Apple researchers have published details of SQUIRE, an experimental AI-powered tool that helped developers explore and refine interface ideas with more control than with typical AI coding tools. Here are the details.
An interesting approach to AI-powered interface prototyping
In a new paper titled SQUIRE: Interactive UI Authoring via Slot QUery Intermediate REpresentations, a group of Apple developers proposes a novel way to approach AI-generated interfaces.
As they explain, natural language has been bringing more flexibility to multiple aspects of the development process, but it also brings two key challenges:
First, natural language on its own can be ambiguous, making it difficult for developers to precisely communicate their intentions. Second, the model may respond unpredictably, requiring the developer to re-prompt through trial-and-error to repair any undesired changes.
That’s where SQUIRE comes in. It is a visual interface that lets developers build and refine UI prototypes step by step, with clearer control over the results.
From the study:
In SQUIRE, users start a project by providing a prompt that describes their goals for the UI, along with sample data containing information for SQUIRE to use as a reference. Users then construct UI as a tree of components in top-down fashion by prompting SQUIRE to fill holes representing missing yet expected functionality. In response to this kind of request, SQUIRE generates a list of appropriate alternatives, each scoped specifically to the targeted hole in the unfinished UI. Clicking on each alternative immediately updates a live rendered preview as well as underlying code, making it easy to visualize the differences. The user can also pose targeted requests to modify the appearance of specific areas of the UI, with the guarantee that no code outside the intended scope will be mutated. In response to this kind of request, SQUIRE generates ephemeral controls that allow the user to apply semantically-related changes quickly and without re-prompting. In all cases, the LLM acts as a companion, presenting reasonable choices for the user to evaluate, but leaving the user with agency to accept or reject its suggestions.
In other words, natural language prompts were still how developers interacted with SQUIRE, but instead of affecting the entire interface at once, each prompt is tied to a specific part of the UI.
Based on observations with 11 frontend developers who used SQUIRE to develop interface prototypes, the researchers found that participants were able to explore and iterate on different UI designs with a strong sense of control, while also rating the system highly for usability and overall satisfaction.
Additionally, they note that this extra sense of control made developers more comfortable exploring paths they might not have tried otherwise, since changes were easy to make, predict, and undo.
From the study again:
Through data collected from a user study of 11 frontend developers, we find that (1) SQUIRE’s interactions encouraged participants to explore frequently, rather than simply use SQUIRE as a code accelerator, (2) participants felt encouraged to take risks when making changes, knowing that the consequences of making atypical decisions could always be undone without friction, (3) participants indicated confidence that SQUIRE matched their intent when making changes, and (4) participants were generally pleased with the quality of code and visuals generated by the system.
SQUIRE under the hood
Rather than generating interface code directly from user prompts, SQUIRE first creates its own intermediate representation of the interface, called SquireIR, which models the UI as a tree of components with named slots that can be filled in over time, like in the example below:
That structure can also include placeholders for parts that have not yet been defined (e.g., a button label, an image, or a section of content), as well as multiple possible UI alternatives. For instance, it could represent the same content as either a list or a grid.
From there, SQUIRE translates that representation into code using HTML, CSS, and JavaScript, with Web Components handling the final UI structure.
Another key aspect of SQUIRE is how it handles changes.
If a developer asks to tweak a button or adjust a layout, only that part is updated, while everything else remains untouched.
According to the researchers, this helps avoid trial-and-error loops seen in many AI coding tools due to the overall unpredictability of LLMs, where the model may make changes beyond what the developer intended.
This structure is what also allows SQUIRE to suggest multiple options at each step, so developers can quickly compare different versions without losing their previous work.
Contrary to many technical papers, this study doesn’t go into detail about model training, architecture, or data. The researchers do note that SQUIRE is powered by OpenAI’s GPT-4o, but the paper’s focus is firmly on the system design and interaction model
SQUIRE is not generally available, and its use was limited to the 11 developers who participated in the study. However, it is not hard to imagine how something like this could be implemented in future versions of Xcode, or other Apple-built development tools.
To learn more about SQUIRE, follow this link.
Worth checking out on Amazon
- David Pogue – ’Apple: The First 50 Years’
- MacBook Neo
- Logitech MX Master 4
- AirPods Pro 3
- AirTag (2nd Generation) – 4 Pack
- Apple Watch Series 11
- Wireless CarPlay adapter
FTC: We use income earning auto affiliate links. More.
Comments