Designers often have very precise requirements for how their creations look. Tiny details that might seem unimportant to the untutored eye have to be exactly right to achieve the intended mood or brand feel. This presents a difficulty when working with LLMs, whose probabilistic approach to creation isn’t best suited to following prescriptive instructions to the letter.

The challenge of imposing structure where it’s needed on the output of visual AI models was taken up two years ago by the co-founders of Weavy, which design vendor Figma acquired earlier this year and is now known as Figma Weave.

Ahead of last week's Config, where the first steps were announced to bring Weave into Figma's existing products, I spoke to Itay Schiff, one of Weavy's founding team, to find out more about its history and what it brings to its new parent. He has spent his entire career in design, starting out in post-production of TV ads. That history speaks to Weavy’s core market, which is more focused on visual designers in marketing, ads and video production rather than the product designers who are Figma's core users. But the advent of generative AI was just as impactful here, as he recalls:

When AI broke out, about three four years ago, I was so shocked by the possibilities that it brings to my industry.

Instead of waiting two weeks for a design agency to send back visuals for a client brief, he could write a prompt and get instant results. He goes on:

I was writing the prompt myself in one of the earliest versions of AI image producing, and I was blown away. I got the answer in seconds. I was also — I mean, I felt that it opened up the creative side of me that I really wasn't aware of, because iterating with the machine was really invigorating for me.

But this new-found creativity came with huge frustrations for him and his co-founders. He continues:

We were really appalled by the fact that you couldn't really control AI. You had to control it using text only, which was totally not native for us. We used to have pixel-level control, manipulating the angle of every asset in the image manually, and that was missing. So that's what we started building.

What sets Weavy — now Figma Weave — apart from other tools is that it is a node-based system. What this means is that it creates a record of all the individual steps, or nodes, in a given design workflow. This framework holds the precise instructions to be followed at each stage, any of which can be separately edited. Those instructions become part of the prompt or harness given to a generative AI model when creating or editing an image or video, and the ability to change any node in the workflow enables very precise control of the results. He comments:

The node-based system... gives you the ability to build your process once and revisit it again and again, which is super-powerful. Chat obviously doesn't deliver that.

Schiff showed several examples on the keynote stage at Config, such as a workflow to apply a consistent visual style to a set of images that come from different sources, or to render a logo on a set of product images. Other examples include exploring different distortion effects or converting a flat image to 3D. More complex examples where customers have used the tool include ensuring brand consistency when creating mini dramas for distribution on social media, or applying one of a range of different visual effects to background lighting in an image. Users can now create their own Weave workflows and share them on the Figma Community.

Another strength that Weavy customers appreciated was the ability to use it with any combination of foundation models, without having to tangle with machine learning tools to get it working. Schiff says:

It saves you all the trouble of learning the machine learning lingo. We're building something aimed at the creative community, and we keep our attention focused on what they need, and not everything.

But it is the ability to define reusable effects and design workflows that is the main advantage of using the tool. This saves designers from spending time on manual work that the tool can automate, and also means that, once they have built a workflow, they can hand off certain editing processes to colleagues, who can simply change settings or inputs to produce their own results. He says:

Building this workflow is a bit of a complex task. Not everybody is up to it, because it looks intimidating, and a lot of artists are reluctant — I mean, it's not natural to think in this way, it's a bit more logical, a bit more technical. We identified it really early on, and we said that, even if one person from the entire organization is able to build this, they are also able to package this complex workflow in a tool kind of way, which is very simple.

Last week's announcement brought over 20 pre-built Weave workflows to the Figma Design canvas, and will be followed soon by the ability for any user to publish their own workflows as Weave tools directly in Figma. The roadmap also includes the ability to insert any Figma frame as a node in a Weave workflow, so that subsequent changes to the design are reflected in the final output.

I spend a lot of my time writing about ERP vendors, and at last week's Config, it suddenly struck me that designers and accountants have a lot in common. Both share a preoccupation with precision. For accountants, it's precision about numbers, transactions and financial projections. For designers, it's precision about the exact look, mood and style of an image, graphic or video.

Interestingly, at Config there were several presentations on the main stage that went into quite a lot of mathematical detail — to the extent that some of them went completely over my head — because it turns out that controlling design elements like animation, shading and perspective involves some very sophisticated math. Understanding how bezier curves work is just as important to designers as regression analysis is to accountants.

Continuing the analogy, just as ERP serves as the system of record for enterprise transactions, Figma can be thought of as the system of record for enterprise design — the platform where the precise look and feel of a brand or a product is set down. The addition of Weave brings crucial control over that precision at a time when generative AI introduces new scope for inadvertent deviation.