Cloning UI in Figma with LLMs

Sometimes you want to take some UI from the browser, whether it’s a full page feature or a single component, and copy it into Figma so that you can iterate over it. There are about a dozen startups who have entire products around this “HTML to Figma” concept, but you can do this now in about an hour with an LLM while you watch TV. Maybe you have the luxury of a fully-staffed, fully-funded design system team with impeccable components that are completely in sync between Figma and code, but for the rest of us, we can leverage LLMs to reduce the minutia of pixel-clicking.

The pattern is really simple:

1. Give an LLM access to the browser
2. Show it the UI you want to copy in one tab
3. Open Figma in another
4. Provide the LLM a way for it to check it’s work via an overlay screenshot
5. Iterate

LLMs will not save you. There is no plugin that will do all of this for you. This is the age of forging your own lightsaber. I’ll show you the broadstrokes so that you can apply the things that work to other problems.

Setup

In order to have an LLM be able to actually manipulate Figma, we need to use the browser. Nothing fancy. You can read all about how to setup a better Figma MCP.

Once you have that setup, in a new folder, create a CLAUDE.md¹ file or AGENTS.md file if you aren’t using Claude. In it, briefly describe what we’re doing:

Our goal is to recreate the UI that’s in Tab 1 in a new Figma file in Tab 2. We want to get as close to pixel-perfect as we can without worrying about subpixel font rendering discrepencies. ONLY interact with Figma via javascript and the global figma API. Do not click Figma UI.

This AGENTS.md file will be our persistent memory. It should contain only the most essential context. Think of it like an onboarding doc. This process can get complicated and will involve resetting the session multiple times so it’s important to keep a small memory file to help the LLM orient each time we start a new session.

Oh, also if you use auto light/dark mode, turn that off if the UI you’re copying also respects the user’s prefers-color-schema…

Phase 1 - Measure

Start by having the LLM open the browser, then navigate to whatever you want to clone in one tab and Figma in another. Open up the Figma file you want to work in. Then start by telling the LLM what you want to do:

Recreate the UI thats in Tab 1 in Figma in Tab 2. Before writing any Figma code, select the first page tab with select_page and extract every measurement using evaluate_script with getBoundingClientRect() Store all measurements as structured data.

If it’s a massively complicated, you’ll need to tweak this abit. Do a research phase and see what you can and can’t do. For this example, I’m going to clone Trello. As part of my tradeoffs, I’ve accepted I won’t deal with icon svgs or loading Atlassian’s custom font because I don’t even know how.

Phase 2 - Rough Draft and Components

At this point, the LLM has a fresh idea of what to build in it’s context. Start by creating some rough components. It’s up to you how involved you want to be here.

If you have opinions about component APIs, this is the place to start a dialog. Iterate with the LLM and decide how you want things to be structured, named, etc. Save those requirements as part of your prompt and restart everything from Phase 1 again.

Working in components first will allow you to make large scale tweaks without burning through a ton of tokens. Funny how that works out. Don’t try to overabstract or force anything at this stage. You can always refactor.

Phase 3 - Introduce feedback

Depending on your model, you can get pretty close with just phase 1 and 2. It’s the last 20% that’s always the hardest. Here’s where a lot of people will dismiss LLMs for not being able to do xyz, but this pareto principle has nothing to do with the “ability” of the LLM—it doesn’t have the right tools to get to 100%. Arguably, neither does a human.

We need better tools.

How would we do this normally? You could keep tabbing back and forth or you could use a screenshot overlay. This is the same techinque that David Luhr used to create the TailwindUI Figma UI Kit.

As I built each component in Tailwind UI’s Figma kit, I made heavy use of the Dev Tools’ “Screenshot Node” feature to create images that I overlaid with 50% opacity on top of my designs in Figma. This helped me double check my work at every level…

We can do the exact same thing and give the LLM a way for it to check it’s work. Now we have a complete loop and LLMs sure love loops. A complete feedback loop allows us to remove ourselves from the equation while the LLM optimizes itself.

Ask the LLM to: create an overlay of the reference UI in Figma and create tools to quickly enable or disable the overlay, control the opacity of the overlay, and control the blend mode of the overlay. Write the instructions on how to use these tools to AGENTS.md

Alternatively, you can just paste in a screenshot yourself. How the LLM gets the screenshot and sets up the overlay is irrelevant.

Once we have a working overlay, it’s time to reset and start a new session.

Phase 3.5 - Reset

We’re resetting here because we just asked the LLM to do a lot. It’s tired. It needs to sleep. LLMs have a finite context window. The longer you go, the dumber it gets. We’ve all seen it. Tools like Claude Code and ChatGPT will skirt around this by continiously summarizing the conversation as the window fills. Ultimately you get a photocopy of a photocopy of a photocopy.

This is a good stopping point because we have our environment setup and all the proper tools. We don’t care about all the lines of code we had to write to set up the tools. From here on out, we’ll perform small iterative loops toggling the overlay. Don’t let the session compact, just create a new session².

Phase 4 - Infinite Iteration

In a new session, tell the LLM:

View the open Figma file and compare and contrast the current design against the overlay. Use the overlay as the source of truth and iterate until you can get as close to pixel-perfect as possible. Start from the top with the highest-level component then branch out to smaller pieces. Your goal is to reduce the amount of layout shift when toggling the overlay from 0 to 100.

Do this over and over and over until you get your desired result. If you aren’t getting the result you want, ask why? Revise your prompt. Try again. Repeat.

Patterns over Tools

There’s no single tool that’s going to be able to do all of this for you. The idea isn’t to completely replace your entire chain. Tools fail because they’re not fit to the individual. We all have our own strengths and weakenesses. You might have a really solid understanding of AutoLayout™️. I might have a really cool keyboard macro that makes this so much faster. My UI might be behind corporate auth. What about animation? Hover states?

Instead of reaching for a tool, make your own. It costs almost nothing now. For most designers, I imagine all you actually need is a way to quickly bootstrap the overlay and some custom key commands to control the toggle. Build that instead.

This entire article can be summarized in one sentence:

If you want to clone UI, add a reference overlay. Give the LLM the ability to use the overlay to check its work.