AI Agents That Write Code From Diagrams: What's Actually Shipping

Can AI agents write code from diagrams?

Yes, and it's already shipping in production. Tools like tldraw's Make Real and Agent SDK, Figma's new use_figma MCP tool, GitHub Copilot's canvases, and MCP-connected diagramming apps like Datadef let an AI agent read a sketch, wireframe, or architecture diagram and generate working code against it, sometimes straight into a live repository.¹²³⁴⁵ This isn't a party trick. Coinbase used exactly this pattern to cut a two-to-four-week feature build down to four days.⁶ The mechanism is real, the output is reviewable code, and the surface you're building on is the same canvas you'd use to explain the idea to a coworker.

That's the shift worth paying attention to. For years, a whiteboard sketch was a communication tool. You drew it, someone photographed it, and an engineer eventually translated it into a ticket. Now the diagram itself can be the interface an agent builds against, and the canvas stays open the whole time as the place where you both work.

From whiteboard to runtime: how the canvas became a dev environment

The lineage here is short and traceable. In November 2023, tldraw shipped an experiment called Make Real, built after a Figma engineer's demo of GPT-4V reading a UI sketch. You'd draw a rough interface on tldraw's infinite canvas, hit a button, and get back a working website rendered as an iframe, on the same canvas.¹⁷ That closed the loop: draw, generate, see it run, without leaving the page.

What's changed since is persistence. tldraw's Agent starter kit gives an AI agent standing access to the canvas itself. It can see shapes, create and edit them, draw freehand, and run multi-step tasks across a session, rather than firing once and handing back a static result.² That turns the canvas from a one-shot generator into an ongoing collaborative surface, closer to a workspace than a demo.

Figma made the equivalent move at the design-tool layer. In March 2026 it opened its canvas to agents through a new use_figma MCP tool, so Claude Code, Cursor, Codex, and other MCP clients can write directly onto a live Figma file, using the team's actual design system, components, and variables instead of generating throwaway markup.³ The diagram stops being a picture of the product and becomes a live, editable model of it.

What does this look like in production?

The clearest production case is Coinbase's internal Figma-to-code agent. It reads Figma screenshots against a reference implementation and a rules document, then writes production code. Their first full pilot, an FCM Derivatives feature, took four days against a normal two-to-four-week timeline, because design-to-code translation had been eating roughly 80% of delivery time on typical internal tooling work.⁶

The first version of the agent generated a full page in one shot, and the result was a pull request nobody could realistically review: 85 files and more than 6,300 additions on the frontend, 19 files and over 14,000 additions on the backend.⁶ Coinbase's fix was procedural, not technical: an orchestrator pattern that plans the work, scaffolds the structure, then splits generation into smaller, parallel per-component draft PRs a human can actually read before merging.

GitHub's Copilot canvases follow a related logic at the workflow level. Instead of burying an agent's decisions in chat scrollback, a canvas keeps the workflow state, decision points, and approval gates visible and inspectable to both the agent and the human reviewing it.⁴ Datadef applies the same idea to architecture diagrams specifically: an agent already working in Claude Code, Cursor, or Codex can create and keep a diagram in sync with the live repo, so the diagram stops being a stale artifact from a design meeting and becomes something the agent maintains as the system changes.⁵

Why this matters now

None of this would matter much if it were confined to a handful of engineering teams running agent pipelines. It isn't. Retool's 2026 survey of 817 builders found that 35% of teams have already replaced at least one SaaS tool with a custom build, and 78% expect to build more custom internal tools this year.⁸ The categories under the most replacement pressure are workflow automations (35%) and internal admin tools (33%), followed by BI, CRM, and project management software.⁸

That's the SaaS economics story underneath this piece: procurement used to be the bottleneck between wanting an internal tool and having one. If a non-developer can sketch the tool on a canvas and have an agent build a working version against it, that bottleneck moves or disappears. This is the same dynamic covered in our framework for deciding what to build, buy, or prototype, just with a lower floor for who can start the build.

The governance side nobody sketching a diagram is thinking about

Here's the part that should give any security or IT lead pause. The same Retool survey found 60% of builders across seniority levels had built something outside official IT oversight in the past year, most often citing speed or an unmet need.⁸ PagerDuty's 2026 survey of 1,250 office professionals found two-thirds had used an AI tool at work they believed was against company policy, and 88% had shared work-related information with a public AI tool.⁹ BetterCloud's 2026 data puts the average company at 118 SaaS apps, with 44% of those not IT-sanctioned, and 52% of IT teams still worried about shadow AI and shadow IT.¹⁰

The canvas-to-code pattern doesn't create shadow IT. It just lowers the skill floor for producing it. Anthropic's Claude Cowork is a useful data point on how fast that can move: Anthropic's own team built it almost entirely with Claude Code, in about a week and a half, spending more time on product decisions than on individual lines of code.¹¹ That's the upside case. The downside case is the same speed applied to a tool nobody in security or compliance ever saw, running against real customer data, drawn on a whiteboard by someone who never filed a request. We covered the mechanics of that risk in more detail in the shadow AI tools employees are already running without a ticket.

What can you actually draw your way to right now?

Coinbase's own findings are the honest ceiling here. The agent was strong at structural, convention-following code: file layout, data fetching, permissions boilerplate, the mechanical scaffolding that eats engineering hours without requiring engineering judgment.⁶ It still needed a human for business-logic edge cases, UI fine-tuning, and integration quirks that don't show up in a screenshot.

Retool's data backs up a more modest reality across the industry generally:

• 72% of builders use AI to write discrete code snippets they still integrate themselves.⁸
• Only 31% prompt their way to a complete application.⁸
• Just 8% ship AI-generated code with no changes at all.⁸

So the honest framing is this: a diagram gets you the scaffolding, the layout, and often a working first draft. It doesn't get you a finished, unreviewed product yet. The gap between those two is exactly where a person with judgment still earns their keep, and it's the same gap our piece on verifying AI-generated code without just reading it harder is built around.

The build vs. buy math, updated for the canvas

Run the numbers the way any build-vs-buy decision should be run. A new SaaS seat for an internal admin tool or workflow automation typically costs a recurring per-user fee, plus the procurement time to get it approved. An agent that turns a sketch into working scaffolding costs a subscription to a coding agent and a few hours of a builder's time to draw the thing and review the output.

Given that workflow automations and internal admin tools are the two SaaS categories facing the most replacement pressure from custom builds,⁸ the math tilts toward building whenever the tool is simple enough that Coinbase-style scaffolding covers most of the work, and specific enough to your team that a generic SaaS product is really just a worse-fitting version of what you'd draw yourself.

How do you try this?

If you want to test this on something real rather than a toy, the on-ramps are already public:

1. Sketch a UI and get a runnable draft. tldraw's Agent starter kit lets you build or use an agent that reads and edits shapes on the canvas directly, good for prototyping a UI idea before you commit engineering time.²
2. Work inside your real design system. If your team already lives in Figma, the use_figma MCP tool lets Claude Code, Cursor, or Codex write against your actual components and variables instead of a generic mockup.³
3. Keep architecture diagrams honest. Datadef's MCP integration keeps a diagram synced to your live repo from inside the coding agent you already use, so the diagram doesn't rot the week after the meeting that produced it.⁵
4. Make the workflow itself reviewable. GitHub Copilot's canvases give you a durable, inspectable surface for agent decisions and approval gates, useful once you're running this pattern on anything that touches production.⁴

For teams that want to skip assembling that stack piece by piece, platforms like Remy are built around the same idea end to end: sketch the internal tool you actually need and let an agent build and own the working version, rather than routing the request through a SaaS vendor.

Start with something small and internal, review the first pull request line by line the way Coinbase learned to, and treat the diagram less like documentation and more like the source of truth it's becoming.