AI Eval

AI Eval is a small repeatable test that scores AI output against a fixed rubric and returns hard numbers. You define success in advance. The test runs on whatever the model spits out whether it is a Figma frame a React component or a full user flow and tells you the exact failures so you fix the prompt not the pixels. At Linear and Vercel in 2026 no prompt leaves the shared library without its paired eval. The orchestrator writes it once. The model grades its own homework forever after. This turns AI from a clever intern that needs constant supervision into a system that ships on spec at 3 a.m. The eval lives inside Claude Skills or as a tiny TypeScript function in Cursor and it compounds with every other skill on the new ladder. Prompt systems without evals are just wishful thinking. Claude Skills without evals are just faster garbage generation. The designers hitting the orchestrator rung at Ramp and Brex all learned this the hard way in 2025 when their first unmeasured AI outputs broke in production and cost real money. Now they measure first.

An AI Eval is not a human critique session. It is not a long comment thread in Figma that changes every Tuesday. It is not vibes. It is not the senior designer staring at three variants and picking the one that feels right. It is not a one time manual QA pass. It is not something you bolt on at the end after the model has already wasted your afternoon. Most ex senior designers from the old ladder still treat it like optional seasoning. They run the prompt look at the output shrug and tweak it manually. Then they complain that AI feels inconsistent. The eval removes the shrug.

Concrete example. In Q1 2025 Brex rebuilt their entire billing surface using AI. Their lead orchestrator wrote an eval with nine weighted criteria. Color adherence scored against their exact Tailwind token set with a 5 percent variance allowance. Spacing checked against their 4 point scale. Accessibility ran axe core plus a custom contrast checker that failed anything under 4.5 to 1 on body text. Component usage verified that every button matched the production library instead of inventing new variants. Copy tone ran through a smaller model trained on Brex voice guidelines and scored from 1 to 10. Mobile rendering ran a simple viewport test. Error states matched their exact illustration library. Performance checked that no generated component added more than 6 kb. The final score had to hit 91 or the output auto rejected and fed the failure reasons back into the next Claude pass. This loop ran 187 times across checkout flows and saved the team from shipping three separate contrast failures that would have gone live under the old process. Ramp copied the pattern for their 2026 dashboard refresh but added a custom calm density metric that counted interactive elements per square inch. Anything too dense failed the eval automatically. Vercel took it one step further in their v0 marketing components. Their design engineer wired the eval directly into their CI pipeline so every generated piece ran Lighthouse performance and a custom brand voice checker before it could merge. The eval JSON even suggested the exact prompt tweak needed. Linear used evals for their command bar redesign. One test simulated keyboard navigation order using a headless browser and failed any output where focus order broke logical sequence. These are not theoretical. These are the exact systems the companies setting the pace actually ship with.

Another concrete example lives in the Claude Skills packs that ship from ClaudeBrainy. One Skill generates variant exploration sets. The attached eval pulls the Figma file through Model Context Protocol then checks auto layout usage fixed pixel widths on text layers variant naming conventions and token substitution accuracy. It returns a score plus plain English fixes like swap this 24 px hard coded gap for var spacing lg. Production assistants run this eval on every batch before anything reaches the shared library. Without it the component library turns into spaghetti within two sprints. Anthropic runs similar evals on their internal design system updates. The eval checks not just visual match but semantic correctness does this new primitive actually belong at this hierarchy level or is the model hallucinating a new primitive that breaks their strict hierarchy rules. Stripe has used eval style checks for years on their design engineer rung. Their version lives in code and runs against every PR that touches the React components.

Use AI Eval when you are operating at the orchestrator or design engineer level and you need the same quality bar across hundreds of generated assets. Use it when you are versioning prompt libraries and want to prove that version 4 beats version 3 on actual metrics instead of gut feel. Use it when handing work to production assistants so they know exactly what good looks like without you babysitting. Use it when you want to turn AI output into shippable product instead of starting points. It is the skill that separates the 140k orchestrators from the 70k prompt monkeys in 2026. Do not use it during the first thirty minutes of wild exploration when you are trying to discover a new visual direction. Do not use it on highly subjective illustration or brand expression work where the success criteria cannot be quantified without killing the soul of the piece. Do not waste time writing one if you are still at the production assistant rung just cleaning up single outputs. And skip it if your workflow is still one shot prompts instead of systems.

The designers winning right now treat evals like their most important deliverable. They spend an hour writing a tight eval so they never spend ten hours fixing bad output again. That trade compounds faster than any portfolio polish ever did on the old ladder. AI Eval is how you stop hoping the AI gets it right and start knowing when it does.