Eval Stack

Eval stack is the four layer filter that sorts AI generated design candidates at scale. It starts with linting and token checks that run in milliseconds. Next comes visual diffing to catch unintended changes. Then an LLM scores survivors against a structured rubric. Human taste sits at the very top. Conversion data flows back from production to tune the whole thing. The concept exists because AI made production free but review brutally expensive. One designer used to make one asset a week. Now thirty briefs spit out eighteen thousand variants overnight. Without this stack teams drown or ship drift at volume.

The old LGTM loop is not an eval stack. Posting a Figma link in Slack and waiting for two thumbs up worked when output was slow. It becomes a coin flip with extra steps at AI speed. Common confusion treats this as regular QA. It is not. QA catches bugs. The eval stack encodes taste, brand voice, and craft signals so they run automatically. Another mistake is assuming better models remove the need for it. The opposite happens. Better models produce more candidates faster. The bottleneck simply moves from making to judging.

ML engineers solved this exact problem in 2023. Models shipped quicker than humans could read output so they built eval suites with cheap checks first. Designers hit the same wall in 2026. A senior designer opens their queue to eighteen thousand candidates and realizes Slack threads do not scale. Stripe runs something close on their design system. Linear tunes writing rubrics monthly on real usage data. Vercel keeps Geist consistent the same way. One team cut human review from four thousand candidates weekly to twenty while improving output quality.

The starter toolchain needs no invention. Playwright captures screenshots. Pixelmatch diffs them against baselines. Chromatic hosts the review. A custom Claude rubric scores layout, color, and voice in JSON. Axe-core handles contrast and WCAG violations in CI. These tools cost pennies at the base layer and a few hundred dollars in API spend at the middle. The entire bottom three layers stand up in one afternoon.

Use an eval stack the moment your team generates more than a few hundred candidates per week. It earns its keep on product surfaces, marketing pages, and component libraries that ship often. The senior designer stops looking at padding errors and starts making real taste calls between three strong finalists. Skip it for early exploratory work or highly nuanced one-off projects like pitch decks. The tradeoff is real. You invest time building and tuning the rubric every month. You lose some serendipitous accidents the raw model might have produced. In return you get brand consistency at a scale no human team could match alone.

Teams that treat the rubric as fixed truth miss the entire point. Conversion data must close the loop. Click-through rates per variant and time-on-page per layout get folded back monthly. Criteria that correlate with results get weighted up. The ones that do not get dropped. Rubrics that never update become stale opinions frozen in 2024 thinking.

Building the rubric alone is a trap. Brand lead, design lead, and senior writers need to sit in the room. One person guessing embeds blind spots. Set a clear pass threshold. Average four out of five with no criterion below three is a solid starter. No threshold turns scoring into theater.

The top layer must stay human on purpose. Automating taste review ships eval-passing mediocrity at industrial volume. The stack exists to amplify senior taste, not replace it. Point expensive human eyes at the last fifty candidates, never the first ten thousand.

Junior designers run the queue and watch lint results. Mid-level ones tune the rubric on shipped data. Seniors own the entire system and the criteria that define the brand. The job title starts to read more like evaluation engineer than visual designer.

Start this week. Write one rubric, wire one Claude prompt, add linting and visual diff to the next surface. The difference hits immediately. Your team ships sharper work with one third the review time.

The eval stack turns taste from a vibe into infrastructure that survives AI volume.