Hybrid Pattern

The hybrid pattern routes the majority of an agent workflow through tool-use APIs while reserving computer use for the awkward edges no one ever wrapped. The system decomposes a goal into steps then checks for a registered tool first. A tool call returns structured JSON in milliseconds at penny-level cost. Only when no tool matches does the host spin up a screenshot loop through Anthropic Computer Use or a Browserbase session. The model receives a focused screenshot plus a narrow sub-goal like "fill vendor form with these values" then returns click coordinates or keystrokes. The host executes them and feeds back the next frame. This 90/10 split appeared everywhere that actually shipped money in 2026. Tool-use handled the repeatable reliable parts. Computer use mopped up the vendor portals dashboards and legacy apps that refused to expose APIs. The pattern killed the pure computer-use hype because those runs routinely hit three to seven dollars while hybrid versions stayed under forty cents.

It is not pure computer use that wastes tokens screenshotting every trivial button click. It is not rigid tool-use that collapses the moment a partner changes an undocumented frontend. It is not a magical auto-router that intuits the right mode without you drawing explicit boundaries in code and prompts. The hybrid pattern demands you map the workflow in advance register crisp tools via the Model Context Protocol and write clear handoff instructions. Skip the mapping and you get the worst of both worlds: surprise hundred-dollar monthly bills and agents that flip to screenshots on tasks a one-line API call could have crushed.

A concrete example runs inside Replit Agent throughout 2026. A developer types "onboard this new client and set up their monitoring." The agent begins with tool-use. It calls your internal GitHub tool to create the repo returns the clone URL then hits the Vercel tool to deploy the starter template. Both calls finish in under 400 milliseconds for four cents total. Next it registers a Sentry alert via another tool. Then it slams into the client's 2019 internal time-tracking system built on deprecated Angular with zero API access. The router switches to computer use inside a Browserbase sandbox. The model gets a cropped screenshot of the login screen clicks the SSO button enters credentials from the prior tool output navigates to the timesheet clicks the project field pastes the commit data from git and submits the hours. Total run cost lands at 31 cents with two supervised checkpoints. Devin follows the exact same dance during security audits using tool-use for AWS inventory but switching to computer use to navigate three separate vendor consoles that only expose web forms. Lutra's workflow builder makes the split visual. Designers drag a tool node for Hubspot updates then a browser node for the government compliance portal that still runs on a 2008 Java applet. Multi-On agents for sales teams use tool-use to pull Salesforce records then computer use to scrape a competitor dashboard that blocks scraping APIs.

Use the hybrid pattern when your narrow agent touches both your own systems and third-party software without clean exports. Deploy it for vendor portal automation dashboard scraping cross-app scheduling and legacy system bridging where the ten percent long tail justifies the fallback cost. Pair it with supervised execution clear success criteria and human handoff buttons. The Model Context Protocol makes the router trivial to code. Do not use it for consumer chatbots that must answer in under a second or for any workflow you fully control with your own database. Never reach for hybrid on open-ended long-horizon tasks above fifteen steps where error rates compound fatally. Skip it entirely if the entire job can live in tool-use or if you lack budget for logging retry logic and circuit breakers around the computer-use leg. Teams that violated these rules in early 2026 watched their agents silently default to screenshots on every step and burn through their runway before product-market fit.

The router layer matters more than the models. Smart teams write a lightweight host that receives the high-level goal decomposes it into atomic steps then iterates. For each step it scans the tool registry. Match found equals instant structured call. No match equals packaged computer-use request with tight context and previous tool results injected as structured text. This keeps vision contexts short cuts token burn and halves hallucinated clicks. Add dry-run mode that simulates the full path before touching real systems. Instrument every mode switch with logs so you can debug why the agent ignored your perfectly good Plaid tool and tried to scrape a bank dashboard instead. On the design side every surface the computer-use leg might touch still needs semantic HTML visible labels predictable layouts strong contrast and zero hover-only affordances. The same checklist that ships accessibility now ships agent compatibility. Ignore it and your hybrid agent becomes blind on your own product.

Failure modes are specific and expensive. Misclassify a task and route it to computer use too early and your unit economics die. Write vague tool schemas and the model ignores them in favor of screenshots. Lose structured data between modes and the agent retypes information it already fetched. The fix is ruthless scoping weekly tests against real sites and cost modeling before launch. Under fifty cents per completed task usually works. Over five dollars never does. The teams winning in 2026 treat computer use like the flaky third-party microservice it is: wrapped bounded instrumented and planned for failure.

Hybrid pattern turns computer use from a budget incinerator into a precision tool you only pull out when APIs fail.