The problem
The model market moves every few months, and the one you built on last year is now slower, costlier, or weaker than what shipped since. If the model is wired into your application by hand, every one of those moves is a rewrite.
What goes wrong
- The prompts are the product. They are tuned to one model's habits. Switch models and the output changes in ways nobody can explain, because nothing was tested against a second model.
- The provider's features become the architecture. Memory, file storage, threads, and tool calling from one vendor's SDK get treated as your own, and a move takes all of them with it.
- Nobody can see the cost. Spend arrives as one provider invoice. Nobody knows which task used which model, and the cheap work never gets routed to the cheap model.
The approach
- Put every model call behind one interface you own. The provider name lives in a config file, not in your code.
- Route by task: heavy reasoning and customer-facing conversation to the strongest model, high-volume routine work to a smaller tier. Each route is a config change, not a code change.
- Run a fixed set of your real cases against every candidate model. A model goes live when it passes that set, not when a benchmark says it should.
- Keep memory, permissions, approvals, and logs on your side of the interface, with the model used and its cost recorded on every task.
When it works
When the system is in production long enough to outlive at least one model generation, and when someone on your team reads the cost and review reports each month and acts on them.
Why the model should be a setting
A provider ships a better model, a cheaper one, or a faster one every few months. The model you chose last year becomes the wrong choice without anything on your side changing. If that model is wired into your application by hand, each of those moves is a rewrite. If the model is a setting, it is a config change and a test run.
The agent, the memory, the permissions, the approval steps, and the logs are yours. The model is a supplier. Build so you can change suppliers. The post The model is a setting, not a bet makes the same case for a buyer.
How we build it
One interface, many models
Every model call goes through one layer you own. It takes a task, a context, and a budget, and returns a result in the shape your application expects. The provider's name appears in a config file, not in your code.
Route by task
Heavy reasoning and customer-facing conversation go to the strongest model. High-volume routine work goes to a smaller, cheaper tier. Multi-step tool use goes to whichever model does it best this quarter. Each route is a line of configuration, not code.
Test across providers before you commit
We keep a fixed set of real cases from your work and run it against every candidate model. A switch happens when the new model passes that set, not when a benchmark says it should.
Keep what makes it yours outside the model
The memory of how your company works, who may ask about what, the approval step before any action that moves money or contacts a customer, and the log of what ran: all of it lives on your side of the interface. A model swap touches none of it.
Count every token
Each task records the model used, the tokens in and out (the units of text a provider bills by), and the provider cost. You see which routes are expensive and you decide where to move them.
What it looks like in hospitality
A club or a hotel runs on requests. A member wants a table. A guest wants a late checkout. The dining team needs the preferences from last visit. An agent handling those requests calls a model many times an hour. A few calls need judgment and a warm tone. Most are routine lookups.
Routing the routine calls to a cheaper tier and the judgment calls to the strongest model puts the spend where the judgment is. When a better model arrives, you turn it on for the judgment calls first, watch the review queue for a week, and then move the rest.
Where Roo fits
Roo is the cockpit a company runs its AI crew from. The crew works inside the company's own systems, under its rules. The Brain belongs to the company. Early access February 2027.
When it does not work
It does not pay off for a short-lived tool that will not outlive one model generation. And it does not work if nobody on your team reads the cost and review reports each month and acts on them.
Where to start
If you are choosing a provider now, choosing a model provider covers what to test. For a bank or lender, banking and financial services covers loan document review and approvals on the record. If you already have an AI build tied to one provider, ask Roo, Webaroo's AI: How do we keep our AI system from depending on one model?
Book 15 minutes with Connor