The problem
Picking a model provider feels like a bet on the company, so teams either stall or sign with whoever demoed last. Either way, the build gets tied to one vendor before anyone has tested it on real work.
What goes wrong
- The vendor's demo is the test. Nobody runs the candidates on the company's own documents, so the first real file is the first real failure.
- The prompt is the product. Business rules live in a long prompt tuned to one model, and a version change or a price change means a rewrite.
- Nobody reads the terms. Data handling, training rights, and retention get discovered by compliance after the system is live.
The approach
- Write down the five to ten tasks the system will do, with real examples from your own files. Test every candidate on those and nothing else.
- Keep the model one layer out, so the provider is a setting. The model-agnostic architecture playbook covers how.
- Keep a fixed set of your real cases, names stripped, and re-run it after every model, version, or prompt change.
- Read the commercial terms before any data flows, and log every request, answer, and action so compliance can audit what ran and why.
When it works
When the people who will own the system test the candidates on their own files, the model sits behind a layer the company controls, and every call is logged. The choice stops being a bet because it is reversible.
Why this decision feels bigger than it is
Every AI build starts with the same question: which model. It feels like picking a bank. It is closer to picking a phone carrier. The work that matters (your data, your rules, your approvals, your logs) sits in your systems. The model is one call out from there, and that call can be pointed somewhere else.
Most of the risk comes from building as if the choice were permanent. Prompts tuned to one provider's habits. Business rules written into the prompt instead of the code. No record of what the model was asked or what it returned. A year later the provider retires a version, changes a price, or a better model ships, and your team has to rewrite instead of reconfigure.
What we do
We pick the model that fits the job. The post Pick an LLM vendor without betting the company makes the longer case.
How to choose without betting the company
Write down the job before you test a model
A model that is strong at drafting may be weak at reading a forty-page statement for exceptions. Define the five to ten tasks the system will actually do, with real examples from your own files, before any vendor demo. Test every candidate on those and nothing else.
Keep the model one layer out
The model name, the prompt version, and the settings live in configuration. Your code calls the model layer, not a specific vendor. When a better model ships, switching is a config change and a round of tests, not a rewrite.
Keep a fixed set of real cases
Pick a set of real cases, strip the names, and re-run them after every change: new model, new version, new prompt. The set tells you whether a switch is safe. Without it, every upgrade is a guess.
Read the terms before the data flows
Commercial API terms matter more than the marketing page. Get three answers in writing: does the provider train on your data, where is it processed, and how long is it kept. For an insurer, a lender or a fund, compliance asks these first. Ask them on the first call with a vendor, not the last. The security page covers how we handle your data.
Log everything the model does
Every request, every answer, every action it took, in a log your team can read. Audit trails are not a feature you add later. In insurance and financial services they are the reason the build is allowed to run at all.
When it does not work
If the whole system is one long prompt tuned to a single model's quirks, you are locked in whether you signed anything or not.
Where to start
If you are not sure which workflow should get a model first, the paid AI Assessment takes ten days with your team in the room and delivers an AI inventory, risk tiers, the top 3 workflows with today's numbers and a 90-day plan. For a straight answer first, ask Roo, Webaroo's AI: How should we choose an AI model provider without getting locked in?
Book 15 minutes with Connor