What is AI model concentration risk?

AI model concentration risk is the exposure a company carries when material work depends on one model provider. Kaykas-Wolff argues the deepest version sits below the API: behaviors only one model performs without being asked, and review steps that run on the same model as the work they check.

Does a model-agnostic routing layer remove concentration risk?

A routing layer makes it cheap to send requests to a different model, but it says nothing about whether the replacement does the job. Kaykas-Wolff describes an agent that produced confident work for eleven days after a model swap while skipping a verification step the old model had performed on its own.

Why should an AI reviewer run on a different provider from the agent it checks?

Two calls to the same model tend to share blind spots, so a same-model reviewer is least likely to catch the errors the drafter is most likely to make. In Kaykas-Wolff's fleet, a Google-backed agent researches, an Anthropic-backed agent drafts, and a local model verifies facts against the source.

How should a company test an AI provider failover?

Pick one production agent whose errors are visible but recoverable, run its frozen evaluation set of fifty to two hundred real past tasks against the second provider, have the owner read the diffs, check the agent's unprompted behaviors, and record the hours spent. Then point the reviewer at a different provider and keep it there.

AI Model Concentration Risk: Run the Failover Before the Board Asks

Q: What should directors see about AI concentration risk in the board pack?

Kaykas-Wolff recommends three columns for each agent in the revenue path: the provider it runs on, the provider its reviewer runs on, and the date of its last real failover test. Matching providers in the first two columns is a finding, and a blank failover date is a bigger one.

Concentration risk has become the AI item on every director’s reading list. The consultants have scoring matrices for it, the compliance vendors have contract checklists, and the standard advice is sensible enough: inventory your model dependencies and put a routing layer between your code and any one provider’s API.

I agree with most of that. It still looks in the wrong place.

I run a fleet of agents across content and operations work, and I sit on the board of ACT | The App Association while running Visiting Media as CEO. From both chairs, the concentration that worries me sits one layer below where the checklists stop. It lives in the behaviors only one model performs without being asked, and in the review steps that run on the same model as the work they are supposed to catch.

What the standard advice gets right

Start with the inventory. Every model sitting inside a material process belongs on it, including the ones buried in SaaS tools your team bought for something else entirely, because that is usually where the surprises are (a support platform that quietly routes every ticket summary through one provider counts, even if nobody in engineering ever signed for it). The same exercise turns up shadow AI agents more often than anyone expects.

Contract terms on model-change notice matter too. Providers retire old versions on their own schedule.

The routing layer is where I part ways with the guides.

Why a routing layer is the weakest hedge

An abstraction layer makes it cheap to send a request to a different model. It tells you nothing about whether the answer that comes back is any good.

I learned this during an ordinary upgrade inside my own fleet, which I wrote up in change control for an AI agent fleet. One agent kept producing confident, well-formatted work for eleven days after the swap while skipping a verification step the old model had always performed on its own. Nobody had written that step down. The old model just did it.

Now picture the same swap during a provider outage, under pressure, across every agent at once, with no canary week and nobody reading diffs because the whole point of a failover is that it happens fast.

That is what a model-agnostic architecture buys you on a bad day.

So the real switching cost of a provider is the time it takes to prove the replacement does each agent’s job, counted per agent and in hours a named person spent. For most companies that number is unknown, because the frozen evaluation sets that would measure it were never built.

The reviewer on the same model as the work

None of the board guides I read mention this one.

Most companies that take AI seriously have built some kind of review layer by now, an agent that checks the drafting agent’s facts or a second pass that catches policy problems before anything reaches a customer. Good. Look at what that reviewer runs on. In most stacks I see, it calls the same provider as the agent it reviews, often the same model version, because that was the API key already sitting in the config file.

That puts the concentration inside the control itself. Two calls to the same model tend to share blind spots, so the reviewer is least likely to flag exactly the mistakes the drafter is most likely to make, and when that provider ships a new version, both sides of the check change on the same afternoon without anyone touching a line of code.

My own fleet splits this on purpose. A Google-backed agent researches, an Anthropic-backed agent drafts, and a local model verifies the facts against the source. I set it up that way for quality. It also turns out to be the cheapest concentration hedge I have, since an outage at one provider slows one stage of the work and leaves the check on it standing.

What a real failover test looks like

In what boards should ask about AI, I suggested directors ask for the substitution cost in engineering weeks and whether anyone has tested it. This is how I would want management to produce that answer.

Pick one production agent whose errors are visible to customers but still recoverable, usually something in tier two of the delegation of authority matrix.
Run its frozen evaluation set (fifty to two hundred real past tasks, each with the output a human judged correct) against the second provider, and have the agent’s owner read the diffs. Scores alone hide the eleven-day problem.
Check the written list of things the agent does unprompted. Each one that disappears is a cost you will pay during an outage.
Record the hours.
Repeat the exercise for the reviewer, pointed at a different provider from the agent it reviews, and keep it there.

The hours from one agent will not scale linearly to the fleet, because the agents with the most judgment in their work take longest to prove. That is fine. Boards govern honestly rough numbers every quarter, and the revenue forecast is one of them.

A note on the contract clauses

The exit-assistance clause your lawyers fought for becomes valuable on the morning after you already know how to leave.

What directors should see

Concentration belongs to the full board, which is one reason I argued against parking it in a separate AI committee. It needs very little space in the pack. I would add three columns to the AI page in the board pack for each agent in the revenue path: the provider it runs on, the provider its reviewer runs on, and the date of its last real failover test.

Matching providers in the first two columns is a finding. A blank date in the third is a bigger one.

Directors without the background to read those columns critically are exactly the case for a technology board advisor, who can sit with the audit chair before the meeting and turn a blank cell into a precise question.

Before the next board meeting

Ask management for the date of the last failover test and the name of the person who ran it.

If the answer arrives as an architecture diagram, you have learned what you needed to know, and the next thing to schedule is an afternoon with one agent on a second provider and someone reading every diff.

This article is part of the Executive Leadership cluster, focused on board governance and the operating discipline required to run AI systems responsibly at the executive level.