What is AI model drift and how do I know if it is happening?

Model drift is the gradual degradation in a model's predictive accuracy as the statistical relationship between its training data and the real world shifts over time. It is not a malfunction — it is the expected behaviour of any model left unattended long enough. The challenge is that it rarely triggers an error message. The model keeps producing scores, reports keep running, and the gap between predictions and reality accumulates silently. The only reliable detection mechanism is a structured monitoring cadence: a named person checking model outputs against independent outcome data on a fixed schedule, with a predefined threshold that triggers action when performance falls below an agreed baseline.

How often should an AI model be monitored after it goes live?

For most commercial models, a quarterly review against a written performance baseline is a practical minimum. High-frequency or high-stakes models — credit scoring, demand forecasting, patient risk stratification — warrant monthly or even weekly checks. What matters more than the specific frequency is that the cadence is agreed in writing before deployment, owned by a named individual, and supported by a live, independent feed of outcome data rather than the original training set. Without that structure, monitoring defaults to never.

Who should be accountable for an AI model's performance in production?

A named individual — not a team, not a generic role — should be formally accountable for each production model's ongoing accuracy. This person must have a scheduled review date, access to outcome data independent of the training set, and a written threshold that defines when retraining or retirement is triggered. In practice, this accountability is almost never assigned at the point of deployment, which is why most models drift undetected. Naming an owner is an organisational design decision, not a technical one.

What is the difference between AI readiness before and after deployment?

Pre-deployment AI readiness assesses whether your data is clean and well-structured enough to train a reliable model. Post-deployment readiness assesses whether your governance is strong enough to keep that model reliable as the world around it changes: a named owner, a monitoring cadence, a validation data feed independent of the training set, and a written threshold for retraining or retirement. Most organisations invest heavily in the first and leave the second entirely unaddressed, treating go-live as the end of the project rather than the beginning of an ongoing obligation.

What should an AI governance review after deployment include?

A post-deployment AI governance review should answer four questions for each live model: who is named as accountable for its performance, what monitoring cadence is in place and what independent outcome data it runs against, what written threshold triggers a retrain or retirement decision, and when that threshold was last reviewed against current business conditions. Models that cannot be answered against all four are unowned — and an unowned model is exactly as exposed to drift as an unowned report is to going stale.

Automation & AI

Your AI Model Went Live a Year Ago. No One Has Checked It Since.

19 June 2026·7 min read

Your AI model launched on schedule and the demo went well. A year later, no one in the building can say whether its predictions still hold. This is not a model problem. It is a governance problem, and it is the one most AI roadmaps leave out.

A churn prediction model went live in the first quarter of last year. The launch email went round. The dashboard was demoed in three separate meetings. The data team got the credit for shipping on time. A year on, nobody in the room can say with any confidence whether the model's predictions still match what is actually happening in the business, because nobody has looked. The model is still running. It still produces a score every morning. The retraining schedule that was meant to be written down never was, and the person who validated the model at launch moved to a different team in the meantime. This is not an unusual story. Among organisations whose AI initiatives survive their first year, it is close to the default outcome.

The launch is not the finish line

Most organisations manage an AI deployment the way they manage a software release: build it, ship it, demo it, move on to the next item on the roadmap. A piece of conventional software, once it works, mostly keeps working. A model is different. Its accuracy depends on a statistical relationship between the data it was trained on and the world that data was describing, and that relationship does not hold still. Customers change suppliers. Pricing shifts. A category that drove ten per cent of volume eighteen months ago drives two per cent now, and the model trained on the old pattern keeps scoring as if nothing happened. The term for this is model drift. It is not a malfunction and it is not an edge case. It is the default behaviour of any model left unattended for long enough. Most of the public conversation about AI failure focuses on what happens before launch. Gartner expects organisations to abandon 60 per cent of AI projects not backed by AI-ready data — a figure it has tracked through 2026. That is a real and well-documented problem. Far less gets written about what happens to the models that make it past that stage. Surviving launch is treated as the achievement, not the start of an ongoing obligation.

Why this gets missed

The reason is not technical ignorance. Data teams know what model drift is. The reason is structural, and it looks the same in nearly every organisation where it happens: nobody was assigned to keep checking. Building a model is a project, with a start date, a budget, and a delivery milestone that someone gets recognised for hitting. Monitoring a model is not a project. It is an ongoing obligation with no natural end date and no demo to show for it, which means it competes for attention against work that produces something visible, and loses. The data scientist who built the model has usually moved to the next priority by the time drift becomes detectable. The business owner who requested the model was never told that "live" and "still working" are two different claims. Gartner's research on agentic AI points to the same pattern: more than 40 per cent of agentic AI projects are expected to be cancelled by the end of 2027, with rising costs, unclear return on investment, and weak risk management named as the leading causes — not failures of the underlying model architecture. The technology works well enough to ship. What is missing, case after case, is the governance to keep it accountable once it is live.

The three questions nobody assigned after go-live

When an AI initiative gets the budget conversation right and still drifts quietly into irrelevance, the cause is almost always that three questions were never assigned to anyone by name. First: who is checking that the model's predictions still match outcomes, and on what schedule — not "the data team", but a named person with a calendar entry, the same way a finance controls owner is named rather than implied. Second: what threshold triggers a retrain or a retirement — without a number agreed in advance, "the model seems a bit off lately" is a conversation that gets postponed indefinitely, because nobody wants to be the one who calls time on a project celebrated eighteen months ago. Third: where the validation data comes from — it cannot be the data the model was trained on, and it must be a live, independent feed of outcomes checked on a fixed cadence. Organisations that can answer all three, in writing, before a model goes live, are answering an organisational question, not a technical one. The model itself rarely needs to change. The accountability around it does.

The audit nobody schedules

Pre-deployment AI readiness checks are becoming a standard part of how serious organisations approach AI investment: a structured look at data quality and governance before any model gets built. That work matters, and it is increasingly common. What is far less common is the same kind of check after deployment, repeated. Most organisations audit once, at the start, and treat that audit as having covered the risk permanently. It has not. A model validated against clean, representative data in month one is being asked, by month twelve, to perform against a version of the business that no longer exists in the same shape, with no one checking whether it still can. A working cadence does not need to be elaborate. A quarterly review against a written baseline, owned by a named person, with a predefined threshold for action, catches the large majority of drift before it becomes expensive. The organisations that build this in are not the ones with the most sophisticated models. They are the ones that decided, at the point of launch, that going live was the start of an obligation rather than the end of a project.

Questions worth asking before the next AI investment

Three diagnostic questions cut through quickly. Who on your team is accountable for each model's accuracy six months from now, by name? If the honest answer is "whoever is around", the model has no owner, and an unowned model is the one nobody notices drifting. What would have to happen to retrain or retire a model, and who decides? If there is no written threshold, the decision gets made by inertia, which almost always means later than it should. When was the last time each model was checked against real outcomes, and against what data? If the answer involves the data used to build the model, or no date at all, the model has not genuinely been checked since launch. Organisations that cannot answer all three for their live models are carrying a governance gap, not a technology gap.

What this actually costs

The model that drifts unnoticed rarely fails loudly. It keeps producing scores. Reports keep generating. Nobody gets an error message. The cost shows up later and elsewhere: a retention campaign aimed at the wrong accounts, a credit decision made on a pattern that stopped holding a year ago, a board presentation built on a number that was accurate once. By the time someone asks why the model's recommendations stopped lining up with what the sales team is actually seeing, the gap has usually been there for months. The fix is not a better model. It is a named owner, a written threshold, and a calendar entry that survives the original team moving on to other work. That is a smaller ask than the original deployment, and it is the one most AI roadmaps still leave blank.