Model Drift Monitor

Pick how the world changes, then track what it does to a model's accuracy over time.

Model drift happens when the customers, language, behavior, or conditions a model sees after launch begin to differ from the data it originally learned from. The model may still be working as designed, but its predictions become less accurate because the patterns it learned no longer match the environment around it.

Choose a drift scenario and decide when to retrain the model, then watch how those choices affect accuracy across the following 24 weeks.

Drift scenario

Each scenario changes the incoming data differently, so you can see how different kinds of drift affect the same model.

A new customer segment enters the mix with behavior the model never saw in training. As it becomes a bigger share of the data, the model's original patterns stop working and accuracy drops.

Accuracy is the share of predictions that match the known correct outcome. 80% means the model got it right in 80 of the 100 cases this simulation represents.

Alert threshold80%

This is the lowest accuracy the business will accept before the model needs attention. Any week below the line is flagged, but the alert does not retrain the model by itself because the owner still needs to investigate what changed and choose the right response.

Retrain at weekNever

The week you'd feed the model newer examples that match the changed environment. Accuracy recovers from that point, though the same drift can start again afterward if the world keeps moving.

Weeks below alert threshold

16 of 24

Accuracy at week 24

73%

No retraining scheduled, so the model keeps relying on what it learned before the environment changed, and whatever this scenario does to accuracy plays out for the full 24 weeks.

What this shows

A model doesn't stay accurate just because it performed well at launch. Customers, language, demand, and operating conditions keep changing, so a business has to keep checking whether a model's predictions still match the world it's making them in, and decide when newer data is worth retraining on.

How it works

Each scenario represents a different way incoming data can move away from the model's original training data. The change may happen through a gradual decline, a sudden shock, a new customer segment, or a repeating seasonal pattern.

Moving the retraining slider updates the model at the selected week using newer data, so accuracy recovers from that point. The environment may continue changing afterward, which means the model can begin drifting again.

Design decisions and guardrails

This smaller-scale replica makes the relationship between drift, monitoring, and retraining visible across one chart. The alert threshold identifies when accuracy needs attention, while the decision to investigate or retrain remains separate. That distinction shows why a monitoring signal should inform a business decision rather than make the decision by itself.

Limitations

The display follows five defined scenarios across 24 weeks, while real environments may experience several kinds of drift at the same time. Production teams also need enough observations to distinguish a lasting decline from normal variation.

Production next steps

A production monitor would compare predictions with known outcomes as those results become available. It would notify the model owner when accuracy crosses an agreed threshold, provide information about which patterns changed, and continue monitoring after any update because the environment may keep moving.

Technology used

TypeScript powers the drift scenarios, accuracy calculations, alert logic, retraining controls, and interactive chart. Keeping the simulation at a smaller scale makes the full monitoring lifecycle visible, including how drift lowers accuracy, how the threshold identifies the decline, and how newer training data helps the model adjust.