Model Drift PSI Calculator
Compare a baseline distribution against current production data and compute the population stability index, KL divergence and Jensen-Shannon distance per bin, with the thresholds that trigger a review.
Last reviewed by the Radiatus Cloud team
Securing AI in production?
We build guardrails, governance & compliance for AI systems.
Models fail quietly when the input distribution moves
A deployed model does not announce that the world has changed. Accuracy degrades gradually while every system-level metric stays green, and by the time the degradation is visible in business outcomes it has usually been happening for months. Monitoring the input distribution catches this earlier than monitoring the output, because inputs shift before predictions become measurably wrong, and because ground-truth labels for the outputs often arrive weeks later or never.
PSI is the industry default and its thresholds are conventions
The population stability index sums, across bins, the difference in proportion multiplied by the log of the ratio of proportions. The widely used thresholds, below 0.1 meaning no significant shift, 0.1 to 0.25 meaning moderate, above 0.25 meaning significant, come from credit risk practice rather than from any statistical derivation. They are useful because everyone uses them, not because 0.25 is a distinguished number, and a PSI of 0.24 on a critical feature deserves more attention than 0.26 on a minor one.
Empty bins are where the arithmetic breaks
PSI and KL divergence both take the logarithm of a ratio of proportions, so a bin that is empty in one distribution produces a division by zero or an infinite term. The usual fix is to add a small constant to every bin, which is fine but changes the result, and the amount added matters more when bins are small. This tool reports how many bins needed that treatment, because a PSI computed mostly from smoothed empty bins is measuring the smoothing rather than the drift.
Related tools
- AI Prompt Leakage Analyzer — Paste a system prompt and a hostile user input to see whether the prompt holds secrets and whether the input carries injection patterns. Local, instant.
- LLM Data Exposure Checker — Check if text contains data likely to be memorized or exposed by LLMs.
- AI Usage Policy Generator — Generate an acceptable use policy for AI tools in your company.
- Model Hallucination Estimator — Estimate risk of hallucinations based on task type and temperature.
Frequently Asked Questions
What PSI value should trigger action?
The common convention is below 0.1 for no significant shift, 0.1 to 0.25 for moderate, and above 0.25 for significant. These come from credit risk practice rather than a statistical derivation, so treat them as shared conventions rather than as thresholds with meaning of their own.
Why monitor inputs rather than outputs?
Because inputs shift before predictions become measurably wrong, and because ground-truth labels for outputs often arrive weeks later or never. Input monitoring is the earlier signal and usually the only available one.
How is KL divergence different from PSI?
KL divergence is asymmetric: it measures the information lost when the current distribution is used to approximate the baseline. PSI is the symmetrised sum of both directions, which is why it is roughly twice the size and why it does not depend on which distribution you call the baseline.
What happens with empty bins?
The logarithm of a zero ratio is undefined, so a small constant is added to every bin. That changes the result, and this tool reports how many bins needed it, because a PSI computed mostly from smoothed empty bins measures the smoothing rather than the drift.
How many bins should I use?
Ten is the usual choice for a continuous feature, using baseline deciles. Too few bins hide drift within a bin; too many produce sparse counts where the smoothing dominates.
Privacy & Security
Everything runs in your browser; nothing is uploaded.
How to Use
Paste your baseline and current distributions to measure drift.
Disclaimer: This tool is provided "as is" without warranty of any kind. Results are for educational and utility purposes.
Related Tools
AI Prompt Leakage Analyzer
AI SecurityPaste a system prompt and a hostile user input to see whether the prompt holds secrets and whether the input carries injection patterns. Local, instant.
LLM Data Exposure Checker
AI SecurityCheck if text contains data likely to be memorized or exposed by LLMs.
AI Usage Policy Generator
AI SecurityGenerate an acceptable use policy for AI tools in your company.