The population stability index (PSI) is a single number that measures how far a variable’s current distribution has moved from a reference distribution. Bin both samples, then sum (Actual% − Expected%) × ln(Actual% / Expected%) across the bins. Below 0.1 is conventionally stable, 0.1 to 0.25 is a moderate shift, and 0.25 or more is significant.
PSI needs no labels, which is why it earns a place in production: when approval rates drift or click-through quietly sags, accuracy alerts wait on ground truth that has not arrived, while PSI flags the shifted input first. It came out of credit-risk scorecard monitoring at banks and spread into general ML monitoring because it collapses a whole feature distribution into one comparable scalar with thresholds a risk committee already recognizes. Several monitoring tools expose PSI, but methods and defaults vary by implementation and feature type.
For the scorecard distinction between score distributions and individual inputs, read PSI vs CSI. Choose complementary checks with PSI vs KS vs calibration, and use monitoring models without ground truth to plan for outcomes that arrive later. The ML model monitoring framework assigns owners and actions to those signals.
The PSI formula
PSI compares a reference distribution (usually your training set, or last week’s traffic) against a current one, bucket by bucket:
PSI = Σ_b (Actual_b - Expected_b) * ln(Actual_b / Expected_b)
For each bin b, Expected_b is the proportion of the reference sample that falls in that bin and Actual_b is the proportion of the current sample. Sum that term across every bin and you get a single non-negative scalar per feature: zero when the distributions match exactly, growing as they diverge.
Why this over a raw KS test or a mean/variance comparison? A KS p-value is tied to sample size, so with millions of rows it rejects the null on noise-level shifts and never says how much the distribution moved; PSI produces a magnitude, so one threshold works across a feature in dollars and one in log-odds. PSI is also symmetric and bounded below by zero whichever side you call “reference,” so dashboards cannot disagree on sign. And because it buckets the distribution first, it catches shape changes such as a bimodal split or a fat tail appearing, which a first-moment comparison sails right past. PSI is not always the right pick, though, and the Drift Test Selector will tell you when a feature’s type, volume, or your detection goal points at Wasserstein or a KS test instead.
Fiddler’s writeup points out that PSI is mathematically a symmetrized form of KL divergence, KL(Actual‖Expected) plus KL(Expected‖Actual), so you inherit the information-theoretic interpretation without having to explain KL divergence to a risk committee. A 2025 arXiv paper on information-theoretic credit risk modeling identifies PSI as Jeffreys divergence, the symmetrized form of KL, and derives standard errors for it. That is why PSI behaves like a proper divergence (non-negative, zero only at equality) rather than an ad hoc statistic someone invented for a compliance memo.
Is PSI logarithmic?
Yes. Each bin contributes the difference in proportions multiplied by the natural log of their ratio, ln(Actual / Expected), so PSI is a sum of log-ratio terms; Arthur, Fiddler and WhyLabs all specify the natural log. Because the difference and the log always share a sign, no term can be negative.
The log weights relative change: a bin doubling from 1% to 2% gets the same log term (ln 2 ≈ 0.693) as one doubling from 20% to 40%, and the (A − E) multiplier scales it by the absolute move, to 0.0069 and 0.139. The base matters too, because the 0.1 and 0.25 bands assume ln. The worked example below scores 0.136 with ln, 0.059 with log10 and 0.196 with log2, so a helper that quietly calls np.log10 reports a moderate shift as stable.
How to calculate PSI step by step
- Fix the two samples: a reference (training data or a validated production window) and a current window, with row counts recorded.
- Fit bin edges on the reference only: ten quantile bins in the scorecard convention, equal-width bins, or one bin per level for a categorical feature. Keep a missing-value bin; Arize’s binning guide warns that dropping missing values hides schema changes.
- Count both samples into those frozen bins and convert to proportions that sum to 1.
- Floor empty bins with a small epsilon, then renormalize.
- Score each bin as (A − E) × ln(A / E) and sum the scores. The total is PSI.
A hypothetical transaction-amount feature cut into five bins fitted on the reference:
| Bin | Expected (E) | Actual (A) | A − E | ln(A / E) | (A − E) × ln(A / E) |
|---|---|---|---|---|---|
| 1 | 0.10 | 0.05 | −0.05 | −0.6931 | 0.03466 |
| 2 | 0.20 | 0.15 | −0.05 | −0.2877 | 0.01438 |
| 3 | 0.40 | 0.35 | −0.05 | −0.1335 | 0.00668 |
| 4 | 0.20 | 0.25 | +0.05 | 0.2231 | 0.01116 |
| 5 | 0.10 | 0.20 | +0.10 | 0.6931 | 0.06931 |
| Total | 1.00 | 1.00 | 0.13619 |
PSI is 0.136, in the moderate band. Bin 5, which doubled from 10% to 20% of traffic, supplies about half of it, so triage starts with large transactions. Bins 1 to 3 lost share yet still add positive terms, because a negative difference meets a negative log.
PSI thresholds: 0.1, 0.25 and sample size
The convention most teams use, per Arthur’s docs:
- PSI < 0.1 — little to no shift, no action
- 0.1 ≤ PSI < 0.25 — moderate shift, worth watching
- PSI ≥ 0.25 — significant shift, treat as a retraining or investigation trigger
Fiddler’s cutoff is tighter, 0.2 instead of 0.25, and WhyLabs lists both, a reminder that these bands are industry convention, not a theorem. Treating inherited numbers as universal is the most common way a PSI monitor becomes noise. Backtesting your own thresholds against a healthy historical window is a step in the ML model monitoring framework, and SLOs and alerting for ML systems covers turning the calibrated number into a page.
The bands also ignore sample size. Yurdakul and Naranjo (Journal of Risk Model Validation, December 2020) showed that when both samples share one distribution, PSI is approximately (1/n + 1/m) times a chi-square variable with B − 1 degrees of freedom, for sample sizes n and m and B bins. The rpsi package turns that into a critical value, χ²(α, B−1) × (1/M + 1/N). With NIST’s chi-square table at α = 0.05 (16.919 for 9 degrees of freedom, 30.144 for 19), the PSI that sampling noise alone exceeds one run in twenty is:
| Reference rows | Current rows | 10 bins | 20 bins |
|---|---|---|---|
| 100 | 100 | 0.338 | 0.603 |
| 200 | 200 | 0.169 | 0.301 |
| 1,000 | 1,000 | 0.034 | 0.060 |
| 10,000 | 10,000 | 0.0034 | 0.0060 |
| 100,000 | 1,000 | 0.017 | 0.030 |
At 200 rows per side and 10 bins, noise alone clears 0.1 more than one run in twenty, so a 0.1 alert there pages on nothing. At 10,000 per side, anything above about 0.003 is statistically real, and the question becomes whether it matters. Twenty bins raise the floor about 1.8 times. Small samples also miss drift: a 2026 preprint extending that work put PSI’s detection power at 32% for n = m = 200 in its credit-default simulations.
Population stability index in credit risk
In credit risk, PSI is the periodic check that a scorecard’s current applicants still look like the population it was built on. Banks compute it on the score distribution, conventionally in ten bands, and on each input characteristic, and keep the result as model-validation evidence. The 0.1 and 0.25 bands come from this practice.
A 2023 review of population stability testing in credit scoring describes the routine: lenders periodically test whether the client attribute distributions seen at scorecard development still hold at review. It also finds classical goodness-of-fit tests struggle on large samples and proposes effect-size and overlap alternatives.
The formula mirrors scorecard feature engineering too. ln(Actual / Expected) has the shape of weight of evidence, and Sudjianto and Burakov prove that Information Value is exactly PSI computed between the good and bad outcome distributions over identical bins. In R, PDtoolkit’s psi() defaults to bin = 10 and alpha = 0.05 and reports chi-square critical values next to the index instead of fixed bands. PSI vs CSI covers the score-versus-characteristic split.
Bins are the whole game
PSI’s number depends entirely on how you bucket. Two conventions dominate:
- Equal-width bins, fixed-width ranges across the variable’s span. WhyLabs uses 30 equal-width bins for its PSI implementation and does not yet support custom binning, and Arthur suggests 10 or 20 equal-width bins between the reference min and max.
- Equal-frequency (quantile) bins, with edges chosen so the reference distribution has equal counts per bin, typically 10 bins in the credit-scoring convention.
Too few bins wash out real shifts; too many turn sampling noise into false alarms, especially on a small production batch, as the noise-floor table above shows.
Whichever you pick, compute the bin edges once from the reference distribution and freeze them. Recomputing edges from each new batch bins both distributions to look uniform, which drives PSI toward zero regardless of actual drift. Arize’s binning guide gives the same rule: fit edges on training or a frozen validation baseline and apply them to production.
Wiring it up
PSI ships in every mainstream drift library, so the choice is rarely about the metric and usually about what else the library has to cover: batch tabular, streaming, or embeddings. Python drift detection libraries compared lays out that split.
Legacy API example: pin evidently==0.6.7 in an isolated environment. This uses that version’s DataFrame API and is not a current-API quickstart. Supply numeric feature DataFrames as train_df and prod_window_df. The current Evidently drift documentation lists psi with a default threshold of 0.1; do not mix API generations.
from evidently.report import Report
from evidently.metric_preset import DataDriftPreset
report = Report(metrics=[
DataDriftPreset(stattest="psi", stattest_threshold=0.2, drift_share=0.7)
])
report.run(reference_data=train_df, current_data=prod_window_df)
result = report.as_dict()
drift = next(
metric["result"] for metric in result["metrics"]
if metric["metric"] == "DatasetDriftMetric"
)
print(drift["number_of_drifted_columns"], drift["share_of_drifted_columns"])
Here, 0.2 is the illustrative per-column PSI cutoff and drift_share=0.7 sets the dataset-level decision threshold on the share of drifted columns. Review individual columns too: an aggregate rule can hide a failure in one important feature. Calibrate both thresholds against representative history.
If you would rather not pull in a monitoring library, PSI is short enough to hand-roll, which is worth doing at least once so you understand where the failure modes below come from:
import numpy as np
def finite_sample(values, name):
values = np.asarray(values, dtype=float)
if values.ndim != 1 or values.size == 0 or not np.isfinite(values).all():
raise ValueError(f"{name} must be a nonempty, finite, 1-D sample")
return values
def reference_edges(reference, bins=10):
reference = finite_sample(reference, "reference")
if isinstance(bins, bool) or not isinstance(bins, (int, np.integer)) or bins < 2:
raise ValueError("bins must be an integer of at least 2")
if reference.min() == reference.max():
raise ValueError("constant reference: use an explicit value-change check")
quantiles = np.linspace(0, 100, bins + 1)[1:-1]
interior = np.unique(np.percentile(reference, quantiles))
return np.r_[-np.inf, interior, np.inf]
def psi(expected, actual, bins=10, eps=1e-6):
expected = finite_sample(expected, "expected")
actual = finite_sample(actual, "actual")
if not np.isfinite(eps) or not 0 < eps < 1:
raise ValueError("eps must be finite and between 0 and 1")
edges = reference_edges(expected, bins)
exp_counts = np.histogram(expected, edges)[0]
act_counts = np.histogram(actual, edges)[0]
exp_pct = exp_counts / exp_counts.sum()
act_pct = act_counts / act_counts.sum()
exp_pct = np.clip(exp_pct, eps, None)
act_pct = np.clip(act_pct, eps, None)
exp_pct /= exp_pct.sum()
act_pct /= act_pct.sum()
return float(np.sum((act_pct - exp_pct) * np.log(act_pct / exp_pct)))
# Hypothetical overflow example: all 12 current observations count.
reference = np.arange(10, dtype=float)
current = np.r_[reference, 100.0, 100.0]
edges = reference_edges(reference)
assert np.histogram(reference, edges)[0].sum() == reference.size == 10
assert np.histogram(current, edges)[0].sum() == current.size == 12
print(round(psi(reference, current), 6)) # 0.164792
Only the interior boundaries come from reference quantiles; infinite outer edges keep values beyond the reference range, and repeated quantiles collapse to one boundary. The function rejects empty, nonfinite, multidimensional and constant-reference inputs, so handle missing data separately. The eps floor prevents zero probabilities, each distribution is renormalized after smoothing, and these choices belong in the metric definition alongside its thresholds.
Push the per-feature PSI value to whatever you already scrape metrics with. A prometheus_client gauge keyed on feature_name is enough to get it on a Grafana panel next to latency and throughput:
from prometheus_client import Gauge
psi_gauge = Gauge("model_feature_psi", "PSI vs training reference", ["feature_name"])
psi_gauge.labels(feature_name="transaction_amount").set(psi_value)
If you are logging with MLflow, attach it as a metric on the scoring run (mlflow.log_metric("psi_transaction_amount", psi_value)) so PSI history lives next to the model version it was scored against, rather than floating in a separate dashboard with no link back to what shipped.
For a wider view of where PSI sits relative to KS tests, Jensen-Shannon distance, and Wasserstein distance, see our guide to data drift detection methods, and SentryML’s monitoring-metrics taxonomy maps the full set and when each one earns its keep.
What you will see
Good: a per-feature PSI line hovering under 0.1 on a rolling 7- or 30-day window, flat enough that you stop looking at it except in a weekly review. Bad: one feature, say transaction amount after a pricing change, or a categorical field after a new signup source gets turned on, climbing past 0.25 while everything else stays flat. That isolation is the useful signal: PSI computed per feature tells you which input changed, not just that something did, so triage starts at the feature instead of the whole pipeline. Credit-risk practice gives that per-feature version its own name, the Characteristic Stability Index, and reads it against the score-level number; PSI vs CSI covers why the pair catches offsetting shifts that either one alone reports as stable.
The other useful signal is persistence rather than peak. A one-hour spike that decays back down is usually a batch job hiccup or a partial data-pipeline delay, not drift worth acting on. Real covariate shift, from a new customer segment, a currency added to checkout, or an upstream schema change that started zero-filling a field, climbs and stays there.
If every feature’s PSI moves in lockstep, suspect the pipeline before the world: a schema change, a null-fill default that changed, a transformation that differs between training and serving (training-serving skew), or a reference window that rolled over a seasonal boundary. Holiday traffic compared against a summer baseline will “drift” every year, and that is a calendar problem, not a model problem. Those pipeline-side shifts sit alongside the textbook covariate, label and concept cases in types of data drift, which is the classification step to run before a PSI spike gets called drift at all.
Caveats
Empty bins break the math. If a bin has zero actual or expected observations, the log term goes to infinity. Fiddler notes an empty bin can leave PSI undefined or unbounded and suggests adding 0.01 to each proportion or a count of 1 to each bin; Arthur floors proportions at 1e-6, like the eps above. Handled carelessly, PSI either blows up or silently reports zero contribution from a bin that should have flagged loudly. Pick an epsilon and document it, because it changes the number.
Categorical cardinality blows up fast. A categorical feature with hundreds of levels effectively becomes hundreds of near-empty bins, and PSI on it is mostly noise. Arize’s PSI explainer puts the useful range at around 50 to 100 unique values and suggests grouping the rest as “other,” the same way you would handle high-cardinality features anywhere else in the pipeline.
Bin count and edges are not portable across teams. Two teams computing PSI on the same feature with different bin counts will get different numbers and possibly different alert outcomes; the chi-square noise floor alone scales with the bin count. Treat PSI as a within-team, within-pipeline standard rather than a cross-org benchmark.
PSI tells you that something moved, not whether it hurts. It is a marginal, univariate statistic. It checks each feature’s distribution independently, has no visibility into how features move together, and none at all into whether P(y|x) has changed. A model can pass every PSI check on inputs while its actual relationship to the label has quietly broken; that is concept drift, and PSI will not see it. A feature can also drift because of a benign product change, such as new UI copy shifting session length, that does not touch model performance at all. Pair PSI on inputs with an accuracy or calibration metric once labels arrive. The ML model monitoring framework keeps input stability separate from performance evidence and response policy.
Decide the reference type up front and say so on the dashboard. A fixed training snapshot catches slow drift away from training but goes stale as the world legitimately changes; a rolling baseline catches sudden shifts but can mask a slow multi-month drift because each window looks only slightly different from the last.
FAQ
what is a good psi value
A PSI under 0.1 is conventionally read as stable, 0.1 to 0.25 as a moderate shift, and 0.25 or above as significant. Treat those as starting points. On small batches, sampling noise alone can exceed 0.1, while on 10,000 rows per side a PSI of 0.03 sits well above the noise floor. Backtest thresholds on healthy history first.
can psi be negative
No. In every bin the difference (Actual − Expected) and the log ratio ln(Actual / Expected) have the same sign, so each term is zero or positive and the sum never drops below zero. PSI is zero only when the binned distributions match. A negative result means the implementation is wrong, typically a ratio flipped inside the log.
does psi work for categorical variables
Yes. Treat each category as its own bin, count the share of reference and current rows in each, and apply the same formula. Give missing values and categories seen only in production their own bins, and group rare levels into “other,” since hundreds of sparse categories turn PSI into sampling noise rather than a drift signal.
how many bins should i use for psi
Ten quantile bins fitted on the reference is the credit-scoring convention and a sound starting point. Tools differ: WhyLabs uses 30 equal-width bins, Arthur 10 or 20. More bins raise the noise floor, because expected pure-noise PSI scales with bins minus one, so pair a higher bin count with larger samples and recalibrated thresholds.