Deep dive — chart 2

Chart 2 — I3 baseline vs recent distribution

The how. Where Chart 1 shows when the I3 line rose, this chart shows how the I3 score distribution shape shifted. Mass moved right, into the decline / refer-SIU lane.

I3 baseline vs recent underwriting-score distribution comparison

I3 underwriting-score distribution — baseline (blue) vs recent (orange). Mass shifted right, into the decline / refer-SIU lane threshold at score 67.

What you are looking at

X-axis: underwriting score (0 to 100)
Y-axis: density of quote intakes at that score for I3
Blue distribution: baseline window (first 30 days), what I3 scoring normally looks like
Orange distribution: recent window (last 30 days), what I3 scoring looked like at Snapshot time
Vertical line at score 67: threshold for routing to decline / refer-SIU lane

What the shape shift means

The orange distribution's mass has moved right compared to the blue distribution. Three things are happening at once:

The peak (mode) rises. The most common I3 score used to be near the standard-auto-bind / manual-review boundary; now it sits well inside the decline / refer-SIU lane.
The right tail grows. Far more I3 quotes score above the decline / refer-SIU threshold.
The left tail shrinks. Fewer I3 quotes score in the standard-auto-bind range.

Underwriting translation: the AI now systematically scores I3 quotes (mid credit + mixed urban-lower-income zip + slightly-higher prior-loss counts) as much higher risk than baseline. The score does not distinguish between "actual outsized risk" and "shifted input mix compounding a baked-in offset." The distribution shape drift is the mathematical fingerprint of that underwriting failure mode.

What KL-divergence quantifies

Kullback-Leibler divergence measures how different one distribution is from another. Values:

KL < 0.05: same distribution (random noise)
KL 0.05-0.2: mild shift, monitor
KL 0.2-1.0: significant shape shift, flag as event
KL > 1.0: substantial shape shift, high-severity event
KL > 2.0: distributions are almost non-overlapping — the applicant pool the AI is scoring is behaviorally different from the pool it was trained on

The sample's I3 KL-divergence hit 4.378 on Day 44 and 4.105 on Day 65. That is far above threshold, and it stayed there.

The underwriting + regulatory read

Underwriting read: the AI has drifted out of calibration for the current I3 applicant mix. Manual underwriter re-review should apply until retraining + validation.

Regulatory read: the shape shift is precisely the kind of "measurable drift in performance on a specific applicant subpopulation" that NAIC AI Model Bulletin, NY DFS Circular Letter No. 7, and Colorado Reg 10-1-1 Section 5.A.10 expect the operator to detect and act on.

The next chart

The baseline-vs-recent chart shows how the I3 distribution shifted. The lane-rate chart shows the operational consequence: which lane are I3 applicants getting routed to now vs baseline.

Chart 3 — lane rate per applicant group →

$499 Snapshot. 3 business days.

Same shape-analysis for your carrier's actual AI surface + full analysis report.

Buy $499

Back to insurance sample hub.