Deep dive — scenario walkthrough
The sample scenario — what the SC auto-claims AI was doing and why it failed silently
Mid-size South Carolina-domiciled P&C carrier. ~150K-500K personal auto policies-in-force + ~20K-80K homeowners across the state. AI-driven claims-severity classifier + adjuster-routing engine at the front of every incoming auto bodily-injury claim. Standard-of-market AI deployment. Standard operational instrumentation. Standard silent failure.
The AI system
Model: auto BI claims-severity classifier + adjuster-routing engine (mock version sc-claims-severity-v2.3.1).
Function: each incoming auto BI claim is scored 0-100 on projected severity + fraud-signal, using structured claim fields + loss-narrative text + prior-loss profile + territory + credit-based-insurance-score inputs. Score routes the claim to one of three lanes:
- Fast-Close (score 0-33) — low-severity, low-fraud-signal, direct-to-close pathway
- Standard Adjust (score 34-66) — human adjuster ownership + standard investigation
- SIU-flag (score 67-100) — hold pending Special Investigations Unit review
Audit period: 90 days. Claim volume in scope: ~12,000 auto BI claims statewide.
Training window: claims through 2026-02-15. Deployed Day 1 of the audit period. Not retrained during the 90-day production run.
The county groups
The audit used five county groupings reflecting the SC market shape:
| County group | Counties |
|---|
| C1 Upstate | Anderson, Cherokee, Greenville, Oconee, Pickens, Spartanburg |
| C2 Midlands | Aiken, Lexington, Newberry, Richland, Saluda, Fairfield, Kershaw |
| C3 Coastal | Beaufort, Charleston, Colleton, Georgetown, Horry, Jasper |
| C4 Pee Dee | Chesterfield, Darlington, Dillon, Florence, Marion, Marlboro, Williamsburg |
| C5 Piedmont | Abbeville, Chester, Edgefield, Greenwood, Lancaster, Laurens, McCormick, Union, York |
What shifted during the 90 days
Three input distributions moved silently and simultaneously in the C3 coastal group:
- Post-storm claim mix shift. Hurricane-season activity in coastal counties raised the share of claims involving vehicle-damage-plus-BI patterns the model had seen less of in training.
- Provider-narrative shift. A larger share of coastal-county BI claims involved a small number of high-volume medical-provider networks whose narrative-text patterns triggered fraud-signal features in the classifier.
- Territory + credit-adjacent input shift. Post-shift C3 claimants skewed toward higher-risk-class territories and slightly lower credit-based-insurance-score bands.
The AI was not retrained. Its baked-in coastal-provider offset compounded with the shifted input mix. C3 SIU-flag lane rate rose from ~5% baseline to ~19% recent — a 14 percentage-point group-differential in the wrong direction, silent to every aggregate metric the carrier was watching. First high-severity drift event landed Day 44.
What the carrier's dashboards showed (green)
- Aggregate cycle-time from FNOL to close — stable
- Mean severity score across all claims — stable
- Aggregate paid-loss vs case-reserve variance — within normal bands
- Model confidence — stable
- SIU-referral rate statewide — slight uptick, within tolerance
Standard claims instrumentation cannot see group-differential in aggregate. That is not a criticism of the claims-ops team; it is a structural property of aggregation.
What the independent-verifier sensor saw
- C3 score distribution shape diverged from baseline (KL-divergence > 2.0) starting Day 44
- C3 SIU-flag routing rate rose across every 7-day window from Day 44 forward
- 18 total drift events over the audit period: 11 high-severity, 7 medium-severity
- ~2,900 C3 claims over the ~46-day drift window; ~551 routed to SIU-flag (~19% recent rate) vs baseline ~5% expected; delta = ~406 potentially misrouted claims
See how the drift-detection chart is read →
The one sentence a Chief Claims Officer would care about
The auto BI claims-severity classifier's SIU-flag lane rate for C3 (Beaufort, Charleston, Colleton, Georgetown, Horry, Jasper) rose from ~5% to ~19% over the audit period — a 14 percentage-point group-differential in the wrong direction that occurred silently while the carrier's claims dashboard showed aggregate cycle-time and closure metrics within normal bands throughout, putting each affected claim at risk under the SC UCSPA "prompt investigation" standard.
$499 Snapshot on your carrier's actual AI system.
Same scenario structure. Your data, your AI, your SC DOI footprint. Same 3 business day turnaround.
Buy $499