Deep dive — scenario walkthrough
The sample scenario — what the portfolio-recommendation AI was doing and why it failed silently
Mid-market RIA, $500M-$5B AUM, 500-5,000 client households, dual-registered or state-registered. Standard robo-advisor / TAMP-adjacent AI deployment. Standard compliance instrumentation. Standard silent failure.
The AI system
Model: portfolio-recommendation engine (mock version robo-portfolio-recommender-v4.2.1).
Function: at client onboarding or quarterly review, each client's stated objective + risk tolerance + account balance + years-to-horizon + account type is scored 0-100 for allocation preference. Score routes the client to one of three allocation lanes:
- Passive Index (Bogleheads) (score 0-33) — low-fee conservative allocation, index-tilt
- Balanced Managed (score 34-66) — moderate blended allocation, mixed active + passive
- Active Alpha / High Fee (score 67-100) — high-fee actively-managed, alternatives, structured products
Training window: recommendations through 2026-02-28. Deployed Day 1 of the audit period. Not retrained during the 90-day production run.
The client groups
The audit segmentation used five common RIA client groups:
| Client group | Definition |
|---|
| R1 | Young accumulator (age 25-40) |
| R2 | Mid-career high-net-worth ($1M-$5M) |
| R3 | Age 50-65 with mid-net-worth ($500K-$1M), conservative near-retirees |
| R4 | Retired income-oriented (age 65+) |
| R5 | Business owner / concentrated position |
What shifted during the 90 days
Two input distributions shifted silently and simultaneously:
- Client-mix shift. A 401(k)-rollover marketing outreach amplified R3 representation in the incoming client stream. R3 share of new advice sessions grew ~30% post-Day 45.
- Account-type + objective shift. Post-shift R3 clients skewed toward 401(k)-rollover account type, stated "retirement" objective, larger consolidated balances, and shorter years-to-horizon. Exactly the profile where a recommendation engine trained on rollover-heavy historical data + implicit revenue-share tier weighting OVER-scores active-alpha suitability.
The AI was not retrained. Its baked-in R3-directed active-lane offset compounded with the shifted-input mix. R3 active-alpha / high-fee lane rate rose from 22% baseline to 68% recent. A 46 percentage-point differential, in the wrong direction for the client segment least positioned to absorb it, silent to every aggregate metric the firm was watching.
What the firm's dashboards showed (green)
- AUM growth — on plan
- Aggregate fee revenue — on plan
- Mean allocation-recommendation score — stable (R3 rise cancelled by R2/R4 stability in aggregate)
- Model confidence — stable
- Client retention / redemption rate — stable within noise
Standard compliance and revenue instrumentation cannot see group-level differential in aggregate. That is not a criticism of the operations team; it is a structural property of aggregation.
What the independent-verifier sensor saw
- R3 score distribution shape diverged from baseline (KL-divergence > 2.0) starting Day 44 and recurring every 7 days
- R3 active_alpha_high_fee routing rate rose sharply from Day 44 forward
- R5 distribution shape also diverged on Day 58 (medium-severity)
- 9 total drift events, 8 high-severity, 1 medium-severity
See how the drift-detection chart is read →
The one sentence a CCO would care about
The AI portfolio-recommendation engine's active_alpha_high_fee lane rate for R3 (age 50-65, mid-net-worth conservative near-retirees) rose from 22% to 68% over the audit period — a 46 percentage-point group differential that occurred silently while the firm's AUM growth + aggregate fee revenue tracked normal throughout.
$499 Snapshot on your firm's actual AI system.
Same scenario structure. Your data, your AI, your jurisdictional footprint. Same 3 business day turnaround.
Buy $499