dc_005 · Deployment: schema drift — feature column missing upstream
Status: ✓ Passing · Last run: 2026-06-09T06:21:11 · Pipeline: deployment
· History: 1/1 runs passed (100%)
What was tested
A feature the model was trained on (noise_2) has disappeared from the source pipeline. Deployment fills the missing column with 0 to keep scoring AND emits a drift warning — predictions may be unreliable because the model is now scoring zero for a feature that used to vary. This is the more dangerous drift case. Silently filling with 0 produces "predictions" that are systematically wrong, but it doesn't crash. Only the warning saves operators from never noticing.
What we planted in the data
- 1,000 entities
- Signal strength: 0.50 (sigmoid slope multiplier)
n_new_rows=100n_prior=500
What the system did
- Training rows built: 1,000
- Features used in model: 3
- Model selected:
DeploymentScoringRunner
What we checked — all assertions passed
| Status | Assertion | Detail |
|---|---|---|
| ✓ | deployment_completed_cleanly |
clean run |
| ✓ | scored_row_count_eq |
scored 100 rows, expected 100 |
| ✓ | drift_warning_present |
drift warning surfaced: [deployment] schema drift detected: 1 feature(s) missing in new data (['noise_2']). Filled with 0 — predictions m… |
| ✓ | prior_predictions_unchanged |
prior predictions row count unchanged (500) |
How this could have gone wrong
(no assertion descriptions provided)
Why this case matters
The catastrophic-failure-mode of missing-column drift is "predictions look fine but are silently wrong." The drift warning is the only thing saving the deployment from this — if it ever stops firing, this case will fail loudly.
Reproducing
# from auto_insight_api/
python -m validation.v2 run dc_005 --pipeline deployment -v
- Case config:
validation/v2/cases/dc_005_schema_drift_missing.yaml - Data shape:
deployment_schema_drift_missing - Analytics type:
deployment