vc_001 · Entity-static control
Status: ✓ Passing · Last run: 2026-08-24T17:41:01 · Pipeline: v2
· History: 16/19 runs passed (84%)
What was tested
One row per entity, no time dimension. A single feature drives the binary outcome with known strength; three other features are noise. This is the baseline 'happy path' case — if this fails, the engine itself is broken before any structural-shape complexity matters.
What we planted in the data
- 5,000 entities
- Planted driver:
usage_score - 3 noise feature(s) (no relationship to outcome)
- Target positive rate ≈ 20%
- Signal strength: 0.70 (sigmoid slope multiplier)
What the system did
- Training rows built: 5,000
- Features used in model: 4
- Model selected:
LogisticRegression - CV AUC: 0.928
Top features by importance:
usage_score_raw(1.947)noise_1_raw(0.035)noise_3_raw(0.019)noise_2_raw(0.018)
What we checked — all assertions passed
| Status | Assertion | Detail |
|---|---|---|
| ✓ | row_count_eq_n_entities |
training rows = 5000, expected = 5000 |
| ✓ | planted_feature_in_top_k_importance |
derivative(s) of usage_score in top 3: ['usage_score_raw'] |
| ✓ | auc_in_band |
AUC = 0.928 inside band [0.70, 0.95] |
| ✓ | noise_features_below_planted |
best planted derivative usage_score_raw is rank 1; no noise above |
How this could have gone wrong
- If
row_count_eq_n_entitiesfailed: df has exactly one row per entity. No fan-out, no row inflation. - If
planted_feature_in_top_k_importancefailed: The model recovered the planted driver — usage_score should rank in the top 3 features by importance. - If
auc_in_bandfailed: AUC inside the band achievable against signal_strength=0.7. Below 0.70 means the model didn't learn the planted signal; above 0.95 means something is leaking. - If
noise_features_below_plantedfailed: Every noise feature should rank below the planted driver. If a noise feature outranks it, the signal isn't being picked up correctly.
Why this case matters
The simplest possible structural shape — if this case ever fails, the failure isn't about shape handling; it's about something foundational (training, encoding, the model selection itself) being broken. Useful as a canary: a green entity-static control means the engine and the basic ML path are healthy.
Reproducing
# from auto_insight_api/
python -m validation.v2 run vc_001 --pipeline v2 -v
- Case config:
validation/v2/cases/01_entity_static_control.yaml - Data shape:
entity_static - Analytics type:
ml_binary