vc_011 · Entity-static regression

Status: ✓ Passing · Last run: 2026-06-04T23:40:19 · Pipeline: v2 · History: 2/2 runs passed (100%)

What was tested

The regression analog of vc_001 (entity-static control). One row per entity, one continuous target driven linearly by a planted feature with Gaussian noise; three noise features carry no signal. This is the canary for the v2 pipeline's regression code path (intent_classification → _train_regression → Ridge/RF/LightGBM studies → MAE/RMSE metrics). If this fails, regression itself is broken — none of the future regression cases on richer shapes will diagnose anything; they'll all fail at the same point.

What we planted in the data

• 5,000 entities
• Planted driver: usage_score
• 3 noise feature(s) (no relationship to outcome)
• Signal strength: 0.70 (sigmoid slope multiplier)
• target_scale = 5.0
• noise_std = 1.0

What the system did

• Training rows built: 5,000
• Features used in model: 4
• Model selected: Ridge
• MAE: 0.774

Top features by importance:

• usage_score_raw (3.510)
• noise_3_raw (0.035)
• noise_2_raw (0.019)
• noise_1_raw (0.005)

What we checked — all assertions passed

How this could have gone wrong

• If row_count_eq_n_entities failed: df has exactly one row per entity. No fan-out, no row inflation.
• If planted_feature_in_top_k_importance failed: The regressor recovered the planted driver — usage_score should rank in the top 3 features by importance.
• If noise_features_below_planted failed: Every noise feature should rank below the planted driver. If a noise feature outranks it, the signal isn't being picked up.
• If regression_signal_recovered failed: Model MAE must be < 60% of the constant-predictor MAE. With signal_strength=0.7 and σ_noise=1.0, target SD ≈ 3.65 so the predict-the-mean MAE ≈ 2.91; a working model should land near the Bayes-optimal MAE ≈ 0.80 — comfortably under the 60% bar.
• If mae_in_band failed: MAE inside [0.5, 1.5]. Lower bound 0.5 is below the Bayes-optimal (0.8) — a model achieving MAE < 0.5 would be impossibly accurate given the planted noise level, suggesting leakage. Upper bound 1.5 is comfortable headroom above Bayes-optimal but well below the constant-predictor MAE (~2.91); landing here means the signal is being recovered without overfitting.

Why this case matters

Without this case, "the platform supports regression" is unverified. Every other regression-focused case (e.g. on panel or multi-table shapes) builds on the assumption that the regression training path works at all — this case is the canary for that assumption.

Reproducing

# from auto_insight_api/
python -m validation.v2 run vc_011 --pipeline v2 -v

• Case config: validation/v2/cases/11_entity_static_regression.yaml
• Data shape: entity_static_regression
• Analytics type: ml_regression

← back to validation report