vc_015 · Mixed-type numeric column (currency as STRING)

Status: ✓ Passing · Last run: 2026-06-04T23:43:41 · Pipeline: v2 · History: 3/3 runs passed (100%)

What was tested

Real-world failure mode: customer revenue data loaded from CSV with values stored as STRING in a mix of formats — "$1,200.00", "950", "1.5k", blanks. The column LOOKS numeric to a human but BQ has it typed as STRING. Tests how the v2 pipeline handles type-mismatched features. Any of these is acceptable behavior: 1. Coerce/clean the STRING values and use revenue as a real feature (best case — signal is recovered) 2. Drop the column at the signal gate (acceptable — model loses the planted signal but the pipeline ran cleanly) 3. Halt with a clear "revenue is STRING but looks numeric" issue (also acceptable — surfaces the data quality issue) Failing behaviors the case rejects: - Treat as a categorical and explode into N one-hot columns - Crash with a "STRING can't be averaged" error in feature_materialize - Silently produce a model with no warning AND no signal recovery

What we planted in the data

• 2,000 entities
• 3 noise feature(s) (no relationship to outcome)
• Target positive rate ≈ 30%
• Signal strength: 0.70 (sigmoid slope multiplier)

What the system did

• Training rows built: 2,000
• Features used in model: 10
• Model selected: RandomForestClassifier
• CV AUC: 0.456

Top features by importance:

• noise_3_raw (0.338)
• noise_2_raw (0.336)
• noise_1_raw (0.325)
• revenue_format_empty (0.000)
• revenue_format_dollar_decimal (0.000)

What we checked — all assertions passed

How this could have gone wrong

• If row_count_eq_n_entities failed: df has exactly one row per entity. Catches any row-inflation from a misguided JOIN on the dirty STRING values.
• If feature_count_below failed: Final feature count must be ≤ 30. With ~2000 distinct revenue strings, naive one-hot would blow past this. A pipeline that coerces revenue (1 col), drops it (0 cols), or top-K caps (~11 cols) all sit well under 30.
• If no_pipeline_errors failed: Clean run — the pipeline must survive the dirty STRING column. Crashing in feature_materialize because AVG/STDDEV was called on a STRING is the specific failure mode this catches.
• If pipeline_halted_with failed: The pipeline must SURFACE the near-random result so the customer knows the model isn't predictive — ml_sanity issues this warning when CV AUC < 0.55. Without it, a silent no-signal model would ship looking fine. Note: assertion uses pipeline_halted_with but the pipeline didn't actually halt — the checker scans issues+errors for the substring, which is what we want here (warning surfaced, not error thrown).

Why this case matters

Currency / amount columns stored as STRING is one of the top three data-quality issues we see in customer uploads (along with dup entity keys and high-null demographics). The pipeline must degrade gracefully — either parse it, drop it, or surface a clear data-quality warning. Catastrophic failure (exploding the feature space or crashing on AVG) is a customer-visible bug.

Reproducing

# from auto_insight_api/
python -m validation.v2 run vc_015 --pipeline v2 -v

• Case config: validation/v2/cases/15_mixed_type_numeric.yaml
• Data shape: entity_mixed_type_numeric
• Analytics type: ml_binary

← back to validation report