vc_017 · Date column stored as VARCHAR (panel)

Status: ✓ Passing · Last run: 2026-06-04T23:45:38 · Pipeline: v2 · History: 2/3 runs passed (67%)

What was tested

Panel data with snapshot_date stored as STRING type (values formatted YYYY-MM-DD but never parsed to DATE). Typical of CSV imports where the loader didn't infer the column type. The pipeline's snapshot detection + temporal feature templates depend on the time column being a real DATE/TIMESTAMP. With a STRING-typed time column, the pipeline either auto-coerces, falls back to treating it as categorical, or halts. Acceptable behaviors: 1. Coerce — recognize the date-shaped strings and CAST in queries 2. Surface a "snapshot_date is STRING but should be DATE" issue 3. Degrade gracefully with a near-random model + ml_sanity warning Rejected behaviors: - Crash on DATE_DIFF / TIMESTAMP_DIFF against a STRING column - Silently produce a confident model with no warning that the temporal structure was lost

What we planted in the data

• 100 entities
• 12 snapshots per entity at 30-day cadence
• Planted driver: usage_score
• 2 noise feature(s) (no relationship to outcome)
• Target positive rate ≈ 30%
• Signal strength: 0.70 (sigmoid slope multiplier)

What the system did

(no metrics available)

What we checked — all assertions passed

How this could have gone wrong

• If pipeline_halted_with failed: Pipeline must halt with a message naming the STRING type problem. The fix shipped alongside this case (_validate_time_column_type) catches the type mismatch at intent_classification, before feature_materialize wastes 4+ minutes generating features that all fail dry-run validation with cryptic BQ errors.
• If pipeline_halted_with failed: Halt message must name the specific column to fix, not a generic "type error". Customer reads it, knows exactly what to fix in their loader / data prep.
• If pipeline_halted_with failed: Halt message must point at the fix — a literal example cast (PARSE_DATE) the customer can copy. Without this, a customer with a STRING date column might know SOMETHING is wrong without knowing how to fix it.
• If no_model_produced failed: A model trained on this data would be silently noise (every temporal feature template would fail dry-run). Confirm the pipeline correctly refused to train rather than producing a misleading model.

Why this case matters

Dates-as-strings is the third major CSV-import wart (after numeric-as-string and high-null). Every customer dataset loaded via CSV has at least one column where the type inference missed. The pipeline must survive — ideally recovering the date semantics, acceptably losing the temporal structure with a clear warning.

Reproducing

# from auto_insight_api/
python -m validation.v2 run vc_017 --pipeline v2 -v

• Case config: validation/v2/cases/17_date_as_varchar.yaml
• Data shape: entity_date_as_varchar
• Analytics type: ml_binary

← back to validation report