vc_009 · Descriptive analytics — multi-table (clients × transactions)

Status: ✓ Passing · Last run: 2026-06-04T05:55:00 · Pipeline: analyze · History: 2/3 runs passed (67%)

What was tested

Same data shape vc_004 uses for ML (multi_table — clients + transactions tables, joinable on client_id), here surfaced to the analyze_agent as an analytical-chat scenario. The generator computes the realized planted facts and emits them as questions in its GeneratedDataset.planted_truth. pipeline_analyze uploads BOTH tables to BQ and registers them in the test Firestore session so the agent sees the multi-table schema. Includes the JOIN-required question (q_top_segment_by_amount — segment lives on clients, amount lives on transactions; must JOIN on client_id). This was originally a known-failing assertion because the chat surface didn't discover non-primary tables; the fix (analyze_agent now reads a session-level data_dict_yaml as a fallback when no org curated dict exists, and pipeline_analyze injects one for multi-table cases) lets the agent see both tables from turn 1 and write JOIN SQL correctly.

What we planted in the data

• 500 entities
• Signal strength: 0.70 (sigmoid slope multiplier)

What the system did

• Model selected: analyze_agent

What we checked — all assertions passed

How this could have gone wrong

• If tool_was_used failed: Distinct-client count across a multi-table source — must touch the data. Either a COUNT(*) on clients or a profile is acceptable.
• If answer_includes_number_near failed: Exactly 500 clients in the primary table.
• If tool_was_used failed: Total transaction count must come from the transactions table — counting clients (500) is the obvious wrong answer here. This assertion's intent: agent looked at the RIGHT table, not the familiar one.
• If tool_was_used failed: JOIN-required question — must write SQL that touches both tables. profile_data alone is insufficient (single-table only).
• If answer_mentions_fact failed: Under seed=42 with the given segment weights, SMB carries ~50% of clients, so even with lower per-client amounts, its total transaction amount dominates. The agent's reply must surface 'smb'.

Why this case matters

Proves the framework's multi-table pipeline works: both tables get uploaded to BQ, both appear in the session, the agent can be asked per-table descriptive questions (count clients, count transactions). Establishes the harness; the JOIN-required assertion is a follow-up once the chat surface reliably discovers all session tables without a curated org dictionary.

Reproducing

# from auto_insight_api/
python -m validation.v2 run vc_009 --pipeline analyze -v

• Case config: validation/v2/cases/09_descriptive_multi_table.yaml
• Data shape: multi_table
• Analytics type: analytical_report

← back to validation report