vc_012 · Descriptive analytics — user event log
Status: ✓ Passing · Last run: 2026-06-04T06:25:25 · Pipeline: analyze
· History: 1/1 runs passed (100%)
What was tested
Event-level data: one row per event, with user_id repeating across many rows. The analyze_agent is asked structural questions whose answers come from aggregations across the event stream (distinct user count, most common event_type, total event count). Fills the previously-uncovered event_level data shape on the coverage matrix. Same chat-surface backend as vc_006-010 (pipeline_analyze), but the data shape itself exercises a path the agent doesn't see in entity-static / panel / segmented cases — many rows per entity, no pre-aggregated target, time as event timestamp rather than snapshot date.
What we planted in the data
- 200 entities
- Signal strength: 0.50 (sigmoid slope multiplier)
What the system did
- Model selected:
analyze_agent
What we checked — all assertions passed
| Status | Assertion | Detail |
|---|---|---|
| ✓ | tool_was_used |
agent used query_bigquery in session up through 'q_total_events' |
| ✓ | answer_includes_number_near |
answer contains 3,984.00, within 0% of expected 3,984.00 |
| ✓ | tool_was_used |
agent used query_bigquery in session up through 'q_total_events' |
| ✓ | answer_mentions_fact |
all expected facts present in answer to 'q_most_common_event' |
| ✓ | tool_was_used |
agent used query_bigquery in session up through 'q_total_events' |
| ✓ | answer_includes_number_near |
answer contains 3,984.00, within 0% of expected 3,984.00 |
How this could have gone wrong
- If
tool_was_usedfailed: Distinct user count — must touch the data, not estimate. - If
answer_includes_number_nearfailed: Exact distinct-user count is 200; tight tolerance because COUNT(DISTINCT user_id) is exact, no sampling noise. - If
tool_was_usedfailed: Most-common-type is a GROUP BY aggregation — must query the data. - If
answer_mentions_factfailed: Under the planted weights (page_view = 0.60), page_view is the most frequent event_type with overwhelming probability across any realized sample. The agent's reply must name it. - If
tool_was_usedfailed: Total event count — COUNT(*) on the table. - If
answer_includes_number_nearfailed: Exact event count for seed=12 is 3984. Tight tolerance — any drift here means the agent's COUNT(*) returned something other than what's in the table.
Reproducing
# from auto_insight_api/
python -m validation.v2 run vc_012 --pipeline analyze -v
- Case config:
validation/v2/cases/12_descriptive_event_level.yaml - Data shape:
event_level - Analytics type:
analytical_report