Assertion Validation

dcl_001 · Deployment Lifecycle: train → compose → transform → deploy → fire → read

Status: ✓ Passing · Last run: 2026-06-09T07:18:42 · Pipeline: deployment_lifecycle · History: 1/2 runs passed (50%)

What was tested

The cherry-on-top integration test. Every production step in order, no mocks:

  1. Train a sklearn LogisticRegression on synthetic history. 2. joblib.dump the trained model to a real GCS blob. 3. Upload source data to a real BigQuery table. 4. Hand-craft a 2-section "build report" (a KPI section + a training section). 5. Invoke understand_deployment_goal (real LLM) — get per- section intents. 6. Invoke transform_section_for_live per section (real LLM for the transform_score path). 7. Build a real DeploymentSpec, write to Firestore at projects/validation_dcl_<uid>. 8. Fire handle_render_live (real handler → real execute_live_refresh → real re_render_html LLM). 9. Read live_artifact back from Firestore; assert html is non-empty AND distinct from the style_reference template (proves re_render_html actually rendered, not fell back).

This is the only test that exercises the FULL deploy stack at the same layer the customer hits it.

What we planted in the data

What the system did

What we checked — all assertions passed

Status Assertion Detail
✓ lifecycle_all_steps_succeeded all 9 lifecycle steps completed; sections_for_live=2, final html=6729 chars
✓ lifecycle_handler_returned_200 handler returned 200 with 2 sections
✓ lifecycle_compose_assigned_intents composer assigned intents: {'row_count': 'keep', 'fit_model': 'transform_score'}
✓ lifecycle_html_distinct_from_style persisted html (6729 chars) is a real render, not the style template

How this could have gone wrong

(no assertion descriptions provided)

Why this case matters

If ANY layer of the deploy stack regresses — the composer prompt, the transform prompt, Firestore serialization, the handler glue, the runtime exec sandbox, the re_render_html LLM — this case fails. The previous tests catch their respective layers in isolation; this case proves they integrate. Slow (~1-2 min) and costly (~$0.05/run) but the most production-faithful test in the suite.

Reproducing

# from auto_insight_api/
python -m validation.v2 run dcl_001 --pipeline deployment_lifecycle -v

← back to validation report