dco_002 · Deployment Orchestration: scheduler fires → backend refreshes → live_artifact updates
Status: ✓ Passing · Last run: 2026-06-11T04:33:41 · Pipeline: deployment_orchestration_runtime
· History: 1/2 runs passed (50%)
What was tested
Where dco_001 locks the contract at the IaC layer (create → verify config → teardown), dco_002 locks it at the RUNTIME layer. Full path:
- Stage GCS model + BQ source + Firestore deployment_spec (mirrors handle_render_live's test setup) 2. Call
deploy_to_cloud(spec, dry_run=False)with SERVICE_URL pointed at the DEV backend 3. Manually trigger the scheduler entry viaCloudSchedulerClient.run_job()4. Poll Firestore forlive_artifact.generated_at/runtime.last_run_atto update (up to 5 min — re_render_html LLM is the long pole) 5. Verify: refresh observed, live_artifact.html non-empty, last_run_status="ok", no section errors
After this passes, flipping dry_run=false in the FE is defended by two green tests: dco_001 contract — provisioner emits the right scheduler config dco_002 runtime — that config actually fires + refreshes
Cost: ~$0.01 (one re_render_html Sonnet call), ~60s wall-clock. TASK_AUTH discovery: the test pulls the live value from the dev Cloud Run service via gcloud run services describe, so the HMAC pre-computed in deploy_to_cloud matches what dev backend's worker expects to verify. Zero local-env setup beyond an authenticated gcloud.
What we planted in the data
- 300 entities
- Signal strength: 0.50 (sigmoid slope multiplier)
n_new_rows=30
What the system did
- Model selected:
DeploymentOrchestrationRuntimeRunner
What we checked — all assertions passed
| Status | Assertion | Detail |
|---|---|---|
| ✓ | orchestration_provisioner_succeeded |
provisioner returned cleanly; scheduler_job=None |
| ✓ | orchestration_refresh_observed |
refresh observed at 2026-06-11T04:33:39.368856+00:00, status=ok, html=5627 chars, section_errors=[] |
How this could have gone wrong
(no assertion descriptions provided)
Why this case matters
Until dco_002 is green, "did Cloud Scheduler → backend actually work?" requires manual gcloud + UI clicking. With it green, the validation page is the gate. Any future change to /worker/deploy-refresh, deploy_to_cloud, handle_render_live, or the HMAC scheme has to keep dco_002 passing.
Reproducing
# from auto_insight_api/
python -m validation.v2 run dco_002 --pipeline deployment_orchestration_runtime -v
- Case config:
validation/v2/cases/dco_002_orchestration_runtime.yaml - Data shape:
deployment_orchestration_runtime - Analytics type:
deployment_orchestration_runtime