Operations and observability

Materialize a generated run into operational storage, verify snapshots, and interpret lag, duplicates, invalid records, late events, and reconciliation signals. Docker services are required for the full lakehouse workflow.

Time	Extras	Output
45–75 min	lakehouse, scale optional	quality audit and checkpoint manifest
45–60 min	postgres optional	repair report and reconciliation
30–45 min	lakehouse, observability optional	snapshot and SLO evidence

The PostgreSQL and lakehouse notebooks include their client installation cells, local startup commands, bounded health checks, and cleanup guidance. Offline Bronze/Silver/Gold projections and reconciliation manifests remain complete without Docker.

Tutorials

• Audit quality faults and build observable projections

• Materialize a lakehouse snapshot and verify it

• Resume a scale run from a checkpoint

• Repair damaged data and replay a bounded interval

• Persist PostgreSQL rows idempotently

• Verify Iceberg time travel and observability signals

Related guides: Kafka reliability, data-quality incidents, and production serving.

Next path: Advanced experiments.