Checkpoint a pipeline

Run an extract → transform → load pipeline that snapshots the container after every stage.

Long pipelines fail partway through. sb.checkpoint() takes a restorable snapshot of the container's filesystem, so a crash in stage 3 resumes from the stage-2 snapshot instead of re-running extract and transform from scratch.

This workflow runs a small ETL pipeline end-to-end and checkpoints after each stage, then lists the snapshot chain it left behind.

Not connected
createSandboxexeccheckpointlistCheckpoints~50s
Connect to a sandbox server above to run this.

What just happened

  • Extract wrote 5,000 rows to /data/raw.jsonl, then sb.checkpoint() snapshotted.
  • Transform aggregated them into /data/agg.json, then checkpointed again.
  • Load wrote the final artifact.
  • The last step called sb.listCheckpoints() — the two snapshot IDs, newest first.

Sandbox.resume(sandboxId) restores the container from the newest snapshot. Execs that completed before that snapshot replay their recorded output from the ledger instantly rather than running again; anything after runs live. The reader never sees the difference.

Full worked version: the Resumable ETL Pipeline cookbook.

CI test runner

Run a suite in a disposable sandbox, watch it fail, apply a fix, watch it pass — then turn the output into a structured report.

Fork a sandbox

Do the expensive setup once, then fork the live container into independent children.

On this page

What just happened