Resumable ETL Pipeline
A multi-stage pipeline that checkpoints after every stage and resumes without redoing completed work.
A multi-stage ETL pipeline (extract → transform → load) that checkpoints after each stage, so a crash — or just wanting to re-run "load" in isolation — doesn't mean paying for extract and transform again.
Setup
Start OpenSandbox in Docker (one-time setup):
bunx alineo-cli initRun it
What it does
Extract — installs pandas, writes raw CSV data, then sb.checkpoint("after-extract").
Transform — aggregates revenue by region with pandas, writes transformed.csv, then
sb.checkpoint("after-transform").
Load — reads back the final output and "publishes" it (prints it).
Simulates picking the pipeline back up later:
client.resume(sandboxId) restores the container from the last checkpoint (after-transform).
The first exec() call after resume replays instantly from the ledger — no network call, no
re-install. This demo re-issues the identical pip install command from the extract stage as a
matter of good practice; the replay itself is positional (see the note below), not a check that
the command matches.
transformed.csv is already present on the restored container's filesystem, so the load stage
reads it straight away — the transform never re-runs.
Replay is positional, not content-matched
Every sb.checkpoint(tag) is a real container snapshot, not just a ledger bookmark — the restored
container genuinely has extract and transform's output on disk. The ledger replay on top of that
is an optimization: the Nth exec() call since resume returns the Nth call's original result,
whatever command is actually passed — there's no check that it matches what was recorded. Always
re-issue calls in the same order as the original run. See Checkpoint &
Resume for the primitive this recipe builds on.
Server proxy
All examples default to useServerProxy: true — traffic routes through the OpenSandbox server so
Docker bridge IPs don't need to be reachable directly. Set USE_SERVER_PROXY=false to disable
(e.g. when using uvx opensandbox-server on the host).