Durability contract
Exactly which failures a swarm survives today — a daemon crash, a lost container, a machine reboot — and which it doesn't yet.
"Resumable" and "durable" mean different things here, and the stack is the first one, not yet the second:
- Resumable: alineod, the agent's container, or the whole machine can go down, and every agent comes back to its last completed turn — without anyone rebuilding state by hand. Everything needed for that lives on the machine alineod runs on.
- Durable: the swarm can be rebuilt on a different machine after the disk it ran from is gone. That needs a backup and restore path that isn't built yet.
Don't take either word further than this page says — if a capability isn't in the table below, it isn't built, regardless of what any other page implies.
What survives what, today
| Layer | alineod crash | Container loss or machine reboot | Disk or host loss |
|---|---|---|---|
Sandbox session (client.resume) | yes | yes, to the last sb.checkpoint() — snapshot + exec replay | no — a snapshot is a docker commit on the sandbox host |
Agent SDK (Alineo.reattach / resume) | yes | yes, to the last turn checkpoint — a fresh container from the setup snapshot, with the latest checkpoint restored into it | no — checkpoints are files next to the SDK ledger |
| alineod swarm | yes — see Crash recovery | yes, to the last completed turn, for specs with "checkpoint": true — the restore ladder | no |
| alineod agent memory | yes | yes — keyed by agent identity, not sandbox | no — one SQLite file; back it up (scripts/backup.ts) |
Three things are called a snapshot or a checkpoint in this stack, and only one of them carries an agent's conversation:
- the setup snapshot — taken once per spec, so
Alineo.start()skips reinstalling Pi and packages. A fresh container restored from it is empty: right packages, no conversation. - the turn checkpoint — a tarball of the agent's home directory (
/root: Pi's session file and what the agent wrote there, package caches excluded), taken after every turn that finishes cleanly when the spec sets"checkpoint": true. alineod takes these; the newest three per sandbox are kept. This is what restores a conversation. sb.checkpoint()— the Core SDK's named OpenSandbox snapshot of a whole container, used byclient.resumefor exec replay.
What a restored agent has
- Its conversation, to the last completed turn. A turn that was running when the container went is not in the
checkpoint. It is recorded as
agent.turn_interruptedand settles with whatever text survived — never silently dropped, never silently repeated. - Its home directory as of that turn. Files written outside
/root—/tmp,/workspace, anywhere else — are not in a turn checkpoint and come back as the setup snapshot left them. ItswaitForinputs (/inputs/*.txt,/inputs.json) are written again by alineod. - Its place in the tree. Parent and child edges are kept, so it can still be prompted, steered, and used as a spawn parent.
An agent whose spec doesn't set "checkpoint": true still comes back after a lost container — onto a fresh one,
with a blank session. It keeps its place in the tree but not its conversation.
Definitions
- Resumable — after any failure in the declared set (process crash, container loss, machine reboot), the system returns to the last committed point without a human reconstructing state by hand.
- Durable — once the system acknowledges a step as committed, no failure in the declared set can undo it. Today's declared set stops at the machine: losing its disk is outside it.
- Honest gap — work done after the last commit is either redone or explicitly reported as interrupted
(
agent.turn_interrupted, orlostwhen an agent can't be brought back at all) — never silently lost, never silently repeated.
Side effects are at-least-once, not exactly-once. A turn that ran git push or called an external API and was
then interrupted may repeat that effect on retry. Nothing in the stack de-duplicates a side effect a model decided
to take mid-turn — only alineod's own commands (spawn, pause, resume, stop, steer) are idempotent, via
idempotencyKey.
Known gaps
- Disk or host loss. Checkpoints, both ledgers and alineod's database all live on one machine. Restoring a swarm onto a new machine from a backup isn't built.
- One instance per database. alineod is a single-node control plane — see Scaling.
- Held credential bindings. A spec with
approval: "hold"credential bindings can't be restored onto a fresh container yet; if its container is gone, the agent endslost. - Non-root agents. A turn checkpoint captures
/root. A spec that runs Pi as a different user isn't covered.
What this unlocks today
Resumable agents, to the last completed turn — including through a machine reboot. Not "durable": a swarm does not yet survive the loss of the disk it runs from. This page will move that into the table above once a fault test proves it, not before.