Crash recovery

Rebuild state from the ledger

The agents and handles tables are discarded and rebuilt by replaying the ledger. Two kinds of agent need reconnecting: any agent in provisioning, spawning, running, or paused (still live), and any finished agent (done or failed) whose sandbox is still open — a turn ending does not close the sandbox, so it stays promptable and usable as a spawn parent. aborted and lost agents are skipped: stop/DELETE /runs/:id already closed their sandbox, and a lost agent is retried again on a later boot, not this one.

Reconnect to agents that have a sandbox

For each one, alineod calls Alineo.reattach(), which binds to the harness bridge already running in the container without restarting it. An in-flight turn is not interrupted.

If the agent was mid-turn at crash time, the process that was reading its stream is gone, so alineod polls the agent until the turn finishes and then records its result — anything waiting on that agent is still released.

If the bridge doesn't answer, alineod falls back to Alineo.resume(), which restarts the bridge (dropping any in-flight turn). If that fails too: a still-live agent ends as lost, since its outcome genuinely is unknown; a finished agent instead keeps its already-recorded outcome — it just can't be prompted or used as a spawn parent again this boot.

Parents whose own turn already finished are reconnected the same way, so a pending child can still fork from them.

Retry spawns that hadn't forked yet

Agents recorded before their sandbox existed — a child held by waitFor, or a root still being provisioned — are retried from what was persisted in their agent_spawned event: the spec, waitFor list, prompt, and budget. A gather agent that was waiting on workers goes back to waiting and starts normally once they settle.

A child whose parent could not be reconnected ends as lost.

Serve requests

The daemon starts listening once reconnection is done; spawn retries continue in the background.