Buildkite "Agent lost" / agent disconnected in CI

By Daniel Zoghalchali·Latchkey

Buildkite expects a periodic heartbeat from each running agent. When heartbeats stop, it declares the agent lost and fails the job it was running, even if the work was still in progress.

What this error means

A running step ends abruptly with "Agent lost" or "The agent ... disconnected". The host may have crashed, been terminated, or run out of memory.

The agent "ip-10-0-1-42-1" was lost while running this job.
Agent lost (no heartbeat received for 60s)

Common causes

The agent host was terminated or crashed

A spot or preemptible instance was reclaimed, or the host OOM-killed the agent process, so heartbeats stopped and Buildkite lost the agent.

The job exhausted host memory or CPU

A heavy step consumed all memory, freezing the agent long enough that it missed heartbeats and was declared lost.

How to fix it

Size the host and retry lost jobs

1. Give the agent host enough memory/CPU for the heaviest step so the process is not starved.
2. Add an automatic retry for the lost-agent condition so a reclaimed host does not fail the build.
3. Prefer on-demand capacity for jobs that cannot tolerate reclamation.

steps:
  - command: "make build"
    retry:
      automatic:
        - exit_status: -1
          limit: 2

Handle spot reclamation gracefully

On spot capacity, drain the agent on the termination notice so in-flight jobs requeue instead of being lost.

How to prevent it

• Right-size agent hosts for peak step memory to avoid OOM freezes.
• Auto-retry the lost-agent exit status for reclaimable capacity.
• Drain agents on spot/preemption notices.

Frequently asked questions

Related guides

References

• GitHub Actions documentation