Kubernetes Job "DeadlineExceeded" from activeDeadlineSeconds in CI
activeDeadlineSeconds caps the total wall-clock time a Job may run. Once that budget elapses the Job is failed with DeadlineExceeded and its pods are terminated - even mid-run and even if backoffLimit retries remain.
What this error means
A Job that previously finished now ends as Failed with reason DeadlineExceeded, and its pods are killed partway through. It is deterministic for a run that legitimately takes longer than the deadline.
Status: Failed
Reason: DeadlineExceeded
Message: Job was active longer than specified deadline
# pods: Status: Failed, Reason: DeadlineExceededCommon causes
The deadline is shorter than the real run time
activeDeadlineSeconds counts from when the Job becomes active and covers all retries combined. A value tuned for the fast path kills a legitimately slower run.
Confusing the Job deadline with the pod deadline
A spec.activeDeadlineSeconds on the pod template bounds a single pod attempt, while the same field on the Job spec bounds the whole Job. Setting the Job-level one too low caps the entire workload.
How to fix it
Raise or remove the deadline to match real runtime
Size the deadline to the worst-case wall-clock time across all retries, or drop it if the Job has no hard time bound.
spec:
activeDeadlineSeconds: 3600 # was 600; cover the slow path + retries
backoffLimit: 4Separate per-attempt and whole-Job limits deliberately
- Set
spec.template.spec.activeDeadlineSecondsto bound one pod attempt. - Set
spec.activeDeadlineSeconds(Job level) to bound the entire Job across retries. - Make the Job-level budget ≥ (per-attempt budget × expected attempts).
How to prevent it
- Measure real worst-case runtime before setting
activeDeadlineSeconds. - Account for all
backoffLimitretries inside the Job-level deadline. - Distinguish the pod-template deadline (one attempt) from the Job deadline (whole Job).