Retries and timeouts as syntax, not boilerplate

Every flaky command eventually grows the same crust of bash:

n=0
until [ $n -ge 3 ]; do
  ./push.sh && break
  n=$((n+1)); sleep $((2**n))
done
[ $n -ge 3 ] && { echo "gave up"; exit 1; }

It's wrong in ways you won't notice until 2 a.m. The backoff is 2,4,8 — no cap, so attempt 6 sleeps over a minute. There's no overall deadline, so a command that hangs makes the whole retry block hang with it. And it's copy-pasted into a dozen scripts, each with a slightly different bug. Retry logic is exactly the kind of thing you want to write once, correctly, and never hand-roll again.

In OrchStep it's not code — it's fields on the step.

Retry, backoff, and both timeouts

name: retry-demo
defaults:
  endpoint: https://api.internal/health

tasks:
  publish:
    steps:
      - name: push
        func: shell
        do: echo "pushing image"
        timeout: "10s"          # each attempt may take up to 10s
        total_timeout: "30s"    # whole step, retries included, capped at 30s
        retry:
          max_attempts: 4       # total attempts, not extra retries
          interval: "1s"        # delay before attempt 2
          backoff_rate: 2.0     # 1s -> 2s -> 4s
          max_delay: "5s"       # cap the growth

      - name: smoke
        func: shell
        do: echo "smoke test {{ vars.endpoint }} -> 200"
        timeout: "5s"
        retry:
          max_attempts: 3
          interval: "2s"
          when: '!result.success'   # only retry on failure

      - name: done
        func: shell
        do: echo "published"

Read the push step top to bottom and you have the whole policy: try up to 4 times, wait 1s then 2s then 4s between attempts, never let one attempt exceed 10s, and never let the entire step — retries and all — exceed 30s. That last line is the one the bash version can't express, and it's the one that saves you.

The field names are load-bearing

This is the part that bites people, so it's worth being blunt: the retry fields have exact names, and getting them wrong fails silently.

retry:
  max_attempts: 4        # NOT attempts, NOT max
  interval: "200ms"      # NOT delay
  backoff_rate: 2.0      # NOT backoff: exponential
  max_delay: "2s"        # cap the growth

Write attempts: or backoff: exponential and the engine doesn't error — it just doesn't retry. The step runs once and moves on, and you don't find out until the flaky call flakes in production. Use max_attempts, interval, backoff_rate, max_delay. (orchstep lint and --dry-run both render the real plan, so a quick dry-run shows you "retry: up to 4 attempts" and confirms it took.)

Two timeouts, because they answer different questions

A single timeout can't protect both a hang and a slow-but-progressing retry chain. OrchStep gives you both knobs:

• timeout bounds one attempt. "If this curl hasn't returned in 10s, it's wedged — kill it."
• total_timeout bounds the whole step, retries included. "I don't care how many attempts it takes, this step gets 30 seconds, full stop."

- name: bounded_operation
  func: shell
  do: ./long-running-script.sh
  timeout: 5s           # each attempt: 5s max
  total_timeout: 15s    # entire step including retries: 15s max
  retry:
    max_attempts: 10    # may not finish all 10 within total_timeout

Here max_attempts: 10 is a ceiling, but total_timeout is the real boss — the step stops at 15s whether or not all ten attempts ran. That's how you bound cost, not just count.

Retry only what's worth retrying

Retrying a 400 Bad Request just wastes time — the request is broken, not the network. when: gates retries on a condition so you only re-run the failures that might pass next time:

retry:
  max_attempts: 3
  interval: 2s
  when: "result.exit_code != 0 && !result.output.includes('invalid')"

And when retries finally fail, catch: recovers and finally: cleans up — the same step, no extra scaffolding:

- name: risky
  func: shell
  do: ./deploy.sh
  retry: { max_attempts: 3, interval: "2s", backoff_rate: 2.0 }
  catch:
    - name: rollback
      func: shell
      do: ./rollback.sh
  finally:
    - name: unlock
      func: shell
      do: rm -f deploy.lock

What you stopped maintaining

When you don't need any of this

A command that never fails and never hangs doesn't need a retry block — don't add ceremony for a step that's already reliable. Reach for these fields when a step talks to something you don't control: a registry, an API, a cluster mid-rollout. That's where "retry as syntax" turns a fragile script into one you can actually trust unattended.

Where to go next

• Error Handling — retry, when, catch, finally, on_error
• shell function — timeout and total_timeout on any step
• Previewing with Dry Run — confirm a retry block actually took effect

The demo's echoes succeed, so no attempt is wasted — the point is the shape: orchstep run publish.