BLOG/TEAM CI/CD
THEME
TEAM CI/CD

True parallel fan-out

Three health checks that don't depend on each other shouldn't run one after another. A parallel block launches steps concurrently, waits for all of them, and merges their outputs back.

Jun 25, 2026 OrchStep Team 6 minROLE: SRESCALE: Any

Three independent things — probe the API, probe the database, probe the cache — have no reason to wait in line. Run them sequentially and the wall-clock cost is the sum of the three. Run them at once and it's the slowest of the three. The shell answer is backgrounding with & and a wait, plus some temp files to smuggle the results back, plus the part where one silent failure gets lost.

OrchStep has a parallel: block. Steps inside launch concurrently, the block waits for all of them, and their outputs merge back into the context for whatever comes next.

Fan out, then aggregate

orchstep.yml
name: fanout
tasks:
  checks:
    steps:
      - name: probe_all
        parallel:
          - name: probe_api
            func: shell
            do: |
              echo "checking api"
              echo "API_MS=42"
            outputs:
              ms: '{{ result.output | regexFind "API_MS=(.+)" }}'
          - name: probe_db
            func: shell
            do: |
              echo "checking db"
              echo "DB_MS=88"
            outputs:
              ms: '{{ result.output | regexFind "DB_MS=(.+)" }}'
          - name: probe_cache
            func: shell
            do: |
              echo "checking cache"
              echo "CACHE_MS=7"
            outputs:
              ms: '{{ result.output | regexFind "CACHE_MS=(.+)" }}'
      - name: summarize
        func: transform
        do: |
          const all = [
            Number(steps.probe_api.ms),
            Number(steps.probe_db.ms),
            Number(steps.probe_cache.ms),
          ];
          return { slowest: Math.max.apply(null, all), total: utils.sum(all) };
      - name: report
        func: shell
        do: 'echo "slowest={{ steps.summarize.slowest }}ms total={{ steps.summarize.total }}ms"'

The three probes run together; the sequential summarize step that follows can read any of their outputs:

  Step: probe_all
│  Starting parallel execution of 3 steps
│  Parallel step 'probe_api' completed successfully
│  Parallel step 'probe_db' completed successfully
│  Parallel step 'probe_cache' completed successfully
│  Parallel block completed: 3/3 succeeded
  [ok] probe_all
  Step: summarize
  [ok] summarize
  Step: report
slowest=88ms total=137ms
  [ok] report
Result: success

That last line is the proof the outputs merged: summarize saw all three ms values even though they were produced concurrently in separate steps.

How it actually works

  1. Each step inside parallel: launches concurrently — one goroutine per step.
  2. Every parallel step gets an isolated copy of the variable context.
  3. The block waits for all children to finish.
  4. Their outputs merge into the main context.
  5. Sequential steps after the block can reference any parallel step's outputs — exactly what summarize does above.

On failure, the block fails — but all steps still run to completion first, and the failures are collected and reported together. You see every probe that broke in one pass, not just the first one to trip.

parallel vs a loop

They look adjacent and solve opposite problems:

Use a parallel: block when…Use a loop: when…
a fixed, named set of different stepsthe same step over a list of items
each does its own thing (build, probe, scan)each item runs the same logic
you want them all at onceyou usually want them in order
outputs read by name (steps.probe_db.ms)outputs collected as an array

Fan-out is "do these N distinct jobs simultaneously." A loop is "do this one job for each of these N inputs." When you genuinely have distinct, independent jobs, parallel: is the right shape.

What you gained

Backgrounding in shellparallel: block
cmd1 & cmd2 & waita named block of steps
temp files to pass results backoutputs merge automatically
one failure gets lostall failures collected and reported
wall time = sumwall time ≈ slowest child
manual $! bookkeepingengine waits for every child

The constraints that keep it honest

Parallel steps must be independent — they can't reference each other's outputs, because each gets a snapshot of variables from before the block, and mutations don't cross between siblings. Step names must be unique across the whole task, including inside parallel blocks (that's how steps.probe_db.ms resolves later). Nested parallel blocks work, but reach for them sparingly — concurrency you can't reason about is worse than the sequential version you can.

Where this is not the answer

If your steps form a chain — B needs A's output, C needs B's — that's a sequence, and forcing it into a parallel block just breaks it. Parallelism only pays when the work is genuinely independent and there's enough of it to matter; firing off three echos concurrently saves nothing real. Use it for the cases where each child does actual work — a network probe, a build, a scan — and the slowest one sets the clock.

Where to go next

Find a task with three steps that don't talk to each other and wrap them in a parallel: block. The run gets shorter and you lose nothing.

#PARALLEL#CONCURRENCY#FAN-OUT#PERFORMANCE
Try it in two minutes — one binary, no signup.
curl -fsSL https://orchstep.dev/install.sh | sh

RELATED — TEAM CI/CD