OrchStep for QA engineers

QA work is full of sequences you run from memory. Bring up the environment, hit the health endpoint, log in, run the suite, notice that two specs flaked again, re-run just those, then write the same "here's how to reproduce it" steps into a ticket for the third time this week.

None of that is hard. It's just undocumented, slightly different every time, and living in your head. When you're out, it leaves with you.

This post turns three of those sequences — a smoke gate, a sharded run with flaky quarantine, and a bug-repro runbook — into one OrchStep workflow. Same commands you already run; OrchStep just gives them a name, an order, and a dry-run.

The pain, concretely

# smoke, from memory, slightly different each run
curl -fsS https://staging/healthz || exit 1
# log in... what was the test account again?
# run the suite, watch it, spot the flakes
go test ./... || go test ./checkout/...   # re-run the flaky package
# then paste repro steps into the ticket by hand

orchstep run smoke                       # health + login
orchstep run test --var shards=4         # sharded suite + quarantine
orchstep run repro --var case=PROJ-481   # deterministic reproduction

The workflow, file by file

Three tasks. smoke is a fast pre-merge gate. test shards the suite and isolates known-flaky specs instead of failing the whole run. repro is the runbook you used to retype into every bug ticket.

name: qa-gates
# QA quality gate: smoke -> sharded run -> quarantine flaky -> report.
defaults:
  suite: checkout
  shards: 4

tasks:
  # `orchstep run smoke`
  smoke:
    steps:
      - name: health
        func: shell
        do: echo "GET /healthz -> 200 ok"
      - name: login
        func: shell
        do: echo "smoke - login flow passed"

  # `orchstep run test --var shards=4`
  test:
    steps:
      - name: gate_smoke
        task: smoke
      - name: run_shards
        loop:
          count: 4
        func: shell
        do: echo "running shard {{ loop.index }} of {{ vars.shards }}"
      - name: quarantine
        func: shell
        do: echo "moving known-flaky specs to the quarantine suite"
        catch:
          - name: note
            func: shell
            do: echo "quarantine step failed, continuing"
      - name: report
        func: shell
        do: echo "suite {{ vars.suite }} green across {{ vars.shards }} shards"

  # `orchstep run repro --var case=PROJ-481`
  repro:
    steps:
      - name: seed
        func: shell
        do: echo "seeding fixture for case {{ vars.case }}"
      - name: replay
        func: shell
        do: echo "replaying reported steps for {{ vars.case }}"
      - name: capture
        func: shell
        do: echo "captured logs + screenshot for {{ vars.case }}"

Three things worth pointing at:

• test calls smoke as a step (task: smoke), so the gate can't be skipped and you never duplicate the health check.
• The shard count is a loop, not a copy-pasted block. --var shards=8 on a beefier machine needs no edit.
• quarantine has a catch: — if isolating the flaky specs itself errors, the run notes it and continues instead of turning a known-flaky into a red build.

Run it, then preview it

orchstep run test --var shards=4
orchstep run repro --var case=PROJ-481

Before you trust the repro you attach to a ticket, see the resolved plan without executing anything:

orchstep run repro --var case=PROJ-481 --dry-run

The dry-run substitutes the case ID and prints the exact steps. That's the runbook — copy it straight into the ticket, and anyone can replay it.

What you gained

This doesn't replace your test runner. go test, pytest, Playwright — they still do the work. OrchStep is the layer that remembers the order and the flags so you don't have to. If a three-line Makefile already covers you, keep it; reach for this when the sequence has grown branches, retries, and tribal knowledge.

Where to go next

• Quick Start — your first workflow in two minutes
• Loops — count, items, and bounded until polling
• Error Handling — retry, catch, finally, timeouts
• Previewing with Dry Run — see the plan before it runs

Install once — brew install orchstep/tap/orchstep — and turn your next "let me just run the smoke tests" into a task anyone can run.