BLOG/AI AGENTS
THEME
AI AGENTS

The replay layer for AI coding agents

An agent's reasoning is expensive and non-deterministic. Its result doesn't have to be. OrchStep is the layer that turns a solved agent session into a deterministic recipe you can review in a PR and run in CI.

Mar 27, 2026 OrchStep Team 7 minROLE: AnySCALE: Any

Here is the uncomfortable thing about coding agents: every run is a fresh roll of the dice. Ask the same agent to "deploy the service" twice and you may get two different sequences of commands. Usually both work. Occasionally one doesn't. You are paying — in tokens and in latency — to re-derive the same answer over and over, and you are trusting a probabilistic process to produce the same result each time.

That is fine for figuring out what to do. It is a terrible property for doing it again.

OrchStep is the replay layer that sits underneath your agent and fixes this. The idea is simple: an agent's reasoning is expensive and non-deterministic, but its result is a fixed sequence of steps. Pay for the reasoning once. Capture the result as a deterministic recipe. Replay the recipe — by itself, with no model in the loop — every time after that.

Reasoning once, result forever

The split looks like this:

The agent's reasoningThe OrchStep recipe
Costtokens + latency, every runa binary, zero tokens
Determinismprobabilisticthe same steps every time
Reviewablea chat transcripta YAML file in a PR diff
Runnable in CInot reallyyes, headless
Failure modere-reason, hopefail loudly on the exact step

The left column is where agents are brilliant — exploring, adapting, recovering from surprises. The right column is where they are wasteful and risky. The replay layer's whole job is to move work from left to right the moment it's solved.

The result is a recipe you can read

When an agent solves a task, OrchStep captures it as a workflow. Not a log — a structured, ordered, reviewable artifact:

AGENT TRANSCRIPT (BEFORE)
> deploy 2.1.0 to staging
[agent] I'll build first...
[agent] running: docker build -t app:2.1.0 .
[agent] now I'll run the tests
[agent] running: go test ./...
[agent] tests pass, deploying
[agent] running: kubectl set image ...
[agent] the registry flaked, retrying...
[agent] ok it's up
ORCHSTEP.YML (AFTER)
name: release
defaults:
  version: "2.1.0"
  target: staging
tasks:
  ship:
    steps:
      - name: build
        func: shell
        do: echo "building {{ vars.version }}"
      - name: test
        func: shell
        do: echo "test suite green"
      - name: deploy
        func: shell
        do: echo "deploying {{ vars.version }} to {{ vars.target }}"
        retry:
          max_attempts: 3
          interval: "1s"
          backoff_rate: 2.0
        catch:
          - name: rollback
            func: shell
            do: echo "rolling back {{ vars.target }}"

The transcript on the left is something you skim once and forget. The YAML on the right is something you put in front of a reviewer. The retry the agent did ad hoc ("the registry flaked, retrying") is now retry: syntax. The recovery it improvised is a catch: block. The version is a variable, so the next release is a one-line override.

PR-reviewable, CI-runnable

Two properties fall out of this, and they are the whole point.

It's reviewable. A captured workflow is a flat YAML file. It shows up in a diff. A teammate can read it, comment on a step, and approve it — the same way they review any other code. You are no longer asking people to trust "the agent did something reasonable." You are showing them exactly what runs.

It's runnable in CI. Because replay needs no model, it runs anywhere a binary runs:

orchstep run ship --var version=2.2.0 --var target=production --output json

No API key, no agent session, no network call to a model provider. The --output json output is structured, so a pipeline can parse step status and gate on it. The thing your agent figured out interactively now runs unattended on every push.

What OrchStep is not

It is not an agent, and it is not trying to replace one. The agent still does the hard part — understanding a fuzzy request, exploring an unfamiliar codebase, recovering when reality doesn't match the plan. OrchStep just keeps the output of that work so you never have to pay for the same answer twice. If your task is a true one-off that will never repeat, there is nothing to replay and nothing to capture. The replay layer earns its keep on the work you do again and again.

Where to go next

Tired of paying full price for the same deploy every week? Capture it once, replay it for free.

#AGENTS#REPLAY#DETERMINISM#CI
Try it in two minutes — one binary, no signup.
curl -fsSL https://orchstep.dev/install.sh | sh

RELATED — AI AGENTS