Parse JSON and YAML without jq golf

You know the line. It started as curl ... | jq '.name' and ended as:

curl -s "$URL" | jq -r '.instances | map(select(.cpu > 70)) | length'

Then someone needed the names too, so it grew a .[] | .name, then a @csv, then a // empty to swallow nulls, and now it's a 90-character one-liner you rewrite from scratch every time because reading it is slower than retyping it. That's jq golf: the answer is in there, but the program is write-only.

The problem isn't jq. It's that shell only speaks strings, so every structured-data question becomes a parsing puzzle. OrchStep gives you two ways out — one for JSON, one for YAML — and both end with real fields you can pass to the next step.

JSON: parse it with an actual expression

func: transform runs JavaScript in a sandboxed VM. No process spawn, no quoting hell — just utils.parseJSON and the language you already know. Filtering hot instances is a .filter(), not a map(select(...)) incantation:

name: parse-demo
defaults:
  cpu_limit: "70"

tasks:
  # JSON: parse with func: transform (a real sandboxed JS VM).
  json_report:
    steps:
      - name: fetch
        func: shell
        do: |
          echo '{"service":"checkout","instances":[{"name":"web1","cpu":42},{"name":"web2","cpu":91},{"name":"web3","cpu":12}]}'
      - name: summarize
        func: transform
        do: |
          const data = utils.parseJSON(steps.fetch.output);
          const limit = parseInt(vars.cpu_limit, 10);
          const hot = data.instances.filter(function (i) { return i.cpu > limit; });
          return {
            service: data.service,
            total: data.instances.length,
            hot_count: hot.length,
            hot_names: hot.map(function (i) { return i.name; }).join(", ")
          };
      - name: report
        func: shell
        do: 'echo "{{ steps.summarize.service }}: {{ steps.summarize.hot_count }}/{{ steps.summarize.total }} over {{ vars.cpu_limit }}% -> {{ steps.summarize.hot_names }}"'

  # YAML: any step whose stdout parses as YAML exposes result.data_object.
  yaml_report:
    steps:
      - name: emit
        func: shell
        do: |
          printf 'region: apac\norders: 124\ncurrency: USD\n'
        outputs:
          region: "{{ result.data_object.region }}"
          orders: "{{ result.data_object.orders }}"
      - name: report
        func: shell
        do: echo "{{ steps.emit.region }} had {{ steps.emit.orders }} orders"

Run it:

orchstep run json_report
# checkout: 1/3 over 70% -> web2

The object you return becomes the step's output. summarize.service, summarize.hot_count, summarize.hot_names — each one is a named field the next step reads as steps.summarize.<field>, not a substring you re-extract. Anything jq does, transform does in plain JS: .filter, .map, .reduce, real variables, and utils.sum / utils.unique / utils.avg for the common reductions.

YAML: it's already an object

jq can't read YAML at all — that's where people reach for yq, a second tool with a second dialect. OrchStep skips it. When a step's stdout parses as JSON or YAML, the engine hands you result.data_object: the parsed structure, ready for field access in an outputs: block.

orchstep run yaml_report
# apac had 124 orders

No yq '.region', no awk -F:. result.data_object.region is the value. The same trick works on real sources — kubectl get ... -o yaml, a docker compose config, a helm values file — pipe it through a shell step and read fields off data_object.

Why this beats the pipe

The deeper win is structured data flows between steps. In a pipeline you have one stream and one shot; the moment you need a value twice, you parse twice. Here, summarize parses once and every later step reads typed fields. The boundary where text becomes an object is explicit and happens exactly once.

When jq is still right

If you're at a terminal poking at one response, curl | jq . is faster than writing a workflow — use it. transform earns its keep when the parse is a step in something larger: a value other steps depend on, a threshold that changes per environment, or a filter you'll read again in six months. At that point "a real expression with named outputs" beats "a clever filter nobody can edit."

Where to go next

• transform function — the JS VM, utils.*, sandbox rules
• Variables & Outputs — result.data_object, toObj, and passing objects between steps
• shell function — where the raw bytes come from

This whole demo is echo and printf, so it runs anywhere: orchstep run json_report.