Scenario Outline column-value substitution for BDD

At a glance

Scenario Outlines run the same step body N times with one row of an Examples: table per run. The Gherkin spec is settled — the gap is in the porting machinery: the bdd-exec skill matches steps to L2 files by alias regex, but there's no documented contract for "the Examples row's column values reach the L2 runspec body." We have one validated upload Outline (discussion-capture.feature) whose L2 already references $FIXTURE_PATH, $FIXTURE_FILENAME, $FIXTURE_CONTENT_TYPE (vars no scenario today supplies), and three more Outlines (pii-boundaries, share-links, video-import) blocked on the same gap.

What we are NOT building: a new YAML schema, a new substitution grammar, a new $VAR scope, a new dispatch phase, or a "scenario outline" awareness in the runner. The smallest viable change is "the skill writes a few more keys into the existing params bag." Authors put the fixture path directly in the Examples block as another column — no second YAML file, no frontmatter declaration, no lookup machinery. (A lookup-table escape hatch was considered and explicitly deferred; see §11.1.)

Scenarios this unblocks (and which to implement alongside the framework PR)

Implementing this plan means — in the same PR or its immediate follow-up — flipping these specific scenarios from @pending to validated. They share the same authoring shape: an Examples block with explicit columns, no hidden lookup. Each row below is one scenario or Outline row that should be running end-to-end against test.sageox.ai before the cluster is considered shipped.

Count: ~30 scenarios flip from @pending to validated in PR 1's footprint. The session and discussion entity-type rows of the share-links Outline (rows 2–3) remain @pending because they're blocked on session-lifecycle and discussion-import BAs entirely — not on this substitution work. They'll flip when those clusters land.

Existing-machinery check (the suspicious read)

Before designing new machinery, I confirmed what's already in the runner. The plan is shaped by what I found.

1. Substitution semantics — column header as $<HEADER_UPPER>

Every column header in an Examples: block is injected verbatim as $<HEADER_UPPER> into the substitution scope for that row's plan. One column → one var. No transformation, no derivation, no second YAML file. If a verb needs three fixture-related vars, the Examples block has three columns — explicit and visible in the feature file.

The substitution rules, in one paragraph

For each Examples row, the skill computes the row's params bag:

1. For each (header, value) in the row, add HEADER_UPPER → value to the bag. format → FORMAT; expected_mime → EXPECTED_MIME; fixture_path → FIXTURE_PATH. Whitespace in the column value is preserved verbatim (one trim of leading/trailing space, no internal collapse).
2. Stamp the resulting bag into step.params for EVERY step in the Outline's plan emit. (Step-scoped params won't break — the merge code in runner.ts handles N steps × M keys.)

Headers are uppercased once at emit time and never re-substituted. The runner's existing $VAR grammar (uppercase, underscore-allowed) handles them transparently.

Edge cases (worth stating up front)

2. Where substitution happens — emit-time, via params

The skill substitutes at plan-emit time, into the existing step.params bag. The runner does not need to know Outlines exist.

4. Format-to-BA dispatch (the live-recording row)

Recording-journey has one Outline whose body is one phrasing — "she uploads a <format> file titled X" — but where one row ("live recording") needs to route to a different BA (record-live-discussion) than the others (upload-a-recording). Today's matcher alias-matches the step text once per Outline; it has no per-row override.

With no lookup table to carry an _ba override, the cleanest path is structural separation in the feature file: keep the upload Outline focused on file-upload formats, and lift live-recording out into a sister scenario (plain Scenario:, not Outline) right next to it. The user-facing spec gains one extra block and loses an awkward "one row that means something different from the others" footnote; the AI authoring path stops needing to remember a per-row dispatch escape hatch.

The structural split is paid for once, in the feature file, by the human author who already knows the two acts are different. It costs nothing at runtime and nothing in the skill.

5. Backward compatibility

The four currently-passing scenarios (onboarding/registration.feature × 2, context-injection/agent-priming.feature × 2) do not use Outlines. They use plain Scenario: blocks with quoted-string captures only. The substitution rules in §1 are additive: a scenario with no Examples block emits an empty row-derived params bag, which is a no-op merge in mergeStepVars.

Migration path for the ~24 blocked Outlines

Most blocked Outlines unblock with column-as-VAR alone:

• pii-boundaries (7 rows) — single page column, no fixtures.
• share-links recording row (1 of 3) — column-only.
• recording-journey audio + video (9 rows) — fat Examples block with explicit fixture_path, fixture_filename, fixture_content_type columns alongside format, media_kind, expected_mime.
• recording-journey live-recording (1 row) — split into a sister Scenario: per §4.
• video-import (5 rows) — fat Examples block with explicit fixture columns.

The session/discussion rows of share-links remain @pending because they're blocked on session-journey and discussion-import BAs entirely — that's an L2 gap, not a substitution gap.

6. Skill update scope (tests/bdd/skills/bdd-exec/SKILL.md)

Phase 6 (RunPlan emit) — section currently at lines 422–547

Today Phase 6 says (lines 442–445):

"The ONLY things you substitute at plan-emit time are scenario-table literals: alias-regex captures, Scenario Outline Examples row values, and persona/pronoun resolution into the persona field."

Update to:

1. The skill MUST iterate Examples rows for any Scenario Outline:, emitting one full RunPlan per row.
2. For each row, the skill builds a row params bag: add HEADER_UPPER → value for every column.
3. The bag stamps into step.params on EVERY step of the row's plan (Background steps included, since some Backgrounds reference column vars indirectly).
4. The skill MUST also substitute <header> tokens in stepText at emit time (cosmetic — the report renders the substituted text, which matches what humans expect from Outlines).

Add a new sub-heading after Phase 6's "Authoring tips" (around line 547): "Scenario Outline expansion" — ~20 lines of contract per the above.

Phase 7 (runner dispatch) — section at lines 549–599

No change. The runner sees one plan per row, indistinguishable from a plain Scenario. The only edit is to refresh the comment at line 446 that lists "scenario-table literals" — change "Scenario Outline Examples row values" from "literal substitution into the spec body" to "stamped into step.params for runner-side substitution."

Phase 2 (step resolution) — lines 158–225

Unchanged. The matched L2 for a step doesn't vary per row; the same alias-match flow as today applies. Outline rows differ only in their params bag.

Phase 4.5 (runspec extract+validate) — lines 262–284

Unchanged. Each row's plan validates independently.

7. Lint mode (T19) extensions

The dry-run lint already catches forward-refs, bare-$VAR-on-RHS, jq-parse failures, and reserved-key shadowing (lint.ts:83–250). The Outline substitution introduces three new failure modes; all become ERROR-severity rules (one stays WARN), all run during the existing single-pass walk (no new dry-run flag).

L1 is load-bearing — without it, an author who forgets a fixture column gets a confusing "substitution failed" mid-VM-spin instead of a one-line pre-run rejection.

8. Fixture story (the physical-world question)

The recording-journey Outline uploads MP3/WAV/M4A/OGG/FLAC/MP4/MOV/WebM/MKV files to S3 via presigned URL. Real bytes. Where do they come from?

Three options, scored

The plan

1. Create a pinned S3 bucket: sageox-bdd-fixtures in us-west-2. Public-read on the fixture objects (they're synthetic public-domain clips, not customer data); IAM-gated on writes (a single S3:PutObject role for the dev who regenerates fixtures).
2. Upload one short clip per format we test (MP3/WAV/M4A/OGG/FLAC/MP4/MOV/WebM/MKV) at canonical key paths: s3://sageox-bdd-fixtures/media/<sha256>/<name>.<ext>. The SHA in the key path doubles as integrity check — if the bytes ever change, the object lives at a different key. Each clip is 2–4s, mono / 480p where applicable, long enough for the transcript pipeline to produce something.
3. The orchestrator's bootstrap step fetches all needed fixtures via aws s3 cp (or signed GET) into the VM's /fixtures/media/ directory. Parallel + region-local, so the cold-cache cost is ~1–3s for the whole set.
4. The Examples block's fixture_path column is the in-VM path (e.g., /fixtures/media/sample.mp3) — written explicitly per row. The orchestrator's fixture-fetch logic is responsible for ensuring those paths exist before any verb runs; missing-file errors surface there, not at the verb's fs.read.
5. A one-time scripts/regenerate-fixtures.sh documents how the seed clips were generated (ffmpeg invocations against public-domain source URLs) and how to re-upload to the canonical SHA-keyed paths. The script is NOT a runtime dependency.
6. Local dev: the bootstrap step caches downloaded fixtures under tests/bdd/.fixture-cache/<sha256>/ (gitignored) so a second make test hits the cache, not the network. Cache-miss path is the same aws s3 cp.
7. CI: the workflow's AWS-credentials step already exists for the rest of the test bench (per the env's IAM role); the bootstrap just adds the fixture-pull command before the first /bdd-exec.
8. Auth: public-read on the bucket means the aws s3 cp can be plain curl against the S3 HTTPS URL — no AWS SDK / credentials needed in the VM. Saves a credentials-plumbing step inside the shuru microVMs.

9. Phasing — one PR, optional second

1. PR 1: Column-as-VAR substitution + LFS-tracked media fixtures

   Skill change: for each Outline row, stamp HEADER_UPPER → value into every step's step.params. Substitute <header> into stepText for report rendering. Phase 6 of SKILL.md updated; new "Scenario Outline expansion" sub-section added.

   Lint rules L1 (unresolved $VAR), L2 (orphan column), L3 (invalid header) land.

   Fixture work: tests/acceptance/fixtures/media/ with LFS-tracked clips; orchestrator VM image bind-mount; CI git lfs pull step. Feature files updated to fat Examples blocks with explicit fixture_path, fixture_filename, fixture_content_type columns where needed. Recording-journey live-recording split into a sister Scenario: per §4.

   Unblocks: pii-boundaries (7 rows), share-links recording row (1 of 3), recording-journey audio + video (9 rows), video-import (5 rows), recording-journey live-recording (1 row). Validation criterion: end-to-end run of pii-boundaries (no fixtures) and discussion-capture recording-journey Outline (full fixture path) both green; LFS pull works in CI.

   ~150 lines of skill changes + ~50MB of LFS fixtures + 1 orchestrator image change (bind mount) + feature-file edits.

2. PR 2 (optional): polish & cleanup

   If PR 1's lint-rule surface area or fixture-mount path needs revision after first contact, ship the polish here. Otherwise the work is done in PR 1.

   Held in reserve; not pre-planned.

PR 1 alone delivers all 23/24 unblockable rows of value. The session/discussion rows of share-links remain @pending pending unrelated L2 work.

10. Worked examples

Example A: recording-journey audio Outline (fat Examples with fixture columns)

Source feature (excerpted from tests/acceptance/features/discussions/discussion-capture.feature) — note the explicit fixture columns:

Scenario Outline: Uploaded <format> becomes a playable, transcribed discussion
  Given Tina is a member of "Acme Engineering"
  When she uploads a <format> file titled "Architecture Review"
  And processing completes
  Then "Architecture Review" should appear in the team's discussions list as Ready
  And it should have a <media_kind> element with a non-empty src and no error state
  And its stored MIME type should be <expected_mime>

  Examples: Audio formats
    | format | media_kind | expected_mime | fixture_path                   | fixture_filename | fixture_content_type |
    | MP3    | audio      | audio/mpeg    | /fixtures/media/sample.mp3     | sample.mp3       | audio/mpeg           |
    | WAV    | audio      | audio/wav     | /fixtures/media/sample.wav     | sample.wav       | audio/wav            |

{
  "feature": "features/discussions/discussion-capture",
  "scenario": "Uploaded MP3 becomes a playable, transcribed discussion",
  "persona": "tina",
  "runId":   "run_...",
  "steps": [
    /* Given Tina is a member ... */
    {
      "phase": "when",
      "stepText": "she uploads a MP3 file titled \"Architecture Review\"",
      "spec": {
        "operations": [{
          "verb": "web/recording/upload-composite/upload",
          "body": {
            "teamId":      "$TEAM_ID",
            "filePath":    "$FIXTURE_PATH",      // unresolved
            "filename":    "$FIXTURE_FILENAME",  // unresolved
            "contentType": "$FIXTURE_CONTENT_TYPE", // unresolved
            "title":       "Architecture Review"
          },
          "exports": { "RECORDING_ID": ".data.recordingId" }
        }]
      }
      // NO params block - dispatch will throw SubstitutionError
    }
  ]
}

{
  "feature": "features/discussions/discussion-capture",
  "scenario": "Uploaded MP3 becomes a playable, transcribed discussion",
  "persona": "tina",
  "runId":   "run_...",
  "steps": [
    /* Given Tina is a member ... - same params bag here too */
    {
      "phase": "when",
      "stepText": "she uploads a MP3 file titled \"Architecture Review\"",
      "spec": { "operations": [{
        "verb": "web/recording/upload-composite/upload",
        "body": {
          "teamId":      "$TEAM_ID",
          "filePath":    "$FIXTURE_PATH",
          "filename":    "$FIXTURE_FILENAME",
          "contentType": "$FIXTURE_CONTENT_TYPE",
          "title":       "Architecture Review"
        },
        "exports": { "RECORDING_ID": ".data.recordingId" }
      }] },
      "params": {
        "FORMAT":               "MP3",
        "MEDIA_KIND":           "audio",
        "EXPECTED_MIME":        "audio/mpeg",
        "FIXTURE_PATH":         "/fixtures/media/sample.mp3",
        "FIXTURE_FILENAME":     "sample.mp3",
        "FIXTURE_CONTENT_TYPE": "audio/mpeg"
      }
    }
    // Then steps carry the same params; assert
    // .data.stored_mime == "$EXPECTED_MIME" works
  ]
}

Example B: pii-boundaries Overview row (single column)

Scenario Outline: Anonymous viewer sees no PII on <page>
  When an unauthenticated viewer opens the <page> of "Acme Engineering"
  Then the rendered page should contain no email addresses

  Examples:
    | page        |
    | Overview    |
    | Sessions    |
    | Repos       |
    | Discussions |

L2 for the When step (business-actions/view-public-team-surface.md) takes one var:

```yaml runspec=primary
operations:
  - verb: web/anon/open-team-page
    body:
      teamSlug: $TEAM_SLUG
      surface:  $PAGE
    exports:
      RENDERED_HTML: .data.html
```

// Skill can pre-substitute <page> into the body literal,
// but only because it's a one-column case. The Then steps reference
// $RENDERED_HTML which IS exported, but $PAGE shows up nowhere in the
// L2 body today - so the only path is hard-coding "Overview" into the
// body during emit:
{
  ...
  "spec": { "operations": [{
    "verb": "web/anon/open-team-page",
    "body": {
      "teamSlug": "$TEAM_SLUG",
      "surface":  "Overview"  // baked in - breaks if L2 author
                              // ever wants to introspect $PAGE in
                              // an assert
    }
  }] }
}

{
  ...
  "spec": { "operations": [{
    "verb": "web/anon/open-team-page",
    "body": {
      "teamSlug": "$TEAM_SLUG",
      "surface":  "$PAGE"     // unchanged from L2 source
    }
  }] },
  "params": {
    "PAGE": "Overview"
  }
}
// Sessions, Repos, Discussions rows emit the same shape
// with PAGE: "Sessions" / "Repos" / "Discussions".

11. What this is NOT

• Not a multi-persona mechanism. Scenarios that need a second actor in scope still emit one persona per plan. That's a separate gap — bd-#27.
• Not a "data-driven test" generalization. We're only handling Gherkin's standard Examples: blocks. No JSON/CSV external data sources, no random-input generation.
• Not a fix for the session/discussion rows of share-links. Those rows remain @pending because the L2s for "Tina has a session" and "Tina has a discussion" don't exist. The substitution work routes through them once they land, but doesn't author them.
• Not a verb-side change. The upload-composite verb already accepts filePath / filename / contentType; the work is plumbing scenario data to those inputs, not changing the verb contract.
• Not a runner refactor. No new schema fields, no new dispatch phases, no new substitution grammar. The runner is unaware Outlines exist before this change and remains unaware after it.
• Not a UI-recorder-mode fix for live-recording. The sister-scenario split (§4) routes that case to record-live-discussion; whether that BA's primary runspec actually drives the web recorder through Stagehand vs. mocks the result is the BA's problem, not the substitution layer's.
• Not a lookup-table mechanism. We deliberately deferred the lookup-table escape hatch — see §11.1.
• Not in scope for migration to cucumber-js or a third-party Gherkin runtime. The skill remains the parser-and-emitter; the runner stays single-scenario. If we ever swap to cucumber-js, the column-as-VAR semantics described here will already match the standard Examples-row binding, so we'd lose nothing.

11.1 Deferred: lookup-table escape hatch

An earlier draft of this plan proposed a second mechanism on top of column-as-VAR: a per-BA YAML lookup table where one column value (e.g., format: MP3) would expand into multiple derived vars (fixture path, filename, MIME, etc.). It's not in this plan, but the sketch is preserved here so a future plan-author doesn't have to re-derive it.

When this would become necessary

• The same format → fixture_path / fixture_filename / fixture_content_type mapping starts appearing in 3+ feature files. (Currently: one.)
• The fixture path columns start materially hurting feature-file readability for non-engineer reviewers. (Currently: the fat Examples block is a Tufte-style win, not a loss.)
• Someone wants to refactor the fixture catalog (e.g., move /fixtures/media/ → /fixtures/audio/) and the change has to ripple through N feature files. (Currently: one feature file, one find/replace.)

What the trigger metric is

Same format → fixture_path mapping repeated in 3+ feature files. Below that threshold the duplication cost is lower than the indirection cost of a second YAML file plus the lint surface to validate it. Above it, the lookup pays for itself.

What it WOULD look like

---
doc-audience: human
status: validated
uses_fixtures: lookups/recording-formats.yaml
aliases:
  primary:
    - "{persona} uploads a {format} file titled \"{title}\""
---

keyColumn: format
entries:
  MP3:
    FIXTURE_PATH:         /fixtures/media/sample.mp3
    FIXTURE_FILENAME:     sample.mp3
    FIXTURE_CONTENT_TYPE: audio/mpeg
  WAV:
    FIXTURE_PATH:         /fixtures/media/sample.wav
    FIXTURE_FILENAME:     sample.wav
    FIXTURE_CONTENT_TYPE: audio/wav

None of this is in the current plan. The current plan puts those same six fixture columns directly into the Examples block, where they're visible to spec readers, version-controlled in one place, and add zero machinery to the skill. The simplification is intentional and reversible.