You are reading a two part series on AIFSD with SDD, TDD and harness engineering. This is part two, part 1 link is here:
Source code available at github: https://github.com/alagesann/aifsd-sdd-tdd-harness
The concurrency RED that found a real bug
The first version of the store looked reasonable. It was written the way you would write one against a remote store such as Redis, with an asynchronous read and an asynchronous write:
// src/rate-limit/store.ts, first version (excerpt)
async increment(key: string, now: number): Promise<Window> {
const current = await this.read(key);
const next =
!current || now >= current.windowStart + WINDOW_MS
? { count: 1, windowStart: now }
: { count: current.count + 1, windowStart: current.windowStart };
await this.write(key, next);
return next;
}The scenario test fires 150 requests at once and asserts the exact split:
it("[scenario: rate-limiting/Burst of concurrent requests]", async () => {
const { app } = setup();
const responses = await Promise.all(Array.from({ length: 150 }, () => get(app, "key-c")));
const allowed = responses.filter((r) => r.statusCode === 200).length;
const limited = responses.filter((r) => r.statusCode === 429).length;
expect({ allowed, limited }).toEqual({ allowed: 100, limited: 50 });
}); × [scenario: rate-limiting/Burst of concurrent requests]
AssertionError: expected { allowed: 150, limited: +0 } to deeply equal { allowed: 100, limited: 50 }Every request got through. Node.js runs your code on a single thread, but each await between the read and the write lets the other 149 requests read the same stale count. A thread-level race detector wouldnt flag this, because it isnt a data race. Its a logic race between asynchronous steps. Only a test that asserts the exact outcome under concurrency catches it. “Send requests in parallel and check nothing crashed” would have passed.
The fix is the one the design already asked for: read, reset and increment with nothing asynchronous in between.
// src/rate-limit/store.ts
import { WINDOW_MS } from "./decide.ts";
export interface Clock { now(): number }
export interface Window { count: number; windowStart: number }
export interface CounterStore { increment(key: string, now: number): Promise<Window> }
/** In-memory store. Any other CounterStore must keep increment() atomic too. */
export class MemoryCounterStore implements CounterStore {
private windows = new Map<string, Window>();
async increment(key: string, now: number): Promise<Window> {
// Read, reset and increment with no await in between: atomic on Node's single thread.
const current = this.windows.get(key);
const next =
!current || now >= current.windowStart + WINDOW_MS
? { count: 1, windowStart: now }
: { count: current.count + 1, windowStart: current.windowStart };
this.windows.set(key, next);
return next;
}
}With this store the burst test passes. The remaining three scenario tests follow the same pattern: key-b gets 200 after key-a has used its 100, and the reset boundaries are clock.advance(59_999) expecting 429 and clock.advance(60_000) expecting 200.
Step 5: Verify
Verification asks two different questions.
Following along? First create
scripts/spec-coverage.mjs(the script is in section 3), whichnpm run checkruns, andstryker.config.json(shown below). Install Stryker withnpm i -D @stryker-mutator/core @stryker-mutator/vitest-runner. Your test counts and scores will differ from the ones below, because your agent’s tests and files differ. What should match is full scenario coverage, and what the survivors tell you.
Does the code pass every check? The check script runs every computational sensor, so the agent, the hooks and CI all mean the same thing by done. The first line of output is OpenSpec’s validation, then Vitest, then the coverage script:
$ npm run check
Totals: 1 passed, 0 failed (1 items)
Tests 6 passed (6)
spec-coverage: 6/6 scenarios coveredspec-coverage (section 3) proves that every scenario has a test. It doesnt prove the tests are any good. Mutation testing does that: StrykerJS plants small bugs (<= becomes <, a condition becomes false) and checks that some test fails for each one. Point it at the rate-limit code:
{
"testRunner": "vitest",
"mutate": ["src/rate-limit/**/*.ts"],
"thresholds": { "high": 90, "low": 80, "break": 80 }
}
decide.ts scores 100%: the boundary scenarios kill the off-by-one mutants, including the one that would block request 100. The survivors are more interesting:
[Survived] ConditionalExpression src/rate-limit/index.ts:13
- if (typeof key !== "string") return;
+ if (false) return;
[Survived] ObjectLiteral src/rate-limit/index.ts:18
- return reply.code(429).header(…).send({ error: "rate_limited" });
+ return reply.code(429).header(…).send({});Neither points to a weak test. Both are behaviour no scenario describes: what happens without an API key, and the shape of the 429 response body. Both came in with the implementation, and no spec asked for either. Thats how mutation testing catches agents quietly expanding the scope.
Survivors arent always unspecified behaviour, though. If your spec does cover keyless requests and the 429 body, the hook file scores 100%. But if your agent’s store also evicts expired keys (sensible, since keys aren’t validated), expect survivors like these:
[Survived] ConditionalExpression src/rate-limit/memory-store.ts:17
- if (previous !== undefined && next.windowStart !== previous.windowStart) {
+ if (false) {
[Survived] ObjectLiteral src/rate-limit/clock.ts:5
- export const systemClock: Clock = { now: () => Date.now() };
+ export const systemClock: Clock = {};The first is a design decision with no test. On rollover, the store deletes and re-inserts a key so that its eviction scan stays in order, and no test checks that order. The second is wiring nobody exercises: every test injects a fake clock, so the production default never runs.
So a surviving mutant means one of three things, and each forces a decision:
Unspecified behaviour: specify it (add a scenario with
/opsx:update, then write the test), or delete the code.An untested design decision: write the test the design implies.
Unexercised wiring: cover it with one integration or smoke test, or accept it knowingly.
Does the code do what the spec intends? Thats an inferential check. Use /opsx:verify if your profile includes it. Otherwise use a reviewer subagent with its own context. Then route each gap:
The code is wrong: go back to
/opsx:apply.The spec is wrong or incomplete: run
/opsx:update, review the change, then apply again.
Step 6: Archive
$ npx openspec archive add-rate-limiting
Task status: ✓ Complete
Specs to update:
rate-limiting: create
Applying changes to openspec/specs/rate-limiting/spec.md:
+ 4 added
Totals: + 4, ~ 0, - 0, → 0
Specs updated successfully.
Change 'add-rate-limiting' archived as '2026-11-01-add-rate-limiting'.Archive merges the deltas itself. You dont need /opsx:sync first: in Claude Code, /opsx:archive offers to sync as part of archiving. Keep /opsx:sync for the rarer case where the main specs need a long-running change's deltas before its archived.
3. Making it repeatable: Harness engineering
Section 2 relied on someone remembering the process. The harness moves it into files. Cheap controls act first, so feedback arrives early:
Guides
config.yaml steers OpenSpec work, CLAUDE.md steers everything else, and an optional skill offers the TDD loop on demand. Keep each rule in one place only.
openspec/config.yaml, the file you set up before starting, is injected into every artifact and every apply:
schema: spec-driven
context: |
Orders API. TypeScript, Node.js 24, Fastify 5, Vitest 4.
Business rules live in pure functions under src/<capability>/; I/O stays in Fastify plugins and stores.
Time is always injected (a Clock); tests never sleep.
rules:
proposal:
- 'End with "## For the reviewer": at most 5 bullets covering assumptions, scope cuts and risky decisions'
specs:
- Every scenario MUST use concrete values (counts, HTTP status, headers)
- Every numeric threshold MUST have a scenario exactly at the boundary
- One scenario per behaviour, plus only the required boundary scenarios
design:
- Any concurrency-sensitive rule MUST say where it is made atomic
- At most 4 decisions; each at most 4 sentences plus 1-2 rejected alternatives
operations:
apply:
guidance:
- Work one task at a time using TDD - write the failing test, run it, and show the RED output before writing production code
- A valid RED is a failing assertion; a compile, import or setup error is not
- 'Name each test that covers a spec scenario "[scenario: <capability>/<exact scenario name>]"'
- Never edit or weaken an approved test to make it pass; stop and ask instead
- Tick a task in tasks.md only after the check that task names passes (for a test-writing task, the expected RED)
- Before finishing the change, run npm test and npm run typecheck; both must passcontext: the stack and architectural ground rules.rules: make scenarios testable, and budget the proposal, spec and design so they stay reviewable (section 4 shows the effect).operations.apply.guidance: the TDD loop, explained in step 4.
Quote any list item containing : , as above. Otherwise OpenSpec silently drops the whole guidance list. To confirm what the agent receives, run openspec instructions apply --change <name> --json and look for operationGuidance.
CLAUDE.md covers bug fixes and refactors. Keep it short:
# Orders API
- Behaviour changes go through OpenSpec (/opsx:propose). Bug fixes and refactors don't need a change.
- Every bug fix starts with a failing test that reproduces the bug.
- "Done" means `npm run check` passes. Show its output; don't summarise it.
- Never weaken an existing assertion to make a test pass. If a test seems wrong, stop and ask.TDD as skill is optional. Its worth adding if you want the same loop on demand outside apply:
Computational sensors
One definition of done. npm run check runs OpenSpec validation, the type check, the tests and spec coverage. The agent, the hooks and CI all run it, so “done” means the same everywhere.
Spec-to-test traceability. OpenSpec has no coverage command, so scripts/spec-coverage.mjs (40 lines, no dependencies, available at github code repo) checks that every scenario has a test tagged with its exact name. It also fails on tags for scenarios that no longer exist, which catches renames.
Hooks run the checks automatically. Three design choices make them work with TDD, not against it:
PostToolUse runs
tsc, not tests. A freshly written test should fail; blocking on it would push the agent to write code first.Stop skips turns that didn’t change code, so explore and propose aren’t blocked.
Stop has a loop guard. Exit code 2 forces Claude to keep working; after three attempts the guard hands control back to you.
Configure them in .claude/settings.json:
Hooks are configured in .claude/settings.json:
{
"hooks": {
"PostToolUse": [
{
"matcher": "Edit|Write",
"hooks": [{ "type": "command", "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/post-edit.sh" }]
}
],
"Stop": [
{
"hooks": [{ "type": "command", "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/stop-gate.sh" }]
}
]
}
}#!/usr/bin/env bash
# .claude/hooks/post-edit.sh
# PostToolUse: type-check after each TypeScript edit. Tests are NOT run here:
# during TDD a failing test is the expected state, so this hook must not fight RED.
file=$(jq -r '.tool_input.file_path // empty')
case "$file" in *.ts) ;; *) exit 0 ;; esac
npx tsc --noEmit 1>&2 || exit 2 # exit 2 = show the errors to Claude#!/usr/bin/env bash
# .claude/hooks/stop-gate.sh
# Stop: don't let Claude end a turn that changed code while the checks are red.
input=$(cat)
counter="${TMPDIR:-/tmp}/stop-gate-$(jq -r '.session_id' <<<"$input")"
# Planning turns (explore, propose) don't touch src/ or test/: never gate them.
if git diff --quiet HEAD -- src test && [ -z "$(git ls-files --others --exclude-standard -- src test)" ]; then
exit 0
fi
# Loop guard: at most 3 forced continuations, then hand back to the human.
[ "$(jq -r '.stop_hook_active' <<<"$input")" = "true" ] || rm -f "$counter"
attempt=$(( $(cat "$counter" 2>/dev/null || echo 0) + 1 )); echo "$attempt" > "$counter"
if [ "$attempt" -gt 3 ]; then
echo "stop-gate: checks still red after 3 attempts; stopping for human review" >&2
exit 0
fi
npm run --silent check 1>&2 || exit 2 # exit 2 = block stopping, feed failures to ClaudeThe scripts need jq and chmod +x, and the repo needs at least one commit.
To see it work, change count <= LIMIT to count < LIMIT and let a turn end. Claude receives:
× [scenario: rate-limiting/Last request within the limit is allowed]
× [scenario: rate-limiting/Burst of concurrent requests]
AssertionError: expected 429 to be 200 // Object.is equality
AssertionError: expected { allowed: 99, limited: 51 } to deeply equal { allowed: 100, limited: 50 }
Tests 2 failed | 4 passed (6)Because tests are named after scenarios, the feedback says which behaviour in the spec broke.
Commit after each step, because the hook compares against HEAD.
CI reruns the checks on a clean machine, plus mutation testing, which is too slow for every turn. "break": 80 fails the job below an 80% score
Inferential sensors: reviewers with their own context
Computational sensors cant tell whether the spec was understood. A subagent can help, because it starts with a clean context. It hasnt absorbed the implementer’s reasoning, so its less likely to repeat the same misreading. Thats the defence against correlated mistakes:
---
name: spec-reviewer
description: Reviews an implementation strictly against an OpenSpec change. Use after /opsx:apply and before /opsx:archive.
tools: Read, Grep, Glob, Bash
---
You are a skeptical, read-only reviewer. Never edit files. The OpenSpec change is your only source of truth.
1. Run `npx openspec show <change> --json --deltas-only` and list every requirement and scenario.
2. Run `git diff main...HEAD` and read the changed files.
3. For EACH scenario report: IMPLEMENTED (file:line) | MISSING | CONTRADICTED.
4. For each scenario's test, say whether its assertions would fail if the behaviour were wrong.
5. List any behaviour in the diff that no requirement describes.
Output a table, then a verdict: PASS or FAIL. No style suggestions.
Treat its verdict as a sensor, not an approval. Its another model’s judgement, so it can be wrong. Its job is to point the human reviewer at the right lines.
4. Conclusion
Each practice closes a different gap. The spec makes intent explicit and reviewable. Test-first development, run inside /opsx:apply, turns that intent into executable checks one task at a time. The harness turns the process from something you ask for into something that happens anyway: config.yaml and CLAUDE.md steer, while hooks, coverage, mutation tests, reviewer agents and CI check.
In this example, the harness caught problems that a “happy path” test would have missed: a race that let 150 requests through a limit of 100, and two pieces of behaviour that no spec asked for.
To start, pick one feature and add three things: boundary scenarios in the spec, TDD guidance in config.yaml, and the Stop hook. Add mutation testing and a reviewer subagent once those feel routine.