Learning AI-DLC building a game, part 4: what the reviews caught

This is part 4 of a series on learning AI-DLC by building a game, World Rift Conflict, in Godot with an AWS backend. Part 3 showed what got built: one command that runs the sign-in checks on four platforms. This part shows the other half of that work: what AI-DLC's reviews and checks caught before anything ran.

Parts so far:

• Part 1: setup
• Part 2: a proof of concept
• Part 3: automated E2E tests
• Part 4: what the reviews caught (this post)
• Part 5: inside a stage

TL;DR

1. Three stages had a separate reviewer agent. Two of them said NOT-READY with three major findings each; the code review said READY.
2. The most useful finding was about me, not the code: the requirements had quietly relaxed a rule I had agreed to less than an hour earlier.
3. The design review found a production guard that checked the configuration but not the build that would actually run.
4. Inception reviews get one pass. The fixes after that pass are not re-checked, so the approval step is where I had to read them.
5. Every lesson went into a project rule that later stages read. Ten rules came out of this piece of work.

How a review works here

A stage that has a reviewer writes its documents first. Then AI-DLC records a review request and hands the documents to a separate agent with a different persona: a product lead for requirements, an architecture reviewer for design and code. The reviewer reads only the files it is given, never the builder's diary, and writes its findings into a table with a severity for each.

Two details matter:

• The reviewer never sees what the author was thinking. It has to judge the documents as written.
• The class decides what happens next. Inception reviews are advisory: one pass, and the findings go to my approval step. Code Generation's review is adversarial: up to two passes, with fixes in between.

Requirements: a rule I had agreed to, quietly relaxed

The product-lead review came back NOT-READY, with 3 major and 4 minor findings. The first major one:

FR19 says deleting test accounts in the test backend "needs no extra go". The team's affirmed Deployment practice says every AWS action, and it names "deleting test users", waits for the person's go. The person was never asked about this.

It was right. Less than an hour earlier, at Practices Discovery, I had agreed that every AWS action, including deleting test users, waits for my go. The requirements draft wanted runs that need nothing from me, so it simply decided that deleting accounts did not count. Both goals were reasonable; picking between them was my call, not the draft's.

So the agent asked me, with the trade-off spelled out: a standing go for test-domain accounts in the test pool only, or a go on every run. I chose the standing go. It was written into the rules as an amendment, and every other AWS action still waits for me.

The other two major findings were the kind I would not have spotted myself:

• The production check had no pass criterion. The requirement said "the sign-in screen shows", but the web game draws on one canvas, so nothing on the page says what is on the screen. The requirement was narrowed to what can be observed: the page answers, the engine starts, and the console shows no error.
• The cleanup corners were missing. A killed run cannot clean up after itself. The fixes: the next run cleans up first; a test account is told apart from a real one by its email domain; stored code emails expire after one day; and only one run at a time is allowed per machine.

Design: a guard that checked the wrong thing

The architecture review of the design also came back NOT-READY with 3 major findings. The one I keep thinking about:

A stale or copied export pointing at production would pass the guard

The design had a production guard: before a run, compare the test user pool ID with production's, and refuse if they match. But the guard read the IDs from configuration files, while the game build that actually runs has its own copy of the configuration baked in. A test build left over from an earlier day, or copied by mistake, could point at production and the guard would never know.

The fix: every test build records which pool it was built for, and the driver compares that with the approved test pool before it launches the game. It refuses on any difference. That check now has its own regression test.

The other two:

• No cost estimate. The requirements said the design would show the test environment stays under about $5 a month, and it did not. It now does: under $0.15 a month at 100 runs.
• Jobs without an owner. Nothing owned deploying the test backend or the mailbox, the steps that need my go, or the email cleanup. A 16th component, TestEnvironment, now owns every AWS action that needs a go. That also became a rule for future designs.

Code: READY, with five notes

The code review ran the local test suites itself (29 Godot tests, 89 TypeScript tests, 8 template tests, all passing). It said READY, with five minor notes. Two of them changed what I did next in Build and Test:

• The new Amplify step was untested where it runs. The Godot tests had only run on my machine, not inside the custom build image, so the first push would have been their first run there. A failure would block the production deploy. The step ran in the image from a clean copy before any push: 29 of 29.
• The web hook had been tested only against fakes. The full runs in Chromium, Firefox and WebKit then used it for real.

The one-pass limit

After fixing the requirements, I tried to have them reviewed again. AI-DLC refused:

Cannot request review pass 2 for "requirements-analysis" because this stage allows 1 review pass. Do not ask the reviewer again; include the findings in the approval summary for the human.

That is deliberate. Inception documents are judgment calls, and a second automated pass mostly delays the person who has to decide. But it means the fixes are mine to check: each approval step listed the findings, the fixes, and the fact that the fixes were not re-reviewed.

The small checks

Not every catch came from a reviewer:

• Traceability. The design check flagged FR2.1 and FR2.2 as missing from the design. They were the proof of concept's IDs, quoted in this work's requirements, so I mapped them to the scenarios that cover them.
• Required sections. Two Build and Test documents had too few headings to read as structured documents. I added the sections before approval.
• Guards on the agent. One hook blocked shell commands while no approved plan existed: "Code generation cannot run mutation-capable shell commands ... code-generation-plan.md is missing or empty." Another made the agent record each review's verdict before doing anything else. Both stopped me doing things in the wrong order.
• A target that could not pass. One requirement wanted "the first month's bill" to confirm the cost. That bill does not exist yet, so AI-DLC counted it as unverified and made me accept it explicitly. That one was my wording's fault.

In numbers: 57 sensor runs, 54 passed.

Where the lessons go

At the end of each stage, AI-DLC shows the notes from the stage diary and asks which ones to keep. A kept note becomes a line in the project's rules, which every later stage reads before it starts. Ten came out of this work. The ones that changed later stages:

• When a requirement or design would change or relax an affirmed practice, ask the person first.
• In a design, every AWS action that needs a go is owned by one named component.
• Before asking for a deploy go, check read-only what the account allows. That check found my DNS zone in part 3 and removed a manual step.
• Test any change to amplify.yml inside the build image before a push.
• Write pass criteria that can be checked within the work.
• Fill in a stage's standard questions from what is already decided, and ask only what is still open.

What I would tell myself

• Read the review findings as questions to you, not as defects. The best one was about a decision only I could make.
• The reviewer's lack of context is the point. It judges what the documents say, not what everyone meant.
• When a stage allows one review pass, read the fixes yourself at the approval step. Nobody else will.
• Keep the rules. The second half of this work went faster because the first half wrote down what it learned.

What is next

The tests run on their own and the rules that shaped them are written down. Part 5 looks closely at one stage, step by step. After that, the real game starts: an mvp piece of work and its first stages, from a game idea to an approved scope.