Key Takeaway
AI-generated code often works on the happy path and fails at the edges, so a green demo is not proof it is correct. Treat every AI diff like an untrusted junior pull request: layer unit, integration, and edge-case tests, run types and lint, and review the code rather than trusting that it ran.
Generative tools have made writing code the easy part. An agent can produce a working feature in minutes, and that speed is exactly what makes the testing question urgent: the bottleneck has moved from typing code to trusting it. The demo runs, but does it hold?
The honest answer is that AI-generated code needs more scrutiny than hand-written code, not less, because it arrives with unearned confidence and no author who understands every line. This guide lays out a layered testing strategy for shipping it safely, and the mindset that makes the strategy stick.
Why AI code needs harder testing
Addy Osmani calls it the 70 percent problem: AI gets you most of the way to a feature astonishingly fast, then the last 30 percent, the edge cases, error handling, and integration, turns into a slog that the speed of the first part hides.
There is a knowledge paradox on top of it. The developers who benefit most from AI code are the ones experienced enough to catch what it gets wrong, while the least experienced are the least equipped to test it. That shapes how you should test:
- Do not confuse runs with correct. A passing demo exercises the one path the author tried, not the inputs a user will actually send.
- Assume unearned confidence. The code reads as if someone thought it through, but no human validated the assumptions baked into it.
- Spend your review budget where AI is weak, on the last 30 percent, rather than re-reading the boilerplate it handles well.
A layered testing approach
No single check catches everything, so stack them. Unit tests pin behavior, integration tests catch the seams, and static tooling catches the sloppiness that tests miss. The goal is many cheap, fast signals rather than one heavy suite.
Wire the layers into one command that the agent and your CI both run, so nobody can skip a step. A minimal gate looks like this:
# Green tests are necessary, not sufficient
npm test -- --coverage # behavior, plus what is left uncovered
npx tsc --noEmit # types the model may have loosened
npm run lint # dead code, unused vars, unsafe casts
npm run build # it compiles in a production buildReview the diff, not just the output. Ask the agent to explain any line whose purpose is not obvious, and if the explanation is vague, that line is where your next test goes. Unexplained code is untested code waiting to fail.
Where AI code breaks most
Direct your tests at the categories with the worst track record. According to Skyramp's analysis of AI-generated code, the defect rates cluster in a few predictable places.
- Error handling, roughly 3.4 times more defect-prone according to Skyramp, where the agent writes the happy path and stubs the failure path.
- Edge cases, roughly 4.1 times more defect-prone according to Skyramp, covering empty collections, boundaries, and unexpected types.
- Concurrency, roughly 4.7 times more defect-prone according to Skyramp, where race conditions survive because they never appeared in a quick manual test.
- Security-sensitive code, where an agent may reach for an insecure default that passes functional tests but fails a threat model.
Keep a human in the loop
Automated tests are necessary but they share a blind spot with the code: if the agent misunderstood the requirement, it can write both the wrong code and the wrong test, and the two will agree with each other perfectly.
That is why a human still has to check the tests against the actual intent. Read the assertions and ask whether they encode what the feature is supposed to do, not merely what the code currently does.
Watch for tautological tests. When the same agent writes both the code and its tests, they can encode the identical wrong assumption, producing a suite that passes while the behavior is wrong. A test that merely restates the implementation proves nothing.
A checklist for shipping AI code
Before an AI-written change merges, run it through the same gate every time. Consistency is what turns these habits into safety.
- Run the full test suite plus types, lint, and a production build, and treat any failure as blocking.
- Add explicit tests for the error paths and edge cases the agent skipped.
- Read the diff line by line, and rewrite anything you cannot explain to a teammate.
- For risky changes, have a fresh agent or a colleague review the diff without the original author's reasoning.
The speed of AI code generation is real, but it moves the work rather than removing it. The last 30 percent, the part that decides whether the feature survives contact with real users, is still yours. Test AI code like the untrusted contribution it is, and you keep the speed without inheriting the defects.