
In short
- AI tools have made writing code cheap, so the amount of change reaching review, testing and production keeps growing.
- The slow, expensive steps are now reviewing that change and knowing it is safe, which is exactly where testers work.
- Teams that keep up test closer to each change, review AI-written tests as carefully as AI-written code, and measure stability, not just speed.
On this page
- 01How much of the code is AI-written now
- 02What actually changes between commit and production
- 03The problem with tests written by the same tool
- 04Review becomes the main quality gate
- 05Test closer to each change
- 06New things to test: the AI features themselves
- 07What to measure
- 08Frequently asked questions
A developer asks an assistant to add a discount field to the checkout API. Ninety seconds later there is a pull request: the field, the validation, a database migration, eleven unit tests, all green. It looks finished. The reviewer has four other pull requests like it waiting, so they skim, approve, and move on.
None of that is careless. It is just the new shape of the work. Writing code used to be the slow part of shipping software. It isn’t any more, and the slow part has moved to the people and processes that decide whether a change is safe. That includes you.
01How much of the code is AI-written now
The numbers moved fast. In October 2024, Google said more than a quarter of its new code was generated by AI. By April 2026 it said 75% of all new code at Google is AI-generated and approved by engineers. That is eighteen months, and Google is not unusual. Your vendors, your contractors and your own team are using the same tools.
Google’s DORA research, which surveyed thousands of technology professionals in 2025, found AI use at work is now the norm and that teams are shipping more. It also found delivery stability is not improving at the same rate, and named what helps: strong automated testing, small batches and fast feedback. In other words, the bottleneck AI created lands on testing.
02What actually changes between commit and production
| Stage | What changes with AI | What it means for quality |
|---|---|---|
| Writing code | Much faster, often by agents working from a ticket | More change per week, more of it in areas the author doesn’t know well |
| Code review | Reviewers see more and bigger pull requests | Skimming goes up; subtle logic changes slip through |
| Tests | Generated alongside the code, by the same tool | Tests can share the code’s mistakes and still pass |
| Duplication | Copying a pattern is easier than finding the shared one | The same bug can live in several places; GitClear’s 2025 research reported a sharp rise in duplicated code |
| Release | Smaller, more frequent deploys become possible | Good if checks run per change; risky if testing still happens per quarter |
03The problem with tests written by the same tool
When the assistant writes both the code and the tests, the tests tend to confirm what the code does, not what it should do. If the code accepts an order with no email, the generated test may happily assert that it does. Everything is green, and the suite is now protecting the bug.
- Read the assertions, not the test names. A test called
rejects_invalid_emailthat only checks a 200 response is not testing what it says. - Look for missing negative cases. Generated suites lean toward happy paths. Ask what input should be refused, and check there is a test for it.
- Check the expected values came from the spec. If the expected result was copied from the code’s own output, the test can’t catch the code being wrong.
- Run mutation testing on critical modules. It plants small bugs in the code and shows whether the tests would notice. More on that here.
04Review becomes the main quality gate
If writing code is cheap, review is where judgement happens. A few habits make a big difference.
- Keep pull requests small. An agent will happily produce 2,000 lines. Ask for the change in slices a person can actually read.
- Make the author explain the risk. A short "what could this break" section in every pull request, whoever or whatever wrote it, focuses the review.
- Review the tests first. If the tests describe the right behaviour, the code review becomes much faster. If they don’t, nothing else matters.
- Pair testers with reviews of risky areas. Payments, permissions, data migrations and anything that deletes. A tester asks different questions from a developer.
05Test closer to each change
When change arrives continuously, a big regression cycle before each release is testing a version that no longer exists by the time it finishes. Move the checks to where the change happens.
- Run the tests affected by a change on every pull request, not the whole suite once a week.
- Keep a fast, reliable smoke suite that runs on every deploy, and treat a flaky test there as a bug.
- Diff what the app requires between builds (API schemas, form validation, permissions) so a weakened rule is visible even when tests pass.
- Use feature flags and canary releases, and decide in advance which signals in production mean "roll back".
06New things to test: the AI features themselves
Many teams are now shipping AI features, not just AI-written code: chat assistants, summaries, smart search. These don’t give the same answer twice, so a single expected value doesn’t work. Testing them means building an evaluation set of realistic questions, defining what an acceptable answer looks like, running each case many times, and tracking the failure rate across releases. It is closer to statistics than to classic assertion testing, and it is one of the fastest-growing skills in QA.
07What to measure
Speed is easy to measure and easy to celebrate. Measure stability next to it, or AI will make your numbers look better while your users have a worse time. The four DORA measures are a good place to start: how often you deploy, how long a change takes to reach production, how often a change causes a failure, and how long it takes to recover.
Adapting your testing to AI-assisted development
- Pull requests stay small enough for a person to read, whoever wrote them
- Generated tests are reviewed for real assertions and negative cases
- Expected values come from the spec, not from the code’s output
- Affected tests run on every pull request; a smoke suite runs on every deploy
- Schema and validation diffs are checked between builds
- AI features have an evaluation set and a tracked failure rate
- Change failure rate and recovery time are measured alongside speed
Frequently asked questions
Is AI-generated code less reliable than human code?
Not by nature, but it arrives faster and in larger amounts, and it is often reviewed less closely per line. That raises the chance that a subtle mistake reaches production unless review and testing keep up.
Can AI write my tests for me?
It can draft them quickly, which is useful. The risk is tests that confirm what the code does rather than what it should do. Review generated tests for real assertions, negative cases and expected values taken from the requirements.
What are the DORA metrics?
Four measures of software delivery from Google’s DORA research: deployment frequency, lead time for changes, change failure rate and time to restore service. Together they show whether you are getting faster without becoming less stable.
How do you test an AI feature that gives different answers each time?
Build a set of realistic test questions, define what an acceptable answer looks like, run each one many times at the settings used in production, and track how often answers fail. Compare that rate between releases.
Will AI replace software testers?
It is changing the job more than removing it. Writing code got cheap, so deciding whether code is safe to ship became more valuable. Testers who can review AI output, design strong negative tests and evaluate AI features are in demand.
Written by
Vihari Panyam
QualityAIQ