AI-generated code should be treated like a contribution from an unknown developer: understand the entire diff, verify the intended behavior, test it independently, and combine human review with automated security checks before merging. A green test run or clean scanner report is useful evidence, not proof that authorization, business rules, supply-chain choices, or agent-driven changes are safe.
This workflow covers code written with an AI assistant or coding agent. It is different from testing an application that uses AI at runtime. The reviewer remains accountable for the change, including files the agent touched outside the requested feature.
Why does AI-generated code need the same or greater scrutiny?
Generated code can look polished while misunderstanding requirements, trusting client input, skipping an object-level authorization check, selecting the wrong package, or changing unrelated configuration. OWASP’s secure-code-review guidance states that “Generated code needs at least the same scrutiny as code from an unknown contributor.” Read the complete diff yourself; an agent’s explanation, summary, or self-review is not an accountable approval. See OWASP’s Secure Code Review guidance.
Start by writing down the change’s intended behavior, affected data, entry points, and trust boundaries. Identify which requests are unauthenticated, which users may access each object or function, and where data crosses from an untrusted source into a query, file path, URL, template, command, log, or response. This gives you a specification against which to judge both the implementation and its tests.
What should I do before running unfamiliar generated code?
Inspect the change surface before installing packages or executing scripts. The safest first pass is about containment and provenance, not whether the feature appears to work.
- List every changed file. Include source, tests, lockfiles, package manifests, build scripts, Dockerfiles, infrastructure, documentation used as agent instructions, CI workflows, and generated artifacts. Unexpected edits are findings until explained.
- Check agent permissions. Keep repository, filesystem, network, credential, and deployment access proportionate to the task. Treat repository text and external tool responses as untrusted input that could influence an agent.
- Examine dependencies before installation. Verify the exact package identity, maintainer history, release history, and requested version. A plausible-sounding package name can be nonexistent, typosquatted, abandoned, or incompatible with the project. Read install and post-install scripts for network access, shell execution, or credential handling.
- Review automation changes. A workflow that runs pull-request code with privileged tokens, modifies release steps, or broadens deployment permissions can create more risk than the feature itself.
- Preserve a clean baseline. Record the existing test commands and assertions before accepting any generated test changes. Do not let an agent remove failures merely to make the branch green.
OWASP’s Secure Coding with AI Cheat Sheet specifically calls out dependency, build-script, CI, and agent-instruction risks.
How do I review AI-generated code before merging it?
Review from the requirement outward: first establish what must happen, then trace every input, decision, side effect, and error path that implements it.
1. Confirm the requirement and state transitions
Turn acceptance criteria into observable cases. Check normal, missing, repeated, out-of-order, expired, and conflicting operations. Generated code often handles the happy path while producing an invalid state at a boundary or during a retry. Verify that transactions, idempotency rules, locking, and rollback behavior match the specification.
2. Trace untrusted input end to end
For each request parameter, header, uploaded file, message, environment value, or external response, identify validation, canonicalization, storage, and every later use. Confirm that queries are parameterized; file names and paths are constrained to an intended directory; URLs are restricted to approved schemes and destinations; and output is encoded for its actual context, such as HTML, JavaScript, SQL, shell, CSV, or a log format.
3. Verify authentication and authorization on the server
Authentication establishes who is calling; authorization decides what that caller may do. Check authorization at every sensitive function and for every object, not only in a front-end route or a visible button. A request should fail closed when identity, role, tenant, ownership, or policy data is missing. Test a signed-in user requesting another user’s object, a lower-privilege user invoking an administrative function, and a request that changes an identifier after an authorization check.
4. Inspect secrets, sessions, and cryptography
Search for embedded API keys, passwords, private keys, tokens, and credentials in source, fixtures, logs, error messages, and generated documentation. Confirm that session cookies use appropriate security attributes, tokens expire and are validated for the intended issuer and audience, and reset or invitation links cannot be reused. Prefer vetted cryptographic libraries and secure random-number generators; do not accept a custom cipher, predictable token, or home-grown password scheme because it is concise.
5. Read error and logging paths
Errors should fail closed and reveal enough for operators without exposing stack traces, secrets, tokens, internal paths, or personal data to callers. Check whether retries can duplicate a payment, message, permission change, or other side effect. Ensure logs do not become an injection channel and that sensitive values are redacted consistently.
6. Compare tests with the code, not just the test result
Look for assertions that actually distinguish correct from incorrect behavior. A generated test that mocks the authorization layer, replaces the database query with a canned response, or asserts only that a function returns without throwing may prove very little. Investigate deleted tests, weakened assertions, broad snapshots, and new mocks around the exact behavior that needs verification.
Use OWASP’s secure review checklist as a concise prompt for authentication, authorization, input handling, output encoding, secrets, cryptography, error handling, and data protection.
Are passing AI-generated tests enough?
No. Tests produced by the same agent that wrote the implementation can encode the same misunderstanding. OWASP warns: “A passing test suite generated by the same agent that produced the code provides no independent assurance.” Preserve existing coverage, add tests from the specification yourself or with an independent process, and compare old and new assertions before trusting a green result.
Build a layered test plan
- Normal cases: valid inputs, expected roles, ordinary state transitions, and successful responses.
- Negative cases: missing fields, malformed encodings, invalid types, unauthorized users, unknown identifiers, and rejected operations.
- Boundary cases: minimum and maximum lengths, zero and negative values, time-zone and date boundaries, empty collections, integer limits, and rate-limit thresholds.
- Credential cases: absent, expired, revoked, malformed, wrong-audience, and wrongly scoped tokens.
- Concurrency cases: simultaneous updates, repeated submissions, retries after partial failure, and conflicting writes.
- Data-flow cases: payloads containing markup, query metacharacters, path separators, unexpected Unicode, oversized files, and external URLs.
For a web application, exercise the running behavior in an isolated environment where appropriate. A unit test can show that a helper returns “allowed” while an actual route still exposes another user’s record. Record what was executed and what remains unverified; do not describe passing tests as a security guarantee.
What security scans should I run?
Use several layers because each examines a different part of the system. The choice depends on language, deployment model, data sensitivity, and change risk; a tiny documentation edit does not require every check, while an authorization or deployment change may require all of them.
| Check | Examines | Typical place to run | What still needs human confirmation |
|---|---|---|---|
| Static application security testing (SAST) | Source-code patterns such as injection sinks, unsafe APIs, and flow violations | IDE and pull request | Business rules, intended data flow, false positives, and missing checks |
| Software composition analysis (SCA) | Direct and transitive packages, versions, licenses, and known advisories | Manifest, lockfile, build, and pull request | Whether the package is the intended one, whether it is actually reachable, and whether upgrade changes behavior |
| Secret scanning | Credentials and key-like material in source, history, configuration, and logs | Commit, repository, and CI | Whether a detected value is live, where it was exposed, and how to revoke and replace it |
| Dynamic application security testing (DAST) | Observable behavior of a running web application | Isolated staging environment | Coverage of authenticated paths, business logic, authorization context, and findings that require safe reproduction |
| Manual secure review | Requirements, trust boundaries, authorization, state changes, and context | Pull request and risk review | Nothing can replace the reviewer’s understanding and accountable approval |
OWASP names Semgrep and Snyk as examples of IDE-oriented checks, and ZAP and Burp Suite as examples for dynamic web testing. They are options rather than a ranking or a claim that one product verifies an entire system. Findings need triage: investigate the code and exploitability instead of accepting an AI-generated “fix” blindly. See the OWASP review guidance and its testing references.
What should I check when an AI agent changes dependencies or CI?
Review infrastructure and supply-chain edits as security-sensitive code, even when the requested feature was unrelated.
Package and lockfile checks
- Confirm the package name, registry, version, integrity data, and transitive additions.
- Compare the lockfile diff with the manifest; unexpected upgrades or registry changes require an explanation.
- Inspect lifecycle scripts and build hooks for downloads, shell commands, credential access, or writes outside the workspace.
- Check advisories and maintenance status, then decide whether the dependency is necessary for the feature.
Agent instruction and repository-content checks
Look for new or modified instruction files that alter how future agents read, write, test, or deploy code. Untrusted issue text, documentation, fixtures, or generated output should not be allowed to override the task’s permissions or safety boundaries. Keep agent credentials scoped and ephemeral where possible.
CI, container, and infrastructure checks
Review changes to workflow triggers, permissions, runners, artifact uploads, container users, base images, secret exposure, network access, and deployment gates. A pull request workflow that executes fork-controlled code with write access can turn a harmless code contribution into credential theft. Confirm that production credentials and deployment actions are separated from ordinary validation.
OWASP’s AI secure-coding guidance covers these agent-specific and supply-chain concerns.
How should review depth change with risk?
Escalate before merge when a change affects authentication, authorization, cryptography, payments, personal data, deployment, CI privileges, or a shared security control. A qualified reviewer who understands the system should approve the change; use code ownership and an additional reviewer for high-impact paths.
OWASP’s Artificial Intelligence Security Verification Standard (AISVS) 1.0 was released in June 2026. It contains 191 requirements across 12 chapters and three appendices. Those figures describe the standard’s scope, not a mandatory checklist for every pull request. Its Appendix C maps AI-assisted-coding controls to verification levels: qualified human review is a level-1 control; broader automated testing and elevated review appear at level 2; and property-based or differential fuzz testing of security-critical behavior appears at level 3. Consult AISVS Appendix C and the AISVS 1.0 documentation when setting an assurance level.
What does published evidence say about vulnerabilities in AI-generated snippets?
One frequently cited study by Fu and coauthors examined 733 AI-generated snippets identified in public GitHub projects. Static analysis and manual filtering flagged 29.5% of the sampled Python snippets and 24.2% of the sampled JavaScript snippets across 43 CWE categories. The figures describe that selected population and method, based on an arXiv manuscript originally posted in 2023; they are not an estimate of the vulnerability rate for all AI-generated code, current assistants, or deployed applications. Read the abstract at arXiv.
The useful conclusion is methodological: generated examples can contain many kinds of defects, and a percentage from one selected corpus cannot replace review of the code in front of you. Treat published rates as context, not as a pass/fail probability for a pull request.
Which standards and guides are useful for a review?
Use the standard that matches the question you are answering.
- Secure-code review: OWASP’s Secure Code Review guidance provides a compact review focus.
- AI-assisted coding controls: AISVS Appendix C addresses human review, testing depth, and escalation by verification level.
- Web-application procedures: OWASP’s Web Security Testing Guide provides versioned procedures. OWASP lists WSTG 4.2 as an available release while version 5.0 is still in development; do not present the in-development version as the stable release.
OWASP also links a printed Testing Guide 4.0 paperback, but that edition dates to 2014. The online 4.2 material is the more relevant reference for current web-testing procedures; neither edition is a substitute for AI-assisted coding controls.
What should a pull-request gate record?
A concise record makes the approval auditable without claiming more certainty than the checks provide.
- The intended behavior, affected assets, trust boundaries, and risk classification.
- Every changed file, including dependency, lockfile, workflow, container, infrastructure, and instruction-file changes.
- Tests run, environments used, inputs covered, and important cases that remain untested.
- SAST, dependency, secret, and dynamic-test results, with each finding’s disposition and reviewer.
- Human reviewers who understand the affected system and any required security or code-owner approval.
- Rollback or remediation steps if a package, credential, workflow, or authorization defect is discovered after merge.
Approve only when the reviewer understands the complete change and its residual risk. “All checks passed” is a status, not a security conclusion.
What are the most common review failures?
- Reviewing only the requested feature: the agent also changed a lockfile, workflow, Dockerfile, or instruction file.
- Trusting plausible package names: the dependency is not the intended project or introduces a dangerous install script.
- Checking only client-side permissions: a caller can invoke the server endpoint directly.
- Testing only valid input: malformed, oversized, expired, repeated, or cross-tenant requests were never exercised.
- Accepting a green generated suite: assertions were weakened or the security boundary was mocked away.
- Treating scanners as proof: clean output cannot establish business-logic correctness or complete authorization coverage.
- Applying generated fixes blindly: a warning is silenced by disabling validation, broadening permissions, or hiding an error.
Correct these failures by returning to the requirement, tracing the data and authority boundaries, and obtaining an accountable human approval at the appropriate risk level.
Frequently Asked Questions
Can I merge AI-generated code if all CI checks are green?
Only after a human has read the full diff and confirmed the intended behavior, authorization, data handling, and test independence. CI proves that the configured checks passed; it does not prove that the checks cover the right cases.
How do I test authorization in an AI-generated API?
Create requests for an unauthenticated caller, a signed-in user accessing another user’s object, and a lower-privilege user invoking an administrative function. Repeat those cases against every relevant endpoint and identifier path, including direct requests that bypass the user interface.
Should AI-generated dependencies be pinned?
Use the project’s normal lockfile and version policy, then verify the exact package, registry, integrity data, and transitive changes. Pinning alone does not make a typosquatted or vulnerable package safe, so provenance and advisory review remain necessary.
When should an AI code change receive a second reviewer?
Require additional or security-qualified review for authentication, authorization, cryptography, sensitive data, deployment, privileged CI, and shared security controls. The higher the impact, the stronger the review and testing evidence should be.