Hypothetical scenario: It's Thursday afternoon in Doral. The owner of a growing logistics company has a pull request waiting for approval. Her two-person dev team used an AI coding assistant to build a customer portal in three weeks instead of three months. The demo looked great. Customers can log in, see their shipments, and download invoices. The team wants to ship Monday. She doesn't read code, and she has to decide whether to say yes.
The title's "half your codebase" describes a situation like hers. It is not a statistic. Whether an AI assistant wrote one function or an entire feature, the decision that matters is what evidence you require before that code reaches customers. Syntax rarely sinks a release. Wrong assumptions do: who can see what, what happens when input is hostile, and how you back out when something breaks.
Why "it works in the demo" isn't release evidence
A demo proves the happy path. A logged-in user with good data clicks the expected buttons. AI-generated code is often very good at the happy path, because that is what the prompt described. The risk sits in everything nobody described: the second customer, the expired session, the malformed upload, the database change that can't be undone.
The tool vendors say the same. GitHub's responsible-use documentation for Copilot agents warns that generated code can look valid while being incorrect or failing to reflect what the developer intended, and that it may contain security vulnerabilities. GitHub recommends reviewing and testing agent output before merging and treating the agent as a tool, not a replacement for human review. Its cloud agent also has built-in limits: it pushes only to its own branch, and workflows on its pull requests need approval from someone with write access. Those limits only help if a person actually does the review.
For a broader frame, NIST's Secure Software Development Framework (SP 800-218) sets out high-level practices meant to reduce vulnerabilities in released software. NIST also notes it gives buyers and suppliers a shared vocabulary. That is useful for an owner: you can ask your team or vendor for evidence in recognized terms, not on instinct. SSDF is a set of recommendations, not a checklist that certifies your app. The questions below are our own working framework, informed by those sources.
Seven areas where you should ask for evidence, not reassurance
1. Authentication vs. authorization
Authentication answers "who are you?" Authorization answers "are you allowed to do this?" The OWASP Authorization Cheat Sheet keeps the two apart. It points out that a logged-in user is often not entitled to every action the system technically allows. It recommends denying access by default, checking permissions on every request, and enforcing checks server-side rather than relying on hidden buttons in the browser. Generated code often gets login right and skips the per-record check. Ask your team to show you where the "is this user allowed to see this record?" check lives.
2. Negative and tenant-isolation tests
Most generated tests confirm that things work. You also need tests proving that things fail when they should. OWASP specifically flags users reaching other users' data by changing an ID in a URL, a flaw class known as insecure direct object reference. If your app serves multiple customers, ask for a test where Customer A's account tries to open Customer B's invoice and is refused. OWASP also maintains a Multi-Tenant Security Cheat Sheet for teams building this kind of separation.
3. Secrets and dependencies
AI assistants will happily paste an API key into a config file or pull in a package nobody evaluated. Ask where credentials are stored and how they're kept out of the repository, and ask for the list of new third-party packages this release adds. OWASP's Secrets Management Cheat Sheet covers the first. Its Authorization Cheat Sheet recommends having a process for detecting and responding to vulnerable components.
4. Input and output trust
Every form field, file upload, webhook and API response is untrusted until validated. That includes output from an AI feature inside your product. Ask which inputs are validated, which outputs are encoded before display, and what happens when a field contains something unexpected.
5. Tests against requirements, not the same assumptions
This one is easy to miss. If the same AI session wrote both the code and its tests, the tests can repeat the code's misunderstanding and still pass. Ask for a short list of business requirements in plain language, written by a person, and a mapping from each one to a test. "Drivers can only see their own routes" should point to a specific test, not to a coverage percentage.
6. Migration and rollback
Database changes are where "we'll fix it Monday" stops being an option. Ask whether this release changes the data structure, whether the change can be reversed, and whether anyone has actually run the rollback on a representative, sanitized staging dataset.
7. Observation and maintainer handoff
After launch, someone has to notice problems and understand the code well enough to fix them. Ask what gets logged, especially failed access attempts. Ask who gets alerted and whether a developer who didn't write the code can explain how it works. If the honest answer is "only the AI session knew," you have an ownership problem as well as a security one. We covered the access side of that in Your Developer Disappeared: Do You Own Your Code, Domain and Hosting?
The owner-friendly request-for-evidence checklist
You don't need to read code to use this. Send it to your team or vendor before release and ask them to answer each row with something you can look at.
| Ask for | Good evidence looks like | Red flag |
|---|---|---|
| Where authorization is enforced | A named server-side check, with deny-by-default behavior | "The button is hidden for non-admins" |
| Cross-customer access test | A test where one account is refused another account's records | Only happy-path tests exist |
| Secrets and new packages | Credentials outside the repo; a short list of added dependencies, each reviewed | Keys in config files; "the AI added what it needed" |
| Input and output handling | Named validation for forms, uploads and webhooks | "Users won't enter that" |
| Requirements-to-tests map | Plain-language rules written by a person, each linked to a test | Tests written in the same AI session as the code, with no human-written requirements |
| Rollback plan | Rollback rehearsed on staging with realistic data | "We'll restore from backup if needed," untested |
| Monitoring and handoff | Failed-access logging, a named alert owner, a short maintainer README | No one but the original session can explain the code |
Run a staging release drill before the real one
This drill is our recommendation, not an official standard. It turns the checklist into a supervised exercise whose duration depends on the release.
- Deploy to staging exactly the way production will be deployed, using the same scripts and no manual shortcuts.
- Log in as two different customers and try to open each other's records by changing IDs in the address bar. You should see a refusal, not data.
- Log in as the lowest-privilege role and attempt one admin action directly.
- Submit bad input on purpose: an oversized file, an empty required field, odd characters.
- Run the database migration, then roll it back, and confirm the app still works on the previous version.
- Check the logs and confirm the failed attempts from steps 2–4 were recorded and someone would have been alerted.
- Ask a developer who didn't build the feature to explain one critical flow back to you.
Hypothetical example: Back in Doral, the owner runs the drill. Step 2 fails: changing an invoice number in the URL shows another customer's invoice. The generated code checked that the user was logged in but never checked who owned the invoice, and the generated tests only covered single-customer scenarios. The fix is small. Finding it after Monday's launch would have meant a disclosure conversation with a client. She ships the following Thursday instead.
Where we fit in
Tech Lab Miami is a South Florida community built by operators, focused on the practical decisions behind technology adoption. AI makes building faster, but the release decision still belongs to a person who has to live with the outcome. Our AI consulting work focuses on that responsible-adoption layer: assessing where AI-assisted development fits, designing review workflows, and helping teams build habits like the drill above. When the risk lives in how systems connect, such as webhooks, integrations and data handoffs, our business automation practice starts with a workflow audit and a measurement plan.
If you're deciding how much to build versus buy, our software strategy overview is a reasonable starting point. If your team wants to build the skills itself, look through what we cover on our learning hub. And if you have an AI-assisted release on your desk right now, tell us about the workflow behind it. We're glad to talk it through with fellow South Florida operators.
FAQs
Is AI-generated code less secure than human-written code?
Not by definition. The risk is that it looks finished before it is. GitHub's own documentation says agent-generated code can contain vulnerabilities and should be reviewed and tested before merging, which is the same standard you'd apply to any code your team didn't independently write.
Do automated security scans replace this checklist?
No. Scanners catch known patterns such as leaked secrets and vulnerable packages. They usually can't tell whether Customer A should see Customer B's invoice, because that depends on your business rules.
I'm not technical. Can I really run the staging drill?
Yes. Most steps are clicking, changing a URL and reading a result. Your developer handles the deployment and rollback. You watch and ask the questions.
Does following NIST or OWASP guidance mean we're compliant?
These are widely used recommendations and reference guides, not a certification. Whether specific regulations apply to your business is a separate question for qualified counsel.
Release review, illustrated


