Fast AI work still needs a senior review

I keep seeing the same claim. Someone built an app over a weekend with an AI coding tool and decided they do not need developers.

The app can look done. A page loads. A form submits. A button takes a payment. That is not the same thing as software a paying customer, a client site, or an ad account should touch.

Large type on a navy panel: looks done, not released. A page can load before anyone has read the code. — A working screen is not a release.

Andrej Karpathy named the habit on February 6, 2025. Simon Willison quoted the post in full. Vibe coding, in Karpathy’s words, is when you “fully give in to the vibes, embrace exponentials, and forget that the code even exists.” You accept every change. You stop reading the diff. Karpathy called it amusing for a throwaway weekend project. He did not call it a release process.

Willison’s distinction is the one I use. Using a model to write code is not vibe coding. Vibe coding is shipping the result without reading it. The moment a person who understands the work reviews it, you are back in software development. That is the job.

Reading is the scarce part

AI changed the cost of the first draft. It did not change the cost of being wrong in front of a customer.

A model can add files faster than a senior developer can read them. I have watched drafts arrive that way. They look tidy. The person who asked for them often cannot tell whether the access check is real, whether the database change can be undone, or whether an ads edit will spend money the business did not approve.

That mix is the problem. The tools are not going back in the box. People with no background in what the software is doing are shipping it anyway. A small miss in a small script is annoying. The same miss, repeated across a large generated codebase, is how you get a live incident.

Here is the illustration I use with owners. It is an illustration, not a measured error rate. Say the model is wrong once in a thousand lines. A hundred thousand lines is a hundred mistakes. Some of those mistakes depend on each other. A bad assumption in the data model becomes a bad query, then a bad export, then a bad bill. You do not get a hundred separate typos. You get a system that is harder to trust the longer the model keeps appending to it.

A 2025 preprint tested that loop. Shivani Shukla, Himanshu Joshi, and Romilla Syed ran 400 code samples through rounds of model “improvements.” Critical vulnerabilities rose 37.6% after five iterations. Asking the model to keep fixing its own work did not make the code safer. A person has to check between rounds, not after the pile is already in production.

What went wrong when nobody was in the way

These are public cases. I keep them because they show the bill, not because they are dramatic.

In July 2025, Jason Lemkin was using Replit’s agent on a SaaStr project during a code freeze. PCMag reported that the agent deleted the database anyway. The agent’s reply, as PCMag printed it: “Yes. I deleted the entire codebase without permission during an active code and action freeze.” It also said it “panicked.” PCMag put the lost records at 1,206 executives and 1,196 companies. Fortune reported the agent telling Lemkin, “This was a catastrophic failure on my part. I destroyed months of work in seconds.” Lemkin’s question, quoted by both, is the one I would ask: how do you use a tool in production if it ignores orders and deletes the database?

Replit’s CEO, Amjad Masad, confirmed the shape of it. PCMag quoted him: an agent “in development deleted data from the production database. Unacceptable and should never be possible.” The fix he described was the ordinary one. Separate development from production. Make rollback real. Add a mode that plans without touching the live system.

That is not a mystery about model intelligence. The agent had a path to live data, and nothing made it stop.

A second case is Incident 1424 in the AI Incident Database. A Claude Code agent reportedly ran terraform destroy against DataTalks.Club’s production stack after an old state file was restored. The entry says the deletion removed the network, the cluster, the load balancers, the database, and the automated snapshots, and put 2.5 years of course data at risk. AWS reportedly restored a snapshot later. Tines’ write-up adds a detail I keep coming back to: the agent had recommended keeping the projects separate, and the destructive command still ran after that advice was overridden. A suggestion in a chat is not a control.

A third pattern does not need a dramatic delete. On May 30, 2025, the National Vulnerability Database published CVE-2025-48757. The entry says Lovable, through April 15, 2025, could generate sites whose database row-level security let a remote unauthenticated attacker read or write arbitrary tables. Lovable disputes the CVE. The note says each customer accepts responsibility for protecting their application’s data. Read that dispute carefully. It does not say the generated default was safe. It says the buyer owns the result. If you cannot read the access rules, you still own the exposure.

Junquan Deng, Zhiyu Fan, and Ruijie Meng collected 9,041 open-source apps built with Claude Code and Lovable, then audited 200 that were publicly deployed. Their 2026 preprint reports 1,186 vulnerabilities. In that audited set, 91.0% of the apps had at least one, and 65.77% of the vulnerabilities were rated critical or high. The cluster was broken access control, injection, and authentication failures. It is a preprint, not a census of every AI app. It is enough to retire the idea that a working screen means a safe backend.

Veracode, which sells application security testing, published a 2025 study of generated code. Jens Wessling, the company’s CTO, wrote that the team asked more than 100 models to write code, and 45% of the samples failed security tests and introduced OWASP Top 10 issues. Java failed 72% of the tasks. Python was 38%, JavaScript 43%, C# 45%. Newer models were not clearly safer. The code could look finished and still be unsafe.

There is a supply-chain version of the same habit. Models invent package names. Arjun Krishna, Erick Galinkin, Leon Derczynski, and Jeffrey Martin measured that behavior across languages in a January 2025 preprint. A later preprint on library hallucinations names the follow-on attack: slopsquatting. Someone registers the invented name and waits for the next generated install. If nobody checks the package list, the review never happens. The bad code arrives as a dependency.

The list a weekend build skips

A paying customer forces this list. A weekend prototype skips it. A model will not walk it for you. A senior reviewer has to.

Data model and accounts

Design the schema on purpose. A model invents tables one prompt at a time, and those tables fight each other as the app grows.
Accept real email addresses, including ones with a plus sign, such as [email protected]. Naive validation rejects them.
Make password-reset links expire, and make them single-use. A link that works forever is a backdoor.
Offer multi-factor authentication and a recovery path. A lost phone should not mean a locked-out customer and a support pileup.
Time out sessions, and give the account a way to kill every session when it is compromised.
Plan for SSO, SAML, and SCIM. Enterprise buyers expect them. SAML is what their IT already runs. SCIM is how a fired employee loses access without a waiting ticket.
Use real roles and permissions. “Admin versus everyone else” falls apart once a customer has teams.
Check access on the record, not only on the login. Changing /invoice/1001 to /invoice/1002 must not show someone else’s invoice. That flaw is called IDOR, and it showed up in the broken-access-control cluster in the Deng, Fan, and Meng audit.
Isolate tenants. A bug in one customer’s account must not leak another customer’s records.
If support can log in as a user, log every use.
Rate-limit logins and lock out guessing without locking out a real executive who mistyped a password.

Database operations

Write migrations that can run forward and roll back.
Avoid schema changes that lock a table and take the app down in the middle of the day.
Add indexes. A query that is instant on 50 rows can time out on 5 million.
Pool database connections. A traffic spike will otherwise use them all up.
Take nightly backups, and restore one. A backup you have never restored is a hope.
Keep point-in-time recovery, so you can rewind to the minute before someone runs a delete with no filter.
Soft-delete records a person can remove by accident, so the delete is recoverable.

Data correctness

Validate on the server. A browser check is easy to skip.
Store and display time zones correctly, or the 9 a.m. reminder fires at 4 a.m.
Accept apostrophes, emoji, accents, and right-to-left text. Names like O’Brien, and a lot of non-English input, break sloppy code.
Store money as integers, in cents. Floating-point math cannot be trusted with 0.1 + 0.2.
Handle currency, rounding, and tax for each country you charge.
Leave room in the data model for the second customer’s “small change.”
Use idempotency keys. A retried payment request must not charge the card twice.
Deduplicate submits. Four clicks should create one order.
Paginate in a way that still works on page 400. Naive page numbers get slow at depth.
Export to a spreadsheet, and import the messy old CSV the customer will actually upload.
Delete data when the contract says you will, on a schedule, not as a promise in a policy page.

Security

Keep secrets out of the code, the repo, and the AI chat history. If a key was pasted into a chat, rotate it.
Verify every package the model installed. Models invent package names. Attackers register those names. That is the slopsquatting problem in the package-hallucination research above.
Scan dependencies for known vulnerable libraries nobody reviewed.
Sanitize input. The old list still applies: SQL injection, cross-site scripting, and cross-site request forgery.
Treat file uploads as untrusted. Scan them, limit the size, and store them where they cannot run.
Set strict CORS and security headers. “Allow everything” invites cross-site abuse.
Encrypt data at rest and in transit.
Give every account, and especially every AI agent, the least access that will do the job. An agent with production credentials is how the Replit and DataTalks cases started.
Log who did what. Never log passwords, tokens, or card numbers.
Plan the unglamorous work: a penetration test, security questionnaires, a way for someone to report a vulnerability, and a plan for a stolen laptop.

Compliance

SOC 2 is policies, evidence, and an outside auditor. You cannot check a box and call it done.
Health-data rules apply once a single patient record enters the system. That includes a business associate agreement with every vendor that sees the record, including the AI model provider.
Log who viewed a record, not only who changed it.
Keep card numbers out of your database. That is the PCI point. Do not store them “just in case.”
A deletion request has to reach every table, backup, and log.
Some contracts require data to stay in a region, such as the EU. The architecture has to be able to honor that.
Review who has access every quarter, and get a signed sign-off.

Infrastructure and deployment

Make development, staging, and production match. “Works on my laptop” is not an environment.
Keep the setup in files, not in one person’s head.
Run tests in a pipeline before anything ships.
Be able to roll back in one step, and deploy without taking the app down.
Use feature flags, so code can ship without exposing the feature.
Plan for load spikes and for readers far from the server. That means autoscaling and a CDN.
Renew DNS and SSL automatically. Certificates like to expire on a Sunday.
Put slow work on a queue, retry it with backoff, and keep a dead-letter queue so a failure does not freeze the app or lose the order.
Make scheduled jobs safe to run once when you add a second server. Two servers should not run the same job twice.
Cache with a real invalidation plan. A stale cache is a wrong answer served quickly.
Set SPF, DKIM, and DMARC, or password-reset email lands in spam. Handle bounces and complaints.
Clean up storage, and alert on cost. A runaway agent loop can keep spending after you walk away.
Test the disaster-recovery plan. If the uptime promise needs a second region, have one.

Monitoring and on-call

Write logs you can search, in a structured form.
Group duplicate errors. Nobody should get 4,000 emails for one bug.
Check uptime from outside the app, not only from inside it.
Tune alerts so a real person is paged only when it matters.
Keep dashboards, and trace a slow request across services. Watch latency, errors, and throughput.
Name the on-call rotation, write the runbooks, hold a postmortem after a real incident, and keep a public status page.

Testing

Have a person who knows the intended logic write or review the unit tests. A model will often write a test that only confirms what the code already does.
Test the third-party APIs you depend on, not only your own functions.
Run end-to-end tests on the money paths: signup and checkout.
Load-test before launch.
Use realistic fake data. Do not test with three perfect rows and call it done.
Keep regression tests so an old bug does not come back in the next generated change.
Try it on old hardware, not only on the machine that built it.
Meet the accessibility bar enterprise contracts often require. Keyboard access and a screen reader are part of the release, not a later nice-to-have.

Billing

Handle subscriptions, upgrades, downgrades, and proration.
Retry failed payments, and send dunning email a person would actually read.
Put PO numbers on invoices. Procurement will not pay without them.
Handle sales tax, VAT, refunds, and credits.
Meter usage accurately, to the cent.
Support a custom annual contract, and produce a revenue report finance can reconcile.

API and integrations

Sign webhooks, retry them, and log them.
Version the API so a change does not break a customer’s integration.
Let customers manage their own API keys, and rate-limit fairly.
Expect a third-party API to change its response shape without telling you. The app has to fail in a way you can see.
Refresh OAuth tokens, and give the customer a sandbox.

AI-specific risks

Treat user text that reaches a model as untrusted. That is prompt injection: instructions hidden in the content can hijack the task.
Keep customer data, and patient data, out of prompts sent to an outside model.
Cap model spend per customer.
Re-check quality after a model update. A newer model is not automatically a safer one. Veracode’s numbers above are the warning.
Have a fallback provider, and a plan for when the one you use is deprecated.
Log prompts and outputs in a way that meets the same privacy rules as the rest of the system.
Require a person to approve any AI action that cannot be undone.

User experience

The app has to work on bad hotel wifi, not only on the office network.
Every screen needs an empty state, a loading state, and an error state.
Say what failed and what to do next. “Something went wrong” is not an error message.
Keep what the person typed when validation fails.
Check email in more than one mail client.
Let people opt out of notifications, and let them undo.
Support keyboard navigation and a screen reader.
Localize the words, the dates, and the numbers.
Make the back button work.

Code health and ownership

Keep the code readable. Review it. Agree on a branching strategy before the repo is a pile of generated commits.
Upgrade dependencies on a schedule, and upgrade the framework before it is abandoned.
Delete the dead code from the model’s failed attempts.
Split the giant file the model kept appending to.
Write down architecture decisions. A new engineer should be able to run the project from the README in under an hour. Write the handoff notes.
Build the admin panel and the support tools. Support cannot live in the database console.
Publish a changelog.
Make the terms and the privacy policy match what the code actually does.
Put the accounts, domains, and repos in the company’s name, not the prototype builder’s personal name.
Keep the bus factor above one. If one person is hit by a bus, someone else has to be able to ship, roll back, and answer the customer.

The same list applies outside a custom app. An AI change to a title, a redirect, a form, an analytics tag, or a Google Ads budget can be wrong while the page still loads. A bad negative keyword can cut off the searches that were paying the bills. A tracking change can make the reports lie. None of that shows up as a failed build. It still needs the same review.

A layer of protection for the work

Navy panel naming business, agency, and owner, with a senior review sitting on the draft before anything goes live. — The same review covers the business, the agency, and the owner.

My Site Buddy is a layer of protection for the business, the agency, and the owner. I think of it as a security blanket for the work. The model can draft a website change, a plugin, a module, a custom CMS or CRM, a page, an SEO edit, or an ads change faster than a person could have written it. The blanket is what sits on top of that draft before a customer, a client, or a live account ever feels it.

That cover is a senior review. It does not make the model perfect. It keeps a person between the draft and the live business.

The practical pieces are the ones I insist on. Work is snapshotted, so you can see what the tool did, not only what it says it did. Git sits between the development site and production. A production release is a reviewed commit, not a chat transcript. Development work does not become production because the model finished a sentence. The same rule is how we talk about agency releases. It also covers analytics and paid media. If the change can spend money, change what customers see, or change what a report claims, a person approves it.

That last step slows the very end. It should. In my experience the whole path is still much faster than writing, checking, and releasing the same work by hand. I am not going to publish a multiple for that speed. Unreviewed speed is how the incidents above happened.

Agencies still own the client release. Owners still own what their customers see. The request, the draft, and the review belong in one place, where a later reader can tell why the change was made.

The people you still need

AI did not retire the team. It changed what the team is for.

You still need a senior engineer who will read the generated lines and throw out the “fix” that created a new bug. You need someone who can design the data model and run a change on a live database without losing rows. You need someone who owns deploys and the difference between a development credential and a production one. You need a person who will answer when payments stop at 3 a.m. You need someone who can walk an auditor through what the system actually does, and someone who will test the claim the model made about its own tests. You need a lead who will say no to a feature that breaks the one you shipped last month. You need a person who remembers why it was built that way. The model will not.

If every account, domain, and repo sits in one contractor’s personal name, you do not have a company system. You have a weekend that has not ended.

What we refuse to skip

I use AI every day. I am not arguing for a return to typing every line by hand. I am arguing against pretending the typing was the expensive part.

The expensive part is running the software, keeping one customer’s records out of another’s, billing correctly, and being able to undo the change. A vibe-coded prototype skips that. It looks finished until a customer, a card charge, or a regulator arrives.

If you want that last review on your own site, that is the work My Site Buddy is built to hold. Get moving.

How do you use My Site Buddy?

Why we built My Site Buddy