Picture a system where you describe an application in plain English, and a team of AI agents generates a working, full-stack app for you - frontend, backend, the works. I’ve been building something like this, and if you’ve built anything like it too, you already know the dirty secret: getting an AI to write code is the easy part. Getting it to write code that actually runs - on the first try, cheaply - is where all the real engineering happens.

For the person using it, when this goes wrong it doesn’t look like a crash. It looks like generation taking forever, costing more than it should, or - worst of all - finishing successfully and handing back an app that looks done but quietly doesn’t work. So the goal isn’t just “catch errors and fix them.” It’s knowing, at every point, whether fixing is actually working, whether it’s wasting money, and when to stop.

How a system like this fits together

Think of it as a small assembly line, where each stage hands its work to the next:

Plain-English request
  -> Planning agent (decides what the app needs)
  -> Architect agent (decides how information is stored)
  -> Backend agent -> backend safety check
  -> Frontend agent -> full-app safety check
  -> Repair agent (only runs if a check fails)
  -> A spend guard watches the entire run, start to finish

That “repair agent” step - and the two checks that feed it - is where I spent the last couple of weeks, and I found three bugs worth sharing. None of them are really “AI bugs.” They’re the same mistakes teams have been making with regular software for decades. AI just gave them a new place to hide.

Why one safety net isn’t enough

The obvious design is: if the generated app is broken, show the AI the error and let it try again. I have that. What I learned is that a single retry loop, on its own, isn’t safe - it either gives up too early on fixable problems, or it keeps trying long after it should have stopped, quietly running up the bill.

So instead of one loop, I run three independent safety nets, each watching for a different kind of trouble:

1. A backend check, done early. Before I bother generating the frontend at all, I make sure the backend actually starts up, connects to its database, and answers basic requests correctly (create something, read it back, delete it, confirm it’s gone). This is cheap and fast to verify, so I catch problems here before they get expensive.
2. A full-app check, done later. Once the whole app is assembled, I check that the frontend and backend actually work together - not just that each one runs on its own, but that, say, tapping “Save” on a screen actually calls the endpoint the backend really built, with the data it actually expects. A generated button calling /api/tasks while the backend only built /api/todos is exactly the kind of mismatch this catches - each half looks fine in isolation, and the app is still broken. This check is more expensive to run, so I only get here after the cheap check already passed.
3. A spending limit, watching everything. Separately from both checks above, I track how much money and how many AI calls a single generation run has used, the whole time. Cross a soft limit, and the system gets more conservative - smaller, targeted fixes instead of broad rewrites. Cross a hard limit, and the run stops completely, and I tell the user exactly what did and didn’t get built, instead of quietly burning money forever chasing a problem that isn’t getting any closer to fixed (what I’d call “not converging”).

For staff-level readers, the underlying idea worth naming directly: retry count, recovery rate, integration health, and spend are four different signals. None of them is a stand-in for the others. A system that only watches one of them will eventually get fooled by a failure mode that only shows up in one of the other three - which is exactly what happened to me, three separate times.

Bug #1: the fix that “worked,” but cost twice as much every time

When my repair agent fixes broken code, it needs to describe what to change in a structured, predictable format - think of it like a list of edits: “in this file, replace this bit with that bit.” Most of the time the AI returns that list correctly.

Sometimes, though, it would return that list as a single wall of text that was supposed to be structured data but wasn’t quite formatted correctly - a bit like getting a spreadsheet back as a paragraph instead of rows and columns. My system was smart enough to notice this, clean it up, and try again automatically. And it worked! The fix always eventually landed.

Here’s the catch: because it always eventually worked, nothing ever looked broken. No error screen, no failed run. It just silently doubled the cost and time for almost every file it touched, and buried my logs in scary-looking error messages that made the system look like it was failing constantly - when really, it was recovering fine, just very expensively, every single time.

The lesson here isn’t really about AI - it’s a lesson every engineer eventually learns the hard way: “it recovered” and “it’s healthy” are two different claims. A system that automatically papers over a recurring problem can hide that problem from you indefinitely, because nothing ever looks red on the dashboard. The fix wasn’t a smarter retry - it was tracking how often the retry was needed and putting that number somewhere a human would actually see it.

Bug #2: giving the repair agent amnesia about its own evidence

This is the one I like best, because the mistake is so easy to make and so obvious in hindsight.

When the generated frontend code has an error (say, a typo that breaks the build), I show the repair agent the error output so it can fix it. That output can get long if there are a lot of errors, so - reasonably! - I trimmed it down to just the last handful of lines before sending it to the AI, to save time and cost.

Here’s the problem: if there were, say, 40 errors, showing only the last few lines means the AI is looking at whatever happened to be at the bottom of a very long list - not necessarily the errors that actually matter, and sometimes missing which file they were even in. It would dutifully fix whatever it could see, I would re-check the code, a different random slice of errors would now be at the bottom, and the whole process would spin without ever really getting anywhere - burning through my limited number of retry attempts on partial, almost arbitrary information.

This wasn’t a “the AI isn’t smart enough” problem. You could swap in the best model in the world and it would still fail here, because the mistake happened before the AI ever saw anything - I was throwing away most of the evidence first. The fix wasn’t cleverness, it was simple: instead of showing the last few lines, I now show one short summary line per unique error, so a 40-error problem becomes a complete, compact checklist instead of a random peek at the bottom of a page. Same amount of text sent to the AI - just organized so nothing important gets left out.

If you’re building anything that feeds one system’s output back into another system to “go fix it,” this is worth remembering: trimming information to save cost is fine. Trimming it by position (just take the last N lines) instead of by importance is how you build a process that looks like it’s trying to improve and never actually does.

Bug #3: the system that forgot what it had just done

The first two bugs were about the repair step being wasteful. This one was about it being pointless - a case where the system was supposed to pick up where a previous run left off, and instead started over from zero, every single time, without ever telling anyone.

Here’s the setup: when someone asks the system to make a small change to an app it already built (rather than build a whole new one), the sensible thing to do is look up what happened last time and continue from there - the AI equivalent of “let me check the file before I start working on it.” So the system was supposed to fetch the record of the previous run before starting the new one.

The bug: the code created this run’s own (empty) record first, and then went looking for “the most recent run’s record” - which, of course, was now its own, brand-new, empty one. It was checking the file, but it had already put a blank page on top of the pile before looking. Every single “small change” request silently behaved as if it were starting from scratch, because the system could never actually see its own history - it just kept finding itself.

Nothing crashed here either. The run would complete, just slower and more expensively than it needed to be, redoing work that had already been done. This is a subtle but common shape of bug in anything with a “remember progress and continue” feature: the order in which you write your own record and read the previous one matters, and it’s easy to get backwards without noticing, because the system still produces a result - just never the efficient one you designed for.

The real takeaway

None of these three lessons are actually about AI. Having more than one safety net, watching your spend independently of your retry count, noticing when something “always eventually works” instead of just letting it, being careful about what information you trim before handing a problem to whoever’s supposed to fix it, and double-checking that a “remember what happened” feature is actually reading its own history in the right order - all of that is just good engineering hygiene that predates AI by decades.

That’s really the point. The scary part of building an AI-driven system isn’t the AI part - it’s that AI output looks plausible enough that it’s tempting to skip discipline you’d never skip in a normal system: clear, structured signals about what went wrong instead of guessing from logs; more than one independent check instead of trusting a single retry loop; and making sure the evidence - and the memory - survives long enough to actually be useful to whoever, or whatever, is supposed to act on it next. Get those things right, and “make AI-generated software reliable” turns out to be a much more familiar problem than it first appears.