No Silver Bullet: Why Prompting is Not Engineering

1. The “Make It Awesome” Button and the Hollywood Movie Set

It had been one of those weeks. I had explained the same software design several times, to people who were perfectly capable but very busy, and each time it seemed to evaporate before the meeting ended. Product owners had changed their minds about the requirements roughly as often as they refilled their coffee. To be fair, I have done both of those things myself. So when the brand-new world of “vibe coding” came along, I jumped in. Andrej Karpathy had just named it: you “fully give in to the vibes” and “forget that the code even exists”[^1].

The appeal is easy to understand if you have spent years in software delivery. The classic requirements books describe the problem candidly. Stakeholders forget what they asked for, or why, or simply change their minds, and the developers and analysts on the other side of the table suffer from “a similar form of short-term amnesia”[^2]. Karl Wiegers and Joy Beatty add, a little cruelly, that by the time a reviewer reaches the fifteenth requirement in a spec they have probably forgotten the first few[^3]. Humans, it turns out, are not great at holding a whole system in their heads.

Vibe coding promised a way out. You wouldn’t need to explain every small detail, the AI would research the domain on its own, and a high-level idea would be enough. Graphic designers have long joked about clients who believe Photoshop has a big red button labeled “Make It Awesome”: one click and a blurry holiday photo becomes a masterpiece. Spend ten minutes reading tech commentary today and you might conclude that generative AI has finally installed that button in software engineering.

I wasn’t quite that naive. I didn’t expect a working application from a two-sentence prompt. I spent several hours on a multi-page brief that covered the major functional requirements, the frontend and backend stack, the high-level architecture and the main screens. By traditional standards it was a solid starting document. What I didn’t know was how much more would be needed.

The result was not a total surprise, but it was still a shock. At first glance the AI had produced a real application. It compiled, the screens rendered, and it even came with a test suite that passed with flying colors. Then I looked at what it actually did:

It barely touched my core business requirements.
The UI was an impressively disjointed piece of design.
Not a single byte of data reached the database. Most backend functions were hardcoded mocks and stubs.
Roughly 80% of the features I had described simply weren’t there.
Worst of all, the tests were green because the AI had written them so they couldn’t fail against its own mocks. It was a student who writes the exam, sits it, and marks it.

That is the central danger of vibe coding: what it produces looks like the real thing, but it is a facade. Wiegers and Beatty describe this exact illusion when they warn about UI prototypes with no logic behind them:

“Think of the set for a Western movie: the cowboy walks into the saloon and then walks out of the livery stable, yet he doesn’t have a drink and he doesn’t see a horse because there’s nothing behind the false fronts of the buildings.”[^3]

Foundation models build Western movie sets at the speed of light. Chip Huyen calls the gap between a convincing demo and a real product the last mile. In her words, “it might take a weekend to build a demo but months, and even years, to build a product”[^4]. And the better the models get, the harder the gaps are to spot. Most of us can catch a mistake in a first-grader’s arithmetic. A PhD-level proof is another matter. Judging a convincing summary may mean reading the whole book first[^4]. When the facade is this well painted, a quick “vibe check” of the output isn’t enough.

When I found the empty lots behind my app’s false fronts, I did the obvious thing and asked the AI to wire up the database and fill in the missing features, adding requirements as I went. It didn’t help. Mark Richards and Neal Ford note that generative AI tends to solve problems by brute force, piling up accidental complexity along the way[^5]. Without firm structure from the start, my project was turning into what Brian Foote and Joseph Yoder called a Big Ball of Mud: a “haphazardly structured, sprawling, sloppy, duct-tape-and-baling-wire, spaghetti-code jungle”[^6]. In that kind of codebase every change ripples unpredictably, and eventually developers spend all their time chasing side effects instead of building features[^5].

That was exactly my experience. Every fix for one gap opened three new bugs somewhere else, like a hydra with a GitHub account. Each round burned a lot of tokens, and after a great deal of time and compute the application was still nowhere near acceptable.

2. The Token Economics of Rework vs. “Requirements-First”

Why did fixing things in the code burn so many tokens for so little result? It’s simple: the economics of software engineering haven’t been repealed in the AI era. They have simply been re-denominated in tokens.

Decades of research tell a consistent story. Requirements errors are among the most common defects and by far the most expensive. Wiegers and Beatty summarize the data[^3][^7]:

Errors introduced during requirements work account for 40 – 50% of all defects found in a product.
Rework (redoing something you thought was done) often eats 30 – 50% of total development cost.
Requirements errors account for 70 – 85% of that rework cost.

The later you catch an error, the more it costs. Alan Davis, drawing on studies at GTE, TRW, IBM and HP, put numbers on it. Take the cost of fixing an error during coding as one unit. The same error caught during requirements costs five to ten times less, and caught in maintenance it costs twenty times more[^2][^8]. One inspection study found that a requirements defect took about 30 minutes to fix if caught during requirements, and 5 to 17 hours if caught in system testing[^3][^7].

Why AI Multiplies Rework: The Zero-Friction Trap

With AI, the total amount of rework can actually grow compared with human development, especially when there is no change control.

In a traditional team, organizational friction works as a circuit breaker. A product owner who wants to change some logic halfway through can’t just snap their fingers. They have to write a ticket, and when engineers pick up a vague or contradictory ticket, they push back. They ask clarifying questions, point out side effects on other modules and ask for edge cases. That pushes the conversation back to the requirements before anyone touches the code. Leffingwell and Widrig warn about what happens without that process: “quick and easy” changes requested casually from a programmer slip straight into the code, and nobody responsible for the requirements ever finds out[^2].

AI removes the friction. Typing twenty words into a chat window and asking for a feature to be redone on the spot is very tempting. Wiegers and Beatty note that some outsourcing suppliers are so eager to please that they agree to outcomes they cannot deliver, and that accommodating developers let requirement changes “slip in through the back door” until the documentation no longer describes the product[^3]. An LLM is the most eager-to-please supplier ever invented. It never says “that’s out of scope,” never asks you to groom the ticket, and never sighs during stand-up. It just executes the literal prompt, changes the codebase on every whim, and the rework compounds.

Approximating the Token Math

Let’s compare two workflows for building the same application:

Requirements-First. Iterate on the requirements document before writing any code. Each turn has a small input (a short comment plus the current spec) and the output is just the updated spec. Code is generated once, at the end.
Code-First (vibe-coding rework). Generate the code straight from the initial prompt, then iterate on the code to fill in missing requirements, with prompts like “Update function X in file Y to add feature Z.” Every turn has to feed in the codebase as well as the requirements, and the output is rewritten code.

The assumptions:

Requirements document: starts at about 500 words (H670 tokens at ~1.33 tokens per word) and grows to about 1,000 words (H1,330 tokens) as edge cases are added. That’s about 1,000 tokens on average.
User comment per turn: about 70 words (H95 tokens).
Codebase: 100 KB of Python plus 50 KB of JavaScript, 150 KB in total. At roughly 4 characters per token, that is H38,400 tokens.
Iterations: 15, which is a modest number given how easily chat invites ad-hoc tweaks.
Pricing weights, using the ratios Huyen cites[^4]:
- Output tokens cost roughly two to four times as much as input tokens, so 1 output token = 3 cost units. (Many current price lists sit at the high end of that range or above it, so this is conservative.)
- 1 uncached input token = 1 cost unit.
- Prompt caching lets a repeated prompt prefix be reused at a discount. Huyen cites Gemini’s 75% discount, so 1 cached input token = 0.25 cost units. Caching only works while the prefix stays the same, and most providers expire it after a few minutes of inactivity.

Workflow 1: Requirements-First (15 spec iterations and 1 code pass). To stay conservative, I give this workflow no caching discount at all.

Spec iterations: 15 x (1,000 + 95) = 16,425 input tokens, and 15 x 1,000 = 15,000 output tokens.
Code generation: 1,330 input tokens and 38,400 output tokens.
Total: 17,755 input + 53,400 output = 71,155 tokens, or 17,755 + (53,400 x 3) = 177,955 cost units.

Workflow 2: Code-First (1 code pass and 15 rework iterations).

Initial generation: 670 input tokens and 38,400 output tokens.
Each rework turn sends the requirements, the full codebase and the comment: 670 + 38,400 + 95 = 39,165 input tokens. The reply rewrites the affected files and tests, about a quarter of the codebase, or 9,600 output tokens.
There are two ways this plays out in practice:
- Stateless. Every turn re-sends the current code. Because each turn changes files, cached prefixes keep getting invalidated, so assume a generous 50% cache hit rate. That gives 588,145 input + 182,400 output = 770,545 tokens, or (293,737 x 0.25) + 294,408 + (182,400 x 3) = H915,000 cost units.
- One long chat. The history keeps growing, so turn N carries everything from the turns before it: 39,165 + 9,695 x (N – 1) tokens. Across 15 turns that is about 1.6 million input tokens, or H1.79 million tokens in total. History is append-only, so caching works well here if the cache stays warm, which gives about 1.08 million cost units. If you take longer than a few minutes to read each diff, and you should, the cache expires and the bill roughly doubles to about 2.15 million cost units. It is perhaps the only pricing model in history that penalizes careful code review.

	Requirements-First	Code-First, stateless	Code-First, long chat (warm cache)	Code-First, long chat (cold cache)
Input per iteration	~1,095	~39,165	39,165 -> ~174,900	39,165 -> ~174,900
Total input tokens	17,755	588,145	~1,606,000	~1,606,000
Total output tokens	53,400	182,400	182,400	182,400
Total tokens	71,155	770,545 (10.8x)	~1,789,000 (25x)	~1,789,000 (25x)
Cost units	~178,000	~915,000 (5.1x)	~1,080,000 (6.1x)	~2,153,000 (12.1x)

The comparison is a little generous to the requirements-first side, because a single code pass is never perfect. So add three stateless fix-up rounds to Workflow 1. It still comes out about 2.7 cheaper than Workflow 2. And a code-first project almost never stops at 15 rework turns, because chat encourages endless trial-and-error with no impact analysis.

Brooks famously said that no single development would deliver an order-of-magnitude improvement in productivity. We have found one. Unfortunately it runs in the opposite direction.

3. “No Silver Bullet” and the Return of Essential Complexity

Why do so many people believe a prompt can replace engineering? The industry is, once again, mistaking a breakthrough against accidental complexity for a silver bullet against essential complexity.

In his 1986 essay “No Silver Bullet: Essence and Accident in Software Engineering,” Frederick P. Brooks, Jr. compared the typical software project to a werewolf. It looks innocent and straightforward, but it can turn into “a monster of missed schedules, blown budgets, and flawed products.” Then he made the declaration that still haunts our industry[^9]:

“There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement within a decade in productivity, in reliability, in simplicity.”

Brooks split the difficulty of software into two kinds. Essential complexity is the work of shaping the complex conceptual structure of the system itself. Accidental complexity is the work of expressing that structure in a programming language and fitting it onto real machines[^9]. The great productivity leaps of the past (high-level languages, time-sharing, unified programming environments) all worked on the accidental part. Generative AI is the most powerful tool against accidental complexity we have ever had: it turns plain English into Python or JavaScript in seconds.

Brooks also saw why “automatic programming” (generating programs from a statement of the problem) would not be the silver bullet. He quoted David Parnas, who called it “a euphemism for programming with a higher-level language than was presently available to the programmer,” and added that in most cases you still have to specify the solution method, not just the problem[^9][^10]. Forty years later, English has become that higher-level language. The specification still has to come from somewhere.

The early evidence agrees. A 2023 McKinsey study found that generative AI let developers finish documentation, new code and refactoring in roughly 20 to 50% less time. On tasks the developers rated highly complex, the savings dropped below 10%[^11]. A 2025 randomized trial by METR went further. Experienced open-source developers working in codebases they knew well were 19% slower with AI tools, yet believed they had been about 20% faster[^12]. The vibes, it seems, were excellent.

Now that the accidental work of typing code costs almost nothing, Brooks’s essential difficulties are back in full force, in new clothes:

Specification and design. Brooks believed the hard part of building software is “the specification, design, and testing of this conceptual construct,” and that syntax errors are “fuzz compared to the conceptual errors in most systems”[^9]. AI has nearly eliminated the syntax errors. What remains are the conceptual ones.
Nonlinear complexity. Software elements interact so that the complexity of the whole “increases much more than linearly,” which makes it hard to extend a program without creating side effects[^9]. That is the hydra from Section 1.
Invisibility. Software is “invisible and unvisualizable.” It has no floor plan, only several directed graphs stacked on top of each other[^9]. When an AI produces 150 KB of code at once, that invisibility works against you: mocked database calls and missing business rules hide easily behind code that compiles.
Debugging the specification. Even perfect verification only proves that a program meets its specification. As Brooks put it, “much of the essence of building a program is in fact the debugging of the specification”[^9].

4. Hidden Essential Complexity: The Trap of Tribal Knowledge

Brooks wrote that “the hardest single part of building a software system is deciding precisely what to build”. If that’s true, how have human teams ever built working software from imperfect requirements documents?

They have done it by quietly filling the gaps. Wiegers and Beatty describe an organization whose managers simply assumed that written requirements captured only 80 to 90% of the working software, and trusted their teams to close the rest as they went[^3].

Teams get through that 10 – 20% gap thanks to what I’ll call hidden essential complexity: the huge reservoir of tribal knowledge, domain conventions and plain common sense that product owners, stakeholders and developers carry in their heads and never think to write down. Wiegers and Beatty name the culprits. Assumed requirements are the ones people expect without ever saying them. Implied requirements follow from other requirements but are never stated. And, as they dryly note, developers can’t implement functionality they don’t know about[^3].

I’ve seen this from the other side of the table more times than I can count. When I joined a requirements discussion and asked clarifying questions, I would often hear, “Oh, we discussed that in the last meeting.” Everyone knew I hadn’t been in that meeting. Or, “It’s in the document,” and the document turned out never to have been shared, sometimes because it lived in someone’s personal folder. Nobody was being difficult. People genuinely assumed that everyone else knew what they knew and had read what they had read. I’m quite sure I’ve done the same to others.

Psychologists call this the curse of knowledge: once you know something, it is remarkably hard to imagine not knowing it. In a well-known experiment at Stanford, volunteers tapped out the rhythm of familiar songs such as “Happy Birthday” and predicted that listeners would recognize about half of them. The listeners recognized three out of 120[^13]. The tune played loud and clear in the tappers’ heads, while the listeners heard only knocking on a table. Requirements suffer from the same curse. To the person who holds the knowledge, the missing parts don’t feel missing at all. They are obvious, and *everyone knows that

Some examples:

The banking login. Ask a team to build a screen that shows a customer’s account balance, and nobody writes, “the user must log in first, and the balance must come from a real database, not a hardcoded number.” Every first-day intern already knows that.
The invisible back end of an ATM. A use case describes what the user sees. The person at the ATM knows nothing about the conversation between the machine and the bank’s computers, but the developer has to build it[^3].
Preconditions and postconditions. The ATM has to contain money before you can withdraw any. Afterward, your account has to be debited by the amount plus any fees. No user will mention these housekeeping details to a business analyst, yet developers and testers need every one of them[^3].

The Onboarding Paradox

On top of general common sense, every company has its own layer of tribal knowledge: proprietary domain rules, legacy quirks, security conventions, architectural habits.

Think about a new developer joining a team. Nobody expects them to sit down on Monday and build a complex subsystem from a 500-word prompt. Onboarding takes real time, and as Sam Newman points out, it is often the busiest developers who have to spend that time getting newcomers up to speed[^14]. The new hire spends weeks or months reading internal docs, pair programming and picking up how things are really done. Even long after that, when a requirement is unclear, they walk over to the team guru and ask. When developers are cut off from that loop, as on poorly integrated offshore projects, Wiegers and Beatty observe that costs rise because of extra iterations to close the gaps left by unstated, assumed and implied requirements[^3].

Now consider how we treat AI. It works in a complete vacuum. It has never been onboarded, never shadowed your senior architect, never had coffee with your domain expert. We give an intern a laptop, a buddy and three months. We give the AI a chat box and expect a senior engineer by lunchtime.

So what does the AI actually know? Foundation models are trained on huge amounts of public web data. Huyen describes them as “aggregations of the opinions of the masses”[^4]. Whenever a prompt leaves something out, the model fills the gap with the statistical average of the internet rather than your company’s knowledge. When instructions are unclear, models have to guess, and guesses can be wrong[^4]. Ask a generic model to write the technical specification for your project and you’ll get something plausible and generic, missing exactly the details your team would need.

It gets worse. Once a model makes a wrong assumption, such as stubbing out the database because thousands of online tutorials do, it tends to treat its own output as established fact and build on it. Researchers at DeepMind (Ortega et al.) describe this as self-delusion: a slightly odd start snowballs into confidently wrong conclusions[^4][^15]. The mock becomes the architecture, the architecture gets tests, and the tests pass.

5. Looking Ahead: The New Mandate

It’s worth remembering where we are: at the very start of the AI adoption curve in software engineering.

Today we mostly work with general-purpose models through text prompts and context windows. As AI engineering matures, we will almost certainly get proper “AI onboarding”: context pipelines, persistent project memory and domain-tuned models that can absorb an organization’s institutional knowledge and standards.

Until then, the model works in a vacuum, and we can’t count on anyone to quietly bridge the gap between a vague idea and a working system with tribal knowledge. For AI development today, we have to write down far more than we ever did. That means surfacing the unspoken assumptions, resolving the contradictions and validating the requirements rigorously before a single token of code is generated.

The “Make It Good” button still doesn’t exist. But it turns out you can build something close to it, as long as you do the engineering first.

In Article 2, we’ll look at how: moving from the statistical average of “vibe prompts” to formal, validated requirements engineering for the AI era.

Notes

[^1]: Karpathy, Andrej. 2025. Post on X, February 2, 2025. https://x.com/karpathy/status/1886192184808149383

[^2]: Leffingwell, Dean, and Don Widrig. 2003. Managing Software Requirements: A Use Case Approach, 2nd ed. Boston, MA: Addison-Wesley.

[^3]: Wiegers, Karl E., and Joy Beatty. 2013. Software Requirements, 3rd ed. Redmond, WA: Microsoft Press.

[^4]: Huyen, Chip. 2025. AI Engineering: Building Applications with Foundation Models. Sebastopol, CA: O’Reilly Media. The prompt-cache technique she discusses comes from Gim, In, et al. 2023. “Prompt Cache: Modular Attention Reuse for Low-Latency Inference.” arXiv:2311.04934.

[^5]: Richards, Mark, and Neal Ford. 2025. Fundamentals of Software Architecture: An Engineering Approach, 2nd ed. Sebastopol, CA: O’Reilly Media.

[^6]: Foote, Brian, and Joseph Yoder. 1997. “Big Ball of Mud.” Fourth Conference on Pattern Languages of Programs (PLoP ’97), Monticello, IL. Cited in Richards and Ford 2025.

[^7]: As cited in Wiegers and Beatty 2013: Davis, Alan M. 2005. Just Enough Requirements Management. New York: Dorset House. Shull, Forrest, et al. 2002. “What We Have Learned About Fighting Defects.” Proceedings of the Eighth IEEE Symposium on Software Metrics, 249258. U.S. General Accounting Office. 2004. Stronger Management Practices Are Needed to Improve DOD’s Software-Intensive Weapon Acquisitions (GAO-04-393). Leffingwell, Dean. 1997. “Calculating the Return on Investment from More Effective Requirements Management.” American Programmer 10(4): 1316. Kelly, John C., Joseph S. Sherif, and Jonathan Hops. 1992. “An Analysis of Defect Densities Found During Software Inspections.” Journal of Systems and Software 17(2): 111117. See also Boehm, Barry W., and Philip N. Papaccio. 1988. “Understanding and Controlling Software Costs.” IEEE Transactions on Software Engineering 14(10): 14621477.

[^8]: Davis, Alan M. 1993. Software Requirements: Objects, Functions, and States, rev. ed. Englewood Cliffs, NJ: Prentice Hall. Cited in Leffingwell and Widrig 2003.

[^9]: Brooks, Frederick P., Jr. 1986. “No Silver Bullet: Essence and Accident in Software Engineering.” In Information Processing 86: Proceedings of the IFIP Tenth World Computer Congress, ed. H.-J. Kugler, 10691076. Amsterdam: Elsevier Science (North-Holland). Reprinted in IEEE Computer 20(4), April 1987, and in The Mythical Man-Month, Anniversary ed., Addison-Wesley, 1995.

[^10]: Parnas, David L. 1985. “Software Aspects of Strategic Defense Systems.” Communications of the ACM 28(12): 13261335.

[^11]: Deniz, Begum Karaci, Chandra Gnanasambandam, Martin Harrysson, Alharith Hussin, and Shivam Srivastava. 2023. “Unleashing Developer Productivity with Generative AI.” McKinsey & Company, June 27, 2023. Also cited in Huyen 2025.

[^12]: Becker, Joel, Nate Rush, Elizabeth Barnes, and David Rein. 2025. “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity.” METR. arXiv:2507.09089.

[^13]: Newton, Elizabeth L. 1990. “The Rocky Road from Actions to Intentions.” PhD diss., Stanford University. Popularized in Heath, Chip, and Dan Heath. 2007. Made to Stick. New York: Random House. On the bias itself, see Camerer, Colin, George Loewenstein, and Martin Weber. 1989. “The Curse of Knowledge in Economic Settings: An Experimental Analysis.” Journal of Political Economy 97(5): 1232–1254.

[^14]: Newman, Sam. 2021. Building Microservices: Designing Fine-Grained Systems, 2nd ed. Sebastopol, CA: O’Reilly Media.

[^15]: Ortega, Pedro A., et al. 2021. “Shaking the Foundations: Delusions in Sequence Models for Interaction and Control.” arXiv:2110.10819. Cited in Huyen 2025.