Ask ten people what “AI” means and you’ll get ten different answers, and most of them will be wrong in the same specific way. They’ll describe a chatbot. That collapse, treating “AI” as a synonym for “large language model,” is doing more damage to how we think about this technology than any actual limitation of the technology itself.

What an LLM Actually Is

AI is not one thing. It’s an umbrella term for algorithms that can learn from data and improve at a task without being explicitly programmed for every case. Under that umbrella you’ve got systems that generate images, systems that decode which patterns of brain activity correspond to which physical movements, and, yes, large language models. An LLM does one very specific thing. Given some input, it generates the output most statistically likely to follow it. That’s it. It’s extraordinarily sophisticated text prediction, not a general reasoning engine, not a decision-maker, not a scheduler, not a moderator.

Part of why that’s so hard for people to internalize is that the chat interface is working exactly as designed. An LLM’s whole purpose is to mimic human language convincingly, and when something responds to you in fluent first person prose, your brain reflexively treats it as a mind on the other end, something that understands, intends, decides. That’s not a glitch people are falling for. It’s the actual design goal being achieved. Knowing intellectually that it’s next token prediction doesn’t stop the interface from feeling like it’s talking to you, because that feeling is the product. I come back to Hanlon’s razor a lot with AI: never attribute to malice what’s adequately explained by stupidity. I’d actually split it here. Some of the confusion is stupidity, sure. But a much bigger share of it is plain ignorance, and that ignorance is being actively cultivated. Companies benefit from people not looking too closely at what these systems actually are, because “it’s basically thinking” sells a lot better than “it’s a very good autocomplete,” and there’s very little incentive anywhere in the industry to correct that impression for you.

That same false confidence is exactly why hallucination isn’t some separate malfunction bolted onto normal operation. It’s the same mechanism doing precisely what it always does. There’s no internal check going off that says “I don’t actually know this,” because there’s no ground truth being consulted at all, just probability. Fluent, confident output gets generated whether the underlying claim is true or completely invented, and the model has no way to tell you which one you’re getting. That’s the real landmine. It’s not that the model is sometimes wrong. It’s that it’s equally confident when it’s wrong as when it’s right, and that confidence is the exact same quality people already mistake for understanding.

That confidence is also what makes companies comfortable putting LLMs where they don’t belong. If a system sounds like it knows what it’s doing, it’s an easy sell to management as ready for the job. So companies that don’t have a good idea take a random product, staple “AI” onto it, and market it as the next wave of whatever they’ve stapled it to, buzzwords included. Worse, they try to force LLMs into roles they were never built for: managing calendars, moderating content, running hiring screens, anything that actually needs deterministic, reliable behavior. A traditional rules based system fails loudly and reproducibly, you get a bug report and you can fix it. An LLM wrapped around that same decision fails quietly and inconsistently instead. The same input can get different treatment on different runs, and there’s no clean, repeatable bug to file, just a general sense that the system is unreliable in ways nobody can quite pin down. That’s a fundamentally different failure mode to traditional software, and it’s exactly why bolting an LLM onto a role that demands consistency is worse than it sounds on paper.

Tribalism, Not Technology

That same misunderstanding doesn’t just get exploited by companies, it drives individuals too, and here I think a lot of it comes down to tribalism dressed up as a technical opinion. A lot of people hear “AI” and immediately assume it must be good, and flock toward it, and I think a decent chunk of that is virtue signaling. If you’ve decided you’re a proponent of AI, there’s a quiet pressure to use every AI product that comes out, because not using product number n starts to feel like a crack in your identity as a supporter. The opposite happens on the other side too. Some people refuse to touch AI at all, on principle, and while I think that stance is easier to justify and often more reasonable than the reflexive embrace, it’s still frequently rooted in the same kind of misunderstanding we’ve been talking about this whole time, just pointed in the other direction. To be fair, some of that refusal is a genuine, considered objection to what’s happening around AI rather than to the technology itself, and I respect that position completely. But I don’t think the answer to bad AI deployment is blanket refusal, and I especially don’t think it’s belittling anyone who uses AI at all. That just makes the person doing the belittling look unreasonable, and it actively damages the legitimate arguments that are actually available, several of which I’m making in this very post.

Omarchy is a good example of this whole dynamic in miniature. Before I go further, I want to set aside its creator, DHH, entirely. Whatever anyone thinks of him is a separate conversation, though it’s worth flagging that there’s a parallel tribalism around him specifically that compounds everything I’m about to say. Omarchy is an agentic operating system built on Arch, and one of its headline features is having LLMs baked directly into the system. I find that genuinely concerning, because it’s threading a non-deterministic system into one of the few pieces of software that should be as deterministic as possible. And you can watch the tribalism play out exactly as you’d predict. The pro-AI crowd flocks to it because it has AI baked in, full stop, regardless of whether that integration actually benefits anyone. The anti-AI crowd rejects it for the same reason in reverse, regardless of whether it might. My honest opinion is that it hurts more than it helps, but I’ll admit some people do seem to get genuine value from having that kind of AI integration woven into their daily use, and that might just be another instance of the same misunderstanding we’ve covered already rather than a real benefit. I’ll leave that aside. The point is that Omarchy’s popularity comes overwhelmingly from two things, its agentic integration and DHH himself, and neither of those is a technical merit. It’s a near perfect example of tribalism alone being able to launch a product into popularity independent of whether the product is actually good. Whatever you end up thinking about Omarchy, that opinion should come from actually forming an informed view of it, not from which side you’ve already picked, and a huge amount of the discourse around AI right now is exactly that, a side already picked, working backward to a justification.

The Bubble

That same tribal energy is a big part of what inflates the bubble in the first place. When the underlying tech is genuinely impressive and also widely misunderstood, and people are already sorting themselves into camps rather than evaluating anything on its merits, it becomes very easy to raise money on vibes alone. The parallels to the dot-com era aren’t subtle. Companies are being built on nothing but “we do AI,” valuations are inflated by other AI companies investing in each other, and VCs keep piling in because everyone else is piling in. NVIDIA is maybe the clearest single example of how circular this has gotten. They manufacture the hardware that AI companies buy up at scale, which makes NVIDIA money, and NVIDIA then turns around and invests some of that money back into those same AI companies, which gives them more capital to buy more NVIDIA hardware. That’s one company, and there are dozens of other investment relationships layered on top of it, all pointing back into the same closed loop. It won’t last forever. Eventually a lot of this will come crashing down. But this is the part the “AI is fake and will vanish” crowd gets wrong: the dot-com crash didn’t kill the internet. It killed the companies that had no real product underneath the hype, and the survivors ended up defining what the internet actually became. The same thing will probably happen here. Whatever AI looks like in ten years will likely be shaped by whoever’s left standing after this shakes out, not by the current noise. LLMs aren’t the revolutionary end state some people claim, they’re just the thing that happened to drag machine learning into the public spotlight, after decades of it already being quietly useful elsewhere. They’re not going to “take over” anything, because the moment a system starts doing something other than modeling language, it stops being an LLM.

Programmers vs. Coders

Nowhere is the gap between what people think an LLM is and what it actually does more visible, or more consequential, than in programming.

The claim that AI will replace programmers only makes sense if you think a programmer’s job is to write code. It isn’t. Anyone can write code. That was never the hard part. The actual job is deciding what the code should be: designing the architecture, structuring the system, and being able to explain why something was built the way it was, not just that it works. Writing code is how a programmer implements those decisions. It’s the output, not the job.

AI is genuinely good at accelerating that output, and I use it myself, regularly. What it can’t currently do, and won’t be able to do as long as it’s fundamentally an LLM, is make the underlying decisions. That takes a meta-understanding of the situation: the possible approaches, their tradeoffs, why one beats another here. That’s reasoning about a system, not predicting the next likely token. An LLM doesn’t have a stake in the decision, it doesn’t have a decision at all. It has a probability distribution.

So here’s the test I’d hold any AI-assisted code to, and it’s not actually a new standard, it’s the standard that should already apply to all code: if you can’t explain why the code does what it does, the same way you’d explain code you wrote yourself, it shouldn’t be in production. Not because AI wrote it. Because code you don’t understand is code you can’t debug, and code you can’t take responsibility for. Someone who tells an LLM to build a whole system with no architectural constraints, doesn’t review the output, and can’t explain how it works afterward, isn’t programming, whatever they call it. “Vibe-coding” is actually a pretty precise name for what that is. It’s coding, not programming, and the two are not the same job even though one usually contains the other.

What worries me more is how far that habit has already spread, including among people who should know better. There are genuinely skilled, experienced programmers who have fully switched over to just prompting and shipping without reviewing what comes back, and a decent chunk of them have a financial reason to insist that this is fine. If your income now comes from YouTube videos, courses, or sponsorships built around the message that AI has made traditional programming obsolete, or from a relationship with an AI company that benefits from that narrative being widely believed, you have a direct incentive to keep saying it, regardless of whether it’s actually true. Their real technical credibility becomes borrowed authority for a claim that their own earlier definition of the job would have rejected outright. That’s a genuinely different problem from a beginner vibe-coding a weekend project. It’s a skilled person using their name to launder an unreviewed shortcut into something that sounds like a legitimate methodology.

The same pattern shows up structurally too, not just individually. Inside companies, engineers and leadership alike are incentivized to report productivity gains from AI tooling, because those numbers justify valuations and investor narratives. That pressure pushes people to overstate results, or to quietly stop reviewing output as carefully, because velocity is what gets measured and rewarded, not correctness. Nobody has to be lying for this to happen. The incentives just point away from scrutiny, the same way they do further up the chain with the funding and the hype.

And there’s a longer term cost underneath all of this that’s easy to miss. If junior developers are learning primarily by prompting instead of by actually working through why things break, you’re not only getting worse code today, you’re thinning out the pipeline of people who could ever become the programmer as opposed to the coder. The people who are supposed to grow into architects and decision makers are supposed to come from today’s juniors doing the slow, often frustrating work of actually understanding failure. Skip that step at scale and it’s not just a code quality problem anymore, it’s a structural one, and it won’t be obvious until the generation that skipped it is the one being asked to make the big decisions.

That’s the pattern running through all of this so far: a genuine capability gets misread as something bigger than it is, and the gap between the story and the reality quietly becomes someone else’s problem later. The environmental side of AI follows the exact same shape, real costs tangled up with misunderstood ones, except here the costs are physical instead of structural.

The Environmental Cost

The water criticism is a good example of the misunderstood part. A lot of the alarm around AI data centers treats their water use like other industrial processes, where water is consumed and gone. Most data center cooling doesn’t work that way. It’s typically thermal dissipation, not consumption, the water isn’t destroyed. I don’t think this comes from bad faith either, just from the same ignorance running through everything else here, not looking closely at how the systems actually work.

The real costs are elsewhere, and there are at least two worth taking seriously. The first is the obvious one: the physical footprint of these facilities and the electricity they draw, which in most countries still comes largely from non-renewable sources. The second gets talked about far less, and it’s the materials cost of actually building all this hardware in the first place. RAM and other components that AI infrastructure depends on require resource-intensive manufacturing processes that produce genuinely damaging byproducts, and that demand doesn’t stay contained to the AI industry. It puts strain on the entire supply chain and pushes prices up for everyone else who needs the same components. It’s also reshaping who these manufacturers even sell to. Micron is a good example: once the money in AI became obvious, they leaned into selling directly to AI companies at scale, a straightforward B2B pivot chasing where the demand and margin now sits. None of this is a new kind of problem though. It’s the same one every wave of industrial expansion has produced, just wearing an AI label. The answer isn’t a referendum on whether AI itself should exist, it’s regulation and a serious push toward sourcing and manufacturing this stuff more sustainably.

Where This Leaves Things

None of this makes me anti-AI, and none of it makes me a true believer either. It’s a tool. Like any tool, its value depends entirely on whether the person using it understands what it’s actually for. The mistake isn’t building with AI or being skeptical of AI, it’s skipping the step where you ask what, specifically, this system does, before you decide what to do with it. Most of the damage here, the hype, the panic, the bad deployments, the eroding standards, comes from skipping exactly that step, and a lot of people currently have very good reasons not to want you to take it.