AI isn't perfect, but you may still be underestimating it

Introduction

I saw a Reddit comment today where someone said the following:

That's not my point : no matter how smart these tools become, these are just tools. There's just no way my product manager will replace me with claude. No matter how hard he tries not matter how smart Claude will be in 5 years. Because there will still need a craftsman to handle the tool. Llm are just intellisense on steroid : it provide a base to work with. Not a final product.

Sure Claude will be able to handle larger code-base in the futur and plan large refactoring, yet the craftsmanship to know when and how will still and always elude the model.

3d models are exactly the same. Sure AI will improve and provide better and better models and sure you'll have less and less grunt work to-do. Yet you'll still have to correctly define how to integrate the model in the end-product.

Yeah our jobs are changing at an always increasing pace. Yeah, there will be job compression. But it's not as if it never happened before. 3d modeling software improvements have already improved your workflow hence your speed at delivering a task which implied already head-count drop. The new tool is just like the previous generation tools on steroids. It boost significantly the productivity... That won't make me a good guy a creating a good 3d models.

I believe the thesis of this Redditor's comment is: "AI is a tool, and you will always need someone who knows how to use the tool - Therefore you will need developers." I think there's a couple obvious assumptions bundled in with this:

It may be a category mistake to call AI a tool in the same sense that a hammer is a tool.
The Redditor posits that tools need craftsmen, therefore his job will be safe since a product manager will never be able to use this tool. I see no basis for this assumption. Technology shifts the "operator" of tools to new roles all the time. What happened to all the typesetters?

What I will argue is that not only is it likely that the role operating AI will change over time, but that it is a category mistake to call AI a tool at all.

Why program?

Something I've seen a lot of programmers specifically say is that AI is not going to replace their jobs because the code it creates constantly requires fixing. To address this idea, I think we need to step back for a moment and ask ourselves Why do we even program to begin with? It seems very easy to be so deep in the trenches on the subject that fascinates you that you begin missing the forest for the trees, or perhaps the forest you exist in doesn't map onto the business world. Your employer does not ultimately pay you for code. They pay you because some business requirement currently happens to be best satisfied by having a human write code. Cleaner abstractions, fewer instructions, better architecture, and elegant implementations matter insofar as they improve cost, reliability, maintainability, performance, or some other outcome the business cares about.

Where I think the argument falls apart for this idea that traditional programmers will be required is that AI generated code being ugly, occasionally wrong, and requiring correction does not actually predict whether businesses will still choose programmers. The business question is which process is sufficiently cheaper, faster, and reliable. The real question to answer isn't "do we still need people to deeply understand programming", it's "do business requirements require someone to deeply understand programming."

AI, a tool?

This Redditor's argument assumes that AI is a tool and tools require operators. To that end, I think this is a category mistake. When we talk about tools, I think most people are thinking about a thing they can deliberately operate to accomplish some task. The tool may amplify force, automate a procedure, or transform an input, but the operator generally remains the source of the goal, the plan, and the next action. With a bandsaw, you simply flick a switch and it turns on.

AI diverges significantly in a couple ways. First is that Traditional tools generally have a deterministic relationship between operation and result. Given the same state and the same operation, we expect the same result. AI systems are different: the same request can produce materially different proposed solutions, plans, and actions.

The second is that AI can originate ideas and actions. In fact it's more apt to describe an LLM as an operator of tools, closer to a human, rather than as a tool itself. One objection is that AI does not truly originate ideas; it only synthesizes from its training data and current inputs. But that distinction is less obvious than it first appears. Human ideas are also synthesized from prior experience, memory, observation, and existing concepts. If synthesis disqualifies something from originating an idea, then it is not clear that humans originate ideas either.

More importantly, the economic question does not depend on resolving that philosophical issue. AI can generate novel candidate actions, choose among them, and execute a sequence that was not explicitly specified by the user. Whether we call that “origination,” “synthesis,” or something else, it is operationally very different from a bandsaw.