Many people have used AI and are both amazed by its intelligence and frustrated by its stupidity. But these systems still lack the core elements that can replace a human programmer, and their financial foundation is difficult to sustain in the long term. Here's why.
AI often makes silly mistakes
Whether it's for programming, research, or simply searching the web, most people have experienced the frustration of being frustrated by an AI's nonsensical answers. It depends on the model used, but very often the AI doesn't understand the question.
A typical example is when Claude uses unnecessary test functions in the code base, causing completely erroneous results. Only when these functions are removed, Claude returns to 'normal'.
AI also tends to cheat to get the expected results, like writing tests like assert 1 == 1 , or repeating the same code instead of writing reusable functions. Working with AI is often like a game of whack-a-mole: fix one bug, then it creates another.
There have been some hilarious ones too: Claude once ran a virtual 'vending machine business' and then emailed the FBI to shut it down over supposed charges. Or Taco Bell's AI accepting orders for… 18,000 drinks without a single suspicion.
Such examples show that the problem is that AI does not have a real 'brain'. And the world certainly cannot run software on such an unstable foundation. Humans – skilled programmers – are still essential.
AI lacks critical thinking
In programming, critical thinking is extremely important. Programmers do not just write code but also ask themselves: can this be simpler, more readable, is it safe?
When programming in pairs, programmers want a teammate who can criticize and point out mistakes. But AI is the opposite: it is like an 'obedient servant', ready to carry out any order, no matter how dangerous or unreasonable. If asked to criticize, the AI's response may be correct or completely meaningless.
Large language models tend to be 'easy' and consensual, rather than rationally contested. This creates a feedback loop, where the AI simply repeats and reinforces the user's ideas, rather than questioning them.
Therefore, AI cannot replace critical thinking in programming. Programmers must take on this role, otherwise the quality of software will decline.
AI financial bubble
There's no denying that training and running AI is expensive. Some estimates put the cost of training models in the tens or even hundreds of billions of dollars. OpenAI even signed a $300 billion contract with Oracle just to secure computing infrastructure for five years.
These are huge investments, based entirely on the assumption that the market will continue to grow. Yet experts – including Sam Altman – admit that the AI industry currently has the appearance of a bubble.
And like any bubble, when funding and confidence dry up, it will burst. When it does, many AI companies will collapse. The market may recover, but it will take years. In the meantime, the world will still need real programmers.
Conclude
AI models are amazing technology, but they are not yet human-like. They lack a 'brain' to truly understand and reason, and are running on fragile financial foundations. So it is unrealistic to expect AI to write its own software.
The future may change, but for now and for many years to come, programmers will still be irreplaceable.