LLMs, AI, and the Sunk Cost Fallacy of Knowledge Acquisition in 2025

This will be a continuation, about one year later, of my previous blog post "LLMs, AI and Software Engineering." In a quick summary, in that blog post I argued that LLMs will eventually write most of the code, and engineers will still be the main people driving them, under a new, unspecified at that time, programming language.

One year later, and I think we have started to see glimpses of this new paradigm. OpenAI with its Codex^[1], Anthropic with Claude Code, the open-source community with opencode, and the list goes on^[2]. All these offer a new way of not only writing code but also thinking about it. With the advent of context engineering, few-shot prompting, and other techniques, we are witnessing in real time a new way to come up with algorithms^[3][4].

These model harnesses are becoming more relevant, providing improvements in LLM benchmarks, to a point where factuality (or LLM hallucinations) has been proven to be greatly reduced by tool use^[5]. This is coherent with the previous theory on the need for physicality for AI: the ability to interact with its environment, when coupled with in-context learning, makes for a powerful reasoning machine--something we have never witnessed outside of humans. The scientific method can finally be conducted by something other than us.

All these breakthroughs and their effect on industry have shown that a great portion of us, technologists, are in fact not always on the side of technology.

While many think that such technology will open up a new door for human scientific endeavour, others see this as a cheap, unreasonable way to reduce our ability to understand the world (or our software).

Make no mistake, however: they are not wrong.

LLMs (with their harnesses), if they manage to maintain this level of year on year improvement, will soon offer an abstraction layer for all symbolic manipulation.

The consequences of such abstraction layer on the scientific world are honestly beyond anything I can imagine.

Yet an abstraction, by definition, acts as a veil on the underlying mechanisms that explain how reality is being manipulated. Yet I argue that this is beneficial, and all of human progress comes down to inventing new abstractions.

The History of Human Abstraction

I have previously talked about the original resistance to early compilers, and mentioned Luddites, but failed to explain that we have repeatedly, across the history of humanity, abstracted foundational skills that used to define who we are.

Agriculture abstracted away hunting and gathering, a skill so foundational that humans from that era were named after it.

Gunpowder abstracted away physical combat prowess; swords and shields became toys in less than a century.

The Industrial Revolution abstracted away human physical effort, another defining feature that civilisations were built upon.

Yet, we are still looking for methods to make food security more economical and ethical. We are still looking for ways to improve on physical combat prowess through different tournaments. We are still looking for new physics to replace the energy that we built the Industrial Revolution on. Today's energy production has evolved beyond anything our ancestors imagined, and still receives constant innovation--as do agriculture and martial arts.

Why?

Humans Are Visionaries

My argument is that we were wrong. We didn't change as we have progressed in the tech tree; we in fact were always the same: visionaries.

In 2025, we are mainly defined by our cognitive prowess. IQ discussions as well as knowledge acquisition are the main drivers of socioeconomic evolution. This will not last for long. I believe that in less than five years, the run-of-the-mill LLM will be smarter than any human that has ever lived. We are not very far from this, and this feeling is spreading through society. Faced with this fear, people who invested most to secure themselves in the era of cognitive prowess - engineers and researchers - are those with the most to lose, and ironically are the people spreading this technology. This violent phase change makes consistent effort across a decade feel like a mistake. When a computational system can find novel mathematical theorems and new chemical compounds, suddenly the time we spent on learning a framework loses a lot of its economic and societal value.

The natural human reaction of denial made our brightest engineers dismiss one of the highest technological levers of our time, but that's the biggest misconception. AI, or more generally Automated Cognitive Ability, will arguably receive more effort on improving it than we did with all the breakthrough technologies. The real power of humans was never cognition; it was vision.

Humans are visionaries.
Not hunter-gatherers.
Not warriors.
Not coal-shoveling mechanics.
Not engineers.
Humans are visionaries.

Our most powerful ability isn't that we are able to do what we do; it is that we want to do what we do. Abstractions open up a novel application layer for our ambitions, and AI is just that.

Conclusion

Don't spend most of your time in a sunk cost fallacy mindset to prove your worth^[6]. Look inside, think about what really drives you, and know that the new frontier is there. You developed discipline and imagination to develop your cognitive skills, and now you will do the same for your vision.