I don't like LLMs

I have a lot of mixed feelings about AI and LLM technology. I'm fascinated by its effect on our profession, excited by the potential gains in productivity—and thus the products we could rapidly build. On the other hand, I'm fearful of the damage AI might cause: agent swarms taking over our virtual and physical infrastructure, designing bio weapons. But, back on my first hand, LLMs might also design miracle cures, and come up with clever ways to raise our prosperity. Fundamentally I don't think we have a choice about riding on the AI technology train. It's a wild ride and I just hope we'll figure out how to steer it before it derails us.

The allure of these models is undeniable. In software engineering, the ability to generate boilerplate code, refactor legacy systems, or even architect complex microservices in minutes is a seductive proposition. It promises an end to the drudgery of repetitive tasks, freeing up human creativity for the high-level design and strategic thinking that actually drives innovation. We are standing on the precipice of a productivity explosion where the barrier to building sophisticated applications drops precipitously. For the small team or the solo developer, this could be the catalyst for scaling ideas that were previously impossible to execute.

However, the shadow side of this technological moonshot is equally real and perhaps more dangerous. The same neural networks that can optimize a supply chain or write a poem can be weaponized to generate disinformation, automate cyberattacks, or create convincing deepfakes that erode the very fabric of trust in our society. The fear of autonomous agents acting without human oversight is not science fiction; it is a tangible risk given how quickly these systems are evolving. If we lose the ability to audit, understand, or control the code these models produce, we risk handing over the keys to our digital infrastructure to entities we cannot fully comprehend.

We must also consider the unintended consequences of relying too heavily on probabilistic outputs. Software engineering has always been about correctness, reliability, and maintainability. Yet, LLMs often prioritize plausibility over truth, generating code that looks perfect but contains subtle, hidden flaws that only manifest under specific conditions. This shifts the burden of quality assurance from the writing phase to an increasingly difficult debugging phase, potentially creating a new class of technical debt that is incredibly hard to trace or fix. We cannot simply outsource our judgment to a black box without developing the critical skills to verify its output.

The path forward requires a nuanced approach that embraces the power of AI while maintaining rigorous human oversight. We need to treat these models not as oracles of wisdom, but as powerful, sometimes erratic, assistants that require constant supervision. Education in computer science must evolve to include literacy in AI limitations, ethical considerations, and the mathematics of probabilistic reasoning. We must build guardrails and alignment mechanisms that ensure these systems serve humanity's best interests rather than amplifying our worst impulses.

Ultimately, the story of artificial intelligence is a story of human agency in the face of unprecedented change. We are not passive observers; we are the architects of this new reality. The challenge ahead is to harness the miraculous potential of these tools to solve the grand problems of our time while vigilantly guarding against the risks that threaten to undermine our safety and stability. It is a delicate balance, but one we must strive to maintain as we navigate this new and uncertain horizon.