Lately we’ve seen many incidents of AI models actively hacking systems or discovering real vulnerabilities in codebases, and that has triggered conversations from “everything is about to get hacked” to people extrapolating that we’re all going to end up as paperclips in the future (as if we humans have never solved a problem).
It’s obvious that AI is already superb at writing code and at finding or exploiting vulnerabilities, and it’s only getting better. The question is what this means for cybersecurity: does everything just get hacked and we’re doomed, or do we also use it for defense and end up balancing out?
My view is the boring one (latter), for two main reasons:
(All of this assumes AI keeps getting democratized in an open ecosystem, which seems almost inevitable now. The biggest risk will always be centralized control of such a powerful technology.)
AI is still a tool. The ultimate tool maybe, but still something that responds to prompts, and that won’t change until we get to AGI and it can actually disobey us (which I believe is still very far away and won’t come from LLMs, but that’s another topic). For now, AI is simply something humans use to augment their abilities, just like many powerful tools we’ve invented before.
AI is not a weapon. A lot of people compare AI to stuff like nuclear weapons, like if we’re birthing such a dangerous tool that unlike nukes, it is going to be available to everyone, so it only takes a few bad guys to hack everything or create bioweapons to kill us all, that couldn’t be further from the truth. Weapons’ only use case is destruction. AI can be destructive, but can also be just as good at defending, and it’ll also be used for many other things that’ll change the world for the better.
Because it is a promptable tool that is excellent at both attacking and defending, it’ll largely end up balancing itself out and we’ll end up in roughly the same place we are now, with a few exceptions on both sides.
By ‘roughly the same place’ I mean: we have always had competent and incompetent builders and competent and incompetent hackers. There have always been situations where the attackers were better than the defenders and won and situations where the opposite happened. That will keep happening, AI is just a tool that’ll supercharge their abilities. This applies to both existing code that can be updated reasonably quickly and to new code being written today.
The exception for the bad side and probably the biggest risk right now is in existing code that is hard or impossible to update. The Coldcard incident is a great example: attackers found a bug in the way seed phrases were created and stole over $100M. Even if the good guys had found it first, they would have needed to make it public so users could move funds, and with that being public, attackers would still have had a window to exploit it. These situations are the riskiest for users and companies as even if the good guys find the vulnerabilities, they can’t patch them with code, the only fix is to migrate every user or data using that specific system and coordinating such a big migration silently is almost impossible.
The exception for the good side is that we will arguably be in a better place for small teams. Before AI, a small team would face a larger group of hackers and be outnumbered, now they can run swarms of agents that audit their code constantly, thus making the playing field more equal.
The other major risk we face is just incompetence. If a company with updatable code doesn’t even bother to care about security (many such cases) or using AI to audit itself, then AI just helps the attackers which can now hit more targets in the same amount of time. Those companies are worse off than before.
On the other side: the cost of continuous AI auditing is low and the reward is extremely high. Builders have a first-mover advantage by being able to use these tools to audit their code before anyone can see it, and you should definitely throw agent swarms at your code before launch and on an ongoing basis afterward. Daily audits should become routine and the way for good guys to stay ahead.
As a user the best thing you can do is to diversify your risk as much as possible and remove single points of failure. Crypto is one of the most dangerous spaces right now because of its decentralized, open nature and the inability to just roll it back in the event of something bad happening (although I think most of it will do just fine and end up as the most anti-fragile system ever).
Mitigating that risk can mean using multisigs that diversify your risk between multiple wallet providers (selflessly shilling my own company: Oasis Vault, though there are other great options too), or split stablecoins deposits across multiple protocols you trust (for example Morpho and Aave) instead of using just one.
Elsewhere I think you just need to assume everything online will get leaked and be as aware as possible of which data you share online.
Also, don’t get hypnotized by marketing words like “air-gapped” or “open source.” Both properties can be valuable, but neither magically makes code unexploitable. Bad code in an air-gapped device still gets exploited (as we just saw with the Coldcard incident) and open-source just accelerates the finding of a vulnerability, hopefully from the good guys, but you can’t be sure.
What it’s true is that open-source software that doesn’t get hacked for a period of time is probably the most secure one out there, open-source lindy-ness is the best property when it comes to cybersecurity.
AI supercharges both sides, the overall equilibrium probably stays intact apart from the few exceptions that I mentioned before and that are worth being aware of. Everything else is just an acceleration of the same cat-and-mouse game we’ve always played, only now both the cats and the mice have much better tools. As long as we avoid the centralization of power we’ll do just fine.