The biggest risk in AI is centralized power. As much as the EAs and rationalists want to convince everyone that an AI can convert all of us into paperclips, that we’re creating something that will treat us like ants or 7-year-old kids, that’s just thinkism, as Kevin Kelly says. The real risk is centralizing the power of such a tool to either a few companies or governments.
There are two ways we can end up with centralized control: governments or companies. On the government front, it’s not a win yet. All kinds of forces are trying to get AI regulation passed: politicians looking for more power, companies looking for regulatory capture, and even the Pope now with his latest encyclical. I think he nailed the centralization risk in companies without realizing there’s the same one when you regulate them. Actually, government centralization is worse, at least companies have the government watching over them, but who watches the watchers?
On the other front, things are looking as good as we could hope. We have all kinds of models and companies competing: SOTA models pushing the limits every other week, open weights models letting users run private and local model AI without asking for permission, American models, Chinese models, we even have a European one. Intelligence is becoming more and more like a commodity, and the app layer seems to be what’s important.
As long as models are swappable and users have the possibility to exit and switch, that risk is completely mitigated. That kind of environment leads to competition, and competition forces companies to behave well. If they misbehave, they start to lose users fast.
So now we only need to worry about the government side, and even there things are looking better than they could. Regulation happens country by country. If one country decides to lock everything down, the rest of the world still has access to unregulated models. That creates a natural incentive against over-regulation, no country wants to handicap itself while everyone else races ahead. This is already happening; China has, ironically, open sourced its models to try and damage US companies’ revenue, giving access to unrestricted models to users around the world.
All the right incentives are starting to align. The future of AI looks less like 1984 and more like the internet: messy, distributed, and impossible to control. The kind of environment that lets a thousand models bloom.
natural selection is happening in real time in AI. China is at the forefront of making this happen. The AI that will survive this mess will be the local AI on your computer and on your phone. We all know how regulations work. The man with the biggest pockets always wins. I think they call it lobbying and political donations. But what we have now is the open source community. We can turn this all on its head just keep doing what we’re doing. And as long as local models keep getting pumped out. The overzealous control mongers will bury themselves in control of AI as the local becomes your laptop and phone. The greatest AI engineers don’t work for a fortune 500 company, they work from there basement. Let’s help them build our open source moat by supporting the people on GitHub and everywhere that person would live on the Internet. And if you start a company, make sure it helps build the moat for open source because that’s the future we want.