That said, I believe that it's already too late. "Entrepreneurs" racing to beat each other to some pot of gold (real or imaginary) have created software with destructive potential (already demonstrated) and which is already "out of the bottle" and could proliferate like ... milk. You only need a cow, and you're in the home milk business, presumably unregulated. Want to issue medical, investing or psychological advice? Set up your own chatbot.
> Top executives at Anthropic, OpenAI and other AI companies are privately gaming out scenarios for a public and political revolt after a catastrophic AI event.[0]
As for Microsoft, they always had the financial reserves to ride out their many blunders. Here are the top FIFTY [1]. The rest of us don't and any one piece of bad advice or continued conversation could be disastrous or fatal.
We're all somewhat lower in status than beta testers, or even guinea pigs. These are experiments, less even the code of conduct that medical and psychological trials demand.
[0] https://news.ycombinator.com/item?id=50047049
https://www.yahoo.com/news/politics/articles/scoop-ai-compan...
[1] https://msftnewsnow.com/the-top-50-biggest-microsoft-flops-o...
[2] https://www.sfgate.com/tech/article/chatgpt-alabama-suicide-...
Milk is consumed almost everywhere in copious amounts, if it isn't produced properly it can create significant illness and potentially death in some. Washing machines can also kill when pets get put in them, or children climb themselves into them.
The reason they are regulated is based on decades of safety incidents.
AI hasn't reached that yet, or to the general public. Yes its has potential, but we just don't have the actual incidents occurring to require it.
But I do like his statement of "Let's start today" which I read that as, lets do it before we realised we should've, which is a good thing.