AI Tracker monitors cutting-edge developments in advanced AI, to help researchers and policy specialists better understand the AI risk landscape.
Since the public release of GPT-3 in mid-2020, AI has entered an era of foundation models, scaling and general-purpose algorithms. As AI systems become increasingly capable, they create new risks to public safety that need to be monitored. These include accident risk, as well as risks borne of new malicious applications that were previously impossible.
Confronting and Overcoming the Risks of Powerful AI
Anthropic is an American company behind the artificial intelligence (AI) Claudeâwhich belongs to the very top tier of models, alongside the more famous GPT from OpenAI. Its investors include Amazon, Google, Microsoft, and NVIDIA. It is the third most valuable private company in the worldâcurrently worth over $350 billion. In January, Dario AmodeiâAnthropicâs founder and CEOâpublished an essay titled: "The Adolescence of Technology: Confronting and Overcoming the Risks of Powerful AI."
In it, he shares concerns regarding the development of AI whose capabilities surpass those of leading experts in many fields. And this is not about the distant future: there is "a very strong chance it comes in the next few [years]."
Why does he think so? Models learn according to predictable laws. They are already automating parts of the work within the companies that create them. The finale will be a loop in which the model works on the next generation itselfâwithout human involvement. As one Anthropic engineer put it:
"We want Claude n to build Claude n+1, so we can go home and knit sweaters."
Companies possess enough computing power to put hundreds of thousands of such AIs to work simultaneously.
Amodei lists four main threats: AI autonomy, misuse by small groups, misuse by states (surveillance, propaganda, weaponry), and economic shock.
I am not saying these predictions should be taken literallyâI disagree with many of them myself; for instance, in my opinion, he underestimates the risk associated with autonomy and overestimates his chances of handling that problem. However, I advise taking them seriously, and taking the stated intentions literally.
I'm glad Dario published this essay.
Counterarguments to Penrose's position on AI. Also, I recommend this short excerpt from interview with Scott Aaronson in regards to his claims about consciousness.
Aaronson's "The Gödel CAPTCHA"
Notes from Aaronson's lecture about the argument.
Turing countering this sort of argument already in his 1950 paper:
