Yudkowsky's Dilemma

Artificial Intelligence (AI) is developing rapidly, and companies like OpenAI are competing to build ever more powerful systems. A group of former and current OpenAI employees have voiced concern about what they describe as a culture of recklessness and secrecy at the company. According to Kevin Roose of the New…

Artificial Intelligence (AI) is developing rapidly, and companies like OpenAI are competing to build ever more powerful systems. A group of former and current OpenAI employees have voiced concern about what they describe as a culture of recklessness and secrecy at the company. According to Kevin Roose of the New York Times, the group says OpenAI is not doing enough to keep its AI systems from becoming dangerous.

These concerns are not unfounded. Eliezer Yudkowsky, a decision theorist, warns that «superintelligent Artificial Intelligence will probably kill us all». In his TED2023 talk, Yudkowsky raises the question of whether it is possible to build powerful artificial minds that are obedient and even benevolent. He stresses the need to act immediately to make sure that AI systems smarter than humans do not lead us to extinction.

The recent departures of prominent AI researchers at OpenAI, such as Ilya Sutskever and Jan Leike, have further heightened concerns about safety at the company. Leike, who led the team in charge of managing the risks of powerful AI models, said on his way out that «safety culture and processes had taken a back seat to shiny products».

Given these challenges, it is crucial that AI companies prioritize safety and transparency in developing this powerful technology. In an open letter, former and current OpenAI employees call for greater transparency and protections for whistleblowers, as well as an end to non-disparagement and non-disclosure agreements. These measures would allow employees to freely discuss the risks and potential harms of AI, without fear of retaliation.

Beyond corporate self-regulation, lawmakers need to regulate the industry to ensure a transparent and democratically accountable governance structure. A great dilemma. As Yudkowsky points out, «the question is not whether superintelligent AI will arrive, but whether we will be wise enough to survive its arrival».

The warnings from former and current OpenAI employees, together with the concerns voiced by experts such as Eliezer Yudkowsky, underscore the importance of developing safe and transparent Artificial Intelligence.

Only through collaboration among companies, lawmakers and society at large can we make sure that AI becomes a tool that benefits humanity, rather than an existential threat.