Former OpenAI Safety Researcher Says ‘Security Was Not Prioritized’

Former OpenAI security researcher Leopold Aschenbrenner says that safety practices on the firm had been “egregiously inadequate.” In a video interview with Dwarkesh Patel posted Tuesday, Aschenbrenner spoke of inner conflicts over priorities, suggesting a shift in focus in the direction of speedy progress and deployment of AI fashions on the expense of security.

He additionally stated he was fired for placing his issues in writing.

In a wide-ranging, four-hour dialog, Aschenbrenner advised Patel that he penned an inner memo final yr detailing his issues and circulated it amongst respected consultants from exterior the corporate. Nonetheless, after a significant safety incident occurred weeks later, he stated he determined to share an up to date memo with a few board members. He was rapidly launched from OpenAI.

“What may additionally be useful context is the sorts of questions they requested me after they fired me… the questions had been about my views on AI progress, on AGI, the suitable stage of safety for AGI, whether or not the federal government ought to be concerned in AGI, whether or not I and the superalignment crew had been loyal to the corporate, and what I used to be as much as throughout the OpenAI board occasions,” Aschenbrenner stated.

AGI, or synthetic normal intelligence, is when AI meets or exceeds human intelligence throughout any subject, no matter the way it was skilled.

Loyalty to the corporate—or to Sam Altman—emerged as a key issue after his temporary ouster: over 90% of workers signed a letter threatening to stop in solidarity with him. Additionally they popularized the slogan, “OpenAI is nothing with out its individuals.”

“I didn’t signal the worker letter throughout the board occasions, regardless of stress to take action,” Aschenbrenner recalled.

The superalignment crew—led by Ilya Sutskever and Jan Leike—was answerable for constructing long-term security practices to verify AI stays aligned with human expectations.The departure of outstanding members of that crew, together with Sutskever and Leike, introduced added scrutiny. The entire crew was subsequently dissolved, and a brand new security crew was introduced… led by CEO Sam Altman, who can be a member of the OpenAI board to which it experiences.

Aschenbrenner stated OpenAI’s actions contradict its public statements about security.

“One other instance is once I raised safety points—they might inform me safety is our primary precedence,” he said. “Invariably, when it got here time to take a position severe assets or make trade-offs to take fundamental measures, safety was not prioritized.”

That is in keeping with statements from Leike, who stated the crew was “crusing in opposition to the wind” and that “security tradition and processes have taken a backseat to shiny merchandise” beneath Altman’s management.

Aschenbrenner additionally expressed issues about AGI improvement, stressing the significance of a cautious strategy—significantly as many worry China is pushing exhausting to surpass america in AGI analysis.

China “goes to have an all-out effort to infiltrate American AI labs, billions of {dollars}, 1000’s of individuals… [they’re] going to try to outbuild us,” he stated. “What will likely be at stake is not going to simply be cool merchandise, however whether or not liberal democracy survives.”

Just some weeks in the past, it was revealed that OpenAI required its workers to signal abusive non-disclosure agreements (NDAs) that prevented them from talking out in regards to the firm’s security practices.

Aschenbrenner stated he didn’t signal such an NDA, however stated that he was provided round $1M in fairness.

In response to those rising issues, a collective of practically a dozen present and former OpenAI workers have in the meantime signed an open letter demanding the suitable to name out firm misdeeds with out worry of retaliation.

The letter—endorsed by trade figures like Yoshua Bengio, Geoffrey Hinton, and Stuart Russell—emphasizes the necessity for AI corporations to decide to transparency and accountability.

“As long as there isn’t any efficient authorities oversight of those firms, present and former workers are among the many few individuals who can maintain them accountable to the general public—but broad confidentiality agreements block us from voicing our issues, besides to the very corporations which may be failing to deal with these points,” the letter reads. “Atypical whistleblower protections are inadequate as a result of they concentrate on criminal activity, whereas most of the dangers we’re involved about will not be but regulated.

“A few of us moderately worry numerous types of retaliation, given the historical past of such circumstances throughout the trade,” it continues. “We’re not the primary to come across or discuss these points.”

After information of the restrictive employment clauses unfold, Sam Altman claimed he was unaware of the scenario and warranted the general public his authorized crew was working to repair the problem.

“There was a provision about potential fairness cancellation in our earlier exit docs; though we by no means clawed something again, it ought to by no means have been one thing we had in any paperwork or communication,” he tweeted. “That is on me and one of many few occasions I have been genuinely embarrassed working OpenAI; I didn’t know this was occurring and I ought to have.”

OpenAI says it has since launched all workers from the contentious non-disparagement agreements and eliminated the clause from its departure paperwork.

OpenAI didn’t reply to a request for remark from Decrypt.