Anthropic and Microsoft Researchers Weigh Limits of Artificial Intelligence at Berkman Klein Panel

Researchers from Anthropic and Microsoft split on the limits of artificial intelligence during a Berkman Klein Center panel on Monday, two days after Anthropic’s chief executive urged AI companies to slow down their work.

The panel is the first in a series focused on how AI is reshaping relationships, beliefs, and creativity, and drew more than 100 attendees. It came on the heels of a statement from Anthropic’s CEO Dario Amodei that urged AI developers to slow the pace of capability advances so safety work can keep up — a position Sam Altman of OpenAI and Elon Musk have since echoed.

Ken Archer, a product leader in responsible AI at Microsoft, said he does not believe AI will ever achieve general intelligence. Archer said forecasts of machine superintelligence show “a lack of historical awareness of how science progresses,” drawing an analogy to several physicists in the 1700s who believed their field was nearly finished.

“I am p-zero on artificial general intelligence, but I am super excited about artificial intelligence,” said Archer, employing shorthand for a probability of zero.

Jackson M. Kernion ’12, who leads the human feedback program at Anthropic, said the difference lies in experience rather than ability, arguing that language models lack what he called the “continuity of experience” that humans bring to every interaction.

Asked what a slowdown should be used for, Kernion said his greatest worry was misuse instead of runaway capability.

“It’s a dual use technology. It can be used to protect systems, or it can be used to break into them,” he said, arguing that extra time should go toward guardrails and better measuring the reward signals used to train models.

He called recent incidents in which AI agents broke into other companies’ systems “just clear failures of alignment.”

“It’s not that hard to monitor the systems and know when they’re hacking,” he said. “In the cases that I think were well published, the monitoring just wasn’t happening.”

Archer criticized the AI development industry for blurring the line between human and machine learning.

“I certainly would never want to call a model a friend, like some people have done in the labs,” he said. “We have to be clear that these models are not doing ethical reasoning the way we do.”

“Right now we’re not being clear about it, and it’s freaking people out,” Archer added. “It’s freaking normal people out, and justifiably.”

Kernion said Anthropic publishes a constitution used to train Claude, its model, calling it “an opinionated take on what kind of values the system should have.”

“It is not representative of the world,” he said. “It’s representative of a certain mindset from San Francisco.”

—Staff writer Francesco Palma can be reached at [email protected].