In 1954, several years after he led the project that created the atomic bomb, physicist J. Robert Oppenheimer was called to testify before the Atomic Energy Commission (AEC). A prominent topic at the hearings was Oppenheimer’s position on the hydrogen bomb, a more destructive version of the atomic bomb that the United States had developed and tested for the first time two years earlier.
Oppenheimer, who in the post-war years was more conflicted about atomic weapons, initially opposed work on the hydrogen or thermonuclear bomb, partly for moral reasons and partly because he doubted that it would work. But later he changed his mind and supported the work on it. AEC lawyers wanted to know why.
It wasn’t because Oppenheimer had changed his mind about the ethics of nuclear bombing the city. Rather, it was because American physicists had developed a new design for hydrogen bombs that was not only easy to operate, but also of fascinating beauty, or “technical delicacy” as he called it. For Oppenheimer, that was enough. Like him he told the AEC session: “When you see something that tastes good, you go ahead and do it, and you argue about what to do about it only after you’ve had your technical breakthrough.”
What Oppenheimer described was a kind of moral impotence dressed up as a solution: the lure of a good scientific answer to a bad problem, and the accompanying tendency to hold off on moral accounting until after technical success. It’s one of the most honest things anyone who built the bomb – or anything else that’s going to change the world – has ever said. And it hasn’t stopped being important, because the people who are now creating the technology that is changing the world of our time continue to say versions of it too.
Jack Clark, co-founder and head of policy at Anthropic, the company behind the Claude models, is one of those people. So it was worth paying attention last week when Clark he sat down for a long public talk and Samuel Kimbriel, founding director of Philosophy and Society of the Aspen Institute, just six days after the federal government. had suddenly cut off access to the two most powerful examples of Anthropic, appearing on the fear of what they could do.
Much of the conversation revolved around one idea that will be familiar to those who read Clark’s work: Powerful AI is coming, and it’s giving us a choice — a choice we’re actively rejecting by failing to control AI. (Disclosure: Future Perfect is funded in part by the BEMC Foundation, whose major sponsor was also an early investor in Anthropic; they have no editorial input into our content.)
We control toothbrushes, Clark said, and cars, and nuclear weapons. “But we seem to have this attitude towards technology that is impossible to control,” he said. “It’s impossible to control … we’re acting like, oh, the tech industry is going to do things, which I think is an option.” Its most striking example was the evolution of the online platform that completely reinvented itself two decades ago. “Social media ran an uncontrolled experiment on the world,” he said. “We all now think and speak a little differently because of social media. That was a choice. We can choose to be different.”
This is the kind of conversation that has has long distinguished Anthropic from other large AI companies: Its leaders are ready to dwell on the great risks of advanced AI, risks that require clear and even strong control. (About a week before the Aspen talks — and just a day before the Trump administration confronts the latest Anthropic models — CEO Dario Amodei published a blog post calls on government authorities to legally prevent or even reverse the deployment of borderline AI designs that fail security tests against threats such as cyber-hacking and biological weapons.)
Anthropic acknowledges that advanced AI is a gamble to be had, but argues that it’s a gamble we must take. In the Aspen talk, Clark talked about the next century that will have brutal challenges – an aging population, evolving institutions, a warming planet – that can only be tackled by AI. Not to move forward with artificial intelligence would be to rob ourselves of the medical miracles we can only imagine, and to openly condemn those who could be saved.
Clark is right that there is choice buried in all of this. But the question of whose path creation is exactly whose choice.
Sure, as Clark said, we control cars and toothbrushes and nuclear weapons, but in each case someone built something first, and the rest of us were left to decide what to do with the world we already have. No one voted on whether the atomic bomb should exist. We were handed the results and had to write the rules afterwards.
The same is true for AI. The choice that Clark wants the public to make to govern was necessary once his industry created something that needed to be governed. He’s giving us a vote on what to do about AI, not a vote on whether to do it – because that vote was already taken, privately, by him, hundreds of colleagues, and. trillions of dollars. But why? he didn’t do that do we have anything to say? Why are we stuck in a world where, as in Oppenheimer’s formulation, “you argue about what to do about it only after you’ve had your technical breakthrough”?
I wasn’t the only person in the audience who was wondering this. Near the end of the conversation, a young woman put a strong version of this question to Clark directly. Every frontier laboratory now admits the technology has serious risks, even present riskhe noted. “So my question is, what gives you, Anthropic, and the other frontier labs the right to continue building something that can destroy everyone, when none of us can get out of it?”
Clark, to his credit, did not ignore the question. But he didn’t answer fully. He changed it – away from the choice to build, towards the need for someone to take responsibility after it was built.
That one cannot be the companies themselves, he said, describing a bright future where “external systems of compliance, control, testing and certification” will decide when each laboratory is allowed to go further. Governments were already moving faster than anyone expected – the US and UK, he said, they had created testing agencies tools that were sometimes better than the company’s.
It was a gracious response — albeit one that sat uncomfortably with the reality that President Trump now appears to be. control the AI at will – but see what it admits. Asked what gives his company the right to build something that could destroy everyone, the head of policy at a leading AI lab said nothing. we have that right. He said the decision should not belong to a company like his, but to explain the system to remove it from their hands which is not yet completely there. He and his colleagues are still building, on the border, as fast as science and mathematics allow, while telling the room that someone else should be in charge. AI is already free in the world. AI control is still the stuff of blog posts.
So why do they really do this? To bring it back to Oppenheimer: because AI is “technically sweet.” Not the race with China, not the trillion dollar calculations, not even the possible desire to cure diseases – although all of that is true. Under them is something simpler and more difficult to control: we have to build what is good. All Clark said was that, marveling that AI is “simpler and easier to build than many other aspects of science,” that its chief scientist jokes they would have AGI already if they just fixed the bugs in their code.
We humans are a tool-using species, Clark said, and AI is the ultimate tool. It’s not that AI is inevitable, exactly, but that it’s so easy to build once the foundations are laid that “almost any way you go, (AI) appears.”
What Clark described is the attraction Oppenheimer called in 1954 – the attraction of a beautiful solution that makes the question of whether you should build it feel beside the point.
I can feel it myself, and I’m just a user. Put a powerful model in your palm, ask it to do something you couldn’t do on your own – write a program, find a bug, solve something you’ve been stuck on – then simply watch. do it what you asked for, and you’ll get a little electric shock that has nothing to do with the aging population or the future of democracy. That joy runs seamlessly from the user at the keyboard through the engineer who trained the model to the manager who shipped it.
That’s why I suspect Clark’s control talk, while sincere, is less of a decision than was ever in doubt. Like Oppenheimer and the hydrogen bomb, the people creating this technology feel they have no choice but to proceed – and then hope the rest of us will make the right decisions to control what they couldn’t help but do.
We’re lucky, so far, with the last sweet device that can still end the world. The hydrogen bomb has existed for 70 years without being used in anger, not because we solved the politics Oppenheimer warned about, but because the rational choice prevailed. And because we were lucky.
Clark may be right that the choice is still ours: The bomb didn’t decide the Cold War, people did, and people can decide this too. But it would help if the people who give us that option would slow down enough to let us do it – instead of building as fast as they can and trusting our luck, and theirs, to hold.
A version of this story originally appeared on Future Perfect journal. Register here!
