Executive directors of Several major counterintelligence companies are urging members of Congress to pass new laws that would make it harder for bad actors to develop biological weapons using their technology.
Demis Hassabis of Google DeepMind, Sam Altman of OpenAI, Dario Amodei of Anthropic, and Mustafa Suleyman of Microsoft AI are among the signatories on public letter calling for laws requiring companies that sell synthetic DNA and RNA to screen customers and mandates to prevent genetic abuse.
Drafted by the nonpartisan Institute for Development and the right-leaning Foundation for American Innovation, the letter acknowledges that given the pace of AI development, “there is a real possibility that the knowledge barriers that have historically prevented bad actors from acquiring biological weapons will erode.”
Scientist Arthur Kornberg was the first to successfully sequence DNA in the 1950s. Now, the process has been automated, with many companies around the world using commercial primers to “print” and sell specific genetic sequences that are used for scientific research, drug development and diagnostics. Most providers sell only to qualified researchers, biotech companies, and academic institutions, but not all are veterinary or gene sequencing customers.
In 2017, Canadian researchers raised the alarm when they used $100,000 worth of mail order DNA to recreate the horsepox virus. Critics said the same technique could be used to make smallpox, a closely related and deadly virus. The gene pool has recovered since then.
Combined with advances in AI, it is now possible to design deadly new toxins and pathogens using large-scale language models, although some biology training may still be required to create a working virus from scratch. Although bioterrorist attacks have been rare, they have the potential to cause significant casualties, public fear, and economic loss. A major concern is that an AI-created pathogen could intentionally or unintentionally trigger a global pandemic.
“AI tools enable the user to identify very quickly where to turn to an order procedure that will not be investigated,” says David Relman, an expert in biology and biosecurity at Stanford University, who signed the letter. “Properly prompted, they can also tell you how to change the nature of your order, so that even those being screened are less likely to detect what you’re trying to make.”
The signatories include other scientists, national security experts, and executives from gene synthesis companies Twist Bioscience and Ansa Biotechnologies. These companies are members of the International Gene Synthesis Alliance, which was founded in 2009 to implement voluntary screening practices. Many companies already use software to review prescriptions for “range of concern” that may contribute to organism toxicity or disease-causing potential.
“If you have technology that’s capable of synthesizing DNA, then you have to make sure it’s used responsibly, and part of that is making sure you understand what you’re making and who you’re making it for,” says James Diggans, vice president of policy and biosafety at Twist Bioscience. The company has helped enforce official laws for years.
Federation guidelines established during the Biden administration required scientists and companies that receive government funding to order gene sequences from providers that make purchases. A bipartisan bill introduced earlier this year in the Senate would require all gene-synthesis providers operating in the United States to screen prescriptions and customers for bad actors or dangerous pathogens.
But diagnostic tools are not perfect. Last year, Microsoft researchers published a read showing that AI protein modeling tools were able to extract potentially harmful gene sequences that had eluded previous company screening programs. The models suggested new protein sequences and similar structures to those known to be harmful.
Geoff Ralston, former president of Y Combinator and partner at the Safe AI Fund, thinks that AI labs with biological models should conduct their own user surveys.
“It should be very difficult, if not impossible, to ask a model to help you do something so dangerous,” says Ralston, who also signed the letter.
Relman agrees that regulations on investigative procedures are only part of the solution. “Given that screening can fail in some cases, we must have other control points,” he says. “That’s where AI companies will have to step up.”
