A new California measure, formally known as SB 1047 or the Safe and Secure Innovation for Frontier Artificial Intelligence Systems Act, would require developers of the largest and most powerful AI systems to test for safety before releasing those models to the public.

The bill would also require companies to tell the state about its testing protocols and guardrails, and California’s attorney general could sue if the tech causes “critical harm.”

AI developers in California have already taken important first steps in pioneering safe development practices. But California’s government cannot afford to be complacent. With Congress paralyzed and the future of the Biden Administration’s Executive Order in doubt, California has an indispensable role to play in ensuring that we develop this extremely powerful technology with basic safety guardrails in order to allow society to experience AI’s massive potential benefits,” (( state Sen. Scott D-San Francisco)) Wiener said in a statement.