California’s SB 1047 bill aims to prevent real-world disasters caused by AI systems by implementing safeguards and holding developers accountable for potential harms. The bill specifically targets large AI models that cost at least $100 million to train and use a significant amount of compute power during training. Companies such as OpenAI, Google, and Microsoft are likely to be affected by these rules as they develop and deploy large AI models.
The bill requires developers to implement safety protocols and create testing procedures to address risks posed by AI models. It also mandates the hiring of third-party auditors to assess AI safety practices annually. Developers must provide “reasonable assurance” that these protocols will prevent critical harms, although absolute certainty is impossible.
Enforcement of the bill would be carried out by a new California agency called the Frontier Model Division (FMD). The FMD would oversee the certification of public AI models that meet the bill’s thresholds and ensure compliance with safety protocols. The agency would be governed by a board consisting of representatives from the AI industry, open-source community, and academia.
Proponents of SB 1047 argue that it is necessary to learn from past policy failures with social media and data privacy and protect citizens before any potential harm occurs. They believe that California should set a precedent in regulating AI technology since Congress has done little in this regard.
Support for the bill comes from influential figures in the AI community, including Geoffrey Hinton and Yoshua Bengio, who are concerned about the potential dangers of AI technology. The Center for AI Safety also supports the bill and emphasizes the need to prioritize mitigating the risks associated with AI.
Opponents of SB 1047 include Silicon Valley players such as a16z, which claims that the bill’s thresholds are arbitrary and will burden startups. Fei-Fei Li, a well-regarded AI pioneer, argues that the bill will harm the AI ecosystem. Other critics, such as Yann LeCun from Meta, believe that the bill is based on an illusion of “existential risk” and will hinder research efforts.
The Chamber of Progress, a trade group representing Big Tech companies, opposes the bill, claiming that it restrains free speech and innovation. U.S. Congressman Ro Khanna also opposes SB 1047, expressing concerns about its effectiveness and potential negative impact on entrepreneurship and small businesses.
SB 1047 will undergo further amendments before being sent to the California Senate’s Assembly floor. If it passes, it will be sent to Governor Gavin Newsom’s desk for approval. The bill is expected to face legal challenges and may not go into effect immediately.
