New York AI Bill: Preventing AI-Fueled Disasters

New York Considers Landmark AI Safety Legislation

Legislators in New York state have approved a bill designed to mitigate potential risks associated with advanced artificial intelligence (AI) models developed by leading companies such as OpenAI, Google, and Anthropic. The legislation focuses on preventing scenarios that could result in widespread harm, including fatalities, injuries to over 100 individuals, or damages exceeding $1 billion.

A Victory for AI Safety Advocates

The passage of the RAISE Act is considered a significant achievement for the AI safety community. This movement has faced challenges in recent years, as emphasis has often been placed on rapid development and innovation, particularly within Silicon Valley and under the previous administration. Prominent figures like Nobel laureate Geoffrey Hinton and AI research pioneer Yoshua Bengio have actively supported the RAISE Act.

Should the bill be enacted, it would establish the first legally mandated transparency requirements for developers of frontier AI technologies within the United States.

Addressing Concerns and Distinguishing from Previous Attempts

The RAISE Act shares some objectives with California’s previously proposed AI safety bill, SB 1047, which was ultimately vetoed. However, New York State Senator Andrew Gounardes, a co-sponsor of the RAISE Act, intentionally crafted the legislation to avoid hindering innovation among startups and academic research institutions – a frequent critique leveled against SB 1047.

Senator Gounardes emphasized the urgency of establishing safeguards, stating, “The window to put in place guardrails is rapidly shrinking given how fast this technology is evolving.” He further noted that experts in the field believe the risks associated with advanced AI are highly probable, a prospect he described as “alarming.”

Next Steps and Potential Impact

The RAISE Act is now awaiting consideration by New York Governor Kathy Hochul. She has the option to sign the bill into law, return it for revisions, or issue a veto.

If signed, the law would mandate that the largest AI labs publish comprehensive reports detailing the safety and security measures implemented in their frontier AI models. It also requires reporting of safety incidents, such as concerning model behavior or unauthorized access to AI systems.

Companies failing to comply with these standards could face civil penalties of up to $30 million, enforced by the New York Attorney General.

Scope of the Regulation

The RAISE Act is designed to regulate the largest players in the AI landscape, including companies based in the U.S. and abroad, such as DeepSeek and Alibaba. The transparency requirements apply to organizations whose AI models are trained using computing resources exceeding $100 million – a threshold currently surpassing the capabilities of most available AI models – and are offered to residents of New York.

Addressing Past Criticisms

According to Nathan Calvin, Vice President of State Affairs and General Counsel at Encode, the RAISE Act directly addresses concerns raised about previous AI safety proposals. Notably, it does not require developers to implement a “kill switch” for their models, nor does it hold companies accountable for harms resulting from post-training modifications to frontier AI.

Industry Pushback and Counterarguments

Despite these adjustments, Silicon Valley has expressed significant opposition to the New York AI safety bill. Assemblymember Alex Bores, a co-sponsor of the RAISE Act, acknowledged this resistance as expected, but maintained that the legislation would not impede technological innovation.

Andreessen Horowitz general partner Anjney Midha voiced strong disapproval, stating on X (formerly Twitter) that the RAISE Act is a “stupid” state-level bill that will disadvantage the U.S. in the global AI race. Andreessen Horowitz and Y Combinator were prominent opponents of SB 1047.

Mixed Reactions from the AI Community

Anthropic, an AI lab known for its focus on safety, has not yet formally endorsed the bill. Co-founder Jack Clark expressed concerns about its broad scope, suggesting it could pose challenges for “smaller companies.”

Senator Gounardes responded to Anthropic’s criticism, asserting that the bill was specifically designed to avoid impacting smaller organizations.

OpenAI, Google, and Meta have not yet provided comments on the proposed legislation.

Concerns About Model Availability

A recurring concern is that AI developers might choose not to offer their most advanced models within New York state due to the regulatory requirements. This outcome occurred in Europe following the implementation of stringent technology regulations, and was also a concern raised regarding SB 1047.

Economic Considerations and Potential Impact on Companies

Assemblymember Bores believes the regulatory burden imposed by the RAISE Act is relatively minimal and should not compel tech companies to withdraw their products from New York. He highlighted the state’s substantial economic importance, as it possesses the third-largest GDP in the U.S., suggesting that companies would be hesitant to abandon such a significant market.

“I don’t want to underestimate the political pettiness that might happen, but I am very confident that there is no economic reason for [AI companies] to not make their models available in New York,” Bores stated.