Why should you care about the RAISE Act?

Could an AI model cause “the death or serious injury of one hundred or more people,” or result in over a billion dollars in damage? Could AI assist in the “creation or use of a chemical, biological, radiological, or nuclear weapon?”

These are among some of the “critical harms” described in New York’s Responsible AI Safety and Education (RAISE) Act, signed by Governor Hochul on December 19th. It is a new law requiring AI developers to disclose security protocols and safety incidents to the Department of Financial Services. The bill was marked by major revisions, both in the Legislature and through recent chapter amendments that were conditional to Hochul’s signing. These changes modeled the bill on California’s SB 53 — the only other state-level AI transparency legislation of this scope, and itself a significantly pared-down successor to last year’s vetoed SB 1047.

While politicians and activists have set their sights on many different facets of AI and its potential impacts on society; legislation like the RAISE Act is focused on holding AI developers accountable to their own security and safety protocols. These safety concerns range from models that can enable users to develop bioweapons to the use of AI to conduct devastating cyberattacks, even risks from models circumventing the controls of their own developers while acting autonomously.

This is not the first time these types of concerns have been raised. In 2023, more than 1,000 tech leaders signed a letter calling for a six-month pause on frontier model development until international safety standards could be established, warning of the dangers of a, “Race to develop and deploy ever more powerful digital minds that no one—not even their creators—can understand, predict, or reliably control.”

Even AI developers like OpenAI have reported that some of their deep research models are, “On the cusp of being able to meaningfully help novices create known biological threats…” and that they expect these models to cross this threshold in the near future. The International AI Safety Report, written by over 100 experts from 30 countries and led by the "Godfather of AI," Yoshua Bengio, identified several emerging risks. Notably, they found that an existing model produced plans for biological weapons "rated superior to plans generated by experts with a PhD 72% of the time and provide[d] details that expert evaluators could not find online."

The RAISE Act focuses on "severe risks that could cause over $1 billion in damage or hundreds of deaths or injuries." The law targets catastrophic risks that could arise from the development and deployment of models by AI developers, but how does it plan to actually do this?

What does (or did) the RAISE Act do?

The RAISE Act has gone through significant changes since its inception. However, its main function largely remains the same: to require “large developers” to create and disclose basic safety and security protocols. These protocols are intended to reasonably reduce the risk of critical harms caused by the deployment or development of a frontier model, including unauthorized access by malicious actors. A copy of this protocol would have to have been provided to the Attorney General and DHSES. Safety incidents must be reported within 72 hours. Furthermore, a version of the bill empowered the Attorney General to sue to prevent the release of a model deemed to create an unreasonable risk of critical harm.

Other earlier versions of the bill also contained specific whistleblower protections for employees reporting incidents with an, albeit mild, $10,000 penalty for retaliation against employees. While most of the transparency requirements remain, the law as it stands today is essentially a different bill from the one presented to Hochul in December.

How has the bill changed since its inception?

Several major changes were made within the Legislature prior to the bill being sent to the Governor. Specific whistleblower protections were removed, with critics arguing that the section would be redundant to existing New York Labor Law (though others have noted this clause would have additionally protected non-paid advisors and contractors, such as eval providers, not currently covered under existing law).

Notably, the requirement for mandatory annual independent third-party audits of safety protocols was removed. Penalties were also significantly decreased from 5% of compute costs for initial violations and 15% for subsequent violations, to a flat fine of $10 million for initial and $30 million for subsequent violations. With industry leaders like Dario Amodei estimating future training costs as high as $10 to $100 billion, this represents a decrease in penalties orders of magnitude smaller than what was originally proposed.

Yet, even after the bill was significantly scaled back, legislative leaders and Hochul reportedly faced a stalemate in December as the Governor sought to amend it to more closely align with California’s less expansive SB 53. The Governor's proposed changes reportedly incorporated sections lifted directly from the West Coast bill.

The agreed-upon chapter amendments resulted in further lowering penalties from $10 million and $30 million to $1 million and $3 million, respectively. The revisions also changed the definition of a “large developer” subject to the law to include only developers with over $500 million in revenue, as opposed to the more inclusive threshold of having spent >$100 million on development. Even more notable, according to reporting from City & State, is the removal of the law’s ability to prevent companies from releasing unsafe models, instead only requiring that developers provide a warning to regulators.

Why were these changes made?

While legislative leaders and sponsors like Assemblyman Alex Bores and State Senator Andrew Gounardes have touted the bill as a success, other advocates have expressed anger and disappointment over the weakened provisions.

According to supporters, these changes were made to align the law with California’s SB 53 and avoid a patchwork of different regulations between states. What is clear, however, is that this bill was subject to intense lobbying by tech companies to roll back and even completely remove many of its initial mandates.

Think Big is one of an array of super PACs under the umbrella of Leading the Future, which released an attack ad against bill sponsor Alex Bores in early December, accusing him and the RAISE Act of “crushing innovation.” They have also promised to spend $100 million in the upcoming midterm elections (in which Bores is currently running in a primary) to support “pro-innovation” candidates. Leading the Future receives financial support from several notable entities, including the venture capital firm Andreessen Horowitz, Palantir co-founder Joe Lonsdale, AI search-engine company Perplexity, and SV Angel founder Ron Conway. The New York Times reported that Ms. Hochul traveled to San Francisco to raise money for her campaign alongside Ron Conway and other tech leaders.

What’s next?

The recently published revised RAISE Act confirms previous reports. Nevertheless, the future of state-level AI regulation is even more unclear in the wake of the Trump administration’s executive order granting the Attorney General authority to sue states and overturn laws inconsistent with “the United States’ global AI dominance.” Despite Congress’s lack of action in developing a national AI framework, the upcoming 2026 midterms may see the ascendance of SB 1047 and SB 53 architect Scott Wiener, as well as Alex Bores in NY-12, to Congress. However, as it stands, it’s hard to see whether their efforts can withstand the growing resistance from Big Tech and the influential AI lobby.