Here is what’s illegal under California’s 18 (and counting) new AI laws

In September, California Governor Gavin Newsom considered 38 AI-related bills, including the highly contentious SB 1047, which the state’s legislature sent to his desk for final approval. He vetoed SB 1047 on Sunday, but signed more than a dozen other AI bills into law over the course of the month. These bills try to address the most pressing issues in artificial intelligence: everything from futuristic AI systems creating existential risk, deepfake nudes from AI image generators, to Hollywood studios creating AI clones of dead performers.

“Home to the majority of the world’s leading AI companies, California is working to harness these transformative technologies to help address pressing challenges while studying the risks they present,” said Governor Newsom’s office in a press release.

So far, Governor Newsom has signed 18 AI bills into law, some of which are America’s most far reaching laws on generative AI yet. Here’s what they do.

AI risk

On Sunday, Governor Newsom signed SB 896 into law, which requires California’s Office of Emergency Services to perform risk analyses on potential threats posed by generative AI. CalOES will work with frontier model companies, such as OpenAI and Anthropic, to analyze AI’s potential threats to critical state infrastructure, as well as threats that could lead to mass casualty events.

Training data

Another law Newsom signed this month requires generative AI providers to reveal the data used to train their AI systems in documentation published on their website. AB 2013 goes into effect in 2026, and requires AI providers to publish: the sources of its datasets, a description of how the data is used, the number of data points in the set, whether copyrighted or licensed data is included, the time period the data was collected, among other standards.

Privacy and AI systems

Newsom also signed AB 1008 on Sunday, which clarifies that California’s existing privacy laws are extended to generative AI systems as well. That means that if an AI system, like ChatGPT, exposes someone’s personal information (name, address, biometric data), California’s existing privacy laws will limit how businesses can use and profit off of that data.

Education

Newsom signed AB 2876 this month, which requires California’s State Board of Education to consider “AI literacy” in its math, science, and history curriculum frameworks and instructional materials. This means California’s schools may begin teaching students the basics of how artificial intelligence works, as well as the limitations, impacts, and ethical considerations of using the technology.

Another new law, SB 1288, requires California superintendents to create working groups to explore how AI is being used in public school education.

Defining AI

This month, Newsom signed a bill that establishes a uniform definition for artificial intelligence in California law. AB 2885 states that artificial intelligence is defined as “an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.”

Healthcare

Another bill signed in September is AB 3030, which requires healthcare providers to disclose when they use generative AI to communicate with a patient, specifically when those messages contain a patient’s clinical information.

Meanwhile, Newsom recently signed SB 1120, which puts limitations on how health care service providers and health insurers can automate their services. The law ensures licensed physicians supervise the use of AI tools in these settings.

AI robocalls

Last Friday, Governor Newsom signed a bill into law requiring robocalls to disclose whether they’ve use AI-generated voices. AB 2905 aims to prevent another instance of the deepfake robocall resembling Joe Biden’s voice that confused many New Hampshire voters earlier this year.

Deepfake pornography

On Sunday, Newsom signed AB 1831 into law, which expands the scope of existing child pornography laws to include matter that is generated by AI systems.

Newsom signed two laws that address the creation and spread of deepfake nudes last week. SB 926 criminalizes the act, making it illegal to blackmail someone with AI-generated nude images that resemble them.

SB 981, which also became law on Thursday, requires social media platforms to establish channels for users to report deepfake nudes that resemble them. The content must then be temporarily blocked while the platform investigates it, and permanently removed if confirmed.

Watermarks

Also on Thursday, Newsom signed a bill into law to help the public identify AI-generated content. SB 942 requires widely used generative AI systems to disclose they are AI-generated in their content’s provenance data. For example, all images created by OpenAI’s Dall-E now need a little tag in their metadata saying they’re AI generated.

Many AI companies already do this, and there are several free tools out there that can help people read this provenance data and detect AI-generated content.

Election deepfakes

Earlier this week, California’s governor signed three laws cracking down on AI deepfakes that could influence elections.

One of California’s new laws, AB 2655, requires large online platforms, like Facebook and X, to remove or label AI deepfakes related to elections, as well as create channels to report such content. Candidates and elected officials can seek injunctive relief if a large online platform is not complying with the act.

Another law, AB 2839, takes aim at social media users who post, or repost, AI deepfakes that could deceive voters about upcoming elections. The law went into effect immediately on Tuesday, and Newsom suggested Elon Musk may be at risk of violating it.

AI-generated political advertisements now require outright disclosures under California’s new law, AB 2355. That means moving forward, Trump may not be able to get away with posting AI deepfakes of Taylor Swift endorsing him on Truth Social (she endorsed Kamala Harris). The FCC has proposed a similar disclosure requirement at a national level and has already made robocalls using AI-generated voices illegal.

Actors and AI

Two laws that Newsom signed earlier this month — which SAG-AFTRA, the nation’s largest film and broadcast actors union, was pushing for — create new standards for California’s media industry. AB 2602 requires studios to obtain permission from an actor before creating an AI-generated replica of their voice or likeness.

Meanwhile, AB 1836 prohibits studios from creating digital replicas of deceased performers without consent from their estates (e.g., legally cleared replicas were used in the recent “Alien” and “Star Wars” movies, as well as in other films).

SB 1047 gets vetoed

Governor Newsom still has a few AI-related bills to decide on before the end of September. However, SB 1047 is not one of them – the bill was vetoed on Sunday.

During a chat with Salesforce CEO Marc Benioff earlier this month during the 2024 Dreamforce conference, Newsom may have tipped his hat about SB 1047, and how he’s thinking about regulating the AI industry more broadly.

“There’s one bill that is sort of outsized in terms of public discourse and consciousness; it’s this SB 1047,” said Newsom onstage this month. “What are the demonstrable risks in AI and what are the hypothetical risks? I can’t solve for everything. What can we solve for? And so that’s the approach we’re taking across the spectrum on this.”

Check back on this article for updates on what AI laws California’s governor signs, and what he doesn’t.