Judge blocks California’s new AI law in case over Kamala Harris deepfake

A federal judge blocked one of California’s new AI laws on Wednesday, less than two weeks after it was signed by Governor Gavin Newsom. Shortly after signing AB 2839, Newsom suggested it could be used to force Elon Musk to take down an AI deepfake of Vice President Kamala Harris he had reposted (sparking a petty online battle between the two). However, a California judge just ruled the state can’t force people to take down election deepfakes – not yet, at least.

AB 2839 targets the distributors of AI deepfakes on social media, specifically if their post resembles a political candidate and the poster knows it’s a fake that may confuse voters. The law is unique because it does not go after the platforms on which AI deepfakes appear, but rather those who spread them. AB 2839 empowers California judges to order the posters of AI deepfakes to take them down or potentially face monetary penalties.

Perhaps unsurprisingly, the original poster of that AI deepfake – an X user named Christopher Kohls – filed a lawsuit to block California’s new law as unconstitutional just a day after it was signed. Kohls’ lawyer wrote in a complaint that the deepfake of Kamala Harris is satire that should be protected by the First Amendment.

On Wednesday, United States district judge John Mendez sided with Kohls. Mendez ordered a preliminary injunction to temporarily block California’s attorney general from enforcing the new law against Kohls or anyone else, with the exception of audio messages that fall under AB 2839.

Read for yourself what Judge Mendez said in his decision:

“Almost any digitally altered content, when left up to an arbitrary individual on the internet, could be considered harmful. For example, AI-generated approximate numbers on voter turnout could be considered false content that reasonably undermines confidence in the outcome of an election under this statute. On the other hand, many ‘harmful’ depictions when shown to a variety of individuals may not ultimately influence electoral prospects or undermine confidence in an election at all. As Plaintiff persuasively points out, AB 2839 ‘relies on various subjective terms and awkwardly-phrased mens rea,’ which has the effect of implicating vast amounts of political and constitutionally protected speech…

[W]hile a well-founded fear of a digitally manipulated media landscape may be justified, this fear does not give legislators unbridled license to bulldoze over the longstanding tradition of critique, parody, and satire protected by the First Amendment. YouTube videos, Facebook posts, and X tweets are the newspaper advertisements and political cartoons of today, and the First Amendment protects an individual’s right to speak regardless of the new medium these critiques may take. Other statutory causes of action such as privacy torts, copyright infringement, or defamation already provide recourse to public figures or private individuals whose reputations may be afflicted by artificially altered depictions peddled by satirists or opportunists on the internet…

The record demonstrates that the State of California has a strong interest in preserving election integrity and addressing artificially manipulated content. However, California’s interest and the hardship the State faces are minimal when measured against the gravity of First Amendment values at stake and the ongoing constitutional violations that Plaintiff and other similarly situated content creators experience while having their speech chilled.“

In essence, he ruled the law is simply too broad as written and could result in serious overstepping by state authorities into what speech is permitted or not.

Because this is a preliminary injunction, we’ll have to wait and see whether this California law is truly blocked for good, but either way it is unlikely to have much effect on next month’s election. AB 2839 is one of 18 new laws relating to AI that Newsom has signed in the last month.

It’s nevertheless a big win for Elon Musk’s camp of free speech posters on X. In the days following Newsom signing AB 2839 into law, Musk and his usual allies posted a series of AI deepfakes that tested California’s new law.