Generative disinfo is real — you’re just not the target, warns deepfake tracking nonprofit

Many feared that the 2024 election would be affected, and perhaps decided, by AI-generated disinformation. While there was some to be found, it was far less than anticipated. But don’t let that fool you: the disinfo threat is real — you’re just not the target.

So at least says Oren Etzioni, an AI researcher of long standing, whose nonprofit TrueMedia has its finger on the generated disinformation pulse.

“There is, for lack of a better word, a diversity of deepfakes,” he told TechCrunch in a recent interview. “Each one serves its own purpose, and some we’re more aware of than others. Let me put it this way: for every thing that you actually hear about, there are a hundred that are not targeted at you. Maybe a thousand. It’s really only the very tip of the iceberg that makes it to the mainstream press.”

The fact is that most people, and Americans more than most, tend to think that what they experience is the same as what others experience. That isn’t true for a lot of reasons. But in the case of disinformation campaigns, America is actually a hard target, given a relatively well -nformed populace, readily available factual information, and a press that is trusted at least most of the time (despite all the noise to the contrary).

We tend to think of deepfakes as something like a video of Taylor Swift doing or saying something she wouldn’t. But the really dangerous deepfakes are not the ones of celebrities or politicians, but of situations and people that can’t be so easily identified and counteracted.

“The biggest thing people don’t get is the variety. I saw one today of Iranian planes over Israel,” he noted — something that didn’t happen but can’t easily be disproven by someone not on the ground there. “You don’t see it because you’re not on the Telegram channel, or in certain WhatsApp groups — but millions are.”

TrueMedia offers a free service (via web and API) for identifying images, video, audio, and other items as fake or real. It’s no simple task, and can’t be completely automated, but they are slowly building a foundation of ground truth material that feeds back into the process.

“Our primary mission is detection. The academic benchmarks [for evaluating fake media] have long since been plowed over,” Etzioni explained. “We train on things uploaded by people all over the world; we see what the different vendors say about it, what our models say about it, and we generate a conclusion. As a follow up, we have a forensic team doing a deeper investigation that’s more extensive and slower, not on all the items but a significant fraction, so we have a ground truth. We don’t assign a truth value unless we’re quite sure; we can still be wrong, but we’re substantially better than any other single solution.”

The primary mission is in service of quantifying the problem in three key ways Etzioni outlined:

1. How much is out there? “We don’t know, there’s no Google for this. You see various indications that it’s pervasive, but it’s extremely difficult, maybe even impossible to measure accurately.”
2. How many people see it? “This is easier because when Elon Musk shares something, you see, ’10 million people have seen it.’ So the number of eyeballs is easily in the hundreds of millions. I see items every week that have been viewed millions of times.”
3. How much impact did it have? “This is maybe the most important one. How many voters didn’t go to the polls because of the fake Biden calls? We’re just not set up to measure that. The Slovakian one [a disinfo campaign targeting a presidential candidate there in February] was last minute, and then he lost. That may well have tipped that election.”

All of these are works in progress, some just beginning, he emphasized. But you have to start somewhere.

“Let me make a bold prediction: over the next 4 years we’re going to become much more adept at measuring this,” he said. “Because we have to. Right now we’re just trying to cope.”

As for some of the industry and technological attempts to make generated media more obvious, such as watermarking images and text, they’re harmless and maybe beneficial, but don’t even begin to solve the problem, he said.

“The way I’d put it is, don’t bring a watermark to a gun fight.” These voluntary standards are helpful in collaborative ecosystems where everyone has a reason to use them, but they offer little protection against malicious actors who want to avoid detection.

It all sounds rather dire, and it is, but the most consequential election in recent history just took place without much in the way of AI shenanigans. That is not because generative disinfo isn’t commonplace, but because its purveyors didn’t feel it necessary to take part. Whether that scares you more or less than the alternative is quite up to you.