It’s Election Day in Arizona and elderly voters in Maricopa County are told by phone that local polling places are closed due to threats from militia groups.
Meanwhile, in Miami, a flurry of photos and videos on social media show poll workers dumping ballots.
The phone calls in Arizona and the videos in Florida turn out to be “deepfakes” created with artificial intelligence tools. But by the time local and federal authorities figure out what they are dealing with, the false information has gone viral across the country.
This simulated scenario was part of a recent exercise in New York that gathered dozens of former senior U.S. and state officials, civil society leaders and executives from technology companies to rehearse for the 2024 election.
The results were sobering.
“It was jarring for folks in the room to see how quickly just a handful of these types of threats could spiral out of control and really dominate the election cycle,” said Miles Taylor, a former senior Department of Homeland Security official who helped organize the exercise for the Washington-based nonprofit The Future US.
Dubbed “The Deepfake Dilemma,” the exercise illustrated how AI-enabled tools threaten to turbocharge the spread of false information in an already polarized society and could sow chaos in the 2024 election, multiple participants told NBC News.
Rather than examining a singular attack by a group or hostile regime, the exercise explored a scenario with an array of both domestic and foreign actors launching disinformation, exploiting rumors and seizing on political divisions.
The organizers and participants in the war game spoke exclusively to NBC News about how it played out.
They said it raised worrisome questions about whether federal and local officials — and the tech industry — are prepared to counter both foreign and domestic disinformation designed to undermine public confidence in the election results.
Current U.S. officials say privately they share those concerns and that some state and local election agencies will be hard-pressed to keep the election process on track.
The exercise illustrated the uncertainty surrounding the roles of federal and state agencies and tech firms seven months before what is expected to be one of the most divisive elections in U.S. history.
Does the federal government have the ability to detect an AI deepfake? Should the White House or a state election office publicly declare that a particular report is false?
Unlike a natural disaster, in which government agencies work through a central command, America’s decentralized electoral system is entering uncharted territory without a clear sense of who’s in charge, said Nick Penniman, CEO of Issue One, a bipartisan organization promoting political reform and election integrity.
“Now, in the last few years, we in America are having to defend assaults on our elections from both domestic and foreign forces. We just don’t have the infrastructure or the history to do it at scale because we’ve never had to face threats this severe in the past,” said Penniman, who took part in the exercise.
“We know a hurricane is eventually going to hit our elections,” said Penniman. But in the exercise, “because patterns of working together haven’t formed, few people understood exactly how they should be coordinating with others or not.”
In a mock “White House Situation Room” around a long table, participants played assigned roles — including as directors of the FBI, CIA and the Department of Homeland Security — and sifted through the alarming reports from Arizona and Florida and numerous other unconfirmed threats, including a break-in at a postal processing center for mail-in ballots.f
Conferring with the tech companies, players who were “government officials” struggled to determine the facts, who was spreading “deepfakes” and how government agencies should respond. (MSNBC anchor Alex Witt also took part in the exercise, playing the role of president of the National Association of Broadcasters.)
In the exercise, it was unclear initially that photos and video of poll workers tossing out ballots in Miami were fake. The images had gone viral, partly because of a bot-texting campaign by Russia.
Eventually, officials were able to establish that the whole episode was staged and then enhanced by artificial intelligence to make it look more convincing.
Source: Experts warn of deepfake disruptions to the 2024 election