Deepfakes, Synthetic Media, and the Misinformation Crisis

The Video Call That Cost $25 Million

In January 2024, a finance employee at a multinational company in Hong Kong received an email that appeared to come from the company's CFO requesting a confidential financial transaction. The employee was skeptical — it had the hallmarks of a phishing attempt. So when an invitation arrived for a video conference with the CFO and several other senior executives, seeing their faces and hearing their voices was reassuring. The employee transferred $25 million.

No one on that call was who they appeared to be. Every person the employee saw — the CFO, the other executives — was a deepfake. Faces and voices had been synthesized from publicly available recordings of the real people and rendered in real time, convincingly enough to pass a live video call. Hong Kong police confirmed the case in February 2024 and described it as the largest known deepfake fraud incident at that time.

This illustrates something important about how deepfakes cause harm: the most dangerous property is not that they are technically perfect — they often are not. It is that they are good enough, fast enough, and cheap enough to deceive most people under realistic conditions.

What Makes a Deepfake

The term "deepfake" combines "deep learning" — a type of machine learning that uses multi-layered neural networks — and "fake." The underlying technology trains on large amounts of video and audio data to learn how a specific person's face moves, how their voice sounds, and how their expressions change. Once trained, the model can generate new content: placing that person's face in video they never appeared in, synthesizing their voice saying words they never said, or creating entirely new footage of a person who does not exist.

Early deepfakes required significant computing resources and technical expertise. By 2024, multiple mobile applications could produce convincing face-swap videos in minutes. The barrier to creation has effectively collapsed.

What has not kept pace is detection. A 2025 study by identity verification company iProov found that only 0.1% of participants correctly identified all fake and real media they were shown. Human subjects correctly identified high-quality deepfake videos only 24.5% of the time — approximately what you would expect from chance. Unaided human judgment is not a reliable defense against modern synthetic media.

Three Categories of Harm

1. Financial Fraud

The Hong Kong case is the most dramatic single instance on record, but it represents a pattern. A 2025 survey of fraud prevention professionals found that 46% had encountered synthetic identity fraud using AI-generated content, 37% had encountered voice deepfakes, and 29% had encountered video deepfakes. Deloitte estimated in 2024 that generative AI-enabled fraud would drive U.S. financial losses from $12.3 billion in 2023 to $40 billion by 2027 — a 32% compound annual growth rate.

Voice cloning is growing fastest: a convincing voice clone can be generated from as little as three seconds of audio. Criminals use cloned voices to impersonate family members in emergency scam calls ("grandparent scams"), impersonate executives requesting internal fund transfers, and bypass voice-based authentication systems used by banks and call centers. Unlike video deepfakes, voice-only attacks require no visual processing — making them faster to produce and harder to anticipate.

2. Non-Consensual Intimate Imagery

By volume, the most prevalent misuse of deepfake technology is the generation of non-consensual intimate imagery (NCII) — synthetic sexual images or videos depicting real people without their knowledge or consent. Targets are overwhelmingly women, spanning public figures and private individuals alike.

In January 2024, AI-generated sexual images of Taylor Swift spread rapidly across social media platforms, accumulating tens of millions of views before removal. The platforms' removal process took significantly longer than the images' initial spread, illustrating the asymmetry between generation and moderation. The episode drew congressional attention in the United States and accelerated federal and state legislative efforts.

Documented harms from NCII extend well beyond reputational damage. Consequences include job loss, family breakdown, stalking, and in some cases coercive demands — for payment or real sexual contact — in exchange for the images' removal. The harm falls disproportionately on women, on people in public-facing professions, and on communities where such images carry heightened social consequences.

3. Journalist and Public Figure Targeting

Between December 2023 and December 2025, Reporters Without Borders (RSF) documented and analyzed 100 cases of journalists targeted by deepfakes across 27 countries. The findings were stark: 74% of the targeted journalists were women. Harms ranged from defamation and financial fraud to threats against physical safety. Deepfakes are increasingly deployed as a tool to silence and discredit reporters — particularly investigative journalists working in contexts where a fabricated video showing a journalist accepting bribes or making extremist statements can be career-ending and physically dangerous.

Political deepfakes follow a similar pattern. In Slovakia's 2023 elections, audio recordings purporting to show a candidate discussing vote-buying circulated in the final days before voting — too recently for effective fact-checking or public correction. Deepfake videos of politicians circulated during Bangladesh's 2024 elections. The threat to democratic processes is less about individual deepfakes definitively changing outcomes than about degrading baseline trust in all political media — a kind of informational pollution where uncertainty about authenticity suppresses confidence in legitimate content alongside fabricated content.

The Scale of the Problem

Deepfake creation accelerated dramatically between 2023 and 2025. Researchers estimated that the number of deepfake files in circulation grew from approximately 500,000 in 2023 to a projected 8 million in 2025 — a 1,500% increase in two years. In the first quarter of 2025 alone, researchers documented 179 deepfake incidents affecting organizations, which surpassed the total count for all of 2024.

A deepfake attempt was estimated to occur every five minutes during 2024. Detection tools exist but require computational resources and expertise that most end users do not have. The gap between creation capability and detection capability is not closing — it is widening, as generative models improve faster than detection models can adapt.

What the Law Is Doing

The legal response to deepfakes is fragmented but accelerating across several jurisdictions:

• EU AI Act Article 50 (enforceable August 2026). Providers of AI systems that generate or manipulate images, audio, or video must mark their outputs in a machine-readable format indicating that the content is artificially generated. The European Commission was finalizing a Code of Practice on AI-generated content labeling in mid-2026, establishing a common disclosure standard including an interim two-letter "AI" icon and shared technical metadata format. Exemptions exist for artistic or satirical expression with adequate accompanying disclosure.
• U.S. DEFIANCE Act (signed July 2024). Created a federal civil cause of action for victims of non-consensual AI-generated intimate imagery. Victims can sue in federal court for damages without needing to prove criminal intent — a lower threshold than criminal statutes. More than 20 U.S. states had passed their own NCII deepfake laws by 2025, with penalties ranging from civil damages to criminal charges.
• U.S. election deepfake laws. More than a dozen U.S. states had passed laws by 2025 restricting the use of AI-generated media in political advertising or requiring disclosure when AI-generated content is used. Federal legislation was pending but had not passed as of mid-2026.
• Platform policies. Major social media platforms require disclosure when AI-generated video is uploaded and prohibit non-consensual synthetic intimate imagery. Enforcement speed remains inconsistent — the Taylor Swift case showed that policy and enforcement can diverge significantly, with removal taking far longer than viral spread.

What You Can Do

• Establish a verification code. For high-stakes communications — financial authorizations, emergency requests, identity confirmation — agree on a shared verification phrase with family members and colleagues that would not appear in any public recording. This is the simplest practical defense against voice and video impersonation.
• Treat urgency as a red flag. Deepfake fraud almost always involves artificial time pressure: a CFO who needs a transfer completed immediately, a relative who needs emergency money now. Urgency is designed to prevent verification. Always slow down and verify through a separate channel you initiate yourself — call a known number, not the number in the message.
• Know your legal options if targeted. If non-consensual synthetic images of you are created or distributed, the DEFIANCE Act (U.S.), state NCII laws, and equivalent legislation in the EU and UK create legal remedies. The Cyber Civil Rights Initiative provides guidance and support for victims navigating removal requests and legal action.
• Use verification tools before sharing. When a video or image is designed to provoke an emotional reaction, pause before sharing. Reverse image search tools (Google Images, TinEye) and video verification tools (InVID/WeVerify) can reveal whether media has appeared in other contexts — a common indicator of manipulated or misattributed content.
• Support mandatory labeling requirements. Machine-readable labeling of AI-generated content — as required by EU AI Act Article 50 — makes automated detection possible at scale. Platforms and fact-checking organizations can use these signals to flag synthetic media before it spreads. Advocating for extension of these requirements beyond the EU is one of the most structurally meaningful positions available to individuals concerned about synthetic media.

The next lesson examines what happens when AI moves from identifying and deceiving individuals to making consequential decisions about them. When AI systems determine who gets a loan, who goes to prison, or who receives a diagnosis — what safeguards exist, and what happens when they fail?