EU enforces AI Act, deepfake fraud hits $3.7 billion

The EU has started to enforce its AI Act, requiring that AI content, including deepfakes, must be labeled and traceable. New data suggests that deepfake-related fraud has reached about $3.7 billion worldwide since 2020, with many organizations reporting at least one attack in the past year. Regulators are pushing for tools that can track the origin of digital content, and more devices are now including features to verify authenticity. These rules and tools may help slow the rise of deepfake scams, but attacks appear to be growing more common and sophisticated.

As the EU enforces its AI Act, concerns about deepfake fraud are accelerating the race for content provenance tools. This weekly watchlist tracks key policy and technology developments in synthetic media.

What does the EU AI Act's August 2024 enforcement mean for deepfake labeling?

The enforcement of the EU AI Act mandates that all deepfakes be clearly labeled as artificial. Furthermore, all AI-generated or modified content must include machine-readable metadata, such as C2PA credentials, to ensure its origin can be traced across different platforms, shifting focus from simple warnings to technical accountability.

The European Commission's enforcement notice specifies that providers must ensure AI-generated content is identifiable, with particular emphasis on deepfakes and text published to inform the public on matters of public interest. The Commission also references a voluntary Code of Practice on Marking and Labelling of AI-generated Content, which provides standardized icons and guidance for disclosing artificial content.

Critically, the EU's approach moves beyond simple visible disclaimers toward technical traceability - requiring interoperable standards that can persist as content travels across platforms.

How significant are deepfake fraud losses?

Documented losses from deepfake-related fraud are growing significantly, with organizations increasingly reporting incidents. Industry reports indicate that a majority of organizations have faced deepfake attacks, with many incidents targeting businesses and involving financial fraud.

The most consequential cases demonstrate how synthetic media bypass traditional security assumptions:

Arup video-call scam: A finance worker transferred significant funds after joining a video call where multiple "colleagues," including senior executives, were deepfakes
Hong Kong corporate scam: Deepfake video calls impersonating coworkers resulted in substantial losses

These incidents show deepfakes have evolved from proof-of-concept demonstrations to sophisticated social engineering tools embedded in normal business workflows.

What provenance tools are seeing mainstream adoption?

C2PA (Coalition for Content Provenance and Authenticity) and its consumer-facing implementation, Content Credentials, are moving from niche applications toward broader adoption across consumer devices and platforms. Recent developments include:

Major device manufacturers exploring native C2PA credential support
The C2PA Conformance Program establishing validation standards for implementations
Updated specifications reflecting production use by artists, journalists, filmmakers, and AI developers

OpenAI has expanded provenance capabilities, with supported audio now including SynthID watermarking, and the company has introduced API access for verification - allowing organizations to embed provenance checks into their own workflows.

This creates a three-layer approach becoming more standard: cryptographic metadata (C2PA), durable watermarking (SynthID), and programmable verification tools.

What risks do deepfakes pose beyond financial fraud?

While financial losses draw attention, deepfakes create systemic trust erosion across multiple domains:

Identity and security systems: Deepfakes can defeat selfie-versus-ID onboarding checks and biometric verification, expanding fraud from social engineering into account-opening systems.

Democratic processes: Fabricated political videos have circulated during election cycles, demonstrating risks to voter trust and information integrity.

Workplace and education: Reports document students using deepfake technology to create harassing and sexually explicit content of classmates and teachers - a new vector for cyberbullying with complex legal implications.

Operational damage: Even unsuccessful attacks incur investigation costs, reputational harm, and degraded trust in video calls and voice communications that modern remote work depends upon.

The common thread: deepfakes exploit contextual trust rather than technical vulnerabilities, making them particularly difficult to defend against with conventional security tools.

How are U.S. AI policies developing compared to the EU?

The U.S. landscape remains fragmented and state-driven while federal frameworks develop slowly. Recent state-level developments include:

States exploring requirements for independent third-party safety audits of frontier AI models
New regulations targeting AI chatbots specifically to protect minors
Federal frameworks continuing to develop amid varying state requirements

This contrasts with the EU's unified approach. Industry is nonetheless moving toward C2PA/Content Credentials as a de facto compliance standard, driven partly by EU market requirements and partly by platform self-regulation.

For multinational organizations, this creates a dual compliance reality: EU rules provide clear technical specifications, while U.S. operations require monitoring evolving state developments without federal preemption certainty.