Amnesty: Generative AI Is 'Unlawful by Design' Due to Web Scraping

Amnesty International declared that standalone generative AI systems based on unlawful web scraping are 'unlawful by design' and fundamentally incompatible with international human rights law. The report 'Unlawful by design: Exposing the human rights costs of generative AI' was released by Amnesty International on May 28 or May 31, 2026, and names major tech firms including OpenAI, Google, Meta, and DeepSeek for their privacy-invasive data collection practices (Amnesty report).

The evidence presented connects the data collection methods for AI text and image generators to clear violations of international human rights treaties, including the right to privacy and protections against discrimination. This human-rights framing, highlighted in a MyPrivacy.blog analysis, has significantly shifted the policy debate from simple risk management toward considering an outright ban on such systems (analysis).

Regulatory ripple effects

Amnesty International's report concludes that generative AI systems are inherently unlawful because their training depends on the mass scraping of web data without consent. This process captures personal information and copyrighted material, which the organization argues is a systemic violation of privacy and other fundamental human rights.

The report's findings are influencing legislation and policy discussions globally:

• European Union: Amnesty International is calling for states to prohibit standalone generative AI systems built using unlawful web scraping, but the provided sources do not confirm that specific amendments to a draft AI Act are currently being debated to explicitly ban such models.
• United States: US AI regulation discussions focus on NIST frameworks or FTC actions, with Amnesty International advocating for prohibitions on non-consensual data scraping for AI.
• United Nations: The briefing's findings on the environmental and climate-related rights impact of high-energy model training are now being cited in submissions to the UN Human Rights Council.

What the report asks for

Amnesty's call to action goes beyond simple transparency, demanding concrete measures:

• Governments: Prohibit generative AI models that are built using unlawful web scraping.
• Companies: Immediately cease the non-consensual collection of personal data for the purpose of training AI models.
• Regulators: Mandate comprehensive human rights impact assessments for AI, including environmental metrics like energy and water usage.

Interaction with ongoing litigation

The report's call for a ban is amplified by a wave of high-profile litigation targeting AI data practices. Kadrey v. Meta is an active copyright infringement case involving allegations of unauthorized use of Kadrey's works to train AI models. Andersen v. Stability AI is a real case involving allegations of unauthorized use of text and code for AI training, further highlighting the legal risks facing AI companies.

Data provenance initiatives

In parallel with regulatory pressure, technical solutions are emerging to ensure lawful data sourcing. Various initiatives are working to develop tools and frameworks that would enable developers to trace dataset origins and verify licensing before use, practices that align with responsible AI development principles outlined in regulatory frameworks.

Outlook noted in policy fora

Policy analysts suggest these converging factors - Amnesty's advocacy, costly lawsuits, and emerging provenance tools - are pushing AI governance toward a supply-chain verification model. The US NIST Generative AI Risk Management Framework emphasizes risk mitigation and responsible AI development rather than outright prohibition.

The future of this regulatory landscape continues to evolve as the EU AI Act, which was adopted in 2024 and is currently in the implementation phase, addresses various AI practices. The EU AI Act prohibits 'untargeted scraping of the internet or CCTV material to create or expand facial recognition databases'. Lawmakers must balance Amnesty's human-rights-based arguments against industry claims that such restrictions will stifle innovation. The outcome in Brussels is expected to heavily influence whether a global standard emerges that either bans certain AI practices or creates alternative frameworks for accountability.

---

What exactly does Amnesty mean when it says generative AI is "unlawful by design"?

Amnesty's 2026 briefing argues that the core training method - mass, non-consensual web scraping - is baked into the architecture of today's large models. Because this scraping routinely vacuums up copyrighted works, personal photos, and private posts, the resulting systems violate privacy, non-discrimination, and expression rights under international law before a single prompt is answered. In short, the illegality is not a bug; it is the default business model.

Which legal rights does Amnesty say are being breached?

The report points to three main areas covered by international human-rights treaties:

• Privacy - scraping personal data without consent
• Non-discrimination - training data reproduces race and gender bias
• Freedom of thought and expression - outputs can distort or suppress information

Amnesty concludes that these breaches are systemic and foreseeable, not incidental.

How have governments reacted since the briefing appeared?

Amnesty International is calling for governments to prohibit rather than merely regulate such models:

• The EU AI Act prohibits 'untargeted scraping of the internet or CCTV material to create or expand facial recognition databases'
• In the United States, AI regulation discussions focus on NIST frameworks or FTC actions, with ongoing debates about federal oversight of AI training practices
• Various jurisdictions are considering privacy-based claims against AI firms, with legal frameworks evolving to address these concerns

Are there practical tools to source training data lawfully?

Various groups are working to develop auditing tools and compliance frameworks that would allow developers to filter datasets by license type and trace creators. Emerging compliance rating systems aim to help teams score datasets on transparency, accountability, and security before training. These tools are gaining attention among enterprise teams that need audit-ready pipelines.

What should corporate buyers ask their AI vendors right now?

• Show a documented data-provenance trail with verifiable licences
• Confirm that no personal data was scraped without consent or legal basis
• Provide a human-rights impact assessment that covers bias, privacy, and environmental cost
• Explain remediation steps if unlawful data is later found inside a model
  Companies that cannot supply this evidence are, in Amnesty's view, shipping products that remain "unlawful by design" and expose customers to regulatory, reputational, and litigation risk.