EU AI Act (Regulation 2024/1689)

The prohibited list draws a hard line that eliminates entire AI product categories. Social scoring systems (like those deployed in parts of China) are categorically banned. Emotion recognition in workplaces is banned — eliminating a product category that several companies have been developing. Untargeted facial scraping criminalises the methodology that many facial recognition companies used to build their training datasets.

The manipulation prohibition (5a) is broad and vaguely defined. "Purposefully manipulative or deceptive techniques" that "materially distort behaviour" could capture recommendation algorithms, targeted advertising, and AI chatbots designed to influence user behaviour. The "significant harm" qualifier provides some limiting effect, but the subjective nature of "manipulation" creates compliance uncertainty. AI developers must assess whether their systems could be characterised as "manipulative" — a term that is not defined in the regulation.

The line between prohibited and high-risk is where the compliance burden concentrates. Many AI use cases that are close to prohibited (e.g., biometric identification not in real-time, emotion recognition outside workplace/education) fall into the high-risk tier instead — where they are legal but subject to extensive requirements. Companies must carefully classify their systems to avoid accidentally crossing into prohibited territory.

The risk-tier system gates AI development at the classification stage. Whether your AI system is classified as "minimal," "limited," or "high-risk" determines whether you face no obligations, transparency obligations, or the full weight of the Act's requirements (risk management, data governance, logging, transparency, human oversight, accuracy, conformity assessment, registration, post-market monitoring). Getting classification wrong — classifying a high-risk system as minimal — exposes the provider to penalties up to €15M or 3% of global turnover.

Annex III's use case list is the regulatory net. The eight categories capture the most commercially significant AI applications: hiring/recruitment, credit scoring, insurance, education, law enforcement, critical infrastructure. Any company building AI products in these domains faces the full high-risk compliance regime. The Commission's power to expand Annex III via delegated acts (Art 7) means the net can widen without legislative action — creating regulatory uncertainty for AI developers whose products are currently outside the high-risk tier.

The derogation (Art 6(3)) is narrow and easily lost. AI systems that "perform a narrow procedural task" or "improve a previously completed human activity" can escape high-risk classification — but only if they don't perform profiling. Since many modern AI systems inherently profile users, the derogation's practical value is limited. The "narrow procedural task" language also conflicts with the reality that AI systems are often deployed precisely because they handle complex, non-procedural tasks.

The data governance requirements (Art 10) create a documentation burden that favours companies with established data pipelines. AI developers must document: data collection processes, data origins, original purpose of collection for personal data, annotation and labelling procedures, bias examination methodologies, and data gap identification. For companies that built their training datasets through web scraping with minimal documentation, retroactively creating this documentation is infeasible. Only companies that designed their data pipelines with provenance tracking from the start can comply efficiently.

The bias detection requirement (Art 10(2)(f-g)) is the Act's most technically demanding provision. AI developers must examine datasets for biases "likely to affect health and safety of persons, have a negative impact on fundamental rights, or lead to discrimination prohibited under Union law" — and implement measures to "detect, prevent and mitigate possible biases." This requires: (1) defining what bias means in the system's context, (2) measuring it across protected attributes, (3) implementing mitigation strategies (reweighting, augmentation, adversarial debiasing), and (4) documenting the process. For large language models, bias detection across all possible use cases is an open research problem.

Human oversight (Art 14) with a "stop" button requirement creates an architecture mandate. High-risk AI systems must include a mechanism for human operators to halt the system — a "stop button or similar procedure." This is a design requirement, not a policy requirement. AI systems that operate autonomously or in real-time pipelines (e.g., automated trading, content moderation at scale) may need fundamental architectural changes to incorporate a meaningful stop mechanism.

The cybersecurity requirements (Art 15(5)) specifically address AI-specific attack vectors. Data poisoning, model poisoning, adversarial examples, and model evasion are called out by name. This means AI developers must implement and document defenses against these attacks — which is an active research area, not a solved problem. For many AI companies, the state of the art in adversarial robustness lags far behind what Art 15 appears to expect.

The conformity assessment system favors large companies by design. For most high-risk AI systems (Annex III points 2-8), the assessment is based on "internal control" — meaning the provider self-assesses and self-certifies compliance. This appears to reduce burden, but the self-assessment must be backed by technical documentation, a quality management system, and post-market monitoring that must withstand regulatory inspection. Only companies with dedicated compliance teams, legal counsel, and quality management infrastructure can produce documentation that will survive scrutiny. A startup building an AI hiring tool faces the same documentation requirements as a multinational — but without the personnel to produce it.

Notified body involvement creates a bottleneck for biometric AI. Biometric systems (Annex III point 1) may require third-party assessment by a notified body. The number of notified bodies qualified to assess AI systems is extremely limited. The assessment process is time-consuming and expensive — creating a de facto gate that slows time-to-market for biometric AI products. Large companies with existing regulatory affairs teams (e.g., companies already in medical device conformity assessment) have a structural advantage.

The CE marking creates consumer trust signaling that benefits incumbents. CE marking signals compliance to procurers and consumers. Companies that can afford the conformity assessment process and achieve CE marking early will have a market advantage — the compliance cost becomes a barrier to entry for new competitors. This is the "regulatory moat" effect: compliance costs that large companies can absorb become competitive advantages against smaller entrants.

Self-assessment with regulatory inspection creates enforcement asymmetry. The internal control procedure means providers self-certify — but regulators can inspect documentation post-market. Large companies with legal teams can prepare documentation that withstands inspection. Smaller companies may produce technically compliant but legally vulnerable documentation. The enforcement risk falls disproportionately on those without regulatory affairs expertise.

The GPAI regime (Art 51-55) is the world's first binding regulation of foundation models. Every provider of a general-purpose AI model — including OpenAI, Anthropic, Google, Meta, Mistral, and any company placing a foundation model on the EU market — must comply with Art 53: maintain technical documentation, provide downstream documentation, implement copyright policy, and publish a training content summary. For models with systemic risk, additional obligations include adversarial testing, systemic risk assessment, and incident reporting. This is the regulatory layer that directly governs the frontier AI industry.

The training content summary requirement (Art 53(1)(d)) forces transparency about training data. GPAI providers must publish "a sufficiently detailed summary about the content used for training" using a template provided by the AI Office. This requirement directly addresses the opacity of training datasets — and creates a disclosure obligation that copyright holders and privacy advocates can use to assess whether their works or personal data were used. The level of detail in the AI Office's template will determine whether this is a meaningful transparency mechanism or a box-ticking exercise.

The systemic risk classification (Art 51) creates a two-tier GPAI regime. Models above the compute/parameter/reach thresholds face additional obligations. The 10,000 registered business users presumption (Art 51 criteria) means any commercially successful GPAI model will likely cross the threshold. The Commission can also designate models ex officio or based on scientific panel alerts — meaning the AI Office has discretionary power to escalate any model to the systemic risk tier. This creates regulatory uncertainty: a model that is below thresholds today could be designated tomorrow.

The open-source exemption (Art 53(2)) is significant but limited. Open-source GPAI models that publish their weights, architecture, and usage information are exempt from the documentation obligations — unless they present systemic risk. This creates a regulatory incentive for open-weight releases, but the systemic risk exception means that the most capable open-source models (e.g., Llama-class models) may still face full obligations. The boundary between "open source" and "systemic risk" will be one of the most contested regulatory questions in AI governance.

The AI Office is the GPAI regulator — and it has no US equivalent. The AI Office within the European Commission has direct authority over GPAI model providers, including those in third countries (via the authorised representative requirement, Art 54). US AI companies must appoint an EU-based authorised representative who maintains technical documentation and cooperates with the AI Office. This is the extraterritorial enforcement mechanism — see the next section for its full implications.

The extraterritorial reach is the Act's most significant feature for US AI developers. Art 2(1)(c) extends the Act to providers in third countries whose AI system output is used in the EU. This means a US-based AI company whose API is called by a user in the EU — or whose model output is embedded in a product used in the EU — is subject to the Act. The "output" criterion is broad: it could capture a US company's AI model used by a European customer through an API, a US model fine-tuned by a European deployer, or even a US model whose outputs are consumed indirectly through a downstream product.

The authorised representative requirement (Art 54) creates a physical EU presence obligation. Third-country GPAI providers must appoint an EU-based representative who can be served by authorities and who maintains documentation. This is not a paper formality — the representative must verify compliance, maintain records for 10 years, and must terminate the relationship (and inform the AI Office) if the provider acts contrary to obligations. This creates a regulatory foothold in the EU for enforcement against non-EU companies.

Penalties exceed GDPR — and GDPR enforcement was significant. The €35M/7% cap for prohibited practice violations exceeds GDPR's €20M/4% maximum. For a company with €50B global revenue, the maximum AI Act fine is €3.5B vs. GDPR's €2B. The 7% figure is deliberately designed to be painful even for the largest tech companies. The SME cap (lower of percentage or fixed amount) provides some protection for startups, but does not exempt them from compliance — only from the most devastating fine levels.

The practical effect on US AI developers: Any US AI company with EU users must: (1) classify its AI systems under the risk tiers; (2) for high-risk systems, implement the Art 8-15 requirements and undergo conformity assessment; (3) for GPAI models, comply with Art 53-55 documentation and transparency obligations; (4) appoint an EU authorised representative for GPAI models; (5) implement post-market monitoring and incident reporting. Companies that ignore the Act face fines that scale with global revenue — not EU revenue. The Act uses worldwide turnover as the fine base, meaning a US company's entire global revenue is the penalty denominator.

Contrast with the US approach. While the EU regulates AI through binding legislation with extraterritorial reach, the US approach under EO 14409 uses "voluntary" frameworks, classified benchmarks, and procurement-mediated compliance. The EU's model creates legal obligations enforceable by fines; the US model creates structural pressure enforceable by market access. AI developers operating in both jurisdictions face the dual challenge of EU legal compliance and US structural compliance — see our GDPR analysis for the underlying data protection layer that intersects with both.