What the GDPR asks of an AI system
The GDPR, Regulation (EU) 2016/679, has applied since 25 May 2018. It says nothing about AI specifically, and that is the point: it applies whenever personal data is processed, whatever processes it. An AI system is not a new legal category. It is a processing operation like any other, and the questions are the ordinary ones.
This article is for operations, IT and compliance leads at companies putting AI into a process that touches customer, patient or employee data. It sits beside our EU AI Act summary, which covers the system. This one covers the data inside it.
Four questions decide almost every case, and they are worth settling before anyone builds anything:
- 01Are you the controller or the processor for this processing?
- 02Which legal basis covers it, and is it written down?
- 03Does any of the data fall under Article 9, the special categories?
- 04Where does the data physically go, and who can reach it there?
Controller or processor: which are you?
You are the controller when you decide why and how personal data is processed. You are a processor when you process it on someone else's instructions. The roles follow the decisions, not the invoices, and in an AI build there are usually three parties.
| Party | Usual role | Why |
|---|---|---|
| Your company | Controller | You decide the purpose: which process the AI supports and whose data it sees |
| Your development partner | Processor | They build and run the system to your specification and act on your instructions |
| The model provider | Processor, or sub-processor | They process the prompts you send under their API terms. Check whether the terms name them a processor or claim a controller role for their own purposes |
Each link in that chain needs an Article 28 processor agreement, in the Netherlands a verwerkersovereenkomst. The one most often missing is the one with the model provider, because the API was signed up for with a company card before anyone asked.
Choosing a legal basis
Every processing operation needs a basis under Article 6. Adding AI to a process you already run does not inherit the old basis automatically, because the basis attaches to the purpose, and an AI step sometimes changes the purpose.
| Basis | When it fits an AI use | What trips it up |
|---|---|---|
| Contract, Article 6(1)(b) | The AI step is needed to deliver what the customer actually asked for | Necessary is read strictly. Making your own work faster is not necessity |
| Legitimate interests, Article 6(1)(f) | Internal efficiency, fraud checks, drafting support, internal search | Needs a documented balancing test. Fails where people would not reasonably expect it |
| Consent, Article 6(1)(a) | Optional features a person switches on themselves | Must be freely given and as easy to withdraw. Rarely valid for employees |
| Legal obligation, Article 6(1)(c) | Retention or reporting that law requires of you | Covers the keeping, almost never the AI analysis itself |
Legitimate interests is the right answer more often than people expect for internal tooling, and it is the one that most often has no paperwork behind it. The balancing test is a short document. Write it once per system and keep it with the record of processing.
What you may put into a model
The practical question in every scoping session. The answer depends on the data and the purpose, not on the model.
| What you want to do | Personal data | Usually workable | What it needs |
|---|---|---|---|
| Search your own policies, manuals and procedures | Rarely | Yes | Check the documents for names and case examples before indexing them |
| Summarise or triage customer emails | Yes | Yes | Contract or legitimate interests, a processor agreement, and a retention limit on the provider side |
| Extract fields from supplier invoices | Limited, usually contact names | Yes | Data minimisation: extract the fields you need, not the whole document, and do not keep the rest |
| Draft replies in a support inbox | Yes | Yes | Tell people AI is involved, and keep a human who can genuinely change the answer |
| Analyse patient or medical records | Yes, Article 9 | Only with an Article 9(2) exception | Explicit consent or a health-care basis in national law, plus a DPIA in almost every case |
| Screen or rank job applicants | Yes | Possible, with real safeguards | DPIA, Article 22 safeguards, and high-risk duties under the AI Act from 2 December 2027 |
| Identify someone from a voice or face | Yes, Article 9 biometric | Rarely | Explicit consent, and an honest look at whether you need identification at all |
| Fine-tune a model on customer data | Yes | Only if the new purpose is compatible | A compatibility assessment, usually a fresh basis and fresh notice. Treat it as a new purpose by default |
Two patterns remove most of the difficulty. Pseudonymise before the data reaches the model when the task does not need the name, and keep the retrieval index separate from the model so you can delete from it. Pseudonymised data is still personal data, but the risk drops and so does the blast radius when something goes wrong.
Health, biometrics and the other special categories
Article 9 covers health data, biometric data used to identify someone, genetic data, racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership, and data about sex life or sexual orientation. Processing it is prohibited unless one of the Article 9(2) exceptions applies, on top of your Article 6 basis.
This catches more companies than expected, because the category follows what the data reveals rather than the field it sits in. A free-text note in a scheduling system that says why an appointment was moved can be health data. A document set at a dentist, a physiotherapist or an occupational health service is health data throughout.
The exceptions that come up most are explicit consent, Article 9(2)(a), and health care provided by or under the responsibility of a professional bound by secrecy, Article 9(2)(h), which depends on national law. In the Netherlands that interacts with the WGBO and the medical duty of confidentiality, which is a separate question from the GDPR and is worth asking your own adviser about.
Does the provider train on your data?
Checked on 16 October 2026
For the major business and enterprise API tiers, the default is that prompts are not used to train the underlying model, and that is usually stated in the terms rather than left to a settings page. Consumer tiers of the same products often behave differently. The tier you are on matters more than the brand on the invoice.
Three things are worth reading in the terms, because they are where the surprises live:
- Training. Whether prompts and outputs feed model improvement, and whether that is on by default.
- Retention. How long prompts are kept for abuse monitoring, who can read them, and whether zero retention is available on your plan.
- Sub-processors. Who else touches the data, where they are, and whether you are told before the list changes.
The European Data Protection Board addressed AI models directly in Opinion 28/2024, adopted on 17 December 2024. It covers when a model can be treated as anonymous, when legitimate interests can support developing and deploying a model, and what follows when a model was trained on unlawfully processed data. It is the closest thing to a settled European position and is worth reading before relying on anonymity as an argument.
When you need a DPIA
A data protection impact assessment is required under Article 35 when processing is likely to result in a high risk to people. Article 35(3) names three cases, and a DPIA is required in at least these:
- Systematic and extensive evaluation of personal aspects based on automated processing, including profiling, where decisions produce legal or similarly significant effects.
- Processing special categories or criminal conviction data on a large scale.
- Systematic monitoring of a publicly accessible area on a large scale.
National regulators publish their own lists of processing that always needs one. In the Netherlands the Autoriteit Persoonsgegevens maintains such a list, and it covers cases that catch ordinary businesses, including large-scale processing of health data and systematic monitoring of employees.
A DPIA is not a formality you do at the end. Done at scoping it is genuinely useful, because it forces the question of whether you need the personal data at all, and that question often makes the build smaller and cheaper.
Automated decisions and Article 22
Article 22 gives people the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects or similarly significantly affects them. It applies unless the decision is necessary for a contract, authorised by law, or based on explicit consent, and even then safeguards are required, including the ability to obtain human intervention and to contest the decision.
Solely is the word that decides it. A human who approves whatever the system suggests is not meaningful intervention. For the safeguard to hold, the reviewer needs the authority, the information and the time to reach a different answer, and in practice that means designing the queue so that disagreeing is possible rather than theoretically allowed.
Where an AI agent takes actions rather than making recommendations, this is the article to check first. Our note on human in the loop for AI agents covers how that is built rather than how it is written down.
Where the data physically goes
Sending personal data outside the EEA is a transfer under Chapter V, and it needs its own ground: an adequacy decision, appropriate safeguards such as standard contractual clauses, or one of the narrow derogations in Article 49. Choosing a European region in a console is a good step and it does not by itself settle the question, because access from outside the EEA is also a transfer.
That is its own subject, and we have written it up separately in EU-hosted AI models: where your data actually goes, including what a European region does and does not cover.
Where to start
- 01List the AI systems you already use, including the ones bought on a card, and name the controller for each.
- 02For each one, write the purpose in a sentence and pick the Article 6 basis that matches it.
- 03Check whether any of the data falls under Article 9. If it does, find the Article 9(2) exception before going further.
- 04Collect the processor agreements, starting with the model provider, and read the training, retention and sub-processor terms.
- 05Run a DPIA where Article 35 or your national list requires one, and do it at scoping rather than at launch.
- 06Record where the data is processed and who can reach it, including support access from outside the EEA.
Most companies find that steps one and two remove more risk than anything technical, because they surface a system nobody had written down. Our AI strategy work starts with exactly that inventory.
Questions about the GDPR and AI
Can I put customer data into ChatGPT or Claude?
On a business or enterprise tier with a processor agreement in place, a documented legal basis and a retention setting you have checked, often yes. On a personal account with no agreement, no. The question is the contract and the basis, not the product.
Does the GDPR apply if the data is pseudonymised?
Yes. Pseudonymised data is still personal data, because it can be attributed to a person with additional information. It lowers risk and it is explicitly encouraged by Article 32, but it does not take the processing outside the GDPR. Only genuine anonymisation does, and that is a higher bar than removing names.
Do I need consent to use AI on customer data?
Usually not. Consent is one of six bases and often the weakest, because it can be withdrawn and must be freely given. Contract or legitimate interests fit most business uses better. Consent becomes necessary mainly for special-category data under Article 9(2)(a).
Who is responsible if the model provider has a breach?
You remain the controller and carry the controller duties, including notifying the regulator within 72 hours where the breach is notifiable. The processor must notify you without undue delay. That obligation lives in the Article 28 agreement, which is why the missing one matters.
How does this relate to the EU AI Act?
They are separate laws that apply at the same time. The AI Act classifies the system by its use and your role as provider or deployer. The GDPR governs the personal data regardless of classification. A minimal-risk system under the AI Act can still breach the GDPR.
Read the full text of Regulation (EU) 2016/679 on EUR-Lex (opens in a new tab), the EDPB guidance pages (opens in a new tab) and, in the Netherlands, the Autoriteit Persoonsgegevens (opens in a new tab). This article is general information and is not legal advice. Guidance and rules can change, so check your situation with your legal adviser.