Control over AI
Blog
AI governance 8 min read

An AI policy for healthcare: patient data, confidentiality and the chatbot

A healthcare AI policy works when it says which tool is allowed on which account, what never goes into a prompt, and what staff do with a tool nobody has decided on.

Abstract frame above a still lake
Quick answer

An AI policy for a healthcare organisation answers the questions staff actually have at work: which AI tools are allowed, on which account, which patient data never goes in, what to do with a tool nobody has decided on yet, and who assesses a new one. Confidentiality and the GDPR make the policy more specific rather than stricter. Health data is special category data, and the GDPR exception for care relies on processing by or under the responsibility of someone bound by professional secrecy. A chatbot the organisation has no agreement with sits outside that.

01

A European regulator received a breach notification from a GP practice where an employee entered patients' medical data into an AI chatbot, against the agreed rules

02

The regulator treats such input as a data breach, and says use that policy allows is often not lawful either

03

Health data falls under Article 9 of the GDPR; the care exception depends on professional secrecy

04

The account is the first rule: a free account and a business licence are two different tools

05

A tool without a decision gets its own status, an instruction and a named reviewer

06

A policy only works if it appears when someone opens the tool

A practice nurse has ten minutes before the next clinic and needs to turn a three-page discharge letter into a short handover for a colleague covering tomorrow. She opens ChatGPT on her personal account, pastes the letter in and asks for five lines. It works, and nobody at the practice knows.

The nurse is made up. The situation is not. In August 2024 the Dutch data protection authority wrote about a breach notification from a GP practice where an employee had entered patients' medical data into an AI chatbot. That is why an AI policy in healthcare needs to be more than a line in the information governance handbook. This article covers what such a policy needs when the reader works in care. The general questions every AI policy answers are in writing an AI policy.

Why "no patient data" is not a policy

In healthcare almost every piece of text is about a patient: a referral, a handover, an email to the pharmacy, the notes from a multidisciplinary meeting. A rule that bans everything with patient data keeps AI tools out of nearly all the work. Then the nurse does what she did, and the work moves to an account the organisation cannot see and has no agreement with.

A policy that works says what can be done and how. Summarising a clinical guideline is fine. Rewriting a patient leaflet in plain English is fine. Summarising a discharge letter happens only in a tool the organisation has approved for it, and even there without the name, date of birth or record number. That is an instruction someone can follow with ten minutes to spare.

What a regulator saw at a GP practice

On 6 August 2024 the Dutch data protection authority published Caution: use of AI chatbot may lead to data breaches (checked 27 October 2026). In one of the breaches reported to it, an employee of a GP practice had entered medical data of patients into an AI chatbot, contrary to the agreements. The regulator calls medical data highly sensitive and notes that the law gives it extra protection for a reason.

Two points in that notice matter most for a healthcare organisation. If an employee uses a chatbot against the employer's rules and enters personal data, that is a data breach, and notifying the regulator and the people affected is mandatory in many cases. If the use is part of the organisation's policy, it is not a breach, but it is often not permitted by law. The regulator says organisations need to prevent both.

A policy that only says "be careful" prevents neither. The authority asks organisations to make clear whether staff may use chatbots at all, and if so, which data they may and may not enter. What to do when it goes wrong anyway is in reporting a data breach through ChatGPT.

Confidentiality follows the data into the chatbot

Health data is special category data under Article 9 of the GDPR (checked 27 October 2026), and processing it is prohibited unless an exception applies. The exception that carries most clinical work is point (h) of paragraph 2: processing for diagnosis, care or treatment. Paragraph 3 attaches a condition. The data has to be processed by or under the responsibility of a professional bound by professional secrecy, or by someone else under a duty of secrecy. The UK GDPR keeps the same condition in Article 9 (checked 27 October 2026), with domestic law in place of EU or member state law.

That is the link between data protection and the confidentiality every clinician already knows. A chatbot supplier the organisation has no agreement with is outside that circle of responsibility. So the policy cannot stop at "is this tool allowed?". It also has to answer "what may a clinician put into this tool without breaking confidentiality?".

The questions a healthcare AI policy answers

QuestionA usable answer in healthcare
Which tools are allowed?A list with names and the account: the organisation's business licence is a different tool from the same chatbot on a personal account
For what work?Per tool, the tasks that fit (summarising a guideline, rewriting a leaflet) and those that do not (clinic notes, a referral with a name on it)
Which data never goes in?Name, date of birth, national ID or NHS number, address, record number, and anything else that identifies the patient, including a rare condition
What about a tool with no decision?No patient or staff data, and this is where you request a review
Who reviews, and how fast?The data protection officer with information security and a clinician, within a fixed period
Who does it cover?Everyone who handles patient data: clinicians, receptionists, admin staff, students and locums
How do people hear about a change?When they open the tool, not in a newsletter
How does the organisation know it holds?By setting the list in the policy against the tools actually in use, as counts, without names

Most policies skip the last three rows. They are the ones that decide whether the policy still says anything useful in six months, about the tools people use by then.

The account decides what happens to the text

The same chatbot can be two different services. On a free or personal ChatGPT account, OpenAI may use the content to train its models (checked 27 October 2026) unless the user turns that off. There is no data processing agreement, and nobody in the organisation can manage members, settings or shared links. On the business versions of ChatGPT, OpenAI does not train on the organisation's data by default (checked 27 October 2026). The differences per supplier are in free vs business AI accounts.

A policy that says "ChatGPT is allowed" without naming the account allows the nurse's personal account too. Write it down precisely. In the example organisation used in this article, Microsoft Copilot on the work account is the approved alternative, ChatGPT is allowed with the condition that no patient data goes in, and DeepSeek is not allowed.

A business account settles the supplier side of the GDPR: training, administration and the data processing agreement that Article 28 requires. It does not settle confidentiality by itself. Whether patient data belongs in a business tool is a question for a DPIA and the data protection officer, and the answer goes into the policy.

What never goes into a prompt, business account or not

The general list is in what not to share with AI. In healthcare it comes down to one rule of thumb: if someone could recognise the patient from the input, it comes out before it goes in. That covers the obvious identifiers, and also combinations that are enough in a small practice, such as an age, a village and a rare condition.

More is possible than people expect. Summarising a guideline, rewriting a leaflet, asking about a process, drafting a general letter with no patient in it. Meetings follow the same logic: an AI notetaker has no place in a case discussion.

What happens to a tool nobody has decided on

A home care provider has a policy with two allowed tools. One community nursing team has been using a translation app for a few months to message relatives of the people they care for. The policy says nothing about it, so nobody asked, and the messages contain names and care arrangements.

Once that comes to light, the policy needs a fourth status: no decision yet. Every tool that is not on the list lands there automatically, with an instruction anyone can follow: no patient data, and report the tool to the data protection officer. The DPO reviews it within the agreed period and records the supplier's answers with a date. Why that status is the honest one is in four statuses for AI tools.

Information security belongs here too. An AI service that patient data can end up in is a supplier service and belongs in your risk assessment. For Dutch care providers that means NEN 7510; how such security baselines relate to AI tools is in BIO, NEN 7510 and AI.

How the policy reaches staff at the moment they need it

The policy sits on the intranet, the chatbot sits in the browser, and the nurse needed her answer while the text box was open. A policy that only lives in a document arrives too late.

A decision works when it appears where the tool is opened. With BeeSensible the organisation records a decision per tool under Decide, and that decision shows in the browser when someone opens the tool. An allowed tool shows nothing. A tool with limited use shows the condition the organisation wrote, such as "no patient data". A tool that is not allowed shows the approved alternative as the first button; whether staff can still continue after that is a setting the organisation chooses. A tool someone opens that has no decision yet lands in the admin's Inbox.

When someone pastes text into a supported AI tool, health details, names and ID numbers are highlighted as they type. What happens next is what the organisation set: the employee removes it, or it is masked on paste or on send. How that works is under Help. The organisation sees counts and patterns, never what anyone typed. What BeeSensible does for care providers is on the healthcare page.

Set your current policy against the table above and see which rows it leaves open. Usually that is the account, the tool without a decision and the moment of typing, which are exactly the three questions the nurse had.

FAQ

Common questions

Can healthcare staff use ChatGPT at work?

That is the organisation's decision, not the individual's. Without agreed rules the answer for anything with patient data is no: the Dutch data protection authority treats entering personal data against the employer's rules as a data breach. If the organisation allows a tool, the policy says which account, for which work, and which data never goes in. A general question with nothing that points to a patient is different from a discharge letter.

Is it a data breach if staff paste patient data into a chatbot?

According to the Dutch data protection authority, yes, if the employee does it against the employer's rules: someone then gets access to personal data without that being permitted. It adds that notifying the regulator and the people affected is mandatory in many cases. Where the use is part of the organisation's own policy, it is not a breach, but it is often not permitted by law either.

Is a business ChatGPT account enough for patient data?

No. A business account settles training, administration and a data processing agreement, which covers part of the GDPR. It does not settle professional confidentiality: the data still goes to a party outside the circle of people involved in the patient's care. The policy therefore also says which data never goes into the allowed tool, and a DPIA and the data protection officer decide whether the rest fits.

Which patient data should never go into a prompt?

Anything that lets someone recognise the patient: name, date of birth, NHS or national ID number, address, record number, and also a rare condition or a recognisable situation in a small community. A clinic letter, a referral or notes from a case discussion do not go in as they are. What usually works is more general: summarising a guideline, rewriting a leaflet in plain language, a question about a process.

Who should assess a new AI tool in a healthcare organisation?

A named role, with a deadline. Often the data protection officer, together with information security and a clinician who knows the work. The questions to the supplier are always the same: where is data processed, what is kept and for how long, is input used for training, who are the sub-processors, and is there a data processing agreement. Until there is a decision, the tool stays on no decision yet.

Does this apply under the UK GDPR?

Yes. The UK GDPR keeps Article 9, including the condition that health data processed for care is handled by or under the responsibility of a professional subject to an obligation of secrecy. The questions in this policy are the same; the regulator to notify after a breach is the ICO rather than an EU authority.

See how BeeSensible works

Sensitive data gets a marking while you type, in the tools your team already works in. You decide what happens before you send.

Portrait of Rens Timmermans from BeeSensible

Prefer someone to walk you through it?

Rens, BeeSensible

I'm happy to give you a twenty-minute tour. Leave your details and I'll find a time that suits you. You're also welcome to call or email.

Prefer to talk to someone first?

Want a live demo, or a quote for 100+ users? Leave your details and we'll be in touch within one working day.