Tuesday morning at an insurer. Three colleagues in the same department open the same chatbot. The first has a clumsy sentence from a newsletter reworded. The second pastes in a complaint email to get a tidy reply drafted; at the bottom of the email sit a name, a policy number and a bank account number. The third is on the train and summarises an internal report on her phone, through the free account she also uses at home.
Same tool, three different risks. Not because the chatbot behaves differently for each of them, but because most of the risk does not sit in the tool. It sits in what goes in, under which account, and in what happens to the answer afterwards.
What decides the risk when the tool is the same?
An AI tool has properties that are fixed: where the vendor processes, whether it trains on input, how long it keeps conversations. Those properties decide part of the risk, and the organisation assesses them once per tool. How that works is in how do you assess the risk of an AI tool.
The rest is decided at the moment of use, and there are three variables. What you put in the prompt: a general question or a customer file. Which account you work under: the business account the organisation set up, or a free account that was already there. And what you do with the answer: read and adjust it, or forward it as it came. Each of those three can make the risk of a properly assessed tool high after all.
Three colleagues, the same tool
| Who | What goes in | Which account | What happens to the answer | Risk |
|---|---|---|---|---|
| Colleague 1: rewording a sentence | A sentence with no personal data | The department's business licence | Read and adjusted | Low |
| Colleague 2: answering a complaint | A customer's name, policy number and bank account | The department's business licence | Sent to the customer after a quick glance | Personal data processed that the task did not need |
| Colleague 3: summarising a report | An internal report | Free account on a personal phone | Pasted into an email | Confidential text under consumer terms, out of the organisation's sight |
The first colleague uses the tool the way the organisation assessed it. The second uses the same account but gives the tool data it did not need: the reply to a complaint does not get better for including the bank account number. The third has in effect swapped the tool for a different one, because a free account on a personal phone is a different processing operation from the licence at the office, even with the same logo at the top.
What is in your own hands?
Two of the three variables, which is more than it often feels like.
What goes in. The task almost never needs the identifying details. A complaint can be answered with "the customer" instead of a name and without the policy number. A report can be summarised without the names of those present. The test you set yourself: if I would not email this text to a stranger outside the organisation, it does not belong in a chatbot either. What can and cannot go in, with the list, is in what you can and cannot share with AI.
What you do with the answer. A language model produces plausible text, not checked text. A date that is wrong, a condition that reads differently in the policy, a tone that does not fit the customer: the model does not notice, you do, provided you read before you send. The answer is a draft. That applies to the first colleague as much as the second, only the consequences of not reading are bigger with a customer letter.
Then the third variable, the account. That one is only half yours. You can choose to use the account the organisation set up instead of the free one that was already open. But whether that business account exists, and for which tool, is the organisation's decision. Why that decision matters more than the choice between brands is in free versus business AI accounts.
What is not in your hands?
The vendor's terms, whether a data processing agreement exists, and the decision on which tools the organisation allows, allows with limits or would rather not see. That is the organisation's work, and it is no reproach to the third colleague that she used the account she had on the train. If nobody has told her which tool is fine and where to look that up, the free account is the logical choice.
That is also why a list of allowed tools in an email from March does little by September. The decision has to be there at the moment you open the tool, and it has to name the alternative. "Not this one, use that one" is an instruction you can act on. "Not this one" on its own ends the conversation, and the work carries on by phone.
How did it go on at the insurer?
The department had allowed Copilot through its Microsoft licence. An inventory showed ChatGPT, Perplexity and three specialised tools in use as well, two of which were not yet in the register. Not an incident: it shows that a list made once quickly falls behind what people have started using in their work.
The organisation took a decision per tool and made it visible at the moment of work. Anyone who now opens the free chatbot sees that a decision exists and which tool is the alternative. Anyone who types a bank account number or a name into a text box sees a highlight on it before the message goes, and chooses: mask, remove, or leave it. The colleague remains the one who acts; the organisation makes sure the decision and the risk are visible at that moment.
