A team at a healthcare organisation wants to use a tool that records and summarises meetings. The privacy officer opens the vendor's website and reads: ISO 27001, EU hosting, "GDPR compliant". Three ticks. She knows that is not an answer, but she does not immediately know what the questions are either.
The risk of an AI tool is not in one score. It is in a set of dimensions you can question one by one, two of which are not about the tool at all but about your organisation. That is why the same tool is low risk for one department and high risk for another.
Why does a single score not work?
A score is useful when you want to sort hundreds of tools into an order. But the moment you take a decision on one tool is exactly the moment you want to know where the number came from. A tool with an average score can be mediocre on every dimension, or excellent on six and unacceptable on one. That difference decides the outcome, and the number hides it.
There is a second reason. A score about the tool can only say something about the tool. Whether a team is going to put client data into it is not in there. An assessment that looks only at the vendor is half an assessment, however thorough.
Which dimensions matter?
Nine, in the order in which they tend to yield the most. Each comes with one question. Whoever can answer all nine has enough to build a decision on; whoever cannot answer three of them at least knows where the remaining work lies.
| Dimension | The question you ask |
|---|---|
| Input | What happens to what someone enters: is it stored, for how long, and who can access it? |
| Training | Is input used to improve the model, and does that differ per account type? |
| Processing and law | Where does processing take place, and which law governs the vendor and its parent company? |
| Data processing agreement | Is a data processing agreement available, and on which plan? |
| Certifications | Which certifications does the vendor hold, and which are relevant to this processing? |
| Integrations | Which other systems can the tool reach: calendar, email, documents, source code? |
| Incidents | Which incidents are known, how were they handled, and is the source verifiable? |
| Intended use | What will the organisation use the tool for: summarising, writing, transcribing, code? |
| Personal data | Which personal data is likely to go in with that use, and how sensitive is it? |
The first seven questions go to the vendor, or to its documentation. The last two go to the team that wants to use the tool. They matter just as much, and they are the ones most often skipped.
What happens to the input?
This is the dimension that weighs most, and the question most often shortened to "is it trained on?". Training is one thing that can happen to your input. Storage, retention, human review on suspicion of abuse and technical logging are four others, and they happen at vendors that do not train too. What the training promise does and does not cover is set out in what "no training" really means.
So ask per category: conversation content, attachments, technical logs. Ask for the retention period and whether it can be configured. And ask whether the answer differs between the free account, the business account and the API, because at nearly every vendor it does.
Where is it, and who owns it?
Two questions that get asked as one. A vendor running in Frankfurt can be an American company, and then the location of the data centre is not the answer to which government can compel disclosure. What is at play in that case, and why the retention period often weighs more than the region, is covered in where your data goes.
For the assessment you note three things: the processing region, the place of establishment of the contracting party, and whether there is a parent company under another jurisdiction. "We host in the EU" is one of the three.
Is there a data processing agreement, and for whom?
If the tool processes personal data on behalf of your organisation, the vendor is a processor and a data processing agreement belongs with that. The major vendors offer one, but only on their business plans. The free account runs under consumer terms, without. When you need one and what it must contain is worked out in do you need a data processing agreement for an AI tool.
For the assessment, then, the answer is never "yes" or "no" but "yes, on plan X". That immediately determines which account the organisation has to buy if it wants to allow the tool.
What does a certification say, and what does it not?
An ISO 27001 certificate says the vendor has organised its information security according to a standard and an external party has audited that. That is valuable. It says nothing about training, about jurisdiction, about the retention of your prompts, or about what your people enter.
So do not count the logos; look at the relevance. For a healthcare organisation NEN 7510 is relevant, for an education provider FERPA in the United States or the Dutch agreements on pupil data. A certification that does not fit the processing does not count. And mind the difference between "our hosting is certified" and "we are certified". Both can be true, and they mean different things.
Which integrations does the tool get?
A chatbot you paste text into sees what you paste. An assistant allowed into your calendar, your email and your documents sees everything it is allowed to reach, including what you did not intend. So the reach of a tool changes with every integration, and a tool that was low risk at purchase can stop being that one feature later.
Ask per integration: under whose permissions, with what scope, and can it be narrowed. A tool allowed to search "everything in SharePoint" inherits every over-broad permission that has been set there over the years.
Which incidents are known?
Every major vendor has had incidents. On its own that is no reason to reject a tool. What counts is the pattern: how quickly it was reported, how it was resolved, and whether the source can be checked. An incident without a verifiable source is a rumour, and a rumour does not belong in an assessment.
Record the source for each incident. An assessment reread a year later still has to show where a judgement came from.
What is the intended use, and which data?
This is where the assessment turns from vendor research into a decision. The same tool for drafting slogans and the same tool for summarising performance reviews are two different processing operations. In the first, personal data is unlikely to come in; in the second it certainly will, and of the sensitive kind.
Ask the team what it is going to do, and derive from that which data is likely to go in. Not "which data is permitted", because that is in the policy, but which data will go in in practice. That answer determines whether a data processing agreement is needed, whether a DPIA is on the table, and which condition belongs with the decision.
Why can the same tool be low and high risk?
Because the dimensions are not all about the tool. Two of them, intended use and personal data, are about your organisation. And of the other seven, several change with the account: training, data processing agreement, retention and admin settings differ between the business account of a tool and its free account. That difference is large enough to warrant two separate assessments, which is explained in free versus business AI accounts.
An accountancy firm assessed a chatbot for two teams. The marketing team wanted to rewrite copy with it, on the business licence with a data processing agreement: in order on every dimension, low risk, allowed. The assurance team wanted to summarise annual accounts with it, and turned out to be doing so already through the free accounts of a few staff: no data processing agreement, training not switched off, client data in the input. Same tool, two decisions. For the assurance team it became: not allowed on a free account, allowed on the licence, on the condition that client names are replaced beforehand.
The questions you put to the vendor to fill in the first seven dimensions are written out in approving an AI tool: what to ask the vendor, with a usable and an evasive answer for each.
How does this become a decision?
An assessment is only worth something once a decision follows from it that is recorded and that people meet at the moment they work. Otherwise it is an archive document. The decision has one of four statuses, and "limited use" carries the condition that follows from the assessment.
BeeSensible maintains a catalogue of 865 AI tools, with for each tool the vendor, its country, certifications, the training policy, whether a data processing agreement is available, and documented incidents with a source. Each tool receives a risk score across six weighted dimensions: data governance (weight 6), jurisdiction (4), compliance (3), incidents (3), integration (2) and vendor trust (2), with a maximum of 100. The bands are Low below 40, Medium from 40 to 59, High from 60 to 79 and Critical from 80. Free and business versions of the same tool are listed as separate rows, each with its own score.
