The auditor asks a privacy officer at an insurer whether staff are complying with the AI policy. She has the policy, the acknowledgements (87 per cent clicked "read and understood") and the attendance list from the training. What she does not have is any idea whether someone is pasting a claims file into a chatbot this afternoon.
The document will not tell you whether an AI policy is followed. Three numbers about practice will. What share of AI use goes through allowed tools, and which way is it moving? What happens after someone sees a notice on opening a tool that is not allowed? And how many tools in real use still have no decision, and is that number growing? All three can be measured without tracking people, and when the result is poor, all three put a question to the policy rather than to the employee.
Why does "read and understood" say so little?
Because it measures what someone knew on the day of the email. The policy is followed or ignored on a different day: the day of the deadline, when a colleague needs a quick summary and the chatbot is already open in another tab. An AI policy on paper describes what should happen; it is not there at the moment that matters.
Article 5(2) of the GDPR requires an organisation to be able to demonstrate that it complies with the principles. A document and a signature show that a policy exists. To show that it is applied, you have to observe practice. Three numbers do that, and none of them needs a name.
Which three numbers show it?
The share of AI use through allowed tools. Of all the times someone in the organisation opens an AI tool, what proportion involve a tool with the status "Allowed" or "Limited use"? This is the number that ties policy to practice. The direction matters more than the level: 70 per cent is good news if it was 55 last quarter, and a warning sign if it was 85.
Notices and what happened next. When someone opens a tool that is not allowed and sees the decision there, there are two possible outcomes: "alternative opened" or "continued anyway". The ratio between them tells you whether the designated alternative is a real alternative. If almost nobody chooses it, the decision may be right but the alternative is not working.
Growing use of tools without a decision. Every organisation has tools nobody has decided on yet. That is an honest status. It becomes a warning sign when such a tool is opened more often every month: it means the policy has a gap exactly where the work is happening.
| Number | What it tells you | What it does not tell you | If it disappoints, look at |
|---|---|---|---|
| Share through allowed tools | Whether practice follows the policy | Why, or why not | Which tools account for the use outside the policy |
| Outcome after a notice | Whether people choose the alternative | What someone did afterwards | Whether the alternative can handle the task |
| Growing use of tools without a decision | Where the policy lags behind | Whether the tool is a risk | The decision still to be made |
One thing missing here is the number of prompts. That tells you how busy things are, not whether the policy is followed. Measuring AI use covers which other numbers about AI use are worth tracking.
What about the rule on sensitive data?
Most policies contain two kinds of rule: which tools, and which data. The three numbers above cover the first kind. The second, "no customer data in a prompt", needs a different observation: how often a sensitive detail is highlighted while someone types, and what happens next. Was it changed before sending, or sent anyway?
That too is a count without people: by app, by data type, with the outcome. Here is what it looks like, with the trend over time and the share changed before sending.
