A privacy officer at an insurer is asked by the auditor whether the AI policy is being complied with. She has the policy, the acknowledgements (87 percent clicked "read and understood") and the attendance list from the training. What she does not have: any idea whether someone is pasting a claims file into a chatbot this afternoon.
Whether an AI policy is followed you do not read from the document but from three numbers about practice. What share of AI use runs through allowed tools, and which way is that moving. What happens after someone sees a notice at a tool that is not allowed. And how many tools with real use still have no decision, and is that number growing. All three can be measured without tracking people, and all three, when the outcome is poor, put a question to the policy rather than to the employee.
Why does "read and understood" say so little?
Because it measures what someone knew on the day of the email. The policy is followed or ignored on a different day: the day of the deadline, when a colleague needs a quick summary and the tab with the chatbot is already open. An AI policy on paper describes what should happen; it does not turn up at the moment that matters.
The GDPR asks in Article 5(2) that an organisation can demonstrate that it complies with the principles. A document and a signature demonstrate that there is a policy. Whether it is applied takes an observation of practice. There are three numbers for that, and none of them needs a name.
Which three numbers show it?
The share of AI use through allowed tools. Of all the times an AI tool is opened in the organisation, what part is a tool with the status allowed or limited use. This is the number that ties policy and practice together. The direction counts more than the level: 70 percent is good news if it was 55 last quarter, and a signal if it was 85.
Notices and what happened next. When someone opens a tool that is not allowed and the decision appears there, two outcomes are possible: the alternative opened, or continued anyway. The ratio between those two says whether the designated alternative is really an alternative. If almost nobody picks it, the decision may be right, but the route is not.
Tools without a decision that keep growing. Every organisation has tools nobody has decided on yet. That is an honest status. It becomes a signal when such a tool is opened more often every month: then the policy has a gap right where the work is.
| Number | What it says | What it does not say | When it disappoints, look at |
|---|---|---|---|
| Share through allowed tools | Whether practice follows the policy | Why, or why not | Which tools carry the use outside the policy |
| Outcome after a notice | Whether the alternative gets chosen | What someone did afterwards | Whether the alternative can handle the task |
| Growing tools without a decision | Where the policy is behind | Whether the tool is a risk | The decision still to be taken |
What is missing here is the number of prompts. It says how busy things are, not whether the policy is followed. Which other numbers about AI use are worth having is in measuring AI use.
And the rule about sensitive data?
Most policies contain two kinds of rules: which tools, and which data. The three numbers above cover the first kind. The second kind, "no customer data in a prompt", takes a different observation: how often a sensitive item gets a marking while someone types, and what happens next. Changed before sending, or sent anyway.
That too is a count without people: per app, per data type, with the outcome. This is what it looks like, with the trend over time and the share that was changed before sending.
