The managing partner of an accountancy firm bought Copilot licences for everyone last year. At the management meeting a partner asks whether it has delivered anything. IT knows how many licences were assigned. Nobody knows how many people work with it, whether they use something else on the side, or whether the work has become any faster.
That question is legitimate, and it can be answered without tracking a single employee. Measure the share of the organisation using AI, how often, with which tools, what part of that use runs through allowed tools, how that changes, and what AI is used for. Leave the number of prompts out as a headline figure. And measure all of it per tool and per day, never per person.
Why does an organisation want to know this?
Two reasons, which often run together.
The first is value. Money was spent on licences and training, and someone wants to know whether AI is changing the work or merely sitting on the budget. That is an ordinary investment question, and a good answer also helps decide where the next training should go.
The second is risk. If half the organisation uses AI, customer information goes through those tools too, and the privacy officer wants to know which tools those are and whether anything has been decided about them. That is the side worked out further in the AI governance topic.
The two questions share one property: neither needs a name. "How many people use AI" and "what share runs through allowed tools" are questions about the organisation. That is what makes the rest of this article possible.
Which figures mean something?
Not everything countable is worth counting. These are.
The share of staff using AI. Against the number of seats or licences. This is the figure the partner in the opening scene wants to see, and it is also the most honest one: forty percent use after a year of Copilot means the licences for the other sixty percent raise a question.
How often: daily, weekly or occasionally. Someone who uses AI every day has built it into their work. Someone who tries it once a month is still looking. The distribution across those three groups says more about where things stand than the total does.
Which tools, and how use is spread across them. One tool carrying everything is a different picture from ten tools each with a small share. The second picture usually means the organisation has not yet named a preferred route, or the named route does not do what people need.
The share of use through allowed tools. This is the figure that ties policy and practice together. If it rises, the decisions and the communication about them are working. If it falls while total use rises, use is growing outside the policy.
Development over time. Each of the figures above is a snapshot. The direction is the information. A seventy percent share through allowed tools is good news if it was fifty last quarter, and a signal if it was ninety.
What AI is used for. Writing communication, summarising, translating, coding, analysis: that distribution says where AI sits in the work, and so where training and agreements pay off most. If half the use is translation, an agreement about customer names in translation tools is more urgent than a prompting course.
| Figure | Which question it answers | What it does not say |
|---|---|---|
| Share of staff using AI | Is use broad or concentrated in a few people? | How intensive |
| Daily, weekly, occasionally | Is it in the work or still being tried? | With which tool |
| Spread across tools | Is there a preferred route, and is it used? | Whether those tools are allowed |
| Share through allowed tools | Does practice follow policy? | Why or why not |
| Development over time | Is what we are doing working? | What the cause is |
| Use-case categories | Where does AI sit in the work? | What exactly was asked |
| Number of prompts | How busy it is | Almost everything above |
Why does the number of prompts say so little?
Because it hides the distribution. One analyst sending two hundred prompts a week and ten colleagues doing nothing produce the same total as two hundred colleagues with one prompt each. The first organisation has an enthusiast, the second has AI in the work. The volume cannot tell the difference.
Prompts over time are useful, as a trend. A doubling in a quarter is a signal to look at which tools carry it. But as the headline figure on a board slide it mostly says that things are busy.
What does this look like in practice?
Back to the accountancy firm. After a quarter of measuring, this is the picture: well over half the seats open an AI tool at least weekly. Copilot is the largest but not the only one: ChatGPT, a translation tool and a tool that summarises annual accounts are opened regularly too. No decision was ever taken about the summarising tool, because nobody knew it existed. The share through allowed tools sits around sixty percent.
Nothing in this is an incident. It is the ordinary picture of an organisation where people make their work easier and the policy has not yet caught up. What the management does with it: decide on the summarising tool, because it sees clients' figures; look at the translation tool; and take the sixty percent as a baseline. When the partner asks the same question in six months, there is an answer with a direction in it.
How do you measure this without a user id?
The difference between measuring and tracking is not in the dashboard but in what gets recorded. An observation without an id looks like this: on this device, this domain was opened today. With that you can count how many tools there are, how often they are opened and how that shifts. What you cannot do with it: speak to someone about it. That is not a gap, that is the design.
Frequency can be determined on the device itself. The device knows whether an AI tool was opened daily, weekly or occasionally this month, and reports only the category as an anonymous count. The organisation sees "three people use AI daily", never who.
That is how BeeSensible does it. The extension notices when someone opens an AI tool and reports only the domain, at most once a day per device, with no user id, no path and no page content. The Adoption screen shows on that basis the share of seats using AI, the daily, weekly and occasional split, the tools, and what AI is used for in seven use-case categories. For that last one, in cloud mode only the first prompt of a conversation is classified, after which the text is discarded and only the category counts. The Risk screen shows the share of use through allowed tools, with its trend. Nowhere in the dashboard is there a breakdown per employee.
