The managing partner of an accountancy firm bought Copilot licences for everyone last year. At the management meeting a partner asks whether it has paid off. IT knows how many licences were assigned. Nobody knows how many people work with it, whether they also use something else or whether the work has got any faster.
That question is legitimate, and most of it can be answered without tracking a single employee. Measure the share of the organisation using AI, how often, with which tools, what part of that use runs through allowed tools, how that changes and what AI is used for. Do not use the number of prompts as a headline figure. And measure all of it per tool and per day, never per person.
Why does an organisation want to know this?
Two reasons, which often overlap.
The first is value. Money was spent on licences and training, and someone wants to know whether AI is changing the work or merely sitting on the budget. That is an ordinary investment question. A count of use does not measure what AI yields or saves; it does measure whether AI is used, by how many people and for what. That also helps you decide where the next round of training should go.
The second is risk. If half the organisation uses AI, customer information goes through those tools too, and the privacy officer wants to know which tools those are and whether anything has been decided about them. That side is covered in more depth in the AI governance topic.
The two questions share one property: neither needs a name. "How many people use AI" and "what share runs through allowed tools" are questions about the organisation. That is what makes the rest of this article possible.
Which figures mean something?
Not everything countable is worth counting. These are.
The share of staff using AI. Measured against the number of seats or licences. This is the figure the partner in the opening scene wants to see, and it is also the most honest one: forty per cent use after a year of Copilot raises a question about the licences for the other sixty per cent.
How often: daily, weekly or occasionally. Someone who uses AI every day has built it into their work. Someone who tries it once a month is still exploring. The distribution across those three groups says more about where things stand than the total does.
Which tools, and how use is spread across them. One tool carrying everything is a different picture from ten tools each with a small share. The second picture usually means the organisation has not yet named a preferred tool, or the named tool does not do what people need.
The share of use through allowed tools. This is the figure that ties policy and practice together. If it rises, the decisions and the communication about them are working. If it falls while total use rises, use is growing outside the policy.
Trend over time. Each of the figures above is a snapshot. It is the trend that tells you something. A seventy per cent share through allowed tools is good news if it was fifty last quarter, and a warning sign if it was ninety.
What AI is used for. Drafting emails and documents, summarising, translating, coding, analysis: that distribution says where AI sits in the work, and so where training and AI rules pay off most. If half the use is translation, a rule about customer names in translation tools is more urgent than a prompting course.
| Figure | Which question it answers | What it does not say |
|---|---|---|
| Share of staff using AI | Is use broad or concentrated in a few people? | How intensive |
| Daily, weekly, occasionally | Is it part of the work or still being tried out? | With which tool |
| Spread across tools | Is there a preferred tool, and is it used? | Whether those tools are allowed |
| Share through allowed tools | Does practice follow policy? | Why or why not |
| Trend over time | Is what we are doing working? | What the cause is |
| Use-case categories | Where does AI sit in the work? | What exactly was asked |
| Number of prompts | How busy it is | Almost everything above |
Why does the number of prompts say so little?
Because it hides the distribution. One analyst sending two hundred prompts a week and ten colleagues doing nothing produce the same total as two hundred colleagues with one prompt each. The first organisation has an enthusiast; the second has AI built into its work. The volume cannot tell the difference.
Prompts over time are useful, as a trend. A doubling in a quarter is a reason to look at which tools account for it. But as the headline figure on a board slide it mostly says that things are busy.
What does this look like in practice?
Back to the accountancy firm. After a quarter of measuring, this is the picture: well over half the seats open an AI tool at least weekly. Copilot is the largest but not the only one: ChatGPT, a translation tool and a tool that summarises annual accounts are opened regularly too. No decision was ever taken about the summarising tool, because nobody knew it existed. The share through allowed tools is around sixty per cent.
Nothing in this is an incident. It is the ordinary picture of an organisation where people make their work easier and the policy has not yet caught up. What management does next: decide on the summarising tool, because it sees clients' figures; look at the translation tool; and take the sixty per cent as a baseline. When the partner asks the same question in six months, there will be an answer that shows which way things are going.
How do you measure this without a user ID?
The difference between measuring and tracking is not in the dashboard but in what gets recorded. An observation without an ID looks like this: on this device, this domain was opened today. With that you can count how many tools there are, how often they are opened and how that shifts. What you cannot do with it is single anyone out. That is not a gap; it is by design.
Frequency can be determined on the device itself. The device knows whether an AI tool was opened daily, weekly or occasionally this month, and reports only the category as an anonymous count. The organisation sees "three people use AI daily", never who.
That is how BeeSensible does it. The extension notices when someone opens an AI tool and reports only the domain, at most once a day per device, with no user ID, no path and no page content. On that basis, the Usage screen shows the share of seats using AI, the split between daily, weekly and occasional use, the tools and what AI is used for, in seven use-case categories. For that last figure, in cloud mode only the first prompt of a conversation is classified; the text is then discarded and only the category is counted. The Compliance screen shows the share of use through allowed tools, with its trend. Nowhere in the dashboard is there a breakdown per employee.
