Your team brings you an AI productivity gain and asks for money to expand. The chart has a baseline, a clean percentage and a confident conclusion. Before approving the next rollout, I'd ask a less impressive question: who is in that number?
A gain can be real and still be too narrow for the decision you're being asked to make. The people who used the tool, the work they tried and the conditions around them define what the result covers. Expansion asks a further question: why should it apply to the next group?
I'd ask the person responsible for the claim for a one-page summary: population, task, outcome and cost, with the underlying records available. Start there before the percentage enters your budget or board deck.
Who was measured?
Which people produced the result, over what period, and what share of the relevant team do they represent?
Distinguish people offered access from people who used the tool, and both from people included in the measurement. If the number covers only continuing users, ask what happened to those who stopped. Also record how many tasks were measured, which kinds and what was excluded.
Consider a hypothetical product team that prepares requirements faster after adopting AI. Only volunteers participated. They worked on routine features; complex integrations stayed outside the trial. That may be a sensible first test. It supports a claim about those participants and those tasks. It doesn't become a result for all of engineering because the slide heading got shorter.
A small, deliberately selected group can tell you something useful. The problem is letting it stand for a larger group without checking the differences.
If the presenter cannot size the group, write unknown rather than letting the number speak for the whole function. The blank is a finding about the claim, not proof that the work failed. Give someone responsibility for resolving it if your decision depends on it.
What counted as finished?
A count of code commits doesn't establish how much usable software was delivered. I'd rather see a defined task measured against a baseline. But a stopwatch doesn't settle the claim either.
Ask when measurement started and stopped. Was the result recorded in the work system or estimated afterward by participants? Does it measure elapsed time, human effort or both? A shorter wait can be valuable even when it doesn't reduce labor hours. Just don't call the two things the same saving.
Include the time spent reviewing and correcting the output, including work passed to someone else. Check whether the work met the same quality standard before and after. In the hypothetical requirements trial, timing an approved specification before AI and an unreviewed draft afterward would compare different completion points.
For agents, distinguish successful task completion from a run that merely stopped. Include failed attempts, timeouts and human intervention rather than reporting only the successful runs.
Then ask what else changed during the period. Early adopters may already have been improving before the tool arrived. Staffing, task mix or a simpler approval process could also explain part of the difference. A before-and-after comparison alone doesn't isolate AI's contribution.
For a large expansion, ask the analyst what comparison would make the estimate more credible, using similar work without the change where feasible. You may be buying a useful combination of software, training and process changes. That's fine. Budget for the combination rather than attributing its whole result to the software.
What return is being claimed?
Ask which business result the productivity number is being offered as evidence for.
If the claim is realized savings, which expense fell, and over what period? If it's faster customer delivery, what happened to delivery time? If the payoff is still ahead, record the expected lag and the assumptions behind it.
A local improvement can be real before a company-level number moves. That doesn't make the financial forecast a demonstrated return. Keep the observed result and the expected benefit separate.
This is a check on the incoming claim, not a request to rebuild every dashboard. My The AI Answer Your Board Actually Wants covers constructing the broader board-facing plan. Here, you're deciding which result can legitimately enter it.
What did it cost, and what will expansion cost?
Put program cost beside the benefit. Include software, infrastructure, implementation, paid learning time, review and rework. Where the program changes roles, include the retraining or severance attributable to that change. Don't assign unrelated restructuring costs to AI merely because they occurred together.
Separate one-time costs from continuing ones. Show what has already been spent, then what the proposed next step would require. The first group's economics may depend on support that won't be available to everyone else.
Have finance check the assumptions, including whether the same saved hours have been counted as both lower cost and extra output without explaining how both happen. A plausible benefit can survive that conversation. It shouldn't need the accounting to look away.
Match the next step to the evidence
Now compare the measured group with the group you intend to add. Look at the work they'll do, the systems and data they'll use, and the support they'll need. Check what training, access and time to learn the next group will receive.
Don't assume they'll benefit less just because they're less experienced or less enthusiastic. Those are reasons to investigate differences, not a ranking of who deserves the tool.
Where a material uncertainty remains, I'd fund a bounded extension that tests it. If the unanswered question is whether AI helps with complex integrations, another trial of routine feature drafts won't answer it. State what result would justify a wider rollout before the extension begins.
A poorly reported real gain is not a failed gain. If the records support it and the problem is an incomplete summary, correct the summary. Don't cancel useful work to make a point about reporting.
Equally, four completed entries don't prove causality, return or readiness to scale. This is a proposed review practice, not a validated ROI test. Security, legal obligations and safe production use still need their own assessment.
Use the depth of review the decision warrants. A consequential investment or a company-wide claim deserves more scrutiny than a low-cost, reversible experiment. You can approve the next limited step while declining to repeat the gain as a company-wide result.
That lets you tell the board what improved, where, and what you're funding next, without pretending the last of those has already happened.
Deciding where AI pays in your company?