A Standard or an Ambush
Any new question you install as a leader lands one of two ways: as a standard or as an ambush. The question is identical in both cases. What differs is whether people knew it was coming.
A question people can prepare for becomes a habit. The same question, sprung as a surprise, teaches your sharpest people to fear the room, and the rational response to fear is rehearsal. You get polished answers and less thinking, which is the exact opposite of what you installed the question for.
Try this. Before your next recurring meeting, announce one question you will keep asking about consequential work. Say in plain words what a good answer contains, so people aim at the substance rather than at your face. Give it a start date. Then keep it, including the weeks when the answers are uncomfortable. The first time you skip it is the moment people learn it was optional, and an optional standard is just a pop quiz with better branding.
The False Signal Got Cheap
The recommendations crossing your desk have never looked better. The pricing memo, the churn analysis, the roadmap options: polished, confident, produced in hours. And you have quietly lost the ability to read what that means. Some of your people did the wrestling those artifacts used to prove. Some didn't. From where you sit, the two versions look identical, and you find out which is which at the worst moment, when a second question lands in the meeting and the presenter goes back to the screen for the answer.
Start with what didn't change, because the nostalgic version of this story is wrong. Polish was never proof. The consultant deck that outran the thinking behind it existed long before the current tools. Effort embedded in an artifact was always a weak proxy for the judgment behind it. What collapsed is the cost of producing the false signal. When a confident memo costs a prompt and a polish pass, the old proxy stops paying, because visible effort has been detached from actual judgment.
The mechanism to watch is quiet. A healthy decision loop runs gather evidence, form a hypothesis, test it, commit. Machine help used well accelerates the gathering and the testing. Used as a substitute, the loop collapses into a single step: ask, receive, comply. The person never forms their own hypothesis. The output arrives fast, fluent and internally consistent, which feels like conviction while being only borrowed conviction. Borrowed conviction has a short shelf life. A recommendation defended with it folds under the second follow-up question, because the presenter cannot go one layer deeper than the tool went. Teams notice. Their confidence in the person presenting erodes quietly, and it is slow to come back.
Meanwhile, the accountability question is arriving from outside whether or not you install a test. A rule of thumb working its way through HR and governance conversations: any decision that could end up in front of a judge needs a human decision-maker, and an answer that amounts to deferring to the tool does not survive the question of how the decision was made. The same boundary runs through boardrooms, not just hiring files. Buyers are learning to put one question to any AI product: how did it arrive at that answer, and can it show its work? A product that cannot has a critical gap. Your meeting rooms deserve the same standard. So before you ask whether a model wrote the memo, ask whether the person behind it can defend the recommendation. That question is older than the tools, and it is the one that does not go away.
A Tell, Not a Verdict
So you need a test that does not depend on how hard the work looks. Here is one, and it costs one meeting and no budget. When a consequential recommendation lands, ask the person presenting two questions. Why this and not that? What did the tool get wrong or miss? Then do the part that actually discriminates: press one layer past the prepared answer, into whichever part sounded most rehearsed, and watch whether the person reasons or retrieves.
Hold the result as a tell, not a verdict. It fails in both directions, and pretending otherwise just swaps one unreliable signal for another. A fluent presenter can perform the reconstruction without owning the judgment, so watch whether the answers get better under pressure or merely faster. A capable person who is nervous, introverted or simply less verbal can stumble on a live question while genuinely owning the work, so read the follow-up, not the polish of the first answer. If the person reasons, you can proceed with more confidence than the memo alone could give you, and no more than that. If the person retrieves, the item is not yet ready to defend, which is different from not having done the work. Send it back without blame; it returns when its presenter can stand behind it one layer past the tool. Until then, you do not have a decision on the table. You have a draft.
Three boundaries keep the check honest. It is not anti-AI: machine help early in the loop, to gather and test, and your own judgment late in the loop, to commit, is the sequencing that makes both halves work. It is graded by consequence: the questions belong to recommendations where being wrong is expensive, while low-stakes reversible work keeps the tool's default speed, so the check protects what the tools bought instead of repealing it. And it is announced rather than ambushed, applied to everyone, because a check people can prepare for builds the habit while the same check sprung as a surprise teaches rehearsal. Underneath sit the human-presence questions buyers and boards are converging on anyway: did the human make the decision, can the human explain why, and will the human be accountable for what happens next.
One distinction, so you do not confuse this with its cousin. It is not the restatement test, where you ask someone afterward to say in their own words what the group decided. That checks shared understanding after a decision exists. This checks who owns the reasoning before a decision does. Different object, different moment, different question.
One Layer Past the Tool
Think of the last consequential recommendation you approved, the pricing call, the roadmap, the hire. If you had stopped the meeting and asked its presenter why this and not that, one layer past whatever the tool produced, would there have been an answer? If you don't know, be exact about what that means: you approved the artifact, and ownership of the reasoning was never tested.
Back to the Screen
Recall the last time a presenter went back to the screen for the answer to a second question. What did the room do with that moment? If the honest answer is the meeting moved on, then your operating rhythm has been quietly teaching everyone that fluency is the standard and retrieval is fine. One such moment, noticed, is a tell. A year of them unnoticed is a culture.
The Reconstruction Check
A proposed practitioner exercise, not a studied protocol, and a tell rather than a verdict. One meeting, one item, no new budget.
Pick the next consequential recommendation on your agenda. Before you use the check, announce it as a standing standard: two questions for recommendations where being wrong is expensive, a start date, and a plain account of what a good answer contains. That announcement is the difference between a standard and an ambush.
Then run it. Ask the two questions. Why this and not that? What did the tool get wrong or miss? Follow with the one that does the real work: press one layer past the prepared answer, into whichever part sounded most rehearsed, and watch whether the person reasons or retrieves.
Then turn the check on yourself. Take the last significant decision you made and explain why it was right given what you knew at the time, without pointing at what a tool told you. If the honest answer requires reconstructing a model's reasoning, you have found something. And while you are at it, notice what your own tools agree with. A tool that agrees with everything you feed it is doing the job of a very articulate yes-man, and you already have enough of those.