One of the biggest challenges you face as CEO is trying to manage and lead people who have far more expertise in their functional area than you do.
Say your CFO is presenting Q4 financial projections to the board. He’s confident, articulate, and fluent in the language of EBITDA, working capital optimization, and capital allocation frameworks. The board members nod approvingly. You nod too. But as you watch the presentation unfold, an uncomfortable question nags at you: Is this excellent financial planning, or just a bunch of well-packaged bovine excrement?
This is the CEO’s expertise problem, and it’s more common than most leaders want to admit. You probably rose through the ranks by building expertise in one functional area, perhaps sales, finance, or technology. But now you’re responsible for evaluating experts across all the functional areas. You might be a former VP of Sales who can dissect a pipeline forecast in your sleep, but you’re now making decisions about cloud infrastructure, actuarial models, and brand positioning, often relying entirely on the judgment of the experts you’ve hired. How do you assess competence in domains where you are not competent yourself?
The Generalist’s Dilemma
This issue of expertise creates what economists call an asymmetric information problem. Your executives possess specialized knowledge that you, by definition, cannot fully evaluate. A talented CTO can make a bad technology decision sound inevitable. A persuasive CMO can defend a failed campaign with industry jargon and attribution modeling complexity. The more sophisticated the function is, the easier it is for executives to hide behind technical language and confident delivery.
The stakes are enormous. The real results of your executive team’s actions often take months or even years to fully show. By the time you realize your head of product has been pursuing the wrong strategy, you’ve burned millions of dollars and critical time. External factors, market conditions, competitor moves, regulatory changes, and all kinds of other circumstances can obscure individual performance, making it difficult to separate skill from circumstance.
The simplest shortcut is one most CEOs are familiar with: credentials. If the resume looks good - degrees from the right schools, maybe a stint at Google or some other massive company - it reassures us that this person knows what they’re talking about. Add to that a bit of executive presence, often rooted in the executive’s sheer self-confidence, and you have what looks like someone you can trust to run a critical function of your company.
Unfortunately, that story is too simple.
Philip Tetlock, who spent decades studying expert prediction and judgment, found that credentials and confidence are poor predictors of accuracy. In his research on geopolitical forecasting, the most credentialed experts often performed worse than simple algorithms. Those people’s traditional signals of expertise (pedigree, articulateness, confidence) don’t reliably correlate with the thing you actually need: sound judgment about future outcomes.
I’ve seen a very similar thing happen on executive teams. Often the slickest, most confident, and most persuasive execs get little scrutiny from the CEO, because they seem like such experts. The CEO, lacking any background in that exec’s domain, goes along with what they say. The executives are evaluated based on the data they curate for the CEO, often receiving high marks even as the performance of the business slips.
The Prediction Accuracy Solution
What’s a better way to evaluate these expert-executives whose domains you don’t fully understand?
Here is a framework I’ve developed over decades leading companies as CEO and another decade training and coaching CEOs and their executive leadership teams:
Instead of evaluating your executives on simple outcomes, evaluate them on the accuracy of their predictions about those outcomes.
In my experience, the strongest predictor of expertise is a leader’s ability to predict performance against meaningful goals. I’ve built that insight into a simple protocol you can follow with your executives. It helps you ensure you’re not blinded by perceptions of credentials and confidence and that you understand the true performance of each leader and their team.
The Quarterly Prediction Protocol:
Step 1. At the start of each quarter, work with each executive to set 3-5 objective, measurable goals. These should be specific, challenging-but-achievable outcomes that align with overall company strategy.
Importantly:
The goals must drive business value that makes sense to the whole business. You as CEO must understand WHY it matters to the company that this goal be achieved, and why it was selected over the endless possible other goals. What you’re trying to prevent here is an executive setting a goal that SOUNDS good but doesn’t support the long-term success of the business. Beware of the vanity metric - the measurement that sounds good to the untrained ear but actually measures nothing of consequence.
Push back when you don’t understand. Get curious. Ask the executive to explain their thinking. Ask them to teach you the things you don’t understand. Do the five whys with them.
Step 2. Every week, executives predict the likelihood of achieving each of these goals on a simple red, yellow, green scale. This isn’t a status update meeting it’s a simple email or dashboard entry. Critically, executives must note significant changes in their predictions. A change in status requires a sentence of explanation: “Confidence dropped because the integration with Salesforce is taking longer than expected” or “Confidence increased because we hired the senior engineer we needed.”
Step 3. At quarter’s end, you evaluate predictions against outcomes and look for patterns:
Consistent accuracy signals genuine domain mastery. An executive who accurately predicts across multiple quarters understands their domain deeply. They have accurate mental models of how their systems work.
Chronic overconfidence, routinely predicting success for much of the quarter on goals that fail, reveals lack of domain knowledge or poor risk assessment. This executive doesn’t understand what they don’t know.
Wild fluctuations swinging from red to green may suggest either poor planning or external factors genuinely outside the executive’s control. Either way, you’ve learned something important.
Sandbagging, by predicting goals won’t be achieved but then achieving them indicates either gaming the system or insufficient ambition in goal-setting. Since you get to approve the goals that are set you can increase the goals the next quarter.
Ultimately, this approach works because expertise lies in having accurate models of reality that generate calibrated predictions.
Consider two CTOs. The first confidently declares that the platform migration will be completed in Q2. It doesn’t, and he is surprised. The second says there’s a 60% chance of Q2 completion, but X, Y, and Z factors could push it to Q3. When it slips to Q3 due to factor Y, he’s not surprised, he’d already modeled that risk. Who has deeper expertise? Who would you rather be getting your information from?
The prediction protocol creates a leading indicator rather than a lagging one. By week 8 of the quarter, you already know which goals are at risk. You’re not waiting for end-of-quarter surprises.
It’s also remarkably difficult to fake over time. Anyone can get lucky once. Consistently accurate prediction over multiple quarters, across different types of goals and market conditions, reveals genuine skill. As Tetlock’s research demonstrated, good forecasters get better with feedback, while poor forecasters plateau or worsen.
The system brings additional benefits beyond evaluation:
It improves planning discipline. Executives can’t set vague goals and hope for the best—they must think rigorously about probability and risk from day one.
It creates early warning systems. When your VP of Sales drops likelihood on the pipeline goal in week 3, you can intervene before the quarter is lost.
It builds intellectual honesty. Organizations develop cultures where admitting uncertainty is valued over false confidence, reducing hindsight bias and Monday-morning quarterbacking.
Most importantly, it generates coaching data. When an executive consistently over-predicts, you can work on their planning and risk assessment rather than waiting for projects to fail.
Beyond Credentials and Charisma
Prediction accuracy won’t solve every evaluation challenge. But it provides what CEOs desperately need: a reliable signal of genuine expertise that cuts through credentials, charisma, and clever explanations.
As CEO, you don’t need to be an expert in every functional area. But you must become expert at identifying expertise in others. The best CEOs demand accurate thinking about results - and that makes all the difference in how a company runs.




