# Philip Tetlock The political scientist who **measured the accuracy of expert forecasts and found them dismal — then identified the small subset of forecasters who beat the baseline**. Tetlock (b. 1954), Annenberg University Professor at Penn, conducted the longest-running empirical study of expert political judgment ever attempted, summarized in *Expert Political Judgment* (2005) and *[[Superforecasting]]* (2015, with Dan Gardner). The findings have direct implications for how anyone should weight pundit confidence, intelligence-community estimates, and the broader infrastructure of confident political prediction. Tetlock's empirical record matters in a media environment that runs on confident prediction. Pundits get paid for confidence; calibration costs them airtime. Tetlock's data shows that the tradeoff is not innocuous — high-confidence pundits are typically *less* accurate than low-confidence forecasters — and the field has been doing it backward for decades. ## Core Contributions ### Expert Political Judgment (2005) Tetlock's twenty-year study tracked thousands of predictions by 284 political-science experts across many topics. The headline finding: experts were on average barely better than chance, and frequently worse than simple statistical baselines. Worse, expertise *increased* overconfidence — the more credentialed the forecaster, the more confident they were and the less their confidence was calibrated to actual accuracy. This is one of the most-cited findings in modern social science and has held up well to replication. The book also distinguished *foxes* from *hedgehogs* ([[Isaiah Berlin]]'s terms): hedgehogs know one big thing and apply it everywhere; foxes know many small things and integrate them. Foxes outperform hedgehogs at forecasting because they update on disconfirming evidence and resist overgeneralization. Most media-prominent pundits are hedgehogs, which partly explains the calibration gap. ### The Good Judgment Project and Superforecasting After the IARPA forecasting tournaments (2011–2015), Tetlock identified a small minority of forecasters — about 2% of his volunteer pool — who consistently outperformed everyone else, including intelligence-community professionals with access to classified information. *Superforecasting* (2015) characterizes this group: they are open-minded but rigorous, update incrementally on evidence, decompose problems into estimable parts, think probabilistically rather than narratively, and resist the seduction of confident sweeping claims. The implication is hopeful: forecasting accuracy is *learnable*. The cognitive style that produces accuracy is identifiable and trainable. The downside is that almost nothing in the public-discourse incentive structure rewards it. ### The Adversarial Collaboration Project Tetlock has also pioneered "adversarial collaboration" as a method — pairing scholars with opposing views to design experiments that would resolve their disagreements empirically. The method addresses the field's pervasive problem of motivated reasoning by requiring participants to specify in advance what evidence would change their minds. It is a structural defense against Burden of Proof manipulation and Special Pleading — both fail when participants pre-commit to the falsifying evidence. ## KB Significance **1. The diagnostic frame for evaluating political punditry.** Tetlock's findings imply that audience trust in confident political pundits is structurally misallocated. The pundits with the highest confidence are usually less accurate than calibrated forecasters; the pundits with the largest media platforms are typically hedgehogs whose forecasting record is weak. The broader infrastructure of confident political prediction is operating on selection criteria (entertainment value, audience-affirmation) that are nearly orthogonal to accuracy. This is a structural critique of the punditocracy that goes deeper than partisan critique. **2. Foxes vs. hedgehogs as a frame for ideological commentary.** The fox-hedgehog distinction maps usefully onto the contemporary political-commentary landscape. [[Curtis Yarvin]] is a hedgehog (one big idea: democracy is bad, philosopher-king monarchy is good, applied to everything); the academic critics of authoritarianism are mostly foxes (many small empirical points integrated into multidimensional analyses). Tetlock's findings predict who will be more accurate over time. This is also why hedgehogs make better ideological entrepreneurs and foxes make better long-term analysts. **3. The structural defense of empiricism.** Tetlock's work — and the broader rationalist/forecasting community his work has helped catalyze — represents a methodological commitment to *let the world tell you when you're wrong*. This is precisely the discipline that conspiracy thinking, [[Falsifiability]] political ideologies, and motivated political reasoning systematically refuse. The Tetlock framework is what separates serious analysis from political mythology. ## See Also - [[Superforecasting]] — The 2015 popular treatment of the tournament findings - [[Daniel Kahneman]] — Tetlock's findings extend Kahneman-Tversky biases into the domain of expert prediction - [[Nassim Nicholas Taleb]] — Friendly disagreement: Taleb is more skeptical that any forecasting beats noise; Tetlock has the empirical evidence that some does - [[Falsifiability]] — The Popperian discipline Tetlock's adversarial-collaboration method operationalizes - Superforecasting (Mental Model) — The mental-model entry for the forecasting framework - [[Keith Stanovich]] — Stanovich's thinking dispositions map onto the superforecaster profile - [[Decision Science]] — The hub where this work lives