FiveThirtyEight.com premiered on March 7, 2008, three days after Hillary Rodham Clinton won the Democratic presidential primaries in Texas and Ohio — victories that were widely described as giving her momentum in her race for the Democratic nomination. Mrs. Clinton was already well ahead in the polls in the next big primary contest, in Pennsylvania.
From reading news reports or watching the nightly gantlet of cable news programs, the message seemed to be that there would be a close fight between Mrs. Clinton and Barack Obama through the final contests in June (and perhaps to the Democratic National Convention in August) — with each candidate about equally likely to prevail.
To those of us who had been following the numbers, however, the outcome was hardly so uncertain. Presidential nominations are not determined on the basis of momentum; they are determined on the basis of delegates, and Mr. Obama had a significant advantage there — thanks to a long string of victories in midsize states throughout February, and huge margins in some smaller states on Super Tuesday that gave him a lead of about 150 pledged delegates. Even if Mrs. Clinton had achieved a 25-point victory in Pennsylvania, she would still have trailed by more than 100 delegates, and it would have been all but impossible for her to catch up with few large states left to vote.
Things went rather badly for Mr. Obama from that point in the campaign on: the Tev. Jeremiah A. Wright Jr. controversy surfaced just a few days after Mrs. Clinton’s wins in Texas and Ohio, and Mr. Obama was feeling critics’ heat after he suggested that small-town voters had been “clinging” to guns and religion. He lost the Pennsylvania primary by 9 points, and managed barely more than a quarter of the vote in some other late-voting states like West Virginia and Kentucky.
Yet he won the nomination easily. There would be no floor flight in Denver. The math trumped the momentum.
A similar pattern emerged in the final stages of the general election campaign. There were some moments — like immediately after the Republican convention — when Mr. Obama seemed in real danger of losing to Senator John McCain of Arizona. But after the financial meltdown became manifest in mid-September, the likely outcome had become quite clear. Mr. Obama held a consistent 6-to-10-point lead in the national polls (historically, few candidates have squandered such a large advantage at the last minute) and also led in most key swing states, giving him an essentially limitless number of permutations to secure 270 votes in the Electoral College. The FiveThirtyEight forecasting model, which was based on a rigorous analysis of polling in past presidential elections, gave Mr. Obama an 85 percent to 90 percent chance of winning throughout this period, and a 98.7 percent chance by Election Day. And yet, on the weekend before the election on the television program “The McLaughlin Group,” three of the five pundits suggested that the election was “too close to call,” and one said she expected a narrow McCain victory.
Politics is not the only place where a poor understanding of probability and statistics can color news coverage. In baseball (which I covered prior to politics), “intangibles” like clubhouse chemistry are sometimes treated as being more important than batting average or E.R.A. But you wouldn’t find very many sportswriters who would claim, in a game in which the Yankees were trailing Boston, 7-2, in the ninth inning, that it was “too close to call,” no matter how shaky the Red Sox bullpen looked, or how confident Mark Teixeira seemed at the plate. That’s the equivalent of what those pundits were doing on “The McLaughlin Group.”
Instead, there seems to be something about politics that can make the rational parts of the brain turn off. FiveThirtyEight was designed to be the antidote to that. For readers just becoming acquainted with FiveThirtyEight, the blog is devoted to the rational analysis of politics, and sometimes other data-rich subjects. In Congressional and presidential elections — for which there is a lot of high-quality data available — this will sometimes take the form of quite explicit forecasts, like Harry Reid having a 42 percent chance of keeping his seat in Nevada (hypothetically) or the Republicans having a 20 percent chance of winning the Senate (again, hypothetically). In other cases, it simply means trying to prioritize objective information over subjective information in dealing with issues in the news.
Although objective and statistical are not quite synonymous, for the most part this approach means that we focus on the numbers, while not losing sight of their context. And it is important to do so in the right way: namely, to use guidance from past elections to inform our understanding of the present.
FiveThirtyEight’s forecasts performed quite well in 2008, predicting 49 of 50 states correctly in the presidential election, as well as all 35 Senate races. Nevertheless, I have re-engineered our Senate model to improve it. It is now much smarter, for instance, in how it estimates the amount of error associated with each forecast. The more the polling tends to diverge in a given race, or the more undecided voters there are, the less confident it is in its projections.
Our new House forecasting model, meanwhile, which we plan to release next week, uses a multiplicity of indicators (national and local polls, forecasts from experts like Cook Political, fund-raising data, and so forth) to project the outcome of each seat, a calibration that is based on the amount of predictive power that each indicator has had in the past. Although polling is a key input in each of our models, we do not use it verbatim: we adjust our analysis to account for polls that show a consistent partisan bias, for instance, and give more weight to polls that have been more accurate in the past.
This approach — not just objective and statistical, but rigorous and empirical — is especially prudent when it comes to evaluating the upcoming midterm elections. The Democrats will almost certainly lose seats in both houses of Congress, but exactly how many is an open question. A casual reading of trends is somewhat unhelpful: Americans are unhappy with the direction of the country and, increasingly, with President Obama. But in contrast to 1994, the Republicans’ favorability ratings are also near all-time lows. Meanwhile, looking at a single statistical indicator does not provide for precision, and some indicators disagree with one another. It is perhaps necessary to dig a bit deeper — to look at more data, and to do so in a more robust way — in order to have a truly good handle on how things are most likely to proceed. And even then there can still be a lot of uncertainty in the forecast. FiveThirtyEight and its statistical models are willing to admit what they don’t know.
Of course, this is now all going to be done under the auspices of a mainstream media partner: The New York Times, which is not only hosting the blog but is enlisting its team of interactive journalists and graphics experts to deepen and enhance it. While I hope this move broadens FiveThirtyEight’s audience, I welcome the many loyal readers who have followed the blog over the past two and a half years.
Fundamentally, I’ve always seen FiveThirtyEight’s mission as being parallel to journalism: objectivity and accuracy have been core values of the blog, but it has also prized clarity of thought and of written and visual communication. Therefore, this is a pretty natural partnership. But I also recognize that this will lead to greater criticism and scrutiny. For the most part, we welcome it: many of the biggest improvements to FiveThirtyEight’s forecasting products have come in response to points raised by our critics, and the blog would never have existed in the first place if I had been more satisfied with the way the press covers politics. We hope to hear frequently from our readers, and we hope that you’ll join us from now until November and in the months and years ahead.
