When Good Enough Isn't Good Enough: On House Forecasts and the Generic Ballot

Emory's Alan Abramowitz, by way of Pollster.com's Brendan Nyhan, has a reaction of sorts to our post from Tuesday in which I talked about some of the ambiguities of the generic Congressional ballot. Here's Abramowitz:

Nate provides a lot of excellent analysis. But there are two pretty silly statements here. First, the generic ballot is a pretty good predictor of both the national popular vote and the national seat results. Second, the national popular vote is a very good predictor of the overall seat results. It definitely is not "relatively irrelevant" to those results.

Let me warn you that some of the ensuing discussion boils down to semantics. What do we mean by a "pretty good" predictor and a "very good" predictor, for instance? There are a lot of times when a particular equation might have a reasonably high R-squared, for example, but would not be all that useful to us in terms of our ability to make predictions about the sort of questions that are most interesting to us, such as what the likelihood is of a Republican takeover of the House.

If we wanted to take the generic ballot today, August 19th, and use it to project out the number of seats that Republicans will gain (or lose, I suppose) in the House, there are basically three sorts of uncertainties that we face.

1. The generic ballot today is an imperfect predictor of the generic ballot on Election Day.

As I wrote on Tuesday, and as others like Columbia's Robert Erikson have found, this is not actually all that big a concern, as the generic ballot is relatively stable as compared with most political indicators. Still, this does produce some additional uncertainty. The generic ballot moved several points toward the Republicans by election day 2004 as compared how it was printing during the summer, and several points toward the Democrats in 2006.

But suppose that we ignore this, for now. Suppose that we take Pollster's current trendline estimate of the generic ballot, which has Republicans winning by 5.6 points, and assume that this is exactly the margin that will separate the two parties on Election Morning. There are still two additional problems that we face.

2. The Generic Ballot on Election Day is an imperfect predictor of the national House popular vote.

This is probably the most significant source of error. In addition to the normal ambiguities surrounding any poll -- the consensus of polls is often wrong in one or the other direction -- we also have the fact that with very rare exception, the generic ballot is framed as presenting "the Republican candidate in your District" against "the Democratic candidate", or some variation of this, rather than naming the candidates specifically. Some voters might react differently if they knew what the names of the candidates were. Also, some voters literally won't have the chance to vote for the candidate from their preferred party, because there are usually several dozen Congressional districts -- and there have been as many as 100 in some past election cycles -- in which one or the other major parties doesn't nominate a candidate. Finally, some states like Florida don't even bother to tally the results when a candidate runs uncontested, so their votes won't count toward the national popular vote at all.

For all these issues, the generic ballot certainly tells you something about the House popular vote, particularly if you make certain adjustments to it, like recognizing the difference between registered voter and likely voter polls. But this contributes a significant amount of uncertainty.

3. The national House popular vote is an imperfect predictor of the seat count.

It's true that if we knew exactly what the popular vote were, we could come up with a not-bad estimate of the seat count. From the chart that Abramowitz posted, it looks like the average error in projecting the seat count from the popular vote is something just a wee bit north of 10 seats, which would imply a 95 percent confidence interval of about X ± 20 or 25 seats. Again -- semantics! -- I would call that "pretty good" rather than "very good": saying, for example, that the Republicans will with 95 percent confidence gain somewhere between 25 and 65 seats, doesn't seem to impart all that knowledge. But even though the distribution of votes into seats is somewhat uneven -- for example, Democratic districts tend to have lower turnout, which somewhat contradicts the fact that the generic ballot tends to overestimate their standing in the national popular vote; but on the other hand, Democratic voters tend to be more concentrated into particular Congressional districts than Republican ones, usually in urban centers -- this is less problematic than Step #2.

But of course, we don't have any knowledge of what the popular vote is in advance -- instead, it has to be estimated from the generic ballot (and perhaps other factors). What happens in the real world where when we have to go directly from the generic ballot to projecting a seat count, processing steps #2 and #3 in one fell swoop? Well, we get a big mess. Here is the direct translation from the parties' generic ballot standing, as inferred from the trendlines that Charles Franklin has generated, into the number of Democratic-held seats in the House, for all elections since 1946.



The forecast misses on average by about 20 seats, which translates into a 95 percent confidence interval of about ±48 (!) seats. At a generic ballot reading of Republican +5.6, for example, where Pollster.com has it now, the regression line above projects a Republican gain of 41 seats -- which sounds reasonable, I suppose -- but the 95 percent confidence interval runs between a gain of 90 seats and a loss of 7 seats. Not very helpful! And of course, this assumes that we know what the standing of the generic ballot on Election Day, which we don't, since there are still 75 or so potentially volatile political days to occur before then.

(As an aside, if you were to apply the same technique only to data from 1994 onward, the regression equation would project Republican gains of between 36 and 107 seats; if you were to use data from 1980 onward, it would project somewhere between a 14 seat loss and a 104-seat gain.)

I don't mean to slam all macro-level attempts at Congressional forecasting: there are significantly more sophisticated versions of macro-level forecasts that political scientists like Abramowitz have worked on. But the generic ballot alone is a very blunt instrument. It basically tells us, "okay, things are probably going to be pretty bad for Democrats, and they could be really bad," something which any sentient observer of politics would already have known.

That's why we're going through the trouble of building a ground-up projection of the House, which attempts to predict the outcome of individual seats, while also understanding that the outcomes and uncertainties in different congressional districts are correlated. This has required collecting lots and lots of data: essentially, every district-level poll, every fundraising record, and every independent forecast since 1998. Although I'm not quite ready to tease at the results, it does seem reasonably clear that taking into account a multiplicity of indicators is helpful -- for example, the generic ballot does have some influence, even if you have lots of other information about the races you're attempting to forecast, but the same is true of each of the other indicators I just mentioned.

I'm sure that even a perfectly-constructed model (and ours won't be) would still be subject to a significant amount of error -- and anecdotally, I suspect that this is a very tricky election to forecast. But given that a lot of this data -- which granted, took weeks and weeks to compile and is not exactly sitting at our fingertips -- has essentially gone unexamined, I hope you'll appreciate my desire to demand a greater degree of precision.

There's More...

Contract Post

Palestinian Refugees Get Qualified Right to Work in Lebanon

he more than 425,000 Palestinian refugees living in Lebanon, who yesterday received a qualified right to work in the country, have a long and complicated history in the country. Unlike those in their neighbors Jordan or Syria, Palestinians in Lebanon have never obtained significant political or social rights, such as citizenship, rights to work, or the ability to own property.

While he bill passed by the Lebanese Parliament formalizes the legal status of Palestinian workers in a number of lower tier occupations, the larger issues of nationality, land and property, and access to Lebanese public social benefits and higher level professions remain unresolved.

Somewhat ironically, the leadership of the many politico-religious communities in Lebanon are in largely in harmony on one element this issue: everyone agrees that the registered Palestinians refugees in Lebanon should not be given Lebanese nationality, nor the trappings of permanent residence, such as the right to own land or major property. It is for very different reasons, however.

When the Palestinians were initially forced from their land by the Israelis during and just after the 1948 war, about 126,000* of the 726,000 total refugees ended up in Lebanon, representing a little over 17 percent. Throughout the next decade, most Christian Palestinians, about 50,000 in total, received Lebanese citizenship. Following the conclusion of Lebanese Civil War in 1990, another group of Palestinians, almost all Shiites, along with a few remaining Christians, received Lebanese nationality, numbering about 60,000. This in concession to the vastly improved political position of the Shiites following the Taif Agreement, which changed the confessional distribution of power in the country.

The more than 400,000 refugees that remain today in Lebanon, in 12 official camps, among other places, are almost entirely Sunni Muslims, their Lebanese brethren being the confessional group that lost the most political power in the wake of the 1975-1990 civil war.

Polling numbers from 2007 shows a remarkable level of agreement about the Lebanese confessionals that an improvement in civil and political rights for the Palestinians, without full citizenship, is the appropriate way forward. That said, there is a definite split between the Christian and Muslim communities, with a significantly larger Muslim minority in favor of citizenship, as well as more vigorous support for improved civil and social rights.

Accordingly, it's unlikely in the foreseeable future that the Sunni Muslim Palestinians in Lebanon will gain citizenship, as they have in Jordon, or full civil rights and semi-nationality, as they have in Syria, in the years ahead. Across confessional groups, there is strong support for the right of return for the refugees to areas that is now part of the Israeli state -- among Christians to avoid upsetting the gentle confessional balance against them, while for Muslim Lebanese it is for reasons of Muslim/Arab solidarity. Giving nationality or full residential rights to the Palestinians, they argue, would admit defeat in this effort, regardless of how fruitless it may seem.

From the outside, of course, it looks far more like political expediency: excluding an unwanted and marginalized community from the benefits that the majority enjoy, regardless of the rhetoric that underpins it.

---
Renard Sexton is FiveThirtyEight's international affairs columnist. He can be contacted at sexton538@gmail.com

* 1950 figure from UNRWA

There's More...

Contract Post

Hatred of Congress

Americans hate their Congress. They hate the long-winded, blowhard speeches, and the parochialism, and the pork-barreling--except, of course, when they like it. As political scientist Richard Fenno once noted, that hatred helps explain why so many candidates--initially as challengers, but even as incumbents--run for Congress by running against it.

As for how this hatred translates into seat losses or gains this November, or any other year, we have been hearing a lot of talking points about how general congressional approval is falling. True. And how the approval of Democrats in Congress is falling. Also true. And how the approval of congressional Republicans isn't much better. True yet again.

Let's take each of these in order to see what we might reasonably infer about the implications for November 2.

First, and without muddying the post with two many pre-existing charts, you can see here that, according to Gallup numbers dating back to 1973, as it has for a long time Congress ranks last in terms of citizen confidence in institutions. (Citizen confidence is not identical to approval, of course, but is very similar.) Looking more closely at approval during the past year, you can see here that, again according to Gallup, the congressional approval has been hovering around the abysmal level of 20 percent throughout 2010.

Not so good.

Turning next to approval of the respective parties, the chart I presented at the top of this post above shows the approval and disapproval ratings for each party since President Obama took office, as reported by the Pew Research/National Journal Congressional Connection Poll and tracked by PollingReport.com for the Democrats and Republicans. It confirms that, indeed, the Democrats in just 18 months have blown a net +10 congressional approval rating, turning it into roughly a 20-point net disapproval rating. The key moment of inversion came in late spring 2009, in the wake of the passage of the stimulus bill and at the outset of sometimes nasty debates on Capitol Hill (and back home in those town halls) about health care reform. The only good news for Democrats is that they've fallen to the depths Republicans have been wallowing at for some time--again, with about net 20-point disapproval score.

So, we can reasonably conclude that the early 2009 high hopes for Congress, no doubt boosted by the high approval ratings and excitement or hope that attended the start of the Obama presidency, have been dashed. Overall congressional approval has turned south, largely because voters now feel about as positively (or rather, negatively) as they do the Republicans. To Americans, the nation's least trusted and least liked political institution has returned to its earthly, dismal levels of scorn and disappointment.

What does all this mean for November? There are (at least) three possibilities to consider:

1. The Democrats are in serious trouble. This is the conventional wisdom, and there is ample reason to subscribe to it. Yes, the GOP's approval numbers stink, but hey, they're not in charge of anything. Their low approval means nothing because the Dems are running the show and their heads are on the electoral chopping block this autumn.

2. The Democrats are in some trouble, sure--but less than people think. Here the logic goes something like this: "The voters are disgusted with the Democrats, but that doesn't mean they like or trust the GOP to do any better. This wasn't the change they were looking for, but they know the change back to what came before is definitely not what they are looking to return to." This is the current political meme being spread by the White House and other national Democrats, a more sophisticated version of saying, "Yup, we suck but they really suck."

3. These congressional approval numbers are meaningless. One can reach this conclusion not only because both parties are suffering the same, 20-point net negative approval, but simply because approval for the Congress is distended from how people will vote on a district-by-district, state-by-state basis. The strongest piece of evidence here is this: If you think the public's view of Congress as a whole is bad now, according to Gallup it was basically the same this time in 2008 when Democrats also controlled both chambers and still picked up seats, not to mention it was higher still in 2006 right before the Republicans had their majorities erased by voters. The counter argument to this is that Democrats, though running Congress two years ago, still had George W. Bush to contend with, so they were not being held responsible for governmental failures then in the way they are now...which more or less brings us back to point #1.

I'm not a polling expert (would love to hear Nate weigh in on this), but I suspect that in this postmodern era of hyper-partisanship these approval numbers, collectively for Congress as an institution or separately for each party, matter less than how voters feel about the president--and how they then express their pleasure or displeasure during midterms toward the one set of national elites they have available to reward or punish via the ballot: members of Congress.

There's More...

Contract Post

Late Night Returns From the West

Returns from today's primaries in Wyoming and Washington have been rolling in for a while, with some interesting results but no big upsets.

In Wyoming, as expected, Leslie Peterson won the Democratic gubernatorial primary over Pete Gosar, by a 48-38 margin. But it doesn't appear there will be an all-female general election: in the GOP primary, with just nine precincts still out, Matt Mead seems to have edged out Rita Meyer by just over 1300 votes; hard-core conservative Ron Micheli finished a strong third, just 840 votes behind Meyer at present. (Colin Simpson finished well back in fourth). Those who consider Mead a RINO will undoubtedly lament the split in the "true conservative" vote.

In Washington, as expected, Patty Murray and Dino Rossi finished first and second in the U.S. Senate primary and will advance to a clawhammer duel in November. With about 55% of precincts reporting (and nearly half of King County--Seattle--still out), Murray leads Rossi 46-34; Tea Party favorite Clint Didier's way back at 12%. Sarah Palin had another bad night with Didier's and Meyer's losses.

In the most competitive House race in WA, in the 3d district, the conventional wisdom won out again, with Democrat Denny Heck and Republican Jaime Herrera moving on to the general election. Would-be conservative spoiler David Castillo is currently running fourth.

Finally, out in California, the special election runoff to fill Lt. Gov. Abel Maldonado's state senate seat looks like a probable win for Republican Sam Blakeslee over Democrat John Laird. I mention this not only because I happen to live in Senate District 15, but because more votes were cast in this special state legislative election than in today's entire Wyoming primary. This central coast district, which runs from Santa Clara all the way to Santa Barbara, is represented by one state senator. Wyoming, as you may know, is represented by two United States Senators. Such is our system.

UPDATE, Wednesday AM: Sean Trende of RealClearPolitics has an analysis of the Washington vote based on the relationship of primary and general election votes in prior "blanket primary" elections, suggesting that Murray's in trouble, the 3d district is almost certain to go Republican, and a couple of Democratic congressmen should be nervous. I haven't had the time or inclination to second-guess Sean's research, but I do wonder if he's noting that Washington has changed back-and-forth from August to September primary dates in the past. You'd figure September primaries would produce higher turnout and a closer relationship to November public opinion, while August primaries would have somewhat less predictive value.

There's More...

Contract Post

Mild West Preview

With most of the chattering classes focused on next week's wild primaries in Florida, this Tuesday's entries, Washington and Wyoming, are not receiving a great deal of attention. That's partly because Washington's "top two primary" system has made many contests essentially a positioning exercise for November, and partly because Wyoming politics just get missed nationally on occasion.

But there is some intrigue surrounding the performance of Republican Dino Rossi in Washington's Senate primary, and there's a close race among Republicans for a spot in the general election in the open 3d congressional seat. In the Cowboy State, there are competitive gubernatorial primaries for both parties, with the Republican race being a close three- or four-way battle in which out-of-state endorsements have been significant.

All year long, Republican long-shot prospects of winning control of the Senate have depended heavily on recruiting a strong candidate against Democrat Patti Murray. DC GOPers got the candidate they wanted in Dino Rossi, who lost two very close gubernatorial races to Christine Gregoire. And while Murray has usually led in general election polls (49-46 in an August 1 PPP poll; 49-47 in a July 28 Rasmussen survey), Rossi has kept it very close.

But two other Republicans with some significant backing have also jumped into the Senate race. Almost no one thinks Rossi will fail to finish second and advance to the general election, although his margin could be lower than originally expected. Rossi has studiously avoided Tea Party events, which have been dominated by former NFL player (for the team in that other Washington, the Redskins) Clint Didier, who has secured endorsements from Sarah Palin and Ron Paul while calling for the phasing out of Social Security and elimination of several federal Cabinet agencies. Conservative businessman Paul Akers, who has also been on the Tea Party circuit, has spent enough of his own money to make a mark, too. The most recent PPP poll showed Rossi at 33%, Didier at 10%, and Akers at 4%, while Murray leads the field at 47%.

The other big primary contest in WA is in the very competitive 3d congressional district, where Democrat Brian Baird is retiring. Former state legislator Denny Heck, a Democrat, is very likely to finish first, but the intra-Republican battle for the second spot has become close and unpredictable. The consensus Republican front-runner is state representative Jaime Herrera, a 31-year-old Latina who is a prize national GOP recruit. But former state legislative staffer and Bush administration bureaucrat David Castillo (not, despite his surname, a Latino) has worked the Tea Party circuit and sports a FreedomsWork endorsement, while a third Republican, disabled veteran David Hedrick, is running to the right of the rest of the field. Interestingly, Herrera has explicitly opposed partial privatization of Social Security, an unusual position for Republicans this year.

In all but one county in Washington, all voting is by mail, which means the primary has been underway for some time. The official estimate of expected primary turnout is 38%, a bit above average for midterm primaries.

In Wyoming, the very popular outgoing Democratic Gov. David Freudenthal decided against a third-term bid (which would have required a legal challege to the state's term limit laws) relatively late, leaving Democrats scrambling for candidates. Ultimately state party chair Leslie Peterson decided to run, with her major competition being former University of Wyoming football star Pete Gosar. A Mason-Dixon poll at the end of July showed Peterson up over Gosar 30-22. There's not a lot of difference between the two candidates on issues; both pledge to continue Freudenthal's legacy.

The Republican primary, however, has had a few ideological flashpoints. The two front-runners have been former U.S. Attorney Matt Mead and State Auditor Rita Meyer. Mead has drawn fire from other candidates for entertaining the possibility of a fuel tax increase, and in general, is suspected by some conservatives of being excessively moderate. Meyer, whose military service in both the Gulf War and in Afghanistan is a key credential, has won backing from Sarah Palin and also from the Susan B. Anthony List, the anti-abortion counterpart to Emily's List. A third candidate, former state legislator Ron Micheli, has been bashing Mead's position on fuel taxes and generally comporting himself as the "true conservative" candidate, obtaining endorsements from Wyoming-based anti-abortion groups. And finally, state House Speaker Colin Simpson, son of former Sen. Alan Simpson, has secured an endorsement from his father's old friend George H.W. Bush.

The late-July Mason-Dixon survey showed Meyer leading leading the field at 27%, with Mead at 24%; Simpson at 17%; and Micheli at 12%.

There's More...

Contract Post

How Stable is the Generic Ballot?

I'm fairly deep in the weeds of building our House forecasting model, which we hope to debut for you at some point late next week. We're basically taking a "kitchen sink" approach -- that is, looking at five or six different sources of information (polls, ratings by professional forecasters like Cook and CQ, fundraising data, etc.), and seeing what has had the most predictive power over the past six election cycles. This is not an easy thing to do -- the data-collection efforts alone are formidable.

One of the challenges I've faced is in coming to grips with the generic ballot, which is the primary indicator of the nationwide standing of the two major parties. The basic question is to what extent the generic ballot ought to take precednece over local-level indicators: for instance, if the generic ballot looks really bad for one party (as it does for the Democrats this year), but the local polls are more favorable, which indicator tends to prevail? I don't have an answer to that yet -- you'll have to tune in next week, I suppose. Still, there are some questions about the generic ballot that I'm now in a better position to address.

For instance: how stable is the generic ballot? I don't mean individual polls of the generic ballot, which in the case of Gallup and some other organizations, can be quite "bouncy" from week to week. Rather, suppose that you're able to remove most of this noise: how quickly can the underlying, macro-level dynamics change when it comes to elections to the Congress?

The way that I've evaluated this is to collect all generic ballot polls since 1998 and looked at what they would have told us at certain intervals before each election. Specifically, I built Pollster.com-style LOESS regression curves around the generic ballot polls, and evaluated the result they would have projected on the morning of the election, and then at 10, 20, 30, 45, 60, 90, 120, 150, 200, 300 and (where there is sufficient early polling) 400 days beforehand. There is no "cheating" allowed: for instance, if a poll came out 199 days before the election, it isn't used in the 200-day forecast, since it wouldn't have been available to us at that time.

The one "fancy" thing I have done is to build in an adjustment to translate registered voter polls into likely voter polls, as we now do for our Senate forecasts. This is worthwhile: although there are some cycles (2006, 2008) where there is little systemic difference between registered voter and likely voter polls, there are other cases (1998, 2000, 2010) where likely voter polls tend to be 4 or 5 points more favorable to Republicans, and accounting for this as early as possible tends to improve the stability of one's forecasts. (There have been no cases recently in which Democrats performed demonstrably better in likely voter polls than in registered voter polls: in years where Democratic mobilization is strong, the two say about the same thing; in bad years for the party, registered voter polls lowball the Republican position by several points.)

Here, for instance, is what the trendline would have looked like at various points during the 2008 election cycle:



As you can see, 2008 was a rather stable cycle. Democrats maintained a consistent lead of about 10 points throughout the entirety of the cycle, and that carried forward to election day, when they won the national popular vote by 11 points.

Several other cycles were also quite stable. For example, 2002:



In spite of the potential idiosyncrasies resulting from redistricting (not to mention 9/11) that cycle, the generic ballot remained quite well-behaved all year.

Likewise, in 2000:



A generic ballot projection a year in advance of election day would have told you pretty much the same thing as one on Election Eve. This is in spite of the fact that the Presidential race that year was one of the more volatile in recent memory.

Nor did the generic ballot move very much at all in 1998, a low-turnout year in which Democrats arguably underperformed given Bill Clinton's favorable standing at that time:



On the other hand, the generic ballot moved a fair bit in 2004. Over the summer, the Democrats built up a generic ballot lead of as large as 8 points, but it evaporated by November; adding insult to injury, the Democrats underperformed their generic ballot standing (losing the national popular vote by 3 points) on Election Day:



There was also a fair amount of movement in 2006 -- and this time, it was uniformly in a direction favorable to Democrats -- although there were some weird polls late in that cycle that perhaps overshot the mark a bit:



***

In spite of these partial exceptions, the generic ballot is generally fairly stable -- almost certainly more stable than something like Presidential polling. This makes a certain amount of sense: whereas something like a gaffe or a victory in a debate can considerably altar the outcome of a Presidential race, there are 435 separate elections to the House, and gaffes in individual districts tend to cancel one another out. Instead, things usually boil down to the national mood that was established during the first half of a Congress's two-year term.

This is basically bad news for Democrats in the context of this cycle: a last-minute reversal of fortunes is unlikely. Where we sit right now, about 75 days before the election, the generic ballot will be off, on average, by only about 2 points from what it will read on Election Morning. The Democrats' standing is poor enough now that a 2- or 3-point shift in their direction would not really be enough to prevent the party from enduring significant losses in the House.

With that said, there is another issue at hand: how much does the generic ballot really tell us about what will happen on Election Day? It might be the case that the generic ballot is fairly stable, but that doesn't necessarily mean it's all that useful an indicator. In addition to the fact that the consensus of polls (however careful we are about calibrating it) might be off in one or the other direction, there's also the fact that the thing which the generic ballot is ostensibly trying to predict -- the national House popular vote -- is relatively irrelevant to the disposition of the chamber, or the number of seats that each party earns. Instead, what we want to know is how the generic ballot translates into each of the 435 congressional districts; this is the sort of problem that we're hard at work upon.

Still, to expect that the national environment will just spontaneously get better for Democrats is probably not realistic. They'll have a poor hand to play, and the task is basically in figuring out exactly how bad the current milieu will translate in terms of a loss of seats.

There's More...

Contract Post

More on FiveThirtyEight's Moving Plans

This has not been the most relaxing summer for me -- lots of Saturday nights spent staring at a spreadsheet, and lots of time on the 2 train shuttling back and forth to Midtown -- but you should start to see the fruits of that labor very soon. We are now planning to migrate FiveThirtyEight to NYTimes.com on Tuesday, August 24th. This is roughly a week later than we had originally planned, but our feeling is that we want to be rolling at pretty much full speed from the very outset of our re-launch there, and so we have indulged in a few extra days of development time.

The next update to our Senate model will likely come on launch day, the 24th. The forecasts will be significantly more rich, interactive and navigable than they are now. The plan is then to debut our House and gubernatorial forecasting models within a week or so of launch -- tentatively, we have the our first House forecast scheduled for Thursday the 26th, and the first gubernatorial forecast scheduled for Monday the 30th, although these dates are subject to change. The House forecasts in particular are, we think, pretty innovative, and will involve our forecasting the outcome of all 435 individual House races as well as the disposition of the entire body. Updates to the Senate, House and gubernatorial forecasts will then begin to cycle through on a regular basis, with each being updated approximately once a week. (For instance, there might be an update to the House forecast each Thursday.)

We also plan to run a quick update of our pollster ratings before re-launching at the New York Times, likely sometime toward the end of the upcoming week.

To answer several further questions about our move to the Times that I've gotten over e-mail:

-- All of our current freelance contributors will be migrating to the New York Times along with me.

-- The main changes you will notice to the content flow are that I will begin posting more frequently after having been on a somewhat reduced schedule for most of the summer. Also, I will be working out of the New York Times newsroom most days and that may somewhat affect the timing and pacing of posts, with a relatively higher percentage of content to be posted on weekday mornings and afternoons; it is unlikely, on the other hand, that you'll be seeing as many posts at times like these, at 1 in the morning.

-- Although, as of the 24th, the front page of FiveThirtyEight.com will re-direct to NYTimes.com, the archives of this site will remain browsable in their entirety.

-- Comments at NYTimes.com will be moderated, which I hope will be a welcome change for 99 percent of you.

-- We will retain our current Twitter feed and it will continue to alert you when new articles have been posted.

-- We will continue to run posts occasionally on non-politics topics, like sports and science, although these will be relatively infrequent at the outset given the immediacy of the midterm elections.

-- As with all content that appears at NYTimes.com, our posts will receive an edit before being published. The most obvious impact of this should be that our copy will be a bit crisper, and that we'll begin to start referring to people as Mr. John Zogby or Mrs. Michelle Bachmann. This is not to say that our teammates on the edit desk will never raise questions when we come to conclusions that are not adequately supported by the evidence -- the Times has high standards, as we do. But the blog should continue to have a strong "voice" and an independent perspective.

-- Finally, I have heard some concerns about the New York Times's metered model, which it says it will implement at some point after the midterms. I would encourage people who have worries about this to browse the entirely of the comments that the New York Times has made on the public record about the model, which is quite different from the versions used by some other news organizations. For example, in addition to the free allotment of pages, users who come to the site through third-party referrers, like other blogs or social networking platforms, will not trigger the pay wall. With that said, I of course hope that you'll at least consider subscribing to the New York Times in print, or one if its various e-reader or digital editions. Having gotten an up-close-and-personal view of the newsroom, I can't emphasize enough how much dedication the New York Times has to its craft, and how much support it provides to its writers in the form of things like editors, photographers, news assistants, its international bureaus, and its exceptional team of graphic and interactive journalists.

***

We very much appreciate your patience and your loyalty to this site and hope you will continue to join us as we embark on new adventures at the Times.

There's More...

Contract Post

Iowa: Eighteen Months And Counting

About eighteen months before the 2012 Iowa Caucuses take place, shaping the next presidential campaign, the state’s second legendary event, the Iowa State Fair, which runs from August 12 until August 22, is being held in a relatively apolitical atmosphere.

But you are never too far from politics in Iowa, and pols were very evident in the vast, three-hour parade that kicked off the parade on Wednesday night. By ancient tradition, current elected officials ride or march at the beginning of the parade, and candidates who are not in office are at the tail-end. Thus, Democratic Governor Chet Culver was right up front, while the once-and-perhaps-future governor, Republican Terry Branstad, was scheduled to appear well behind the Iowa Pork Queen, the Shriners, the vintage John Deere tractors, two roller-derby teams, and the Southwest Iowa High School Honor Band, among many others (which may be why he ultimately let his running-mate represent him in the event). It was, joked many, the first time Culver’s led Branstad all year.

There were only three putative presidential candidates on this year's Fair schedule, Tim Pawlenty, Newt Gingrich and Rick Santorum, but they are simply the first wave. As one of my Iowa friends put it: “Next year at the Fair there'll be so many candidates, you won’t be able to stir 'em with a stick.” Presumably they will be better briefed than doomed 2008 candidate Fred Thompson, who showed up wearing Gucci loafers and spent the day tooling around on a golf cart (a definite no-no unless you are a Major Fair Sponsor; everyone else must walk the goo-encrusted dust or mud).


Even in the dog days of an unusually hot summer with widespread recent flooding, the midterm campaigns here are gearing down for a heavy stretch-run, fueled by the generous subsidies that would-be presidents routinely lavish on the state parties, and watched by an unusually checked-in electorate acutely aware of its role in national politics.

Iowa Democrats had a breakthrough year in the last midterms in 2006, winning control of the state legislature and the governorship together for the first time in 42 years. (They also picked up two congressional seats.) And in 2008, obviously, Iowa Democrats played a big role in the eventual nomination of Barack Obama, who carried the state handily in November. Iowa’s reputation for progressivism was also burnished in 2009, when its Supreme Court legalized same-sex marriage, making the state an unlikely spot for destination weddings. (One t-shirt I spotted at the State Fair had this legend: “8-4-10: California Is Finally As Gay As Iowa”).

But even though Iowa’s unemployment rate is well below the national average (6.8% in June), it has not escaped the sour mood that’s affecting politics, and particularly Democrats, nationally.

The President’s job approval ratings in Iowa aren’t that bad; according to a Rasmussen poll in early July, they stood at a 48-52 ratio, better than the 44-55 ratio the same pollster showed nationally at about the same time. But Chet Culver’s job approval ratio has been in the negative range for months in every poll taken since last fall; the Des Moines Register had it at 36-53 in February (with only 37% approval in union households, reflecting Culver's stormy relationship with public sector unions), PPP showed it at 28-56 in early June; and Rasmussen had it at 37-61 in early August.

Unsurprisingly, Terry Brandstad, who served as governor from 1983-1999 before "retiring," has been leading Culver by a minimum of fifteen points in every major poll taken in the last year. Branstad's biggest hurdle so far was winning the Republican nomination against 2006 nominee for Lt. Gov. (and 2008 Mike Huckabee Iowa campaign chairman) Bob Vander Plaats, whose underfunded campaign was a redoubt for restive social conservatives whose mistrust of Branstad goes back quite some time. Branstad's underwhelming 50-41 win in the June 8 primary--even after he was endorsed by Sarah Palin--was a reminder of the power of conservative activists in this state, particularly in lower-turnout caucuses like those that serve as an abbatoir for presidential candidates. Indeed, Branstad's camp had to endure post-primary reports that Vander Plaats was considering a third-party run that would have instantly made the general election a barnburner, before Vander Plaats finally announced he was devoting his immediate future to an effort to recall the State Supreme Court Judges (two of whom were appointed by Branstad) who legalized same-sex marriage. Vander Plaats has not, significantly, endorsed Branstad.

There are recent signs that Chet Culver's repairing his relationship with key elements of the Democratic base in Iowa (including those public-sector unions who have bad memories of the Branstad Administration) and the gubernatorial race could well tighten up. Moreover, Republicans are not currently favored to pick off any of the allegedly vulnerable Democratic U.S. House Members; top target Leonard Boswell's race is currently rated "Lean Democratic" by the Cook Political Report. The fight for control of the state legislature will be vicious and probably close.

But for those interested in Iowa primarily because of its role in the presidential nominating process, the things to watch are more limited. If Branstad wins, will his close association with Mitt Romney matter a lot going into the Caucuses, or will Palin's late (and unsolicited) endorsement of Branstad help keep him neutral? Will the Iowa conservative crusade against same-sex marriage in Iowa help mobilize right-bent activists and help candidates other than Romney (including perhaps Huckabee, who beat Romney here in 2008 despite a vast financial disadvantage)? Will candidates like Tim Pawlenty and long-shot Rick Santorum become viable by spending a lot of early time here? And will some potential president make Fred Thompson's mistake and violate the unwritten but iron rules of Iowa culture between now and then, perhaps disdaining a bite of Hot Beef Sundae or deep-fried Oreos, or failing to express admiration for the winner of the Big Boar Contest, with cameras nearby?

Those of us in the commentariat wondering about this right now are few, but a year from now, like the candidates at the Iowa State Fair, you won't be able to stir 'em with a stick.

There's More...

Contract Post

ESPN Umpire Study Blows The Call

Suppose that an organization studied videotape of Major League Baseball umpires over a two week period. They reviewed every play in every game, other than balls and strikes. And they found that, in the 184 games they studied, there were only 47 missed calls -- about one for every four games.

That'd be quite vindicating for the umpires, I would think. Just 0.2 or 0.3 missed calls a game? A 2004 study of NFL games, by contrast, found 40 reversals on challenges initiated by the replay booth in the final two minutes of each half**; that would extrapolate out to 600 miscalls over the course of the entire game, or about 2.3 per contest (and not all calls are reviewable). And several NBA insiders that I spoke with for the book chapter that I'm writing about hoops said there were 15 or 20 "questionable" calls a game in their sport.

Indeed, this is exactly what a study by ESPN just found. Baseball umpires very rarely blow calls.

But that's not the framing that ESPN used. Instead, they said that umpires missed 1 in 5 "close" calls, which sounds much more damning. The question, of course, is how one defines a "close" call -- something which is completely arbitrary. If ESPN had used a more expansive definition of a "close" call, perhaps umpires would only have missed 1 in 10 "close" calls, or 1 in 20, rather than 1 in 5. If they used a narrower definition, perhaps the umpires would have missed 1 in 3. All of which tells you nothing about the performance of umpires and a lot about the semantic proclivities of a bunch of research assistants sitting around a conference room somewhere in Bristol.

__

** The reason it's preferable to look at this statistic, rather that at the number of reversals on coach-initiated challenges as occur in the first 28 minutes of each half, is because coaches are limited in the number of challenges they may make and penalized for incorrect ones with the loss of a timeout. Thus, many incorrect calls will go undetected, because it is not worth it for a coach to initiate a challenge, even if there is some likelihood that the call was incorrect. The reply booth has no such restrictions, however, and should therefore provide for a more reliable estimate.

There's More...

Contract Post

Palin's Endorsements: More Method Than Madness

Yesterday my esteemed FiveThirtyEight colleague Tom Schaller published a post suggesting that Sarah Palin isn't doing herself or her party any favors by endorsing candidates in Republican primaries around the country (with the possible exception of accidentally helping candidates whose moderate credentials she indirectly burnishes by endorsing their opponents, like Bobby Ehrlich in Tom's home state of Maryland).

Since a big chunk of Tom's argument is based on the testimony of a congressman from my own home state of Georgia, and the results of Tuesday's gubernatorial runoff there, I feel entitled to dissent, at least in part. You don't have to agree with Michelle Cottle's assessment of Palin as a strategic genius to conclude that St. Joan of the Tundra is not just messing around pointlessly.

In an interview, Congressman Jack Kingston implied that Palin was "meddling" in Georgia to the detriment of her endorsee, who was, he complained, "clearly the more moderate person in the race." Kingston was a supporter of Nathan Deal, who edged Palin's candidate, Karen Handel, in the runoff, so he's not exactly objective on the subject. In fact, Palin's original endorsement of Handel occurred just as she was beginning her ascent from second or third in the primary field to first (the same uncanny timing she showed in South Carolina with her endorsement of Nikki Haley). Her single personal appearance with Handel, the day before the runoff, probably occurred too late to matter much either way, and in any event, Handel's loss by a couple of thousand votes after more than a year as an underdog doesn't seem that bad a performance.

As for Handel's ideology, which is presumably germane to the question of whether Palin knows what she is doing, Kingston is faithfully repeating the Deal campaign's "liberal" spin on Handel. But for the record, her platform included abolition of the state income tax, a very hard line on immigration, and support for a ban on abortions that displeased Georgia's right-to-life lobby only because she insisted on rape-and-incest exceptions and wouldn't support sharp restrictions on IV fertility clinics--hardly raging liberalism. Aside from Palin, Handel was also strongly supported by RedState's Erick Erickson (a Georgian), whom nobody would describe as a "moderate."

More generally, it's important to remember that Palin's "meddling" in Republican primaries has involved very different levels of activity. In several cases (most famously her last-minute, unsolicited endorsement of Terry Branstad in Iowa) she's put up a statement on Facebook and left it at that. In a few others (e.g., Carly Fiorina of California, Todd Tiahrt of Kansas) she's recorded robocalls for endorsees, a very common and low-risk tactic so long as the calls don't involve negative attacks on other Republicans. Only in five so far has she personally campaigned with "her" candidates: in New Mexico for Susana Martinez (who won), in Idaho for Ward Vaughan (who lost), in South Carolina with Nikki Haley (another win), in Georgia, with Karen Handel (who made a runoff, which she lost by an eyelash), and in Arizona, with the man who made her famous, John McCain (McCain's primary is on August 24, but he's heavily favored to win).

It's also worth remembering that other potential 2012 presidential candidates have heavily engaged in endorsements and robocalls, along some personal campaigning (e.g., Mike Huckabee and Newt Gingrich with Kingston's candidate Nathan Deal) without raising nearly as many questions about "meddling" as has Palin.

Tom suggests that the only rational factors that would lead Palin to make endorsements in competitive races involve either strategic goals or personal connections, and he's right, Palin's endorsement of the obscure Maryland candidate Brian Murphy (or for that matter, of Bob McConnell in Colorado) doesn't seem to meet those criteria. But that's not to say that many of her other endorsements don't make sense on those grounds.

Most obviously, she's endorsed successful Republican gubernatorial primary candidates in two of the four states that will kick off the 2012 presidential nominating contest (Iowa and South Carolina), and has also endorsed the front-running candidate for U.S. Senate in a third (New Hampshire). Neither the State of Maryland or Bobby Ehrlich is likely to play a major role in 2012, so while her endorsement of Murphy may not help a possible presidential campaign, it probably won't hurt it, either.

As for "personal connections," there's clearly one at play in Arizona. Family issues have also been a factor. According to some reports, her ill-fated involvement in Ward Vaughan's campaign in Idaho could be attributable to her father, Chuck Heath, who endorsed Vaughan back in 2009 after meeting him during the presidential campaign (Heath also was an early backer of Danny Tarkanian in Nevada, which may well explain why his daughter didn't jump into that primary and endorse "Mama Grizzly" Sharron Angle or early front-runner Sue Lowdon).

Family aside, I'd argue that the whole "Mama Grizzly" phenomenon is deeply personal to Palin. She's very invested in the idea that she's a pioneer for a new breed of conservative women who are shaking up the GOP and politics generally, and her endorsements of Martinez, McGowan, Bledsoe, Fiorina, Haley, Fallin, Heil, Handel and Ayotte all meet that criterion as well.

There may be something else going on with Palin's career generally that is difficult for those of us who don't share her ideology to comprehend, but that could be quite real to her: she truly does think of herself and most of her endorsees as, well, "mavericky." A number of her endorsees, including some of the less likely ones such as Heil and McConnell, but also bigger names like Haley and Handel, have been candidates with unusual backgrounds who were struggling to raise the money necessary to become or remain competitive candidates. For all her disdain for the "lamestream media," the one thing Sarah Palin knows she can offer, instantly, is free media attention.

And to get back to Jack Kingston's characterization of Palin's "meddling" in Georgia, that's exactly what she offered Karen Handel, a "Mama Grizzly" whose message as a "conservative reformer" was, from Palin's point of view, a "mavericky" assault on the good ol' boys of the Georgia GOP, and who was struggling to keep up with John Oxendine, Eric Johnson and Nathan Deal on the fundraising trail. These are all qualities that made her campaign remarkably similar to that of Nikki Haley in next-door South Carolina, who never raised a lot of money and was ultimately lifted to a big primary and runoff victory by clumsy sexual and ethnic allegations that made every other factor, including Palin, largely irrelevant.

I still don't know what business Sarah Palin has endorsing Brian Murphy. But describing her general pattern of endorsements in the midterms as irrational or counterproductive strikes me as far too sweeping a generalization. I invite Tom to retreat to a more natural posture of disagreement with Jack Kingston.

There's More...

Contract Post