Ballot measures tend to be much harder to accurately poll than partisan elections. Each initiative is a unique question with its own quirky set of parameters regarding which demographics and psychographics are most likely to support or oppose it. In comparison, all partisan elections between a Democrat and Republican are, at a core level, fairly similar.

In partisan races, voters have party affiliations that are often an excellent indicator of how they will vote. For example, registered Democrats almost always vote for the Democratic nominee, and registered Republicans almost always for the Republican. Even without polling, it is often possible to make a fairly accurate guess about the eventual result of a partisan election based just on the past voting history of the electorate.

In addition, pollsters have been basically asking the “do you prefer the Democrat or the Republican for office X?” question for decades. Pollsters have had a long time to refine their models about this basic question. They have learned what is the important and unimportant demographic data you need to make sure to weight to make those polls accurate.

Of course the most important demographic data for a partisan election might not be the most important for a ballot measure. It could be region, education, religion, job, number of kids, or a myriad of other defining characteristics, based on the specific issue of the ballot measure.

Let’s look at the polling for propositions in California in 2008. There were two highly controversial ballot measures: Prop 4, a measure requiring parental notification for abortions, and Prop 8, a measure to deny marriage to same-sex couples in California. There was also the important but lower profile Prop 11, which concerned electoral redistricting. Below are the last polls taken before the elections. Field and PPIC are California-specific pollsters with very good track records, and SurveyUSA is one of the most accurate national pollsters.

President SurveyUSA Field Poll PPIC Final
Obama 60 55 56 61
McCain 36 33 33 37
Prop 4
Yes 40 45 46 48
No 46 43 44 52
Prop 8
Yes 47 44 44 52.2
No 50 49 52 47.8
Prop 11
Yes 36 45 41 50.9
No 29 30 34 49.1

Not surprisingly, the polling for the presidential election was fairly accurate, and noticeably more accurate than the polling for the propositions. Despite Prop 8 being a very high-profile measure, with high voter awareness and major campaigns on both sides, the final margin of victory was outside all three polls’ margin of error. Interestingly, there wasn’t just an across-the-board under-sampling of conservative voters—“Yes” on Prop 8 outperformed the polls, but Prop 4 polling from Field and PPIC did not meet any major swing to the right, but missed the surge in leftward “No” votes.

The accuracy of the polling for Prop 11 is even more telling. While a relatively important measure, it didn’t attract the high-profile campaigns that Prop 8 did. I wouldn’t be surprised if a quarter or more of voters didn’t know anything about the measure when they entered the voting both. With such a large unaware and undecided vote, truly accurate polling is basically impossible.

By their nature, ballot initiatives are more difficult to poll than partisan elections. It can become even more difficult with measures like 2008’s Prop 8 and this year’s Proposition 19, where people might not feel comfortable telling pollsters their true feelings. It is just a reminder that polling of ballot measures is an imperfect indicator of the final result, and certainly shouldn’t be treated as gospel.