Advocacy Guide

5. Poll your policies

If you are pushing an important policy and spending a good deal of your organization’s money, it makes sense to conduct a poll. How extensive and expensive the poll should be depends on how much you’re spending on the overall policy campaign, including the cost of staff time. You might spend up to ten percent on public opinion research.

The point of polling is not to decide what policies you favor or oppose, which is a question of your values, it is to:

• Decide which policies to advocate in which situations. You should test what people think are problems as well as alternative solutions.
• Decide how to describe your policies. This is message testing: what works, what doesn’t, and why.
• Determine if your policies are important enough to voters that they might consider switching candidates based on their issue positions.
• Persuade policymakers, grasstops leaders, opinion writers and donors to support or join the coalition. They want to see poll results in order to decide whether to invest their own reputations and resources in the cause.
• Generate press coverage and raise the credibility of the campaign.

To understand what kind and size of poll you need, here are some basics:

If poll respondents are selected in a proper random manner then, based on the principles of statistics, they will accurately represent the opinions of the population at-large. Polls express a mathematical probability—traditionally a 95 percent likelihood—that the poll results are within the margin of error of what the whole population believes. The margin of error overwhelmingly depends on the number of poll respondents.

With 95 percent confidence:

Sample of 2,401 respondents = 2 percent margin of error

Sample of 1,067 respondents = 3 percent margin of error

Sample of 600 respondents = 4 percent margin of error

Sample of 384 respondents = 5 percent margin of error

Sample of 96 respondents = 10 percent margin of error

In other words, in a well-executed poll answered by roughly 384 people, the results will be accurate 95 percent of the time within plus or minus 5 percent. Or, for example, if a poll of 600 people finds that 65 percent favor increasing the minimum wage to $15 an hour, that means that in the represented population there is a 95 percent chance that support for the policy is as high as 69 and as low as 61 percent. In one poll out of every 20, support is actually above 69 or below 61 percent. (Although the math is more complicated, this simplification is what advocates need to know.)

In short, you almost never poll fewer than around 400 respondents. The results become too imprecise and both reporters and policymakers may discount the reliability of a smaller poll. If you can afford it, it’s preferable to poll between 600 and 1,000 respondents, because this allows for reliable cross-tabulations. Taking the same minimum wage example, if about 100 respondents are African Americans and 80 percent support the minimum wage increase, that means among the whole population of African Americans there is a 95 percent chance that support for the policy is as high as 90 percent and as low as 70 percent. If you wanted to know what a smaller group believes, perhaps people under age 30, there probably wouldn’t be enough respondents to give a statistically significant answer.

There are several different ways to conduct a poll: with a live phone bank calling landlines or a combination of landlines and cellphones, entirely automated calls, entirely online surveys, or a combination of phones and online. If you have an experienced pollster, any method can be made to work.

If you plan to use a poll to gain donors and supporters, impress the media, and persuade policymakers, you’ve got to use a pollster that they will believe. In some places, you can use a local pollster who has built up a good deal of credibility with opinion leaders. Otherwise, national pollsters—although more expensive—are probably your best bet.

Displayed below, for example, see topline results from a survey commissioned by the Public Leadership Institute. It polled over 1,000 likely voters to get statistically significant results for a variety of demographic groups, it used live callers to both landlines and cellphones, and it was conducted by a top national pollster, Ann Selzer of Selzer & Company.

Conservatives perform public opinion research constantly. Virtually all of their major efforts are poll-tested. If we want to win major battles, progressive advocates need to test legislation, messages and talking points.