Skip to Content
Sign up for Grass Roots Public Affairs Quarterly Newsletter Twitter Facebook

Polling with Purpose: When WHO you ask is more important WHAT you ask

Sun 5th Feb 2017
By Adrian Macaulay, Polling Specialist, MA

Political pollsters put a great deal of effort in designing questions that explore the opinions people have towards a particular topic or issue, and what behaviours those people are likely to exhibit. However, the most important question a pollster needs to both ask and answer is: Why?

Why did/didn’t voters cast their ballots for a particular candidate?
Why do voters in a certain geographic district hold a negative view towards a certain policy?
Why did voters who identified with Party X last election plan to vote for Party Y in this election?

While there are a number of questionnaire or survey techniques that can help answer ‘why’ questions, the pollster’s client needs to keep something very important in mind:  data-collection agencies are likely to charge for each completed interview – so be sure to give serious thought as to who you want to make up your sample before considering what you want to ask them. Even the most thought-out and considerate question will yield little-to-no value if the wrong people were selected to participate in your survey.

When conducting opinion research, think carefully about the type of respondent that would help explain the why factor of your problem or issue(s). Start by establishing who you want to be in your sample: what do the demographics of your ideal sample look like? How are those demographics likely to be reached? What line of work would they be in? Do respondents work at all? Consider at what time certain respondents should be contacted to take your survey. Are there any sensitivities among certain demographics that you need to consider?

If you are experiencing difficulty in determining who would be a good fit for your sample, perhaps think about who you don’t want to be part of your sample to help eliminate respondents who could skew your findings.

Once you determine who will be in your sample, give thought as to your sampling frame: where are you going to draw your respondents from? If you’re drawing respondents from a phone list, be sure to consider the types of people who would have a land-line and those who would be reached on a mobile-device (for example new-comers are unlikely to have land-lines, whereas older respondents are more likely to have a land-line than a mobile-device).

Now you need to think about how you’re going to go about collecting responses from the people you have identified in your sampling frame. A good sampling technique can make all the difference! Most public opinion polls you see in the media are performed through probability or random sampling whereby pollsters are trying to gather “public opinion” at large, and therefore give each eligible member of the population an equal chance of being selected to participate in the survey. There are situations, however, where the client has a firm understanding of who are trying to gather responses from and use judgemental or purposeful sampling to collect data. Just keep in mind that the findings from non-probability samples cannot be attributed to the public at large.

Another important aspect you need to consider when planning your sampling technique is the actual process of data collection. What method are you going to use: Online? Live-Caller? IVR (robo-polling)? Some pollsters swear by a particular method as the best way to collect opinion data, while others may find a use for all three methods. Be sure to consult with your pollster to find out which method best fits with your proposed study – and your budget!

By this point you have an idea as to who will be in your sample, where they are likely to be found, and how they are going to be contacted. Now you need to determine a sample size. Sample sizes vary depending on a number of variables such as the nature of the research or the rate at which respondents are willing to take your survey (response rate). However, the client’s budget usually influences the size of the sample, so keep an eye towards the cost-per-completed-interview (CPI).
Time to execute the survey! Be sure to work with a data collection firm that will listen in or check up on interviews while the poll is in the field to ensure that the responses they are collecting come from the people they need to be talking to.

Once you have your sample of completed interviews, you need to validate the sample. Ensure respondents that completed your survey were the respondents that you needed to hear from and remove responses that may skew your sample. Often times weighting or other corrective measures are often used to help validate the sample. If you are conducting an opinion poll of a broad population (ex: Canadian citizens or residents of Manitoba), ensure that the data you get is representative of that population.

Following these steps can help ensure your survey gets to the heart of your issue in an efficient and cost-effective manner.

Adrian Macaulay is President of Delphi Polling & Consulting Inc., a research firm located in Downtown, Toronto.