Issue #24 - August 2012
A Brief Guide to Market Research
When undertaking market or social research, we are often asked the question "how can you evaluate the opinions of the entire city when you only interviewed a small number of people?"
In survey methodology, random sampling is used to select a representative sample of a population. For a sample to be random, every member of the population must have an equal opportunity to be selected. Random sampling is a statistical/mathematical process that allows for calculation of the number of sample interviews to be completed relative to the size of the population, in order to produce the desired level of accuracy.
Accuracy is based on the confidence interval, which is plus or minus a percentage, and the confidence level, which is an overall percentage. The market and social research standard is a confidence interval of ±5% and a confidence level of 95% and these are used to determine sample size. What this means is that if 50% of the sample population give a particular answer to question, we can be 95% certain that between 45% and 55% of the total population will also give the same answer. The further the response of the sample population is from 50% (for example, 87% gave the same answer), the smaller the confidence interval and the more accurate the result.
We recently undertook a survey of households connected to the town water supply in Hinchinbrook Shire, which was very simple and comprised only two questions. We completed the survey using computer aided telephone interviews (CATI), randomly selecting telephone numbers from across the Shire. The first question asked respondents if their property was connected to the town water supply and eliminated those who were not. From there, we went on to complete 450 (single question) interviews, although only 375 were needed to provide the minimum sample size required to achieve standard accuracy. Council decided we should do an additional 75 interviews because this amounted to 10% of the total population. These additional interviews provided a slight increase in accuracy to a confidence interval of 4.3%.
When analysing research, it is important to keep in mind that the overall accuracy of survey results are indicative, not absolute. It is also important to remember that any survey is only as good as the questions asked, which is why we prefer to test a survey instrument before undertaking a full survey, rather than just diving in. From time to time, we find that what seems like a sensible and straightforward question elicits responses that are quite unexpected and not providing the information required.
At Balfour Consulting, we use specialist commercial survey software that allows us to easily input data and extract results in tables, charts or direct comment. The more difficult task is interpreting results to get a real understanding of the overall findings of the survey and what we are trying to learn. The one message we give all our clients is that we will provide them with the most accurate data possible, what they do with it is their business. What we will never do is undertake research in order to establish a predetermined result.
As a regionally based research company we are significantly more cost effective than statewide or national survey companies and are happy to undertake small market or social research projects to meet the needs of local government or small business. Contact us if you have any questions or want to discuss how Balfour Consulting can help with your research needs.