Sample is often treated as a commodity, with price and ease of availability often driving choice. But what about sample quality? What about reliability? Does sample source, how the panel was recruited or routing make a difference?
If sample is variable and unreliable, what are the implications for business decision making? If results cannot be reliably reproduced, can we claim that market research is based on science?
This paper raises critical questions and provides provocative evidence of the state of sample quality in Canada.
In this paper we draw on an extensive body of research on research regarding sample quality and reliability. Comparing river sample, routed sample and sample from multiple panels with different recruitment techniques, we report on variation between panels in the data collected, variation in answers from wave to wave and the effects of recruitment source on representativeness.Since sample is the soil and water of market research, it is critical that we pay attention to quality of the source from which our research springs.