Which of these approaches to measuring sponsor recall do you think is best?
Best practice: Select one sponsor and two non-sponsor competitors in each category. Ask for “the” correct partner (Option A).
Why not select two sponsors if the category is not exclusive? Suppose you have two casino/resorts as partners. Why not ask about both at the same time?
- Specificity/Validity: Asking which brand is “a” corporate partner is a different mental process than asking which is “the” corporate partner.
- Asking which is “a” corporate partner using check boxes (as many as apply) induces more guessing and second-guessing.
- It’s like the dreaded test questions you had in school with “all of the above,” “none of the above,” or “All of the below are true except….”
- The best multiple choice questions that measure what you say you’re measuring have one correct answer.
- Results: Respondents develop different guessing strategies that may actually reduce recall results under conditions of greater uncertainty.1
- With years of experience experimenting with myriad methods of measuring recall, we found allowing for multiple correct choices reduces recall levels.
- Many will guess one or the other (not both) sponsors when they would have most certainly selected the correct brand against two non-sponsors.
- More respondents also guess the wrong brand. (“Hey, maybe I just didn’t notice or have forgotten?”) Since no penalty exists for guessing extra sponsors, they add choices they otherwise would never consider.
- Cognitive load: We reduce cognitive load with one correct answer that stands in contrast to two incorrect answers. Providing two potentially correct answers (sponsors) increases cognitive load and leads to mental errors.
- Asking respondents to identify the single correct sponsor allows it to stand in contrast to the competing non-sponsor brands, thereby enhancing brand equity effects.
- Having two major sponsors in the same category is like asking a mother to evaluate which is her favorite child. The question is much easier, requires less cognitive resources, to compare one sponsor (or one’s own child) to others that do not belong.
- Contrast effects: The elements of brand equity will stand in more stark contrast to non-sponsors, but will diminish when compared to similar others. In other words, the brand lift will not be accurately measured versus non-sponsors (viz., where the brand would be without the team) if we include more than one actual sponsor.
Finally, for business purposes, we offer and price reports based on one partner in each category. Adding a second partner within a category would mean another report. This adds additional programming for us and additional cost to the team. If both partners in a category would like to compare how they are doing with each other, then we simply replicate that category in a separate set of surveys–using the same non-sponsor competitors if desired. Then, we’re comparing apples-to-apples and we know respondents were not confused. We’re measuring what we say we’re measuring (validity) and we know if we repeated the measures in the same way, we’d get similar results (reliability).