Once partnership sales & service complete the customized brand actions and the surveys are complete, the team receives a URL for each survey that covers three partner brands and two competitors each.
Anonymous & customized URLs
The survey URL we send the team is in this format to be embedded in a survey link (e.g., “Join Your Fan Panel.”):
Anonymous survey link: https://teamsportsmarketing.iad1.qualtrics.com/jfe/form/SV_7WjOuk6NQCMjr0N
Some teams track respondents by unique identifiers to store in the database. To know who responded without asking for emails, we need a unique ID for each user. That unique ID is added to the URL of the relevant survey for each email distributed. We insert a hidden value in the survey (G8HX7A, where A stands for survey group #1). The email system pulls the unique user ID and places it after the ?G8HX7A= in the URL the team embeds in the emails:
Customized survey link: https://teamsportsmarketing.iad1.qualtrics.com/jfe/form/SV_7WjOuk6NQCMjr0N?G8HX7A=123456789
The URL to the survey persists (doesn’t change) even if we make changes to the survey itself. Changes to the survey can be made at any time in the entire process.
Sample design, confidence intervals & response/completion rates
To generalize to the entire population of the team’s fans, we need a sample (N = 25,000) representative of season ticket holders, single-game buyers and other users for both preseason and postseason for each survey. Substantive incentives may reduce sample size needed. More on that later.
Suppose a team has 300,000 addressable users in the database, with 30,000 season ticket holders, 100,000 single-game buyers and 170,000 other users. For each sample of 25,000 we’d need to select from each group in a random 1 fashion:
- STHs: 2500 (10% of 25,000)
- Single-game buyers: 8,333 (33% of 25,000)
- Other users: 14,167 (56.7% of 25,000)
If the team studies 18 partners (with 3 partners in each survey), that means 6 surveys & 6 samples for both preseason and postseason for a total of 12 samples. Ideally, that requires a total addressable database of at least 300,000 (25,000 X 12). We can experiment with smaller sample sizes and try/hope for higher response rates using incentives. Some fans, typically of winning teams, respond better than others. Response rates are often higher or lower in postseason (compared to preseason) based on team performance.
Once the population of the fan base gets over 250,000 the sample size needed for a 99% confidence level with a margin of error of 5% (N =663) or 3.5% (N =1347) remains the same. So, bottom line, we need at least 663 to be 99% confident that we are +/- 5% on our measures. More is always better, but +/- 2.5% (N=2626) is cost-prohibitive. If the team’s database is limited, we might use samples of 12,500 and know we are unlikely to be as accurate if we get a response rate below 663.
Typical completion rates with no incentive (“please help”) or little incentive (drawing) for NFL, NBA, and MLB teams range from 2.5% (N = 625) to 7.5% (N = 1875). The trick is getting fans to begin the survey, as 75% will complete once started, even when the survey takes 5-10 minutes. Response rates (click-thru’s) are considerably higher than 2.5-7.5%, so sending reminders a few days later can bump up completion rates substantially.
The most effective are simple requests to help the team. Personalizing helps. Telling them how long it will take helps completion rates. Giving a due date within 3-5 working days provides urgency. Research shows longer time periods encourage procrastination and reduces response rates.
Incentives are tricky. Incentives with high value can help, but we’ve done as well asking, “Will you help?” followed by the survey link. If inventory is available, free or discounted tickets may help, but may bias the sample toward those responding to the incentive. That is partially remedied by adding, “Redeem for yourself or for family and friends who love the team.” Fanatics reports offering a free cap to redeem at the team store increases response rates and may be offset with additional purchases by fan visits. So, if your merch partner is willing, it’s worth exploring. We’ve found offering discounts on merchandise may backfire in postseason because fans compare the discount to other (lower) clearance prices.
Non-fans comparison group
Partners may choose to compare results with a non-fan group representative of the DMA. In conjunction with the postseason studies, we will measure the same brand measures (regard, relevance, differentiation, trust, usage, brand preference and brand engagement) for up to nine partners and one competitor each. The sample is provided by Qualtrics with the following characteristics representative of the population according to the latest census data:
- Gender (typically 50/50)
- Age groups: 18-29, 30-39, 40-49, 50-59, 60+
- Based on zip codes within the selected market (typically within 120 mile radius)
- Attended one or fewer games AND watched 3 or fewer games
The costs of the subjects for this study (~$5,000) are passed through directly to the team and billed accordingly. We administer the survey and provide the data & results.