Where sample sizes are concerned, sometimes more isn't better, it's just ... more.
It's pretty common for people to take comfort in large numbers, because they assume that with smaller samples, the results might not be as stable. That's often not the case, however, and working with smaller samples actually has a few benefits:
- You can still get reliable results at the 95% confidence level, even if you don't have 16 million participants. There are some useful sampling guides (see below), but so long as you follow them, you should wind up with stable, projectable results.
- If you have your own panel, running large-sample activities means that you're contacting a large chunk of your panel every time you launch an activity. If you're selective and endeavor to get the number of completions needed to hit that 95% confidence mark, and not much more, you run a far lower risk of fatiguing your panelists. Trust us: you'll see less panel attrition if you throw fewer activities at any given panelist. Keeping panelists is cheaper than recruiting their replacements.
- It saves you money. If you only need 1,000 completions but pay for 5,000, you're spending serious money for data that's of incremental value. Especially if you're recruiting through a reputable panel provider, where cost-per-completion is somewhere around $8 at the lower end, that's $32,000 of unnecessary sample expense!