I sat in on a meeting recently in which a rep from a TV station said with apparent seriousness that their research showed that people who saw a TV ad three times would always call the advertiser.
This sounds like a bold-faced lie, especially since it followed a discussion in which we were assured that there was no way to track the results.
But what if it had been a more plausible statistical claim? What if the TV ad salesperson had dressed it up and said something like, “Of customers surveyed who remembered seeing a specific TV ad three times, 67.8% said that they intended to find more information about the brand.” Would that have been more believable?
Here’s some data about the effectiveness of TV ads:
- 72% of ad execs surveyed felt that online videos are as effective or more effective than TV ads… but “as effective” and “more effective” are two different things.
- Interactive online ads perform better than TV ads shown on the web… but TV ads are designed for TV, not for the web.
- Over the past 30 years, TV outperforms other media… but 30 years ago online ads didn’t exist.
- Nielsen reports that only 47% of consumers around the world trust TV ads… but is that percentage true for your target audience?
- 30-40% of TV watchers are using their phones while they watch… but are they just checking their mail during commercials or are they discussing the show with friends?
- A recent survey found that only 14% of viewers watch commercials; the rest avoid them by recording shows, leave the room while the ad is on, or check their phones during the ads… but this was an online survey, so maybe the die-hard TV fans were not included.
Nothing in there about everyone who sees an ad three times making a call, but there are issues with all the stats we found, and some have more issues than others. We have no idea how the TV station came up with their claim, but we’re pretty confident that it didn’t involve the kind of research methodology Nielsen uses.
On the other hand, the alternative to using statistical data is often just “going with your gut” (IBM says 80% of traditional marketers do that, by the way). We know that’s not effective.
So how can you make statistics useful?
- Consider bias. If the person who shares the stats with you is trying to sell you something, dig deeper before accepting it.
- Find the raw data if you can. The headlines for most of the stories using the “72% of ad execs say…” information above said that online ads are more effective. That’s not actually what the data says. Nielsen is all about research and their sample sizes are huge, so we feel pretty confident about their claims. For Polly C. ‘s hub page, we want to see the specific numbers and find out how they were gathered. Either way, we know that headlines aren’t always reliable, even if the reporting is. Track down the actual data and make sure it says what you’re being told it says.
- Collect and use your own data. General statistics can be useful and instructive, but your own data should be the basis for most decisions. Larger sample sizes and controlled research settings can give you much more useful information than a small data set from your website, but you should at least compare your data with the third-party data before you make up your mind.
Leave a Reply