Apples and watermelons may be a good combination for Nerds, but terrible for analytics.
If you're in digital media measurement, you've probably heard this from your boss, "ComScore's got it wrong, our site has twice as many uniques. Have them fix it!"
Well, he/she is right; you probably do have twice as many. But guess what … in the end, it doesn't really matter [I'll explain why later], but first let's explain why they're different.
My favorite analytics guru Avinash Kaushik recently listed 7 common analytics mistakes and first on the list was ... "Never Compare Apples to Watermelons."
And one of the most common "Apples to Watermelons" examples I can think of happens in media measurement -- comparing internal metrics (Omniture, Web Trends, Google Analytics) to external metrics (comScore, NielsenNet, Compete).
There are 2 main reasons why this is a fruitless task ... let me count them.
1. Differences in the data source -- Most metric measurements are based off a specific data set. Typically it's the data the vendor or company has access to, has purchased or has built. Different sources will generate different results. Internal metrics generally rely on a javascript beacon that passes information. External sources don't have permission to drop a beacon on your pages, so they often use panel-based methods (large panels like Compete, or smaller targeted panels such as comScore).
Internal metrics depend on users' computers to accept cookies and run javascript. External metrics depend on mathematicians to extrapolate what 1 million people in a DMA do based off the actions of 200 monitored panelists.
2. Differences in the definitions -- a pageview is a pageview right? Well yeah, but what about reloaded pages, what about pages viewed by people not represented in panels, what about pageviews viewed from a mobile device? You can see how each source defines their metric by what they have the ability to count or estimate.
Great, then what's the solution?
Well, first is a discussion with your boss to help him/her understand that his competitors deal with the same 40% to 60% comScore metric discount that you do.
Second, measure yourself versus the competition based on same external metric source -- so that you're measuring "apples to apples". Then identify exactly how they compute their measurements and ensure your site is registered with all means possible. Adding tags to your site pages can aid a company like comScore or Quantcast in measuring your audience more accurately.
Third, measure your individual site performance based on the more detailed internal metrics. Focus on relative metrics like percentage growth, month-to-month and year-over-year. Focusing on magic number milestones can obscure recognition of true progress and serious problems.
As with most problems, there’s no black-and-white answer – but understanding the core issue is half the battle enroute to a more actionable strategy.
Comments [0]