Google Analytics is a fantastic and powerfully tool, it can be easily misunderstood and inform bad decisions and investments. I know, its got amazing features, it's really easy to set up, it's a doddle to get figures out of, on the surface it is a great example of superb product management. But is it?
Before going any further it must be clear, I am a really big fan of Google Analytics. I really like custom segments, reports, cross domain tracking, revenue goals, attribution modelling and user explorer.
As a "business intelligence" system it is amazing. Unfortunately its failures are due to its successes. Google Analytics is mega accessible. In the past business intelligence tools were hard to use and very complex to set up. Only expert analysts trained in such matters could abstract learnings. Google has done lots of difficult work for everyone (yeah!), but this means very few people who use it actually know what it is telling them.
Consider something as simple as "users", what is a Google Analytics "user"? When some marketing manager and decision makers hears this word they think it means a "person", but it doesn't - it means a unique device with a unique cookie. So a successful media site that engages with its users may look at their 3m returning "users" and assume they have 3m engaged people using their site frequently - this might be the case, but it might also be 1m engaged people who engage with that site on their mobile, their home laptop and their work computer. So what you may ask, as long at is always measuring the same thing? Consider this made up example, a media company decides to launch a paid for media service assuming their addressable audience was 3x bigger than it actually is - I would expect it might well fail to deliver on their business case expectations.
What about a less obvious misinterpretations - who understands the concept of dimensions in Google Analytics, sessions vs users vs hits? What does "unique page" actually mean? When should you segment by users not sessions?
The challenge to a product such as Google Analytics is how to make powerful tools accessible, while ensuring users truly understand the meaning to draw valid conclusions. I have made mistake where I thought I understood what I was interpreting to then look at a different report to prove myself wrong - I had not understood what I did in the first place.
From a product management position this is a conundrum. Products like Google Analytics have basic features which make them feel easy to use while actually delivering very little value, the real value is gained only when the user engages with complex features - that assume user understands. As product managers we want to empower users and make complex capabilities accessible, but as the saying goes 'a little knowledge is dangerous'.
So think twice next time you create a filter or segment - what is your slicing and dicing actually telling you.
What about your products? Do you assume your users all have PHDs in your field? What assumptions about users knowledge have you made? How can you learn if it is valid. How can you onboard users?