This is a rant about the state of analytics :)
Let me give you a simple example of why analytics sometimes have the tendency to make me want to pull out all of my hair ... or rather I already did, just look at my profile pic :)
Earlier today, I wanted to look at what the bounce rate was for a client of mine, for each of their pages. This should be a simple thing to do. Create a report by ‘page’, add ‘page views’ and ‘bounce rates’ … and you are done.
Except, the result looked weird.
So, I went back into the report building mode and searched for ‘bounce rate’ as a metric, and found two of them. One was called ‘Bounce Rate’, the other simply called ‘bounce rate’. Yes, that is exactly the same name.
So… I added both, and the result is what you can see in the picture below. Two identical metrics, with completely unidentical numbers. They do not even follow the same pattern.
Now, I’m sure that the analytics company behind this have a perfectly reasonable explanation (at least to them). They will probably tell me that Bounce Rate (capital) is based on a session level of analytics, whereas ‘bounce rate’ (lower case) is based on a hit based/calculated result.
And they will explain this to me as if I’m the one being the idiot, because, surely, I should know this.
It’s same with other analytics suites. For instance, if I want to know how many unique visitors (aka users) my site has on a given day, I would look at ‘users’. But if I then want to see how many users each page has, creating a report that looks at users for each page will give me completely wrong and misleading numbers.
The reason is that, on a page level, users can’t be used (because the metric is session based). Instead, if I want to learn how many users a page has, I shouldn’t be looking at users, but instead look at unique page views.
I happen to know this because I’m an analytics geek, and from a technical perspective, I can even understand why this is from the perspective of a database query.
But just stop and think about this for a moment. This is wrong in so many ways. If I want to know how many users a page has, I should be to select ‘users’ as my metric. I don’t care that it’s session-based, hit-based, or whatever-based.
A user is a user. A bounce rate is a bounce rate. A page view is a page view.
The very idea that I can go into my analytic system and create a report that outputs completely wrong data (like in the case with bounce rate above or with users on a page) is wrong.
I don’t care why this is. I don’t care how many excuses or explanations the analytics community can come up with to explain it. It’s wrong!
Pure and simple.
The analytic providers need to get their act together.
I can spot when something isn’t right with the data because I do data analysis every day. But most people can’t do that. Most people would just ask for the data that they need, and blindly trust it. So what do you think happens when people choose the wrong bounce rate metric, the one with the capital label rather than the one with the lower case label?
I repeat, this is wrong. This is not a user problem. It is unacceptable for an analytics system to give wrong answers. It’s unacceptable to ask for a metric, and be told the wrong result.
And here, I have highlighted only two examples. I can highlight 100 more, easily.
One more example. If I look at a client’s analytics, and ask it to tell me how many unique visitors/users they had last week, it might tell me something like this:
But then, at the bottom of this report, it will say that the total is: 997,732 unique visitors for that week.
No!!! ... Just no!!!
The entire purpose of a unique visitor is to know how many people you reach. You cannot add them up as a total. That’s wrong, because it doesn’t take into account that some people might come back more than one day.
The correct number of unique visitors, in this case, is 654,442 unique users for that same week. In other words, the correct number of unique visitors for that week is 34% lower than what the analytics system is reporting in the standard report.
If you were a sales manager for a company, and you started reporting your sales this way, you would be fired. But this is how our analytics systems work today.
It’s just wrong in so many ways!
And I could go on and on and on. Because there are so many similar examples in almost every single report that you see.
Analytics is hard. We all know this in the analytics community. But these problems are not what’s hard about them. This is lazy analytics based on totaling or averaging numbers from a table, rather doing real analytics.
We need to stop this. The analytics community needs a rethink. We are not talking rocket science here. We are talking about the most basic of the basics… and it’s wrong.
Bounce rates not being reported correctly. Users not being reported as actual users. Unique Visitors not being reported as unique.
Seriously. This needs to change!