Shared publicly  - 
 
"Sitecatalyst 15, for all of the excitement Adobe has tried to gin up, is a laundry list of incremental fixes to functional shortcomings that the industry has simply complained about for years."

Smack.

Read more from +Tim Wilson on his blog. What do you think? Is web analytics hard because the tools are hard to use? I have an opinion … but would love to hear from my circles.
31
4
Chris Grant's profile photoAlex Brasil's profile photoAdam Greco's profile photoMatt Lillig's profile photo
79 comments
 
I think it's complicated if you allow it to be complicated. I think Omniture has allowed it be complicated by giving there clients more control over their reporting. GA is simpler because frankly there are fewer options.Oh, and I've been using SiteCatalyst 15 for over 3 months now and think that just using that version will solve a lot of implementation issues that more advanced users encounter.
 
So you agree with Tim's "incremental fixes" comment? I'm not close enough to it to know, and I'm not sure if +Adam Greco actually uses Google+ yet ...
 
As an implementation specialist that came from web development I can say that it is clear that SiteCatalyst's back-end systems drive unnecessary front-end complexity. For instance, look at the variables passed in the HTTP request of a track page call and those of a track link call. Web analytics is event-based, but, you will see no obvious events listed to identify either. There is no "pageView" event or "userInteraction" event. SiteCatalyst looks for the presence of "pe" and "pev2" variables to determine the type of call.

Tracking tags should be so simple that you do not need a full time implementation specialist. Instead, they are complex. Thus, the rise of the new TMS industry.
 
Good example of complexity, Matt. Also, good catch that the existing complexity in data collection essentially gave rise to the Tag Managers. Not exclusively, but the status quo certainly didn't help prevent their emergence.
 
+Eric Peterson I think that any upgrade will contain incremental fixes to a degree. Being vendor agnostic, sure, there are no easy solutions out there, but I do think you need the people (i.e. us) that understand the minutia of the tools to help you take the most advantage of your investment in analytics. The correct answers to many analytics questions will not have a cookie-cutter answer that work across all verticals.industries.

At the same time, I do see where Tim W's coming from here. I specifically look at the options for simply telling a retail customer (in SiteCatalyst) how much sales was provided from Google natural search. I can think of 5 ways to answer that question, all giving different answers. But I do think that's where experts come in to help companies best understand how to customize tools for a business' requirements AND increase user adoption as well as ease of use.
 
I think they're overly complex. Like Tim mentioned in his article, there are certain issues you can be aware of and move around, such as Table Limits in Webtrends, but you probably never will even know those really exist until you're trying to compare data and you notice something is horribly wrong. By then it's completely inconvenient and might not be recoverable, and good luck trying to explain to a client/business partner what is happening on the back end.

I think the issues stem from engineers designing the system to do what it does but no one taking that information and presenting it in the interface or documentation so it makes sense. Maybe there should be an alert that you're about to hit limits. Maybe there should be something more up front in the documentation. But the bottom line is there's so many little things that significantly can impact results that most people just won't ever understand in its current setup.
 
Great post, +Tim Wilson. Yes, there is much that vendors can do to make implementation, analysis, data extraction/integration, and program/product management easier. Will I ever be satisfied with any vendor out there? I am betting no. Will I still use those same vendors to make incremental profits from digital measurement and analysis? I sure will. I still use the same old tired approach. Match the tool to the needs of the organization. Yes, it is hard. As a consultant in this industry that is wonderful for my bank account, but as a passionate analytics practitioner it needs to change, and no one is changing fast enough. I will say again what I have said before, if you are completely satisfied with your web analytics provider you are not doing web analytics right.
 
It's hard, if you're a newer user. But just like anything else, with time and repetition it gets easier. Fixing your own car is hard too....it's why we have mechanics.
 
Does there exist secondary dimensions that platform providers may compete upon? Absolutely. The primary axis focuses on degrees of freedom. If you want it dead simple, you have to limit the capacity for people to make it complicated. If you have really complex queries, then you're going to get complexity. There's a path through making the extremely complex, flat, by way of technology and good old fashioned science. That assumes a certain level of sophistication on the part of customers, because any simplification involves choice and sacrifice. And every analyst has a bone to pick with everybody over which to cut and which to keep (See: The Engagement Debate, 2007-2009).

We stand no chance, none, under the current paradigm to tackle with Geo-Social or IoT.
 
The problem with web analytics is the same problem with every other project in business that doesn't go well: the sunk cost fallacy.

A lack of planning causes any tool to produce crappy results. This leads to additional requirements from Marketing, and tech guys start implementing more garbage tags (and custom JavaScript if you're really lucky!). Then, no one understands the reports, and everyone is mad...but starting over isn't an option. It costs too much, we'll lose our data (that no one trusts!) So we blame the tools for our lack of understanding of OUR businesses. The tool becomes the problem for our lack of education, lack of training, lack of everything.

Frankly, for every ridiculous thing that I've heard needs to be "tracked" by marketers, I'm glad it's not easier to implement web tools. Like the quote from Avinash about investing in people, that's where the industry is currently lacking. We need people who understand analysis AND implementation, people who can translate gibberish from marketers into gibberish from implementers to understand, and be true leaders of business.

Omniture isn't too complicated, GA isn't too simplistic...we make them that way, because assigning blame to a tool that can't defend itself is easier than directing the blame to ourselves for lack of business focus.
 
I commented on Tim's post but I'll comment over here too. First, I'm really not a fan of the blame game. The user has the power to make an omniture implementation fail, and the power to make it succeed- same is true for any of the established vendors. You'll get out of omniture what you put into it- if you really do put an investment into it (as is required because of the complexity of it) then you can get a lot of flexibility and power out of it (which is allowed because of the complexity of it)
People aren't looking for a tool that's easy to use, they're looking for something idiot-proof that can do the job for them. Any tool is an investment- you have to put something into it to get something out of it.
 
Heh-heh. I guess I could now complain about how I'm not going to capture this whole thread for archival purposes in the blog comments. Damn you, Google+! You sucky suck, too! (Kidding!).

I really wasn't looking to wholly throw Sitecatalyst under the bus, accuse them of not listening to customer feedback, or propose that there is Magical Simple Fix that I'm just not sharing.

I have little patience for people who go through their workday pointing fingers rather than trying to be part of the solution (AND taking responsibility -- even more than they deserve -- when things go wrong). This post, admittedly, dipped into that territory. But, as I started out by saying, I was blaming myself a bit for some of the unkind thoughts and words I've had for client implementations.

The point that I'm not making well (and probably won't make well here, either), is that, with Sitecatalyst in particular (hmmm...Webtrends, too, now that I think about it), by the time we've gotten over the hump of how the pieces fit together, we're so happy to be there that we simply accept that the size of that hump was As It Needed To Be. There's a Clayton Christensen / Innovator's Dilemma reference floating in here somewhere -- the more we provide feedback based on our understanding of What Can't Be Different (and, as humans, once we understand how something IS we have a hard time envisioning how it could be otherwise -- see John Medina, Malcolm Gladwell, Daniel Gilbert), the more we're driving incremental improvement rather than fundamental change.

But I'm not going to budge on my claim that SC 15 segments are almost laughable.
 
@tim You can ask Eric do data extract, google+ function, never try it though.
 
To Tim Wilson's post I'm inclined to agree. I hear far, far more stories about botched implementations than I do about success stories for the same. I'm sure those success stories are out there, but those don't get much attention. I've even heard SC experts slag off on implementations by firms that have some pretty robust brain power supporting SC.

My poor analogy is that SC may be the $200k sports car that few people can handle. Those that can deserve to get paid for their expertise. Those that can't should stick with the sedan and focus on analyzing what they can (see, the analogy fell apart).

To Eric's question about is WA hard because of the tools, I'd still maintain that most difficulties are still around focusing on analysis not simply data and getting folks to take action on data amidst all the other distractions in the digital space.
 
Whoa, I did some actual work for a few hours and look what happened. Nice! Except now I have to actually read all of this and respond since some really bright folks commented.

I know there is a beer in this hotel somewhere ...
 
While +Tim Wilson 's post was largely SC15 focused, I think the true point of his article focuses on the Analytics industry as a whole and how it is trying to evolve. While newer and improved tools are being released every day, the industry as a whole suffers from fundamental issues (i.e. implementation difficulties, data interpretation, etc.) that cannot be addressed with improved tools. There will always be the inertia of current industry culture, and it will take someone with guts to change that. It's a big bet, and few are willing to go "all in" on a potentially disruptive approach to #measure without a certain knowledge that it will be a huge success--especially in this economy!
 
+Bryan Cristina and +Alan K'necht your responses both remind me of a brilliant (ahem) post I wrote back in February of 2010 about "The Coming Bifurcation in Web Analytics":

http://blog.webanalyticsdemystified.com/weblog/2010/02/the-coming-bifurcation-in-web-analytics-tools.html

My thesis (in a nutshell) was that "core" solutions have become far too complex for lay-users within the Enterprise and that a change was coming. In essence I agree with Tim here --- SiteCatalyst has been forced to become incredibly powerful and thusly complex. This opened the door to Google and their "simplicity" (although they too have moved away from this simplicity in favor of an increasingly complex UI, IMHO, which is a damn shame …)

The rub is that what folks like +Tim Wilson and honestly most of you in this thread need is Adobe Insights or the IBM "Explore" product … a powerful ETL solution that is able to take web data (and more) in to meet your specific business/customer/client needs. The problem? Insights is still very expensive … and so you're stuck trying to use SiteCatalyst to meet your evolving and increasingly complex business/customer/client needs.

Am I wrong here? Tell me if I'm wrong …

I feel bad for the folks that thought my comments were a criticism of Adobe SiteCatalyst --- they were not. I'll let +Tim Wilson defend his comments himself --- and apparently he's gonna make fun of the new segmentation functionality until someone like +Ben Gaines gets all up in his grill (which, by the way, I will pay to see). But at the end of the day Tim captured a key challenge we face in web analytics today --- the gap between functionality and business value. If the only way to succeed in digital measurement, analysis, and optimization is to be +Adam Greco, +Ben Gaines, or +Tim Wilson we have all already failed.

(More responses soon … where is my waiter with that next beer!)
 
+Matt Lillig I disagree, and I think this entire thread bears that out. Web Analytics is hard no matter how experienced you are, and perhaps it gets more difficult with time. No disrespect to mechanics, but I would love to have a client who had as few moving parts as a car … or even 1/10th of the moving parts of a car …

I, and I think a lot of you, end up working for or with real businesses who have a seemingly uncountable number of stakeholders, complexities, and internal obstacles regardless of their origin. When you superimpose these challenges onto the overarching clusterf*@k that web analytics becomes (paraphrasing +Tim Wilson here) you end up with a situation that is generously referred to as "hard."

When someone tells me that "web analytics is easy" I immediately know that they are A) a vendor with a vested interest in a short sales cycle, B) a consultant with a vested interest in disagreeing with me personally, or C) an idiot. Since you're "A" I challenge you to go back to your client base and ask them sincerely whether the entire, overarching process of "web analytics" is getting easier over time.

Let me know what you find out.
 
"Omniture isn't too complicated, GA isn't too simplistic...we make them that way, because assigning blame to a tool that can't defend itself is easier than directing the blame to ourselves for lack of business focus."

+Randy Zwitch any chance you live in Dallas? I ask, because I want to give you a hug. This comment is brilliant --- even if you got the wrong attribution for the original call for staffing in web analytics (see http://www.forrester.com/rb/Research/web_analytics/q/id/51262/t/2 ;-)

Blaming tools --- and this is a consistent thread here --- is far easier than addressing the real, fundamental challenges we all face doing web analytics professionally. And I'm not saying that we're a bunch of whiners --- far from it, the folks on this thread are some of the hardest working, most intelligent, passionate analysts I know --- but at the end of the day a little introspection about our organizations and/or clients would go a long, long way.

How about it, people: is there any chance you're blaming technology for something that might better be blamed on people or process?
 
Eric I agree with much in these posts and with your point that many of us have needs that have graduated from what a tool like SiteCatalyst is for. The trick is to use SiteCatalyst for what is does very well and find the right tools to meet the needs you have that are probably not what SiteCatalyst should try to do.
 
+Eric Peterson Trust me, I understand first hand the complexities and internal obstacles of trying to manage web analytics for a real business (SonyStyle for a short time). And yes, I agree it was hard trying to manage the mess that it was right after they switched from WebSideStory to Omniture (that's why I got the he'll out of there and went back to the vendor side!....j/k). I just don't agree that "hard" is the correct term. Always "challenging", absolutely!

By the way, it ain't always a party on the vendor side either, as you well know.
 
Not in Dallas +Eric Peterson, but if you're at the industry conference in San Diego in Sept, I'll trade that hug for a beer ;)

My comment is speaking from a beginner in web analytics, starting on a botched Omniture implementation. The smartest thing the company did was send me to training in combination with getting a consultant to make recommendations. So I got a great, if not extremely difficult, on-the-job training about thinking about the business questions first, then figuring out what data I need. As opposed to starting with data, and wondering what it was "telling" me...

Ultimately I moved on to an agency, but that lesson is helping me to help my client move their focus beyond what "Omniture can do" to "what our business wants to do".


 
+Lee Isensee I'm not so sure we can blame leadership and HR, at least not in one fell swoop like that. Having worked for vendors at various points in my career I can assure you with absolute certainty that our/your/their sales organizations used that lack of understanding to our distinct advantage. Don't believe me? Go read this 2008 post from an Executive at Omniture:

http://blogs.omniture.com/2008/04/21/analytics-goes-global-and-guess-what-it-is-easy/

To be fair the post is dated and the author has changed roles at Adobe a few times … but that post highlights the mindset we have to overcome, still, three years later. When folks like +Jason Thompson and +Tim Patten (strategic partners of mine, whose experience I respect) absolve the vendors of responsibility saying "it's easy to point fingers, but creating change is more difficult" it gives me pause; I mean, it's not as if the vendors don't have their fair share of smart folks! Even with +Ben Gaines walking off into the sunset, leaving Adobe for his dream job at ESPN, Orme still has some incredibly bright, hard working, honest people … but still smart guys like +Tim Wilson are inspired to blog.

Why is that?

At the end of the day it's not about data accuracy, is it? It's about creating business value. And on this point, in many regards, we continue collectively to fail.
 
Every problem looks simple... once you know the answer.

However when you don't, trying to find it can be pretty complicated. If you work on a discipline that requires "analysis" to distil some value, then don't expect it to be that simple. If it was so simple, may be we should be called "collectors" or "pickers" and not analysts. As many before me added to this discussion, complexity is expected and you have to deal with it.

We use computers, software, calculators..., nails and hammers, any technology, because they are the best fitted tool do a particular task that is available. And yet tons of projects involving these technologies fail every day. Because tools and technology are perhaps the easiest part, managing people and politics being the most complex ones.

I like to think I am good working and finding my way with the tool I have at hand, even with the one that are not so usable ( I have to admit it, I have a harder time with nails and hammers). In any case I kind of feel having this attitude helps you deal with this complexity. Better tools will help... of course. I would love to have better tools, and as many I have been frustrated when things don't work as expected, or when I found that the great feature I need to answer this X or Y question "...is there, but charged extra" (did I mention that my favourite analogy for SiteCatalyst is one of those Bell helicopters from the 60's so powerful but with so many levers, lights and gauges that only a few can fly them without crashing? Ammo not included. Yes, tools and data could be better). But as you well mentioned Eric, tools are the least of our concerns, and there is not much that can be done without the process and the business behind, plus a lot of energy and effort put behind. Did I forgot a dose of patience?

I any case there is so much that has been said on this page that I much rather stop before paraphrasing...too much.

One last thought, though. We do analytics because we see the value behind (I least I do, and let's face, is really interesting, even exciting!), and it is valuable because it is not so easy to get. Of course, the difficulty of making something look simple so it can be used by many is also part of the mix. Nevertheless, let's face it, will you keep doing what you do if it was dead simple?
 
So I am still new to Google+ so hopefully this works! A few thoughts from my vantage point:

1. I am glad our podcast helps create healthy debates!
2. I agree that v15 is a collection of feature requests (many of them mine!) that people had wanted for a while, but as was mentioned, also transitioned to a new platform which may help in the future. While I think addressing what clients want via the Idea Exchange is great, I can see how some might want the next paradigm to emerge vs. adding a bunch of enhancements. However, if Omniture had some radical new release with a new paradigm, many people could revolt! Not many companies are Apple-like and can define new products and paradigms and have people shell out money more for them!
3. I have always agreed that web analytics is hard and that you get out of it what you put into it. Specifically, SiteCatalyst is not the easiest tool in the world, but it can do some amazing things with it if you know what you are doing. Unfortunately, there is a steep learning curve and even after hundreds of blog posts, I still get questions from confused customers...
4. Lastly, I don't think people should blame the tool. In the past, I had clients who went to Omniture after failing with Coremetrics and failing with Hitbox. Then they told me how they wanted me to set-up their Omniture stuff. When I challenged them, they said, "that's how we have always done it!" to which I responded "...and how has that worked out for you?" You need to be vigilant in your set-ups of these tools and laser-focused on what business questions you need to answer. Too many people look at an implementation as checking off a box...That's not how it works...
 
Since nothing in life is free.....

Say Adobe promised a revolutionary step for SiteCatalyst (w/ defined specifics), but to hire the necessary resources they need to triple billing for 18 months prior to release.

Following launch, bills would be 50% higher for the following 18 months.

Would anyone take the offer?
 
I don't think the problem here is seeing an implementation as "checking off a box." So many of you have preached that that is the wrong way of doing things that it would be foolish to think in that manner. Not to say that it doesn't happen, but I would be surprised to see a competent person dedicated to Analytics or the implementation thereof act in such a manner.

The core issue was nailed by Eric earlier in this thread. We get tunnel vision when developing new implementation. Our organization requests new functionality from the implementation. We deliver, IT messes it up, we fix it, and by the time its out there, we're being asked to serve up something else. With this constant stream of requirements, we forget to take a step back, look at what is really needed, develop a LONG-TERM plan for implementation evolution, acquire executive sponsorship, and fight the good fight by pushing back against the ad hoc implementation requests. I'll say it right now: There is no solution that will fit your company's needs that does not require an ever changing, customized implementation and there never will be.

But that doesn't make web analytics hard. It's not hard. It's not easy, but it requires that we fight against our human nature, take a look at the 30,000 ft. view and ensure that we're still on the right path.
 
+joe orlet If I know the new feature will generate me a high ROI, then should cost matter?
 
+Joe Orlet I'm not sure what you're asking. If the question is "would you pay more for a theoretical improvement from Adobe (or any vendor)" then I am gonna guess most folks would say "no."

CPM and event-based analytics costs have been in decline for years … why would that change? And forgive me if I sound old and cynical here --- but what "revolutionary" changes have we truly seen in this sector since the introduction of Visual Sciences back in 2002? I'm gonna get slapped around for that comment, but objectively, what have we seen in the last ten years that was truly a revolutionary or better, evolutionary, step forward for the industry?

If you force my I would answer "tag management" but that's a cop out since it's a big old band-aid on process breakdown, negligence, and otherwise lack of governance. Honestly I don't feel smart giving that answer … it's too obvious.

What do you all think? As a counterpoint to +Tim Wilson's original post, who is willing to detail the next big jump forward technologically? One that would justify increased pricing, hype and hyperbole, or, better yet, widespread vendor switching such as we saw during Omniture's original heyday?

Keep in mind free is not a technological advancement … for you all you GA-wonks out there ;-)
 
Too bad all to often the people cutting the checks just see this big cost that can't get them EXACTLY (no 90% accepted) what they want.
 
Eric, since SC15 didn't seem to be enough progress, are people willing to pay a significant amount more for faster feature releases?
 
I have been following this lively discussion all day. I was tempted to chime in several times and now I am giving in to temptation. I arrived at Web Analytics via the development/custom reporting world, so I very often get frustrated by products that create queries and present data for me. The well intended user interface - in my opinion - is what makes analytic tools cumbersome, bulky, bloated and overly complicated. I think we would be better served by having access to raw data; having expert proficiency in data extraction; and having an in-depth understanding of web analytics.
 
Are feature releases really what we want here? Can you think of anything that you would want to do but can't assuming you have a perfect implementation? (Also assuming the feature you suggest is not offered in another package)
 
+Joe Orlet ah, okay, that is more clear. I'm guessing the answer is "no" but it's a good question … of course, Google keeps giving us more features for the same low, low price of … yeah, you know.
 
+John Pettinger +Wesley Hall you guys arguing in Google+ is comical. You work, what, four feet from each other? I'm gonna have Mike send me pictures of you two cat fighting in the halls tomorrow LOL!
 
+John Pettinger That is exactly what I am saying! You know me too well!! +Eric Peterson ha! We are not arguing. JP knew what my point was - and how to say it in less words than me.
 
Eric, I disagree that Google Analytics is free.

Google collects, owns, and uses the data from sites where it is deployed. I think that is worth exponentially more than the price of SiteCatalyst.
 
+Wesley Hall I agree that for many of us, SQL will solve many many problems, but SiteCatalyst has its place. That is: the screens of our execs when they say "I wonder if..."

And when SC doesn't answer the question, it should be our job to update the implementation so that next time, it will. I have a quote on my bed right next to me that I try to remember whenever we look into implementation:

"My job is to answer the questions that management will ask next month."
-Hal Varian
Chief Economist
Google
 
LOL, yes SQL is the answer to web analytics being to complicated.
 
Lee, better point. Google isn't an option for me. I'd exceed its traffic limits in less than a day.
 
+John Pettinger SC, GA, etc all have their places. No doubt. For monitoring day to day, I agree those tools are great. But for real data dives, and for applying specific business rules, nothing beats handling your own data. Even to your quote (which is a good one) being proactive comes from knowing your business and knowing how your data reflects your business. My mantra is be tool agnostic and be knowledge focused. Now go to bed.
 
+Joe Orlet +Wesley Hall Now that I think about it, you're right Joe. Even if we got the raw data, it still requires implementation. We still have to tell it what to track.

Go to bed? Please. I don't have to get kids to school in the mornings. ;-)
 
Plus, you have to have someone who knows and maintains the SQL database.
 
BUT BUT BUT..... you could just spend that on implementation resources....
 
So two things, as this conversation continues. Google+ needs threaded conversations, and I like this style of interaction.
 
Hmmm. Wow. I'm thinking this round of debate-discussion is winding down. I knew my thoughts might touch a nerve or two, but I hadn't really thought through what would happen if, say, +Eric Peterson dropped it on Google+. Because of the temporal nature of the medium, I am concerned that the discussion will be lost, so I'll try to figure out a way to capture the themes in a subsequent post (am I clinging too long to this old school "blog" medium? Aren't the comments on Eric's post from last year an important part of the post itself?).

I'm also pushing out a blog post that was slated for today to next week (more about people/roles than technology), as it might touch a few nerves, too, and I really don't want to find myself labeled The Cranky Bastard of Web Analytics.

I appreciate the discussion, as it's pushed me to think more deeply about the ramifications of the original claim along several dimension. And, I appreciate a real side benefit of how this has played out -- it helped me fill some holes in my Web Analytics circle!
 
I'm a little late to the dance, but here is my $.02: Web analytics is hard if you are doing it right. There is always another, more complicated and harder hill to conquer in this world. If all you do is spit out the same reports over and over with no insight, no deep dives, no new perspectives, then it can be easy, but you aren't doing it right.

The tools themselves don't necessarily make the job easier or harder. They are designed for specific functions, and do okay at those functions, but no UI for analytics is a deep dive tool. Even the segmentation tools are not deep dive tools. Doing a proper deep dive requires business knowledge and not just an understanding of your website.

Web analytics is hard, because business is hard. Things change (internally and externally), and you have to understand how those changes affect your business, and how they are reflected in your reporting. Knowing your business and its needs are where it all starts from - the well-planned implementation, the regular (automated) reporting based on that implementation, the deep dives, the ad hoc work. All of it comes from knowing what helps to drive your specific business. Once you have mastered that, you can get some insights from the UIs. Until you have mastered that, direct access to the raw data and all the SQL skill in the world will not really help.
 
+Michael Helbling agree, this is a much better format for this type of thing than A) the Yahoo! group ever really managed to be and B) Twitter is. +Tim Wilson props to you for kicking this off! I'm sure the thread isn't done (yet) and hey, let's meet here once a week and do this again.

Or, are you thinking what I'm thinking? I bet you are ...
 
I screencapped Eric's statements on the overlap of social complexity with technical complexity.

Marketing reporting, as practiced today, falls far short of the marketing analytics that most industry professionals want to deliver. That gap causes significant frustration. This thread is a good exhibit.

The only way out will ultimately be because a few of us decides to change the game completely, and are willing to break the market off its bad habit.
 
OK, so rather than going on for days here, I decided to sum this up in a post: "How to avoid web analytics douchiness."

http://www.atlantaanalytics.com/practicing-web-analytics/how-to-avoid-web-analytics-douchiness/

In essence, we are stuck on the tools, we are stuck defending something that shouldn't be defended, and we are stuck trying to look like the smartest people in the world and saying that people that disagree with us "don't get it." That is a sad state. Agree/disagree?

Ted's post is brilliant, correct, and timely. And most importantly, it's actually constructive.

I think we are "stuck" with implementation where it is today. But I do think tag management solutions will get us out of this mess, when designed not to make the task easier for the same people, but completely change who is doing the task. Better, more readily available tools and supplies allow the artists to focus on the art, rather than the tools. IT is a tool-making group, not an art-making group. The solution is to remove them from the process entirely, which can only happen if this unnecessary complexity is addressed. I believe it can be.

Yes, the focus can't stay on the tools, but it does need to go to the tools for long enough to fix what sucks.
 
+Eric Peterson Funny you should mention VS-- just yesterday I was talking to a fairly large client that still uses it. I agree that there has been no real innovation in the field for quite some time (heck, VS is still usable , which is pretty sad given how much it has rotted since the acquisition).

As for the "next big jump". This is a semi cop-out because it's sort of here (though I argue not nearly as successfully as anyone claims), but the whole disparate data sets merged into a warehouse of analyticy goodness is the golden goose, and the first vendor to make it fairly simple and cost effective (I do not think either of these conditions are close to being met), will garner a lot of business.

I also think you'll start to see certain organizational strengths manifest itself in the analytics field. e.g, I think Multi-Channel funnels is a really great step in the right direction and will push proper attribution modeling into the spotlight/show some tremendous value.

I wouldn't be surprised if IBM leveraged their considerable advantages to tackle that data warehouse golden goose, and of course, Omniture has the expertise and technologies to make a great push at that as well (arguably the closest to this goal with Insights, but the entire stack is both extremely expensive and non-trivial to do right).

This is somewhat odd as we're discussing how broken WA is, whilst arguing that the future will see WA as a small part of the overall picture. I do think this is true, but I also agree that WA is brutally difficult which, to me, indicates just how insanely hard it will be to get it right, across multiple data sources.
 
You all should go read +Evan LaPointe's quirky yet spot-on response to a lot of what is in this thread. Tip: the real meat (as with many of Evan's posts) is in the past three or four paragraphs.

http://www.atlantaanalytics.com/practicing-web-analytics/how-to-avoid-web-analytics-douchiness/

Some of my favorite "Evan" in this post includes:

"… let’s be nicer to each other and stop playing secret service for Omniture. Stop jumping in front of bullets for a tool that doesn’t thank you and wouldn’t do the same for you (no, these improvements are not based on user feedback, they are based on the imminent reality that the tool will become irrelevant without a feature roadmap: that’s how products work). The tool is a decade behind the times in terms of back-end architecture, and that is a fact. 15 is designed to get them to zero so they can continue to build an amazing suite of front-end utilities for smart people like you to use (and keep behind the face). Stop defending their HORRIBLE implementation. It is not a Ferrari. It is a Fortran mainframe that takes 200 square feet and 20 IBM engineers to do what an iPhone can do. They will catch up because they are brilliant, but today is a joke.

Let’s move past all of this inconsequential tool obsession, let’s extinguish the burning desire to look smarter than the next person, let’s be the classy and incredibly valuable resource the world needs us to be, and let’s reap the rewards. We are one community stuck in a rut, and we are digging. We are better and smarter than this."
 
+Alex Brasil Hear hear! Both of us have been crossing the chasm for the better part of three years on this file.

There is no entry-level SMB big data whole product.

There is no whole product that replaces the good old fashioned NOSQL database under your desk, the copy of SPSS (powered by IBM) on the Mac, Google Refine, also sitting on your mac, and then the development cycle we have to go through to express an algorithm and make it usable. The process of sustainable marketing science on such a stack is painful to say the least, and extremely dangerous to say more.

Fundamentally, tagging is so easy to break by the well-intentioned, and impossible to do properly through the ignorant. Analysts don't have complete sovereignty over their instrumentation. Big problem and a big disruption opportunity.

The Datameer's doesn't want to come down the SMB pricing and I've watched too many startups choke and die crossing the data chasm on SAS. It's non-trivial.

That said, though I wish it wasn't this bad, somebody is going to have to take a risk and radically democratize statistical analysis on big data.
 
While I didn't really want to throw my dog into this fight, I'm compelled to chime in here. For starters, I think this discussion has devolved from Tim's original premise in that it's time for a paradigm shift. He's absolutely right.

My thoughts on this also dovetail with Evan’s recent post on the complexity of what we do. That complexity is awfully close to cementing our demise. Nearly everyone on this thread has been asked to -- or had the foresight to -- “dumb it down” for a business audience to understand and appreciate what we do. The complexity allows us to have the wherewithal to deliver brilliant insights and recommendations, but throwing that in the faces of peers and colleagues doesn’t win favor. Our tools are complicated. Our processes are convoluted. And guess what, we’re a community of thousands instead of tens of thousands as a result. But just because we know how the “machine” works doesn’t mean that others cannot enjoy the ride. The rut that we need to dig out from and the paradigm we need to shift is driving business value.

Back in 2008 when I wrote that the free IndexTools solution had 77% of the functionality that Omniture did, I took flamethrower heat for saying that. It was blasphemy. But it was true. It was also true that both were incredibly complex and only a handful of craftsmen were capable of handling these machines. Not much has changed in three years.

The way I see it, we’re not going to turn our business counterparts into analytics mechanics, so it’s us that need to change. Us being the vendors and the operators of digital analytics. I’ll submit that there’s lessons to be learned from burgeoning Social Analytics vendors. I’ve been frustrated and even criticized the lack of customization and the inability to created calculated metrics, but the limited functionality and straightforward mode of delivering analytics is something that we can benefit from. A reset if you will. An opportunity to use our brains to solve business problems and not rely on the tools as a technical crutch. While these vendors are certainly not perfect, perhaps simplified tools can help us to focus less on the complexity of operations and more on the value that we’re driving for our clients and employers.
 
+Ben Gaines and I are having a very good conversation on twitter, much better had here. Sorry!
 
Hey folks … +Adam Greco, +John Lovett and I would invite all of you --- or rather the first 10 (or 25, we're not sure which) of you --- to discuss this topic in greater depth next week on Monday, August 15th at 11 AM Pacific / 2 PM Eastern here in a Google+ Hangout. We will moderate this topic based on all that has been shared thus far (and all that gets shared between now and Monday) as best we can, with the caveat being we are new to Google+ Hangouts.

Who? +Adam Greco, +John Lovett, +Eric Peterson and a bunch of our Google+ pals!
What? Web Analytics Demystified Google+ Hangout
Where? Here, of course!
When? Monday, August 15th at 11 AM Pacific / 2 PM Eastern
How? As best we can, since we are new to Hangouts

We hope to see your comments between now and then, and hope to see some of you next week!
 
Awesome! My latest "step away from the desk to do real work and return to find another dozen comments on this thread" included a discussion of hangouts. I'll be traveling back from Cooperstown to Ohio, so my attendance will be dicey.
 
Here is my response to Tim Wilson's blog post. I hope you find it helpful:

Thank you Tim for the insightful post (and the mention).

Here is my take, from a client, turned vendor (ForeSeeResults), then back to a client (ProQuest) perspective:

1. On the vendor side, I believe there is the following mindset: "Measure more variables because we just figured out how to do so, then make cool visualizations, dashboards, 3D Fly throughs, spinning piecharts :) etc." This works great in helping vendor sales teams sell the sizzle, but the steak is still the same.

2. Vendors naturally feel that they are the center of the universe. Is it better to integrate ForeSee Results data (disclosure: my previous company) into the Omniture environment? Or is it better to integrate Omniture data into the ForeSee Results environment? Depends on which vendor you talk to first. Regardless, you are not going to get a "seamlessly integrated holy grail"... and the business reality is that you need more than one vendor if you truly want to have a credible measurement ecosystem.

3. On the client side, we do not think enough about the business questions our team needs to answer. We gravitate towards what vendors give us and try to amplify the value of those variables, even if they are not really that actionable. Why? Because it is hard to admit that we do not have the answer to a particular business question in the data and of course we are constrained by budget, time, bandwidth etc.

4. I like your "Jobs-like Visionary" concept. Apple made itself the hub. It simplified its offerings. It focused. It changed its business model completely. 3 of the top 4 Clickstream Analytics tools (Adobe/Omniture, IBM/Coremetrics, and Google/GA) are managed by parent organization for which #measure is not their main business. (Webtrends is still independent).

Is it realistic to think that companies structured in this way can pull a "Steve Jobs" and revolutionize this industry?
 
+Eric Peterson Eric, regarding the next big thing--my potentially baseless guess is that tag management is an intermediate stop toward a "pageless" tagging approach. As "page views" becomes an increasingly muddy metric, the "page" as the atomic level of measurement will become equally challenged. TMS' enable object manipulation through a metalayer. I'm going to put a stake in the (quick)sand and say that the next generation of analytics solutions extend that model and operated solely at an object layer, rather than page (or custom click) level.

The benefit of this approach would be the flexibility -- a page, tab, button, anything with which the Customer can interact, on any platform -- would be visible to to the analytics solution. The downside would be the increased need to craft an efficient and effective measurement strategy. Just because you can tag everything, doesn't mean you should. Tirtiery server calls, anyone? Concievably, you could create a "basic" tagging that treated pages as the atomic object. Or you could go whole hog and measure every little Ajax, HTML5, Flash geegaw you can find. But a granular tagging method would blend very nicely with a big data solution.

So that's my prediction. Then again, I was one of two people in Austin who actually bought an Apple Newton way back when.
 
This is a great conversation.

I'm just going to give my answer to Eric's original question.

My schooling was survey research, including time-series survey research. With that as background for my answer, I confidently say: Web analytics tools of today aren't even on the same planet as the tools I used to use to organize and analyze huge complicated multi-point surveys. The web analytics tools are much more primitive in scope and much harder to use than survey research tools available to students. In the ensuing years, I have never been able to do an analysis with web analytics tools that is on a par with analyses I was routinely doing in my first year of graduate school. (SPSS, SAS, MIDAS, BMED, even Systat and Clementine as examples)

Web analytics (i.e. using the available tools) is MUCH harder than it needs to be.
 
I'll double up on +Nancy Koons's first entry into the web analytics blogosphere (welcome!) an point you all to her post:

http://nanalytics.wordpress.com/2011/08/10/5/

"If the people who are supposed to have the most knowledge about a tool are struggling to advise me how to implement their tool, what am I supposed to think? And what am I supposed to do? Is all the complexity of the tool worthwhile if no one can tell me how to leverage it? Today, it makes me think twice, and I am loathe to make major changes."

This harkens back to +Tim Wilson's original thesis --- how is it possible that we have ended up in a situation where so much knowledge, experience, patience, and determination is required to do our jobs?

Great post, Nancy.
 
+Evan LaPointe I'm glad you considered that a good discussion. I did, too, even though I got it started off on the wrong foot. I appreciate your grace in not holding it against me!

So I'm giving in to +Eric Peterson and commenting, even though I'm the new analyst on the block and feel like I should probably just sit quietly in the corner and observe the professionals for a few months.

I'm not sure whether this even relates to the discussion anymore, but I can't help but throw out this point, which I almost turned into a blog post a few months ago: If you want to make your analytics implementation simple, just copy-and-paste the code on to your pages. You can definitely do this with Google Analytics, and I'm not sure whether people realize that you can very easily do this with Omniture code, too. You'll get basically the same data out of both tools that way. You can be implemented in about five minutes, with no complaints from your developers.

That's patently stupid, right? The kind of data you'll get would make any analyst cringe because it's not actionable and it's not relevant to the business. (Not to mention that you're paying thousands of dollars for Omniture, so you want more value out of it than that.) This is where complexity comes in. Regardless of your solution of choice, to get good, actionable data you are going to need to invest in customization. Customization adds complexity, and complexity leads to frustration.

+Tim Wilson said that analytics platforms should be able to lighten the customization load somewhat, and I agree with him completely, which is why (much to the disappointment of +Eric Peterson, who wanted to see us go at it!) his post didn't bother me, as a former vendor, in the least. (In fact, I really liked it!) If nobody can figure it out, that's a problem. So how do we get there? Well, that's another story.

This is not the kind of radical innovation that +Tim Wilson is hoping for, but at least there are some ideas in the Idea Exchange that would certainly help. For example, the very first idea ever (I know this because I submitted it myself) deals with automated SAINT classifications. You can do a lot of powerful things with very basic data and SAINT. And the recently-built Processing Rules concept goes a long way as well, as do Context Data variables. I'm sure Omniture will continue to expand the power of these features as a way to relieve the pressure on the relationship between Marketing/Analytics and IT. That relationship will always be necessary, but it can be made easier.

I think the important thing to keep in mind is that Adobe always positioned SiteCatalyst 15 as the stepping stone to bigger and better things. Some of the changes to the platform were so dramatic that it was neither wise nor possible to make every necessary change at the same time without disrupting the users of the product. Here's a case in point, and one that Tim referenced: pulling Visits in various reports. SiteCatalyst 15 takes a major step toward improving the consistency of data, but that means that a new report (in v15) might not match an old report run on the previous platform. The feedback that we got on this was that it was significantly painful for many users to have that change occur, but that they acknowledged the improvement. So, if you're Adobe, do you make the change and alienate some users, or do you hold off and alienate other users? Long-term, I think you will see some of the significant platform improvements that +Tim Wilson is hoping for. . . at least, I certainly hope so. Just because they didn't all happen with the first launch on the new platform doesn't mean they won't happen.

Change is always necessary. I won't claim that Omniture is perfect, or even close to perfect. The folks there are not complacent. I think that responsibility for improvement lies just about everywhere: certainly with the vendor, and certainly with clients, too. I can't see how a vendor solution can be both flexible enough to meet the needs of thousands of different kinds of businesses and simple enough to work without some degree of implementation pain, but I'm also not Steve Jobs (to use Tim's analogy). Based on everything I've heard, those that claim to do so may not require a tremendous amount of tagging effort, but make up for it in other areas which we can also loosely term implementation.

The charge that vendors use lack of understanding to their advantage is, of course, true, as it is in virtually any industry. (If we all understood an industry perfectly, salespeople would be useless. We'd already know which product/solution/brand is best for our needs based on the strengths/weaknesses of each.) That said, I don't think it is true to the point that a vendor would go out of its way to keep implementation hard. Certainly that was not the case when I was at Omniture. On the contrary, I was part of many, many conversations with the consulting team about the struggles that clients faced in deploying SiteCatalyst, and about how the product could make implementation easier.

One final note: The reason nobody can agree on whether web analytics is easy or hard is that everyone has a different definition of web analytics. For some, it's doing analysis. For others, it's doing analysis and selling that analysis to executives and others who don't care and/or don't want to listen or believe what they are being told. For still others, it's implementation plus analysis plus selling their analysis. For a fourth group, it's merging web data into their EDW. I could go on. These tasks have varying levels of difficulty based on about 400 factors including (but not limited to): company culture, tenure at company, executive sponsorship, technical aptitude, marketing aptitude, storytelling capabilities, budget, and susceptibility to heartburn or fits of delusion. Seriously, are we really even debating this? It's disturbingly similar to the iPhone versus Android argument. Which is right? Gosh, it really depends.

Gaines out.
 
I really like what +Chris Grant mentioned in his comparisons to survey research tools, and I agree that our tools aren't at the same level with other analysis solutions in BI. The reason? I think it's because the creating motivation for web analytics came more from the marketing industry rather than OPS or BI. Initial tool requirements suffered from industry inexperience and simplistic expectations. As the industry has evolved, so has the requirements for analytics. It seems like the industry is in an adolescent stage between simple metrics and a true intelligence tool, and we all want to be grown-ups now rather than suffer from growing pains.
 
+John Lovett, if you were any cuter, I'd kiss you. You said it all.
 
I have to admit, it's been awesome watching all of the comments roll in on this topic. There are so many different perspectives.

In one corner you have team that is looking 10 steps in front, challenging the vendors to look 5 or 10 years down the road to what the industry is going to need (or really what we need now).

In the other corner you have the people looking at the step that is happening now and the one that will come directly after that, the ones trying to make the tool a little bit better a little bit at a time, the ones trying to keep all of their very different clients happy.

Is there really a continuous path from where the people trying to look 10 steps ahead are looking to go and where the people looking at the next step are headed?

Can you really expect Omniture to implement a major paradigm shift without alienating a ton of customers? People love their comparison data. Look at how difficult it is for some companies to shift to SiteCatalyst 15. If a relatively small change compared to what you are suggesting causes that much pain, a huge paradigm shift isn’t going to go over too well with many of their clients. What is that? You say they could make a totally new product to try and ease the pain?

A) Can Omniture really justify the cost of building a product with the intention of phasing out a product that is making them a ton of revenue currently?
B) Many of their customers may not want to make the jump. They already have a ton of money invested in implementation, training,and past data. The added value would have to be pretty amazing to sink costs in the implementation of a new tool and new training. Honestly, the new tool will have to have some awesome inertia-fighting skills to be successful.

Could a brilliant, game-changing product be built? Sure. If you were an Omniture exec would you approve the start of that project? Probably not.

With all of that being said, I’m not sure there is a continuous path between the two teams’ paths. So here we sit, passionately frustrated and arguing with each other.

If a real paradigm change really is on the horizon (which I hope it is), I would bet it comes from somewhere other than Omniture.
 
+Alyson Murphy You bring a very interesting point. Almost 20 years ago, the first vendors opened a log file, and realized they could comment on the data. Since then, we have been boxed in the session paradigm. Not that we couldn't say anything valuable, mind you. However, we now all feel we need to be able to comment more on the business value, on prospects, and how to turn them into customers, and we are growing increasingly frustrated with the anonymous data, and the limitations of the session. We know the real valuable stuff often happens over multiple contact points, and we mostly have to rely on the darn deleted cookie to do that!

But you're right: what would need to be done would require rewriting the code, the genetics of Web Analytics, and no vendor is ready to take that risk without fearing for the farm, especially when the most dangerous competitor uses dumping (yes, giving a product that costs you money is dumping, at least in terms of international trade) to dominate the market. Or maybe that paradigm shift will actually come from the (benevolent?) Giant, since the analytics product team doesn't have to be a P&L, I don't know.

I'm with you; I'm not entirely optimistic that those changes will come from companies who need to meet short-term targets. Eric is right to point out that no fundamental revolution has happened in the last 10 years in our field. Maybe there is room for a new product, a new company, a new way. And I thought that market had been saturated ages ago...
 
Wow, some great comments on the Web Analytics landscape. Before I start with my opinions I have to warn you that I may be a bit of a Tealeaf Fanboy. I work for Tealeaf. I also worked for Omniture for 6 years (2003-2009) and was the go-to guy for anyone that had issues. Yes, it was fun and challenging to get problems that SiteCatalyst couldn't solve out of the box-- with some tweaking, we were able to make it work... But over the years the requests became harder and harder to deal with. I can only imagine how complex things have become since I left. I discovered Tealeaf by working with them in an Omniture partnership. When I discovered Tealeaf I felt like I had found the answer to solving all the crazy problems that came my way. Capture EVERYTHING, pull out what you need in an EASY way, REPLAY the user's experience! To me Tealeaf is the Paradigm shift that everyone is just discovering.

The biggest question, when it comes to any tool, is: "Am I learning something?". If you are not learning, then any reporting is just policing the status quo. If you are not learning you are not using your tool to enable innovation.

So, spoken from a Tealeaf Fan, learning is the easiest thing to do with Tealeaf tools. Sure we lack a lot of the fancy reports you get in a robust analytics tool, but that gap is closing fast. Sure, we are more focused on Customer Experience than on Online Marketing, but it's all in how you use the tool. Tealeaf's core competency is the replay. Replaying user's experience based on segments of users dives deeply into the "why" of your business. Learning is commonplace with the Tealeaf tools.

That's the Paradigm I discovered a few years ago. And there are many out there doing the same...

(I am a fan of Tealeaf, but don't think I won't listen to reason)...
 
The self described fanboy's comment makes me wonder if the needs of the industry are so diversified that there is no one size fits all solution anymore.
 
While I should not be commenting while on vacation, its a great thread - but I must be quick while the blueberry muffins are cooking. The shift the industry needs is to move from reporting to deeper analysis. Today, reports have to be interpreted, often by the one who created it. Functions that show root cause, predict the future and suggest actions to take will make things easier. This problem is universal in the analysis market, not limited to online marketing. The people/process problem has been solved by many organizations. They have a leader that demands fact based decisions, they built an expertise team, train people by role to use analysis, use consultants to get over technical humps and market themselves to their internal communities.

-bill
 
I think the reason the tools can be a pain in the ass to use is largely a function of our "roots" as an industry. Web analytics started in the IT space, measuring log files to generate operational reporting. Most of the marketing and analytic functionality we crave as analysts were largely bolt-ons to that original framework. In other words, I don't believe a web analytics tool has been built yet that has analytics at its core. My two cents.
 
Let's see if we can make this sharper. 10 years from now web analytics will be different. How do you see the data accessed in 10 years? Tell a story if you can.
 
(one month later...) +Ariel Assaf - I envision a client, with a similar interface & capabilities to Adobe Discover, using a datastore that has been built from pages using natural language tagging - something that will be as instinctive as HTML - with totally customisable datapairs & arrays according to your own website needs.

Tagging in 2021
var tags = {
"Number of commas on page": 100,
"Fruit on page": [[apple], [orange], [pear]]}

Drag & drop client query in 2021

[Page views] where page has [over X Number of commas] and contains [Fruit type oranges].
 
The best thing Adobe could do would be to fully realize the power of ReportBuilder. Make ORB the primary front end to the data.
 
Almost 2 years too late to add to the debate - my own view - why web analytics is hard, but SiteCat specifically makes it harder than it needs to be, because of a couple of bad architectural decisions (that weren't so bad when they were made, but are pretty indefensible now): http://snowplowanalytics.com/blog/2013/06/28/is-web-analytics-easy-or-hard-distinguishing-different-types-of-complexity/

Kind of nuts that the points raised above are still as valid today as they were two years ago...
Add a comment...