Picking the right web analytics tool / vendor is perhaps one of the most critical decisions you'll make. Given the general struggle by all Practitioners in the ecosystem it is apparent that many end up making a less than optimal decision.
There are lots of different web analytics vendors comparison reports / blog posts / analyst reviews / hidden, or not so hidden, agenda pieces.
[sidebar] [update]
Here are a couple of blog posts you might find to be of value, as you are trying to decide which web analytics vendor to pick for your company.
[/sidebar] [/update]
When Eric requested me to be on the panel at the Search Engine Strategies conference in San Jose it provided a interesting opportunity for me to think about the web analytics industry. You see the talk was part of the track: Analyzing The Analytics Players.
Creating any presentation is part labor of love and part scary adventure, this one was a bit more of the latter due to the many hats that I wear. Author, Blogger, Founder, Analytics Evangelist.
In the end it was both a huge challenge to evaluate each web analytics vendor and a lot of fun to see if I could do it in a way that was very different.
My chosen methodology was to strip away all the marketing spin and micro-nuanced pages and pages of complexity – where everyone ends up having every feature – if only after paying enough or implementing "special" code (while standing on your hands and making a omelette and solving the Iraq problem all at the same time).
I call it the soundbite methodology. Take away everything so that you are just left with the essence. Two sentences that try to do justice by capturing the most essential thought.
The result, I think, is a interesting presentation that, I think, you'll find to be of value, regardless of how long you've been in the field and regardless of which vendor you like or work with. In this post I want to share that presentation with you in a screencast format.
Here is what the video contains:
- Brief introduction to the talk & yours truly.
- Overview of how clickstream data is collected.
- One big problem (and three challenges) that are driving vendors in new directions.
- Comparison of web analytics vendors / tools (Omniture, WebTrends, CoreMetrics, WebSideStory / Visual Sciences, Indextools, Unica, ClickTracks, Google Analytics & Microsoft).
- Parting thoughts: a call to action, standards rock.
And here are some caveats:
- Running time is 47 mins 44 seconds.
- If from the tone of my voice it sounds like I made the video in one sitting in the middle of the night then it is true. :) 0127 hrs to be precise, I am sure you'll forgive the lack of quality production.
- You might not agree with some, or all, of it. I request you to give it a listen and you know where to find the feedback/comments form. :)
Here is the first version of the video, a high resolution version.
You can play / pause using the controls. To skip forward simply click on the long strip you see after the stop button.
Here is the second version from Google Video. You know what to do with this one (although because of quality I do encourage you to give the above version a try):
If you want to download the video visit the Google Video site and you'll find a handy download button there.
If you would like to embed the video on your website / blog then visit the Google Video site and click the blue "Email – Blog – Post to MySpace" button and then click on the "Embed HTML" link. Copy / paste the code into any html document and you are in business.
I would very much love to have your feedback on any aspect of it (production quality, value of the content, positions taken, what's there, what's not there….).
What do you all think? Please share your perspectives, critique, brickbats via comments.
[Like this post? For more posts like this please click here, if it might be of interest please check out my book: Web Analytics: An Hour A Day.]
Via 
Great video Avinash! I am very impressed.
I have been in the web analytics world for ten plus years and I have yet to see such a great articulation of what makes each vendor unique.
As someone who is on the non-practitioner side I found the three challenges part most interesting. I suspect that every vendor will benefit from hearing that becuase it is rare to get a perspective from someone (you) who is not just sitting outside and acting like a expert.
Jason.
Avinash thanks for the very informative video.
The comparison between the different Analytics tools was interesting.
From watching the video i get the impression that the future analytics tools are more focussed on the javascript based tracking as opposed to weblog analyzing tools like webalizer or awstats. Therefore I would like to ask you what role do you see of weblog analysis in the future of web analytics ?
Dear Mr. Kaushik,
I am a(n Avinash Kaushik groupie,) fledgling web analyst and really learning a lot from your book and media presentations. I always enjoy your video presentations but find myself wandering away from the tab after 20 minutes, and passively listening to the rest of the lesson but not because I've lost interest. The only logical reason I can think of is the limited human attention span – probably why youtube does so well.
I was wondering if you've ever thought of breaking lessons into chunks, similar to a standard debate format of: intro, background, main point 1[examples], and conclusion where each comma is another video installment. I'm not a debater, it's just an example.
I know people can hit pause at any time and I've del.icio.us-ed many linked for later consumption, but we all strive for user centric design right? :)
Thank you and best wishes for MarketMotive!
al
Jahangir : Weblogs still provide some value, though with each passing day they provide less value – with the web evolving at the clip that it is. (And with that statement I am inviting the wrath of Steve, whose comment you will surely see in a few hours – it is still late night in Australia at the moment!)
A small reason why weblogs have fallen by the wayside is becuase of technological reasons. The bigger reason is becuase of business needs – both the desire for rapid access to data in a flexible environment and becuase of a lack of fully staffed IT department to meet the needs of business users.
There are companies who have replicated IT skills and people that you might find at Omniture / WebTrends / Google Analytics / HBX. These companies can make weblogs sing. If you don't have that then it is harder (given the business needs and what it takes to collect, store and process data accurately in a timely fashion).
Here's more if you want: The Great Web Data Capture Debate: Web Logs or JavaScript Tags?
And to mitigate some of Steve's wrath let me say again that weblogs can be a source of clickstream data if you have Steve. Or Judah (though he is very expensive! – and if he is not then people at Reed need to pay him more!! :).
I do not believe that javascript based mechanisms are what the future holds for us. They are not sturdy enough, and not flexible enough. Amongst the choices we have they are the most business friendly.
The future will probably bring something like robust real, not fake, event tracking. Or maybe some other technology that is is custom built for a RIA (rich internet application) or desktop software monitoring type experience.
As I say in the video, we are early in the evolutionary cycle.
al : Excellent feedback, I appreciate that very much.
The video is too long. My hope was the content would be compelling enough to hold attention. But as Bryan said to me this morning, nicely, "its missing a talking head"! :)
Your advice is great and if I do these in the future my plan will be to set aside more time so that I can break them into pieces etc. I also need to brush up on my Camtasia editing skills.
I am very grateful for your honest feedback, thank you.
-Avinash.
Very valuable presentation Avinash, I wish that I was there to see it in person (I have seen you present at Emetrics twice and it is hard to replicate the "avinash magic" in a video!).
I found the vendor comparison to be very insightful, really cut through the clutter (I have already sent off a summary of the soundbites to my boss).
There was only one thing I'll agrue with, IndexTools is not quite at the same par as Omniture / WebTrends in terms of feature-set though I'll agree that it is much cheaper.
Your posts are very valuable, thanks for keeping up all the work. I am sure it is not easy.
Matt
Avinash,
Too funny. I'm logging all of your comments for my next review!
Truth be told, I'm "data collection agnostic." As you know, the data collection method is only one element to consider in the big picture of the selection process…
Best,
Judah
Avinash – where's the link to the shootout report? Great seeing you in San Jose, BTW – hope to keep in touch :)
Avinash,
As Steve mentions in
The Great Web Data Capture Debate: Web Logs or JavaScript Tags? I feel being able to be able to look at both weblog and js analytic tools is the best approach. And as Dr. Turner also mentions, its always a good idea to have independence from a single vendor.
So my final thought is, we cant totally ignore weblog just yet.
Maaaaaaaaaaaate! :-) Great Video! Well probably more visual podcast than true video. Which, IMHO as a format/delivery, actually worked very nicely. I could switch to other tabs and do minor tasks while still listening, and flip back for the slide change.
So can't say I'm unhappy with that format!
One slightly personal question to start with. Where *did* you get that picture of me in the IT Power Center slide???? I had no idea I used to look that handsome! And minus my propeller beanie too! :-P
The Lack of People meme, is a signature :-) one from yourself, and dare I say, not exclusive to WA either. Neither of which makes it any less correct. Remind me to point my boss at this video.
One recent development I'm noting in the open source world, is where the analytics side is being embedded in the core underlying website application. CMS. Whatever. The interesting part with that, being that it has a strong understanding of the site structure. At this stage, most are little more than glorified counters, and hence no better than older style log analysers. Promising, but long way to go. IMHO.
You also mentioned talking about Omniture?, I think it was, the cross pollination to surveys and your delight in that? What about the idea of ditching tagging, logs, panels etc and moving to a semi-"tagged" survey model?
We'd lose details, sure, but could we gain in other ways? I'm thinking the ultra simple :-) or :-( that Google Webmaster Console has/uses. To start with. Simpler in some ways. Possibly very powerful if done in line with basic flow analysis. "Smile on this transaction; Unhappy here. WHY!!?!?" type of thing.
My thinking being to try and reduce the complexity, which is overwhelming, and getting back to a high level view of a site or sites.
Part of this thinking comes from your description of the category features of all the tools. Basically – complex, data rich, *APPLICATIONS*. I disagree with your description of GA being Easy. It may well be "EASIER", but it ain't easy. Which reminds me, I should have been giving a lesson to my director on GA and what it all means while I was IM chatting with Judah this arvo. ;-)
What I'm looking for is the magic Green Button. Tells me what's wrong, and where I can fix things. Pipe dream currently? Sure. Be nice when we get there though!
The idea being to have "smarter" tools to have less capable people. And hence really open analytics up to the organisation. So they don't need to be trained at all.
Perhaps a value add for WAA members in some ways: Case studies from members on their insights/analyses of sites.
Start to build a rule/exception analysis repository for members. That'd be worth joining for! ;-)
As for Clickstream and Logs? Well, let's be honest with ourselves for starters. :-) Tagging is just a funky way of generating logs. If you can clickstream with tagging, you can clickstream with raw apache/iis logs. The big issue differentiator for raw logs, being use of the backbutton. Which tagging handles, and logging doesn't.
I regard that as a challenge to solve via alternate methods of analysis. Some of the data mining research I've read – based on physical shopping @ the supermarket – has solutions that look pretty good to this type of problem. We. Shall. See!
I can even see some use for this style of analysis in a true Web 2.0 world. eg Define "markers" which matter, and ignore the rest as gumpf. So to speak.
The end goal not being to be "perfect", but derive actionable insights that increase the ROI. Can't think who taught me that one… ;-)
Wrap up time: I couldn't find my Mr Potatohead "Angry Eyes", so you'll have to settle for "Tired Eyes" instead. And hence get the above wimpy wrath and not the True WRATH OF STEVE. My Apologies. I'll be grumpy(er) next time.
Cheers!
– Steve
Great video/slidecast (?)!
From the non-practitioner side, I'd have to agree with the challenges that you highlighted. Education is a key element to nearly everything that we do, but this can often be difficult to justify.
As Steve mentions, the 'real' is to produce a tool that tells you exactly what to do. The line between usability and web analytics is becoming increasingly blurred resulting in not only richer insights, but better web analytic tools!
Survey integration is certainly an extremely rich form of integration, as it allows for another level of segmentation – based on not only the survey responses, but also the online behaviour. Surveys can also provide a 'quick win' for integrating demographic information, without fully integrating with CRM systems.
With luck WebAbacus will be in the next vendor comparison ;)
Hi Avinash, On format, this material may work perfectly well as an audio download (no offense to the lengths you've gone to find appropriate pictures). The pictures add, but the content can hold it's own via audio. Best, Suresh.
My company bought a $1,200 web analytics report to compare web analytics vendors, it only left us more confused and dazed about which vendor was right for us.
Your free screencast helped demystify the vendors and their strategy in a very incisive way. Just based on your screencast we know exactly which two vendors we are going to invite for a RFP and trial (and we had a list of six!).
I found the format of the screencast to be quite helpful.
Thanks for saving us the aggravation!
I just viewed your video – all of it – some of it twice – I accidentally clicked on a link while it was playing which stopped the video, so I had to start over – oops!
The commentary about silos grabbed my attention – your observation about organizational and data silos is spot on. I believe, however, that the web analytics practitioners are complicit in keeping themselves in the silos. I have met, talked to and corresponded with many web analytics people over the years, and the attitude seems to be that web analytics, if studied granularly enough will reveal all, and this is simply not the case. Web analytics can not provide context. It is a record of actions taken, not actions abandoned or actions interrupted or attempted actions unsatisfied or even actions taken but invisible due to imperfections in the web analytics implementation.
Web analytics practitioners have a lot of data at their disposal, no doubt an overwhelming amount. Unfortunately, what I observe all too often is that the focus is very deep into what people did and not broad about the factors that impact why they might have done what they did.
Those that take the broader view, bringing together disparate views of what and why and how, increase their value to the organization AND increase the possibility of breaking out of the silo.
There is so much value to be derived by weaving it all together, and so much to do to get there. If “web analytics” were a discipline that brought together structure, quality, search optimization (paid and organic), VOC, and THEN overlays visitor analysis, that discipline would sit across the entire organization and not be siloed inside Marketing, Communications or IT. In my opinion, until the discipline itself is broader, it doesn’t offer a compelling value that can break free of the silo.
I know you will tell me if you disagree :-)
Krystol
Krystol : I agree with you, and you are right I would tell you if I did not! :)
All three issues that are working against Web Analytics Vendors in the Vendors ability to monetize their offerings effectively are not Vendor problems. They are "client" problems, and we the practitioners have to fix them.
I am firm believer in the principle of Multiplicity when it comes to being able to make effective decisions about "is my website working".
Multiplicity calls for having the right "basket" / "portfolio" of tools (SEO, Analytics, Quality, VOC, etc), integrated to some extent, that allow you to get a holistic picture.
But I think this poses a huge challenge for Practitioners. Most organizations, people, Analysts / their bosses / VP's etc who touch the web analytics space are:
1) in a constant quest of finding one tool / source that will be god's gift to them and answer all their questions and
2) they struggle with establishing a comfort level living in a world of multiple data inputs (tools or methodologies) and make decisions
Maybe it is that as humans we can't deal with that level of complexity or we are not wired to have our brains be flexible to adapt to this world of "multiplicity"
And yet that is the key to massive and overwhelming success.
Thanks for provoking this thought (and just for the record Ball is a cute last name! :)).
-Avinash.
DONE! :-) I brought my daughter to work today, so just relaxed enough an environment to do a 45 min. video, AND I of course saw the presentation live in San Jose as well.
>>Web Analytics Vendor Tool Comparison
I am a big fan of the 90/10 rule (perhaps a tad too aggressive in the split, me being a vendor and all) – nevertheless, it is an absolute must for any enterprise to actually employ analysts!
But coming back to your comment about IndexTools as a vendor (I feel obligated to comment…) . Us having the ability to “Compare Anything to Anything!” – I believe you spotted very well, where we excel today. And this “feature” sounds so simple, but this is essentially (to some extent) what “the others” call; Data Warehouse, Marketing Warehouse, Visitor Profiles etc… :-) And what they charge through the nose for in the end!!
– we simply do not have the energy (nor people) to deploy too many marketing gimmicks around our offering.
In general a super presentation Avinash.
N.B.
I need to send you another picture of me.. because I look like 17 on the one you used in San Jose.. ha ha
Cheers
Dennis R. Mortensen, COO at IndexTools
My Web Analytics Blog
Hi Matt,
I am sure you somehow expected me to comment on this :-)
>>There was only one thing I’ll agrue with, IndexTools is not quite at the same par as Omniture / WebTrends in terms of feature-set though I’ll agree that it is much cheaper.
Unless of course you are Matt Belkin :-)
I would love to show you that there is not much difference between Site Catalyst 13 and IndexTools Web Analytics 9.
I know that we (the industry in general) is supposed to believe that their product is superior, but in real life there is only a set number of features that we do not do.
So we stick to our mantra of 90% of the features of Omniture for a fraction of the costs!!
..And I am not saying that Site Catalysts is not a great tool – it is. But I also think there is a difference between paying (quote from the last RFP we participated in, competing against Omniture) $70K a year for IndexTools and $240K for Site Catalyst. You can actually employ one or two additional Analyst for the difference.
BUT call or email me Matt and I will be happy to do a walkthrough of our enterprise solution – so you can see that I am not that far off..
Cheers
N.B.
Avinash. Please forward this response to Matt as I am keen to convert any non-believers out there.. :-)
Dennis R. Mortensen, COO at IndexTools
My Web Analytics Blog
avinash
thanks for sharing the presentation. i want to ask a question – what software did use to create that? can you throw some light on the actual execution of a video presentation like that…i'd like to do one for my company but I am a bit tech challenged…can you help
I guess the answer for dividing the gold bar would be to cut it twice so that we can get a 1-egmented, 2-segmented, and a 4-segmented piece. On the first day, I would give the 1 segmented piece, exchange it for a 2-segmented piece at the end of the 2nd day, give back the 1-segmented piece on the 3rd day, exchange the 4-segmented piece for the other two at the end of the 4th day…..
Dear Avinash Ji,
I am reading your book. I am so delighted & happy. In fact you have become my inspiration. I have done my degree from Marathwada University.
Currently working as webmaster at mumbai.
Salute to you sir, from bottom of my heart.
Hello
I have seen that there are only big companies which are reviewed for their analytics software.
How about Stuffed Tracker ?
http://www.stuffedtracker.com
Its run by a small russian company
Hello
Thanks for an excellent article.
In running a small-med size ecommerce business, which shopping cart to use keeping in mind the compatibility with Web Analytics such as Google Analytics etc.
I have not seen anyone discussing these issues as it relates to ecommerce.
What platform would work with Web Analytics.
I have come across a few shopping carts but none support any feature for Multivariate Testing, leave apart any compatibility with Google Analytics or any other service.
As a Memetrics customer, and they offer mostly testing, not analytics, they have a very interesting article about the limitations of javascript.
http://www.memetrics.com/marketing_resources/javascript.html
Insightful words Avinash, however I do have a different take on your comments wrt HYBRIDS:
http://www.advanced-web-metrics.com/blog/2007/10/07/hosted-v-software-v-hybrid-tools/
Hi, Avinash thanks for the very informative video. The comparison between the different Analytics tools was really interesting.
From watching the video I get the impression that the future analytics tools are more focussed on the javascript based tracking. I am working as a Web Analytics Professional in ACS and using WebTrends Analytics in my company for various clients.
Personally I use Google Analytics, as well as everyone I know.
To me, no app even comes close to being as useful, especially with the new Site Speed feature.
Thanks for such a nice article and video post.