Introduction

Most companies now have an established web presence with sites that have gone through many iterations and relaunches. These web sites typically represent substantial investments for their owners, but what are they delivering in return? This is an obvious question to ask and there are now many web analytics or web metrics solutions offering to answer this question, but surprisingly few books have been written on this topic. In 2002, after a long time waiting, like the proverbial number ten bus, two have come along at once. This month we review:

Measuring the Success of your Website: A customer-centric approach to website management by Hurol Inan, an Australian consultant.

and

Web Metrics: Proven Methods for Measuring Web Site success by Jim Sterne, a consultant based on the west coast of the US.

In the one day CIM workshop Improving E-marketing Performance I focus on four key areas that marketers need to address in order to achieve web site success. These are the four areas, against which I will review these books:

1. Process.

Defining an e-marketing improvement process for an organisation. The who, what, when and why of web metrics goal setting, collection, reporting, review and action.

2. Measures.

Identifying key metrics for improving e-marketing performance. What are the measures that matter for improving performance.

3. Tools.

Selecting the best techniques and tools for collecting & reporting metrics. This includes selection of tools such as WebTrends which are used to measure the activity of visitors on the web site and ask questions such as how many visit, why do they visit, what do they view and which actions do they take? It also includes more traditional marketing research techniques for assessing customer opinion and satisfaction such as focus groups and surveys.

4. Actions.

Taking appropriate actions based on e-marketing results. The tools allow us to collect and report many metrics, but what counts is how we make use of these metrics. What are the implications of the metrics for how we should change the promotion, design and content of our site?

The Process

Both authors have set themselves a tough challenge since any book with the dreaded M-words Metrics or Measurement is never going to be a light read. I have found that what marketers seem to be most interested in is not metrics, but how to improve their marketing process, hence we have changed the name and emphasis of the CIM workshop from Measuring Online Marketing Effectiveness to Improving E-marketing Performance.

I would liked to have seen more content in both books on the measurement process since this is often the most difficult aspect of using metrics to improve performance. It is relatively easy to select the metrics and tools for collection, but in my view, the more difficult and important questions are how the metrics are distributed for review and action. This raises further questions such as how often do you review different types of metrics, who is involved in the review and who is responsible for taking action? This problem becomes more difficult for larger organisations with different sites for each brand or country.

The final two chapters of the Hurol Inan book address this area: Chapter 11 on Resourcing Web Measurement and Chapter 13 on Implementing Measurement practices. The Chapter titles look both promising, but both fail to deliver on detailed guidelines. Jim Sterne refers to these issues in Chapter 3 Raising the Flag, Convincing the Boss and Chapter 15 Making It Work by Bringing It Together. These give some suggestions, but again more detailed guidelines would be welcome.

The metrics

Both books are much stronger when giving guidance on e-marketing metrics. Hurol Inan adopts a widely used approach of specifying metrics for different stages of the customer lifecycle and then improving effectiveness by minimising the dropouts from each.

Reach (increasing the number of referrers to the site)

Acquire (minimise leakage)

Convert (minimise abandonment)

Retain (minimise attrition)

Four chapters detailing metrics in these areas and including the need for multi-channel metrics form the core of this book. There certainly arent a shortage of metrics in these chapters and the clear structure is good, but for the busy reader it would be nice to know what are the measures that matter which need to be reported on each week or month. This insight isnt provided. If you are looking for a way to summarise key e-marketing metrics, Marketing Insights (www.marketing-insights.co.uk) provides a monthly reporting spreadsheet that highlights key measures such as traffic volumes, engagement and conversion rates.

Jim Sternes book does not provide such a clear framework as that by Hurol Inan, although similar principles of acquisition, conversion, engagement and retention underpin the chapters. They are partly obscured by the American style which is evident from chapter titles such as Sawing logs, How good are you at making noise and How good are you at buying noise. This definitely makes for a more entertaining read, but it also makes getting value from the book harder work some will prefer the more direct approach of Hurol Inan.

Hurol Inan uses 4 fictitious generic examples to illustrate the approach in each chapter. This does provide a consistent approach, but it lacks the immediacy of Jim Sternes book which uses real-world cases based on interviews he has conducted with US companies. These examples from companies such as Compaq, Hallmark and Excite work best for me since they illustrate real solutions to real problems.

The Tools

There are a plethora of different tools for collecting web metrics (the dictionary definition for plethora is superfluity or excess, overabundance). Traditionally, server-based tools such as WebTrends have been popular. These work by summarising across a month or a week, all the events recorded in a transaction log file every time a web page or graphic is downloaded for viewing by a site visitor. More recently, browser-based measurement tools such as HitBox, WebTrends Live and Red Sheriff have become more popular since these are more accurate (See WNIM4 for details). Hurol Inan covers the characteristics and advantages of client and server side tracking well. He also covers other techniques such as traffic volume measurement from panels (e.g. Jupiter MMXI and Nielsen Netratings) and from analysis of ISP traffic (e.g. Hitwise). More sophisticated analysis of site behaviour as available from vendors such as Clickstream, Lumio, Site Intelligence and WebAbacus is not referred to, perhaps because these are UK-based. Jim Sterne stresses that his book is not about the technology. As a consequence he does not cover the tools in as much depth as Hurol Inan. He does cover server-based analysis in some detail, but surprisingly I couldnt find any reference to client-side tracking or traffic volume measurement techniques.

Surprisingly, neither book devotes much space to assessing customer perceptions of the site and a companys services. Hurol Inan makes brief reference to online surveys, but no reference to focus groups. This is a surprising omission since one can gain a lot of insight from the numerical analysis of visitor patterns, but direct feedback from the site users has got to be important if youre serious about improving the performance of e-marketing. In the CIM workshop we briefly cover best practice for online surveys, focus groups and mystery shoppers. For example we look at practical issues such as how to ensure a representative sample for online surveys, techniques for gaining respondents and the right questions to ask. More detail on these issues would be welcome in both books.

The Actions

It is one thing to collect any type of metrics, it is quite another to know how to interpret and then act on the results. Take for example, the metrics about the mix of referring sites. What should you be looking for to improve traffic volumes and quality from the mix of referrers? What are the implications of number of returning visitors or top entry and exit pages or site path analysis? Interpreting and acting on specific data is not described in depth in either book. However, there is more guidance in Jim Sternes book, with useful sections on improving ad and e-mail marketing and optimal site path analysis.

So, if you are looking for guidelines on both how to measure and how to improve I would recommend Jim Sternes book. Here there are best practice guidelines on media relations, web site advertising, e-mail marketing and personalisation. Note though that there is a lot less on the principles of web site re-design in order to enhance effectiveness. Jim Sterne thanks Jakob Nielsen at Useit.com in his acknowledgements for doing what he does (guidance on web site usability), so I dont have to. While Jakob Nielsens site and books do provide expert advice on site usability, I think it does leave a gap in this book. Some basic principles of revising site design to achieve conversion to the outcomes the company seeks from site visitors would be useful.

The Conclusion

Which of these books you prefer will probably depend on how much time you have to read them. Web Metrics at 430 pages is much longer than Measuring the Success of your Website at 230 pages. If youre short on time then you may prefer the clear, direct approach of Hurol Inan which summarises the possible metrics in a well-defined framework. It also offers the best overview of the range of measurement tools available. You will have to work, though, to sift through the metrics to see which are applicable to your organisation. If you already have a metrics framework in place and want a more thought provoking read with insights from managers at a range of organisations, then Jim Sternes book is the one for you.

Next month

In 2000, customer relationship management was attracting a lot of attention, not to say hype, with vendors and solutions providers exhorting marketers to introduce CRM systems. Around this time, surveys started to show that early adopters were struggling to implement systems that were on time, within-budget and met the organisations needs (difficult for any large-scale IT system). Next month, I will update you on the latest, looking specifically at how companies are using the web component of CRM systems.

References

Hurol Inan (2002) Measuring The Success Of Your Website: A customer-centric approach to website management. Pearson Education / Prentice Hall, Frenchs Forest. NSW, Australia.

Jim Sterne (2002) Web Metrics: Proven Methods for Measuring Web Site success. John Wiley & Sons, Inc, New York, NY.