benry blog

I’m off to my first EMetrics Summit — in San Francisco. I’m going down on Saturday night and will be there Sunday for the Web Analytics Association Training Day. Even though I’m making my way through the UBC courses I decided to signup for the WAA Training Day as it will give me a recap of the Web Analytics for Site Optimization course I just finished and the Creating and Managing the Analytical Business Culture session relates to one of my three outcomes.

BTW: Jim Sterne’s 3 outcomes exercise to ensure all participants get the most out of the summit is a fantastic idea. I’m surprised this isn’t used at other conferences. Here’s my 3 Outcomes:

1. Identify 3 improvements we could make to our site to better persuade online visitors to take action (such as applying online) so that we can increase the monetary contribution our site makes to the bottom line of the credit union. These improvements could be in design, usability, content or calls-to-action. Locate an individual or company that can help educate our web and marketing communication team on persuasive writing for the web. Implement these three improvements and education by the end of Q4 or earlier.
2. Understand how others are moving web analytics learnings, reporting and outcomes into their organizations. Specifically what information they communicate, how they communicate it and how they maintain momentum so that we can move from tactical reporting to reporting in alignment with our overall business goals as a company and illustrate how the web contributes to these outcomes. By the end of Q4 implement a strategically aligned scorecard/dashboard report that can be used across our marketing team to better understand the impact of our online efforts.
3. Learn 3 methods others are utilizing with organic and/or paid search to ensure relevance in geographical areas and improve click-through and conversion. Determine the steps required to establish a SEO/SEM effort (we do not have one currently) that will help us increase the exposure of our credit union within our geographic area of BC (critical since we are moving towards acquisition rather than retention strategies and are planning to merge with another credit union making us the third largest credit union in BC with a wide operating area from Pemberton to Kelowna). Talk with others — especially those who may be from the banking sector — about their experiences and also connect with Robbin Steif about potentially working together on this effort. By June 15 (post the launch of our new site), develop a strategy for our SEO/SEM efforts leveraging these three methods and begin executing by Q3.

My fourth outcome is to have fun, meet some great people and take it all in.

The agenda for the conference is jam packed full of fantastic presentations. The hard part will be picking which ones to hit during the Success Tactics streams. Additionally I’ll be at the WAA AGM on Sunday, the analytics blogger lunch on Tuesday Monday and the WAW event.

I have to admit though that I’m also feeling a bit apprehensive being around so many knowledgeable web people, many of whom have influenced my development as an analyst or written the many books that I use daily.

Recently I began work on a small analytics project. I wanted to establish an a KPI measurement of our site content effectiveness. The plan was to develop a scorecard that could be leveraged on a regular basis by our communications staff (the content writers) to help them determine what pages and content needed to be fixed. I wanted to be able to use this scorecard to set priorities without having to do a ton of additional analysis (something like a number that yelled ‘fix me’ at a glance) and wanted to be able to monitor pages over time to see if the changes being made were in fact resolving problems.

Robbin at LunaMetrics agreed to help (and was a fantastic sounding board throughout this project).

We began by looking at a number of options for measurement. A number of metrics and KPIs came to mind: Ratio page views to visits; Exit ratio; Bounce rate and Percent one page visits. Here’s an explanation of each of the KPIs I’ve mentioned in more detail:

Ratio page views to visits
Formula: page views of a particular page / visits to the same page
A high ratio, especially on deep pages, seemed to indicate that the visitor was stuck and using the back button.

Exit ratio
Formula: exit visits / visits to the same page
While we might have a high number of visits for a page (meaning that we were getting the traffic there), this KPI would highlight which pages were causing the user to leave when they couldn’t find the information they were looking for.

Bounce rate
Formula: single access visits / entry visits to the same page
A measure of how many people leave without viewing any other pages. A low number tells you that the page is effective in moving the user deeper into the site, a higher bounce rate, the less effective a page is at keeping the user engaged. (Note: HBX, unlike Google Analytics, doesn’t display a bounce rate metric like Google Analytics does, so you have to calculate this in Excel).

Percent one page visits
Formula: single access visits / visits to the same page
Jim Novo noted this one in a Google Conversion University article. It’s the percentage of visitors bouncing off the site (like ‘plexiglas’ says Jim). It’s and usually is tied to global navigation issues and should trend down over time as you make changes to the site or copy. Robbin took the Bounce Rate and Percent One Page Visits KPIs and charted them and they correlated nicely, so using either KPI would work for us.

Looking at each of this KPIs on a per page basis told us we had some problems. Each by themselves was partially effective, but missing bits of information. So we set out to create a KPI mashup. This became known as the Steif Mashup Content KPI. Here’s what we came up with:

Visits * Bounce Rate / Time Spent on Page in Seconds

We liked this. Visits were a good indicator of the traffic volume and page importance to users, Time Spent on Page was a good indicator of the engagement of the user with the page content when they got there and Bounce rate was a good indicator of whether the page itself was working or not.

• A high visit page with a high bounce rate needed to be fixed ASAP. People were going here and we were losing them
• A low visit page with a high bounce rate, could be attended to when priority allowed
• A High visit page with a low bounce rate we could leave for now and come back to tweak later
• A low visit page with a low bounce rate we’d likely not even look at.

Using HBX Report Builder I created an Excel worksheet that pulled in the page path and the metrics related to that page path. For those who have never done this using Report Builder, here’s what you need to do.

• For the page path select ‘Most Requested Pages’ and bring in the page name and page path. If you set a filter you can bring in only pages that meet certain criteria (e.g. I used this to bring in only pages for the ’Banking’ section of our site - which you can set either within the Report Builder interface or reference to a cell in your Excel spreadsheet)
• Adding dependent requests off the page name, I grabbed each of the other values (visits, single access, entries, exits and time spent on page) and added them to their own columns.
• Since HBX reports time spent on page as D:HH:MM:SS you need to break this apart to get a value for the time overall in seconds. That’s easy using the MID function in Excel (cell G3 in the example is the time spent on page cell):
  (MID(G3,6,2)*60)+(MID(G3,9,2)

Then doing the calculations for each KPI was just a matter of referencing the data pulled down into Excel.

Problem was that this still wasn’t giving me an “at-a-glance” view of what pages were working or not. For example for a high traffic sick page, the resulting number was higher than that of a low traffic sick page, even if the bounce rate was the same for both pages. So setting priority at a glance was difficult. So we tried changing the mashup KPI taking out the ‘visits’ and ended up with this:

Bounce Rate / Time Spent on Page in Seconds

This was starting to look better. Now if a page had a higher “score” the more this page needed attention.

• A high bounce rate page with high read time could mean that the content was missing some information the visitor was looking for
• A low bounce rate page with a high read time meant that the content was likely working.
• A high bounce rate page with a low read time meant we were losing the visitor quickly. Content wasn’t working or providing the necessary information.
• A low bounce rate page with a low read time wasn’t getting used much, but was meeting the needs when it was accessed.

Now with this new KPI we’re starting to prioritize content changes that need to be made. As an overlay on this we’ve also segmented these KPIs looking at how the values change based on the traffic source, new vs. returning visitor, member vs. non-member, etc. Where we have pages with lite traffic volumes, we can extend it out to cover a longer period (e.g. last quarter instead of the last month).

So what do you think? Have we tackled this the right way? How do you measure your site’s content effectiveness?

I’m off to the Net.Finance conference next week. A bunch of us from the credit union’s here in BC are heading down — should be fun. I’m not presenting this year (did last year) but will be catching William’s presentation on Vancity’s ChangeEverything project.

There are many topics being covered, but not too much on the agenda covering web analytics specifically (surprising considering the number of FI’s using major providers) but a couple worth seeing:

Web Analytics: Building A Culture Of Measurement And Accountability - James McGuire, Vice-President, Online Strategy & Client Experience, Canadian Personal & Business Clients, Royal Bank of Canada
Sounds like this one will connect well with the UBC courses.

Keynote - Jason Palmer, Vice President of Product Strategy, WebTrends Inc.
He’ll likely touch on the importance of analytics.

There’s also a CMO panel on day two titled Evaluating The ROI Of Using Customer-Centric Marketing Strategies And Its Impact On Customer Satisfaction that has Stephen W. Pardue, Senior Vice President, Financial Services, WebSideStory, Inc.

I’ll post on the conference a bit once I get back and make slides available if they exist.

My interview with Anil Batra on his web analytics site.

I’ve had an interesting week with web analytics:

First, I spent some time chatting with my favorite analytics consultant (bless her patient heart) trying to work out a new mashup KPI and content scorecard to help us measure our content effectiveness and set some priorities (seems like there is a never-ending list of things to fix on the site). It’s amazing that she takes the time to answer my many novice questions.

Second, my annual performance review will for the first time contain web analytics KPIs from my scorecard as success measures. Nice to have a boss that gets what I’m trying to do and the importance of us talking about the success/failures of the site in the context of real data.

Third, I got a chance to do an interview that will appear sometime soon on the site of a favorite blogger of mine.

Forth, we completed a number of recent IA and design changes that will soon be built based mostly on decisions supported by our web analytics data.

Not a bad week.