This session provides an overview of web analytics and how it can be used to analyze search marketing performance, followed by Q&A with web analytics experts and tool representatives. Moderator: Jim Sterne, Founding President and Chairman, Web Analytics Association Q&A Moderator: John Marshall, CTO and Founder, Market Motive
Speaker: Gary Angel, President, Semphonic
Q&A Speakers: Wes Funk, Omniture Brett Crosby, Senior Manager, Google Analytics Richard Zwicky, CEO, Enquisite
Gary gives an overview of what we're going to cover. One of the problems with these tools is that there is so much information, trying to decide what to do can be difficult. The start of this presentation is summarizing some of the ways to decide what to do.
Tool Basics
1. Data Collection There are two main "flavors" of web analytics - Software-as-a-Service systems implementing using tags - Software you can run in-house that process web log files (and sometimes tags) - Each of these can provide similar data and capabilities.
Tag-Based SaaS systems tend to be easier to implement and have better data-quality out of the box. The most popular systems on the market are currently SaaS-based.
If you have a lot of internal data about visitors (from a gated site, for example), it can be easier to integrate with software you run in-house.
2. Data Quality. Web Analytics data is notoriously unreliable -- and even at best, it's never a "system of record." The idea that "trending" data protects you from data quality problems is only a half-truth.
Key Data Quality issues include: - Tracking visitors over time - Knowing how long visitors spend on each page - Knowing where visitors go when they leave - Counting robots - automated agents as real visitors
Web analytic systems will never reconcile. It's never going to correlate with what the search engines are sending you regarding your advertising.
* What Really Matters?
1. Needs Drive Differentiation. Your measurement needs drive real-world tool requirements.
Some of the key factors that really matter: - Visitor Segmentation. Ability to take a group of visitors and track their behavior. - Dimensional Reporting. How many people that did x also did y. Look at people based on two different variables. Need to be able to move numbers around, not have a vendor that locks up the data. - Setup. Even SaaS solutions take time to set up. - Data Integration (Online). Data can come from lots of sources, the more the system lets you bring together different data the better. - Data Integration (Customer) - SEM Data
2. Visitor Segmentation. Many types of visitor segmentation.
Key Segment Capabilities - Can Segments be created without tags? You don't always know what you want to look for in advance, you need to be able to go back and look at data later. - Can full logical operators be used to define segments? - What data can be used to define segments? Can you use every piece of data you get to segment visitors? - Can external data be used natively and combined with web data in segment creation? - Can Segments be created via data-drive techniques like neural networks? - Can Segments focus on visit or visitor behavior? - Can Segments be defined based on time and event sequences? Most tools don't show what happens over time, what someone did the next week. - Can distributions be produced on key behaviors to assist in segment creation? How many visitors visit one time? How many visit three times? Are people doing the same thing over and over? Are you only getting average? Average is not the same as distribution.
Segment Methodology - Are Segment samples or against all data? - Are segments created in real-time or delayed?
3. Dimensional Reporting. N-Way Cross-tabulation (Viewing the counts of variable by one or more other variables) is an ESSENTIAL part of analysis.
Key Dimensional Reporting Capabilities: - Visitor and Visit Distributions on variables - Cross-Tabulation of all variables in Reporting - Multi-Dimensional Tabulation of variables for Analysts - Ability to export N-Dimensional Tables to Excel - Ability to apply visitor segments to N-Dimensional Views - Ability to distribute N-Dimensional Views once created - NO DATA CROPPING on high-cardinality variables like PATH and SEARCH KEYWORDS
4. Management Reporting. Every online business spends time on management reporting. It is an essential element of telling the business story to key decision-makers. How easy is it to pull out data? Need to figure out how long this might take when you are evaluating a program.
Key Management Reporting Capabilities - Ability to combine and tailor views of the data - Ability to export data to Excel flexibly - Near real-time segmentation that integrates with Excel Automation - API to the reporting data
5. Setup. Building a tag is not rocket science. Many web analytics packages do require a fair amount of work on the tag if you want to take full advantage of their system. How much work do I have to do in advance? How much can I do on the fly? More you have to put in in advance, less flexible things are, and more of a chance of getting settings wrong.
Key Setup Capabilities: - Ability to create most analysis (segmentation, campaigns, funnels, hierarchies) without tag changes. - Light-weight tag - Ability to capture data that is available only in real-time in custom variables - Ability to capture customer identification and use it for data integration
6. Data Integration. Most businesses will ultimately need to combine online and offline data AND website and other online data. You need to be able to get all of your data in one place. You need to think about what type of data you need, and make sure that you can integrate that data. You need to look at this yourself, not just have the vendor say that it's easy.
Key Integration Capabilities: - Ability to integrate with key online systems include email, paid search, competitive analysis, and banner systems. - Ability to provide a data feed back to the client for customer integration. - Open architecture, web service, API, or other seamless data access.
7. SEM Capabilities. Search engine marketing has its own unique demands and requirements. Hare are some of the key points if your primary web analytic interest is in search. Some of these capabilities may not be covered by any existing tool, but are things that would be really good to have.
Key SEM Measurement Capabilities: - Track results by both actual search term used and search term purchased - Track content match scores - Day parting and time parting in the web analytics reporting - Flexible attribution models. Nobody wants to deal with just the first or last campaign. How do campaigns work over time? Can you assign different weights to different campaigns? - Cross-attribution reports (how much of Campaign X overlaps with Campaign Y) - Ability to collapse search terms and analyze them as a unit (important for analyzing the tail). Most tools don't let you do this. - Side-by-side performance of SEO and PPC - Cross-Tabulation of geography by keyword
8. Additional SEM Measurement Capabilities: - Tracking creative in the web analytics tool - Ability to show common "combined" search terms (Term X then Term Y entry) - Ability to Path over time at the Event Level -- X visitors entered on PPC Search - Y re-entered on visitor search - produced a success - Over-time report (visitors who entered on PPC during July did what in August, September, etc.) - Page performance by entry type report (SEO, PPC, other campaign, direct, previous page)
Key Concept: Most tool evaluations focus on things that turn out not to matter at all when you actually have (and use) the tool. Pay attention to what you really need, and evaluate in-depth.
* Summary: Thinking About Fit How important is web analytics to you? - It's a function of how important the website is. - You can't have a great website without analytics, but you can have a satisfactory one. - Some tools demand that you invest more (in time and money). You need to decide how likely that investment is to pay off for you.
The following points weren't covered in the session, but included in the printed handouts. I typed them in already, so I'm going to use them.
How to get started: - Think about your organization, culture, and knowledge - Choose a tool/resource direction that is realistic - Take the time to build a roadmap of what you want to accomplish
What you should worry about: - Getting people is hard - A good implementation is harder than people say - Web analytics won't happen without both tool and resources - The market is immature and there are no safe approaches - What are you getting back -- if you don't demand interesting analysis, you won't get any.
----- Jim introduces the panelists and asks for people to text/send questions in.
First question: Why should you use a paid solution instead of free?
Wes from Omniture: Sometimes data you need to get into is really deep and disparate systems. The next generation of that type of interaction is not just seeing those relationships, but able to do something with that data. Seeing a type of visitor, then treat that visitor accordingly.
Jim asks Richard and Gary for input. When do I need to move from Google to something else?
Richard: We often don't know. About 75% Our Enquisite customers also use Google analytics, and 20% use Omniture. Some of the features in the wish lists Enquisite released yesterday, and that's the type of thing you can get from a paid solution.
Gary: Analytics helps you look at the hard dollars from SEM, problem is so much else is soft dollars. This helps you look at the cost/benefit aspects. Biggest differentiator is not cost, but is the solution the right one for you. Free solution may be just fine.
Jim: asking Brett asking what's the sweet spot for Google Analytics, what problems can they fix. Brett: We try to address as many problems as possible. Money should be spent on doing the analysis and taking action based on the analysis. Observation about industry after they released the tool..before they released, it was a fairly niche industry. Once they released Google Analyticds, a lot more companies got interested in analytics. Noticed that everyone in the company wanted to have access to the data. Challenge was how to serve both the pros, and the people new to the tool. New version in May tried to put things in context to help people understand what is happening.
Audience Question: Recommendations of how to communicate to managers who still talk about hits ("how idiots track success").
Wes: Ask the executives the questions they should be asking. Do some of the work for them. Ask them the questions they should be asking the stakeholders.
Richard: Educational process. Give them a number, then explain what the number means and give them context.
Brett: Show a simple example of what the data means. Brett also points out how many people in the room are very knowledgeable in this field.
Gary: Believes the fault lies with us more than we want to admit. Execs are not stupid. They understand what matters to business, but we (analysts) don't know what language to use with them. People didn't understand value in analytics until SEM, then that helped make things clearer to everyone. SEM is an example of hard money and ROI that can be seen. Talk money, that's what matters to the business.
Audience Question: What are people's thoughts on the move to time on site, and the new trend of looking at user engagement. What does engagement mean? Look at goal of website. In anything, you need to look at the goal and see what it is you want people to do, then look to see if it's successful.
Help people to ask what is the right question. You may or may not care about time on site for your particular business model. Not every company needs to care about branding, but it does for some. You need to find out what matters. Don't just rely on traffic. Need to think about what traffic does.
Audience Question: Could each of you please give an example for a customer where their analytics data helped them to make $150k or more in money.
Richard: Customer doing Africa tours. They looked at the geographic location to see where they had higher conversions but little exposure. They did a campaign to target that area (UK), using localized spelling and other information specific to the UK. They had a 25% increase in world business by doing this.
Brett: He used an example of a surf apparel company. They had a large drop in sales from one month to the next, and couldn't figure out why. They looked at the visitor map, and saw that there was nobody coming from the East Coast. It turns out that the server was misconfigured and blocking IP addressed from east of the Mississippi. Another example is to look at bounce rates to see where content can be changed to reduce the bounce rates and increase conversions.
Wes: He gives an example of looking at where a user came from (via IP address) and adding text in the user's language. For example, a user coming from Spain would see a small text portion in Spanish letting them know where to go for more information in their language.
Gary: With big clients it's easy to get that level of benefit, but harder with small clients. Natural search visitors don't perform as well as those coming in direct, in many cases. Companies will go through lots of time/effort/money to get people to come in from natural search, but may be too expensive. Looked at where people came in, they came in deep into the site, so moved some of the good engaging stuff off of the front page and put it deeper to catch people's interest. Resulted in increasing number of pages people viewed when they came in from search. Also having internal search nearly doubled number of page views for people coming in from search.
Another example: wanted to identify which websites to get rid of. Got rid of about 100 sites after looking at value of sites. Look at which PPC campaigns to get rid of.
Richard: Sort referrals based on which page of search results people came from (page two, page three), target those to come through on page one. Has immediate effect on campaigns, increases authority of site as a whole.
Jim asks Wes: How do you value your search traffic when you're doing a company focused on lead generation, doesn't take credit cards. For them, downloading white paper or viewing webinar is what is important to that client. Look at keywords and understand what groups of keywords lead to what behavior.
Good thing is being able to tag types of keywords, visitors, etc. For example, do I need to focus on executives? Figure out which keywords execs use, up PPC campaign spend for that group.
Audience Question: How do we best integrate analytics info with offline dashboards to show how offline stuff is correlated with online?
Brett: Several common techniques. Easiest way to do this is to use some type of landing page or unique URL. Good way to tell if ads have effect. Not everyone goes to that specific URL though. Another way is to look at geographic visitors. Even if visitors go straight to website, you can see if place where you put radio ad is sending traffic. Try one creative in one geography, another creative in another geography, see which is better.
Richard: Who was typing in a query that was on a billboard, then looked at when and where visitors came from.
Brett: look at ads that have "go to Google and search for term x", people remember how to do that, good results.
Next question: What do you have that will help me with organic keyword ranking?
Wes: Ability to easily see the clicks that come in, add organic words to SEM campaign. Now easy to see report that helps you figure out which keywords to add to SEM campaign. Vice versa. If I'm ranked number one on a paid keyword, do I want to spend the effort to rank number one for that organically?
Brett: Lots of reports to help with this. You can see list of keywords, split paid vs organic, which SEs keywords come from. For paid, it shows where your keyword was placed (top of page, right of page, which slot). New feature is internal site search reporting. What did user type into SE to find your site, then what specific did they search for on your site? Example of searching for cookies, finding your site, then searching on chocolate chip cookies on your site. Decide to then send paid results to chocolate chip cookies. Another tool is website optimizer, multi-variate testing application.
Richard: Looking at extended reporting. Customer acquisition side is lacking, they try to fulfill this. Missed some here.
If we install Google analytics on our site, isn't that telling them how much money we make off of them and then they would charge us more? Brett says no, that information is not shared. A few more questions happened after this that I didn't catch.
