Ad Testing: Research and Findings Track: Advertising Focus: Ads Skill: Advanced
Session Overview: Have you been doing deep testing of your ad campaigns? Fear not -- the panel in this session has and will be sharing tips you might want to try yourself. How important are headlines, descriptions and other elements in your control? How do consumers interact with the ads?
Moderator: * Gordon Hotchkiss, President & CEO, Enquiro Search Solutions Inc.
Speakers: * Gordon Hotchkiss, President & CEO, Enquiro Search Solutions Inc. * Anton E. Konikoff, Founder and CEO, Acronym Media * Jonathan Mendez, Founder & Chief Strategy Officer, OTTO Digital * Nico Brooks, Director of Search Technology, Atlas * Clay Bavor, Product Manager, Ads Quality, Google
Gord Let's kick things off with Anton Konikoff from Acronym Media
Anton Ad testing is more than a one dimensional activity. Will go into when, where and how to test ads. Anton looks at the world through keywords – "keyword driven marketing"
Keyword Driven Marketing - Equates a keyword with the customer - Focus on the language the customer uses to decipher the meaning and target the messaging accordingly. - Important to keep customer voice and intent in mind when testing. - Equally important for B-B and B-C Why Do Ad Testing - Gain insights in a controlled environment – quantifiable results. - Help understand the effectiveness of the messaging - Identify the appropriate products/offerings Key Metrics - CTR – but as a process metric and a final metric - Conversion Rate by Ad – this is the ultimate measure of success Setting Up Ad Testing - How many ads should you test? -- Whatever you're comfortable with really. (though he says some may disagree) - Sample Size -- How much data --- 100 clicks might be enough but 1000 is certainly better – it depends on the campaign. Ad Copy Testing - Test Headlines and Display URLs - Keep your headline consistent and test the description. Then move back to test headline. - Test the call to action – is it price point?
Pull out the best performer of description test and then test headline.
"The Click Through Rate is Irrelevant" – gets in trouble for this view, but has seen low CTR generate high conversion rates.
More Granular Tests - Punctuation - Capitalization - Proper Case vs. Sentence Case - Accent Usage - Keyword Insertion Sirius Radio Case Study - Tested $30 rebate vs. One Month Free - Period: 1 month - $30 rebate conversion 3.5%, free month 1.5% Common Pitfalls - High CTR may be bad -- Is the traffic qualified - Too many variables? -- Be sure you know which change made an impact Google Image Ad Testing - Tested 3 TV commercials on Google site targeting network & Quigo Video beta test - 3 different ads, different actors and slightly different messaging. Initial screen was the same. - CTR was consistent – but weren't only testing for that. - Metrics -- How far did the customers watch -- What was the CTR - Findings -- Winner 60% completion and 30% higher CTR using simple split test. Path to Success - Start with the most simple/straightforward test -- Apply more variable over time - Be brave and bold in testing radical hypothesis. Challenge the status quo. - Set aside a budget only for testing - Set up a testing process that is ongoing and consistent. Find the right mix of elements and continue to push testing. - Always test something! - Don't settle for current results. Improvement is always possible through structured testing.
Nico Brooks Let's talk about suggestions for doing ad testing, how to measure performance and share an idea about the research. Plus going to share a little bit of data
Suggestions - Separate branded and non-branded keywords - Look at history of the ad clicks/interaction that lead to a conversion – -- Don't just look at the last click. Defining route knowledge – it's how we think about where things are. Know where something is (like a business) without really knowing the exact physical location.
Big Idea - Much of search activity if motivated by route knowledge. - Analogy – Ants and Compost Bucket -- Ant problem at commune. So long as the bucket was moved, the ants would not get to it. -- The ants used route knowledge to find the bucket and when it moved, the ants couldn't find it. - How we think about where something is has to do with how we get there.
Applying to search data - Users remember how they got there before. They may not remember the URL, but they know how they got there. So they go the same way hoping to get the same result.
Search Study - Segmented 275k clicks between first visit and reports as well as branded vs. non-branded Results - Repeat visits were nearly % of total clicks. - Brand data was more than 50% of clicks - Repeat and Branded was nearly 1/3 of all clicks. - Repeat plus branded accounted for nearly 70% of clicks Suggestions - #1 -- Separate branded and non-branded keywords -- Branded keywords aid navigation and boost interest -- Non-branded keywords create interest – which is good from a customer acquisition perspective. - #2 -- Navigation clicks may be masking the true value driven by other keywords and ad types.
Jontahan Mendes It doesn't have to be scary or overly quantitative. Start simple and small then get more advanced. Great tools – some free, some not so free. You can also do it without a tool, just using a spreadsheet.
Keys to Successful Testing - Always have a single question to answer. -- The best test will answer that question and ask many more. - Have a large differentiation between elements that you are testing. -- Draw out the differences - Temporal Behavior -- Results during the week may be different than those over the weekend - Collect enough data to reduce margin of error and increase confidence. -- Too little data may induce false positive. - Segment results - Act! Take action on results and keep on testing. Why Test - Testing is better than guessing. - We test because despite how smart we are we can learn. - Don't Guess!
Ad copy testing itself can yield a significant change in conversion.
Why do ads work? - Impression upon the user - Interest Ads generate interest in the product/service - Expectation that you will find what you want - Persuasion – our goal is to sell the product/service
Test Ads by Match Type - Created redundant ad groups and test against one another by match type. - Understand, test and measure the impact on your campaigns. Brand Keywords – waste of money? - Tested 1 week on 1 week off - No seasonality. - Ad went to homepage - Single version of the ad - 30 branded keywords Did they drive traffic? NO. Did having the ads increase conversion? YES – up to 32% Having the ads was a huge win.
Case Study - Best ads to increase conversion? - Took 4 elements and 3 variations and dropped it into an array for testing. - Design was 9 tagged ads (URL tagging) - Tagged landing page & thank you page. - Ad optimizer was turned off. Results - Some ads really tanked. Some lift and one gave 36% conversion lift. - Gave a window into each factor that had something to do with success. - Description line was found to be the winner – but beware because it varies from test to test. Bake Off - Winning recipe versus predicted recipe - Final winner gave a 52% lift for the client
Gord Was going to talk about a Google research project – brand, non-brand – top versus side sponsored. 8am today came the realization that the presentation would have been about how people respond to different parts of the real estate, but he decided to toss the presentation.
Instead he is breaking the box and looking at the new testing box.
What's Next? - Search results page has been pretty static the last several years. We are introducing many new variables to testing so a heads up is probably better. - Pace of change on the SERP is going to increase. - On the cusp of search experience changes
Strive to maintain balance between ads and organic result relevance. Need to raise the bar on ads to provide a richer experience. Value of images, video in addition to text. Leads us to better targeting – the keyword – knowing more about the customer through personalization.
If the natural results get better – will users stop clicking ads?
When an image is introduced on a page, users in eye tracking study used the image to divide the page. Users looked at result immediately adjacent to the image. Scanning ran down from the image (in slot 3-4) which is different from what is seen in non-image based search results.
Ask users are learning how to navigate the more complex page and are doing it quickly. This will impact advertising.
Tested a personalized search page. Using various groups of data, paid and organic results. Also used personalization to identify the user. Displayed products, stock and stores that carry your favorite product.
Results: Users oriented themselves to the organic and top sponsored listings. Note – the interface was a surprise. - Interaction with the page was pretty typical with about 8 seconds. - Page had a left side bias - Sponsored locations – top sponsored did well, other 2 did almost nothing - Time Spent on page – Organic results got lions share however, shop local section was pleasant surprise. Wrapping Up - Cannot count on same linear scanning activity – which has driven testing historically. - As images appear we "berry pick" - Richer ad choices and personalization will drive the experience. - Lines will become blurred between organic and sponsored listings. - Information scent and top left relevance are going to continue to be dominant behaviors.
Clay Bavor Spend time providing perspective on how Google views ad testing and campaign optimization. Plus talk about a tool that was recently released.
- Goal is to show relevant ads, making sure the user experience is good. - In the long run, a good user experience is good for everyone. How to make the user experience very good? - Ad testing will drive the best user experience - 2 things to do -- Step back and look at keywords. Are they conceptually grouped? --- The more targeted t -- Don't drop the ball on the landing page. Take some time and make sure they correspond to the ad text and keywords that are driving traffic there. -- - Think of ad testing as putting yourself in the user's shoes to drive high quality traffic. Check out "Search Query Report" in Adwords. - You can find a summary of keywords that are driving traffic which will help you improve the overall performance of campaigns and ad groups Circle back to the user experience - Target effectively for the user experience your users want - Keep the users in mind - Test and re-test
Q and A Q – Can some shine light on ad with highest CTR is not the ad being served the most? A – Anton – Idea of ad optimizer is to distribute traffic to the best ads. Other factors are involved – landing page and other quality factors. A – Google – Not sure without looking at the specific instance.
Q – Did the eye tracking study, did you track the demographics? A – Gord – all classified as heavy searchers. Sophisticated searchers.
Q – How can we manage client's historical data if we split out keywords? A – Clay – It shouldn't be a problem. Google stores account history and is never lost. Keywords and ad test history will be there. Don't be afraid to experiment. Some small churn in quality score, but should adjust pretty quickly. A – Anton – You should always look at keyword groupings. It's especially important for contextual ads and looking at ad groups for contextual matching.
Q – When I a/b testing ads, they perform better together than either would on its own, can someone explain that? Can you help me think about landing page testing? A – Jonathan – Look at landing pages, they take some work but looking holistically is the right approach. Test all elements together if you can. A – Anton – Landing pages are second in testing. Make sure you are following best practices first. Make sure they are targeted and crawlable. Best practices will get you 80% there.
