Google Analytics Gets New Testing Functionality: Content Experiments

Google Analytics Content ExperimentsToday the Google Analytics team made an important announcement for marketers and analysts: Google Website Optimizer will be fully integrated into Google Analytics. Google will be discontinuing Website Optimizer as a stand alone product (by August) and the testing functionality will be available to Google Analytics users only. The testing tool will be found in the Content section under the header “Experiments”.

If you are an existing Google Website Optimizer, you will automatically be in the first round of whitelists, but Google is not manually whitelisting users after the public roll-out begins. Also, it is important to note that Google will not offer a migration option for Google Website Optimizer users, if you have tests running you can run them up to August and then you will have to re-create the tests on the new Content Experiments.

content experiments results

Content Experiments Advantages (over Website Optimizer)

  1. Only the control script will be necessary to run tests, the script that redirects the traffic from the original page to the variations. The standard Google Analytics tracking code will be used to measure goals and variations, including existing goals.
  2. Google Analytics advanced segments and page metrics will be available along with optimizing goal conversions based on your test pages. That means we will now be able to segment our results based on any advanced segment, which can help understand for which visitors the winning pages works best.
  3. Improved statistical engine for analyzing experiments, which will help making decisions faster about the winning/loosing pages.
  4. Tests results will not appear for at least 2 weeks. According to Google this is a mechanism to encourage statistical significance and avoid fluctuations in traffic patterns. Post Script: The Google Analytics team clarified that “While we won’t declare a winner for the first 2 weeks, we will be sharing test results from the very first days. It’s a small distinction, but an important one.”
  5. Tests will automatically expire after 3 months to prevent leaving tests running if they are unlikely to have a statistically significant winner.
  6. “Dynamic Traffic Allocation” functionality: traffic will be shifted away from low-performing variations, over to higher performing ones. This feature can’t be turned off. This is to prevent poor-performing variations from doing extensive damage)

It is interesting to analyze the new features and the ones being left behind for now. By removing (almost all) tag implementations, forcing users to test 2 weeks (to take care of statistics), and allocating traffic dynamically, Google is clearly focusing on the beginners, making it simple for companies starting their testing efforts.

Another interesting change is that tests will be limited to 3 months. For experienced testers, it is clear that a test that runs more than 3 months will suffer from several statistical problems, as the data will be very scarce. But this will also stop websites that use Website Optimizer as a cloaking tool, where your original page is optimized for search engines and receives a small percentage of traffic and the variations are showed to users.

Content Experiments Features Not Available Or Missing

  1. Multivariate Testing will not be available anymore. Google will rebuild this functionality at some point in the future (read this testing framework for more on the differences between A/B and multivariate testing).
  2. E-commerce transactions as the goal will not be available
  3. Maximum of 5 variations per test: this is a very big disadvantage and will limit tests significantly.
  4. Maximum of 12 tests per profile at this time: another big disadvantage for companies doing many tests. Post Script: The Google Analytics team clarified that ”the maximum is 12 active/draft tests per profile. People can create more than 12 tests per profile over time, just not run them all at once.”
  5. Implement the control snippet code once and leave it on the page for future tests. As of now, each test will have its own code.

Again, we can notice that features that would be extremely important for advanced users have not been added to the Content Experiments. Limitations such as variations limits, test limits per profile, one code per test, and a lack of multivariate testing are extremely problematic for advanced users. Hopefully Google will lift some of those barriers quickly.

Content Experiments Help Articles

Below are the official help articles from Google. In addition, also check this step-by-step guide on how to create A/B Tests using Google Analytics Content Experiemnts.

Overview Topic

Run an Experiment Topic

Notifications and Errors Topic

Opinions expressed in the article are those of the guest author and not necessarily Marketing Land.

Related Topics: Analytics | Channel: Analytics | Google | Google: Analytics | Top News

Sponsored


About The Author: is the Founder of Conversion Journey, a Google Analytics Certified Partner. He is also the founder of Online Behavior, a Marketing Measurement & Optimization website. You can follow him on Google+ or Twitter.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



Marketing Day:

Get the top marketing stories daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. You can read more about our comments policy here.
  • http://www.seoskeptic.com/ Aaron Bradley

    Great overview, Daniel – thanks!

    Isn’t another advantage over GWO one might list the ability to configure events as conversion goals?  I know this is possible in GWO, but despite Google’s claims that you can set this up “easily” in GWO, the process strikes me as bafflingly complicated, and required quite a lot of additional coding.

    Am I right that Content Experiments offers better integrated support for setting up tests using events as goals, or am I overstating the advantages of CE over GWO here?

  • http://www.kezber.com/ Etienne Dupuis

    What a great week for Google. Replacing Google Places with G+ was solid, now moving Optimizer within GA ! Thank you!

    I wonder what will happen to the Website Optimizer Partners?

  • http://online-behavior.com/author/daniel-waisberg Daniel Waisberg

    You are right Aaron, this is a huge advantage. The capability to use Google Analytics goals makes the conversion tracking much easier. As you said, with events this is especially important.

  • http://fotohelp.kz/ Блог фотографа

    there is no Experiments in russian GA :(

  • http://www.vejaisso.com/ Felipe Veiga

    How can I test these changes on a dynamically generated content, in my case wordpress?
    Should I have X number of posts, all with rel canonical to the main one and put the script on these pages alone? 
    Also, can I set Adsense revenue as goal? I can’t seem to do that, which makes testing of publicity based websites extremely hard. 

  • Mark Castaldo

    I have been using GWO for a while, this update is only going to take away functionality that I used to have. 
    I could always integrate GWO with GA by using custom variables, with that i got advanced segments, bounce rates, and any other slew of stats I could need from GA. I also had the ability to do as many tests as I wanted, combinations of A/B and multivariate, multivariate tests inside of A/B tests, and other more complicated tests. I’m glad this is easier to implement now, (1 copy / paste section of code) but the update hurts the power users. Highly dynamic sites will still require custom setups to get tests operating properly. I guess its a little early to moan and groan too much.. I’m going to wait a few months to see what google does. 

  • Alba_s


    Improved statistical engine for analyzing experiments, which will help making decisions faster about the winning/loosing pages.”

    Can you elaborate on how they’ve changed statistical analysis– apart from declaring no test winner for at least 2 weeks? 

    It appears that the user will still have no control over the confidence level a test page variation must hit to be declared the winner.

  • http://online-behavior.com/author/daniel-waisberg Daniel Waisberg

    Etiene, they will also be renamed to something related to Google Analytics…

  • http://www.facebook.com/dennisgjordan Dennis Bunchofgrapes Jordan

    I dont have access to this tool yet. (working from Germany for a German Website).

  • http://www.convert.com/ Convert.com Experiments

    Daniel, great summary. When I saw this immediately launched Convert Experiments (today) on http://www.convert.com A/B AND multivariate testing, GA integration, live-stats, not variation limits, and slipt testings… ahh ohh and ecommerce revenue, orderitems and transaction tracked with script or automatic from Google Analytics… sweet right.

    Check it out on http://www.convert.com/convert-experiments/ and for anyone stuck with lots of multivariate tests contact me Dennis, and we will migrate the GWO multivariate test to our system.

    Thanx for the post Daniel… still got room for new guest-post on your personal OB site? My old one was months ago.

    Dennis van der Heijden

  • http://www.convert.com/ Convert.com Experiments

    Hi Alba,

    You need control over confidence level, test duration (min and max) keeping winner and cutting losers? Try http://www.convert.com we launched last week and can do all that while having a seamless integration with Google Analytics.
    Dennis

  • http://www.convert.com/ Convert.com Experiments

    Hi Felipe, 

    Testing dynamic pages is possible but adsense revenue as a goal is not an option at this moment. Even ecommerce revenue testing in this tool is not possible even they are both Google Analytics.

    If you are looking for quick dynamic page testing you can use http://www.convert.com and we also do ecommerce revenue testing (we pickup GA ecommerce revenue). Unfortunatly Adsense revenue is not a goal in our tool either.

    Dennis

  • http://www.convert.com/ Convert.com Experiments

    Hi Mark,

    While you wait for Google, why don’t you try Convert Experiments in the mean time. We have 30 day trial (free) offer unlimited tests and variations, multivariate tests, one code for the entire site, dynamic page testing (large groups of pages at the same time).

    Send me an email once you signed up I will give you a tour fit for your site.
    Dennis

  • http://twitter.com/Kelly_Wikads Kelly Watt

    Thanks for the detailed overview of content experiments. It will be interesting how Google rolls out more advanced features to make this tool of greater value for marketing companies over the average user.

  • http://www.servicecentral.com.au/ Service

    Good move Google (although we can’t see it for our Australian site yet). Now, if only they would migrate across Webmaster Tools as well.

  • Online Team

    +1 for a great article. 

    Does anyone know how to share experiement beteeen people ?  
    (at least just possibility to view) 

    When I grant access to the account or profile – Experiements is not there for that person i have given access. 

  • TimFromMA

    Hi Daniel,
    When you say, “E-commerce transactions as the goal will not be available,” what do you mean? Im not a power user, but I thought the Goals were defined as pages. Do you mean I can not test cart and checkout pages?
    Thanks.

  • Jennifer Zibrin

    Although I think you’ve got a great overview here, I don’t agree that “Dynamic Traffic Allocation” is a benefit… That DEFINITELY should be a setting we can turn on and off. The traffic to my site fluctuates significantly – sometimes hour by hour – so the even rotation of page variations is crucial to my getting accurate test results. Do you think they’ll make it a setting?

Get Our News, Everywhere!

Daily Email:

Follow Marketing Land on Twitter @marketingland Like Marketing Land on Facebook Follow Marketing Land on Google+ Subscribe to Our Feed! Join our LinkedIn Group Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Marketing News!

Marketing Day is a once-per-day newsletter update - sign up below and get the news delivered to you!