Today, Google announced a new AdSense experiments feature that allows publishers to run A/B tests on ad unit settings including ad types and ad styles such as borders and font colors. The feature is available within AdSense as part of Content Experiments.
The “Traffic split” option at the bottom of the set-up page is checked by default. When checked, Google will automatically optimize the traffic split between the two ad settings using a multi-armed bandit approach. The ad setting that is performing better (has a better click-through rate) will be served more often.
Roughly 24 hours after launching a test, Google will display a confidence score indicating the statistical significance of the results. Google suggests waiting to reach 95% confidence before declaring a winner.