5 Surprising Facts On Quality Score Change

In the paid search world, ad rank and average cost per click (CPC) are determined by three factors: your own bid (max CPC), your competitors’ bids, and a numeric representation of ad relevance known as quality score. This quality score acts as a modifier, making it possible for an ad with higher “quality” to outrank a competitor’s ad at a lower bid.

While the relationship between bids is straightforward to understand, quality score is much more opaque, being affected by many factors such as click-through rate (CTR), keyword and ad copy relevance, quality of landing page, etc.

Many marketers have studied quality score extensively, and there are some excellent overviews on the subject. However, the specific drivers of quality score change, and how exactly they act, remains largely unknown.

Below are a few facts that will hopefully help shed some light on the inner workings of this mysterious metric.

1. Quality Scores Can Rise Or Fall Rapidly From Day To Day

Below is aggregated Google advertiser data showing how often quality score changed by a certain amount on a daily basis. Although larger shifts tend to occur less frequently, there are still a good number of cases where the quality score moved by 5 points or more from one day to the next.

image001

The distribution appears linear on a log scale, where the difference in frequency for each unit change in quality score is around 40%. This means that if there are 100 keywords with a 1 unit shift in quality score during a period of time, around 40 keywords will have shifted by 2 units, 16 by 3 units, and so on.

Quality score change is not always stepwise or incremental: depending on the account and competitive landscape, there are cases where quality scores shift from one extreme to the other.

2. Some Quality Score Shifts Bounce Back Immediately

A good portion of quality score shifts last only temporarily. Of all keywords that saw a quality score change, 21% bounced back to the former score within one day, 32% within two days, and 39% within three. This means that even if you implement a change and see a quality score shift for better or for worse, it is best to wait a few days before arriving at a final conclusion.

3. A Significant Shift In CTR Can Trigger A Rapid Change In Quality Score

After analyzing data on what metrics were likely to trigger changes to quality score, the two that I found to be significant were CTR and average position.

As expected, when CTR increased for a certain keyword, quality score was also likely to rise, in some cases reacting as soon as the following day. However, seeing a higher average position for a keyword without a rise in CTR increased the odds that the quality score will fall.

This shows that when Google factors in CTR to calculate quality score, it models the expected CTR by each position. If recent CTR for the keyword or creative is above Google’s projected curve, the quality score will adjust upwards. Similarly, falling below Google’s projections has the potential to negatively impact quality score.

4. Quality Score Is More Volatile On Head Terms Than Tail Terms

For most metrics, greater volume generally means more stability. Not so for quality score: this is a metric in which head terms change more frequently and by a larger amount than on tail terms (excepting strong brand keywords that are locked in at a high quality score).

image003

A possible explanation for this trend is this: if Google’s quality score algorithm waits until a certain threshold of significance is reached before making changes, then more traffic will allow the threshold to be reached faster.

When trying to gauge the effect of a landing page or ad copy test, it may help to focus on the head terms, not only because of the larger impact, but also because they will be a stronger and faster directional indicator of quality score change than mid or tail terms.

5.  A Unit Change In Quality Score Affects CPC By 5%.

It is well known that a lower quality score means that you have to pay more per click to maintain a given position. The distribution of quality score is relative to competition, so the actual effect of quality score on CPC will be different for every advertiser.

For the accounts I looked at, a quality score difference of 1 had a 5% effect on average CPC for a keyword when controlling for bid and average position. Though this may seem insignificant at first glance, the effect is cumulative, which means that a 5 point difference in quality score can result in a 30% difference in CPC.

Quality Score can be impacted by many different factors, many of them being qualitative and only allowing assessment via extensive trial and error.

While this process can become frustrating, some variables such as CTR, CPC, and average position are immediately measurable. By analyzing how quality score behaves with respect to each of these metrics, we can better assess impact and make faster and more effective decisions on how to improve it.

Opinions expressed in the article are those of the guest author and not necessarily Marketing Land.

Related Topics: Analytics & Marketing Column | Channel: Analytics | Google | Google: Search | Search Marketing

Sponsored


About The Author: leads product marketing at Origami Logic, a cross-channel marketing intelligence solution for modern marketers. With a career of 8 years in marketing and analytics spanning various functions, Kohki's focus has always been on translating data into strategy, simplifying the complex, and bridging the gap between data and organizational silos.



Sign Up To Get This Newsletter Via Email:  


Share

Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. You can read more about our comments policy here.
  • http://www.andykuiper.com/ Andy Kuiper – SEO Analyst

    Thanks Kohki :-) I especially found interesting this takeaway: “..a quality score difference of 1 had a 5% effect on average CPC for a keyword when controlling for bid and average position.”

  • http://twitter.com/kohkiyamaguchi Kohki Yamaguchi

    Glad you found it interesting, Andy! The exact percentage probably
    varies depending on account and vertical, but most studies seem to see
    somewhere in between 5% to 10% effect on CPC for each QS unit.

  • http://www.facebook.com/mr.larry.kim Larry Kim

    wow great stuff

  • http://www.toptiertools.com/ Frederick Vallaeys

    Great data Kohki! I’m sorry I missed your talk in Seattle about this… We just did a few tests at Optmyzr with our Quality Score Tracker tool and we found that there is a bigger shift in CPC in the lower range of QS. So moving from QS 4 to 6 has a bigger impact on average CPC than a move from QS 7 to 9.

    It’s also true that Google does position normalization so like you said, an increase in CTR due to a better position is not a guarantee that quality score will improve.

 

Get Our News, Everywhere!

Daily Email:

Follow Marketing Land on Twitter @marketingland Like Marketing Land on Facebook Follow Marketing Land on Google+ Subscribe to Our Feed! Join our LinkedIn Group Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Marketing News!

Marketing Day is a once-per-day newsletter update - sign up below and get the news delivered to you!