For Google AdSense publishers experiencing recent ad fulfillment issues, crawler access might be the problem

AdSense has made some changes in the way it buys ads on the Google Display Network that are impacting monetization for some publishers.

Chat with MarTechBot

Google Adsense Icon9 1920

A small segment of Google AdSense publishers have noticed changes in ad delivery on their sites over the past month or so. Publishers say ads are not being served and the ad slots on their pages appear as blank spaces, causing drops in revenue.

There has been speculation on the AdSense forum that the issue might be caused by policy changes, or by dynamic pages or URL parameters, or when a new page has not yet been crawled by the AdSense bot. It turns out that last guess is closest to being correct.

Google is making some changes to buying processes on the Google Display Network. As a result, Google will not automatically monetize sites or pages that the AdSense bot can’t crawl, Marketing Land learned Friday. The AdSense crawlers categorize, classify and label content for monetization.

Therefore, the issues some publishers are experiencing may be the result of not properly granting AdSense crawler access to their sites. Google says the problems aren’t widespread, but the changes — which are still in early stages — are affecting some publishers more than others. (Blank ad spaces can also appear if the ad impression simply isn’t fulfilled.)

If you’re experiencing problems with ads not being served properly — and there are no policy violation notices in your AdSense account — check the crawler access error tab located under “Status” in AdSense for notice of crawler implementation errors. It’s possible the crawler is being blocked in the site’s robots.txt file (see details in this Help Center article) or getting tripped up by when content is served behind a login (see how to give access to login-protected pages here).

Google says it is working on identifying publishers that are impacted to help them resolve issues and will be issuing further guidance in the short term. The team will be evaluating results of the early-stage changes over the coming weeks.


Opinions expressed in this article are those of the guest author and not necessarily MarTech. Staff authors are listed here.


About the author

Ginny Marvin
Contributor
Ginny Marvin was formerly Third Door Media’s Editor-in-Chief, running the day-to-day editorial operations across all publications and overseeing paid media coverage. Ginny Marvin wrote about paid digital advertising and analytics news and trends for Search Engine Land, Marketing Land and MarTech Today. With more than 15 years of marketing experience, Ginny has held both in-house and agency management positions. She can be found on Twitter as @ginnymarvin.

Get the must-read newsletter for marketers.