Taking feeds out of our web search results

Tuesday, December 18, 2007 at 6:40 AM

Written by Bogdan Stănescu, Software Engineer

As a webmaster, you may have been concerned about your RSS/Atom feeds crowding out their associated HTML pages in Google's search results. By serving feeds, we could cause a poor user experience:

1. Feeds increase the likelihood that users see duplicate search results.
2. Users clicking on a feed may miss valuable content available only in the HTML page.

To address these concerns, we prevent feeds from being returned in Google's search results, with the exception of podcasts (feeds with multimedia enclosures). We continue to allow podcasts, because we noticed a significant number of them are standalone documents (i.e. no HTML page has the same content) or they have more complete item descriptions than the associated HTML page. However, if, as a webmaster, you'd like your podcasts to be excluded from Google's search results (e.g. if you have a vlog, its feed is probably a podcast), you can use Yahoo's spec for noindex feeds. If you use FeedBurner, making your podcast noindex is as simple as checking a box ("Noindex" under the "Publicize" tab).

As a user, you may ask yourself whether Google has a way to search for feeds. The answer is yes; both Google Reader and iGoogle allow searching for feeds to subscribe to.

We're aware that there are a few non-podcast feeds out there with no associated HTML pages, and thus removing these feeds for now from the search results might be less than ideal. We remain open to other feedback on how to improve the handling of feeds, and especially welcome your comments and questions in the Crawling, Indexing and Ranking subtopic of our Webmaster Help Group.

For the German version of this post, go to "Wir entfernen Feeds aus unseren Suchergebnissen."