Tuesday, July 31, 2007 at 3:06 PM
When Google originally introduced Supplemental Results in 2003, our main web index had billions of web pages. The supplemental index made it possible to index even more web pages and, just like our main web index, make this content available when generating relevant search results for user queries. This was especially useful for queries that did not return many results from the main web index, and for these the supplemental index allowed us to query even more web pages. The fewer constraints we're able to place on sites we crawl for the supplemental index means that web pages that are not in the main web index could be included in the supplemental. These are often pages with lower PageRank or those with more complex URLs. Thus the supplemental index (read more - and here's Matt's talk about it on video) serves a very important purpose: to index as much of the relevant content that we crawl as possible.
The changes we make must focus on improving the search experience for our users. Since 2006, we've completely overhauled the system that crawls and indexes supplemental results. The current system provides deeper and more continuous indexing. Additionally, we are indexing URLs with more parameters and are continuing to place fewer restrictions on the sites we crawl. As a result, Supplemental Results are fresher and more comprehensive than ever. We're also working towards showing more Supplemental Results by ensuring that every query is able to search the supplemental index, and expect to roll this out over the course of the summer.
The distinction between the main and the supplemental index is therefore continuing to narrow. Given all the progress that we've been able to make so far, and thinking ahead to future improvements, we've decided to stop labeling these URLs as "Supplemental Results." Of course, you will continue to benefit from Google's supplemental index being deeper and fresher.
58 comments:
Supplemental results helps us search marketers target a specific user and bring them what they are looking for better. When a user searches for a specific person at a certain school wouldn't they rather land on a page that has information for that school - or even a list of others that went to the school that they could contact rather than a general page about finding classmates all over the US?
Keep the supplemental results coming!
Not knowing which pages are in the supplemental index hurts webmasters trying to improve the quality of their site.
PLEASE!!!!! Leave the supplemental tag on the results so we have a chance of improving our web sites content.
Supplemental is STILL Supplemental, we just don't get to see it now.
I take a reasonably long quote from a page I know was in the Main index and the page I took it from comes up in the SERPs.
I take a similar length quote from a page I know was in the Supplemental index and it is no where to be seen in the SERPs.
So there is "less" difference? I see no difference where it counts, in the SERPs.
I do see it though where it hurts, my ability to discern why a given page doesn't show up where it had, while in the Main index but not in the Supplemental index.
Google just took one of the few reliable methods, short of executing numerous searches and still knowing little more about the health of a site or a given page.
Either give us back Supplemental or totally remove any difference between the two because the only thing that has changed now is that we are now blind as to what is happening.
Craig
Thank you. Now people can start worrying about things that matter.
"The distinction between the main and the supplemental index is therefore continuing to narrow."
If I am not mistaken, supplemental pages do not pass PageRank, and links from these types of pages have no effect on rankings... is this correct? Is there a reason you would not want webmasters to be able to identify these pages on their own websites and do something about them?
Aside from making it much harder for webmasters to see how Google views the importance of pages on our websites, what exactly will this accomplish? How can this move possibly improve user experience?
This falls right in line with what I blogged about the other day, having to do with Google's priorities being slightly out of whack, you know? The supplemental label never really did anything for the searchers who aren't already webmasters, the vast majority of them probably never even realized it was there. How is removing it going to help them?
-Michael
Thanks a lot for the information, but i agreed with Jennifer, the maketeers and optimizers need "supplemental results" to bring readers valuable and quality search results
"We're also working towards showing more Supplemental Results by ensuring that every query is able to search the supplemental index, expect to roll this out over the course of the summer."
So we have to operate blind until the end of the Summer?
What does "showing more Supplemental Results" mean? Why not show all Supplemental results where they would be were it not the fact that they are effectively considered "supplemental" whether anyone outside of Google is allowed to see that distinction or not?
Why can't we have the "Supplemental Result" designator until there finally is NO distinction?
Why get rid of it at all? It is easily the most informative and reliable tool webmasters have to understand how Google views a given page.
As for worrying about things that matter, isn't worrying about why a given page doesn't show up in SERPs where it once did or where one thinks it should something useful to wonder about?
The Main and Supplemental index was the only reliable tool we had from Google to determine the health of a given page or site.
Craig
Is Google listening? We are very concerned that this supplemental tool will be taken away.
Please either hear our views and respond, or...maybe close off comments so the webmaster community will loose it's voice (once again).
Martin
Martin, you can be sure that Google, and more specifically, the people at Google that matter, are listening.
I may not like what Google does now and again but one thing I do know for sure, they do listen.
And although they might not reply to comments, although they often have in the past and they might not actually do what posts in comments ask for, they definitely to follow what is being said.
This IMHO is sad as the supp indexed was an indicator to possible navigational errors on our sites.
Removing this indicator just leaves us in a position to guessing the best flow.
Hmm.. despite all the propaganda to the contrary, I know, we know, everyone knows that having pages in supps is BAD BAD BAD.
Yes, certainly, sometimes pages in supps will appear in the main index for very narrow niche searches.. but that's rare.
Instead of giving more tools to webmasters to help users assess their 'penalties' Google has just taken away a tool, THE ONLY TOOL, that webmasters can use to assess how important Google considers their pages to be.
I'm sorry, but in the absence of a replacement tool, I'd have to say the removal of the supp warning is a big step backwards.
Yes, google stop to labeling supplemental result, when today i check my site http://www.3mobilephonedeal.co.uk/
i found all of my pages i got surprise then i check other sites. i found all pages of all sites are out of supplemental result.
Well, I can't say we weren't warned.
The problem is in many cases Google will only index top-level navigational pages instead of deep pages that would make better search results.
That means obscure queries often pull up crap - for example, a search for "blue widget" pulling a foot-long blog category page instead of the blog post thats most relevant to the search query, which is supplemental because it doesn't have enough PageRank.
How user-friendly are search results restricted by PageRank, so that the most relevant URLs to specific queries aren't found in the main index?
Hopefully what I see over the summer will put all this into perspective.
Hopefully this will remove the fear factor from everyone's heart and it will be a sigh of relief for SEO consultants from their clients.
Yet as the feature will still work, it will help the webmaster's to make the website more relevant.
Dear Mr. Google,
You are a popular person I can see, however, us skinny nerdy kids can't compete with you without weapons of supplemental index mass destruction in our arsenal. Since I used Google from the time it launched, and also with a PhD in Googling, this is my first compliant.
P.S.
In regards to a certain website I recently analyzed, with 50,000 organic rooted results:
1. All key content is in supplemental.
2. All key content appears lower in "site:" search; and has much poorer ranking compared to non-key pages, with similar competitiveness factors.
As a conclusion, the supplemental index indicates a certain level of quality within SERPS.
"That means obscure queries often pull up crap - for example, a search for "blue widget" pulling a foot-long blog category page instead of the blog post thats most relevant to the search query, which is supplemental because it doesn't have enough PageRank."
That happens quite often. I have one page that WAS Supplemental that doesn't show up even for long string searches but the site's entrance page and article list page, both of which have abstracts of the page in question show up for searches.
That would hopefully no longer happen if the distinction between Main index and Supplemental index were totally removed but at this point, without knowing a page is Supplemental, it is going to be hard as hell to have any idea what is going on.
I can't wait for all the -950/Full Moon/It's Tuesday after the second Saturday penalty discussions to flare up as they surely will.
I am not sure you all are quite spot on in saying the removal of the supplemental tag on the search results page means you have no leading indicator. It just means you have to do more homework. How so? Here is my 0.02 worth of advice.
1) If PR0, look more closely at the page (this one is obvious)
2) Using the site: search command I always found Google to have a fairly accurate return of page importance. Marry these results of important to the PR of each page and you may see this. When supplemental result tags existed, you would often see this after the first N pages of your site, right? Lower quality pages listed later on in site:
3) Using analytics or a program like www.tracewatch.com you can analyze what pages get visited and which ones do not. You can see direct influence of organic results to a page as well as path analysis (often a search gets you to page A and then they navigate to Page B and then page C where the real desired landing should have been, etc).
By using these simple techniques and some other analysis that I won't get into you are surely well armed to figuring out what pages are deemed desirable and which ones have been thrown into the dredge of SH.
This is bad news for webmasters, but more importantly bad news for searchers. To me it points out the flaw in Google's singleminded focus on page rank and links - a focus which often ignores the pages that are the most relevant. Suppplemental pages are basically - pages that Google knows about, but doesn't choose to let the searcher know about. True, most of those pages are without pedigree. But many of them are far, far more relevant than the pages it is going to tell them about. The pages in the primary index are often less relevant, but benefit from being around long enough to have some page ranks or incoming links (probably from some non-relevant origin). Bring back the supplementals, please - or ease up on page rank and links in favor or relevance.
Seems like the most important thing is for Google to always improve the results for searchers. This, IMO, has always been the #1 goal and still is, correct?
Then allowing for webmasters to see, fix, and allow their once mis-coded, mis-directed, or other problem pages to has a chance for indexing should improve results and give site owners a chance to fix things.
Whether supplemental results can be viewed live or not, Google should just add this capability to the Webmaster Console so that webmasters can fix their own problems there.
I would just say that eliminating it altogether is ridiculous since you already indicate problem areas in the Webmaster console for webmasters to look at and fix whenever they log in.
Best Regards,
Brian
SEOposition.com
Regarding the comments about the loss of supplemental results indicators as an SEO tool:
site:www.domain.com/& works fine for listing supplemental results, even without the explicit marking of URIs as supplemental.
I agree with most of the comments here. I don't think the existence of supplemental pages are good for searchers at all. All it does is force webmasters to buy links, which of course google doesn't like.
I get frustrated when a page is on page no1 and then disapears totally because it falls into supplemental. I then try and get more backlinks via ezinearticles, squidoo, gather etc but it doesn't help.
I’ve been in the industry for many years and I think stripping the label is a smart step for Google. Yes I know that our ability to easily identify poorly performing pages is going to become much harder for us and certainly a pain. However what did we all do before Supplemental Results ever existed?
It does take more work for us to monitor our favorite analytics reports for pages that are not performing well in conversions or visitors but it is still completely doable for all of us with the right software.
I like the fact that we are trading in the ability to see the ‘label’ for the ability to have our pages in the Supplemental Index crawled more often or at least a faster opportunity to get the pages out of that index.
I am confused. The blog post states that pages in the supplemental index are "often pages with lower PageRank and complex URLs". My site don't use complex urls so I'm focusing on PageRank. OK, the way to increase PageRank is for other webmasters to link to your site. Of course, we all know that buying links is a no-no. OK, I'll wait for webmasters to stumble upon my site, see how wonderful it is and provide a natural link. But wait. 90% of my site is in the now-invisible supplemental index so the chances of people stumbling upon those pages are slim. So I may never gain more PageRank on those pages, especially if I do not even know which pages are supplemental. For webmasters in my position, this is a frustrating, never-ending circle.
Thanks for listening,
Teri Fritts
AliceAccents.com
Yeah but having your content put into supplementals simply because it does not have enough incoming links...sux, period.
Ya, That's a good idea to remove supplimental Result Tag. At the end of the Day Google will Show Only best Relevant results so if you want be on the top. Work on the pages & Not Tricks. It does not Matter at all to Tag it as supplemental for users.
As long as you parse and index every word on a Supplemental Results Page and give them an equal chance to rank for queries, I don't care what you do behind the scenes.
My concern all along has been that Google should not be hiding Supplemental Results pages as a way of silencing criticism without giving the many Supplemental pages that provide unique content and value a fair chance to participate in search results.
If all you're doing is removing the label but continuing to practice Web Apartheid, I promise you there will be more criicism, much louder criticism, much more frequent criticism.
I'm encouraged by the fact you say the gap between the two indexes is narrowing. But if pages that don't violate your Webmaster Guidelines fail to rank and fail to pass anchor text, then you're just playing games and being sneaky.
We won't forget what the Supplemental Index has done to the visibility of millions of innocent pages. Restore that visibility and all will be forgiven
I guess you guys have something in mind of fighting with duplicate content issues. Well being put into Supplemental Results made people to no copy others stuffs. Now, as they are being eliminated then this may cause a little bit of problem.
Well I know you may have something in your mind. Share it!
Oh I'd say it's about time. So many people think of the supplemental index as a punishment by Google rather than a natural consequence of how their pages are doing in relation to others.
I had noticed supplemental results with fresh cache dates, so there no reason to think of them as badly indexed because they are supplemental or supplemental because badly indexed (though this latter could certainly be true). It's just that most likley the contents are last week's flavor and there is stuff that's much more interesting and up to date NOW (and not that's not decided by Google but rather by the rest of the web including our own sites). And if my page on blue widgets is supplemental and yours is not, all that this means is that my page isn't as relevant to the query as yours (looking at it with a cool head)- or we don't know how to search to get the results (often the case). Can't all be tops. Otherwise maybe Google should only return 5 or 10 results and be done.
is it possible to totally remove my website from all results msn google tahoo, then resubmit same domain, with hole new site?
I'd second the suggestion another poster made for a supplementals tracker in webmaster tools. Working on supplementals is a simple way to improve a site, and Google is taking away the best method of measuring your progress.
I certainly like having them labeled. There is still easy way to tell if a page is supplemental but when dealing with large site with 20% of thier pages supplemental, well, ouch. Surely someone will write an app that automates the new process for isolating these pages.
Thats what Mr.Web Guru thinks. Yeah, I'm going to have to blog on my own about this also.
Cheers,
Robert
Please, Mr. Google, Im trying to do SEO and you are not being fair to me!!!!! Help!
Yes, the supplmental index still exists, but you cant see it. Most importantly, here is a cheat sheet for avoiding and removing yourself from the supplemental results:
http://webmarketing.veracart.com/2007/08/complete-short-list-on-googles.html
whoopos ! link didnt post:
Supplemental cheat sheet
does google frown upon using the site:mysite.com& as a command to reveal the pages that are still in the supplemental?
I ask because having to add the & makes me feel nervy that i might be breaking new guidleines.
I suspect it would not worry them if you use the & but I can only get about half of the pages to list from a large website so I dont know all the supplemental pages.
Perhaps a supplemental page should be shown by something less obtrusive than the word supplemental so that webmasters know the page is supplemental but it is not meaningful to the general user.
I'm not sure why Google wants to remove the supplement tag on pages that are in the supplement index. Now it is more difficult to fix quality issues of pages if you don't know which pages are in supplement.
Like others I used the supplemental results to identify low value pages eg pages with too many graphics not enough text, similar titles and meta data etc.
I could identify supplemental pages, fix them, or block them with a robots.txt instruction.
Now I can't and one of two things will happen. I'll lose rankings and won't know if it's a supplemental problem or not. The second possibity is I'll be competing against spammers spewing valueless, auto-generated content.
Really this is a big news for all the seo webmasters.
Is Google now able to indexing URL'S with more parameters and conditions ?
Danish Kamal
I think people paid too much attention to the supps - they didn't return less traffic and would rank well (top 10) for relevant terms.
All webmasters need to work hard at getting quality pages into google if that's how they want to be found.
I welcome the changes, and hope google continues index what it can
Please make supplemental results available in Google Web Master Tools. They are very useful in debugging multiple URLs to the same content, linking issues and other SEO issues that are not as easy to understand without the distinction.
The suggestion by Catfish seems to be excellent approach to achieve what I suggeasted in my earlier comments.
Make a different section in webmaster tool for supplement result, this will help up to see the supplement status.
Hi All,
I was writing an article about Google Supplemental Index and the way that webmasters can get their pages out of the Supplemental Index but found out that Google has removed the label.
When I read the article posted on this page, I realized that now we don't need to try to get our pages out of the Supplemental Index because as it is explained in this article, Google is trying to narrow the differences between Supplemental Results and main result. So it means the pages that are in Supplemental Index can also be seen in the search result and get ranked.
I think Google is trying to remove the Supplemental Index little by little because now they have become able to crawl the web pages much more precisely than the past and two pages that were used to be looked so similar to Google and so one of them had to be placed in the Supplemental Index, now are different and distinguishable to Google.
I think this is a big progression and webmasters should be happy. This is what the this article is trying to say.
Best regards,
Vahid
My main problem is that a person does a search, gets a link to my blog for a description of the post they want, and it links to something else on the blog. My people are not blog savey or patience enough to search the whole thing, even though there is a "search this blog" feature.
Just bringing up the right post would be a big help, and wouldn't cost me readers!
Well, it may work for you, but for my site, you put pages in the supplemental that DID have a page rank, and as high as some of the other pages you left in the main index. They were popular pages for my site. You also removed the PR for pages that I have had rank on for YEARS.
What Google ALWAYS fails to realize is, that you are painting the criteria for searching/ranking with a VERY broad brush, and not all sites should be looked at the same. Such as sites that sell things. If someone comes to my site and goes to 3 pages, and buys what they need, my visitors will only view a few pages, which is a criteria you base PR on. How deep they go.
You all need to rethink the entire criteria. Information sites, ecommerce sites, etc. should NOT be all treated the same. Especially in my industry where there is so much keyword stuffing, and importance placed on backlinks, and their text, when they might not be relevant at all!! The evidence that you are looking at backlink text more these days is that a site with NO search keywords on the page, AT ALL, can come out of the blue and OUTRANK a site that has been ranked for YEARS in the same spot.
NOT FAIR.
I agree Lynne22. Right now I have a site that was ranked in the top 3 for a very, very narrow search (keywords specific to the client's biz plus the city they operate in). 2 or so weeks ago, Google dropped it back to the second to last page. We went from page 1 to page 78. What's frustrating is that I don't know why. Did I fall into the supplemental index? If so, why should we be penalized in the case of such a narrow search. Our pagerank was nada (we didn't register on the radar) but a month ago it didn't matter which was correct in the sense pagerank on such a narrow search shouldn't matter. I feel like roots level control of the web is being taken out of our hands. If our site is doing something against Google "rules" then I wish we could be informed. This up and down ride has happened 2 other times in 6 months. Each time, it takes 6 weeks or so to get our page 1 standings back.
Pulling supplemental checking out from under us and other tactics undermines Google's credibility. Heck, why is it that the site in question ranks consistent in MSN and Yahoo month after month. Google is great but the quality of the search results on the competition is just as good.
Keep messing with "the formula" Google and people will just move on...no not in a mass exodus...but with enough momentum that you'll be looking back thinking where the hell did we go wrong.
complex URL is a dynamic url?
There is a thread in webmasterworld.com about the reasons why Supplemental index exists. Here is a summary of the availiable fixes for this google penalty.
Supplemental results helps us search marketers target a specific user and bring them what they are looking for better. When a user searches for a specific person at a certain school wouldn't they rather land on a page that has information for that school - or even a list of others that went to the school that they could contact rather than a general page about finding classmates all over the US?
Keep the supplemental results coming!
So no more supplemental results this is really helpful and might help small pages rank in the search results specially if a web page gets more inbound links it will be moved back to the main index and rank even higher again.
What does "showing more Supplemental Results" mean? Why not show all Supplemental results where they would be were it not the fact that they are effectively considered "supplemental" whether anyone outside of Google is allowed to see that distinction or not?
I think there is a lot of confusion over the supplemental pages issue. We all know about writing good original content but sometimes even so pages end up in the supplemental pages.But often there is good information on the pages in the supplemental pages.
1st Health News
Thank you. Now people can start worrying about interested things , and immune everything.
www.jesuschrist-shop.com
Supplemental Pages are still big question. From last 15 days google is not showing supplemental pages for my Presidential elections 2008 related site. Sometime it becomes difficult to understand what the funda behind all this type of results from supplemental results.
As a beginner in this game, I'm definitely in favor of the supplemental results. Some people may not like it but in the grand scheme of things it's nothing but an improvement!
Hi everyone,
Since several months have passed since we published this post, we're closing the comments to help us focus on the work ahead. If you still have a question or comment you'd like to discuss, free to visit and/or post your topic in our Webmaster Help Group.
Thanks and take care,
The Webmaster Central Team
Post a Comment