Google Webmaster Central Blog - Official news on crawling and indexing sites for the Google index

Sitemaps offer better coverage for your Custom Search Engine

Tuesday, May 06, 2008 at 12:04 PM



If you're a webmaster or site owner, you realize the importance of providing high quality search on your site so that users easily find the right information.

We just announced today that AdSense for Search is now powered by Custom Search. Custom Search (a Google-powered search box that you can install on your website in minutes) helps your users quickly find what they're looking for. As a webmaster, Custom Search gives you advanced customization options to improve the accuracy of your site's search results. You can also choose to monetize your traffic with ads tuned to the topic of your site. If you don't want ads, you can use Custom Search Business Edition.



Now, we're also looking to index more of your site's content for inclusion in your Custom Search Engine (CSE) used for search on your site. We figure out what sites and URLs are included in your CSE, and -- if you've provided Sitemaps for the relevant sites -- we use that information to create a more comprehensive experience for your site's visitors. You don't have to do anything specific, besides submitting a Sitemap (via Webmaster Tools) for your site if you haven't already done so. Note that this change will not result in more pages indexed on Google.com and your search rankings on Google.com won't change. However, you will be able to get much better results coverage in your CSE.

Custom Search is built on top of the Google index. This means that all pages that are available on Google.com are also available to your search engine. We're now maintaining a CSE-specific index in addition to the Google.com index for enhancing the performance of search on your site. If you submit a Sitemap, it's likely that we will crawl those pages and include them in the additional index we build.

In order for us to index these additional pages, our crawlers must be able to crawl them. Your Sitemap will also help us identify the URLs that are important. Please ensure you are not blocking us from crawling any pages you want indexed. Improved index coverage is not instantaneous, as it takes some time for the pages to be crawled and indexed.

So what are you waiting for? Submit your Sitemap!
The comments you read here belong only to the person who posted them. We do, however, reserve the right to remove off-topic comments.

20 comments:

borisz said...

Great news, I really like the fact that visitors will find more pages then before using my CSE.

Omar Ismail said...

Custom search is awesome. We've been using it for a while now and have only just started to appreciate its power. Seeing more advances and importance given to it with moves like this is great. Creating a supplemental index should help greatly especially in regards to freshness and more obscure/difficult to find pages (though we're pretty lucky in that regard).

I have some other suggestions, that I think would go a long way towards improving the custom search program. They would for us anyway!

I tried to write it in this comment box, but it got too long, so I put it in this blog post

Bill said...

It seems odd to me that Roxy's comments were posted (considering how absolutely irrelevant they are). I do, however, agree with the other posters - Custom Search is great and we look forward to continued improvement in search technology. Nice Job Google!

My description said...

Hey I didn't have your email but I guess you're responsible for the main website. I have one proposition. Here on the main page: Support victims of the cyclone in Myanmar (Burma). why don't you imply the principle of "social proof" and tell (I'm sure many people donate) the money collected or THE PEOPLE who donated! I'm sure if it stands something like: Support victims of the cyclone in Myanmar (Burma). and then: 15456565 people already donated (optional: in the last 24 hours), I'm sure you'll increase the number of donations.

Also, you can offer some cheap from google as exchange for donating. There are many options to enhance this.

Regards :)

BRAHAM said...
This post has been removed by a blog administrator.
Michelthapa said...

Good News to improve our search visibility in Google's SERP. I'm really happy with you

Katty said...

You asked us to submit the sitemap. Well, I tried to do this for my blog on blogspot.com, and asked some help/tips by mail, but I didn't get any answer yet, nor managed it to post the sitemap in google webmaster tools! Please, RĂ­ona, Pete or colleagues could you please send me some help? Thanks in advance!

Susan Moskwa said...

Hi Katty,
Take a look at your robots.txt file--Blogger automatically submits your Sitemap for you via robots.txt

If you also want to submit it via your Webmaster Tools account, check out my advice on how to do so.

If you need any more help, please post in our Webmaster Help Group.

PARVEZ SHAIKH said...

How we can get latest update in google

Katty said...

I think and hope I solved the problem with the adding of a sitemap in google
webmaster tools.

In the past I added in the text-field of the form after selecting general
sitemap the url: http://www.example.blogspot.com/atom.xml?redirect=false
But the http...tihing was already there, that's why everything went fault.

After I only added in the form this: atom.xml?redirect=false
I got the message that thesitemap was succesfully submitted.

I'm so happy the sollution is near!
Hoping I helped some other dummies with this feedback, I send you
My best regards!

Katty

Rafael Montilla Mercadeo Electronico said...

Hi,

Google does not reconize my 404 Not Foud file

http://www.mercadeoporinternet.com/404-notfound.html

can you tell me why?

acumed said...

Help,
I am using a shopping cart for my site and when I checked with Google Webmaster I noticed a warning on the sitemap section that said 303 moved temporarily with a file name. When I followed the file I got a page on my site that said not a valid category. I cannot find the category on the shoppong cart to delete it. I am concerned when I see the word "warning" because I don't understand what or why?

Please educate me,

Scott
www.acumedsupplies.com

shawn said...

I've gotta question...

I understand that we get more indexed pages using the google custom search option, my question is, do we get paid for the advertisements that appear on the results page. My Site is a web hosting site, and I'd love to blog about this should that be the case.

Thanks

Randy Davis said...

I have errors in my sitemap-404. The missing pages are from my blog that was deleted when I changed hosting services.

Does anyone know how to remove the errors from the lost pages?

rdavis@episodexistudios.com

hagay2386 said...

I agree with you Bill

Free pdf, software for you said...

I can summarize the benefits:
1. I really like the fact that visitors will find more pages then before.
2. Seeing more advances and importance given to it with moves like this is great.
3. I do agree with the other posters that custom search is great and we look forward to continued improvement in search technology.

I hope that helps.

Khalid
------------------
WoW Collection 2008

www.luxurybedsets.com said...

What are the disadvantages of using sitemaps.

FinishHard said...

I can't sitemap! pLEASE hELP ME! i DON'T KNOW WHATS GOIN ON NOTHING SEEMS TO BE WORKING

iiricafortii said...

Yeah, Is there any disadvantages by using sitemaps, how can I apply it into my sites?

Registry Cleaner Reviews

Ivan Walsh said...

CSE is probably the most underrated tool in the webmaster toolkit.

Ivan