Monday, October 12, 2009 at 3:15 PM
The Webmaster Tools team is lucky to have passionate users who provide us with a great set of feature ideas. Going forward, we'll be launching some features under the "Labs" label so we can quickly transition from concept to production, and hear your feedback ASAP. With Labs releases, you have the opportunity to play with features and have your feedback heard much earlier in the development lifecycle. On the flip side, since these features are available early in the release cycle they're not as robust, and may break at times.Today we're launching two cool features:
- Malware details
- Fetch as Googlebot
Malware details (developed by Lucas Ballard)
More information on this cool feature is available at our Online Security Blog.
Fetch as Googlebot (developed by Javier Tordable)
"What does Googlebot see when it accesses my page?" is a common question webmasters ask us on our forums and at conferences. Our keywords and HTML suggestions features help you understand the content we're extracting from your site, and any issues we may be running into at crawl and indexing time. However, we realized it was important to provide the ability for users to submit pages on their site and get real-time feedback on what Googlebot sees. This feature will help users a great deal when they re-implement their site with a new technology stack, find out that some of their pages have been hacked, or want to understand why they're not ranking for specific keywords.
67 comments:
GRRRRREAT work, I think that being able to fetch as Googlebot will be a big help to webmasters!
It is a very nice feature to try, I did it and typed a wrong webpage and it got a wrong message. Now I couldn't erase it.
What it really does is that we get to see the code just as a search engine will, how ever it will be nice to know which are the most important sentences or paragraphs for Google from those try web pages, so we let them stay just as they are to qualify for some keywords.
yea, noticed it and expected a blog post
however for Arabic language written in encoding windows ISO-1986 is not readable by Google: gives invalid characters
however in search results it is written normally
Nice. I am anxious to see it in action.
This looks to be just the HTTP response. Is there any way we can see the delineation of parsed keywords/text that were derived from the response, along the lines of tool like submitexpress.com/analyzer/?
Looks good! It didn't find any malware on my sites, which is as it should be!
i still do not understand abaot what "Fetch as Googlebot" is, how, what's exactly happened wit my website if i click fetch button.
thank's for your attention
Hi, John from Li'l engine.
Is there a cut off of 585 lines of code returned from the new "fetch as Googlebot" feature?
Or is this actually telling us that Googlebot can see pass 585 lines of code!?
Rgds
simply superb tools... i remeber the malware thing from a link i recieved a while back which ran google diagnostics on your site to check for malware... pretty damn cool stuff
also can't wait to play with fetchbot
very informative post.
RE Lil (October 12, 2009 9:02 PM)
I just checked with 1500 lines of code and 1000 lines of content-heavy code, and both were "fetched" fine. First successful diagnostic from the 'fetch' feature?
One more new few feature from Google.
Ok, so no one has spotted the potential problem here?
Surely this tool will help black hats and spammers out there to create cloaked pages, or to spoof a falsly high PR by using a 301 redirect on detection of the google bot to a high PR site? or is it just me being paranoid?
I know there are several different googlebot signatures that are designed to detect cloaking, but having this tool to debug your black-hatting (if such a verb exists) will only help to create more spam and trickery in the SERPs.
Maybe we should help them some more, eg have a google mass mailing facility, or how about a hand clickfraud generator?
"Fetch as" shows exactly/around 100 000 bytes [100KB].
The question is: does it fetch more than it is shown?
Wonderful, just yesterday I was trying different options to see how my site looked to googlebot. Like this one. Great troubleshooting tool.
It appears on my webmaster tool and U google it right away bring me here.
Thanks for the info.
I like Fetch as Googlebot. I used to search to know how Google see my site, but I didn't believe what other websites displayed. Now I know I can trust this because it is officially from Google.
How to remove URL from "Fetch as Google"?
That's a nice feature and I tried it right out,
but I don't see the difference between the output from it and a regular http fetch - is it just to make sure the google bot can see all the text on my page?
I hoped I would get more details regarding what the google bot sees, and not just the website's source code.
Hey that's cool.....labs in GWT !!!
But have you removed the page with highest page rank from crawl stats? or is it a problem only in my account?
Great new feature to webmasters tool
Nice feature (I think) but what does fetch show me apart from a truncated view of my source code, with which I'm already very familiar? Presumably it would be of use if my HTML was generated rather than hand crafted? Or am I missing something here? Happy to be enlightened.
Does anyone have an answer regarding the cut-off of 585 lines, or the 100kb cut-off as per Leon_linde ?
Googlers, Matt Cutts? Please? :)
@John / Lil: What's the URL for which you're seeing this cut-off?
I noticed this update when I logged in last night and viewed our top 10 pages as Googlebot.
It's also reassuring that we're not reporting any malware errors.
Thanks for these developments.
WoW, Great tools! Keep up the great work!
I'm still hunting for a link to Fetch. George.Campbell@gmx.fr
I'm sorry, but I don't get it. The html-code that googlebot fetches looks exactly the same as in my browser. Is it supposed to be different?
Or what is the benefit of the new fetch as googlebot feature?
I'm just trying to understand...
God bless google. I was about to spend $50 to purchase a licensed version of Malware remover. Keep it up.
Web Design Firm
@Susan Moskwa - We're looking at our domains homepage http://www.lilengine.com
As you can see the page has 'a lot' of content, which is over 100kb.
If googlebot is not crawling greater than 100kb than we would be much better off not having all that content on the homepage.
Has anyone found anything useful on the SEO blogs relating to this?
amazing , this will make sites more secure !
i was trying out other demos of softwares , bless g bot ! :D
http://www.qwertyweb.blogspot.com
@John,
As with 100KB-issue, there's an answer in GWM forum:
http://www.google.com/support/forum/p/Webmasters/thread?fid=09d32bc88ed7a866000475d3ce7c98d9&hl=en
Absolutely fantastic.
I have ripped my hair out in the past hunting for possible malicious code... Written to Google several times begging for a human response... This is brilliant. Thanks Guys.
Great Tasks done.... I think these will help webmasters a lot...
This is great, just today I was trying different tools to see how my site looked to google. This should help webmasters with unexplained indexing issues.
it's nice to see the response HTTP headers.
I believe you can set User Agent Sting to http://www.useragentstring.com/pages/Googlebot/
in about;config in Firefox
Great
great features!!
This is really a great help by Google developer for the rest of the web masters.
Keep it up with the good work!
How do you get into it? The instructions say to login to webmaster tools, select your site, then go to labs->fetch as googlebot, but there is no such selection from webmaster tools.
How do I "Fetch as Googlebot"? It doesn't show up under Webmaster Tools.
This is surely another great SEO tool for us webmasters to keep up with our content creation techniques. Thanks, Gabriel Osagie Irowa
I think this is a wonderful feature in Google webmaster tool. one of my site got malware code and it helps me lot. Thanks !!
Do you warn the webmaster, or give the webmaster the option to get warned, when there is malware detected?
Good work! very informative post
I've tried this feature, but my page lose from google index after that, could anybody help me to solve this problem. thanks
how can i use this new feature (Fetch as Googlebot) because i can't understand hows it's working ?
some body can explain me
hey it's really nice one.. Now we can sure get result soon..
but, i have one questions, if we can add by mistake some other things and we need to remove than what we have to do.. there is no options for delete.. ????
Eager for Reply..
Thanks n' Regards
Fetch as Googlebot helped get a site out of the banned and back into the index.
Nice addition to WMT.
This is a great Feature!! Thanks for that
Hi,
whats difference between "Show the following text as the page owner entry for www.example.com:" and "Show this notice as the page owner entry for www.example.com"
Thanks google for the 2 new features.
I tried this tool for my website http://www.raymeds.com . This fetch as Googlebot was really helpful to me. A great tool.
Thanks for developing a tool like this.
i submitted my site in fetch tool. i get status as success. what is it means.
i submit my new website http://www.lifetimefitnesslocations.com/ but webmaster not very my website why
Yes this is good, but if you also want to compare other pages that are NOT in google yet, or not yours have a look at this little tool we built today (took us just 4 hrs):
http://www.evolvedwebsites.com.au/spiderbot/
Wonderful, just yesterday I was trying different options to see how my site looked to googlebot. Like this one. Great troubleshooting tool.
Hi..
Great Blog!! Your write very well.Thanks for sharing this blog....
I submit my website to google with the "fetch as googlebot",the next day my site Was collected.
Thanks for thise features. THese help me find Malware on my site http://sildenafilohnerezept.com
We've had 2 malware attacks on our website SexMedicinOnline and I used this Google tool every time to clean the site pages every time.
The link to "submit to index" on my "fetch as googlebot" page in "webmaster tools" stopped functioning on 4/27/12. I'm wondering if this excellent tool has been discontinued. http://www.wtv-zone.com/phyrst/
How can I delete what I have added to the fetch as google?
Hi everyone,
Since over a year has passed since we published this post, we're closing the comments to help us focus on the work ahead. If you still have a question or comment you'd like to discuss, free to visit and/or post your topic in our Webmaster Central Help Forum.
Thanks and take care,
The Webmaster Central Team
Post a Comment