Tuesday, December 04, 2007 at 11:53 AM
Written by John Mueller, Webmaster Trends Analyst, Zürich
In writing and maintaining accurate meta tags (e.g., descriptive titles and robots information), you help Google to more accurately crawl, index and return your site in search results. Meta tags provide information to all sorts of clients, such as browsers and search engines. Just keep in mind that each client will likely only interpret the meta tags that it uses, and ignore the rest (although they might be useful for other reasons).
Here's how Google would interpret meta tags of this sample HTML page:
| <!DOCTYPE …><head> | |
| <title>Traditional Swiss cheese fondue recipes<title> | utilized by Google, accuracy is valuable to webmasters |
| <meta name="description" content="Cheese fondue is …"> | utilized by Google, can be shown in our search results |
| <meta name="revisit-after" content="14 days"> | not utilized by Google or other major search engines |
| <META name="verify-v1" content="e8JG…Nw=" /> | optional, for Google webmaster tools |
| <meta name="GoogleBot" content="noOdp"> | optional |
| <meta …> | |
| <meta …> | |
| </head> |
<meta name="description" content="A description of the page">
This tag provides a short description of the page. In some situations this description is used as a part of the snippet shown in the search results. For more information, please see our blog post "Improve snippets with a meta description makeover" and the Help Center article "How do I change my site's title and description?" While the use of a description meta tag is optional and will have no effect on your rankings, a good description can result in a better snippet, which in turn can help to improve the quality and quantity of visitors from our search results.
<title>The title of the page</title>
While technically not a meta tag, this tag is often used together with the "description." The contents of this tag are generally shown as the title in search results (and of course in the user's browser when visiting the page or viewing bookmarks). Some additional information can be found in our blog post "Target visitors or search engines?", especially under "Make good use of page titles."
<meta name="robots" content="…, …">
<meta name="googlebot" content="…, …">
These meta tags control how search engines crawl and index the page. The "robots" meta tag specifies rules that apply to all search engines, the "googlebot" meta tag specifies rules that apply only to Google. Google understands the following values (when specifying multiple values, separate them with a comma):
- noindex: prevents the page from being indexed (see "Block or remove pages using meta tags")
- nofollow: don't follow links from this page when looking for new pages to crawl (also see "Block or remove pages using meta tags")
- nosnippet: don't show a snippet of this page when displaying it in the search results (see "Prevent or remove snippets")
- noodp: don't use text from ODP (The Open Directory Project a.k.a. dmoz.org) to generate a title or snippet for this page (see "How do I change my site's title and description?")
- noarchive: don't display a "Cached" link for this page in the search results (see "Prevent or remove cached pages")
- unavailable_after:[date]: remove this page from the search results after the specified date and time (see "Robots Exclusion Protocol: now with even more flexibility")
The default rule is "index, follow" -- this is used if you omit this tag entirely or if you specify content="all." Additional information about the "robots" meta tag can be found in "Using the robots meta tag." As a side-note, you can now also specify this information in the header of your pages using the "X-Robots-Tag" HTTP header directive. This is particularly useful if you wish to fine-tune crawling and indexing of non-HTML files like PDFs, images or other kinds of documents.
<meta name="google" value="notranslate">
When we recognize that the contents of a page are not in the language that the user is likely to want to read, we often provide a link in the search results to an automatic translation of your page. In general, this gives you the chance to provide your unique and compelling content to a much larger group of users. However, there may be situations where this is not desired. By using this meta tag, you can signal that you do not wish for Google to provide a link to a translation for this page. This meta tag generally does not influence the ranking of the page for any particular language. More information can be found in the "Google Translate FAQ".
<meta name="verify-v1" content="…">
This Google webmaster tools-specific meta tag is used on the top-level page of your site to verify ownership of a site in webmaster tools (alternatively you may upload an HTML file to do this). The content value you put into this tag is provided to you in your webmaster tools account. Please note that while the contents of this meta tag (including upper and lower case) must match exactly what is provided to you, it does not matter if you change the tag from XHTML to HTML or if the format of the tag matches the format of your page. For details, see "How do I verify my site by adding a meta tag to my site's home page?"
<meta http-equiv="Content-Type" content="…; charset=…">
This meta tag defines the content-type and character set of the page. When using this meta tag, make sure that you surround the value of the content attribute with quotes; otherwise the charset attribute may be interpreted incorrectly. If you decide to use this meta tag, it goes without saying that you should make sure that your content is actually in the specified character set. "Google Webauthoring Statistics" has interesting numbers on the use of this meta tag.
<meta http-equiv="refresh" content="…;url=…">
This meta tag sends the user to a new URL after a certain amount of time, sometimes used as a simple form of redirection. This kind of redirect is not supported by all browsers and can be confusing to the user. If you need to change the URL of a page as it is shown in search engine results, we recommended that you use a server-side 301 redirect instead. Additionally, W3C's "Techniques and Failures for Web Content Accessibility Guidelines 2.0" lists it as being deprecated.
(X)HTML and Capitalization
Google can read both HTML and XHTML-style meta tags (regardless of the code used on the page). In addition, upper or lower case is generally not important in meta tags -- we treat <TITLE> and <title> equally. The "verify-v1" meta tag is an exception, it's case-sensitive.
revisit-after Sitemap lastmod and changefreq
Occasionally webmasters needlessly include "revisit-after" to encourage a search engine's crawl schedule, however this meta tag is largely ignored. If you want to give search engines information about changes in your pages, use and submit an XML sitemap. In this file you can specify the last-modified date and the change-frequency of the URLs on your site.
If you're interested in more examples or have questions about the meta tags mentioned above, jump into our Google Webmaster Help Group and join the discussion.
Update: In case you missed it, the other popular picks were answered in the Webmaster Help Group.
33 comments:
Hi John,
I notice that you do not mention the keywords meta tag - does this mean that Google totally ignores it?
Thanks for asking, Glen! You're right in that we generally ignore the contents of the "keywords" meta tag. As with other possible meta tags, feel free to place it on your pages if you can use it for other purposes - it won't count against you.
Thanks for this nice summary John!
Excellent summary John, it's great to have it all in one place for future reference. Thank You.
Good post John, but can you tell me if Google considers a 0 second meta refresh as a 301? I have a site on a server that does not support htaccess files so I cannot set up a 301 redirect, but I would really like to move the site to another server.
Hi John,
how about language information?
I use the html element attribute "lang" and the meta tag "language" for language settings.
Which is used by Google?
I generally use the keyword meta tags to help maintain the focus of the page. Each page is not only optimized for search engines, but for the demographic.
The three focus points of each page ::
1) Target Audience
2) Value Proposition
3) Call to action
Having the keywords in the meta tags helps the copywriters know what the main focus of the content is.
If your keyword are in the content, then chances are you should start to see results in rankings for those pages.
Jen
Thanks for that info John. I didn't know about a couple... now I know :)
Many years ago, the major search engines would penalize a site for using any meta refresh because it was often abused by SPAMmers. Then about 4 years ago, I noticed Google and Yahoo! had started to accept them as they would a 301 server redirect. This has proven to be a very beneficial procedure for webmasters who started out in a shared or free environment who want to move to a conventional, independent domain of their own. However, this practice is still undocumented by Google and Yahoo! and your post here makes no mention of it. Could you please find and post Google's official policy on meta refreshes? Thanks!
It really would be to your advantage to use Meta Keywords - this would probably discourage Good Webmasters from keyword Stuffing and using hidden text.
Unfortunately,when doing a query - even if one word is missing - a site won't come up on the SERPs.
So Meta Keywords would be a great opportunity to place Synonyms that searchers may be using
I think the placement of synonyms in meta keywords would be a good idea.
Here's a question: How come for a couple sites I released had pages that had the noindex tag but were still craweled and listed in the index? (The pages had the noindex tag BEFORE they were ever crawled)
I also had the pages robots.txt'ed out.
About a month after the first page of the site was crawled they disappeared from the site search...
Don't those meta tags do what they are supposed to do?
Hi SD, I posted something about meta refreshes in the webmaster help groups at http://groups.google.com/group/Google_Webmaster_Help-Indexing/browse_thread/thread/9092c894fdd52749/ - in short: you should not rely on them being treated the same as server side 301 redirects.
Hi Olaf - at the moment we do not use language information from that attribut or meta tag. Is Google having trouble identifying the language on your pages? If so, it would be great if you could start a thread in the webmaster help groups (don't forget to post your URL :-)): http://groups.google.com/group/Google_Webmaster_Help/topics
Jen, that's a great use of the keywords meta tag, thanks for sharing!
Hey Timothy, that's an interesting question. It's hard to say for certain without seeing the URL, but in general, if we crawl a page which you later block via robots.txt, we will never notice any further changes in the meta tags since we will have stopped crawling it. It leaves us in a kind of tough situation - we know that the page exists, it might have value for our users (and yours!), but you do not allow us to look at it again.
One solution would be to remove these kinds of pages with the URL removal tool built in to our webmaster tools. Another possiblity would be to remove it from the index with only a "noindex" robots meta tag (once it is out of our index, you can block crawling of it with the robots.txt).
Thanks John for the Information.
the language detection is fine. (just want to be sure)
search-engines-web and Mick, I don't believe we will be using the keywords meta tag for anything like that in the near future. If you want to target synonyms you will have to include them in your content - doing that generally also improves the quality of your writing, which makes it good for all sides!
Hi John,
That's the first time I see:
META EQUIV="...
Didn't you mean :
META HTTP-EQUIV="... ?
Thanks for catching that, Jean-Luc!
lots of new meta. The content expired after date is good for news site IMO.
Anyway, Out of topics. Does GoogleBot read Stylesheets files (CSS)?
ChaosKaizer: Yes Googlebot does download CSS and JS files. I blogged the details back in January:
http://ekstreme.com/thingsofsorts/seosem/googlebot-requested-a-css-file
Others have seen similar behavior. Would be nice if John would officiall confirm it :)
Pierre
El tutorial está muy bien, lo que pasa es que es muy difícil saber si sería efectivo, ya que el tiempo estimado para comprobar si ha resultado es demasiado tiempo.
Hello! Is it possible for the robot to crawl a specific content first? I mean to say, suppose a content is at the bottom of the page..But I want to implement sonmething so that the bottom content is crawled by the spider first. is it posible?
Ah! So glad to have found this... no need for theories now about the mysterious meta tags. Thanks!!
Hi John,
We recently uploaded sitemaps for our website TolMol.com. The sitemap/crawl report shows warning message that there are too many redirects in our pages. We can't understand why Google Bot is seeing redirects.
We do use "refresh" meta tag on many of our pages for redirecting users to home page after 15-20 minutes. Is this the cause?
Thanks in advance.
Hi, Ramna:
Yes, those meta refreshes are most likely causing your problem. Redirecting after 20 minutes seems like a strange thing to do from a usability point of view (aside from the problematic nature of meta redirects, as mentioned in the blog post); you might want to consider removing the refreshes, or seriously reexamining why you think it's helpful for a user to randomly get redirected to a different page than the one they were choosing to look at.
Thanks, Good post John ;)
Hi there, I've had my site verified since about 2006. All of a sudden it is no longer verified. I cant get either of http://www.juliatrops.com or http://www.okanaganeroticartshow.com verified - each comes back with a dns error. godaddy, the hoster, says my dns is resolving correctly with no problems. Do you have any suggestions for me to follow? I've also been trying to put on the analytics on the juliatrops.com site - but google does not see it. Sitemaps work okay. Any help would be totally appreciated. Thank you! Julia
Hello.
Our WEB site is intended to show different things (translations) depending on the accept-language HTTP header rxvd. Of course the meta description, abstract etc. are then different cause they are translated. Does Google crawler systematically try out all possible languages ? In case not, how to ensure that both a dutch-speaking and a french speaking person will find the info ?
Other question : does Google crawler store the cookies and keep them
- during a same crawling session ?
- during normal cookie lifetime ?
Thanks.
Dimtiri.
Hi, im sorry but i just cant understand how do it upload something, like my sitemap to my blog? thanks
do you recommend the use of site like http://www.uklinkexchange.co.uk/ to manage links on your site and generate more traffic?
Post a Comment