Smaaz’s Blog

New Yahoo! Meta Tags and Weather Update

December 6th, 2007

Yahoo! today announced support for tags that give webmasters even more flexibility over which pages and documents are crawled and indexed by Yahoo! Search. Specifically, they are extending their support of page level exclusion tags — NOINDEX, NOARCHIVE, NOSNIPPET, NOFOLLOW — to provide additional control for archiving and summarization of ANY file type. Previously, these page level tags could only be expressed within html pages through the META directive (for e.g. <META NAME=”Slurp” CONTENT=”NOARCHIVE”>), but based on feedback from webmasters, Yahoo! now enables these tags to be expressed through X-Robots-Tag directive in the http header, giving webmasters the flexibility to achieve exclusions on PDF, Word documents, PowerPoint, video, and other file types, including html files, and increasing their coverage through a simplified process. Additionally, webmasters no longer need access to html templates in order to express exclusions for html files. To take advantage of this feature, simply add the following page level tags to the X-Robots-Tag directive in the HTTP Header. Here are a few examples:

X-Robots-Tag: NOINDEX — If you don’t want to show the URL in the Yahoo! Search results.
Note: Yahoo! will still need to crawl the page to see and apply the tag, so if you don’t wish to have the page crawled, use robots disallow on robots.txt.
X-Robots-Tag: NOARCHIVE — If you don’t want to display cache link in the search results page.
X-Robots-Tag: NOSNIPPET — If you don’t want to display summary in the search results page.
X-Robots-Tag: NOFOLLOW — If you don’t want Yahoo! to crawl links in the page.

Along with this change, Yahoo! will be rolling out additional changes to site crawling, indexing and ranking algorithms over the next few days.

Entry Filed under: Yahoo!