Yahoo! Slurp obeys the rel="nofollow" attribute for links. Web publishers and blog owners can apply a rel="nofollow" attribute to any hyperlink on their page to indicate that the link may not be an approved or trusted link. Note: Slurp may use a "nofollow" link for discovering content, but the link will not be considered an "approved" link for consideration for ranking of the target page.
This attribute works to reduce the benefits of comment abuse. For instance, websites with public comment areas can apply a "nofollow" attribute to publicly entered links to fight comment spam.
Applying the rel="nofollow" Attribute:
Example:
- <a href="http://spammer.example.com/">buy now</a>
changes to:
- <a href="http://spammer.example.com/ rel=nofollow">buy now</a>
The rel="nofollow" attribute applies to single HTML tags. To prevent Slurp from following all links on a page, place a robots "nofollow" Meta tag in the <HEAD> section of the page or use the X-Robots-Tag as shown:
Applying the "nofollow" Meta Tag:
Example:
- <meta name="robots" content="index,nofollow">
Applying the "nofollow" Directive:
Configure your web server to place the following directive in the HTTP header used to serve the page:
X-Robots-Tag: nofollow
For additional details see: About the Robots <META> tag