Yahoo! recognizes and honors the access policy for robots specified within the robots.txt file that lives (or can be created) on any WWW server.

To exclude your entire web site or a specific section (directory) of your server from the Yahoo! Product Search index, simply place a file at the root of your server called robots.txt.

To prevent most robots, including Yahoo!'s, from scanning your site, you can add these lines to the /robots.txt file on your server:

User-agent: *

Disallow: /

To specifically prevent Yahoo! from crawling your site, you can add these lines to the /robots.txt file on your server:

User-agent: YahooSeeker

Disallow: /

You can also be more selective and indicate which parts of your server robots cannot visit. Learn more about robots.txt.