Search Help:  
 
Enter keywords to search help.

How can I stop Yahoo! from crawling my news source?

Yahoo! recognizes and honors the access policy for robots specified within the robots.txt file that lives (or can be created) on any WWW server.

To exclude your entire web site or a specific section (directory) of your server from the Yahoo! News Search index, simply place a file called robots.txt at the root of your server.

To prevent most robots, including Yahoo!'s, from scanning your site, you can add these lines to the /robots.txt file on your server:

User-agent: *
Disallow: /

To prevent just Yahoo! News from crawling your site, add these lines to the /robots.txt file on your server:

User-agent: yahoo-newscrawler
Disallow: /

You can also be more selective and indicate which parts of your server cannot be accessed by robots. Learn more about robots.txt here.

Blocking RSS Feed Only

If you don't want to be included in the RSS feed offered by Yahoo! News Search, but want your content to remain accessible in the normal news results, please use the News feedback form. Select the "Remove Site From RSS Feed" option from the pull-down menu. Make sure to include the following information in the feedback window to assure that we can process your request:

  • Name of news source
  • URL of news source
  • Contact email address
  • Business phone number of authorized contact person
  • Statement that you're requesting that articles from the source noted above be removed from Yahoo! News Search RSS Feed.
  • Statement by you that the above information in your notice is accurate and that you are the intellectual property owner or authorized to act on behalf of the intellectual property owner

After you complete the form with the information requested, we'll need to verify that you're the content owner or authorized by the content owner to make the request.

Yahoo! will make every effort to remove your products and domain within seven (7) business days after verifying your authorization.

After you make this request, all news articles from your source will be removed from the RSS index.

Please note: Articles removed from RSS may otherwise be available through Yahoo! News Search or Yahoo! Web Search.

Was this article helpful?

Yes   No
Click to contact Customer Care for further assistance.