FAQs
Contact Us
Go Back to Yahoo! Search
Search Help:
Enter keywords to search help.
Yahoo! Search
>
Yahoo! Search Help
>
Yahoo!'s Web Crawler
Yahoo! Slurp—Yahoo!'s Web Crawler
Why are you crawling my site?
How do I prevent my site or certain subdirectories from being crawled?
How does the Yahoo! Web Crawler handle redirects?
How do I use the rel="nofollow" attribute to help combat weblog comment spam?
How can I reduce the number of requests you make on my web site?
How do I mark web page content that is extraneous to the main unique content on the page?
How do I prevent you from indexing certain pages?
How do I keep my page from being cached in Yahoo! Search?
Why would I see repeated download requests?
Why is your crawler asking for strange URLs that have never existed on my site?
Why can't I find my web pages in your search engine?
How does your crawler find documents on my site?
Does your crawler follow dynamic links?
Couldn't Find What You Were Looking For?
Ask
another question
Tips
on how to ask questions
Yahoo! Help Central
Top Questions
Search Tips
What are the different sections on the Search results page?
How do I allow you to index pages, but not place them in your cache?
What is Yahoo! Video?
Intro to Yahoo! Search
Related Links
Search Basics
Search Indexing
Search Ranking
Search Spam and Deletions
Yahoo! Site Explorer
Yahoo! Search Blog
Yahoo! Directory Help