FAQs
Contact Us
Go Back to Yahoo!Xtra Search
Yahoo!Xtra Search
>
Search
>
Webcrawler
Yahoo!'s Web Crawler
Why are you crawling my site?
How do I prevent my site or certain subdirectories from being crawled?
How does the Yahoo! webcrawler handle redirects?
How can I reduce the number of requests you make on my web site?
How do I prevent you from indexing certain pages?
How do I allow you to index pages, but not place them in your cache?
Why am I seeing repeated download requests?
Why is your crawler asking for strange URLs that have never existed on my site?
Why can't I find my web pages in your search engine?
How does your crawler find documents on my site?
Does your crawler follow dynamic links?
Top Questions
How do I prevent my site or certain subdirectories from being crawled?
How does the Yahoo! Search webcrawler handle redirects?
How do I prevent you from indexing certain pages?
How do I allow you to index pages, but not place them in your cache?
Does your crawler follow dynamic links?
Related Links
Yahoo!Xtra Help Central