Yahoo! Search
>
Yahoo! Search
Help
>
Webcrawler
Yahoo! Slurp—Yahoo!'s Web Crawler
- Why are you crawling my site?
- How do I prevent my site or certain subdirectories from being crawled?
- How does the [property_webcrawler/] handle redirects?
- How can I reduce the number of requests you make on my web site?
- How do I prevent you from indexing certain pages?
- How do I allow you to index pages, but not place them in your cache?
- Why am I seeing repeated download requests?
- Why is your crawler asking for strange URLs that have never existed on my site?
- Why can't I find my web pages in your search engine?
- How does your crawler find documents on my site?
- Does your crawler follow dynamic links?
More Help Links
Copyright © 2008 Yahoo! Southeast Asia Pte. Ltd. (Co. Reg. No. 199700735D). All Rights Reserved.
Privacy Policy - Terms of Service
Privacy Policy - Terms of Service