In general, Yahoo!7 Slurp should only download one copy of each file URL from your site during a given crawl. Occasionally the crawler is stopped and restarted, and it re-crawls pages it has recently retrieved. These re-crawls should happen infrequently, and should not be any cause for alarm.

Yahoo!7 Slurp re-reads /robots.txt fairly often so that any changes to the robots exclusion rules are applied promptly.

There are other Yahoo! crawlers which read content from your site as well as Yahoo!7 Slurp. Check your server access log or user-agent log for firm identification of the crawler. Other Yahoo! crawlers include Yahoo! MMCrawler.