User-agent: * Disallow: /admin/ Disallow: /config/ Disallow: /files/ Disallow: *.doc Disallow: *.rar Disallow: *.zip Request-Rate: 1/60 Visit-time: 0600-0800 Crawl-delay: 216000