## Don't allow search engines to crawl the domains directory tree. ## It's too large, and we don't want to reward spammers in any way. User-agent: * Disallow: /*.bind Disallow: /*.rbldns Disallow: /surbl.bind.txt Disallow: /surbl.rbldns.txt Disallow: /surbl.zone.txt Disallow: /cgi-bin/