#Allow aacceptable BOT's access to everything except .... User-agent: googlebot Disallow: /cgi-bin/ Disallow: /ccount/ User-agent: msnbot Disallow: /cgi-bin/ Disallow: /ccount/ User-agent: Slurp Disallow: /cgi-bin/ Disallow: /ccount/ #Allow Adsense crawler otherwise Google Ads won't work. User-agent: Mediapartners-Google Disallow: /cgi-bin/ Disallow: /ccount/ #Disallow all others access to anything User-agent: * Disallow: /