# This is a file retrieved by webwalkers a.k.a. spiders that # conform to a defacto standard. # # The webmaster for this site is # # I've been using this basic robots.txt for a long time, and you can still find unaltered versions of it scattered around different # sites I've worked on like http://www.geographic.com/ # # Format is: # User-agent: # Disallow: | # # Flag Date By Reason # $l1- 20000228 rpd Installed robots.txt # 20000814 rpd commented out the disallow on /images/ # # ----------------------------------------------------------------------------- User-agent: * #Disallow: /images/ Disallow: /CGI/ Disallow: /cgi-bin/ Disallow: /carlos/