Loading ...If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!
Atrios stumbled upon the robots.txt file for the White House. In case you don’t know, the robots.txt file is the file that tells the search engine robots what to do. Most people use it to keep it from accessing certain parts of their site that might be redundant or just not relevant to a search, such as cgi-bin code or something. The White House uses to disallow archiving virtually anything on their site.
If something gets archived, you can’t go back in re-write history. Archiving? Disallowed.
—admin
Subscribe to Metadish using an 
Post a Comment