I'm happy that the White House has a robots.txt file.
- Thread starter Jung
- Start date
so um... what exactly does this mean?
It appears to be a bunch of disallowed folders located on their servers. But I could be wrong.
i believe it's disallowed for search engine spiders to crawl those directories. but don't quote me on that
Gumercules is correct. Robots.txt basically tells search engine spiders to fuck off; it tells them which directories to ignore when crawling the site. Obviously though, robots.txt shouldn't be publicly viewable. It's common practice to block access to it via .htaccess. The Whites House's site admins/developers are just in need of some security training.
yeah, it looks like most of those files have been taken down or really don't matter at all. But it's still funny.
apprently they put it in a couple of months ago
-
Want to get something off your chest? You've found the right place! To learn more about our site, please visit our About Us section.
Registration takes less than 30 seconds to gain access to posting, and many other hidden features available. Why not create your free account today directly or via Google, Facebook, or Twitter?