WTF ... IS WTF!?
We are a collective of people who believe in freedom of speech, the rights of individuals, and free pancakes! We share our lives, struggles, frustrations, successes, joys, and prescribe to our own special brand of humor and insanity. If you are looking for a great place to hang out, make new friends, find new nemeses, and just be yourself, WTF.com is your new home.
question...
- Thread starter Unforgiven
- Start date
It's a web crawler name...
"yahoo! will be modifying its web-indexing robot (crawler) user-agent to reflect our company name. This is the crawler we use to build a searchable index for the yahoo! search services. To ensure consistency and minimal disruption, we will continue to maintain the "slurp" name within our web crawler user agent and continue to support "slurp" as part of any robots.txt files that references this. "
"yahoo! will be modifying its web-indexing robot (crawler) user-agent to reflect our company name. This is the crawler we use to build a searchable index for the yahoo! search services. To ensure consistency and minimal disruption, we will continue to maintain the "slurp" name within our web crawler user agent and continue to support "slurp" as part of any robots.txt files that references this. "
-
Want to get something off your chest? You've found the right place! To learn more about our site, please visit our About Us section.
It takes less than 30 seconds to gain access to posting, and many other hidden features available. Why not register your free account today? You also have the option via Google, Facebook, or Twitter!