# http://en.wikipedia.org/wiki/Robots_exclusion_standard # http://en.wikipedia.org/wiki/Web_robot # http://www.robotstxt.org/wc/norobots.html # Keekp all robots from all the pages: User-agent: * Disallow: /