# Robots.txt file # For domain: http://www.nelson-atkins.org/ # All robots will spider the domain User-agent: * Disallow: /cafe-search/ User-agent: gsa-crawler Disallow: /cafe-search/