# Robots.txt file # For domain: http://www.nelson-atkins.org/ # All robots will spider the domain User-agent: * Disallow: /cafe-search/ User-agent: gsa-crawler Disallow: /cafe-search/ User-agent: * Disallow: /in-gallery/ User-agent: gsa-crawler Disallow: /in-gallery/ User-agent: * Disallow: /blog/ User-agent: gsa-crawler Disallow: /blog/ User-agent: * Disallow: /cahn-tumblr/ User-agent: gsa-crawler Disallow: /cahn-tumblr