# gsa-crawler should be able to crawl. User-agent: gsa-crawler Disallow: User-Agent: * Disallow: /scripts Disallow: /KelloggInsight