Fetch a site’s live robots.txt and test a path for Googlebot, Bingbot, GPTBot, ClaudeBot, or a custom user-agent.
Only public web targets are accepted. Private and reserved network ranges are blocked.
See the live robots.txt decision, winning directive, matching rules, and sitemap declarations for a crawler/path pair.
Use this check when you need a focused answer about the specific signal described above rather than a broad, unexplained SEO score.
Uses transparent longest-match logic against the live public robots.txt file; it does not impersonate a search engine crawler.
Because website responses can change by time, location, resolver, or server configuration, treat the result as a current observation of the submitted public target.