verisourcemcp.verisource.dev/mcp
Crawler SpecificationREAD-ONLY · ROBOTS.TXT OBEYEDGET & HEAD ONLY

VerisourceScan

If this crawler name showed up in your Cloudflare, Vercel, or Nginx logs, someone ran an on-demand site health and AI readiness check on your domain—either from our home-page scanner or via npx @verisource/cli scan on their own machine. This page documents every request the scanner sends and how to block it in 2 lines.

01 · User-Agent Identification

How it identifies itself

Mozilla/5.0 (compatible; VerisourceScan/1.0; +https://verisource.dev/scanner)

Every automated HTTP request carries this exact User-Agent header. It never spoofs consumer browsers or third-party search crawlers.

02 · Scope & Crawl Limits

What VerisourceScan checks

  • Fetches /robots.txt first and respects both Disallow paths and Crawl-delay directives.
  • Inspects structured AI retrieval endpoints (/llms.txt, /agents.md, JSON-LD entity/FAQ schemas, and Shopify /api/ucp/mcp).
  • Follows same-origin HTML links up to 5 pages (1 link deep) from the unauthenticated home-page box, or up to 20 pages (2 links deep) inside verified workspaces and CLI runs.
  • Verifies linked pages, scripts, stylesheets, images, and #anchor targets using HEAD requests where supported, flagging HTTP 4xx/5xx responses and dead links.
03 · Hard Safety Boundaries

What it never does

  • Never sends POST, PUT, PATCH, or DELETE mutations during site health crawls (GET and HEAD only).
  • Never submits forms, clicks destructive buttons, or transmits session cookies or credentials.
  • Never follows paths matching state-changing actions: logout, signout, delete, remove, unsubscribe, or cancel.
  • Refuses all loopback, RFC 1918 private networks, cloud metadata IPs (169.254.169.254), and non-standard ports on every redirect hop.
04 · Opt-Out Instructions

How to keep VerisourceScan out

Add these two lines to your domain’s root /robots.txt file to block all automated VerisourceScan checks immediately:

User-agent: VerisourceScan
Disallow: /
Explore All Features →Scan Your Own Site →