Some cookies are necessary for SecondDoor to work. Others are optional, and tell us how the site and the product are used. Accept all, or decline all optional ones. Without a selection, nothing optional is set. More in our cookie policy.
Basic scanPublisher · Blog property/Scanned 8 Sept 2026, 19:26 UTC/1 browser pass
A publisher that welcomes AI crawlers in robots.txt but blocks them at the network edge, creating a gap between stated and actual policy.
Older scan
This report was produced by an earlier version of the scanner. Some findings may not match how the site is measured now.
Ordered by what the evidence says is costing most. Written for whoever owns the site, not whoever builds it.
6 pieces of evidence and the recommended fix are recorded for this issue, with notes written from your own pages.
This is a basic scan, so it raises the three issues the deterministic evidence supports. A deep scan sends two agents over the site and raises up to ten, each with cited evidence and the screens to prove it.
A deep scan walks the site with two agents, records cited evidence and screenshots, and keeps a mission timeline. It comes with the paid plans. Making an account costs nothing and gives you one basic scan a month.
The score and the audit trail stay public. The evidence and the files open with an account.
The audit trail. Not measured is not zero.
Machine discoverability
robots.txt explicitly names claudebot and allows 14 AI crawlers including gptbot, perplexitybot and oai-searchbot. A sitemap index declares 590 URLs.
4 observations recorded.
Programmatic onboarding
Not scored: a blog is not measured on this.
Pricing legibility
Not scored: a blog is not measured on this.
Agent-aware instrumentation
robots.txt names claudebot explicitly and allows it alongside 13 other AI crawlers. No MCP manifest exists at the 3 standard paths checked. No ai-plugin.json or llms.txt file is published.
4 observations recorded.
Machine-fetchable trust
The homepage answers automated requests with HTTP 200 and serves structured data. No security.txt file is published at /.well-known/security.txt. No Web Bot Auth signals are present.
2 observations recorded.
Commercial rails
Not scored: a blog is not measured on this.
Steps to first value
Could not verify: this scan did not reach the pages that would show it.
Required fields
Could not verify: this scan did not reach the pages that would show it.
Verification walls
Could not verify: this scan did not reach the pages that would show it.
Error recovery
Could not verify: this scan did not reach the pages that would show it.
It got through 4 of 12.
The agent got through 4 of 12 requests, but could not ask whether the door opens for requests carrying the AI crawlers' names. It came up empty on 7 other checks too. That is enough missing for a machine to give up before it reaches a purchase.