Directory / web.openverbs.com
web.openverbs.com/v1/robots working
https://web.openverbs.com/v1/robots
Before you pay
- Advice
ok- Protocol
- x402 · http · POST
- Asks for
- 0.006 USDC on base
- Catalogue says
- 0.006 USDC
- Pays to
0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A- Last check
- 19 min ago · 400 ms · HTTP 402
- Detail
- 402 with valid terms
Reliability
- Uptime 24 h
- 100.0%
- Uptime 7 days
- 100.0% (12 checks)
- Uptime 30 days
- 100.0% (signed daily)
- Paid calls 30 d
- 6 from 5 payers (x402 Bazaar)
- Listed by
- x402-bazaar
- First seen
- 2026-11-05 22:09 UTC
Description (from the catalogue, unverified)
Fetch a site's robots.txt and evaluate crawl permissions: given a URL (plus optional extra paths) and a user-agent, return whether each path is allowed or disallowed, the rule that matched, the user-agent group, the crawl-delay and any declared sitemaps. Implements the Robots Exclusion Protocol (RFC 9309) with longest-match-wins, Allow-over-Disallow tie-breaking and * / $ wildcards.
Signed history
| When | Statement | Log index |
|---|---|---|
| 2026-11-06 00:12 UTC | 729ff04a…56e016 daily summary | 39847 |
Verify any statement offline: how.
For agents and owners
JSON: https://alalanet.com/v1/services/nccteunvkjzypbfy
Badge:
<a href="https://alalanet.com/s/nccteunvkjzypbfy"><img src="https://alalanet.com/badge/nccteunvkjzypbfy" alt="alalanet status"></a>Own this service? Get verified. Do not want it checked? See the prober page.