Directory / agent-data-tools.revmesh2074.workers.dev
agent-data-tools.revmesh2074.workers.dev/robots working
https://agent-data-tools.revmesh2074.workers.dev/robots
Before you pay
- Advice
ok- Protocol
- x402 · http · GET
- Asks for
- 0.01 USDC on base
- Catalogue says
- 0.01 USDC
- Pays to
0x6293ded2684F1A76E26bEcef87c137D691Fbf800- Last check
- 22 h ago · 11 ms · HTTP 402
- Detail
- 402 with valid terms
Reliability
- Uptime 24 h
- 100.0%
- Uptime 7 days
- 100.0% (7 checks)
- Uptime 30 days
- 100.0% (signed daily)
- Paid calls 30 d
- 4 from 2 payers (x402 Bazaar)
- Listed by
- x402-bazaar
- First seen
- 2026-11-05 22:10 UTC
Description (from the catalogue, unverified)
Robots.txt check. Read a site's robots.txt and say whether a crawler may fetch a URL, by RFC 9309 rules as Google applies them (most specific user-agent group, longest matching Allow or Disallow, Allow on ties, * and $ wildcards). Returns the deciding rule, the group used, the rule count and the sitemaps listed. A missing robots.txt (4xx) allows everything. Use before scraping or crawling a site politely; for the sitemap URLs themselves use sitemap.
Signed history
| When | Statement | Log index |
|---|---|---|
| 2026-11-13 00:17 UTC | 5b752662…141d7f daily summary | 641437 |
| 2026-11-12 05:50 UTC | 2c4b472b…8b7618 status change | 571128 |
| 2026-11-10 00:19 UTC | d8ed0ae3…bf83bb daily summary | 399999 |
| 2026-11-09 00:17 UTC | 220db96c…0631c6 daily summary | 308538 |
| 2026-11-08 16:54 UTC | 55f14918…552985 status change | 242206 |
| 2026-11-07 00:18 UTC | 39a7e3ac…3918fe daily summary | 151620 |
| 2026-11-06 00:14 UTC | 12344e58…87c7d4 daily summary | 58608 |
Verify any statement offline: how.
For agents and owners
JSON: https://alalanet.com/v1/services/wywdc4xd7o75bpa2
Badge:
<a href="https://alalanet.com/s/wywdc4xd7o75bpa2"><img src="https://alalanet.com/badge/wywdc4xd7o75bpa2" alt="alalanet status"></a>Own this service? Claim it (free). Do not want it checked? See the prober page.