ToolRancherBot
Last updated: 7 August 2026
Crawler Identity
Every request ToolRancherBot makes carries these headers. The User-Agent below is the exact string to match on if you want to allow-list or block us.
User-Agent: ToolRancherBot/1.0 (+https://toolrancher.com/bot; hello@toolrancher.com) From: hello@toolrancher.com
We do not send browser fingerprint headers alongside that agent, and we do not disguise the crawler as a browser.
What It Does
The crawler reads public product, category, sitemap, price, stock, and product detail pages from selected UK tool retailers and brands. The data is used to help ToolRancher users compare products and retailer offers, and to link shoppers through to the retailer to buy.
What It Does Not Do
ToolRancherBot never accesses checkout, basket, customer account, or any page behind a login. It does not collect personal or customer data, and it does not attempt to work around blocks, rate limits, or bot protection.
Crawl Policy
ToolRancherBot reads and enforces robots.txt before fetching, honouring the rules for its own user agent and any Crawl-delay directive. We prefer sitemaps, product feeds, affiliate feeds, and public APIs over broad page crawling — if you can give us a feed, we would rather use it than crawl you at all.
Rate Limits
Default settings are 3 concurrent requests with a 2 second delay between them, applied per domain. We back off when a site returns rate-limit or blocking responses, and we will run to any slower limit or overnight window you ask for.
How To Block Us
Add this to your robots.txt and we will stop:
User-agent: ToolRancherBot Disallow: /
You can also block the user agent at your CDN or WAF. We will not attempt to circumvent either, and you do not need to tell us first.
Contact
For crawl concerns, allow-listing, rate limits, data correction, or feed/API access, email hello@toolrancher.com. We reply within one working day.
