Hello! This page explains what our crawler does and how to control it.
Hunter Logos is a free project anyone can use to display company logos in their product or website They can be used in CRMs, sign up forms, lists of companies, etc.). This free API helps make the web easier to parse and more pleasant to look at. It's our small contribution to making the web nicer.
To use it yourself, visit https://hunter.io/logos.
When we fetch logos, our crawler identifies itself as:
User-Agent: HunterLogosRobot/1.0 (+https://logos.hunter.io/robot)
We only fetch what we need to find a logo. Typically, that means:
https://YOURDOMAIN/robots.txthttps://YOURDOMAIN/ (your homepage)We do not crawl your whole site.
We crawl slowly and infrequently. In practice:
We respect robots.txt. If you don't want us to fetch anything from your site,
you can block our user agent:
User-agent: HunterLogosRobot Disallow: /
(Blocking all bots with User-agent: * + Disallow: / also blocks us.)
If you notice any bugs or have any questions, please let us know at contact@hunter.io.