VertarBot
VertarBot is the crawler that reads UK car dealers’ stock pages for Vertar.
It identifies itself as VertarBot/2.0 (+https://vertar.com/about/vertarbot).
- It makes one request at a time to any website, with a pause of at least two seconds between requests.
- It reads a dealer’s listing pages every few hours, and a car’s own page only when the car is new or its listing changed, plus about once a week to check it’s still for sale.
- It follows
robots.txt, includingCrawl-delay. To stop it, addUser-agent: VertarBotandDisallow: /, or email us.
Every car we list links back to the dealer’s own page. Questions or opt-outs: [email protected].