This is the web crawler behind this search engine. It visits public pages to build a search index, the same way any other search engine's crawler does.
Every request it makes identifies itself with the User-Agent
Linkhub-crawler/1.0.
It respects robots.txt. To block it from part or all of
your site, add a rule for Linkhub-crawler, e.g.:
User-agent: Linkhub-crawler
Disallow: /If it's causing trouble for your site, crawling too aggressively, or you'd like a page removed, please reach out: admin@linkhub.dk.