The RelatedBot is the automated indexing component of the |related| content recommendation system.

Its purpose is to identify and process publicly available articles on partner websites through the |related| widgets displayed on their pages. The system indexes only those URLs where the widget is actually present.

In rare cases – when a website cannot technically display the widget – content is retrieved via an RSS feed. In such cases, only the URLs listed in the feed are processed.

Operation

• RelatedBot does not perform general web crawling.
• Indexing occurs only on URLs where the widget is present. Therefore, in certain cases it may access non-public but publicly reachable pages (without login).
• The system respects meta tag settings.
• It does not collect personal data, cookies, or form submissions.
• Requests are made at a limited rate and do not cause extra server load.

Identification

RelatedBot uses the following User-Agent identifier:

Mozilla/5.0 (compatible; RelatedBot/1.0; +https://related.hu/bot)

Exclusion

If you wish to prevent specific pages from being included in the recommendation system, you can use one of the following meta tags on the respective page:

<meta name="robots" content="noindex">
<meta name="relatedbot" content="noindex">

For more details about exclusion and deletion options, please see this article.

Found an issue?

If you notice that our system is indexing a page incorrectly, or accessing a website you no longer use with the |related| service, please contact us at [email protected].