Yandex crawlers: what each bot does
Yandex runs 5 bots that we document: YandexBot, YandexImages, YandexVideo, YandexAdditional, YaDirectFetcher. They have different jobs (search engine crawler, special-purpose crawler), so a robots.txt rule for one does not apply to the others.
| Bot | Category | robots.txt token | Honors robots.txt? | What blocking does |
|---|---|---|---|---|
| YandexBot | Search engine crawler | YandexBot | Varies by Yandex robot | Your pages can drop out of Yandex Search. Yandex's other robots, for example for images or video, have their own names and rules. |
| YandexImages | Search engine crawler | YandexImages | Yes, per Yandex | Your images can drop out of Yandex Images. YandexBot, Yandex's main indexing robot, has its own token. |
| YandexVideo | Search engine crawler | YandexVideo | Yes, per Yandex | Your videos can drop out of Yandex video search. YandexVideoParser, which Yandex also lists for video search, is a separate robot with its own name. |
| YandexAdditional | Special-purpose crawler | YandexAdditional | Yes, when named (ignores *), per Yandex | Per Yandex, a rule for it keeps page content out of Search with Yandex AI responses. It makes no indexing requests itself; the Yandex index is built by YandexBot. |
| YaDirectFetcher | Special-purpose crawler | YaDirectFetcher | No, per Yandex | Per Yandex, a robots.txt rule changes nothing, because YaDirectFetcher does not use the robots.txt file and ignores the directives set for it. |
158 requests from Yandex bots on this site since 19 Oct. Bot traffic data