dp-discover (Digital Pigeon): what it is and how to allow or block it
Updated · checked against the operator's documentation on
dp-discover is a crawler of Digital Pigeon, an email-marketing agency, that checks whether a domain serves a live website, what platform it runs on and what broad category it is in. Digital Pigeon documents no robots.txt rule for it; it says to block it by user agent at your web server or WAF. Digital Pigeon documents no robots.txt token for it; the name in its user agent is dp-discover.
Seen dp-discover/1.0 in your logs?
dp-discover/1.0 (+https://godigitalpigeon.com/bot)The example string from Digital Pigeon's documentation. Version numbers change; the token dp-discover stays.
- What it is
- A special-purpose crawler run by Digital Pigeon.
- Honors robots.txt
- UnclearNot documented.
- Real or fake?
- No official checkThe user agent is only a claim, and Digital Pigeon publishes no IP list or DNS check we could find.
- If you block it
- Digital Pigeon says it stops reaching your site when you block its user agent at the web server or WAF, or when you email it to be added to its exclusion list.
Facts
- Operator
- Digital Pigeon
- Category
- Special-purpose crawler
- robots.txt token
dp-discoverDigital Pigeon documents no robots.txt token for it; this is the name in its user agent.
- Honors robots.txt?
- Not documented. Digital Pigeon's page does not mention robots.txt. It names two ways out: block the user agent at your web server or WAF, or email for an entry on its exclusion list. A robots.txt rule may not be read; use the server rule on this page.
- User-agent example
dp-discover/1.0 (+https://godigitalpigeon.com/bot)
- Official source
- Digital Pigeon: dp-discover
- Compared with the source
What it is used for
Per Digital Pigeon, it visits publicly reachable domains to support email-deliverability and marketing research: whether a domain serves a live website, the general technology and platform from public response headers and homepage markup, and a broad category. Requests are low-volume, typically a single homepage fetch, and it does not log in, submit forms or probe for vulnerabilities.
Not the crawler behind a search index. What blocking it changes is specific to its tool, so read the next section.
What happens if you block it
Digital Pigeon says it stops reaching your site when you block its user agent at the web server or WAF, or when you email it to be added to its exclusion list. Nothing about your search visibility changes.
Digital Pigeon's page publishes no IP list or reverse-DNS method, so the user agent alone cannot show that a request is genuine.
Is it really dp-discover?
Anyone can send dp-discover as a user agent, and Digital Pigeon publishes no IP list or reverse-DNS method that we could find, so a log line cannot prove a request is real. Check whether it fetched /robots.txt and whether the address belongs to the operator's network.
- We did not find an IP list or a reverse-DNS method in the official source we compared on 2026-11-09, so the user agent alone cannot show that a request is genuine. Check the operator's page for updates.
- Paste any user agent from your logs into the user-agent lookup, or follow the scripts in Fake Googlebots and PHP scanners.
How to allow or block dp-discover
Add one of these groups to the robots.txt at the root of your domain. A group that names the bot overrides a general User-agent: * group.
Digital Pigeon names no robots.txt token for dp-discover, so these groups use the name from its user agent and may have no effect. Digital Pigeon's page does not mention robots.txt. It names two ways out: block the user agent at your web server or WAF, or email for an entry on its exclusion list. A robots.txt rule may not be read; use the server rule on this page.
User-agent: dp-discover Allow: /
User-agent: dp-discover Disallow: /
Block dp-discover at the server
robots.txt asks; a server rule refuses. These answer 403 to any request whose user agent contains dp-discover (case-insensitive). They stop only bots that name themselves honestly, so keep the robots.txt group as well.
# server { ... }
if ($http_user_agent ~* "dp-discover") {
return 403;
}RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} dp-discover [NC]
RewriteRule .* - [F,L]Step by step, with a test command: Block AI bots in nginx by user agent · Block AI bots in Apache with .htaccess · Block AI crawlers on Cloudflare · Block AI bots on Vercel
Does your site allow dp-discover today? The checker evaluates your robots.txt for this and every other bot.Check my site
What we see on this site
No request identifying as dp-discover has been recorded in the last 30 days, so there is nothing to show yet. We never fill this in with estimates.
Counts come from this site's own server; they hold no IP addresses or URLs, and a user agent is only a claim. See the bot traffic data and the methodology.
Questions about dp-discover
What is dp-discover?
Digital Pigeon's crawler that checks domains for a live site, its technology and a broad category. Per Digital Pigeon, it visits publicly reachable domains to support email-deliverability and marketing research: whether a domain serves a live website, the general technology and platform from public response headers and homepage markup, and a broad category. Requests are low-volume, typically a single homepage fetch, and it does not log in, submit forms or probe for vulnerabilities.
Is dp-discover safe and legitimate?
The real dp-discover is a special-purpose crawler run by Digital Pigeon. A user-agent string is only a claim, though, so anyone can send it. We did not find an IP list or reverse-DNS method in the official source we compared on 2026-11-09, so the user agent alone cannot prove a request is real.
Does dp-discover obey robots.txt?
Not documented. Digital Pigeon's page does not mention robots.txt. It names two ways out: block the user agent at your web server or WAF, or email for an entry on its exclusion list. A robots.txt rule may not be read; use the server rule on this page.
How do I block dp-discover?
Add a group to the robots.txt at the root of your domain: User-agent: dp-discover followed by Disallow: /. A group that names the bot overrides a general User-agent: * group. Digital Pigeon documents nothing about robots.txt for this agent, so we cannot promise it is honored.
Will blocking dp-discover hurt my Google rankings or AI visibility?
This is not the crawler behind a search index, so blocking it is not a ranking decision. Digital Pigeon says it stops reaching your site when you block its user agent at the web server or WAF, or when you email it to be added to its exclusion list. Nothing about your search visibility changes.
What does "dp-discover/1.0 (+https://godigitalpigeon.com/bot)" mean in my logs?
It is the user-agent string in Digital Pigeon's documentation for dp-discover, a special-purpose crawler run by Digital Pigeon. The part that robots.txt rules match is the token dp-discover. The string is only a claim, because anyone can send it; the section "Is it really dp-discover?" above shows how to check a request.
Related bots
Other Special-purpose crawlers
- GoogleOther GoogleGoogle's generic crawler that product teams can use for their own purposes.
- Google-InspectionTool GoogleThe Google fetcher behind Search testing tools such as the Rich Result Test and URL Inspection.
- archive.org_bot Internet ArchiveReported to be the crawler the Internet Archive uses for Wayback Machine and Archive-It captures.
- WebMCPIndexBot WebMCP ShieldA survey crawler that measures how widely websites adopt WebMCP.
All Special-Purpose Crawlers · Bot directory · Which bots to allow or block · robots.txt generator · See a page as a bot