NetAPI Bot (NetAPI): what it is and how to allow or block it
Updated · checked against the operator's documentation on
NetAPI Bot is the crawler NetAPI uses to collect public homepage content for its domain data and datasets. NetAPI says it fetches only the home page, at most twice a month, and publishes its four IP addresses and a reverse-DNS check. Block it with User-agent: NetAPI. Its robots.txt token is NetAPI.
Seen NetAPI/1.1 in your logs?
NetAPI/1.1 (+https://netapi.com/bot.html)The example string from NetAPI's documentation. Version numbers change; the token NetAPI stays.
- What it is
- A special-purpose crawler run by NetAPI.
- Honors robots.txt
- YesYes, per NetAPI.
- Real or fake?
- Can be verifiedThe user agent is only a claim. Check the requesting IP against NetAPI's published IP ranges.
- If you block it
- NetAPI stops fetching your home page for its domain datasets.
Facts
- Operator
- NetAPI
- Category
- Special-purpose crawler
- robots.txt token
NetAPI
- Honors robots.txt?
- Yes, per NetAPI. NetAPI says it respects robots.txt and matches the product token NetAPI (the name before the version), not the full user agent. A rule for it or for all bots stops the fetch; it also offers manual exclusion by contact.
- User-agent example
NetAPI/1.1 (+https://netapi.com/bot.html)
- Official source
- NetAPI: NetAPI Bot
- Published IP ranges
- https://netapi.com/bot-ips.json
- Compared with the source
What it is used for
Per NetAPI, it collects publicly available homepage content for domain research and for dataset enrichment in the NetAPI service. It requests only the main page, not sitemaps, blogs or product catalogs, and fetches each site at most twice per month.
Not the crawler behind a search index. What blocking it changes is specific to its tool, so read the next section.
What happens if you block it
NetAPI stops fetching your home page for its domain datasets. Nothing else changes: it is not a search engine and does not affect rankings.
NetAPI says genuine requests come only from its listed IPv4 addresses (the crawler does not use IPv6) and that a reverse lookup ends in .bot.netapi.com and resolves back to the same address. The list on its page has four addresses today; the machine-readable file can change, so compare with the current file.
Is it really NetAPI Bot?
Anyone can send NetAPI Bot as a user agent, so a log line proves nothing by itself. The way to tell is the requesting IP address.
- Compare the IP address with the NetAPI: published IP ranges. A request from outside those ranges is not the operator's crawler.
- Guide: robots.txt syntax: groups, wildcards and matching rules.
- Guide: Fake Googlebots and PHP scanners: what hit our new server in 48 hours.
- Paste any user agent from your logs into the user-agent lookup, or follow the scripts in Fake Googlebots and PHP scanners.
How to allow or block NetAPI Bot
Add one of these groups to the robots.txt at the root of your domain. A group that names the bot overrides a general User-agent: * group.
User-agent: NetAPI Allow: /
User-agent: NetAPI Disallow: /
Block NetAPI Bot at the server
robots.txt asks; a server rule refuses. These answer 403 to any request whose user agent contains netapi/ (case-insensitive). They stop only bots that name themselves honestly, so keep the robots.txt group as well.
# server { ... }
if ($http_user_agent ~* "netapi/") {
return 403;
}RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} netapi/ [NC]
RewriteRule .* - [F,L]Step by step, with a test command: Block AI bots in nginx by user agent · Block AI bots in Apache with .htaccess · Block AI crawlers on Cloudflare · Block AI bots on Vercel
Does your site allow NetAPI Bot today? The checker evaluates your robots.txt for this and every other bot.Check my site
What we see on this site
Live counter, last 30 days: 2 requests on 1 day identified as NetAPI Bot (latest: ). That is a small sample: read it as an example, not as a measurement of the web.
- Home page: 1
- robots.txt: 1
Counts come from this site's own server; they hold no IP addresses or URLs, and a user agent is only a claim. See the bot traffic data and the methodology.
Questions about NetAPI Bot
What is NetAPI Bot?
NetAPI's crawler that fetches the home page of a domain to enrich its domain datasets. Per NetAPI, it collects publicly available homepage content for domain research and for dataset enrichment in the NetAPI service. It requests only the main page, not sitemaps, blogs or product catalogs, and fetches each site at most twice per month.
Is NetAPI Bot safe and legitimate?
The real NetAPI Bot is a special-purpose crawler run by NetAPI. A user-agent string is only a claim, though, so anyone can send it. Compare the requesting IP address with the published IP ranges.
Does NetAPI Bot obey robots.txt?
Yes, per NetAPI. NetAPI says it respects robots.txt and matches the product token NetAPI (the name before the version), not the full user agent. A rule for it or for all bots stops the fetch; it also offers manual exclusion by contact.
How do I block NetAPI Bot?
Add a group to the robots.txt at the root of your domain: User-agent: NetAPI followed by Disallow: /. A group that names the bot overrides a general User-agent: * group.
Will blocking NetAPI Bot hurt my Google rankings or AI visibility?
This is not the crawler behind a search index, so blocking it is not a ranking decision. NetAPI stops fetching your home page for its domain datasets. Nothing else changes: it is not a search engine and does not affect rankings.
What does "NetAPI/1.1 (+https://netapi.com/bot.html)" mean in my logs?
It is the user-agent string in NetAPI's documentation for NetAPI Bot, a special-purpose crawler run by NetAPI. The part that robots.txt rules match is the token NetAPI. The string is only a claim, because anyone can send it; the section "Is it really NetAPI Bot?" above shows how to check a request.
Related bots
Other Special-purpose crawlers
- GoogleOther GoogleGoogle's generic crawler that product teams can use for their own purposes.
- Google-InspectionTool GoogleThe Google fetcher behind Search testing tools such as the Rich Result Test and URL Inspection.
- archive.org_bot Internet ArchiveReported to be the crawler the Internet Archive uses for Wayback Machine and Archive-It captures.
- WebMCPIndexBot WebMCP ShieldA survey crawler that measures how widely websites adopt WebMCP.
All Special-Purpose Crawlers · Bot directory · Which bots to allow or block · robots.txt generator · See a page as a bot