What is HeadlessChrome?

HeadlessChrome is a bot in the headless browsers category, operated by Google.

Bot name
HeadlessChrome
Operator
Google
Type
Bot
Respects robots.txt
No
robots.txt user agent
HeadlessChrome
Main crawl location
United States (87%)
Signs requests (Web Bot Auth)
No

About HeadlessChrome

HeadlessChrome is the user agent Chrome sends when it runs in headless mode, without a visible window. Testing tools, page renderers and scrapers use it to load pages that need JavaScript.

Headless browsers run a full browser engine without a screen. They are used for testing, rendering and scraping sites that need JavaScript. Most scrapers that evade simple checks run in a headless browser, so client side detection matters more than the user agent here.

HeadlessChrome user agent

Sample user agent strings sent by HeadlessChrome:

  • Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/153.0.8010.12 Safari/537.36
  • Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/151.0.7922.34 Safari/537.36
  • Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/112.0.5615.49 Safari/537.36
  • Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/156.0.0.0 Safari/537.36
  • Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/155.0.0.0 Safari/537.36
  • Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/136.0.7103.25 Safari/537.36

Anyone can copy a user agent string. Transfon Security also checks where each request comes from, so a request that claims to be a known crawler but comes from the wrong network is treated as a fake, not as the real crawler.

Where HeadlessChrome crawls from

Share of HeadlessChrome requests to sites protected by Transfon Security in the 7 days to 27 October 2026.

Countries

United States87%
Turkey7%
China1%

Networks

Amazon.com, Inc. AS1461858%
Amazon.com, Inc. AS1650926%
Equinix (EMEA) Acquisition Enterprises B.V. AS158307%
Google LLC AS3969821%

How to block HeadlessChrome in robots.txt

HeadlessChrome does not read robots.txt. The rules below record your policy, but they will not stop it. To actually block HeadlessChrome, use a WAF rule as shown in the next section.

Block HeadlessChrome from the whole site

User-agent: HeadlessChrome
Disallow: /

Block HeadlessChrome from some folders

User-agent: HeadlessChrome
Disallow: /private/
Disallow: /search

Allow HeadlessChrome only on some folders

User-agent: HeadlessChrome
Allow: /blog/
Disallow: /

Allow HeadlessChrome and block other crawlers

User-agent: HeadlessChrome
Allow: /

User-agent: *
Disallow: /

Things to know

  • Each host needs its own file. Rules on www.example.com do not apply to shop.example.com.
  • A crawler obeys only the most specific group that names it. Once you add a User-agent: HeadlessChrome group, HeadlessChrome ignores your User-agent: * rules, so repeat any that should still apply.
  • The user agent name is not case sensitive. We take HeadlessChrome from HeadlessChrome's user agent; check Google's own documentation if it publishes a different robots.txt name.
  • Crawlers cache robots.txt, so changes can take up to a day to apply. Disallow stops crawling but does not remove pages already collected or indexed.

Limits of robots.txt

  1. HeadlessChrome may not follow the rule. HeadlessChrome does not read robots.txt at all, so a rule for it has no effect.
  2. The user agent could be fake. robots.txt matches the name a request claims. Anyone can send HeadlessChrome's user agent, and a scraper using it is not bound by Google's robots.txt policy. Only checking the source network tells a real HeadlessChrome from a fake one.

Treat robots.txt as a statement of your policy, and enforce it where requests arrive.

Who should care about HeadlessChrome

Publishers

HeadlessChrome is a common tool for scrapers. Real readers never use it, so block it on article pages and keep it out of ad requests.

Transfon Security for publishers

Media buyers and advertisers

Traffic from HeadlessChrome on landing pages is automated. It inflates clicks and visits, so filter it from campaign data and raise it with the ad network.

Transfon Security for advertisers

E-commerce

Price scrapers, stock checkers and card testing scripts often run on HeadlessChrome. Block it on product, cart and checkout pages.

Transfon Security for e-commerce

SaaS and APIs

HeadlessChrome is normal for API clients but not for your web app. Allow it only on API routes that need a key, and rate limit it there.

Transfon Security for SaaS and APIs

Block HeadlessChrome with Transfon Security

HeadlessChrome ignores robots.txt, so it has to be stopped when the request arrives. Transfon Security shows HeadlessChrome by name in the Top Bots report, so you can see how often it visits and which pages it reads. Block or allow it with a User Agent rule, or slow it down with a rate limit decision rule.

Transfon Security

Control HeadlessChrome on your site

  • See every HeadlessChrome visit and the pages it reads
  • Spot bots that fake trusted crawler names
  • Block, allow or rate limit HeadlessChrome with one rule
  • Works with or without robots.txt support

Frequently asked questions

What is HeadlessChrome?

HeadlessChrome is a bot in the headless browsers category, operated by Google. HeadlessChrome is the user agent Chrome sends when it runs in headless mode, without a visible window. Testing tools, page renderers and scrapers use it to load pages that need JavaScript.

Does HeadlessChrome respect robots.txt?

No. HeadlessChrome does not follow robots.txt, so a Disallow rule will not stop it. Block it by user agent at your server, CDN or WAF instead.

How do I block HeadlessChrome?

Block requests whose user agent contains "HeadlessChrome" at your server, CDN or WAF, because HeadlessChrome does not read robots.txt. In Transfon Security that is one User Agent block rule.

Is robots.txt enough to block HeadlessChrome?

No. robots.txt is voluntary, so HeadlessChrome may not follow it, and anyone can send HeadlessChrome's user agent, so a fake HeadlessChrome ignores your rule. Enforce the rule where requests arrive, at your WAF, and check the source network.

Should I block HeadlessChrome?

Headless browsers run a full browser engine without a screen. They are used for testing, rendering and scraping sites that need JavaScript. Most scrapers that evade simple checks run in a headless browser, so client side detection matters more than the user agent here.

Related bots

All headless browsers · Full bot directory

See Which Bots Visit Your Site

Transfon Security identifies every bot in this directory in your traffic reports, flags fake search crawlers, and lets you allow, block or rate limit each bot.

Stay ahead of the publishing curve

Privacy, security, monetization, and performance insights. A few emails a month, no spam.

By subscribing you agree to our privacy policy.