Headless browsers run a full browser engine without a screen. They are used for testing, rendering and scraping sites that need JavaScript.
Most scrapers that evade simple checks run in a headless browser, so client side detection matters more than the user agent here.
Of the 4 headless browsers with a stated policy, 0 follow robots.txt and 4 do not. Bots that ignore robots.txt can only be stopped by user agent or IP at your server, CDN or WAF.
Transfon Security identifies every bot in this directory in your traffic reports, flags fake search crawlers, and lets you allow, block or rate limit each bot.
Privacy, security, monetization, and performance insights. A few emails a month, no spam.
