What is Playwright?

Playwright is a bot in the headless browsers category, operated by Microsoft.

Bot name
Playwright
Operator
Microsoft
Type
Bot
Respects robots.txt
No
robots.txt user agent
Playwright
Signs requests (Web Bot Auth)
No

About Playwright

Playwright is an open source browser automation framework from Microsoft. It drives Chromium, Firefox and WebKit for end to end tests, and is also widely used for scraping.

Headless browsers run a full browser engine without a screen. They are used for testing, rendering and scraping sites that need JavaScript. Most scrapers that evade simple checks run in a headless browser, so client side detection matters more than the user agent here.

Playwright user agent

Anyone can copy a user agent string. Transfon Security also checks where each request comes from, so a request that claims to be a known crawler but comes from the wrong network is treated as a fake, not as the real crawler.

How to block Playwright in robots.txt

Playwright does not read robots.txt. The rules below record your policy, but they will not stop it. To actually block Playwright, use a WAF rule as shown in the next section.

Block Playwright from the whole site

User-agent: Playwright
Disallow: /

Block Playwright from some folders

User-agent: Playwright
Disallow: /private/
Disallow: /search

Allow Playwright only on some folders

User-agent: Playwright
Allow: /blog/
Disallow: /

Allow Playwright and block other crawlers

User-agent: Playwright
Allow: /

User-agent: *
Disallow: /

Things to know

  • Each host needs its own file. Rules on www.example.com do not apply to shop.example.com.
  • A crawler obeys only the most specific group that names it. Once you add a User-agent: Playwright group, Playwright ignores your User-agent: * rules, so repeat any that should still apply.
  • The user agent name is not case sensitive. We take Playwright from Playwright's user agent; check Microsoft's own documentation if it publishes a different robots.txt name.
  • Crawlers cache robots.txt, so changes can take up to a day to apply. Disallow stops crawling but does not remove pages already collected or indexed.

Limits of robots.txt

  1. Playwright may not follow the rule. Playwright does not read robots.txt at all, so a rule for it has no effect.
  2. The user agent could be fake. robots.txt matches the name a request claims. Anyone can send Playwright's user agent, and a scraper using it is not bound by Microsoft's robots.txt policy. Only checking the source network tells a real Playwright from a fake one.

Treat robots.txt as a statement of your policy, and enforce it where requests arrive.

Who should care about Playwright

Publishers

Playwright is a common tool for scrapers. Real readers never use it, so block it on article pages and keep it out of ad requests.

Transfon Security for publishers

Media buyers and advertisers

Traffic from Playwright on landing pages is automated. It inflates clicks and visits, so filter it from campaign data and raise it with the ad network.

Transfon Security for advertisers

E-commerce

Price scrapers, stock checkers and card testing scripts often run on Playwright. Block it on product, cart and checkout pages.

Transfon Security for e-commerce

SaaS and APIs

Playwright is normal for API clients but not for your web app. Allow it only on API routes that need a key, and rate limit it there.

Transfon Security for SaaS and APIs

Block Playwright with Transfon Security

Playwright ignores robots.txt, so it has to be stopped when the request arrives. Transfon Security shows Playwright by name in the Top Bots report, so you can see how often it visits and which pages it reads. Block or allow it with a User Agent rule, or slow it down with a rate limit decision rule.

Transfon Security

Control Playwright on your site

  • See every Playwright visit and the pages it reads
  • Spot bots that fake trusted crawler names
  • Block, allow or rate limit Playwright with one rule
  • Works with or without robots.txt support

Frequently asked questions

What is Playwright?

Playwright is a bot in the headless browsers category, operated by Microsoft. Playwright is an open source browser automation framework from Microsoft. It drives Chromium, Firefox and WebKit for end to end tests, and is also widely used for scraping.

Does Playwright respect robots.txt?

No. Playwright does not follow robots.txt, so a Disallow rule will not stop it. Block it by user agent at your server, CDN or WAF instead.

How do I block Playwright?

Block requests whose user agent contains "Playwright" at your server, CDN or WAF, because Playwright does not read robots.txt. In Transfon Security that is one User Agent block rule.

Is robots.txt enough to block Playwright?

No. robots.txt is voluntary, so Playwright may not follow it, and anyone can send Playwright's user agent, so a fake Playwright ignores your rule. Enforce the rule where requests arrive, at your WAF, and check the source network.

Should I block Playwright?

Headless browsers run a full browser engine without a screen. They are used for testing, rendering and scraping sites that need JavaScript. Most scrapers that evade simple checks run in a headless browser, so client side detection matters more than the user agent here.

Related bots

All headless browsers · Full bot directory

See Which Bots Visit Your Site

Transfon Security identifies every bot in this directory in your traffic reports, flags fake search crawlers, and lets you allow, block or rate limit each bot.

Stay ahead of the publishing curve

Privacy, security, monetization, and performance insights. A few emails a month, no spam.

By subscribing you agree to our privacy policy.