Bot Detection: How Sites Spot Headless Browsers

Bot detection is the set of checks a website runs to decide whether a visitor is a person using a browser or an automated program, so it can allow, challenge or block the request.

For screenshots this matters because a headless browser is, honestly, a bot. When detection fires, you get a picture of a challenge or an "Access denied" page instead of the site. This entry is part of the screenshot glossary.

What it means in plain terms

A site cannot see who sits behind a request. It sees evidence: headers, the network connection, what JavaScript finds in the browser, and how the visitor behaves. Bot detection adds that evidence into a score. A low-risk score passes. A doubtful score gets a CAPTCHA or a JavaScript challenge. A high-risk score gets blocked.

The layers of evidence

 request arrives
     │
     ├─ 1. network   IP reputation, data-centre ranges, TLS and HTTP/2 fingerprint
     ├─ 2. headers   User-Agent, sec-ch-ua client hints, Accept-Language, header order
     ├─ 3. browser   JavaScript probes: navigator.webdriver, plugins, screen, WebGL
     └─ 4. behaviour mouse movement, timing, cookies and history from earlier visits
     │
     ▼
 score → allow │ challenge (CAPTCHA) │ block (403 / "Access denied")

Layers 1 and 4 are outside the control of a script. A fresh headless session has no cookies, no history and often runs from a cloud IP range. That is why patches to layers 2 and 3 alone rarely give stable results.

What a default headless Chrome exposes: a test

We loaded https://example.com and read the signals that page JavaScript and the server can see:

Tested on 2026-10-15 with Node 22.18.0, [email protected] (Chrome 148) and [email protected] (Chromium 156) on macOS.

Two results stand out. Every setup reported navigator.webdriver === true, which the WebDriver standard requires for automated browsers. And the word HeadlessChrome reached the server in the User-Agent in all three, so even a site with no JavaScript checks can see it.

How to check it yourself

Puppeteer

import puppeteer from 'puppeteer';

const browser = await puppeteer.launch();
try {
  const page = await browser.newPage();
  await page.goto('https://example.com/', { waitUntil: 'load' });
  const signals = await page.evaluate(() => ({
    webdriver: navigator.webdriver,
    userAgent: navigator.userAgent,
    uaBrands: navigator.userAgentData?.brands.map((b) => b.brand) ?? null,
    languages: navigator.languages,
    plugins: navigator.plugins.length,
    screen: `${screen.width}x${screen.height}`,
  }));
  console.log(JSON.stringify(signals, null, 2));
} finally {
  await browser.close();
}

Playwright

This version also prints the headers the server actually received, which is what a network-level bot check sees:

import { chromium } from 'playwright';

const browser = await chromium.launch();
try {
  const page = await browser.newPage();
  const response = await page.goto('https://example.com/');
  const sent = await response.request().allHeaders();
  const webdriver = await page.evaluate(() => navigator.webdriver);
  console.log({ webdriver, 'user-agent': sent['user-agent'], 'sec-ch-ua': sent['sec-ch-ua'] });
} finally {
  await browser.close();
}

RenderScreenshot

Our renderer runs on Cloudflare's browser rendering service, and we chose not to fight detection. There is no stealth setting. Cloudflare adds headers to every request from that service that mark it as automated, and changing browser.user_agent does not remove them. The bot protection guide explains this policy and cites Cloudflare's documentation.

What the API gives you instead is a clean failure. Wait for an element that only the real page has, so a challenge page times out and uses no credit:

curl -X POST https://api.renderscreenshot.com/v1/screenshot \
  -H "Authorization: Bearer $RS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://shop.example.com/product/42",
    "wait": { "for_selector": "[data-product-id]", "timeout": 20 }
  }' \
  --output product.png

If you own the site, let the renderer through with a shared-secret header in network.headers or a WAF rule. The Cloudflare guide shows the rule. The API request on this page was not run for it.

Common mistakes

• Changing only the User-Agent. navigator.webdriver, client hints and the network fingerprint still say "automated".
• Retrying in a loop. Each retry is a new session with the same signals. It usually meets the same block, and many sites raise the score after repeated attempts.
• Trusting a pass on your laptop. A residential IP scores better than a cloud server. The same script often fails in production.
• Saving whatever comes back. A challenge page returns HTTP 200 on many sites. Check the content before you store the image.
• Allowlisting by IP for your own site. Rendering services rarely have fixed addresses. Match a header or a signature instead.

Related terms

• CAPTCHA: the challenge a site shows when detection is unsure.
• User agent: the header most scripts change first.
• Headless browser: why these signals exist.
• Browser context: why every capture starts with no history.

To see how your own site treats an automated browser, get a free API key and capture a page with the wait condition above.