Blocking Cookie Banners Automatically: How It Fails

When you set block.cookie_banners: true on a RenderScreenshot request, the renderer does two things, in this order: it clicks the first "accept" button it recognizes, and if it finds none, it hides known banner containers. That is all. This post shows the actual code, the reasons for that order, and a reproducible test that shows where it fails. It is part of our engineering notes on building RenderScreenshot; the user-facing options are in the cookie banner guide.

Code excerpts are from our rendering Worker as of October 15, 2026. Fixture results from a run on the same date.

Where the handler runs

Blocking happens inside the Cloudflare Worker that drives the browser, after navigation and the wait, and before the capture:

goto(url) → wait (until / delay) → hide ad containers → cookie banner handler → page options (click, scripts, styles, hide, remove) → capture → upload to R2

Two consequences follow from that position:

1. The handler runs once. A banner that a consent script injects after this point is not touched.
2. Your own page options run after it. If the handler misses a banner, a page.click or page.remove in the same request still gets a chance. That is the escape hatch for every failure below.

Step 1: click an accept button

The first step loops over a list of selectors and clicks the first element that exists. This is a shortened excerpt; the full list also has a second Cookiebot button id, a [data-cc="accept"] attribute and the text selectors discussed below.

const COOKIE_ACCEPT_SELECTORS = [
  '[data-testid*="accept" i]',
  '[data-testid*="agree" i]',
  'button[id*="accept" i]',
  'button[id*="agree" i]',
  'button[class*="accept" i]',
  'button[class*="agree" i]',
  'a[id*="accept" i]',
  'a[class*="accept" i]',
  '#onetrust-accept-btn-handler',
  '#CybotCookiebotDialogBodyLevelButtonLevelOptinAllowAll',
  '.cc-accept',
  '.cc-allow',
  '.cc-dismiss',
  // ...
];

for (const selector of COOKIE_ACCEPT_SELECTORS) {
  const button = await page.$(selector);
  if (button) {
    await button.click();
    await new Promise((resolve) => setTimeout(resolve, 300)); // let the banner animate out
    return;
  }
}

Why click instead of hide? Because a click is the only action that tells the consent platform the visitor answered. The platform then removes its own banner, its backdrop and any scroll lock it put on <body>. Hiding leaves all of that state in place.

The cost of clicking "accept" is that the site may now load the analytics and marketing tags it was holding back. That is why the guide recommends combining cookie_banners with block.trackers.

A selector list we carried but could not use

Our list also contains Playwright-style text selectors such as button:has-text("Accept all"). The renderer runs Cloudflare's fork of Puppeteer, whose page.$() takes plain CSS, so the loop skips every entry that contains :has-text. They are dead weight today. The consequence shows up in fixture 3 below: a button whose only clue is its visible text is invisible to step 1.

Step 2: hide known banner containers

If no accept button matched, the handler sets display: none on every element that matches a list of container selectors: common ids and classes (#cookie-banner, .cookie-consent, .cc-window), the root elements of the large consent platforms (#onetrust-consent-sdk, #CybotCookiebotDialog, #usercentrics-root, #qc-cmp2-container), and two attribute patterns:

'[aria-label*="cookie" i]',
'[aria-label*="consent" i]',

The attribute patterns catch accessible custom banners. They also catch a footer link labelled "Cookie settings", which then disappears from the capture. We accept that collateral damage because a hidden footer link costs less than a banner across the screenshot.

display: none was chosen over visibility: hidden because it also gives the space back: a banner that pushes content down leaves no gap.

Eight fixtures, and what happened

A unit test cannot tell you how a consent banner behaves, and the large consent platforms change their markup without notice. So we built small HTML fixtures, one per banner pattern we have seen in captures, and ran the same two-step handler against each in headless Chrome.

Two of eight are failures we would expect any selector-based handler to have (iframes, late banners). The other three are design limits of this handler that are worth knowing about before you rely on it.

Fixture 3: no clue but the text

<div class="privacy-modal" role="dialog" aria-label="Your privacy choices">
  <p>We value your privacy</p>
  <button>Accept all</button>
</div>

Nothing in the markup says "accept" or "cookie" to a CSS selector. Step 1 finds nothing, and step 2 finds nothing either, because the label says "privacy", not "cookie" or "consent". Fix it in the request with your own selector:

{
  "url": "https://example.com",
  "block": { "cookie_banners": true },
  "page": { "remove": [".privacy-modal"] }
}

Fixture 5: the banner is in an iframe

Some consent platforms render the banner inside an <iframe> in a fixed container. page.$() searches the top document only, so the button inside the frame is never found, and the container's id is not on our list. Remove the container in the top document instead; the frame goes with it:

{ "page": { "remove": ["[id^='sp_message_container']"] } }

Removing the container does not register consent. If the page also locks scrolling, add a style: "styles": ["html, body { overflow: auto !important; }"].

Fixture 6: the backdrop is a separate element

Many banners are two elements: the banner and a full-screen semi-transparent backdrop, plus overflow: hidden on <body>. Step 2 hides the banner, and the capture shows the page dimmed under an empty overlay. In a full-page capture, the scroll lock can also cut the page at the viewport height. When the click step works, the consent platform cleans up both. When the hide step runs, it does not. The fix is the same as fixture 5: page.remove the backdrop selector and restore overflow.

Fixture 7: the wrong button

<button class="accept-invite">Accept invitation</button>
...
<div class="cookie-banner">Cookies <button>Allow</button></div>

button[class*="accept" i] matches the invitation button first, so the handler clicks it, waits 300 ms and returns. The banner stays, and whatever that button does has happened. On a real page that could open a modal or navigate. This is the strongest argument against a long list of loose substring selectors, and the reason the platform-specific ids should be tried before the generic patterns. In our current list the generic patterns come first; reordering them is on our list of fixes.

If you hit this, turn block.cookie_banners off for that URL and click the right button yourself with page.click, which runs in the same place in the pipeline:

{ "page": { "click": ".cookie-banner button", "click_required": false } }

Fixture 8: the banner arrives late

Some consent scripts wait for idle time or a timer before they inject the banner. If the handler runs first, there is nothing to click or hide, and the banner appears in time for the capture. Make the wait longer than the banner's delay so the banner exists when the handler runs:

{ "wait": { "until": "networkidle", "delay": 2000 } }

wait.delay is capped at 10 seconds. For banners that only appear on scroll or interaction, no wait helps; block the consent script with block.requests instead (the guide lists the script hosts of the common platforms).

Run the fixtures yourself

The harness below contains the same two-step logic in shortened form and the eight fixtures. Use it to test a banner pattern from a site you capture often: copy the banner markup into a new fixture and see which step catches it.

// cookie-fixtures.mjs: run the two-step consent handler (click "accept", else hide)
// against synthetic consent-banner fixtures and report which ones it clears.
// Node 22+, puppeteer@24.  Usage: node cookie-fixtures.mjs
import puppeteer from 'puppeteer';

// --- The handler under test (same logic and order as our Worker, shortened lists) ---
const ACCEPT_SELECTORS = [
  '[data-testid*="accept" i]', 'button[id*="accept" i]', 'button[class*="accept" i]',
  'a[id*="accept" i]', 'a[class*="accept" i]',
  '#onetrust-accept-btn-handler', '#CybotCookiebotDialogBodyLevelButtonLevelOptinAllowAll',
  '.cc-accept', '.cc-allow', '.cc-dismiss',
];
const BANNER_SELECTORS = [
  '#cookie-banner', '#cookie-consent', '#onetrust-consent-sdk', '#CybotCookiebotDialog',
  '#usercentrics-root', '#sp-cc', '#qc-cmp2-container', '.cookie-banner', '.cc-window',
  '.consent-banner', '[data-cookieconsent]', '[aria-label*="cookie" i]', '[aria-label*="consent" i]',
];

async function dismissCookieBanners(page) {
  for (const selector of ACCEPT_SELECTORS) {
    const button = await page.$(selector);
    if (button) {
      // DOM click: same target element as the Worker's ElementHandle.click(), without mouse emulation.
      await button.evaluate((el) => el.click());
      await new Promise((r) => setTimeout(r, 300));
      return `clicked ${selector}`;
    }
  }
  const hidden = await page.evaluate((selectors) => {
    let n = 0;
    for (const s of selectors) {
      for (const el of document.querySelectorAll(s)) { el.style.display = 'none'; n++; }
    }
    return n;
  }, BANNER_SELECTORS);
  return hidden ? `hid ${hidden} element(s)` : 'no match';
}

// --- Fixtures: each has a banner (#t-banner or a host) and optional extra checks ---
const acceptJs = `document.querySelectorAll('[data-banner]').forEach(e => e.remove()); document.body.style.overflow='';`;
const FIXTURES = {
  'OneTrust-style ids': `
    <div id="onetrust-consent-sdk" data-banner><div id="t-banner">We use cookies
      <button id="onetrust-accept-btn-handler" onclick="${acceptJs}">Accept</button></div></div>`,
  'Cookiebot-style ids': `
    <div id="CybotCookiebotDialog" data-banner><span id="t-banner">Cookies</span>
      <button id="CybotCookiebotDialogBodyLevelButtonLevelOptinAllowAll" onclick="${acceptJs}">Allow all</button></div>`,
  'Custom banner, text-only button': `
    <div class="privacy-modal" role="dialog" aria-label="Your privacy choices" data-banner>
      <p id="t-banner">We value your privacy</p><button onclick="${acceptJs}">Accept all</button></div>`,
  'Banner inside shadow DOM': `
    <div id="usercentrics-root" data-banner></div>
    <script>
      const root = document.getElementById('usercentrics-root').attachShadow({ mode: 'open' });
      root.innerHTML = '<div id="t-banner" style="position:fixed;bottom:0">Cookies <button class="accept">OK</button></div>';
    </script>`,
  'Banner inside an iframe': `
    <div id="sp_message_container_42" data-banner style="position:fixed;inset:0">
      <iframe id="t-banner" srcdoc="<p>Consent</p><button class='accept-all'>Accept</button>" style="width:100%;height:200px"></iframe></div>`,
  'Banner plus separate backdrop and scroll lock': `
    <style>body{overflow:hidden}</style>
    <div class="cmp-backdrop" id="t-overlay" style="position:fixed;inset:0;background:rgba(0,0,0,.6)"></div>
    <div class="cookie-banner" id="t-banner" data-banner>Cookies</div>`,
  'Unrelated "accept" button earlier in the page': `
    <button class="accept-invite" onclick="document.title='invite accepted'">Accept invitation</button>
    <div class="cookie-banner" id="t-banner" data-banner>Cookies
      <button onclick="${acceptJs}">Allow</button></div>`,
  'Banner injected 1.5 s after load': `
    <script>
      setTimeout(() => {
        const d = document.createElement('div');
        d.id = 't-banner'; d.className = 'cookie-banner'; d.textContent = 'Cookies';
        document.body.append(d);
      }, 1500);
    </script>`,
};

async function isVisible(page, id) {
  return page.evaluate((id) => {
    const host = document.getElementById('usercentrics-root');
    const el = document.getElementById(id) ?? host?.shadowRoot?.getElementById(id);
    if (!el) return false;
    for (let n = el; n; n = n.parentElement ?? n.getRootNode().host) {
      if (getComputedStyle(n).display === 'none' || getComputedStyle(n).visibility === 'hidden') return false;
    }
    const r = el.getBoundingClientRect();
    return r.width > 0 && r.height > 0;
  }, id);
}

const browser = await puppeteer.launch({ headless: true });
const results = [];
try {
  for (const [name, html] of Object.entries(FIXTURES)) {
    const page = await browser.newPage();
    await page.setContent(`<!doctype html><title>fixture</title><body><main>Article text</main>${html}</body>`, { waitUntil: 'load' });
    const action = await dismissCookieBanners(page);
    await new Promise((r) => setTimeout(r, 2000)); // late banners get their chance to appear
    const bannerVisible = await isVisible(page, 't-banner');
    const overlayVisible = await isVisible(page, 't-overlay');
    const scrollLocked = await page.evaluate(() => getComputedStyle(document.body).overflow === 'hidden');
    const title = await page.title();
    const clean = !bannerVisible && !overlayVisible && title === 'fixture';
    results.push({ fixture: name, action, bannerVisible, overlayVisible, scrollLocked, sideEffect: title === 'fixture' ? '' : title, clean });
    await page.close();
  }
} finally {
  await browser.close();
}
console.table(results);

Tested on October 15, 2026 with Node.js 22.18.0 and Puppeteer 24.43.1 (Chrome 148). The output of that run is the fixture table above. The fixtures are static HTML and use no network, so the results do not depend on where you run them. The fixture markup is trusted test content; never build fixture HTML from data you did not write.

What the fixtures do not tell you

These fixtures are patterns, not a sample of the web. They show which banner shapes the handler can and cannot handle. They do not tell you what share of real sites shows each shape.

That share depends heavily on where the browser is. Consent platforms decide whether to show a banner from the visitor's location, and many show nothing outside the EU and the UK. A run from North America would undercount banners, and our renderer's location depends on which Cloudflare data center handles the request. A meaningful "share of EU sites where the banner survives" figure needs a run from EU egress, a fixed URL list and a person who looks at every image. We have not done that run yet, and we will not publish a success rate until we have. If you need one for your own URL list, the render-with-and-without script in the guide produces the image pairs to review.

For the related problem of pages that refuse automated browsers outright, see our headless detection study.

What we are changing

From the fixtures, in order of value:

1. Try platform-specific selectors before generic substrings, so fixture 7 cannot happen on pages that run a known platform.
2. Remove the :has-text entries, or match button text in the page with a small script, which would fix fixture 3.
3. After a hide, also remove full-screen fixed backdrops and restore overflow on html and body, which would fix fixture 6.
4. Look inside same-origin frames and open shadow roots during the click step.

Until those ship, the per-request page options are the reliable fix, and they cost nothing extra.

Try it on your page

Run one URL with and without the option and compare the two images. A free account is enough; the request shape is in the blocking reference.

Related

• Block cookie banners and ads: the options, the fallback steps and a table of consent platforms.
• How we built RenderScreenshot: where the Worker sits in the request.
• Hide cookie banners in Puppeteer: the same problem in your own scripts.