Scaling Puppeteer Screenshots: Pools, Queues, Throughput

To take bulk screenshots with Puppeteer, run a fixed number of workers that pull URLs from a queue, spread those workers over several browser processes with one or two pages each, open a fresh browser context per job, and retry a failed job once. The split matters more than the total: in our tests, 8 captures in flight ran more than twice as fast as 8 separate browsers than as 8 tabs in one browser.

The core loop looks like this:

const queue = [...urls];
await Promise.all(workers.map(async (browser) => {
  for (let url = queue.shift(); url; url = queue.shift()) {
    const ctx = await browser.createBrowserContext();
    try {
      const page = await ctx.newPage();
      await page.goto(url, { waitUntil: 'load' });
      await save(url, await page.screenshot({ type: 'webp', quality: 80 }));
    } finally {
      await ctx.close();
    }
  }
}));

This post measures what that loop does at 1, 4, 8 and 16 concurrent pages, shows where one browser stops scaling, and then turns the loop into a pool that survives crashed browsers. It is part of our Puppeteer screenshot guide.

Tested with: Puppeteer 25.14.0, Chrome 155.0.8059.79 (the browser Puppeteer downloads), Node.js 22.18.0, macOS on an Apple M1 Pro (10 cores, 32 GB). Tested on the night of 2026-10-18.

How we measured

The numbers on this page come from one machine and a local test page, so the network is out of the picture and only the browser work changes between runs.

• Target: a 1200×630 social card served by a local Node.js server. Each card loads one web font (the server waits 150 ms before answering) and one PNG (120 ms). Twelve variants rotate.
• Job: a new browser context, setViewport({ width: 1200, height: 630 }), goto with waitUntil: 'load', a WebP screenshot at quality 80, then close the context.
• Run: 200 jobs per configuration. We report throughput (jobs per second over the whole run), p50 and p95 time per job, failures, and peak memory. Memory is the resident size of every Chrome process (browser, GPU, renderers) summed, sampled every 250 ms.
• Caveat: the laptop ran other work at the same time, and absolute numbers moved by up to 30% between runs. The comparisons held in every run, which is why we trust them. Run the scripts on your own host before you size a server.

One browser, more pages

The first question is how far one Chrome process goes. Same 200 jobs, one browser, 1 to 16 workers sharing it:

Two earlier runs had the same shape at lower absolute speed (2.0, 4.1, 4.3 and 4.4 per second in one of them). In all three, going past 4 pages added less than 10% throughput.

Look at the p50 column. From 4 pages up, it roughly doubles each time concurrency doubles, while throughput stays flat. That is the signature of a queue: the extra pages are not working in parallel, they are waiting inside the browser. A capture that took a third of a second alone took three seconds at 16 pages, for no gain. We did not profile where Chrome serializes the work; the practical result is that adding tabs to one browser stops paying off very early.

The same eight slots, split across browsers

Next we kept 8 captures in flight and changed only how they are split between Chrome processes. Two runs each:

What this says:

• Separate browsers scale; tabs in one browser do not. 8 × 1 did 2.4 times the work of 1 × 8 with the same number of captures in flight.
• Memory is the price. Each extra browser added about 600 MB on this page. Heavier pages cost more per renderer.
• The middle is the sweet spot. 4 × 2 reached about 90% of the 8 × 1 throughput with about 57% of its memory. That is the setting we would start a 10-core, memory-limited server with.
• Past the CPU, more slots only add latency. 4 × 4 matched 8 × 1 throughput but doubled the time per job. The machine was full.

Your numbers will differ with page weight and core count. The shape is what to plan around: find the browser count where throughput flattens, then stop.

A pool that survives real failures

The bare loop from the top of the page has a problem we hit during these runs. On one of the two 4 × 4 runs, a browser disconnected in the middle (TargetCloseError: Protocol error (Target.disposeBrowserContext): Target closed). The ctx.close() in the finally block threw, nothing caught it, and the whole Node.js process exited with 150 jobs still in the queue. We did not find out why that browser died; on a busy host, you should assume one will.

A production pool needs five things the bare loop lacks:

1. A fixed number of browsers and pages per browser, from the tables above.
2. A fresh browser context per job, so cookies, storage and cache from one URL never leak into the next.
3. A hard timeout per job that closes the context even if a goto or a screenshot hangs.
4. A retry, once, for jobs that failed for reasons that are not the page's fault.
5. Browser replacement: relaunch any browser that disconnects, and retire each browser after a set number of jobs so slow leaks reset.

Here is the complete pool. It is plain Puppeteer with no other dependency.

// pool.mjs
import puppeteer from 'puppeteer';
import { mkdir, readFile, writeFile } from 'node:fs/promises';

const BROWSERS = Number(process.env.BROWSERS ?? 4);       // Chrome processes
const PAGES_PER_BROWSER = Number(process.env.PAGES ?? 2); // concurrent captures in each
const RECYCLE_AFTER = Number(process.env.RECYCLE ?? 100); // restart a browser after N jobs
const JOB_TIMEOUT_MS = 20_000;
const MAX_ATTEMPTS = 2;

// One Chrome process. Retires itself after RECYCLE_AFTER jobs: new jobs go to a
// fresh browser, and the old one closes when its last job finishes.
class BrowserSlot {
  constructor() { this.current = null; this.launching = null; }

  async acquire() {
    if (this.current && (this.current.jobs >= RECYCLE_AFTER || !this.current.browser.connected)) {
      const old = this.current; this.current = null;
      old.retired = true;
      if (old.active === 0) await closeBrowser(old.browser);
    }
    if (!this.current) {
      this.launching ??= puppeteer.launch({ args: ['--disable-dev-shm-usage'] })
        .then((browser) => { this.current = { browser, jobs: 0, active: 0, retired: false }; })
        .finally(() => { this.launching = null; });
      await this.launching;
    }
    const entry = this.current;
    entry.jobs++; entry.active++;
    return entry;
  }

  async release(entry) {
    entry.active--;
    if (entry.retired && entry.active === 0) await closeBrowser(entry.browser);
  }

  async close() { if (this.current) await closeBrowser(this.current.browser); this.current = null; }
}

async function closeBrowser(browser) {
  await browser.close().catch(() => browser.process()?.kill('SIGKILL'));
}

async function capture(slot, url) {
  const entry = await slot.acquire();
  const context = await entry.browser.createBrowserContext(); // isolated cookies and storage
  const timer = setTimeout(() => context.close().catch(() => {}), JOB_TIMEOUT_MS);
  try {
    const page = await context.newPage();
    page.setDefaultTimeout(JOB_TIMEOUT_MS);
    await page.setViewport({ width: 1200, height: 630 });
    await page.goto(url, { waitUntil: 'load' });
    return await page.screenshot({ type: 'webp', quality: 80 });
  } finally {
    clearTimeout(timer);
    await context.close().catch(() => {});
    await slot.release(entry);
  }
}

export async function run(urls, onResult) {
  const slots = Array.from({ length: BROWSERS }, () => new BrowserSlot());
  const queue = urls.map((url, i) => ({ url, i, attempts: 0 }));
  const failures = [];

  async function worker(slot) {
    for (let job = queue.shift(); job; job = queue.shift()) {
      job.attempts++;
      try {
        await onResult(job, await capture(slot, job.url));
      } catch (err) {
        if (job.attempts < MAX_ATTEMPTS) queue.push(job);
        else failures.push({ url: job.url, error: err.message });
      }
    }
  }

  try {
    await Promise.all(slots.flatMap((slot) => Array.from({ length: PAGES_PER_BROWSER }, () => worker(slot))));
  } finally {
    await Promise.all(slots.map((s) => s.close()));
  }
  return failures;
}

// node pool.mjs urls.txt   (one URL per line)
if (import.meta.url === `file://${process.argv[1]}`) {
  const urls = (await readFile(process.argv[2], 'utf8')).split('\n').map((s) => s.trim()).filter(Boolean);
  await mkdir('shots', { recursive: true });
  const t0 = performance.now();
  const failures = await run(urls, (job, img) => writeFile(`shots/${job.i}.webp`, img));
  const secs = (performance.now() - t0) / 1000;
  const ok = urls.length - failures.length;
  console.log(`${ok}/${urls.length} ok in ${secs.toFixed(1)} s (${(ok / secs).toFixed(1)}/s)`);
  if (failures.length) { console.error(failures); process.exitCode = 1; }
}

Run it with BROWSERS=4 PAGES=2 node pool.mjs urls.txt. The process exits with code 1 and prints the failed URLs if any job failed twice, so a cron job or CI step notices. Our run of this exact listing on 200 card URLs plus two bad entries (an unsafe port and a string that is not a URL) printed 200/202 ok in 17.7 s (11.3/s), listed both bad entries with their errors, and exited with code 1.

Why each piece is there

• createBrowserContext() per job costs little compared to a new browser and gives every URL a clean profile. It is also what makes cleanup reliable: closing the context closes every page and popup the site opened.
• The timer closes the context, not the page. In our test, a goto stuck on a page that never finished loading rejected within about 10 ms of its context closing (Navigating frame was detached), so the worker moves on instead of waiting forever. setDefaultTimeout covers the normal cases; the timer is the backstop.
• Retries go to the back of the queue, so a job that failed because its browser died is picked up by whichever worker is free, usually on a different slot.
• browser.connected is checked on every acquire. A dead browser is replaced on the next job, and the slot's two workers keep working.
• launching ??= makes the two workers of a slot share one launch instead of starting two browsers at the same moment.
• --disable-dev-shm-usage matters in containers, where /dev/shm defaults to 64 MB and Chrome crashes when it fills. The Puppeteer Docker guide covers the rest of the container setup, including fonts and the sandbox.

What we tested against it

When we killed a browser mid-run, the two jobs it was working on failed within half a second (Protocol error (Emulation.setTouchEmulationEnabled): Target closed), went back into the queue and succeeded on the retry. The slot launched a new browser on its next job. Recycling every 20 jobs cost about 7% in run time on this page; every 100 cost nothing we could see. Pick the longest interval that keeps your memory flat over a long run.

The pool does not handle every failure. A page that crashes its renderer (an out-of-memory tab) fails the job and is retried once, which is correct. A page that is simply slow fails twice and ends in failures, which is also correct: retrying a 20-second timeout a third time rarely helps. For the timeout and leak cases in depth, see Puppeteer timeouts and memory leaks.

Beyond one process: a durable queue

The in-memory array is fine for a script that runs to completion. It is not fine for a service: if the process restarts, the queue is gone. Once screenshots arrive from a web app or must survive a deploy, move the queue out of the process.

flowchart LR
  A[Web app or cron] -->|enqueue URL| Q[(Durable queue)]
  Q --> W1[Worker process 1<br/>pool: 4 browsers x 2 pages]
  Q --> W2[Worker process 2<br/>pool: 4 browsers x 2 pages]
  W1 --> S[(Object storage)]
  W2 --> S
  W1 -->|job done / failed| Q
  W2 -->|job done / failed| Q

The pool's design carries over. Each worker process owns one pool sized for its machine, takes at most BROWSERS × PAGES jobs from the queue at once, and acknowledges a job only after the image is stored. Three rules to keep:

• Cap in-flight jobs at the pool size. A consumer that pulls 50 jobs into a pool of 8 slots recreates the "16 tabs in one browser" row: same throughput, much worse latency, and 42 jobs that time out on the queue's side.
• Let the queue own retries across processes. Keep the pool's single in-process retry for crashed browsers, and let the queue retry jobs whose worker died.
• Make jobs idempotent. Name the output after a hash of the URL and options, so a job that runs twice overwrites the same file instead of producing a duplicate.

Scale out by adding worker processes or machines, not by raising pages per browser.

Production checklist

• Size from your own measurement. Run the 200-job benchmark at 1 × N, N/2 × 2 and N × 1 on the target host, with your real pages. Stop adding browsers when throughput flattens.
• Watch memory, not just CPU. At 8 × 1 our run peaked at about 5.6 GB. Set the container limit above your measured peak, or the kernel will kill Chrome mid-job.
• Wait for the right thing. waitUntil: 'load' was right for our card. For client-rendered pages, a wait that is too long holds a slot for nothing and a wait that is too short produces a blank image; choosing the wait for page readiness shows the trade-offs with timings.
• Cut requests you do not need. Ads and trackers keep pages busy and cost bandwidth on every job. Blocking requests in Puppeteer shows what each blocked type saves.
• Run Chrome under an init process in containers (docker run --init or dumb-init), so killed renderers are reaped instead of piling up as zombies.
• Log per job: URL, attempt, slot, time, bytes and error. When throughput drops, this tells you whether pages got slower or browsers started dying.

Troubleshooting

The same bulk job with one API call

If you would rather not size, run and patch a browser fleet, RenderScreenshot's batch endpoint takes a list of URLs and renders them on our side. A batch of 10 URLs or fewer with no webhook returns results directly; a larger batch, or one with a webhook, returns a job id that you poll, or sends a webhook when it is done.

import { Client, TakeOptions } from 'renderscreenshot';

const client = new Client(process.env.RS_API_KEY);

const batch = await client.batch(
  ['https://example.com/', 'https://example.org/', 'https://example.net/'],
  TakeOptions.url('').preset('og_card')
);
console.log(batch.completed, batch.total);

How many URLs one batch holds and how many render at once depend on your plan; the bulk screenshots guide lists the limits and includes a runner that splits any list into batches and retries failures. The rate limit is shared by all keys on an account. You can try it with 150 free credits a month: create a free account.

Related guides

• Taking screenshots with Puppeteer, the hub for this series
• Puppeteer timeouts and memory leaks, for the failures a pool has to absorb
• Puppeteer in Docker, a tested image to run this pool in
• Element screenshots in Puppeteer, when each job captures part of a page