Screenshots for Directory Websites

A directory of text links reads like a spam list; a grid of real screenshots reads like a curated product. But nobody wants to open each submitted site, take a screenshot, crop it and upload it by hand. Trigger a capture when the submission arrives, let a signed webhook write the image into the listing record, backfill the existing listings with one batch, and let a cache TTL decide when each listing is captured again.

This page is the submission-driven version of the website thumbnails pattern. That page covers sizes and placeholders; this one covers the pipeline from form to record.

The request gallery

Each listing gets one request. These are the body settings that matter for a directory.

The check is a case-sensitive match on visible text, so list the exact phrases you see on parked pages. The content checks guide has the details.

How it works

sequenceDiagram
  participant F as Submission form
  participant S as Your server
  participant A as Airtable
  participant R as RenderScreenshot
  F->>S: POST site URL
  S->>A: create record (status: pending)
  S->>R: POST /v1/screenshot with webhook.url ?record=recID
  R-->>S: 202 Accepted, job id
  R->>S: screenshot.completed (signed)
  S->>S: verify X-Webhook-Signature
  S->>A: PATCH attachment from screenshot_url
  A->>A: download and keep its own copy

The form answers the visitor at once, and the record moves from pending to ready (or failed: <code>) a few seconds later. The record ID travels in the webhook URL, so you need no job table.

The code

One module does both halves: requestCapture queues the capture with a per-request webhook, and the Express route receives the event, checks the signature and updates the Airtable record.

// directory.mjs: capture each submitted site, then write the screenshot back to its Airtable record.
// Env: RS_API_KEY, RS_WEBHOOK_SECRET, AIRTABLE_TOKEN, AIRTABLE_BASE, PUBLIC_URL (where this app is reachable)
import crypto from 'node:crypto';
import express from 'express';

const RS_API = process.env.RS_BASE_URL ?? 'https://api.renderscreenshot.com';
const AIRTABLE_API = process.env.AIRTABLE_API ?? 'https://api.airtable.com';
const TABLE = `${AIRTABLE_API}/v0/${process.env.AIRTABLE_BASE}/Listings`;

// 1. Called by your submission form handler after it creates the Airtable record.
export async function requestCapture(recordId, siteUrl) {
  const res = await fetch(`${RS_API}/v1/screenshot`, {
    method: 'POST',
    headers: { Authorization: `Bearer ${process.env.RS_API_KEY}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({
      url: siteUrl,
      viewport: { width: 1280, height: 800 },
      output: { format: 'webp', quality: 80, resize: { width: 640, height: 400, fit: 'cover' } },
      block: { cookie_banners: true, ads: true, chat_widgets: true },
      wait: { until: 'networkidle', timeout: 20 },
      // Parked and for-sale domains fail the check, cost 0 credits and come back as screenshot.failed.
      options: { fail_if_contains: ['This domain is for sale', 'Buy this domain'] },
      webhook: {
        url: `${process.env.PUBLIC_URL}/webhooks/screenshot?record=${encodeURIComponent(recordId)}`,
        events: ['completed', 'failed'],
        secret: process.env.RS_WEBHOOK_SECRET,
      },
    }),
  });
  if (res.status !== 202) throw new Error(`Capture not queued: HTTP ${res.status} ${await res.text()}`);
  return (await res.json()).id; // job id, equal to the id of the webhook event
}

// 2. Receives screenshot.completed / screenshot.failed and updates the record.
function verified(raw, signature, timestamp) {
  if (!signature || !timestamp) return false;
  // Retries arrive over about 7 minutes; reject anything much older than that.
  if (Math.abs(Date.now() / 1000 - Number(timestamp)) > 600) return false;
  const expected = `sha256=${crypto.createHmac('sha256', process.env.RS_WEBHOOK_SECRET).update(`${timestamp}.${raw}`).digest('hex')}`;
  const a = Buffer.from(expected);
  const b = Buffer.from(signature);
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

export const app = express();
app.post('/webhooks/screenshot', express.raw({ type: 'application/json' }), async (req, res) => {
  const raw = req.body.toString('utf8');
  if (!verified(raw, req.get('x-webhook-signature'), req.get('x-webhook-timestamp'))) return res.sendStatus(401);

  const event = JSON.parse(raw);
  const recordId = req.query.record;
  if (typeof recordId !== 'string' || !/^rec[A-Za-z0-9]{14}$/.test(recordId)) return res.sendStatus(400);

  const fields = event.event === 'screenshot.completed'
    ? { Screenshot: [{ url: event.data.screenshot_url }], 'Screenshot status': 'ready', 'Captured at': event.timestamp }
    : { 'Screenshot status': `failed: ${event.data.error_code}` };

  const update = await fetch(`${TABLE}/${recordId}`, {
    method: 'PATCH',
    headers: { Authorization: `Bearer ${process.env.AIRTABLE_TOKEN}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({ fields, typecast: true }),
  });
  // A non-2xx answer makes RenderScreenshot retry the delivery.
  res.sendStatus(update.ok ? 200 : 502);
});

if (import.meta.url === `file://${process.argv[1]}`) app.listen(process.env.PORT ?? 3000);

Details that are easy to get wrong:

• Raw body. The signature is an HMAC-SHA256 of timestamp.body with your secret. express.raw keeps the exact bytes; parsing and re-serializing the JSON can change them and break verification.
• Status codes. Any non-2xx answer makes the API retry, up to five attempts over about seven minutes. Return 502 when Airtable is down so the event comes back, and 401 or 400 for requests you never want again.
• The attachment. Airtable's Web API takes an attachment as [{ url }], downloads the file and stores it. That matters because the screenshot_url is a CDN link that expires with the cache TTL.

Tested on 2026-10-18 with Node.js 22.18.0 and Express 5.3.0, against a local stub that answers 202 and then sends a signed screenshot.completed event, and a mock of the Airtable PATCH endpoint. The record received the attachment URL, and a forged signature got a 401.

The split fits both kinds of stack. A code template you deploy, such as Dirstarter, calls requestCapture from its own submission handler. A hosted builder such as Directify runs no code of yours, so the record must live where both sides can reach it, which is the Airtable step's job.

Backfill the listings you already have

For the existing catalogue, send a batch instead of one request per site. More than 10 URLs, or any webhook.url, makes the batch asynchronous:

{
  "urls": ["https://example.com", "https://example.org", "https://example.net"],
  "options": {
    "viewport": { "width": 1280, "height": 800 },
    "output": { "format": "webp", "quality": 80, "resize": { "width": 640, "height": 400, "fit": "cover" } },
    "block": { "cookie_banners": true, "ads": true, "chat_widgets": true }
  },
  "webhook": { "url": "https://directory.example/webhooks/batch" }
}

Post it to /v1/batch. When the batch.completed event arrives, read data.results_url and match each result's url to your records. A batch needs as many credits left as it has URLs before it starts, and its size is capped per plan, so send the backlog in chunks. The batch endpoint reference lists the limits.

Refresh cadence through the cache TTL

A directory full of last year's designs looks abandoned. Run this job daily over every listing; the TTL decides who gets a new capture:

// refresh.mjs: run once a day. A site is re-rendered only when its last render is older than
// REFRESH_DAYS; every other call is a cache hit that costs 0 credits.
const RS_API = process.env.RS_BASE_URL ?? 'https://api.renderscreenshot.com';
const REFRESH_DAYS = 30; // cache.ttl allows 1 hour to 30 days
const PAUSE_MS = 60_000 / 50; // 50 calls a minute: set it below your plan's per-minute limit; cache hits count too

export async function refreshAll(sites, onNewImage) {
  for (const site of sites) {
    const res = await fetch(`${RS_API}/v1/screenshot`, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${process.env.RS_API_KEY}`,
        'Content-Type': 'application/json',
        Accept: 'application/json',
      },
      body: JSON.stringify({
        url: site.url,
        viewport: { width: 1280, height: 800 },
        output: { format: 'webp', quality: 80, resize: { width: 640, height: 400, fit: 'cover' } },
        block: { cookie_banners: true, ads: true, chat_widgets: true },
        wait: { until: 'networkidle', timeout: 20 },
        cache: { available: true, ttl: REFRESH_DAYS * 86400 },
      }),
    });
    const body = await res.json();
    if (!res.ok) {
      console.error(`${site.url}: ${body.error?.code}`); // failed renders cost nothing; tomorrow's run retries
    } else if (!body.cache.hit) {
      await onNewImage(site, body.image.url, body.metadata);
    }
    await new Promise((resolve) => setTimeout(resolve, PAUSE_MS));
  }
}

With cache.available and a 30-day TTL, a listing captured 12 days ago comes back as a cache hit for 0 credits, and one captured 31 days ago gets a fresh render. Every call, hit or not, counts toward your account's per-minute rate limit, which all your keys share, so keep the pause. Tested on 2026-10-18 with Node.js 22.18.0 against the same stub: the second run over the same list produced no new images, and a failing URL was logged and skipped.

What it costs

Take a directory with 1,000 listings, 100 new submissions a month and a 30-day refresh. That is about 1,100 renders a month, inside the Starter plan's 2,500 credits for $10, or about $0.0091 per render. Failed captures of dead and parked sites are free. See the pricing page for the other plans.

Next steps

If your directory is about design rather than discovery, the design inspiration galleries page adds full-page and dark-mode captures. Read the webhook reference for the payload fields, then create a free account and point requestCapture at your own form.