Screenshots in background jobs

A screenshot call is slow (seconds, sometimes up to a minute), depends on a page you do not control, and uses a credit when it succeeds. Those three facts decide how the job should look: run it off the request path, make every run safe to repeat, and retry only the failures that can get better. This guide, part of the guides hub, shows that pattern in six job systems.

The Node (BullMQ 5.81), Python (Celery 5.6), Ruby (Sidekiq 8.1) and Go (asynq 0.26) workers were run on 2026-10-15 against Redis and a local stub of the API. Each one enqueued five requests with a duplicate among them, ran four jobs, retried a 422 and a 429 until they succeeded, and stopped at once on a 400. The Laravel and JobRunr listings were not run.

The pattern

enqueue(params):
    key = sha256(canonical JSON of params)[0:32]
    add job with id = key            # duplicates collapse here

job(params):
    if result exists at key: return  # retries after success cost nothing
    image = take(params)             # may raise
    store image at key
on error:
    HTTP 429           -> retry after Retry-After seconds
    408, 422, 5xx, net -> retry with exponential backoff
    400, 401, 403, 404 -> fail now; retrying cannot fix it

One key for job ID and result

Build the key from the request itself: serialize the parameters as JSON with sorted keys and no spaces, hash with SHA-256, and keep the first 32 hex characters. All four tested implementations below produce the same key for {"preset":"og_card","url":"https://example.com/a"}, so a Python producer and a Node worker can share a queue.

Use the key twice:

• As the job ID. A user who clicks "regenerate" three times, or a deploy that replays an import, enqueues one job. BullMQ, asynq and JobRunr reject a duplicate ID on their own. Sidekiq (open source) and Celery do not, so the examples claim the key in Redis with SET NX first.
• As the storage path. The job checks for an existing result before it calls the API. If a worker crashed after the upload but before it acknowledged the job, the retry finds the file and exits without a second render.

If any request option can change (viewport, format, wait settings), include it in the hashed parameters. Otherwise a changed request reuses an old image.

Retry by status, not by message

Use the HTTP status in your retry logic. It is exposed on the error type of every SDK and does not depend on how a client library maps error codes. For the full error envelope, see error handling and retries.

Limits that shape the queue

Set the worker concurrency of the screenshot queue at or below your plan's concurrent limit. More workers do not make it faster; they produce concurrent_limit errors that come back as 429.

Optional: let the API deduplicate a re-render

The POST body accepts "cache": { "available": true }. With it, a request that matches a successful request from your account within the cache TTL returns the earlier result and uses 0 credits. This protects you when your own "result exists" check cannot run (for example, the upload to your storage failed). The SDK builders do not expose this field, so send it with a direct POST as shown in the raw HTTP guide. Details are in caching.

Node.js with BullMQ

// screenshots.mjs - Node 20+, bullmq 5, renderscreenshot SDK
import { createHash } from 'node:crypto';
import { mkdir, writeFile, access } from 'node:fs/promises';
import { Queue, Worker, UnrecoverableError } from 'bullmq';
import { Client, TakeOptions, RenderScreenshotError } from 'renderscreenshot';

const connection = { host: '127.0.0.1', port: 6379 };
const client = new Client(process.env.RS_API_KEY, { timeout: 90_000 });
export const queue = new Queue('screenshots', { connection });

// Same inputs, same key: the key names the job and the stored file.
// params is flat here; use a stable stringify if you hash nested options.
export function screenshotKey(params) {
  const canonical = JSON.stringify(params, Object.keys(params).sort());
  return createHash('sha256').update(canonical).digest('hex').slice(0, 32);
}

export async function enqueueScreenshot(params) {
  const key = screenshotKey(params);
  // BullMQ ignores an add() whose jobId already exists, so double clicks enqueue once
  return queue.add('capture', params, {
    jobId: key,
    attempts: 5,
    backoff: { type: 'exponential', delay: 15_000 },
  });
}

const RETRYABLE = new Set([408, 422, 429, 500, 502, 503, 504]);

export const worker = new Worker(
  'screenshots',
  async (job) => {
    const path = `screenshots/${job.id}.png`; // in production: S3/R2 key or a DB row
    if (await exists(path)) return { path, skipped: true }; // a retry after success costs nothing

    try {
      const image = await client.take(TakeOptions.url(job.data.url).preset(job.data.preset));
      await mkdir('screenshots', { recursive: true });
      await writeFile(path, image);
      return { path };
    } catch (err) {
      if (err instanceof RenderScreenshotError) {
        if (err.httpStatus === 429) {
          // wait as long as the API asks, without using up an attempt
          await worker.rateLimit((err.retryAfter ?? 60) * 1000);
          throw Worker.RateLimitError();
        }
        if (!RETRYABLE.has(err.httpStatus)) throw new UnrecoverableError(err.message);
      }
      throw err; // network errors, 422, 5xx: BullMQ retries with backoff
    }
  },
  { connection, concurrency: 2 }, // keep at or below your plan's concurrent request limit
);

async function exists(path) {
  try {
    await access(path);
    return true;
  } catch {
    return false;
  }
}

worker.rateLimit() pauses the whole queue, which is what you want: a 429 means every worker on this key is over the limit, not only this job.

Python with Celery

# tasks.py - Python 3.9+, Celery 5, redis-py, renderscreenshot SDK
import hashlib
import json
import os
from pathlib import Path

import redis
from celery import Celery
from renderscreenshot import Client, RenderScreenshotError, TakeOptions

app = Celery("screenshots", broker="redis://localhost:6379/0")
app.conf.task_acks_late = True  # a worker crash puts the task back on the queue
app.conf.worker_prefetch_multiplier = 1

r = redis.Redis()
client = Client(os.environ["RS_API_KEY"], timeout=90)
OUT = Path("screenshots")  # in production: S3/R2 or a database row
RETRYABLE = {408, 422, 429, 500, 502, 503, 504}


def screenshot_key(params: dict) -> str:
    canonical = json.dumps(params, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(canonical.encode()).hexdigest()[:32]


def enqueue_screenshot(params: dict) -> str:
    key = screenshot_key(params)
    # Celery does not deduplicate on task_id, so claim the key first
    if r.set(f"screenshot:queued:{key}", 1, nx=True, ex=3600):
        capture.apply_async(args=[params], task_id=key)
    return key


@app.task(bind=True, max_retries=5)
def capture(self, params: dict) -> str:
    key = screenshot_key(params)
    path = OUT / f"{key}.png"
    if path.exists():
        return str(path)  # a retry after success costs nothing

    try:
        image = client.take(TakeOptions.url(params["url"]).preset(params["preset"]))
    except RenderScreenshotError as e:
        if e.http_status not in RETRYABLE:
            # 400, 401, 403, 404: retrying will not help
            raise RuntimeError(f"screenshot failed with HTTP {e.http_status}") from None
        delay = e.retry_after if e.http_status == 429 and e.retry_after else 15 * 2 ** self.request.retries
        raise self.retry(exc=e, countdown=delay)

    OUT.mkdir(exist_ok=True)
    path.write_bytes(image)
    return str(path)

task_acks_late plus the existence check make the task safe to run twice. Without late acknowledgement, a worker killed during a render loses the task. We ran this worker with -P threads; the default prefork pool works the same way.

Ruby with Sidekiq

# app/sidekiq/screenshot_job.rb - Sidekiq 7 or 8, renderscreenshot gem
require 'digest'
require 'fileutils'
require 'json'
require 'renderscreenshot'
require 'sidekiq'

class ScreenshotJob
  include Sidekiq::Job
  sidekiq_options retry: 5

  CLIENT = RenderScreenshot.client(ENV.fetch('RS_API_KEY'), timeout: 90)
  RETRYABLE = [408, 422, 429, 500, 502, 503, 504].freeze

  # 429: wait as long as the API asks. Other retryable errors: Sidekiq's backoff.
  # Anything else (400, 401, 403, 404): send the job straight to the Dead set.
  sidekiq_retry_in do |count, error|
    status = error.respond_to?(:http_status) ? error.http_status : nil
    if status == 429
      error.retry_after || 60
    elsif status && !RETRYABLE.include?(status)
      :kill
    end
  end

  def self.key_for(params)
    Digest::SHA256.hexdigest(JSON.generate(params.sort.to_h))[0, 32]
  end

  def self.enqueue(params)
    key = key_for(params)
    # Sidekiq (OSS) has no unique jobs, so claim the key in Redis first
    claimed = Sidekiq.redis { |c| c.call('SET', "screenshot:queued:#{key}", '1', 'NX', 'EX', 3600) }
    perform_async(params) if claimed
    key
  end

  def perform(params)
    path = "screenshots/#{self.class.key_for(params)}.png" # in production: S3/R2 key or a DB row
    return if File.exist?(path) # a retry after success costs nothing

    image = CLIENT.take(RenderScreenshot::TakeOptions.url(params['url']).preset(params['preset']))
    FileUtils.mkdir_p('screenshots')
    File.binwrite(path, image)
  end
end

Call ScreenshotJob.enqueue('url' => page.url, 'preset' => 'og_card') with string keys, because Sidekiq passes arguments through JSON. With Active Job instead of Sidekiq, express the same rules with retry_on and discard_on, and decide by error.http_status rather than by error class: the gem raises ValidationError for some 422 responses, and those are worth a retry.

PHP with Laravel queues

<?php
// app/Jobs/CaptureScreenshot.php - Laravel 10+, renderscreenshot/renderscreenshot (not run for this guide)
namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Support\Facades\Storage;
use RenderScreenshot\Client;
use RenderScreenshot\RenderScreenshotException;
use RenderScreenshot\TakeOptions;

class CaptureScreenshot implements ShouldQueue, ShouldBeUnique
{
    use Dispatchable, InteractsWithQueue, Queueable;

    public int $tries = 5;      // a release() after a 429 also counts as a try
    public int $timeout = 120;  // longer than the 60 s maximum page wait

    public function __construct(public array $params) {}

    // ShouldBeUnique: a second dispatch with the same key is dropped while this one is queued
    public function uniqueId(): string
    {
        ksort($this->params);
        return substr(hash('sha256', json_encode($this->params, JSON_UNESCAPED_SLASHES)), 0, 32);
    }

    public function backoff(): array
    {
        return [15, 30, 60, 120];
    }

    public function handle(): void
    {
        $path = "screenshots/{$this->uniqueId()}.png";
        if (Storage::exists($path)) {
            return; // a retry after success costs nothing
        }

        $client = new Client(env('RS_API_KEY'), null, 90.0);
        try {
            $image = $client->take(TakeOptions::url($this->params['url'])->preset($this->params['preset']));
        } catch (RenderScreenshotException $e) {
            if ($e->httpStatus === 429) {
                $this->release($e->retryAfter ?? 60);
                return;
            }
            if (!in_array($e->httpStatus, [408, 422, 500, 502, 503, 504], true)) {
                $this->fail($e); // 400, 401, 403, 404: no retry
                return;
            }
            throw $e; // retry with backoff()
        }

        Storage::put($path, $image);
    }
}

JSON_UNESCAPED_SLASHES keeps the key equal to the one the other languages produce. In real code, read the key from config() rather than env() so that config caching works.

Go with asynq

// screenshots.go - Go 1.21+, github.com/hibiken/asynq, github.com/Render-Screenshot/rs-go
package main

import (
    "context"
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "errors"
    "fmt"
    "os"
    "slices"
    "time"

    "github.com/hibiken/asynq"
    rs "github.com/Render-Screenshot/rs-go"
)

type CaptureParams struct {
    Preset string `json:"preset"` // fields in alphabetical order, so the JSON is canonical
    URL    string `json:"url"`
}

func screenshotKey(p CaptureParams) string {
    b, _ := json.Marshal(p)
    sum := sha256.Sum256(b)
    return hex.EncodeToString(sum[:])[:32]
}

// EnqueueScreenshot adds the task once; a second call with the same params is a no-op.
func EnqueueScreenshot(c *asynq.Client, p CaptureParams) error {
    payload, _ := json.Marshal(p)
    _, err := c.Enqueue(asynq.NewTask("screenshot:capture", payload),
        asynq.TaskID(screenshotKey(p)), asynq.MaxRetry(5), asynq.Timeout(2*time.Minute))
    if errors.Is(err, asynq.ErrTaskIDConflict) {
        return nil
    }
    return err
}

var retryable = []int{408, 422, 429, 500, 502, 503, 504}

func HandleCapture(client *rs.Client) asynq.HandlerFunc {
    return func(ctx context.Context, t *asynq.Task) error {
        var p CaptureParams
        if err := json.Unmarshal(t.Payload(), &p); err != nil {
            return fmt.Errorf("bad payload: %v: %w", err, asynq.SkipRetry)
        }
        path := "screenshots/" + screenshotKey(p) + ".png" // in production: S3/R2 key or a DB row
        if _, err := os.Stat(path); err == nil {
            return nil // a retry after success costs nothing
        }

        img, err := client.Take(ctx, rs.URL(p.URL).Preset(p.Preset))
        var apiErr *rs.Error
        if errors.As(err, &apiErr) && !slices.Contains(retryable, apiErr.HTTPStatus) {
            return fmt.Errorf("%v: %w", err, asynq.SkipRetry) // 400, 401, 403, 404
        }
        if err != nil {
            return err // asynq retries; RetryDelay below picks the wait
        }
        if err := os.MkdirAll("screenshots", 0o755); err != nil {
            return err
        }
        return os.WriteFile(path, img, 0o644)
    }
}

// RetryDelay honours Retry-After on 429 and falls back to asynq's default backoff.
func RetryDelay(n int, err error, t *asynq.Task) time.Duration {
    var apiErr *rs.Error
    if errors.As(err, &apiErr) && apiErr.HTTPStatus == 429 && apiErr.RetryAfter > 0 {
        return time.Duration(apiErr.RetryAfter) * time.Second
    }
    return asynq.DefaultRetryDelayFunc(n, err, t)
}

func main() {
    redis := asynq.RedisClientOpt{Addr: "localhost:6379"}
    client, err := rs.New(os.Getenv("RS_API_KEY"), rs.WithTimeout(90*time.Second))
    if err != nil {
        panic(err)
    }

    if len(os.Args) > 1 { // enqueue mode: go run . https://example.com
        c := asynq.NewClient(redis)
        defer c.Close()
        if err := EnqueueScreenshot(c, CaptureParams{URL: os.Args[1], Preset: "og_card"}); err != nil {
            panic(err)
        }
        return
    }

    srv := asynq.NewServer(redis, asynq.Config{
        Concurrency:    2, // keep at or below your plan's concurrent request limit
        RetryDelayFunc: RetryDelay,
    })
    mux := asynq.NewServeMux()
    mux.HandleFunc("screenshot:capture", HandleCapture(client))
    if err := srv.Run(mux); err != nil {
        panic(err)
    }
}

The SDK also has rs.WithMaxRetries(n) for retries inside one call. In a job system, leave it at the default of 0 and let the queue retry, so one place owns the retry budget.

Java with JobRunr

// ScreenshotJobs.java - Java 17+, JobRunr 7, rs-java (not run for this guide)
import com.renderscreenshot.sdk.Client;
import com.renderscreenshot.sdk.RenderScreenshotException;
import com.renderscreenshot.sdk.TakeOptions;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;
import java.util.Set;
import java.util.UUID;
import org.jobrunr.jobs.annotations.Job;
import org.jobrunr.scheduling.BackgroundJob;

public class ScreenshotJobs {
    static final Client CLIENT =
        new Client(System.getenv("RS_API_KEY"), "https://api.renderscreenshot.com", Duration.ofSeconds(90));
    static final Set<Integer> RETRYABLE = Set.of(408, 422, 429, 500, 502, 503, 504);

    // Keys must be canonical JSON with sorted fields, as in the other languages
    public static void enqueue(String key, String url, String preset) {
        UUID id = UUID.nameUUIDFromBytes(key.getBytes(StandardCharsets.UTF_8));
        BackgroundJob.enqueue(id, () -> capture(key, url, preset)); // same id is created once
    }

    @Job(name = "screenshot %1", retries = 5)
    public static void capture(String key, String url, String preset) throws Exception {
        Path path = Path.of("screenshots", key + ".png");
        if (Files.exists(path)) return; // a retry after success costs nothing

        try {
            byte[] image = CLIENT.take(TakeOptions.url(url).preset(preset));
            Files.createDirectories(path.getParent());
            Files.write(path, image);
        } catch (RenderScreenshotException e) {
            if (!RETRYABLE.contains(e.getHttpStatus())) {
                // JobRunr retries every exception; log and return to stop on 400, 401, 403, 404
                System.err.println("screenshot failed for good: HTTP " + e.getHttpStatus());
                return;
            }
            throw e; // JobRunr's exponential backoff
        }
    }
}

JobRunr's backoff does not read Retry-After. If you hit 429 often, lower the worker count for this queue rather than relying on retries.

Where webhooks fit

Every request in these jobs is synchronous: the job waits for the image. If you also configure a webhook URL in the dashboard, RenderScreenshot sends a screenshot.completed or screenshot.failed event for each of these requests too. You can use that stream for monitoring, but do not let it start a second copy of the work. If your handler enqueues jobs, use the same event + id deduplication shown in verify webhook signatures. For a design where the webhook drives the flow, see async screenshots with webhooks.

Troubleshooting

Related

• Error handling and retries: the full error table and backoff code.
• Batch endpoint: many URLs in one request, with its own async mode.
• Test code that calls the API: fakes and stubs for these jobs.
• Rate limits: per-plan limits and headers.