Working with Drupal's Queue API

Enqueue stable references

Queue items should contain identifiers and immutable context, not serialized service objects or entire entities. Load fresh state in the worker and decide whether the work is still necessary.

Make processing idempotent

Assume a job may execute more than once. Use external idempotency keys, state checks, or transactional writes so retries do not duplicate effects. Throwing an exception should leave enough context to diagnose the failure.

Operate the queue

Track backlog size, oldest-item age, throughput, failure counts, and external latency. Cron is adequate for modest work; larger systems may run dedicated workers with bounded execution.

Working example

#[QueueWorker(id: 'journal_reindex', title: new TranslatableMarkup('Reindex journal'), cron: ['time' => 30])]
final class JournalReindexWorker extends QueueWorkerBase {
  public function processItem($data): void {
    $this->indexer->indexArticle((int) $data['nid']);
  }
}

Design the queue item as a message

A queue payload should contain enough information to identify the work without trying to freeze the whole application state. Entity IDs, external identifiers, operation names, and small immutable values are generally easier to retry than serialized entities or service objects.

The worker can load current state when it executes and decide whether the requested work is still valid. That matters when an item waits in the queue while the underlying content changes.

Understand retries and partial failure

Workers can fail after part of an operation succeeds. An external service may accept a request just before the local process times out, or a database change may complete before a later notification fails. Retrying the whole job without a plan can duplicate those effects.

Use idempotency keys, stored status, transactions where appropriate, and explicit ordering so a repeated item reaches the same desired state instead of repeating irreversible side effects.

Operate workers as a service

Knowing that a queue exists is not enough. Track whether items are being consumed, how old the oldest item is, whether throughput is keeping up with creation, and which failures are repeating. A healthy worker system makes delay and failure visible before users report that background work never finished.