Building High-Throughput Background Job Workers in PHP and Redis

Synchronous HTTP request cycles must be kept under 200ms. Whenever a web request involves sending transactional emails, generating dynamic PDF invoices, resizing high-resolution screenshots, or calling third-party APIs, that work must be pushed to an asynchronous background worker queue.

In this breakdown, we examine how to scale PHP worker pools with Redis to process intensive workloads with guaranteed delivery.

Queue Architecture & Priority Segregation

Do not put all asynchronous jobs into a single default queue. A sudden burst of 10,000 marketing newsletter dispatches could delay critical password reset emails by 30 minutes.

Instead, allocate prioritized worker processes across dedicated Redis queues:

# High priority queue always processes first
php artisan queue:work redis --queue=high,default,low --tries=3 --timeout=60

Handling Rate Limits with Redis Token Buckets

When dispatching webhooks to external services, your workers must respect third-party rate limits. Use atomic Redis throttling to prevent account suspension:

Redis::throttle('stripe-api-sync')
    ->allow(100)
    ->every(60)
    ->then(function () {
        // Safe to execute API request
    }, function () {
        // Could not acquire lock; release back to queue with backoff
        return $this->release(10);
    });

Graceful Worker Restarts & Memory Management

PHP workers running daemonized loops can accumulate memory leaks if third-party packages retain static state. Always configure workers with memory ceilings (`--memory=128`) and use Supervisor or systemd to supervise process lifecycles automatically.