Laravel Queue Workers: Processing Background Jobs at Scale

Imagine your e-commerce platform just went viral. Orders are flooding in, confirmation emails need sending, PDFs need generating, and third-party payment webhooks are firing every second. If you're handling all of this synchronously within a single HTTP request, your users are staring at a spinning loader — and your server is quietly crying. This is exactly the problem Laravel's queue system was built to solve.

Queue workers allow you to defer time-consuming tasks to the background, keeping your application fast and responsive. But there's a significant gap between spinning up a basic queue and running one reliably at scale. In this article, we'll close that gap.

Understanding the Queue Architecture

At its core, Laravel's queue system has three moving parts: jobs (units of work), queues (named channels jobs are pushed onto), and workers (processes that consume jobs from those queues).

When you dispatch a job, Laravel serializes it and stores it in your configured driver — Redis, database, SQS, or others. A worker process then polls that driver, picks up pending jobs, deserializes them, and executes the handle() method.

// Dispatching a job
ProcessOrderConfirmation::dispatch($order)->onQueue('emails');

// With a delay
SendAbandonedCartReminder::dispatch($cart)->delay(now()->addHours(2));

Creating Jobs the Right Way

A well-structured job is self-contained, idempotent where possible, and handles its own failure gracefully.

namespace App\Jobs;

use App\Models\Order;
use App\Mail\OrderConfirmed;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Mail;

class ProcessOrderConfirmation implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 3;
    public int $backoff = 60; // seconds between retries
    public int $timeout = 120;

    public function __construct(public Order $order) {}

    public function handle(): void
    {
        Mail::to($this->order->customer->email)
            ->send(new OrderConfirmed($this->order));
    }

    public function failed(\Throwable $exception): void
    {
        // Notify the team, log the failure, trigger a fallback
        \Log::error('Order confirmation failed', [
            'order_id' => $this->order->id,
            'error' => $exception->getMessage(),
        ]);
    }
}

Notice the explicit $tries, $backoff, and $timeout properties. These are non-negotiable in production. Without them, a poorly behaved job can exhaust your worker or retry indefinitely.

Running Workers in Production

Supervisor: The Missing Piece

Running php artisan queue:work directly on your server is fine for development — it'll die the moment your SSH session closes. In production, you need a process manager. Supervisor is the standard choice.

Here's a typical Supervisor configuration for a Laravel application:

[program:laravel-worker-default]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work redis --queue=default --sleep=3 --tries=3 --max-jobs=500
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=4
redirect_stderr=true
stdout_logfile=/var/log/supervisor/laravel-worker-default.log
stopwaitsecs=3600

The --max-jobs=500 flag is important — it restarts the worker after processing 500 jobs, which prevents memory leaks from accumulating over time. Pair this with --max-time if you prefer time-based recycling.

Using Multiple Named Queues

Not all jobs are equal. A password reset email is more urgent than a weekly analytics report. Use named queues with priority ordering:

php artisan queue:work redis --queue=critical,emails,default,low

Workers process critical first, falling through to lower-priority queues only when higher-priority ones are empty. This prevents heavy background jobs from blocking time-sensitive operations.

Scaling Horizontally with Redis

Redis is the recommended queue driver for high-throughput applications. It supports atomic operations natively, making it safe to run multiple worker processes without duplicate job processing.

// config/queue.php
'redis' => [
    'driver' => 'redis',
    'connection' => 'default',
    'queue' => env('REDIS_QUEUE', 'default'),
    'retry_after' => 90,
    'block_for' => null,
    'after_commit' => true, // Only dispatch after DB transaction commits
],

The after_commit flag is subtle but critical. Without it, a job dispatched inside a database transaction might be picked up by a worker before the transaction commits — meaning the job runs against data that technically doesn't exist yet.

Monitoring with Laravel Horizon

Once you move to Redis, install Laravel Horizon. It gives you a real-time dashboard for queue metrics, throughput, failure rates, and job runtimes.

composer require laravel/horizon
php artisan horizon:install
php artisan horizon

Horizon also handles auto-balancing workers based on load:

// config/horizon.php
'environments' => [
     'production' => [
        'supervisor-1' => [
            'maxProcesses' => 20,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
        ],
    ],
],

This dynamic scaling is what separates a hobby setup from a production-grade queue system. When demand spikes, Horizon spawns more workers automatically. When things quiet down, it scales back.

Handling Long-Running Jobs and Timeouts

For jobs that involve file processing, API polling, or report generation, timeouts become a real concern. A few patterns help:

Chunking large datasets:

public function handle(): void
{
    User::query()
        ->whereNull('report_generated_at')
        ->chunkById(100, function ($users) {
            foreach ($users as $user) {
                GenerateUserReport::dispatch($user);
            }
        });
}

Instead of one massive job iterating over thousands of records, dispatch individual child jobs. Each child is small, fast, and retryable independently.

Job chaining for sequential workflows:

Bus::chain([
    new ValidateOrder($order),
    new ChargePayment($order),
    new FulfillOrder($order),
    new SendConfirmationEmail($order),
])->dispatch();

Chaining ensures each step only runs when the previous one succeeds, giving you transactional-like behavior across multiple jobs.

Deployment Considerations

When you deploy new code, your running workers still have the old code loaded in memory. You need to restart them gracefully:

php artisan queue:restart

This signals workers to stop after finishing their current job — no abrupt termination. Add this to your deployment script (Envoyer, Forge, or your CI/CD pipeline) as a mandatory step.

At our agency we handle queue-heavy Laravel applications regularly — if you're curious how teams approach background processing for AI-driven features and high-traffic platforms, you can learn more about the kind of architecture decisions that come into play at that scale.

Conclusion

Laravel's queue system is genuinely powerful, but its reliability at scale depends on the details: proper job design, Supervisor configuration, named queue priorities, Redis as your driver, Horizon for observability, and graceful restarts on deployment. Get these right and background processing becomes one of the most dependable parts of your stack — rather than a source of silent failures and user frustration.

Start with one well-structured job, observe it through Horizon, and build your queue architecture incrementally. The investment pays off every time your application stays fast under load.