Retry mechanisms in Laravel

When something doesn't work on the first try, what do we do? We try it again, right? Expecting that it might work this time.

We, as a human, do this all the time in our day-to-day lives.

This same 'retry technique' is also a useful concept in Software engineering for building reliable Software systems. We can retry failed operations to recover from transient errors or network outages. By automatically retrying failed operations, retry mechanisms can help software systems recover from unexpected failures and continue functioning correctly.

Laravel provides some excellent 'Retry' mechanisms out of the box for handling transient failures!

In this article, we'll learn about these mechanisms to build a reliable system using Laravel!

First of all, What is a ‘transient failure’?

There are certain types of random failures due to service unavailability, network issues, timeouts, etc. Most of the time, these kinds of failures can be solved simply by 'Retrying' the operation. These are known as 'Transient' failures.

Is there anything called ‘Permanent failure’?

Yes, all the other kinds of failures due to faulty business logic, buggy code, wrong credentials, etc. are permanent failures. No matter how much we retry, these failures won’t be resolved automatically unless we fix the code or config.

Here is an excellent article describing a lot more about the theoretical aspect of the 'Retry mechanism'.

Now, let's learn about the 'Retry' mechanisms provided by Laravel.

Retry in Database transactions

Deadlock might occur while executing a Database transaction. This can happen when a transaction is unable to complete due to contention with another concurrent or recent transaction attempting to write to the same data.

In DB::transaction method in Laravel, we can simply pass a second parameter to specify how many times it'll retry the transaction if deadlock occurs.

use Illuminate\Support\Facades\DB;
 
DB::transaction(function () {
    DB::update('update users set votes = 1');
 
    DB::delete('delete from posts');
}, 5);

It's better to always pass a number to this second parameter because the default value for this is 1.

public function transaction(Closure $callback, $attempts = 1) {...}

See more about it in the doc: https://laravel.com/docs/10.x/database#handling-deadlocks

Retry in HTTP call

We all know that third-party APIs or services are not reliable! We need to build our application in a way that we can handle those service failures gracefully.

If we use HTTP Client provided by Laravel, we can use the retry method to automatically retry if the API call is not successful.

$response = Http::retry(3, 100)->post(/* ... */);

We can also pass a second parameter to the method to specify how long Laravel should wait before retrying. It's a good idea to wait for some time before retrying again. Because if the service is down, the immediate retry might fail as well.

We can also customize this retry behavior by providing a 'Closure' as the third parameter.

Check out the details in the doc: https://laravel.com/docs/10.x/http-client#retries

Retry queued jobs

When executing a queued job, it might fail due to some error. If the failure is caused by transient errors, it’s a good idea to attempt again to execute the job.

When running the 'Queue worker', we can pass an argument to mention the maximum number of times a job will be retried if it fails.

php artisan queue:work --tries=3

Otherwise, we can mention in each job separately how many times this job might be retried, and how long it should wait:

namespace App\Jobs;

class ProcessPodcast implements ShouldQueue
{
    /**
     * The number of times the job may be attempted.
     *
     * @var int
     */    
    public $tries = 5;

    /**
     * The number of seconds to wait before retrying the job.
     *
     * @var int
     */    
    public $backoff = 3;
}

We can also define a tries method, backoff method or use a time-based approach by defining a retryUntil method in the Job class.

Check out the details in the doc:

https://laravel.com/docs/10.x/queues#dealing-with-failed-jobs

https://laravel.com/docs/10.x/queues#max-job-attempts-and-timeout

Retry failed jobs

Even if we retry multiple times, some jobs might fail due to permanent errors. These failed jobs require some fix in the code or config. After the fix, we might want to retry the job. Laravel will store these failed jobs in a database table called failed_jobs.

Using the php artisan queue:retry command we can retry one single job, multiple jobs, all jobs from a queue, or all failed jobs.

Here is how we can do that: https://laravel.com/docs/10.x/queues#retrying-failed-jobs

Retry in maintenance mode

When running php artisan down for maintenance mode, we can also provide a retry option to the down command, which will be set as the Retry-After HTTP header's value. But it's not very useful because browsers generally ignore this header,

php artisan down --retry=60

Retry helper

Laravel provides an excellent helper method called ‘retry’ which can be used to retry any piece of code. This is really helpful when we want to retry some logic with different parameters.

Here is a simple example to generate a Unique number that is not already used based on some criteria:

function getUniqueNumber() {
    return retry(5, function() {
        $number = rand(1, 100);

        // 'isUsed' is a fictional method here which checks some logic                
        if (isUsed($number)) {
            throw new Exception('Number is already used');
        }

        return $number;
    });
}

If some alternative methods/packages are used instead of HTTP Client or the DB::transaction provided by Laravel, we can still use this retry helper to retry Database or API calls.

The doc also described more robust ways we can customize this retry helper's behavior: https://laravel.com/docs/10.x/helpers#method-retry

That's a wrap-up for now! I hope this quick go-through over the 'Retry mechanisms in Laravel' was helpful for you.

Did I miss any retry techniques in Laravel?

Feel free to add in the comment! 🙌