When using Rabbitmq in cluster, there is a common need of having to upgrade the nodes, while keeping the existing connections. The way this is normally done is by putting nodes in cluster tomaintenance mode, ensuring cluster is healthy at all times. However, symfony/messenger nor php-amqp handle this use case at the moment. What happens instead is that exception when getting the message is thrown, worker crashes, error is logged and process manager has to respin it. This all happens without having a way in user space to handle this case better. Messenger's retry mechanism does not work here, because that one kicks in only when exception is thrown in handlers. Concrete exception is following:

[AMQPConnectionException (320)]
Server connection error: 320, message: CONNECTION_FORCED - Node was put into maintenance mode

What I'm proposing in this PR is that if connection error is detectedtry to reconnect once before throwing exception. That should handle the outlined case. This goes line in line with recommendation from AWS's support we got:

kindly ensure that the client connecting to the broker attempts a retry in case the above error message is observed during a maintenance window. In RabbitMQ cluster deployments, the nodes are restarted one-by-one, meaning at least two nodes will be up and running at all times. Even if a connection is severed, a connection retry will result in the other nodes accepting the connection, and the clients can keep using the broker.

I've also reported issue atphp-amqplib/php-amqplib#1161 with hope that this could be fixed at some point upstream, but I don't give it a big chance.