In this test, we seem to be discretely doing things:

sending + consuming
(kill connection + wait for reconnect)
sending + consuming

I think that's not really what happens in the background worker, or in gundeck for forwarding notifications. Those processes would be waiting for new AMQP messages all the time, and suddenly the disconnect would kick.

What happens if the RabbitMQ connection is killed while sending/consuming messages?

It's correct that we test that sending and receiving works, then kill RabbitMQ, restart it and wait for reconnect and then test sending and receiving again.

But I don't understand what you mean with "that's not what happens in background worker". The background worker is always connected to the queue, also when the outage starts. Or more precisely, it's a thread in the background worker, that should constantly and indefinitely try to reconnect or restart if killed.

Gundeck in not involved as from the logs we can see that it's the notification-pusher that while trying to establish a connection to RabbitMQ.

Trying to kill RabbitMQwhile bg worker is consuming messages is an interesting idea. What we do is, we kill the broker, while the bg is connected, however there are no messages being processed when the connection is killed, because the queue is empty. Killing the broker while the queue contains unprocessed messages is technically a bit difficult because when we take RabbitMQ down we also cannot produce messages. Still maybe worthwhile to try to find a workaround and test this?