- Notifications
You must be signed in to change notification settings - Fork4k
(to be written)#14902
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
(to be written)#14902
Conversation
... by using `keep_while` conditions on bindings and auto-deleteexchanges.[Why]The `delete_queue` transaction's anonymous function has to be beextracted by Horus, like any Khepri transaction. This is an expensiveoperation, but Horus uses caching to avoid most work after the firstextraction.The problem is when there are many concurrent executions of the sametransaction, before it has been executed once: the cache is not hot andHorus has to extract the same transaction many times in parallelcurrently.An example of this situation is when there are massive disconnectionsfrom RabbitMQ clients that trigger massive queue deletions. This can puta lot of load on RabbitMQ.[How]This patch removes the entire transaction. Instead, it uses `keep_while`conditions on bindings and auto-delete exchanges to let Khepri handlethe deletion of semantically related tree nodes. RabbitMQ just has tomake a simle "delete this queue" command.
082e24d to3a75b6cComparethe-mikedavis commentedNov 6, 2025
I was looking into eliminating the transaction for queue deletion a while back but I didn't get the changes over the finish line. I just pushed up the branch I was working on:https://github.com/rabbitmq/rabbitmq-server/tree/md/khepri-q-delete-command. It's quite old but maybe it has some interesting leads for this branch. I also looked into using stored procedures where we use transactions now - wouldn't help us with the BEAM file changes in OTP 28 but it avoided some performance impacts. The code-server work for transactions is expensive but also the size of transaction commands can be quite large (the compiled module binary + env) and we could do a lot less work with commands + keep-while conditions. But, if I am remembering correctly, we would need some sort of migration for existing Khepri DBs to transition from transactions to keep-while and regular commands so that they could add the keep-while conditions. |
dumbbell commentedNov 7, 2025
Yes, exactly. This patch is incomplete on purpose (the needed feature flag is a simple hard-coded Thank you very much for sharing your work! I will take a look at it :-) |
(to be written; CI doesn’t run on a commit without an explicit pull request…)