Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork9.6k
[Messenger] [Amqp-messenger] Support content encoding and compression#47592
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:7.4
Are you sure you want to change the base?
Conversation
fb9ae8d to278ff6fComparesrc/Symfony/Component/Messenger/Bridge/Amqp/Compressor/CompressorFactory.php OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
What about allowing compression/decompression with any transports? I guess we could add a middleware to do so? |
9ac9934 tofff81c6CompareHi Nicolas! Thanks for your comments. I have worked on a draft. You can see it in thelast commit. In the draft, I try to implement the (de)compression using a middleware. If the draft is close to what you had in mind, I think that we have a problem. The compression expects a string in order to compress it. In the opposite direction, the decompression expects a string to decompress it. When we send a message, the string is only available after the serialization. But the serialization process is inside theTransports. TheTransports receive themessage object, serialized it and send it. A similar problem arises when we consume the message. The stack of middlewares always has themessage object. These are my ideas. What do you think about them? |
When I send a message using the AMQP transport together with theAmqpStamp and thecontent-encoding property, the message is not compressed.
TheAMQP protocol, allow through thecontent-encoding property, to “indicate what additional content encodings have been applied to theapplication-data, and thus what decoding mechanisms need to be applied in order to obtain the media-type referenced by the content-type header field”.
The values admitted according to the protocol are:gzip, compress, deflate, identity, etc.
I expect that the body of the message will be compressed and decompressed respectively by the publisher
and the subscriber.
For example, we can send a messages:
and, With the current code the body message (payload) is not compress using gzip:
$ rabbitmqadmin get queue=messages -f pretty_json [ {"exchange":"messages","message_count": 0,"payload":"{\"name\":\"Foo bar\"}","payload_bytes": 18,"payload_encoding":"string","properties": {"content_encoding":"gzip","content_type":"application/json","delivery_mode": 2,"headers": {"X-Message-Stamp-Symfony\\Component\\Messenger\\Stamp\\BusNameStamp":"[{\"busName\":\"messenger.bus.default\"}]","type":"App\\Message\\DummyMessage" },"timestamp": 1662922329 },"redelivered": true,"routing_key":"" }]but, if we are aware of the value of thecontent_encoding property we should compress the body message.
$ rabbitmqadmin get queue=messages -f pretty_json [ {"exchange":"messages","message_count": 0,"payload":"H4sIAAAAAAAAA6tWysxNTE8NyC8u8UxRsrI0qwUAri6HchIAAAA=","payload_bytes": 38,"payload_encoding":"base64","properties": {"content_encoding":"gzip","content_type":"application/json","delivery_mode": 2,"headers": {"X-Message-Stamp-App\\Messenger\\UniqueIdStamp":"[{\"uniqueId\":\"631e4bdc3ad3b\"}]","X-Message-Stamp-Symfony\\Component\\Messenger\\Stamp\\BusNameStamp":"[{\"busName\":\"messenger.bus.default\"}]","type":"App\\Message\\AddPonkaToImage" },"timestamp": 1662929884 },"redelivered": true,"routing_key":"" }]Finally, the subscriber must be able to uncompressed the body message. I made some changes in order to figure out how
this can be accomplished.
Thanks!, I should be grateful for any comments.
Postscriptum: I closed the PR47545 by mistake :).