Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork9.7k
[Messenger] Remove base64_encode & use addslashes#30957
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
ed0775f to1900a1aCompare| <xsd:elementname="default-serializer"type="xsd:string"minOccurs="0" /> | ||
| <xsd:elementname="symfony-serializer"type="messenger_symfony_serializer"minOccurs="0" /> | ||
| <xsd:elementname="encoder"type="xsd:string"minOccurs="0" /> | ||
| <xsd:elementname="decoder"type="xsd:string"minOccurs="0" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
why are the encoder and decoder gone ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I can't find them referenced anywhere - there is noencoder ordecoder in the Configuration class. I think these were incorrectlynot removed from some earlier change to messenger.
| $serializeEnvelope =base64_decode($encodedEnvelope['body']); | ||
| $serializeEnvelope =$encodedEnvelope['body']; | ||
| if (true ===$this->base64Encode) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I suggest usingif ($this->base64Encode) as it is already a boolean anyway
nicolas-grekas left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
once stof's comments are addressed
nicolas-grekas commentedApr 8, 2019
Another proposal would be to always use addslashes/stripslashes on the serialized payload. |
weaverryan commentedApr 8, 2019
I think thisdoes fix the problem. It would not allow message objects themselves to contain binary data, but probably that's not supported? Here is what SQS says about character sets:
Is addslashes sufficient? |
1900a1a tob9530f5Comparelyrixx commentedApr 8, 2019
I don't like this option. I often copy/paste message from/to AMQP. More over I often have different language between my producer and my consumer. (But I'm not sure the messenger component is designed for such use cases). Instead of an option, why don't you add |
weaverryan commentedApr 8, 2019
What's the format you usually use? serialized PHP or JSON?
It absolutely is. But in that case, you'd be using the Symfony serializer (JSON) or something custom. Indeed the PhpSerializer is (mostly) only for the situation where you plan to send and consume from your Symfony app.
That's an interesting idea. In fact, the transport is responsible for calling the serializer... and so it could even pass this as an option to |
nicolas-grekas commentedApr 8, 2019 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
addslashes encodes
then you're not using the PhpSerializer - my comment applies only to it. we could also and/or alternatively define a new |
weaverryan commentedApr 8, 2019
I like this way.
On 2nd thought, this might be hard to implement, as this whole escaping thing only applies if you're using the PhpSerializer... and the user may or may not be using it. So, it would be a weird transport option that only applies sometimes. |
lyrixx commentedApr 8, 2019
JSON of course :) |
b9530f5 toc12b8acCompareweaverryan commentedApr 10, 2019
Ready for review again - I've updated the description, etc. This changes to use |
nicolas-grekas left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
messenger.transport.base64_encoded_php_serializer
for another PR when needed?
| if (false ===$serializeEnvelope) { | ||
| thrownewMessageDecodingFailedException('The "body" key could not be base64 decoded.'); | ||
| } | ||
| $serializeEnvelope =\stripslashes($encodedEnvelope['body']); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
\ should be removed
| */ | ||
| publicfunctionencode(Envelope$envelope):array | ||
| { | ||
| $body =\addslashes(serialize($envelope)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
\ should be removed
c12b8ac to70b448dCompareweaverryan commentedApr 15, 2019
Yep, exactly. It's trivial anyways - you could even decorate the Ready to go! |
fabpot commentedApr 15, 2019
Thank you@weaverryan. |
…verryan)This PR was merged into the 4.3-dev branch.Discussion----------[Messenger] Remove base64_encode & use addslashes| Q | A| ------------- | ---| Branch? | master| Bug fix? | no| New feature? | yes| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets | none| License | MIT| Doc PR | already covered by existing issueIn#30814, we base64_encoded messages because some transports (specifically DoctrineTransport + Postgresql & SQS) do not allow binary data.The downside is that the messages become unreadable, which makes it much less convenient to debug your messages with 3rd party monitoring tools, for example.This PR replaces base64_encode with addslashes. Another alternative (that I first tried in this PR) was to use a blob type, which Drupal does in its code (https://www.drupal.org/project/drupal/issues/690746). But, it still meant that binary data could cause problems with other transports, like SQS.I also put all the serializer config under a nice, neat `serializer` key under messenger.Best seen with `?w=1`.Cheers!Commits-------70b448d Reorganizing messenger serializer config and replacing base64_encode with addslashes
Uh oh!
There was an error while loading.Please reload this page.
In#30814, we base64_encoded messages because some transports (specifically DoctrineTransport + Postgresql & SQS) do not allow binary data.
The downside is that the messages become unreadable, which makes it much less convenient to debug your messages with 3rd party monitoring tools, for example.
This PR replaces base64_encode with addslashes. Another alternative (that I first tried in this PR) was to use a blob type, which Drupal does in its code (https://www.drupal.org/project/drupal/issues/690746). But, it still meant that binary data could cause problems with other transports, like SQS.
I also put all the serializer config under a nice, neat
serializerkey under messenger.Best seen with
?w=1.Cheers!