Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[Messenger] Remove base64_encode & use addslashes#30957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
fabpot merged 1 commit intosymfony:masterfromweaverryan:messenger-blob-db-type
Apr 15, 2019

Conversation

@weaverryan
Copy link
Member

@weaverryanweaverryan commentedApr 7, 2019
edited
Loading

QA
Branch?master
Bug fix?no
New feature?yes
BC breaks?no
Deprecations?no
Tests pass?yes
Fixed ticketsnone
LicenseMIT
Doc PRalready covered by existing issue

In#30814, we base64_encoded messages because some transports (specifically DoctrineTransport + Postgresql & SQS) do not allow binary data.

The downside is that the messages become unreadable, which makes it much less convenient to debug your messages with 3rd party monitoring tools, for example.

This PR replaces base64_encode with addslashes. Another alternative (that I first tried in this PR) was to use a blob type, which Drupal does in its code (https://www.drupal.org/project/drupal/issues/690746). But, it still meant that binary data could cause problems with other transports, like SQS.

I also put all the serializer config under a nice, neatserializer key under messenger.

Best seen with?w=1.

Cheers!

<xsd:elementname="default-serializer"type="xsd:string"minOccurs="0" />
<xsd:elementname="symfony-serializer"type="messenger_symfony_serializer"minOccurs="0" />
<xsd:elementname="encoder"type="xsd:string"minOccurs="0" />
<xsd:elementname="decoder"type="xsd:string"minOccurs="0" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

why are the encoder and decoder gone ?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I can't find them referenced anywhere - there is noencoder ordecoder in the Configuration class. I think these were incorrectlynot removed from some earlier change to messenger.

$serializeEnvelope =base64_decode($encodedEnvelope['body']);
$serializeEnvelope =$encodedEnvelope['body'];

if (true ===$this->base64Encode) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I suggest usingif ($this->base64Encode) as it is already a boolean anyway

Copy link
Member

@nicolas-grekasnicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

once stof's comments are addressed

@nicolas-grekas
Copy link
Member

Another proposal would be to always use addslashes/stripslashes on the serialized payload.
That would allow removing the option - ie less complexity. WDYT?

@weaverryan
Copy link
MemberAuthor

Another proposal would be to always use addslashes/stripslashes on the serialized payload.
That would allow removing the option - ie less complexity. WDYT?

I think thisdoes fix the problem. It would not allow message objects themselves to contain binary data, but probably that's not supported? Here is what SQS says about character sets:

A message can include only XML, JSON, and unformatted text. The following Unicode characters are allowed:

#x9 | #xA | #xD | #x20 to #xD7FF | #xE000 to #xFFFD | #x10000 to #x10FFFF

Is addslashes sufficient?

@lyrixx
Copy link
Member

That would allow removing the option - ie less complexity. WDYT?

I don't like this option. I often copy/paste message from/to AMQP. More over I often have different language between my producer and my consumer. (But I'm not sure the messenger component is designed for such use cases).


Instead of an option, why don't you addTransport::needEscaping() ?

@weaverryan
Copy link
MemberAuthor

I often copy/paste message from/to AMQP.

What's the format you usually use? serialized PHP or JSON?

More over I often have different language between my producer and my consumer. (But I'm not sure the messenger component is designed for such use cases).

It absolutely is. But in that case, you'd be using the Symfony serializer (JSON) or something custom. Indeed the PhpSerializer is (mostly) only for the situation where you plan to send and consume from your Symfony app.

Instead of an option, why don't you add Transport::needEscaping() ?

That's an interesting idea. In fact, the transport is responsible for calling the serializer... and so it could even pass this as an option toencode() anddecode()

@nicolas-grekas
Copy link
Member

nicolas-grekas commentedApr 8, 2019
edited
Loading

It would not allow message objects themselves to contain binary data

addslashes encodes\0,'," and\

More over I often have different language between my producer and my consumer

then you're not using the PhpSerializer - my comment applies only to it.

we could also and/or alternatively define a newmessenger.transport.base64_encoded_php_serializer service instead of adding a new option

@weaverryan
Copy link
MemberAuthor

we could also and/or alternatively define a new messenger.transport.base64_encoded_php_serializer service instead of adding a new option

I like this way.

Instead of an option, why don't you add Transport::needEscaping() ?

On 2nd thought, this might be hard to implement, as this whole escaping thing only applies if you're using the PhpSerializer... and the user may or may not be using it. So, it would be a weird transport option that only applies sometimes.

@lyrixx
Copy link
Member

What's the format you usually use? serialized PHP or JSON?

JSON of course :)

@weaverryanweaverryan changed the title[Messenger] Remove base64_encode, but make optional[Messenger] Remove base64_encode & use addslashesApr 10, 2019
@weaverryan
Copy link
MemberAuthor

Ready for review again - I've updated the description, etc. This changes to useaddslashes instead ofbase64_encode(). This only applies to the PhpSerializer.

Copy link
Member

@nicolas-grekasnicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

messenger.transport.base64_encoded_php_serializer

for another PR when needed?

if (false ===$serializeEnvelope) {
thrownewMessageDecodingFailedException('The "body" key could not be base64 decoded.');
}
$serializeEnvelope =\stripslashes($encodedEnvelope['body']);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

\ should be removed

*/
publicfunctionencode(Envelope$envelope):array
{
$body =\addslashes(serialize($envelope));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

\ should be removed

@weaverryan
Copy link
MemberAuthor

messenger.transport.base64_encoded_php_serializer
for another PR when needed?

Yep, exactly. It's trivial anyways - you could even decorate thePhpSerializer

Ready to go!

@fabpot
Copy link
Member

Thank you@weaverryan.

@fabpotfabpot merged commit70b448d intosymfony:masterApr 15, 2019
fabpot added a commit that referenced this pull requestApr 15, 2019
…verryan)This PR was merged into the 4.3-dev branch.Discussion----------[Messenger] Remove base64_encode & use addslashes| Q             | A| ------------- | ---| Branch?       | master| Bug fix?      | no| New feature?  | yes| BC breaks?    | no| Deprecations? | no| Tests pass?   | yes| Fixed tickets | none| License       | MIT| Doc PR        | already covered by existing issueIn#30814, we base64_encoded messages because some transports (specifically DoctrineTransport + Postgresql & SQS) do not allow binary data.The downside is that the messages become unreadable, which makes it much less convenient to debug your messages with 3rd party monitoring tools, for example.This PR replaces base64_encode with addslashes. Another alternative (that I first tried in this PR) was to use a blob type, which Drupal does in its code (https://www.drupal.org/project/drupal/issues/690746). But, it still meant that binary data could cause problems with other transports, like SQS.I also put all the serializer config under a nice, neat `serializer` key under messenger.Best seen with `?w=1`.Cheers!Commits-------70b448d Reorganizing messenger serializer config and replacing base64_encode with addslashes
@weaverryanweaverryan deleted the messenger-blob-db-type branchApril 18, 2019 10:42
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@stofstofstof left review comments

@fabpotfabpotfabpot approved these changes

@nicolas-grekasnicolas-grekasnicolas-grekas approved these changes

Assignees

No one assigned

Projects

None yet

Milestone

4.3

Development

Successfully merging this pull request may close these issues.

6 participants

@weaverryan@nicolas-grekas@lyrixx@fabpot@stof@carsonbot

[8]ページ先頭

©2009-2025 Movatter.jp