Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[Messenger] Adding failure transport support#30970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@weaverryan
Copy link
Member

@weaverryanweaverryan commentedApr 7, 2019
edited
Loading

QA
Branch?master
Bug fix?yes
New feature?yes
BC breaks?yes
Deprecations?no
Tests pass?yes
Fixed tickets#31231
LicenseMIT
Doc PRsymfony/symfony-docs#11236

This adds "failure" transport support for messenger, so that messages that fail onall their retries can be collected in one spot and retried later if wanted:

framework:messenger:failure_transport:failedtransports:async:dsn:'amqp://'failed:dsn:'doctrine://default?queue_name=failed'routing:'App\Message\SmsNotification':async

In this setup,SmsNotification would be retried 3 times on theasync transport (current behavior) and then finally sent to thefailed transport. Thefailed transport can be consumed like a normal transport, but should usually be handled & consumed by one of the new commands:

> bin/console messenger:failed:show
Screen Shot 2019-04-10 at 3 15 45 PM

> bin/console messenger:failed:show 217
Screen Shot 2019-04-10 at 3 15 55 PM

> bin/console messenger:failed:purge 217
Screen Shot 2019-04-10 at 3 16 07 PM

> bin/console messenger:failed:retry 217
Screen Shot 2019-04-10 at 3 16 29 PM

> bin/console messenger:failed:retry 218 -vv
Screen Shot 2019-04-10 at 3 20 39 PM

Note (This screenshot is ugly - need to make the dump of the message and the exception more attractive)

Or you can runbin/console messenger:failed:retry without any argument, and it will consume the failed messages one-by-one and ask you if you want to retry/handle each. By passing

Cheers!

onEXHovia, hhamon, benja-M-1, datp4ddy, yceruto, raress96, andreybolonin, and maks-rafalko reacted with thumbs up emojijoshlopes, yceruto, prgTW, andreybolonin, fbourigault, and maks-rafalko reacted with heart emoji
* @param bool[] $sendAndHandle
*/
publicfunction__construct(array$senders,array$sendAndHandle = [])
publicfunction__construct(array$sendersMap,ContainerInterface$sendersLocator,array$sendAndHandle = [])
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Instead of each sender class having its own service locator object of senders, this$sendersMap is now a simple array of string sender aliases (or service ids, if no alias) and$sendersLocator is a service locator that contains all senders.

*
* @experimental in 4.3
*/
class FailedMessagesRetryCommandextends Command
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Should we make the commands final?

@weaverryanweaverryanforce-pushed themessenger-reject-transport branch 3 times, most recently from8e203fc toe151d3cCompareApril 10, 2019 21:23
@weaverryanweaverryan changed the title[Messenger][WIP] Adding failed transport support[Messenger] Adding failed transport supportApr 10, 2019
@weaverryanweaverryanforce-pushed themessenger-reject-transport branch from9085c63 to9ccf3bcCompareApril 10, 2019 22:47
@weaverryan
Copy link
MemberAuthor

weaverryan commentedApr 10, 2019
edited
Loading

This is ready! And it worksawesome. Here are some technical notes:

  1. SendersLocatorInterface changed... because for the first time, we need to look up the sender with more logic than just the message class - we need to look up by a specific alias name

  2. Sender aliases are still optional... but the system is increasingly dependent on them being available - retry, failure transport, etc.

@weaverryanweaverryanforce-pushed themessenger-reject-transport branch 3 times, most recently fromc7e613d to16f2b32CompareApril 11, 2019 05:01
@weaverryanweaverryanforce-pushed themessenger-reject-transport branch from16f2b32 to04a985aCompareApril 13, 2019 18:52
@weaverryanweaverryanforce-pushed themessenger-reject-transport branch 2 times, most recently from2c448d8 to905febbCompareApril 23, 2019 14:38
Copy link
MemberAuthor

@weaverryanweaverryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is ready! I think it's important to get into 4.3. We're beyond feature freeze, but this component is experimental and this contains one more BC break. This also contains various bug fixes, which I've outlined.

$throwable =$throwable->getNestedExceptions()[0];
}

$flattenedException =\class_exists(FlattenException::class) ? FlattenException::createFromThrowable($throwable) :null;
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Using FlattenException was my way of making sure we had an Exception that was serializable. The original exception would be even better (as we already have logic to render this very "pretty" in the console), but I think this is the only safe way.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

maybe at least pass file+line info when the Debug component is not found?
or maybe remove this and replace by the throwable cast as string?

publicstaticfunctiongetSubscribedEvents()
{
return [
WorkerMessageFailedEvent::class => ['onMessageFailed', -100],
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Low priority so that users could handle it first if they wanted.

$envelope =$envelope->with(newBusNameStamp($this->busName));
if (null ===$envelope->last(BusNameStamp::class)) {
$envelope =$envelope->with(newBusNameStamp($this->busName));
}
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixes an unrelated "bug" where retries would cause many BusNameStamp to be added, which was just unnecessary and wasteful. Test was added for this.

{
if (null !==$redeliveryStamp) {
return [
$redeliveryStamp->getSenderClassOrAlias() =>$this->sendersLocator->getSenderByAlias($redeliveryStamp->getSenderClassOrAlias()),
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

For redelivery, the sender "alias" really becomes more important. This is communicated on the phpdoc ofSendersLocatorInterface::getSenders(), which mentions that the iterator should be keyed by a sender alias, when possible.

nicolas-grekas reacted with thumbs up emoji
publicfunctionisRetryable(Envelope$message):bool
{
if (0 ===$this->maxRetries) {
if (null ===$this->maxRetries) {
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Unrelated: changes behavior slightly so that0 can be used to mean "don't retry on this transport". Previously, there was no easy way to say "don't retry" as 0 meant "retry infinitely".

Test added for this.

nicolas-grekas reacted with thumbs up emoji
publicfunctiongetMessageCount():int
{
return ($this->receiver ??$this->getReceiver())->getMessageCount();
}
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Unrelated: the interface & this method were just missing.

@trigger_error(sprintf('The 2nd argument to "%s::__construct()" requires ContainerInterface 2nd argument. Not passing that was deprecated in Symfony 4.3 and will be required in Symfony 5.0.',__CLASS__),E_USER_DEPRECATED);
$this->sendersLocator =newServiceLocator([]);
$this->sendAndHandle =$sendersLocator;
$this->useLegacyLookup =true;
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

BC added so that FW bundle 4.2 can work with Messenger 4.3 (otherwise cross-version tests fail)

nicolas-grekas reacted with thumbs up emoji
* @param bool[] $sendAndHandle
*/
publicfunction__construct(array$senders,array$sendAndHandle = [])
publicfunction__construct(array$sendersMap,/*ContainerInterface*/$sendersLocator =null,array$sendAndHandle = [])
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is really the "messiest" part of the PR. Because we need to send to the failure transport, identified by its "name", we need a way to ask the system: "please give me the sender namedmy_failed_transport". The newgetSenderByAlias() accomplishes this. And it works great, but it was a "shift" in the system, as - until now - we only ever asked "Give me the senders for this envelope/message".

"conflict": {
"symfony/event-dispatcher":"<4.3"
"symfony/event-dispatcher":"<4.3",
"symfony/debug":"<4.1"
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Because, IF symfony/debug is available, we need theFlattenException::createFromThrowable() method.

chalasr reacted with thumbs up emoji
@weaverryan
Copy link
MemberAuthor

Ready again!

Copy link
Member

@chalasrchalasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Very nice work!

@weaverryanweaverryanforce-pushed themessenger-reject-transport branch 2 times, most recently frome6bb425 to1692281CompareApril 29, 2019 16:31
@weaverryan
Copy link
MemberAuthor

Thanks for the review@chalasr!

This is once again ready to go!

Copy link
Member

@chalasrchalasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Retested locally, works well now.
👍 once my last comment addressed

@weaverryan
Copy link
MemberAuthor

Comment addressed!

@weaverryanweaverryan modified the milestones:next,4.3Apr 30, 2019
@weaverryanweaverryanforce-pushed themessenger-reject-transport branch froma8923a9 todd1edf3CompareApril 30, 2019 12:45
@weaverryan
Copy link
MemberAuthor

Test failures are unrelated. Ready for review!

@fabpotfabpotforce-pushed themessenger-reject-transport branch fromdd1edf3 to36487e5CompareMay 1, 2019 06:22
@fabpot
Copy link
Member

Thank you@weaverryan.

@fabpotfabpot merged commit36487e5 intosymfony:masterMay 1, 2019
fabpot added a commit that referenced this pull requestMay 1, 2019
This PR was squashed before being merged into the 4.3-dev branch (closes#30970).Discussion----------[Messenger] Adding failure transport support| Q             | A| ------------- | ---| Branch?       | master| Bug fix?      | yes| New feature?  | yes| BC breaks?    | yes| Deprecations? | no| Tests pass?   | yes| Fixed tickets |#31231| License       | MIT| Doc PR        |symfony/symfony-docs#11236This adds "failure" transport support for messenger, so that messages that fail on *all* their retries can be collected in one spot and retried later if wanted:```ymlframework:    messenger:        failure_transport: failed        transports:            async:                dsn: 'amqp://'            failed:                dsn: 'doctrine://default?queue_name=failed'        routing:            'App\Message\SmsNotification': async```In this setup, `SmsNotification` would be retried 3 times on the `async` transport (current behavior) and then finally sent to the `failed` transport. The `failed` transport can be consumed like a normal transport, but should usually be handled & consumed by one of the new commands:**> bin/console messenger:failed:show**<img width="861" alt="Screen Shot 2019-04-10 at 3 15 45 PM" src="https://user-images.githubusercontent.com/121003/55917329-ddc54280-5ba3-11e9-878c-af3c653643de.png">**> bin/console messenger:failed:show 217**<img width="804" alt="Screen Shot 2019-04-10 at 3 15 55 PM" src="https://user-images.githubusercontent.com/121003/55917360-f33a6c80-5ba3-11e9-9f12-a8c57a9a7a4b.png">**> bin/console messenger:failed:purge 217**<img width="835" alt="Screen Shot 2019-04-10 at 3 16 07 PM" src="https://user-images.githubusercontent.com/121003/55917383-ff262e80-5ba3-11e9-9720-e24176b834f7.png">**> bin/console messenger:failed:retry 217**<img width="737" alt="Screen Shot 2019-04-10 at 3 16 29 PM" src="https://user-images.githubusercontent.com/121003/55917396-09482d00-5ba4-11e9-8d51-0bbe2b4ffc14.png">**> bin/console messenger:failed:retry 218 -vv**<img width="1011" alt="Screen Shot 2019-04-10 at 3 20 39 PM" src="https://user-images.githubusercontent.com/121003/55917503-6512b600-5ba4-11e9-9365-4ac87d858541.png">**Note** (This screenshot is ugly - need to make the dump of the message and the exception more attractive)Or you can run `bin/console messenger:failed:retry` without any argument, and it will consume the failed messages one-by-one and ask you if you want to retry/handle each. By passingCheers!Commits-------36487e5 [Messenger] Adding failure transport support
@weaverryanweaverryan deleted the messenger-reject-transport branchMay 1, 2019 17:24
@Tobion
Copy link
Contributor

If I see it correctly, the messenger:failed:show command cannot show messages when rabbitmq is used as a failed transport because it does not implement the listing and getting individual messages.
But in theory that could still be possible by consuming all messages and then putting them back into the queue. Was that considered?

webdevilopers reacted with eyes emoji

@weaverryan
Copy link
MemberAuthor

weaverryan commentedMay 5, 2019
edited
Loading

Hi@Tobion!

Yes, it’s something i talked about with Sam, but wasn’t ultimately sure about. The problem is if there aremany messages. Consuming them all, just to list them and put them back could be a big operation. I don’t have any experience in how rabbit behaves in this way. I’d love to hear any experiences related to this.

Another consideration is that (from feedback I’ve heard) people could theoretically have many failed messages (due to a 3rd party service failure, for example) and want to retry all of them, or maybe all failed messages within a date range. Retrying all of them would work fine, but doing it by date range would require the “consume all” strategy.

Finally, one other unrelated thing from feedback: retry should perhaps just requeue. This is especially important if we offer some sort of “bulk” retry/requeue: requeuing them and allowing their (potentially many) workers on the original queue do their work normally makes more sense then retrying them one-by-one of the server where someone is manually looking at and retrying the failed messages.

@weaverryan
Copy link
MemberAuthor

Also, generally speaking, I’m not sure if Amqp is a good candidate for the failure transport. The failure transport could get “large” - even in a normal app - if you rarely check it and handle those messages. That could hurt Amqp performance - it doesn’t (as I understand it) like to have queues with many messages.

@Tobion
Copy link
Contributor

Hey@weaverryan. I love this feature.I think if we provide a way to configure the failed storage, then we should also try our best to make them all feature complete.

The problem is if there are many messages. Consuming them all, just to list them and put them back could be a big operation.

I don't think it's very common to have many failed messages after retry. But as a sanity check, we could just do the listing if the amount of messages is below a threshold and ask for confirmation if not.

retry should perhaps just requeue

There are use-cases for both approaches in my opinion. You either just want to retry one and see if it works now or why it doesn't work. Or the queue is full and you don't want to wait.
But requeing also makes sense when there are too many messages and you want to leverage your workers.
So ideally both solutions would be offered.

@weaverryan
Copy link
MemberAuthor

I'm going to work on a PR for the requeue. When I've open that, let's continue discussing there to make sure we're happy with everything. I also like the "retry" for at least some situations (just retry it right now and show me if it worked!).

publicfunctionget():iterable
{
if ($this->hasReceived) {
return [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

since this method uses a generator, it should not return array. but justreturn;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

actually it does not seem worth to use a generator here, but just return the envolope in an array.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@fabpotfabpotfabpot approved these changes

@nicolas-grekasnicolas-grekasnicolas-grekas left review comments

@OskarStarkOskarStarkOskarStark left review comments

@chalasrchalasrchalasr approved these changes

+3 more reviewers

@TobionTobionTobion left review comments

@srozesrozesroze left review comments

@onEXHoviaonEXHoviaonEXHovia left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Projects

None yet

Milestone

4.3

Development

Successfully merging this pull request may close these issues.

9 participants

@weaverryan@fabpot@Tobion@nicolas-grekas@sroze@OskarStark@onEXHovia@chalasr@carsonbot

[8]ページ先頭

©2009-2025 Movatter.jp