Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[Mesenger] Add support for reseting container services between 2 messages#41163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
fabpot merged 1 commit intosymfony:5.4fromlyrixx:worker-reset-on-message
Sep 10, 2021

Conversation

@lyrixx
Copy link
Member

@lyrixxlyrixx commentedMay 10, 2021
edited
Loading

QA
Branch?5.4
Bug fix?no
New feature?yes
Deprecations?no
Tickets
LicenseMIT
Doc PRsymfony/symfony-docs#15796

Without this patch, services are not resetted. For example Monolog
Finger Cross handler is never reset nor flushed. So if the first
message trigger and "error" level message, all others message will log
and overflow the buffer.

So, when a transport isasync (it means it is run in a worker), it's highly preferable to this configuration on

Usage with framework:

framework:messenger:transports:async:dsn:'%env(MESSENGER_TRANSPORT_DSN)%'reset_on_message:truefailed:'doctrine://default?queue_name=failed'sync:'sync://'

sweoggy, vuongxuongminh, Zeryther, and maxlipsky-ca reacted with thumbs up emoji
@carsonbot

This comment has been minimized.

@lyrixx
Copy link
MemberAuthor

Hello, I have rebased this PR, and addressed all comments. I should be OK now

@lyrixx
Copy link
MemberAuthor

lyrixx commentedSep 1, 2021
edited
Loading

I could rebase this PR, but before doing so, I prefer to be sure it's mergeable. so WDYT?

Edit: I have rebased the PR

alxndrbauer reacted with eyes emoji

@lyrixxlyrixxforce-pushed theworker-reset-on-message branch fromd9b324a toc52c434CompareSeptember 3, 2021 12:49
@Jeroeny
Copy link
Contributor

Jeroeny commentedSep 3, 2021
edited
Loading

So if the first message trigger and "error" level message, all others message will log and overflow the buffer.

From my experience, a worker running without processing messages can also go out of memory. For example if you have an app running in debug mode the tracing/profiling will keep event-dispatcher events in memory, eventually causing the process to go out of memory.

Because of this, should there also be a possibility to trigger the resetonWorkerRunning instead of just theWorkerMessageHandledEvent andWorkerMessageFailedEvent ?

And continuing on that, would it be nice if you could do a reset when the worker is inactive (Can be readWorkerRunningEvent->isWorkerIdle()) ? . So that if your reset functionality has a bit of performance impact, it could have less impact on the message handling performance.

@lyrixx
Copy link
MemberAuthor

From my experience, a worker running without processing messages can also go out of memory. For example if you have an app running in debug mode the tracing/profiling will keep event-dispatcher events in memory, eventually causing the process to go out of memory.

Indeed, cf#32422

Because of this, should there also be a possibility to trigger the resetonWorkerRunning instead of just theWorkerMessageHandledEvent andWorkerMessageFailedEvent ?

I don't understand :/

And continuing on that, would it be nice if you could do a reset when the worker is inactive (Can be readWorkerRunningEvent->isWorkerIdle()) ? . So that if your reset functionality has a bit of performance impact, it could have less impact on the message handling performance.

Indeed, that would be possible, in a next PR :)

@Jeroeny
Copy link
Contributor

If I understood the logic in this PR correctly, a reset is only triggered by on of thoseWorkerMessage.. events right? So it's only going to reset if messages are being consumed. What about a worker/consumer that's idle for a while? It could go out of memory for the same reasons and might benefit from reset of its services.

@lyrixx
Copy link
MemberAuthor

Ohhh, I see, good point. I need to check when the running event is triggered to ensure it does not break something

@Jeroeny
Copy link
Contributor

when the running event is triggered

I think it should be just as safe as the other events, it at least not during the handling of a message:

https://github.com/symfony/symfony/blob/5.4/src/Symfony/Component/Messenger/Worker.php#L95
https://github.com/symfony/symfony/blob/5.4/src/Symfony/Component/Messenger/Worker.php#L111

@lyrixxlyrixxforce-pushed theworker-reset-on-message branch fromc52c434 tof0fbce3CompareSeptember 6, 2021 15:34
@lyrixx
Copy link
MemberAuthor

@Jeroeny Here we go 👍🏼 I pushed a new version where the listener listenWorkerRunningEvent too. Thanks for the hint 👍🏼

Copy link
Member

@fabpotfabpot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Can you update the description to explain when and why one should enable or disable this flag? If you can create a PR on the docs, that would be the cherry on the cake :)

…ssenger message.Without this patch, services are not resetted. For example MonologFinger Cross handler is never reset nor flushed. So if the firstmessage trigger and "error" level message, all others message will logand overflow the buffer.Usage with framework:```yamlframework:    messenger:        transports:            async:                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'                reset_on_message: true            failed: 'doctrine://default?queue_name=failed'            sync: 'sync://'```
@lyrixxlyrixxforce-pushed theworker-reset-on-message branch fromf0fbce3 to488bb88CompareSeptember 10, 2021 09:13
@lyrixx
Copy link
MemberAuthor

lyrixx commentedSep 10, 2021
edited
Loading

@fabpot I haved edited the PR description, and the configuration.
I'll open the doc PR asap (edit:symfony/symfony-docs#15796)

@fabpot
Copy link
Member

Thank you@lyrixx.

lyrixx reacted with heart emoji

@fabpotfabpot merged commitba7f746 intosymfony:5.4Sep 10, 2021
@lyrixxlyrixx deleted the worker-reset-on-message branchSeptember 10, 2021 09:45
->tag('kernel.event_subscriber')

->set('messenger.listener.stop_worker_on_stop_exception_listener', StopWorkerOnCustomStopExceptionListener::class)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is it correct that theStopWorkerOnCustomStopExceptionListener service looses thekernel.event_subscriber tag?

dmaicher reacted with thumbs up emoji
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks, fixed in#43002

fabpot added a commit that referenced this pull requestSep 13, 2021
…rkerOnCustomStopExceptionListener (lyrixx)This PR was merged into the 5.4 branch.Discussion----------[Messenger] Add back kernel.event_subscriber tag on StopWorkerOnCustomStopExceptionListener| Q             | A| ------------- | ---| Branch?       | 5.4| Bug fix?      | yes (but the bug was not released)| New feature?  | no| Deprecations? | no| Tickets       || License       | MIT| Doc PR        |refs:#41163 (comment)Commits-------00a34c6 [Messenger] Add back kernel.event_subscriber tag on StopWorkerOnCustomStopExceptionListener
symfony-splitter pushed a commit to symfony/framework-bundle that referenced this pull requestSep 13, 2021
…rkerOnCustomStopExceptionListener (lyrixx)This PR was merged into the 5.4 branch.Discussion----------[Messenger] Add back kernel.event_subscriber tag on StopWorkerOnCustomStopExceptionListener| Q             | A| ------------- | ---| Branch?       | 5.4| Bug fix?      | yes (but the bug was not released)| New feature?  | no| Deprecations? | no| Tickets       || License       | MIT| Doc PR        |refs:symfony/symfony#41163 (comment)Commits-------00a34c623b [Messenger] Add back kernel.event_subscriber tag on StopWorkerOnCustomStopExceptionListener
javiereguiluz added a commit to symfony/symfony-docs that referenced this pull requestOct 4, 2021
…lyrixx)This PR was squashed before being merged into the 5.4 branch.Discussion----------[Messenger] document reset_on_message transport optionrefssymfony/symfony#41163Commits-------3f7ebce [Messenger] document reset_on_message transport option
This was referencedNov 5, 2021
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@fabpotfabpotfabpot approved these changes

@srozesrozeAwaiting requested review from sroze

@nicolas-grekasnicolas-grekasAwaiting requested review from nicolas-grekas

@chalasrchalasrAwaiting requested review from chalasr

@dunglasdunglasAwaiting requested review from dunglas

+1 more reviewer

@stefangrstefangrstefangr left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Projects

None yet

Milestone

5.4

Development

Successfully merging this pull request may close these issues.

6 participants

@lyrixx@carsonbot@Jeroeny@fabpot@stefangr@nicolas-grekas

[8]ページ先頭

©2009-2025 Movatter.jp