Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[RateLimiter] Add limit object on RateLimiter consume method#38257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
fabpot merged 1 commit intosymfony:masterfromvasilvestre:limit-object-on-ratelimiter
Sep 30, 2020
Merged

[RateLimiter] Add limit object on RateLimiter consume method#38257

fabpot merged 1 commit intosymfony:masterfromvasilvestre:limit-object-on-ratelimiter
Sep 30, 2020

Conversation

@vasilvestre
Copy link

@vasilvestrevasilvestre commentedSep 21, 2020
edited
Loading

QA
Branch?master (should be merged in 5.2 before 31 September if possible)
Bug fix?no
New feature?yes
Deprecations?no
TicketsFix#38241
LicenseMIT
Doc PRNot yet :/

@vasilvestrevasilvestre changed the titleAdd limit object on RateLimiter consume methodWIP Add limit object on RateLimiter consume methodSep 21, 2020
@nicolas-grekasnicolas-grekas added this to thenext milestoneSep 24, 2020
@vasilvestre
Copy link
Author

@wouterj how should I handle the Redirect login fail attempts ?

@vasilvestrevasilvestre changed the titleWIP Add limit object on RateLimiter consume method[RateLimiter] Add limit object on RateLimiter consume methodSep 26, 2020
Copy link
Member

@wouterjwouterj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thank you@vasilvestre! And I'm sorry for not being available at second half of the week..

This PR looks good to me - all pending comments are minor ones no big design changes. The only thing I'm wondering about is whether to use FQCN in the configuration - that might not be very easy to use for Yaml and XML users. Would love feedback from the core team about this.

vasilvestre reacted with thumbs up emoji
@vasilvestre
Copy link
Author

vasilvestre commentedSep 26, 2020
edited
Loading

Thank you@vasilvestre! And I'm sorry for not being available at the second half of the week.

This PR looks good to me - all pending comments are minor ones no big design changes. The only thing I'm wondering about is whether to use FQCN in the configuration - that might not be very easy to use for Yaml and XML users. Would love feedback from the core team about this.

I just finish the PR on a live coding session, it was a nice moment so no problem!

Yeah FQCN isn't fine for YAML and XML, how is the core team dealing with it recently?
I switched from 'fixed_window' to FQCN because of previous discussion, what should we go for?

@fabpot
Copy link
Member

I'm not comfortable using a FQCN in configuration, I would prefer to keep using strings instead.

@vasilvestre
Copy link
Author

I'm not comfortable using a FQCN in configuration, I would prefer to keep using strings instead.

Strings are back.

@kbond
Copy link
Member

It was my understanding the FQCN was only being dropped in config, notLimit::getStrategy()?

@vasilvestre
Copy link
Author

vasilvestre commentedSep 27, 2020
edited
Loading

It was my understanding the FQCN was only being dropped in config, notLimit::getStrategy()?

Well I do not really understand what you mean actually, where should we use it?

@kbond
Copy link
Member

kbond commentedSep 27, 2020
edited
Loading

This should still be an option, no?

I thought Fabien's comment was about removing FQCN from the configuration.

@vasilvestre
Copy link
Author

vasilvestre commentedSep 27, 2020
edited
Loading

Limiter isn't set at this moment, we set it in limiter itself in create()

publicfunctioncreate(?string$key =null):LimiterInterface{$id =$this->config['id'].$key;$lock =$this->lockFactory ?$this->lockFactory->createLock($id) :newNoLock();switch ($this->config['strategy']) {case'token_bucket':returnnewTokenBucketLimiter($id,$this->config['limit'],$this->config['rate'],$this->storage,$lock);case'fixed_window':returnnewFixedWindowLimiter($id,$this->config['limit'],$this->config['interval'],$this->storage,$lock);default:thrownew \LogicException(sprintf('Limiter strategy "%s" does not exists, it must be either "token_bucket" or "fixed_window".',$this->config['strategy']));    }}

Edit : OK I got it mate sorry ! I haven't worked yet on headers and this isn't the goal here so I didn't think about it. This will be useful in future, when we will be able to attach a response or whatever and set headers, that's it ?
Fixed now !

@vasilvestre
Copy link
Author

vasilvestre commentedSep 27, 2020
edited
Loading

Flaky test here but fixable through#38320

Edit : It's already merged so I rebased and test should all pass :)

@symfonysymfony deleted a comment fromvasilvestreSep 28, 2020
Copy link
Member

@fabpotfabpot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

minor comments

@nicolas-grekas
Copy link
Member

So, now, I'm wondering about the design:

Why didn't we addgetRemainingTokens andgetRetryAfter toLimiterInterface instead?
Also, I don't see the point ofgetStrategy andgetMetadata, those look like leaking configuration to me, nothing that can be dealt with from an abstract pov.

@vasilvestre
Copy link
Author

So, now, I'm wondering about the design:

Why didn't we addgetRemainingTokens andgetRetryAfter toLimiterInterface instead?
Also, I don't see the point ofgetStrategy andgetMetadata, those look like leaking configuration to me, nothing that can be dealt with from an abstract pov.

See#38257 (comment) for history details, it's not needed right now but it will be useful for test, as shown there :#38257 (comment)

@nicolas-grekas
Copy link
Member

I'm not convinced at all that the abstraction needs to provide reflective information. Wiring should be tested by another mean IMHO.

Why didn't we add getRemainingTokens and getRetryAfter to LimiterInterface instead?

We should also resolve this question :)

@vasilvestre
Copy link
Author

@wouterj I think Nicolas is right here, the Limit object isn't that useful. WDYT?

@wouterj
Copy link
Member

Why didn't we addgetRemainingTokens andgetRetryAfter toLimiterInterface instead?

Please note that I'm far from experienced with rate limiting (I just wanted login throttling), but my ideas on this:

  • This state information differs per$key, so it's not a real clean getter but insteadgetRemainingTokens($key) andgetRetryAfter($key). Not a deal breaker, just seems a bit code-smelly.
  • This state information can be changed at any moment by another process - which is whyconsume() is protected by a lock. My idea was that it's the most useful to provide information on the state at the moment the decision was made - not at an arbitrary moment in time.
  • Returning this object fromconsume() has the advantage of IO blocking the retry time in the process. E.g:
    while (!($limit =$limiter->consume())->isAccepted()) {$limit->wait();}
vasilvestre reacted with thumbs up emoji

@nicolas-grekas
Copy link
Member

Thanks@wouterj, that makes sense!
We now need to settle aboutgetStrategy andgetMetadata (which I'd currently prefer removing, please explain why if you disagree), then good to go on my side.

@wouterj
Copy link
Member

wouterj commentedSep 29, 2020
edited
Loading

We now need to settle about getStrategy and getMetadata (which I'd currently prefer removing, please explain why if you disagree), then good to go on my side.

I don't have a strong opinion on this one (and not a real use-case). If you think it's better to remove it, let's do that. It's easier to add it if we find good use-cases than to remove it in the future :)

@vasilvestre
Copy link
Author

The ci check fail happens on a code part I haven't modified, what to do ?

@fabpot
Copy link
Member

Thank you@vasilvestre.

vasilvestre reacted with hooray emojivasilvestre reacted with rocket emoji

@fabpotfabpot merged commitaa66149 intosymfony:masterSep 30, 2020
fabpot added a commit that referenced this pull requestSep 30, 2020
…ded IO blocking (wouterj)This PR was merged into the 5.2-dev branch.Discussion----------[RateLimiter] Call all compound limiters on failure and added IO blocking| Q             | A| ------------- | ---| Branch?       | master| Bug fix?      | yes| New feature?  | no| Deprecations? | no| Tickets       | -| License       | MIT| Doc PR        | -Two small improvements that I missed in#38257:* Added `wait()` method, to be able to do logic like this:   ```php   while (!($limit = $limiter->consume())->isAccepted()) {       $limit->wait();   }   ```* Fixed the `CompoundLimiter` to always call all limiters (even when one already blocks). This was the original behavior of the compound limiter and imho the best behavior (at least, it's always consistent and doesn't rely on the order of limiters - which is unsorted).Commits-------0279f88 Call all compound limiters on failure and added IO blocking
@nicolas-grekasnicolas-grekas modified the milestones:next,5.2Oct 5, 2020
@fabpotfabpot mentioned this pull requestOct 5, 2020
@NikosDevPhp
Copy link

@wouterj We have a rate limiting for failed attempts let's say 3 for a sliding window of 1 hour.
I want to know preemptively if the request should go through, reducing load on the server for let's say a public api (my use case is grpc call where 429 equivalent response code is not respected).
Could I use $rateLimiter->consume(0) to get the current state to prohibit further code execution if remainingAttempts is zero(0). (In this case isAccepted is always set to true)

Then if the attempt is failed I would consume(1).

Is this how it is supposed to work?

@NikosDevPhp
Copy link

Anyone able to respond to this one? I think it's a pretty valid case to get the current state

@vasilvestre
Copy link
Author

@wouterj We have a rate limiting for failed attempts let's say 3 for a sliding window of 1 hour. I want to know preemptively if the request should go through, reducing load on the server for let's say a public api (my use case is grpc call where 429 equivalent response code is not respected). Could I use $rateLimiter->consume(0) to get the current state to prohibit further code execution if remainingAttempts is zero(0). (In this case isAccepted is always set to true)

The RateLimiter evolved a LOT since this PR. AFAIK the documentation says to consume 1 and if an exception is thrown then you don't have load on your server.

Pseudo code fromdocumentation:

class ApiController extends AbstractController{    public function heavyProcess(Request $request, RateLimiterFactory $anonymousApiLimiter): Response    {        $limiter = $anonymousApiLimiter->create($request->getClientIp());        if (false === $limiter->consume(1)->isAccepted()) {            throw new TooManyRequestsHttpException();        }        // you can also use the ensureAccepted() method - which throws a        // RateLimitExceededException if the limit has been reached        // $limiter->consume(1)->ensureAccepted();        // Heavy process    }}

Anyone able to respond to this one? I think it's a pretty valid case to get the current state

Maybe you could make a new issue or find wouterj on a social network.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@kbondkbondkbond left review comments

@nicolas-grekasnicolas-grekasnicolas-grekas left review comments

@wouterjwouterjwouterj approved these changes

@fabpotfabpotfabpot approved these changes

+1 more reviewer

@GuikingoneGuikingoneGuikingone left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Projects

None yet

Milestone

5.2

Development

Successfully merging this pull request may close these issues.

[RateLimiter] Expose the limiter state

8 participants

@vasilvestre@fabpot@kbond@nicolas-grekas@wouterj@NikosDevPhp@Guikingone@carsonbot

[8]ページ先頭

©2009-2025 Movatter.jp