Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[FrameworkBundle] Dont store cache misses on warmup#40645

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@Nyholm
Copy link
Member

@NyholmNyholm commentedMar 30, 2021
edited
Loading

QA
Branch?4.4
Bug fix?yes
New feature?no
Deprecations?no
TicketsFix#38694
LicenseMIT
Doc PRsymfony/symfony-docs#15172

When we are warming the annotation cache, we are reading all annotation into anArrayAdapter. When we are done we move the values to aPhpArrayAdapter to store them in a file.

@Seldaekfound out that when you are using a custom constraint with aSymfony\Component\Validator\Constraints\Callback, there is a strange error like:

Can use "yield from" only with arrays and Traversables

That is because theClosure in theSymfony\Component\Validator\Constraints\Callback cannot be serialised and saved to cache. But since theArrayAdapter is alsostoring misses as null, the null values are understood as real values.

When all values are moved to thePhpArrayAdapter and we ask the cache for a value (via Doctrine'sCacheProvider), it will returnnull as a value instead offalse as a cache miss. Andnull is not something one could "yield from".

@Nyholm
Copy link
MemberAuthor

@carsonbot could you find me a reviewer please?

@carsonbot
Copy link

I'm sorry. I could not find any suitable reviewer.

@Nyholm
Copy link
MemberAuthor

Oh. I guess it was too long ago@dmaicher wrote this class =)

David, do you mind having a look at this PR?

@Seldaek
Copy link
Member

Thanks for the quick fix :)

If I understand correctly though, this means using lambdas like that will prevent caching of the resolved config and it'll reload on every request needing it. I guess it's thus worth avoiding lambda callbacks..

@Nyholm
Copy link
MemberAuthor

Yes, that is correct.

@Nyholm
Copy link
MemberAuthor

I updated the docs with this observation:symfony/symfony-docs#15172

@nicolas-grekas
Copy link
Member

nicolas-grekas commentedMar 31, 2021
edited
Loading

Storing misses is an important part of the performance provided by system caches.
We should look for another solution if possible.

@Nyholm
Copy link
MemberAuthor

Nyholm commentedMar 31, 2021
edited
Loading

Yes, but these "misses" are interpreted as "hits". A "hit" with null is not the same as a miss. There should never be "null" stored inthis cache.

I have two alternative solutions. The first would be to find a way to serialize a\Closure. The second one is to throw an exception on cache warmup when you fail to save a value in cache.

@nicolas-grekas
Copy link
Member

nicolas-grekas commentedMar 31, 2021
edited
Loading

The test case passes even without the change if I didn't mess up :)

Yes, but these "misses" are interpreted as "hits". A "hit" with null is not the same as a miss. There should never be "null" stored in this cache.

For system caches, aka the ones sourced by the code itself, we can do this - cache misses.
We built this logic because it saves CPU cycles, eg you don't want to read the annotations again on methods that don't have any annotation.

I have two alternative solutions. The first would be to find a way to serialize a \Closure. The second one is to throw an exception on cache warmup when you fail to save a value in cache.

Why a closure ends up in the cache? That's the main question I have here indeed. An annotation should not lead to a closure because then of course it cannot be cached - while the source is a plain descriptive annotation.

The second solution would be a major regression actually.

Can you try this patch instead? It tries to not store misses for failures-to-save().

--- a/src/Symfony/Component/Cache/Traits/ArrayTrait.php+++ b/src/Symfony/Component/Cache/Traits/ArrayTrait.php@@ -145,6 +145,7 @@ trait ArrayTrait             try {                 $serialized = serialize($value);             } catch (\Exception $e) {+                unset($this->values[$key]);                 $type = \is_object($value) ? \get_class($value) : \gettype($value);                 $message = sprintf('Failed to save key "{key}" of type %s: ', $type).$e->getMessage();                 CacheItem::log($this->logger, $message, ['key' => $key, 'exception' => $e]);

@Nyholm
Copy link
MemberAuthor

The test case passes even without the change if I didn't mess up :)

You did mess up =)

❯ ./phpunit src/Symfony/Bundle/FrameworkBundle/Tests/CacheWarmer #!/usr/bin/env phpPHPUnit 9.5.3 by Sebastian Bergmann and contributors.Testing /src/Symfony/Bundle/FrameworkBundle/Tests/CacheWarmer....F..............                                               19 / 19 (100%)Time: 00:00.091, Memory: 16.00 MBThere was 1 failure:1) Symfony\Bundle\FrameworkBundle\Tests\CacheWarmer\AnnotationsCacheWarmerTest::testWarmupRemoveCacheMissesFailed asserting that actual size 3 matches expected size 2./src/Symfony/Bundle/FrameworkBundle/Tests/CacheWarmer/AnnotationsCacheWarmerTest.php:149

Why a closure ends up in the cache?

Because we cache an object ofSymfony\Component\Validator\Constraint\Callback. That object is given a closure by the user in its constructor.

Can you try this patch instead? It tries to not store misses for failures-to-save().

I think that would work. But the end result would be the same. We would reread the annotations at runtime.
Note that we still cache the empty array. Which means "we have read the annotations, and there weren't any".

@nicolas-grekas
Copy link
Member

You did mess up =)

I confirm :P

Because we cache an object of Symfony\Component\Validator\Constraint\Callback. That object is given a closure by the user in its constructor.

I just read the doc PR, nice one.

I think that would work. But the end result would be the same. We would reread the annotations at runtime.

True, but we would still store misses, and this is important for the other system caches.

Note that we still cache the empty array. Which means "we have read the annotations, and there weren't any".

OK great, then I suppose other system caches need this. Annotations are not the only system caches, but the current patch changes the behavior for all of them.

Nyholm reacted with thumbs up emoji

javiereguiluz added a commit to symfony/symfony-docs that referenced this pull requestMar 31, 2021
… (Nyholm)This PR was squashed before being merged into the 4.4 branch.Discussion----------[Validator] Add warning about closure not being cachableFollow up fromsymfony/symfony#40645The annotation cache will not work with `Closure`.If I understand correctly, it is not possible to have external dependencies in your custom validator AND get your annotation cached. You have to chose one or the other.Commits-------ebddb78 [Validator] Add warning about closure not being cachable
Comment on lines 82 to 85
$values =array_filter($values,function ($val) {
return$val !==null;
});

// make sure we don't cache null values
parent::warmUpPhpArrayAdapter($phpArrayAdapter,array_filter($values));
parent::warmUpPhpArrayAdapter($phpArrayAdapter,$values);
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

array_filter($values) will also remove everything that isempty.

https://3v4l.org/0m3In

There is no test for this code and I am not sure whynull should be removed. I can only assume it is because of the callback =)

@Nyholm
Copy link
MemberAuthor

I was wrong patchingAbstractPhpFileCacheWarmer. What I really wanted to do is to only modify the behaviour of theAnnotationCacheWarmer.

Can you try this patch instead? It tries to not store misses for failures-to-save().

This is not reliable. If a user does something like this, the bug will occur again.

$i =$cache->getItem('foo');$i->set('data');$cache->save($i);// Save failed for some reason.// ...$cache->getItem('foo');

@Nyholm
Copy link
MemberAuthor

The PR is updated

Nyholm added a commit that referenced this pull requestMar 31, 2021
…pter (nicolas-grekas)This PR was merged into the 4.4 branch.Discussion----------[Cache] skip storing failure-to-save as misses in ArrayAdapter| Q             | A| ------------- | ---| Branch?       | 4.4| Bug fix?      | yes| New feature?  | no| Deprecations? | no| Tickets       |Fix#38694| License       | MIT| Doc PR        | -In addition to#40645Commits-------ba66987 [Cache] skip storing failure-to-save as misses in ArrayAdapter
@dmaicher
Copy link
Contributor

Oh. I guess it was too long ago@dmaicher wrote this class =)

David, do you mind having a look at this PR?

I think my work on that was related to#23558

But now I see you changed the PR anyway 😊

@nicolas-grekasnicolas-grekasforce-pushed the4.4-annotation-cache-warmer branch fromd249500 to27a22b3CompareApril 1, 2021 10:17
@nicolas-grekas
Copy link
Member

Thank you@Nyholm.

@nicolas-grekasnicolas-grekas merged commitcc7d126 intosymfony:4.4Apr 1, 2021
@NyholmNyholm deleted the 4.4-annotation-cache-warmer branchApril 6, 2021 09:47
This was referencedMay 1, 2021
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@nicolas-grekasnicolas-grekasnicolas-grekas approved these changes

@chalasrchalasrchalasr approved these changes

Assignees

No one assigned

Projects

None yet

Milestone

4.4

Development

Successfully merging this pull request may close these issues.

6 participants

@Nyholm@carsonbot@Seldaek@nicolas-grekas@dmaicher@chalasr

[8]ページ先頭

©2009-2025 Movatter.jp