Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork9.7k
[PropertyAccess] Major performance improvement#16294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
dupuchba commentedOct 20, 2015
👍 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
persisting the cache across requests does not make any sense, as your cache key relies onspl_object_hash, which is not shared across requests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Good catch, using the class fully qualified name should work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Is it actually caching values per instance::property or per class::property? Because the first is not really a case you can cover across requests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Actually it's by instance (because ofspl_object_hash) but I suggest to change it by class (it will also improve performances when different objects of the same type are serialized).
stof commentedOct 20, 2015
Using the object hash in the cache key can cause issues if your object is garbage collected and the same hash is reused (which is possible as it is based on the memory location of the object, which is available for reuse). Btw, your benchmark is not relevant for the API platform use case: you are getting the values thousands of times on the same instance. Your API platform loop would be looping over objects, and so use separate instances. And most of your cache would not be usable in this case, because the object hash would be different and so the cache key would be different. Only the PropertyPath cache would enter into effect. I suggest you to update your benchmark. What about caching only the logic relying on the class definition instead of caching the logic depending on the object ? This means that you would only need to check dynamic properties each time (and do it before calling the magic methods to respect the existing order of checks). But most cases would still benefit from the cache, and it would be shareable between all objects of the same class, making it much more likely to be reused (and making it shareable across requests too) |
dunglas commentedOct 20, 2015
@stof I just changed the strategy (updated the PR) before you posted. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This cannot be cached, as it depends on the instance, not on the class
stof commentedOct 20, 2015
@dunglas I suggest you to split this PR in 2 separate PRs:
You really have 2 independent optimizations here |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
return$this->propertyPathCache[$propertyPath] =newPropertyPath($propertyPath);
aitboudad commentedOct 20, 2015
IMO this should be considered as new feature ? |
dunglas commentedOct 20, 2015
Benchmark and Blackfire profile updated. |
dunglas commentedOct 20, 2015
@stof I'll move the |
sescandell commentedOct 20, 2015
Hi, IMO, your benchmarks is not exhaustive. What about performance impact on "non-similar" objects? If I remember well, form component is mainly based on PropertyPath one. Your benchmark perfectly feet for scenarios you described (Serialization), but PropertyPath might be used somewhere else. Maybe it has no impact (or improving it), just guessing. |
stof commentedOct 20, 2015
@sescandell the impact on form would be increased memory usage (because of the cache being kept), but the only difference in the code is an array key lookup each time, which is O(1), and much faster than all the reflection-based logic. And if we store the cache in a persistent cache to reuse it across requests, forms would also benefit from it, as a previous request might have warmed up the cache. |
stof commentedOct 20, 2015
@dunglas your Doctrine cache integration is broken currently anyway: it cannot be merged as is as it would break BC due to the second argument being used already in newer branches. |
stof commentedOct 20, 2015
Btw, I'm not sure that the PropertyAccessor is the right place to put the PropertyPath caching. We have many places instantiating a PropertyPath outside it (for instance the Form and Validator components always create the instance externally and passes it to the PropertyAccessor). We may want to introduce a PropertyPathFactory in 2.8, with several implementations:
This is already what we do for the ChoiceListFactory in 2.7+ for instance. Then, any place needing to create a PropertyPath would rely on the factory (making it an optional argument for BC, and creating a |
stof commentedOct 20, 2015
Btw, looking at the code of the PropertyPath constructor, I found a simpler optimization: the regex used in it can be optimized simply. I will send a PR for it. Then we will see whether your property path cache is still worth it or no. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Same goes here, it might depend on the instance, and not on the class.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
@sescandell there is nothing in this condition depending on$object (except the fact that the case of dynamic properties must have been checked first)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
this is wrong. You need a separate access type for magic methods, because you must check the dynamic properties before calling the magic method (otherwise it changes the current behavior)
dunglas commentedOct 20, 2015
@stof OK with your proposal.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
the reflection part could be cached too, to avoid redoing it.
dunglas commentedOct 20, 2015
@stof problems you raised should be fixed now |
dunglas commentedOct 20, 2015
AppVeyor errors not related. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
You should not cache the not found case.
Imagine the following situation:
$a =newstdClass();$a->a ='a';$b =newstdClass();$b->b ='b';try {$accessor->getValue('b',$a);}catch(...) {...}try {$accessor->getValue('b', $b);} catch(...) {...}
In that situation it will just throw the exception (except that for $b it should work).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It will work with your example:https://github.com/dunglas/symfony/blob/propertyaces-perf/src/Symfony/Component/PropertyAccess/PropertyAccessor.php#L284-L293
Only method names are cached but dynamic properties are always tried.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
👍
stof commentedOct 21, 2015
@dunglas to make the code more readable, the cache building may be extracted to separate private methods |
dunglas commentedOct 25, 2015
@stof done. Did you have the time to take a look at the PropertyPath regex? |
dunglas commentedOct 29, 2015
ping @symfony/deciders |
fabpot commentedOct 30, 2015
Thank you@dunglas. |
This PR was squashed before being merged into the 2.3 branch (closes#16294).Discussion----------[PropertyAccess] Major performance improvement| Q | A| ------------- | ---| Bug fix? | yes| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets |#16179| License | MIT| Doc PR | n/aThis PR improves performance of the PropertyAccess component of ~70%.The two main changes are:* caching the `PropertyPath` initialization* caching the guessed access strategyThis is especially important for the `ObjectNormalizer` (Symfony Serializer) and the JSON-LD normalizer ([API Platform](https://api-platform.com)) because they use the `PropertyAccessor` class in large loops (ex: normalization of a list of entities).Here is the Blackfire comparison:https://blackfire.io/profiles/compare/c42fd275-2b0c-4ce5-8bf3-84762054d31e/graphThe code of the benchmark I've used (with Symfony 2.3 as dependency):```php<?phprequire 'vendor/autoload.php';class Foo{ private $baz; public $bar; public function getBaz() { return $this->baz; } public function setBaz($baz) { $this->baz = $baz; }}use Symfony\Component\PropertyAccess\PropertyAccess;$accessor = PropertyAccess::createPropertyAccessor();$start = microtime(true);for ($i = 0; $i < 10000; ++$i) { $foo = new Foo(); $accessor->setValue($foo, 'bar', 'Lorem'); $accessor->setValue($foo, 'baz', 'Ipsum'); $accessor->getValue($foo, 'bar'); $accessor->getValue($foo, 'baz');}echo 'Time: '.(microtime(true) - $start).PHP_EOL;```This PR also adds an optional support for Doctrine cache to keep access information across requests and improve the overall application performance (even outside of loops).Commits-------284dc75 [PropertyAccess] Major performance improvement
sstok commentedOct 31, 2015
I really like Stofs idea for caching the determined strategy, and caching the property-path string to object 👍 |
dunglas commentedOct 31, 2015
I removed these features from this PR because they cannot be merged in 2.3 but I'll open PR new PRa against 2.8 yo add them back. |
fabpot commentedNov 2, 2015
@dunglas I've just merged 2.3 into 2.7 without including your modifications as it conflicts badly with all the changes made since 2.3. Instead of trying to merge everything, I think it would be better to just create a new PR on 2.7. Can you take care of that? Thanks. |
dunglas commentedNov 2, 2015
Ok I'll do it this week. |
This PR was merged into the 2.3 branch.Discussion----------[PropertyAccess] Fix dynamic property accessing.| Q | A| ------------- | ---| Bug fix? | yes| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets | n/a| License | MIT| Doc PR | n/aFix a bug regarding dynamic properties access introduced by#16294.Commits-------916f9e0 [PropertyAccess] Test access to dynamic properties352dfb9 [PropertyAccess] Fix dynamic property accessing.
… 2.3 (dunglas)This PR was squashed before being merged into the 2.7 branch (closes#16463).Discussion----------[PropertyAccess] Port of the performance optimization from 2.3| Q | A| ------------- | ---| Bug fix? | yes| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets |#16179| License | MIT| Doc PR | n/aPortage of#16294 in the 2.7 branch.Commits-------aa4cc90 [PropertyAccess] Port of the performance optimization from 2.3
webmozart commentedNov 26, 2015
This is awesome, thanks@dunglas!! |
This PR was squashed before being merged into the 2.8 branch (closes#16547).Discussion----------[Serializer] Improve ObjectNormalizer performance| Q | A| ------------- | ---| Bug fix? | no| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets |#16179| License | MIT| Doc PR | n/aCache attributes detection in a similar way than in#16294 for PropertyAccess.As for the PropertyAccess Component, I'll open another PR (in 2.8 or 3.1) allowing to cache these attributes between requests using Doctrine Cache.@Tobion, can you try this PR with your benchmark to estimate the gain?Commits-------683f0f7 [Serializer] Improve ObjectNormalizer performance
This PR was squashed before being merged into the 3.2-dev branch (closes#16838).Discussion----------[PropertyAccess] Add PSR-6 cache| Q | A| ------------- | ---| Bug fix? | no| New feature? | yes| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets | n/a| License | MIT| Doc PR | n/aFollow#16294Commits-------4ccabcd [PropertyAccess] Add PSR-6 cache
This PR improves performance of the PropertyAccess component of ~70%.
The two main changes are:
PropertyPathinitializationThis is especially important for the
ObjectNormalizer(Symfony Serializer) and the JSON-LD normalizer (API Platform) because they use thePropertyAccessorclass in large loops (ex: normalization of a list of entities).Here is the Blackfire comparison:https://blackfire.io/profiles/compare/c42fd275-2b0c-4ce5-8bf3-84762054d31e/graph
The code of the benchmark I've used (with Symfony 2.3 as dependency):
This PR also adds an optional support for Doctrine cache to keep access information across requests and improve the overall application performance (even outside of loops).