Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork9.7k
[Cache] AddMarshallerInterface allowing to change the serializer, providing a default one that automatically uses igbinary when available#27645
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
d7d91b3 tocfd7492Compare3cc3d37 to6622803Compare| * Serializes a list of values using igbinary_serialize() if available, serialize() otherwise. | ||
| */ | ||
| protectedstaticfunctionserialize(array$values, ?array &$failed):array | ||
| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Why is it placed into AbstractTrait? It didn't used not by AbstractTrait nor by AbstractCache. It would be surprise to override serialize method and got nothing. Only 3 of 7 AbstractCache subclasses requires serialization: FilesystemCache, PdoCache and RedisCache.
| */ | ||
| protectedfunctiondoSave(array$values,$lifetime) | ||
| { | ||
| if (!$values =parent::serialize($values,$failed)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
memcache API allows to store plain PHP vars. And its serialization method can be configured in connection strings. Do we need double serialization here?
| thrownew \DomainException('Failed to unserialize cached value'); | ||
| thrownew \DomainException(error_get_last() ?error_get_last()['message'] :'Failed to unserialize cached value'); | ||
| }catch (\Error$e) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Do we need that complex logic here? Why simple call to unserialization function and result check would not be enough?
| 4.2.0 | ||
| ----- | ||
| * automatically use igbinary when available, with graceful error handling |
palex-fptJun 20, 2018 • edited by nicolas-grekas
Loading Uh oh!
There was an error while loading.Please reload this page.
edited by nicolas-grekas
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
IMHO it is OK to have meaningful defaults for framework. But it is not OK to force settings for components.
[edit:see answer below]
palex-fpt commentedJun 20, 2018
Opps. I'm late with review. Just checked our issue board: there was tickets when we was forced to revert igbinary serialization for some components. Having ability to select serialization method without rebuilding operation environment is critical. |
nicolas-grekas commentedJun 20, 2018
💯 that's exactly what is provided here: when data is read from the backend, we adaptatively unserialize it using igbinary or native-serialize depending on the payload. Of course, if igbinary is not loaded but the data had been serializing with it, we report a cache miss - there is no other way around.
That's actually required to provide the auto-adaptative behavior just described above. I'd recommend tonot use igbinary at the extension level for this reason also. Maybe even deprecate having it enabled?
What's the downside? If there is none, better keep this as an internal detail. If you have igbinary, there is no downside to using it, and since this extension is opt-in, you can always remove it. |
36820be to9a5c2ffCompare| $namespaceVersion =2; | ||
| try { | ||
| foreach ($this->doFetch(array('@'.$this->namespace))as$v) { | ||
| foreach ($this->doFetch(array('/'.$this->namespace))as$v) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Changing this creates a separate set of keys when using 3.4 to 4.1 vs 4.2 & up. This is required for Memcached adapters because of the added serialization that was delegated to the extension before. Not doing so breaks loading values using 3.4 with a cache that has been populated with 4.2 (can happen during migrations.) Let's be safe. Since versioning is enabled only for Memcached by default, this is nicely selective.
palex-fpt commentedJun 20, 2018
This fallback is required due fact that from application point 'nothing' changed. When I reconfigure application cache (change file to memcache, change directory or change serialization mode) I'm pretty OK with repopulating cache. And I prefer fast fail when igbinary is configured for application but is not available on deployment node, than slow running application with hard to detect reasons.
It is now igbinary is faster. It can be changed. When some other component Well, can we have selectable modes: adaptive, igbinary_only, php_only? |
nicolas-grekas commentedJun 21, 2018 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
No way: that explicitly conflicts with PSR-6 spirit and wordings (for good reasons IMHO.)
Done with a new constructor argument: |
MarshallerInterface allowing to change the serializer, providing a default one that automatically uses igbinary when availablenicolas-grekas commentedJun 22, 2018
Now with marshaller injection instead of boolean option. |
99f0534 tofd8d65cCompareToflar commentedJun 25, 2018
Just a general question: I haven't checked out the code in depth but it seems to me like the |
…lon" controller service refs (nicolas-grekas)This PR was merged into the 4.1 branch.Discussion----------[TwigBundle] bump lowest deps to fix issue with "double-colon" controller service refs| Q | A| ------------- | ---| Branch? | 4.1| Bug fix? | yes| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets |#27645| License | MIT| Doc PR | -Bumping http-kernel to ~4.1 and conflicting with framework-bundle <4.1.The rest is transitivity.Commits-------f75723f [TwigBundle] bump lowest deps to fix issue with "double-colon" controller service refs
nicolas-grekas commentedJun 28, 2018
ping @symfony/deciders |
| * | ||
| * @author Nicolas Grekas <p@tchwork.com> | ||
| */ | ||
| class DefaultMarshallerimplements MarshallerInterface |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Maybe this has been asked, but why do we have aDefaultMarshaller with some internal logic in all methods to differentiate between PHP and IgBinary ... instead of defining a PhpMarshaller and IgBinaryMarshaller (moreover given that we define a MarshallerInterface).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Because there is only one implementation of MarshallerInterface: DefaultMarshaller, which auto-adaptatively uses serialize() or igbinary_serialize().
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
But that looks to me like creating aDefaultMailer which makesif($use_smtp) { ... } elseif ($use_gmail) { ...} inside its methods. In other parts of the framework we define 1 interface and multiple different implementation classes. Here it looks like we have multiple implementations inside a single class. It looks confusing to me, sorry.
nicolas-grekasJun 28, 2018 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
@javiereguiluz it's not the same. Here, by dealing with both serialization methods (and there are none others for PHP objects), we provide an important feature: auto-adaptative unserialization, providing safe forward and backward compatibility at the data level. This is not possible without having something that knows about both formats.
teohhanhuiJun 28, 2018 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
IgBinaryMarshaller could take PhpMarshaller as a fallback marshaller (which is optional), no?
teohhanhuiJun 28, 2018 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
What is the purpose of the MarshallerInterface then if there are no other reasonable implementations?
nicolas-grekasJun 28, 2018 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks to the interface, the serialized string itself can still be processed, e.g compressed and/or encrypted.
One could also build a serializer that would be dedicated to and optimized for very specific data structures (eg only numbers, etc.), etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hmm... But actually it's not true that igbinary is the only serialization format that might benefit from a fallback to PHP serialization. There are / were other contenders like msgpack (MessagePack) and Protobuf, both of which have PHP extensions.
nicolas-grekasJun 28, 2018 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Nice, another reason to have the interface. Let's keep the topic: should we split the default marshaller in two. I think we would do a misfeature by doing so: things would be more complex to understand, more complex to wire, less capable, and less performant while in a critical code path. Would the individual envisioned marshallers empower our users in comparison to these downsides? Not at all in my opinion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think you're right, and those other serializer formats are not PHP-specific, so it wouldn't make sense for them to fallback to PHP serialization, actually. 😆
joelwurtz commentedJun 28, 2018
What will happen if someone install the igbinary extension on their server but already have a lot of cache ? |
nicolas-grekas commentedJun 28, 2018 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
@joelwurtz see test cases: native |
| { | ||
| private$allowIgbinarySerialize =true; | ||
| publicfunction__construct(bool$allowIgbinarySerialize =true) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
A way to force only igbinary (disable the PHP serialization fallback)?
teohhanhuiJun 28, 2018 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
@palex-fpt said:
I prefer fast fail when igbinary is configured for application but is not available on deployment node, than slow running application with hard to detect reasons.
To which@nicolas-grekas you replied:
No way: that explicitly conflicts with PSR-6 spirit and wordings (for good reasons IMHO.)
But if the fail-fast behavior means a cache miss, that's not violating PSR-6 in any way. (Bad idea, as it'll result in exactly "slow running application with hard to detect reasons".) It could fail-fast by throwing an exception in the constructor when igbinary is forced but the extension is not available. That would not violate PSR-6.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Correct. Not convinced yet. Really worth it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
TBH, I prefer fail-fast feature over auto-adaptive feature. Reconfigured application cache can and should be warmed up. But having spoiled node that writes to shared cache in php format and forces all other properly configured nodes to use slow method is not good. If someone concerning about preserving old cache data he can use workaround with distinct namespaces and chain cache instances with igbinary an php-serialize.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Here we are: the default mode is auto-adaptative, but if you passtrue, you'll get an exception when igbinary is not loaded.
73ea8dd to3a7517dComparenicolas-grekas commentedJul 1, 2018
Any other comment here? |
3cff315 to9c328c4Compare…providing a default one that automatically uses igbinary when available
fabpot commentedJul 9, 2018
Thank you@nicolas-grekas. |
…he serializer, providing a default one that automatically uses igbinary when available (nicolas-grekas)This PR was merged into the 4.2-dev branch.Discussion----------[Cache] Add `MarshallerInterface` allowing to change the serializer, providing a default one that automatically uses igbinary when available| Q | A| ------------- | ---| Branch? | master| Bug fix? | no| New feature? | yes| BC breaks? | no| Deprecations? | yes| Tests pass? | yes| Fixed tickets |#19895| License | MIT| Doc PR | -With this PR, when igbinary is available, it is automatically used to serialize values.This provides faster and smaller cache payloads.The unserializing logic is autoadaptative:- when an igbinary-serialized value is unserialized but the extension is missing, a cache miss is triggered- when a natively-serialized value is unserialized and the extension is available, the native `unserialize()` is usedPing@palex-fpt since you provided very useful comments on the topic and might be interested in reviewing here also.Commits-------9c328c4 [Cache] Add `MarshallerInterface` allowing to change the serializer, providing a default one that automatically uses igbinary when available
Koc commentedJul 15, 2018
Is it possible decoration for serialization? Create |
Uh oh!
There was an error while loading.Please reload this page.
With this PR, when igbinary is available, it is automatically used to serialize values.
This provides faster and smaller cache payloads.
The unserializing logic is autoadaptative:
unserialize()is usedPing@palex-fpt since you provided very useful comments on the topic and might be interested in reviewing here also.