I think this could be a nice addition for 2.7, the default cache is useful in 80% of apps but when the size of catalogue became more than 10 MB it's consumes too much memory.
So the solution is to use DoctrineMessageCache, I tested with a catalogue(~100 mb) and the result look really interesting:

#config.ymlframework:translator:{ fallbacks: [en], cache: translation.cache.doctrine }doctrine_cache:providers:translation_cache:memcached:servers:localhost:11211

#services.ymltranslation.cache.doctrine:class:Symfony\Component\Translation\DoctrineMessageCachearguments:[%kernel.debug%, @translation_cache]

https://blackfire.io/profiles/compare/82c496fe-200a-451d-a2c8-e2fd458f0193/graph

/cc@fabpot

@Tobion can you review this PR ?

ping @symfony/deciders

any feedback, I really want it to be available in 2.7 :)

👍

I tend to agree with@mpdude

anyway the main thing here is to provide a doctrine cache for translation, what I didn't like for ConfigCache is it mainly related to resource file.

any decision about this feature the current implementation look good to me :),
the#14178 look nice but didn't solve the issue at all,
another thing to consider here is that users can use the cache without the config component deps.

#14178 tries to solve a completely different problem.

But even with sticking to the current way of usingConfigCache, we should be able to improve theTranslator by possibly having several caches (loading smallerMessageCatalogues) and/or improved memory efficiency by keeping messages in a shared memory cache (which this PR is about, if I understand it correctly).

If I got the code right (not sure about that), your goal is to keep messages in a key-value-store type cache.

That way, theTranslator does not have to load a possibly hugeMessageCatalogue (or even several of them) from theConfigCache only to return a single message. It might be able to retrieve the message directly from the key-value-cache.

Also, the message catalogue will mostly be kept in shared memory and reduce the memory footprint for a single PHP process. With the current implementation, the entire Catalogue will be read from theConfigCache into per-request memory.

So, if this understanding is correct, can't we find a way to make the Translator (key-value-)cache aware and use a lookup strategy based on (locale, domain, message-id) without having to touch theConfigCache related parts at all?

Here's the steps such a Translator would need to do:

1. Figure out if the catalogue is still fresh/valid

   For this, we could stick with the currentConfigCache validation. If we detect that we need to rebuild theConfigCache, we subsequently also need to copy/map the Catalogue (after we loaded it) into the key-value-store/cache.

2. Look into the cache

   Beforerequireing the Catalogue from theConfigCache, check whether the requested (locale, domain, message-id) is available in the (key-value-)cache. If so, return it from there.

3. Cache miss

   If a requested translation cannot be found, it does either not exist or it may be missing because the key-value-cache has been purged.

   We could detect the latter by having an extra key that is written once the cache was loaded. If that key is missing, re-populate the key-value-cache with theMessageCatalogue loaded from theConfigCache (!).

Does that make sense?

@mpdude you understand this topic now better than me :) , below my answers:

If I got the code right (not sure about that), your goal is to keep messages in a key-value-store type cache.

The main think here for me is to move the cache outside of Translator and adding an initial implementation of key-value-store(doctrine/cache).

can't we find a way to make the Translator (key-value-)cache aware

Right I think missed this part even if we use key-value-store we need ``ConfigCache` to make cache aware better.
I work now on this part so I'll fix it soon.

@mpdude Hi,
If we decide to use the Translator (key-value-) we don't need anymore to save cache in file !! and I think the current limitations ofConfigCache is it doesn't support key/value cache.

So what I made now with69225b8 is refresh cache when resources file change.

And to understand how I make the Translator (key-value-)cache aware:

1 - Figure out if the catalogue is still fresh/valid.

isFresh

2 - Look into the cache

check whether the requested (locale, domain, message-id) is available in the (key-value-)cache

3 - Cache miss

If that key is missing, re-populate the key-value-cache with the MessageCatalogue loaded from the Loaders and not ConfigCache !

If you're trying to go withoutConfigCache, you will end up re-implementing the freshness check it provides (for examplehere). You also seem to need theDoctrineMessageCache as a replacement forConfigCache.

My suggestion was to keep theConfigCache part as-is. It's just a file sitting on the disk and does no harm. That way, you can still use the resource validation logic it provides. You will see from withinTranslator when the catalogue has to be rebuilt, that's when the cache-is-fresh check fails.

In other words, theDoctrineCache would just be another caching layer on top ofConfigCache: InTranslator, before you load the entire Catalogue from disk, see if you can find that particular message in the DoctrineCache first.

When you have the Catalogue at hand anyways (because you had to rebuild itor load it because the message was not in DoctrineCache), also save it to the DoctrineCache.

@mpdude well I agree to some extent, but I didn't like the idea of writing cache twice in ConfigCache then DoctrineCache.

@fabpot what do you think?

I'd hold off and do not merge this into 2.7. Note that I didn't review this PR in details just yet, and I only scanned through comments. Forgive me if I'm repeating someone else here.

We need to think of how to organise doctrine integration more carefully and make it consistent across the framework. With the current implementation I don't think that doctrine cache classes belong to the Doctrine bridge, as they're only use is the translator.

Serializer and Validator have their own cache implementations, slightly different from each other - first relies on doctrine/cache directly, the later adapts its interface. This PR introduces yet another approach. All I'm saying we should stop and consider if we can make it a bit more consistent.

The safe way is to split this PR into 2, let's discuss about addingmessage cache interface and once accepted I'll open another one for doctrine cache.