Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[FrameworkBundle] Make cache:clear "atomic" and consistent with cache:warmup#25117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
fabpot merged 1 commit intosymfony:3.4fromhkdobrev:cache-clear-atomic
Dec 31, 2017
Merged

[FrameworkBundle] Make cache:clear "atomic" and consistent with cache:warmup#25117

fabpot merged 1 commit intosymfony:3.4fromhkdobrev:cache-clear-atomic
Dec 31, 2017

Conversation

@hkdobrev
Copy link
Contributor

@hkdobrevhkdobrev commentedNov 22, 2017
edited by nicolas-grekas
Loading

QA
Branch?master
Bug fix?yes
New feature?no
BC breaks?no
Deprecations?no
Tests pass?yes/no
Fixed tickets-
LicenseMIT
Doc PR-

Here's what happens before/after this PR oncache:clear andcache:clear --no-warmup.

Before PR

cache:clear

  1. rm var/cache_old
  2. Clearing cache invar/cache  <--- This is not in line and not atomic
  3. rm var/cache_warmup
  4. Warming up cache invar/cache_warmup
  5. mv var/cache var/cache_old
  6. mv var/cache_warmup var/cache
  7. rm var/cache_old

cache:clear --no-warmup

  1. rm var/cache_old
  2. Clearing cache invar/cache  <--- This is not in line and not atomic
  3. mv var/cache var/cache_old<--- The old cache dir is completely obsolete in this workflow
  4. rm var/cache_old

After PR

cache:clear

  1. rm var/cache_old
  2. rm var/cache_new
  3. Clearing cache invar/cache_new
  4. Warming up cache invar/cache_new
  5. mv var/cache var/cache_old
  6. mv var/cache_new var/cache
  7. rm var/cache_old

cache:clear --no-warmup

  1. rm var/cache_old
  2. rm var/cache_new
  3. Clearing cache invar/cache_new
  4. mv var/cache var/cache_old
  5. mv var/cache_new var/cache
  6. rm var/cache_old

The main differences:

  • Unify the flows and have each distinct operation only once in the code
  • Clear the cache in the new cache directory and then switch to it atomically instead of clearing in the current one.
  • Always have the cache directory present in the end. It was missing aftercache:clear --no-warmup before.

I think this brings more consistency and is aligned with the present goals of the command as well.

However, I'm not really familiar with the Symfony framework and I might have wrong assumptions.

@nicolas-grekas
Copy link
Member

I like the goal (but didn't really look at the code yet.) I think this could target 3.4, as a bug fix.
Note that this wouldn't make the change atomic, as there is still a small window where the cache folder doesn't exist at all (between steps 5&6.) But there is no portable way to fix it on our side, so that if ppl want to workaround the issue, they should just use a proper deployment script with offline cache creation.
There is one step which can be improved: the new cache folder could get the legacyContainerBlablah directory copied into it before the swap happens, so that a concurrent request that's still using the legacy container could still complete.
This legacy container directory should then be removed, but only after a bit of time, so that these concurrent requests could fully complete.

amici reacted with thumbs up emoji

@nicolas-grekasnicolas-grekas changed the titleMake cache:clear --no-warmup atomic and unified[FrameworkBundle] Make cache:clear "atomic" and consistent with cache:warmupDec 27, 2017
@nicolas-grekasnicolas-grekas changed the base branch frommaster to3.4December 27, 2017 09:36
@nicolas-grekasnicolas-grekas modified the milestones:4.1,.4,3.4Dec 27, 2017

$this->filesystem->rename($realCacheDir,$oldCacheDir);
if ('\\' ===DIRECTORY_SEPARATOR) {
sleep(1);// workaround for Windows PHP rename bug

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This has been introduced in#8767, butlinks to a behavior that I'm not able to reproduce on PHP 5.5/7.1
I propose to remove it, since it slows down DX on Windows, and breaks "atomicity".

hkdobrev reacted with thumbs up emoji
Copy link
Member

@nicolas-grekasnicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@hkdobrev FYI I merged my suggestion to your initial patch.
With this PR, the window for newly started concurrent requests to start with an empty container is very tiny, and the legacy container is also copied, so that already started concurrent requests can still use the legacy service factories.
This reuses the*.legacy files used byKernel, so these will be cleaned up in the end.

@nicolas-grekas
Copy link
Member

Re-qualified as bug on 3.4 btw.

hkdobrev reacted with thumbs up emoji

@Taluu
Copy link
Contributor

Wasn't the--no-warmup option forcache:clear deprecated at some point ? So thatcache:clear wouldnever warmup the cache ?

@nicolas-grekas
Copy link
Member

@Taluu this has been un-deprecated in#23792

Taluu and hkdobrev reacted with thumbs up emoji

@hkdobrev
Copy link
ContributorAuthor

hkdobrev commentedDec 30, 2017
edited
Loading

@nicolas-grekas Thanks for the review!

There is one step which can be improved: the new cache folder could get the legacy ContainerBlablah directory copied into it before the swap happens, so that a concurrent request that's still using the legacy container could still complete.
This legacy container directory should then be removed, but only after a bit of time, so that these concurrent requests could fully complete.

Do you think we should do this as part of this PR?

P.S. Or I just realised you've implemented that suggestion as well, I thought you were talking only about the Windows-specific thing.

@nicolas-grekas
Copy link
Member

@hkdobrev that's already the case, I pushed on your branch.

hkdobrev reacted with thumbs up emoji

@fabpot
Copy link
Member

Thank you@hkdobrev.

@fabpotfabpot merged commit8b88d9f intosymfony:3.4Dec 31, 2017
fabpot added a commit that referenced this pull requestDec 31, 2017
… with cache:warmup (hkdobrev)This PR was merged into the 3.4 branch.Discussion----------[FrameworkBundle] Make cache:clear "atomic" and consistent with cache:warmup| Q             | A| ------------- | ---| Branch?       | master| Bug fix?      | yes| New feature?  | no| BC breaks?    | no| Deprecations? | no| Tests pass?   | yes/no| Fixed tickets | -| License       | MIT| Doc PR        | -Here's what happens before/after this PR on `cache:clear` and `cache:clear --no-warmup`.### Before PR**`cache:clear`**1. `rm var/cache_old`1. Clearing cache in `var/cache`    &nbsp;<i>&lt;--- This is not in line and not atomic</i>1. `rm var/cache_warmup`1. Warming up cache in `var/cache_warmup`1. `mv var/cache var/cache_old`1. `mv var/cache_warmup var/cache`1. `rm var/cache_old`**`cache:clear --no-warmup`**1. `rm var/cache_old`1. Clearing cache in `var/cache`    &nbsp;<i>&lt;--- This is not in line and not atomic</i>1. `mv var/cache var/cache_old`    <i>&lt;--- The old cache dir is completely obsolete in this workflow</i>1. `rm var/cache_old`---### After PR**`cache:clear`**1. `rm var/cache_old`1. `rm var/cache_new`1. Clearing cache in `var/cache_new`1. Warming up cache in `var/cache_new`1. `mv var/cache var/cache_old`1. `mv var/cache_new var/cache`1. `rm var/cache_old`**`cache:clear --no-warmup`**1. `rm var/cache_old`1. `rm var/cache_new`1. Clearing cache in `var/cache_new`1. `mv var/cache var/cache_old`1. `mv var/cache_new var/cache`1. `rm var/cache_old`---The main differences:- Unify the flows and have each distinct operation only once in the code- Clear the cache in the new cache directory and then switch to it atomically instead of clearing in the current one.- Always have the cache directory present in the end. It was missing after `cache:clear --no-warmup` before.I think this brings more consistency and is aligned with the present goals of the command as well.However, I'm not really familiar with the Symfony framework and I might have wrong assumptions.Commits-------8b88d9f [FrameworkBundle] Make cache:clear "atomic" and consistent with cache:warmup
@hkdobrevhkdobrev deleted the cache-clear-atomic branchJanuary 1, 2018 15:57
@fabpotfabpot mentioned this pull requestJan 5, 2018
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@fabpotfabpotfabpot approved these changes

@nicolas-grekasnicolas-grekasnicolas-grekas approved these changes

Assignees

No one assigned

Projects

None yet

Milestone

3.4

Development

Successfully merging this pull request may close these issues.

5 participants

@hkdobrev@nicolas-grekas@Taluu@fabpot@carsonbot

[8]ページ先頭

©2009-2025 Movatter.jp