NotificationsYou must be signed in to change notification settings
Fork7.9k
Star21.3k

Circle: Build docs in parallel.#20718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

tacaswell merged 2 commits intomatplotlib:mainfromQuLogic:circle-parallel

Aug 9, 2022

Merged

Circle: Build docs in parallel.#20718

tacaswell merged 2 commits intomatplotlib:mainfromQuLogic:circle-parallel

Aug 9, 2022

Conversation

Copy link

Member

PR Summary

We are supposedly on medium Docker VMs, which offer 2 vCPUs, so we should try to build in parallel.

PR Checklist

[n/a] Has pytest style unit tests (andpytest passes).
IsFlake 8 compliant (runflake8 on changed files to check).
[n/a] New features are documented, with examples if plot related.
Documentation is sphinx and numpydoc compliant (the docs shouldbuild without error).
[n/a] Conforms to Matplotlib style conventions (installflake8-docstrings and runflake8 --docstring-convention=all).
[n/a] New features have an entry indoc/users/next_whats_new/ (follow instructions in README.rst there).
[n/a] API changes documented indoc/api/next_api_changes/ (follow instructions in README.rst there).

QuLogic added Documentation Build labels

Jul 22, 2021

timhoffm approved these changes

Jul 22, 2021

View reviewed changes

Copy link

Member

jklymak commentedJul 22, 2021

Looks like the build ran out of memory?

jklymak marked this pull request as draft

July 27, 2021 13:53

jklymak added the status: needs revision label

Jul 27, 2021

QuLogic force-pushed thecircle-parallel branch from049af8a to1162ac7Compare

August 4, 2022 21:12

Copy link

MemberAuthor

QuLogic commentedAug 4, 2022•
edited
Loading

On recent Circle builds, I noticed it said we have access to a higher resource class, so I bumped it up tolarge, which will hopefully mean it won't run out of RAM this time, and will be a little bit faster with double the vCPU.

QuLogic added2 commits

August 4, 2022 22:53

Circle: Increase resource class to large

9ac36f3

This doubles the vCPU and RAM, which will hopefully be enough to run inparallel.

Circle: Build docs in parallel.

afef2e6

We are now on large Docker VMs, which offer 4 vCPUs, so we should try tobuild in parallel.

QuLogic force-pushed thecircle-parallel branch from1162ac7 toafef2e6Compare

August 5, 2022 02:54

Copy link

MemberAuthor

QuLogic commentedAug 5, 2022

Re-reading the resource class docs again, it looks like the system really has 32 cores, and since I passed-jauto, it probably was running in 32 processes, which is likely to OOM on any amount of RAM. Testing locally, single-core takes 2.4GB, 2-process takes 2.5GB, 3-process takes 2.65GB, and 4-process takes 3.1GB. The times are 19.6min, 17.1min, 15.7min, and 15.3min, respectively. Unfortunately, sphinx-gallery takes half the time and doesn't run in parallel, so we can't save too much.

But we might as well save the 5 minutes by going to-j4.