Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7.9k
CI: Ensure code coverage is always uploaded#28164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
OK, this looks good; coverage is being uploaded even on failure on all the runs. I'm not sure if we should upload Cygwin coverage, since it's not done on all PRs too. Maybe there's a way to keep that separate. Also, the script to remove passing images is now working, though I see it doesn't really clean up as much as it can. One other thing is that it appears that Azure only uploads on Linux; it is using the Bash uploader, which is supposed to be broken, but still works there. However, it stopped working on other platforms. I think we would need to set up a token in order to get the new uploader working. |
I think preventing cygwin from uploading is probably fine. If having it would cause Azure may be worth either fixing (could be here or in a followup) or removing coverage entirely (if its not getting us additional coverage/hasn't been working anyway). I would leave any adjustments to the upload cleaner for a separate PR. |
If code coverage is not uploaded on failure, then overall coverage cansuffer mysteriously (e.g., if a mac system failed, then it would appearas if `backend_macosx.py` was not tested). Codecov would then showseveral "indirect changes" that are spurious.On GitHub Actions, if you don't have any status check function in the`if` entry, then it is treated as `success() && (whatever else youhad)`, so put in an explicit check.Azure also had no condition, so defaulted to only-on-success.Cygwin is only run on `main` and sometimes on PRs, so disable codecoverage reporting from it. That would cause random "reductions" incoverage for all PRs that don't run it (which is most of them.)Also, fix the image-cleanup script, which would only run on successinstead of failure, where it would be useful.
45d5dc0 to25847ddCompareI removed Cygwin, as it would just cause extra flapping, since only |
Agree about both not uploading from cygwin and defering fixing uploading on non-linux azure to a follow on PR. |
3ee509d intomatplotlib:mainUh oh!
There was an error while loading.Please reload this page.
Owee, I'm MrMeeseeks, Look at me. There seem to be a conflict, please backport manually. Here are approximate instructions:
And apply the correct labels and milestones. Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon! Remember to remove the If these instructions are inaccurate, feel free tosuggest an improvement. |
…3.9.xBackport PR#28164 on branch v3.9.x (CI: Ensure code coverage is always uploaded)
PR summary
If code coverage is not uploaded on failure, then overall coverage can suffer mysteriously (e.g., if a mac system failed, then it would appear as if
backend_macosx.pywas not tested). Codecov would then show several "indirect changes" that are spurious.On GitHub Actions, if you don't have any status check function in the
ifentry, then it is treated assuccess() && (whatever else you had), so put in an explicit check.Azure also had no condition, so defaulted to only-on-success.
Also, fix the image-cleanup script, which would only run on success instead of failure, where it would be useful.
PR checklist