Unfortunately, the way thatstubtest outputs its error messages makes it difficult to post a message onboth the.py and.pyi stub, so we decided to take the full message and post it only on one.

As a test, I have made PR in my repoQuLogic#30 which started from the previously-unfixed#25542, and then I also added a commit with multiple argument mismatches to confirm thatstubtest didn't output some different format.

Results are at the bottom ofhttps://github.com/QuLogic/matplotlib/pull/30/files

PR Checklist

Documentation and Tests

[n/a] Has pytest style unit tests (andpytest passes)
[n/a] Documentation is sphinx and numpydoc compliant (the docs shouldbuild without error).
[n/a] New plotting related features are documented with examples.

Release Notes

[n/a] New features are marked with a.. versionadded:: directive in the docstring and documented indoc/users/next_whats_new/
[n/a] API changes are marked with a.. versionchanged:: directive in the docstring and documented indoc/api/next_api_changes/
[n/a] Release notes conform with instructions innext_whats_new/README.rst ornext_api_changes/README.rst

QuLogic added topic: typing CI: testingCI configuration and testing labels

Mar 31, 2023

QuLogic added this to thev3.8.0 milestone

Mar 31, 2023

Copy link

Member

ksunden commentedMar 31, 2023•
edited
Loading

The case that may need to be looked at a bit is the missing stub case, as that does not have a stub path to parse.

Edit: on second thought, it may give the path of the parent object or just line 0 of the module, but still worth a look

Copy link

MemberAuthor

QuLogic commentedMar 31, 2023

Would that be for a new Python file entirely?

Copy link

Member

ksunden commentedMar 31, 2023

No, just for a new method is what I was thinking

Copy link

MemberAuthor

QuLogic commentedApr 1, 2023

Thestubtest output for that case doesn't include a line number, so it doesn't get posted, but the other errors do work fine. I'm not sure if I can get it to fall back to posting on the.py or if it would confuse it.

Copy link

MemberAuthor

QuLogic commentedApr 1, 2023

I also just added an extra function in the stub, but not the Python file, and I see that that doesn't get posted either; it looks like there's a slightly difference in output in that case, which I should be able to cover.

Post stubtest results to GitHub checks

2d05200

QuLogic force-pushed thereviewdog-mypy-stubtest branch fromeff69d5 to2d05200Compare

April 6, 2023 08:06

Copy link

MemberAuthor

QuLogic commentedApr 6, 2023

OK, this is now able to post 1) differences between stubs and runtime; 2) items in stubs but not in runtime. Unfortunately, the case with items in runtime but not stubs cannot be caught, AFAICT.

Seehttps://github.com/QuLogic/matplotlib/pull/30/files

QuLogic mentioned this pull request

Apr 6, 2023

stubtest: More information in concise errorspython/mypy#15011

Open

Copy link

MemberAuthor

QuLogic commentedApr 6, 2023

I opened an issue on stubtest to see if concise error messages could contain a bit more information (linked above). This could simplify this posting process as message would not be multiline.