Thanks for tackling this! Just one quick idea comes to mind for:

effcf01 : Split the docscrape tests into two test modules...

Have you consideredsplitting while keeping blame? I did this recently inmne-tools/mne-qt-browser#350 (comment) for example and it seems to have worked for preserving blame. But I understand if this doesn't seem worth it here!

Your TODO list looks good to me as well.

Have you considered splitting while keeping blame?

Thanks for the pointer@larsoner , this is definitely worthwhile ! I will give it a shot

AFAICT the test failures are not obviously related: the failures seem to be related to path formatting on windows for the validator.@stefmolin@mattgebert any idea what might be going on here?

(or a dependency change in the action runner).

Good observation - I opened #804 to test whether it was something in the runner environment - everything there runs as expected so it's likely not that.

This is definitely related to Windows paths, but it is failing here and not in main, which makes it seem like it is related to these changes

Indeed that seems to be the case, but I still can't tell how. After looking at this a bit, there are two things I'm confused by:

1. How did these changes affect these test results (having re-looked at everything there doesn't seem to be any direct connection to the changes and thehooks/test_ tests).
2. How did these tests ever pass on windows in the first place? The problems that were tripping it up do indeed seem to be entirely related to windows path rendering... why wasn't this a problem before? FWIW7708ca0 "fixes" the issue by robust-ifying the tests themselves to different platforms. That doesn't answer either of these two questions though.

Normally I'd address these questions bybisect-ing to ID which specific commit caused the issue, but I don't have ready access to a windows machine.

I'm also at a loss for why these changes would cause the tests to suddenly fail on Windows. The only thing I can think of is that the proposed changes don't touch our GitHub Actions, meaning that we are no longer testing any of the sphinx behavior – it's not installed. Maybe sphinx comes with something that changes how the paths behave on Windows? I don't have access to a Windows machine either.

I'm also at a loss for why these changes would cause the tests to suddenly fail on Windows. The only thing I can think of is that the proposed changes don't touch our GitHub Actions, meaning that we are no longer testing any of the sphinx behavior – it's not installed. Maybe sphinx comes with something that changes how the paths behave on Windows? I don't have access to a Windows machine either.

I believe I've figured it out: the issue is thetest/hooks test werenever being run in CI. See#653 , specificallye728f61 and the associated action logs, for proof. Basically the tests are run with the--pyargs flag which runs tests from the installed package, but thetests/hooks tests were not included in the package, so they were never run.

I suspect the reason they started being run with this PR was the introduction ofconftest.py, which IIRC can subtly influence test collection¹.

Anyways - I will port the test fixes in7708ca0 over to#653 to fix all of that separately so we can remain focused on the sphinx-dep changes in this PR.

@rossbar Sorry I haven't had time recently to get on top of this, and am just having a quick look at this now!

I noticed the hook tests never being called (see#622 (comment) ) a while ago when I was writing#622

I did a few things to rectify this, not sure if this is the same as what you've now done in#653. You can check thefile changes.

• I had to add an explicit call in the CI for thenumpydoc/tests/hooks directory to ensure it runs.
• The other thing in the hook tests is the use of delimeters that don't match the Windows filepath. Added aos.replace("/", os.sep)

#622 is also now ready to integrate btw since we move to Python 3.10+.

Now that#653 is in and testing gremlins are all chased down I intend to keep pushing this!

The first order of business was taking up@larsoner 's excellent suggestion about preserving history in the newtest_docscrape_sphinx.py file. Ifollowed the linked procedure and indeed was able to verify withgit blame that the history is preserved as expected.

Unfortunately the "split the files" commit(s) are relatively early in the stack by necessity, so the only way I could get these changes up was with a rebase+force-push, which screws up the linked commit summary in the original post. I'll go ahead and update that to reflect the new hashes.

The next order of business is to work throughthe checklist... I will share results for each task once I've had a chance to work through it.

This is awesome,@rossbar! I useNumpyDocString for the early prototype myst API generator, and it would be great not to have to bring in Sphinx. Same in CI etc.