PR Summary

NumPy dropped these builds first in 1.21.2 (Aug 15, 2021) for Python 3.10 only, and for all builds in 1.22 (Dec 31, 2021).
SciPy dropped these builds first in 1.8.0 (Feb 5, 2022) for Python 3.10 only, and for all builds in 1.9.2 (Oct 8, 2022).

Also, simplify workflow file to remove some duplication.

Theoretically we could join all theBuild wheels for CPython 3.X steps, but I left them separate so that there wouldn't be a timeout for a single step taking too long (though I don't think that'd happen).

PR Checklist

Documentation and Tests

• [n/a] Has pytest style unit tests (andpytest passes)
• Documentation is sphinx and numpydoc compliant (the docs shouldbuild without error).
• [n/a] New plotting related features are documented with examples.

Release Notes

• [n/a] New features are marked with a.. versionadded:: directive in the docstring and documented indoc/users/next_whats_new/
• API changes are marked with a.. versionchanged:: directive in the docstring and documented indoc/api/next_api_changes/
• Release notes conform with instructions innext_whats_new/README.rst ornext_api_changes/README.rst