Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.7k
gh-62432: unittest runner: Exit code 5 if no tests were run#102051
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
As discussed inhttps://discuss.python.org/t/unittest-fail-if-zero-tests-were-discovered/21498/7It is common for test runner misconfiguration to fail to find any tests,this should be an error.Fixes:python#62432
Member
terryjreedy commentedFeb 20, 2023
Tests/macOS is routinely failing; ignore currently. This is a design bug fix, hence enhancement (feature change), as much as I would like to pretend otherwise. |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Misc/NEWS.d/next/Library/2023-02-19-12-37-08.gh-issue-62432.GnBFIB.rst OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
ContributorAuthor
stefanor commentedFeb 20, 2023
Urgh, that removal wasn't intentional, I just hit the re-request review button. |
ContributorAuthor
stefanor commentedApr 26, 2023
@gpshead: Ping? (it's been 2 months) |
ContributorAuthor
stefanor commentedApr 27, 2023
Thank you! |
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 14, 2023
- Either test filtering is used- Sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 14, 2023
- Either test filtering is used- Sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 14, 2023
- Either test filtering is used- Sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 14, 2023
- Either test filtering is used- Sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used via bazel's environment variable. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 14, 2023
- Either test filtering is used- Sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used via bazel's environment variable. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 14, 2023
- Either test filtering is used- Or sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used via bazel's environment variable. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 14, 2023
- Either test filtering is used- Or sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used via bazel's environment variable. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 15, 2023
- Either test filtering is used- Or sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used via bazel's environment variable. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 15, 2023
- Either test filtering is used- Or sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used via bazel's environment variable. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 15, 2023
- Either test filtering is used- Or sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used via bazel's environment variable. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565190992
copybara-servicebot pushed a commit to abseil/abseil-py that referenced this pull requestSep 15, 2023
- Either test filtering is used- Or sharding is used and the shard index > 0, i.e. only the first shard will fail when no tests ran on Python 3.12+.Context: Python 3.12 unittest will now fail when no tests ran after the change frompython/cpython#102051. Since `absltest` is built on top of `unittest`, it will follow this behavior change in Python 3.12.However, when test filtering is used in `absltest`, often used via `bazel test --test_filter=<my_filter>`, the current user expectation is the `bazel test` command should NOT fail is at least one test ran. Since the test runner here has no visibility of the overall `bazel` invocation, we'll make the test not fail when test filtering is used via bazel's environment variable. This is the existing behavior before Python 3.12.Also test absl-py on Python 3.12.PiperOrigin-RevId: 565771935
Member
vstinner commentedJan 8, 2024
The new Python 3.12 message surprises users when all tests are skipped: see issue#113661. |
fishy commentedJan 8, 2024
The PR description says:
Which seems to suggest that it should not fail test if all tests areskipped, as skipped tests are still "found", just intentionally skipped from running. |
ContributorAuthor
stefanor commentedJan 9, 2024
Yeah, I'd agree with that. |
texhnolyze added a commit to bit-bots/bitbots_main that referenced this pull requestDec 12, 2024
to ensure that the CI `colcon test` run works, because with a change topython 3.12 the `unittest` standard library used by default with colconnow exits with an error code of 5 for an empty test suite.See:colcon/colcon-core#678See:python/cpython#102051
texhnolyze added a commit to bit-bots/udp_bridge that referenced this pull requestDec 16, 2024
to ensure that the CI `colcon test` run works, because with a change topython 3.12 the `unittest` standard library used by default with colconnow exits with an error code of 5 for an empty test suite.Additionally removed old ros 1 integration tests.See:colcon/colcon-core#678See:python/cpython#102051
6 tasks
texhnolyze added a commit to bit-bots/dynamic_stack_decider that referenced this pull requestDec 16, 2024
to ensure that the CI `colcon test` run works, because with a change topython 3.12 the `unittest` standard library used by default with colconnow exits with an error code of 5 for an empty test suite.See:colcon/colcon-core#678See:python/cpython#102051
6 tasks
texhnolyze added a commit to bit-bots/udp_bridge that referenced this pull requestDec 16, 2024
to ensure that the CI `colcon test` run works, because with a change topython 3.12 the `unittest` standard library used by default with colconnow exits with an error code of 5 for an empty test suite.Additionally removed old ros 1 integration tests.See:colcon/colcon-core#678See:python/cpython#102051
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading.Please reload this page.
As discussed inhttps://discuss.python.org/t/unittest-fail-if-zero-tests-were-discovered/21498/7
It is common for test runner misconfiguration to fail to find any tests,
this should be an error.
Fixes:#62432