- Notifications
You must be signed in to change notification settings - Fork516
Make tests easier to run#4620
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Open
cwalther wants to merge5 commits intomicrosoft:mainChoose a base branch
base:main
Could not load branches
Branch not found:{{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline, and old review comments may become outdated.
Open
+148 −63
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
Otherwise tests that have `cmake_minimum_required(VERSION 3.0.0)` or similarfail when run with the current CMake 4.2.0, whose lowest supported versionis 3.5.0.I have not individually checked whether the test cases rely on any policiesthat changed between 3.0.0 and 3.5.0, but the tests still pass.
Author
cwalther commentedNov 25, 2025
@microsoft-github-policy-service agree company="Indel AG" |
Server Mode was removed in favor of the File API in CMake 3.20, so skip therespective tests when it is not supported. Whether server mode is supportedwas tracked already, but only with a lower version bound. Availability ofserver mode is explicitly named in the `-E capabilities` response, soadditionally take that into account, effectively adding an upper bound.
Skip configure-time compiler checks when passing a fake sysroot to thecompiler, they would fail because it is unable to link an executable thatway.I am not sure under what circumstances this has ever worked.
Make the tests that rely on executables in test/fakebin/ check whether theyexist, and if not give a helpful error message, instead of failing inunclear ways when they find things in the system $PATH instead.Add a command `yarn pretest-buildfakebin` that documents how to build theseexecutables, which was otherwise only documented in.github/workflows/ci-main*.yml and .vscode/settings.json.
Author
cwalther commentedNov 26, 2025
I had forgotten to runFormat Document again on modified TypeScript files. Fixed in this force push. |
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
None yet
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This changes the developer experience (but not the user experience)
The following changes are proposed:
(More details in the individual commit messages.)
The purpose of this change
Make it easier for a new developer of CMake Tools to run tests.
Other Notes/Information
I was working on a new feature for CMake Tools and, being new to the project, was having some trouble running the tests. After some investigation, it all came down to two hurdles:
To fix 1., instead of just writing instructions to install CMake 3.18, I decided to make the tests work with the current version 4.2, because thefeature I am working on requires CMake 4.3 (not released yet) and I will want to test with that too. The changes required for that turned out to be relatively minor, and are contained in the first three of the attached commits.
To fix 2., I made the tests that without fakebin executables would fail in obscure ways, instead fail with a clear message telling the user how to build the missing executables. This is the last commit.
Together, these changes allow me to successfully run all tests (
yarn smokeTests,unitTests,integrationTests,endToEndTestsSuccessfulBuild,endToEndTestsSingleRoot,endToEndTestsMultiRoot,backendTests) with both CMake 4.2.0 and 3.18.6 on my Ubuntu 20.04 system.Would these changes be welcome in the project? Or would you prefer other ways to mitigate these hurdles?
Since these are not user-visible changes, I did not add an entry to CHANGELOG.md. Should I?