- Notifications
You must be signed in to change notification settings - Fork836
Comments
chore(ci): Shard runtime tests and add heartbeat logging#22627
chore(ci): Shard runtime tests and add heartbeat logging#22627agneszitte wants to merge 5 commits intomasterfrom
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Pull request overview
This PR updates the runtime test infrastructure to reduce CI timeouts by increasing Android runtime test sharding and adding “heartbeat” reporting so CI can surface the last started test during hangs, along with additional diagnostics artifacts/markers for reruns.
Changes:
- Add runtime test heartbeat support by writing the currently running test name to a file (and logging it) during execution.
- Increase Android runtime test sharding to 8 groups (native + Skia) and enable rerun behavior for Android native runtime tests.
- Update platform-specific CI scripts to pass/collect the heartbeat file and emit additional timeout/no-results diagnostics.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/SamplesApp/SamplesApp.UnitTests.Shared/Controls/UnitTest/UnitTestsControl.cs | Writes heartbeat updates per-test and logs active shard test counts. |
| src/SamplesApp/SamplesApp.Shared/App.Tests.cs | Adds--runtime-current-test-file arg handling and logs the configured heartbeat file path. |
| build/test-scripts/wasm-run-skia-runtime-tests.sh | Passes heartbeat file path via URL and prints heartbeat at end of run. |
| build/test-scripts/run-windows-skia-runtime-tests.ps1 | Sets heartbeat env var and prints heartbeat at end of run. |
| build/test-scripts/macos-skia-runtime-tests.sh | Sets heartbeat env var and prints heartbeat at end of run. |
| build/test-scripts/linux-skia-runtime-tests.sh | Sets heartbeat env var and prints heartbeat at end of run. |
| build/test-scripts/ios-uitest-run.sh | Adds iOS runtime heartbeat file plumbing and prints heartbeat in both success/timeout paths. |
| build/test-scripts/android-uitest-run.sh | Adds emulator reuse improvements, creates diagnostics artifacts, pulls/prints heartbeat, adds rerun marker behavior. |
| build/test-scripts/android-run-skia-runtime-tests.sh | Adds emulator reuse improvements, creates diagnostics artifacts, pulls/prints heartbeat. |
| build/ci/tests/.azure-devops-tests-android-skia.yml | Aligns Android Skia runtime sharding to 8 groups. |
| build/ci/tests/.azure-devops-tests-android-native.yml | Expands native runtime shard list to 0–7, sets group count to 8, enables reruns for runtime tests. |
💡Add Copilot custom instructions for smarter, more guided reviews.Learn how to get started.
src/SamplesApp/SamplesApp.UnitTests.Shared/Controls/UnitTest/UnitTestsControl.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
unodevops commentedFeb 10, 2026
🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here:https://unowasmprstaging.z20.web.core.windows.net/pr-22627/wasm-skia-net9/index.html |
src/SamplesApp/SamplesApp.UnitTests.Shared/Controls/UnitTest/UnitTestsControl.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/SamplesApp/SamplesApp.UnitTests.Shared/Controls/UnitTest/UnitTestsControl.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
💡Add Copilot custom instructions for smarter, more guided reviews.Learn how to get started.
src/SamplesApp/SamplesApp.UnitTests.Shared/Controls/UnitTest/UnitTestsControl.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/SamplesApp/SamplesApp.UnitTests.Shared/Controls/UnitTest/UnitTestsControl.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
unodevops commentedFeb 10, 2026
🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here:https://unowasmprstaging.z20.web.core.windows.net/pr-22627/wasm-skia-net9/index.html |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Pull request overview
Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.
💡Add Copilot custom instructions for smarter, more guided reviews.Learn how to get started.
| HEARTBEAT_MESSAGE="$(cat "$RUNTIME_CURRENT_TEST_FILE")" | ||
| echo "Last runtime test heartbeat: $HEARTBEAT_MESSAGE" |
CopilotAIFeb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
runtime-current-test is written as UTF-16 (see SkiaSamplesAppHelper / the WASM file-creation server), but this script reads it withcat and then runs it throughtr/sed. UTF-16 content will contain NUL bytes and will be truncated/garbled in bash, so the heartbeat XML message will be incorrect. Decode to UTF-8 first (e.g., viaiconv -f utf-16 -t utf-8 when available) before escaping/writing XML.
| HEARTBEAT_MESSAGE="$(cat"$RUNTIME_CURRENT_TEST_FILE")" | |
| echo"Last runtime test heartbeat:$HEARTBEAT_MESSAGE" | |
| ifcommand -v iconv>/dev/null2>&1;then | |
| HEARTBEAT_MESSAGE="$(iconv -f utf-16 -t utf-8"$RUNTIME_CURRENT_TEST_FILE"| tr -d'\000')" | |
| echo"Last runtime test heartbeat:$HEARTBEAT_MESSAGE" | |
| else | |
| echo"Warning: iconv not available; cannot decode UTF-16 runtime heartbeat file.">&2 | |
| HEARTBEAT_MESSAGE="Runtime test heartbeat file present but could not be decoded (missing iconv)." | |
| echo"Last runtime test heartbeat:$HEARTBEAT_MESSAGE" | |
| fi |
| if ! $ANDROID_HOME/platform-tools/adb shell test -e "$UITEST_RUNTIME_AUTOSTART_RESULT_FILE" > /dev/null; then | ||
| if [ $SECONDS -ge $END_TIME ]; then | ||
| # Capture a timeout marker so CI artifacts explain why the run ended early. |
CopilotAIFeb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The wait loop above still uses[[ ! $(adb shell test -e ...) && ... ]] (command substitution), which checks the (empty) output string rather than the command exit code. That means the loop won’t terminate when the results file appears unless the app exits or the timeout hits. Update the loop condition to use the command’s exit status (e.g.,while ! adb shell test -e ... >/dev/null 2>&1 && ...; do ...).
| if ! $ANDROID_HOME/platform-tools/adb shell test -e "$UITEST_RUNTIME_AUTOSTART_RESULT_DEVICE_PATH" > /dev/null; then | ||
| if [ $SECONDS -ge $END_TIME ]; then | ||
| # Capture a timeout marker so CI artifacts explain why the run ended early. |
CopilotAIFeb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thewhile loop above uses[[ ! $(adb shell test -e ...) && ... ]] (command substitution), so it evaluates the (empty) output instead of the exit status. If the app keeps running after writing results, this can cause the script to wait until timeout even though the results file exists. Use the command exit code in the loop condition (e.g.,while ! adb shell test -e ... >/dev/null 2>&1 && ...; do).
unodevops commentedFeb 10, 2026
🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here:https://unowasmprstaging.z20.web.core.windows.net/pr-22627/wasm-skia-net9/index.html |
Uh oh!
There was an error while loading.Please reload this page.
This updates runtime test sharding (Android native/Skia) and adds heartbeat logging across Android, iOS, Windows, Linux, macOS, and WASM so CI can surface the last started test during hangs and reduce long-running job timeouts.
It also adds rerun markers and timeout/no-results artifacts to improve diagnostics and rerun behavior.
Note: No related issue (Internal maintenance / Approved by Team member:@agneszitte).