Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

fix(site): fix flaky Chromatic tests#20808

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@ssncferreira
Copy link
Contributor

@ssncferreirassncferreira commentedNov 17, 2025
edited
Loading

Problem

TheOrgsSortedAlphabetically test fromOrganizationSidebarView.stories.tsx was failing on Chromatic. Test logic was attempting to verify organization sorting order programmatically. This was identified in the Chromatic build of PR:https://www.chromatic.com/build?appId=624de63c6aacee003aa84340&number=26015

After fixing this test, two additional tests started failing:

  • VanillaJavascriptError fromGlobalErrorBoundary.stories.tsx: Test was making incorrect assertions about stack trace content
  • MarkAllNotificationsAsReadError fromNotificationsInbox.stories.tsx: Test was flaky due to competing WebSocket error messages

Solution

These are Chromatic snapshot tests, so implementation details (like sorting order or exact error message content) are already validated by the visual snapshots. Programmatic assertions were causing flakiness and are redundant.

Emyrk and dannykopping reacted with heart emoji
@ssncferreirassncferreiraforce-pushed thessncferreira/fix-site-orgs-sorted-alphabetically branch fromefeb716 to83c9847CompareNovember 17, 2025 18:06
@ssncferreirassncferreira changed the titlefix(site): OrgsSortedAlphabetically testfix(site): fix OrgsSortedAlphabetically testNov 17, 2025
@ssncferreirassncferreiraforce-pushed thessncferreira/fix-site-orgs-sorted-alphabetically branch from83c9847 to0bb3de1CompareNovember 17, 2025 18:17
@ssncferreirassncferreira marked this pull request as ready for reviewNovember 17, 2025 18:33
@Emyrk
Copy link
Member

I am also seeingVanilla Javascript Error fail, and not sure exactly what is going on.

Copy link
Member

@aslilacaslilac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think a better solution might be to remove this logic all together. stories are all snapshot tests, so we already know that the names are present and in the right order because they're in the screenshot. if this logic is flakey, then lets just rely on the snapshot instead.

@ssncferreira
Copy link
ContributorAuthor

I think a better solution might be to remove this logic all together. stories are all snapshot tests, so we already know that the names are present and in the right order because they're in the screenshot. if this logic is flakey, then lets just rely on the snapshot instead.

@aslilac I think the test might actually make sense. It validates that the organizations are listed in the correct order: first the active organization (Omega org), then the remaining organizations alphabetically.

AFAIU, if a bug is introduced where the order of the orgs changes, Chromatic will detect those visual changes in the screenshot, but it won't fail automatically, a reviewer would need to catch it manually. The programmatic assertion ensures the sorting logic works correctly and fails immediately if it breaks.

This new logic fixes the flakiness of the test by querying the.truncate span directly instead of relying on the fulltextContent (which included the avatar fallback text). The other failing tests in the CI are unrelated to this test and are failing on other builds as well.

@aslilacaslilac self-requested a reviewNovember 19, 2025 17:05
@ssncferreira
Copy link
ContributorAuthor

After discussing with@aslilac on the best practices around Chromatic tests, the focus should be on visual validation through snapshots rather than programmatic assertions. The snapshot will catch visual changes resulting from any programmatic changes during the Chromatic review process. The programmatic assertions on these tests were redundant and introduced flakiness, so removing them makes the tests more stable while still catching issues through visual diffs.

@ssncferreirassncferreira changed the titlefix(site): fix OrgsSortedAlphabetically testfix(site): fix flaky Chromatic testsNov 19, 2025
Copy link
Member

@aslilacaslilac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

thank you!

ssncferreira reacted with heart emoji
@ssncferreirassncferreira merged commit17edeea intomainNov 19, 2025
35 checks passed
@ssncferreirassncferreira deleted the ssncferreira/fix-site-orgs-sorted-alphabetically branchNovember 19, 2025 18:17
@github-actionsgithub-actionsbot locked and limited conversation to collaboratorsNov 19, 2025
Sign up for freeto subscribe to this conversation on GitHub. Already have an account?Sign in.

Reviewers

@aslilacaslilacaslilac approved these changes

Assignees

@ssncferreirassncferreira

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ssncferreira@Emyrk@aslilac

[8]ページ先頭

©2009-2025 Movatter.jp