Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Use extended cache for caching integration configuration details#6650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
brant-livefront wants to merge2 commits intomain
base:main
Choose a base branch
Loading
frombrant/add-caching-to-event-integrations-configurations

Conversation

@brant-livefront
Copy link
Contributor

📔 Objective

This PR moves the most high-traffic piece of event integrations over to the new extended cache. This part is the processing that happens every time when receive an event. We need to fetch the integrations from the database and determine if an integration should be fire. Previously, we had a home-grown database that would load all integrations into memory on a fixed interval. This PR replaces that with extended cache.

There are a number of benefits to having this in extended cache:

  • We don't need to pre-load any data. We can fetch from the database as events first come in (with stampede protection) and the results will exist in the cache
  • Due to extended cache's redis support, that newly cached item will be in redis for other nodes that see the same event type / organization / integration type later.
  • As a result of the backplane, we can invalidate the cache (flushing it from L2 / redis) as soon as an admin changes any integration or integration configurations and this will be proposed to all nodes (flushing their L1 / memory cache). The next event that would need this will then load the brand new data on first request and refill the L1 / L2 caches.
  • Since we no longer need to worry about stale data, we can use a very long (1 day) TTL on these records which allows us to lean even more heavily on cache.
  • We are relying on known implementations and don't need to worry about maintaining our own hand-coded in-memory cache.

In addition I've cleaned up the docs and fixed an error that I noticed while testing this (Trying to create an integration of the same type as an existing one would 500 error. It now returns aBadRequestException - and we have a new test to validate that)

Finally, I have added a method / stored procedure / query for fetching wildcard configurations. These are configurations where EventType isnull, which match all events. We were not calling this directly since we were loading all configurations at once into our old cache. But we now need a way of asking for these directly.

⏰ Reminders before review

  • Contributor guidelines followed
  • All formatters and local linters executed and passed
  • Written new unit and / or integration tests where applicable
  • Protected functional changes with optionality (feature flags)
  • Used internationalization (i18n) for all UI strings
  • CI builds passed
  • Communicated to DevOps any deployment requirements
  • Updated any necessary documentation (Confluence, contributing docs) or informed the documentation team

🦮 Reviewer guidelines

  • 👍 (:+1:) or similar for great changes
  • 📝 (:memo:) or ℹ️ (:information_source:) for notes or general info
  • ❓ (:question:) for questions
  • 🤔 (:thinking:) or 💭 (:thought_balloon:) for more open inquiry that's not quite a confirmed issue and could potentially benefit from discussion
  • 🎨 (:art:) for suggestions / improvements
  • ❌ (:x:) or⚠️ (:warning:) for more significant problems or concerns needing attention
  • 🌱 (:seedling:) or ♻️ (:recycle:) for future improvements or indications of technical debt
  • ⛏ (:pick:) for minor or nitpick changes

@claude
Copy link
Contributor

claudebot commentedNov 26, 2025
edited
Loading

Claude finished@brant-livefront's task ——View job


Code Review Complete

I've completed a comprehensive review of this PR. Below are my findings organized by severity.


Summary

This PR successfully migrates event integration configuration caching from a custom in-memory implementation to the extended cache framework. The migration provides significant benefits: stampede protection, Redis-backed L2 caching, cache invalidation via backplane, and longer TTLs. The implementation is generally solid with good test coverage for the new features. However, there isone critical bug that must be fixed before merging, along with several other issues requiring attention.


Critical Issues

Finding 1: Incorrect JOIN logic produces Cartesian product (STILL UNRESOLVED from previous review)

src/Infrastructure.EntityFramework/AdminConsole/Repositories/Queries/OrganizationIntegrationConfigurationDetailsReadManyByEventTypeOrganizationIdIntegrationTypeQuery.cs:22-23

This query contains redundant and incorrect JOIN syntax that will produce incorrect results:

joinoiin dbContext.OrganizationIntegrations on oic.OrganizationIntegrationIdequalsoi.Idinto oioicfrom oiindbContext.OrganizationIntegrations

Line 22 creates a LEFT JOIN but never uses the result (into oioic). Line 23 creates an unfiltered CROSS JOIN because it's not connected to the previous join, resulting in a Cartesian product.

Required fix:

varquery=fromoicindbContext.OrganizationIntegrations            joinoiindbContext.OrganizationIntegrations onoic.OrganizationIntegrationId equalsoi.Idwhereoi.OrganizationId==_organizationId&&oi.Type==_integrationType&&oic.EventType== _eventType

This matches the correct pattern used inOrganizationIntegrationConfigurationDetailsReadManyWildcardByOrganizationIdIntegrationTypeQuery.cs:19-23.


Minor Issues

Finding 2: Potential cache invalidation race condition in configuration updates

src/Api/AdminConsole/Controllers/OrganizationIntegrationConfigurationController.cs:107-112

When updating a configuration, the code invalidates the cache using the OLDEventType from the existing configuration, but if the admin changed theEventType in the update, the new cache entry won't be invalidated:

varnewConfiguration=model.ToOrganizationIntegrationConfiguration(configuration);awaitintegrationConfigurationRepository.ReplaceAsync(newConfiguration);awaitcache.RemoveAsync(EventIntegrationsCacheConstants.BuildCacheKeyForOrganizationIntegrationConfigurationDetails(organizationId:organizationId,integrationType:integration.Type,eventType:configuration.EventType// ← Using OLD EventType));

Suggested fix: Invalidate both the old and new event type cache entries when they differ:

// Invalidate old cache entryawaitcache.RemoveAsync(...,eventType:configuration.EventType);// If EventType changed, also invalidate new cache entryif(newConfiguration.EventType!=configuration.EventType){awaitcache.RemoveAsync(...,eventType:newConfiguration.EventType);}

Finding 3: Cache invalidation logic inconsistency between Integration and Configuration controllers

src/Api/AdminConsole/Controllers/OrganizationIntegrationController.cs:50-56 vssrc/Api/AdminConsole/Controllers/OrganizationIntegrationConfigurationController.cs:65-73

TheOrganizationIntegrationController usesRemoveByTagAsync (batch removal) whileOrganizationIntegrationConfigurationController usesRemoveAsync (single key removal). This is actually correct but could benefit from a code comment explaining the difference:

  • Integration changes affect ALL configurations → use tag-based removal
  • Configuration changes affect ONE specific event type → use key-based removal

The comment at line 65-66 partially explains this but could be clearer.

Finding 4: Potential null reference in wildcard configuration query comment

src/Core/AdminConsole/Repositories/IOrganizationIntegrationConfigurationRepository.cs:25

The XML comment states "rows in theIOrganizationIntegrationConfigurationRepository that have null EventType" but should reference the table/entity, not the repository interface. This is a documentation clarity issue.

Suggested fix:

/// rows in the OrganizationIntegrationConfiguration table that have null EventType are considered

Finding 5: Missing cache invalidation for wildcard configurations

src/Api/AdminConsole/Controllers/OrganizationIntegrationConfigurationController.cs:65-73, 107-112, 137-142

When creating, updating, or deleting a configuration, the code only invalidates the specific event type cache entry. However, ifeventType isnull (wildcard), both the wildcard cache entry AND any specific event type entries should be invalidated, since wildcard configurations apply to all events.

Current code:

awaitcache.RemoveAsync(EventIntegrationsCacheConstants.BuildCacheKeyForOrganizationIntegrationConfigurationDetails(organizationId:organizationId,integrationType:integration.Type,eventType:configuration.EventType// Could be null for wildcards));

Suggested enhancement: WheneventType isnull, consider using tag-based invalidation instead to clear all related cached entries.


Code Quality & Technical Debt

Finding 6: Improved error handling and validation

src/Api/AdminConsole/Controllers/OrganizationIntegrationController.cs:42-46

The duplicate integration check now properly returnsBadRequestException instead of 500 error. This is a good improvement with proper test coverage (OrganizationIntegrationControllerTests.cs:132-149).

Finding 7: Cache TTL increased from interval-based to 1 day

src/Core/AdminConsole/Services/Implementations/EventIntegrations/EventIntegrationHandler.cs:150, 162

The new 1-day TTL is appropriate given the proactive cache invalidation strategy. However, consider documenting this decision in the code or ADR to explain why such a long TTL is safe (i.e., because of backplane-based invalidation).


Test Coverage Concerns

Finding 8: Low patch coverage reported by Codecov

The PR shows 59.12% patch coverage with 56 lines missing coverage. Key areas lacking coverage:

  • OrganizationIntegrationConfigurationDetailsReadManyWildcardByOrganizationIdIntegrationTypeQuery.cs - 26 lines (0% coverage)
  • DapperOrganizationIntegrationConfigurationRepository.cs - 13 lines (0% coverage)
  • EFOrganizationIntegrationConfigurationRepository.cs - 13 lines (0% coverage)

While the newGetWildcardConfigurationDetailsAsync methods are tested indirectly through the handler tests, direct unit tests for these repository methods would improve confidence and maintainability.


Good Practices Observed

  • Proper use of stampede protection via FusionCache
  • Clean separation of concerns between event and integration tiers
  • Comprehensive test coverage for the duplicate integration check
  • Good use of tags for bulk cache invalidation
  • Proper cache key/tag generation viaEventIntegrationsCacheConstants
  • Excellent documentation updates in README.md

Action Items

  1. MUST FIX: Correct the JOIN logic inOrganizationIntegrationConfigurationDetailsReadManyByEventTypeOrganizationIdIntegrationTypeQuery.cs (Finding 1)
  2. SHOULD FIX: Handle EventType changes in cache invalidation (Finding 2)
  3. SHOULD FIX: Improve cache invalidation for wildcard configurations (Finding 5)
  4. CONSIDER: Add unit tests for the new repository methods to improve coverage (Finding 8)
  5. CONSIDER: Add code comments explaining Integration vs Configuration cache invalidation strategies (Finding 3)
  6. CONSIDER: Fix documentation clarity in IOrganizationIntegrationConfigurationRepository.cs (Finding 4)

Comment on lines 19 to 21
var query = from oic in dbContext.OrganizationIntegrationConfigurations
join oi in dbContext.OrganizationIntegrations on oic.OrganizationIntegrationId equals oi.Id into oioic
from oi in dbContext.OrganizationIntegrations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Critical Issue: Incorrect JOIN logic

This query has redundant and incorrect JOIN syntax:

  • Line 20: Creates a LEFT JOINinto oioic but never uses the result
  • Line 21: Thisfrom oi in dbContext.OrganizationIntegrations creates a CROSS JOIN because it's not filtered by the previous join

This will likely produce incorrect results (Cartesian product) or duplicate records.

Fix: Remove theinto oioic and the redundantfrom clause:

varquery=fromoicindbContext.OrganizationIntegrationConfigurations            joinoiindbContext.OrganizationIntegrations onoic.OrganizationIntegrationId equalsoi.Idwhereoi.OrganizationId==_organizationId&&oi.Type==_integrationType&&!oic.EventType.HasValue

This pattern matches the existing query inOrganizationIntegrationConfigurationDetailsReadManyByEventTypeOrganizationIdIntegrationTypeQuery.cs in the same directory.

@github-actions
Copy link
Contributor

github-actionsbot commentedNov 26, 2025
edited
Loading

Logo
Checkmarx One – Scan Summary & Details1ad3d629-1bb4-40dc-ab15-d42602019f0a

Great job! No new security vulnerabilities introduced in this pull request

@codecov
Copy link

codecovbot commentedNov 26, 2025
edited
Loading

Codecov Report

❌ Patch coverage is59.55882% with55 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.11%. Comparing base (1334ed8) to head (9a7c65e).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
...anyWildcardByOrganizationIdIntegrationTypeQuery.cs0.00%25 Missing⚠️
.../OrganizationIntegrationConfigurationRepository.cs0.00%13 Missing⚠️
.../OrganizationIntegrationConfigurationRepository.cs0.00%13 Missing⚠️
...SharedWeb/Utilities/ServiceCollectionExtensions.cs0.00%4 Missing⚠️
Additional details and impacted files
@@            Coverage Diff             @@##             main    #6650      +/-   ##==========================================+ Coverage   53.27%   57.11%   +3.83%==========================================  Files        1906     1906                Lines       84973    85025      +52       Branches     7639     7638       -1     ==========================================+ Hits        45268    48558    +3290+ Misses      37952    34638    -3314- Partials     1753     1829      +76

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report?Share it here.

🚀 New features to boost your workflow:
  • ❄️Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@claudeclaude[bot]claude[bot] left review comments

@withinfocuswithinfocusAwaiting requested review from withinfocus

@r-tomer-tomeAwaiting requested review from r-tomer-tome is a code owner automatically assigned from bitwarden/team-admin-console-dev

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@brant-livefront

[8]ページ先頭

©2009-2025 Movatter.jp