NotificationsYou must be signed in to change notification settings
Fork715
Star17.5k

feat: add metrics workgroup#9400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

uddhavdave merged 14 commits intomainfromud/add-metrics-workgroup

Dec 4, 2025

Merged

feat: add metrics workgroup#9400

uddhavdave merged 14 commits intomainfromud/add-metrics-workgroup

Dec 4, 2025

Conversation

Copy link

Contributor

uddhavdave commentedNov 30, 2025

This PR adds:

Workgroups concurrency control to metrics
Modularized version of work group checks
Introduce StopWatch feature to measure splits of the search operation and print pretty summary at end of trace

github-actionsbot added the ✏️ Feature label

Nov 30, 2025

Copy link

Contributor

github-actionsbot commentedNov 30, 2025

Failed to generate code suggestions for PR

Copy link

Contributor

greptile-appsbot commentedNov 30, 2025

Greptile Overview

Greptile Summary

This PR modularizes workgroup concurrency control and introduces aStopWatch utility for performance timing in search operations. The changes extract workgroup logic into a dedicated module that supports both OSS (distributed lock-based) and Enterprise (workgroup-based) concurrency control, and adds detailed timing breakdowns for metrics and log searches.

Key Changes

NewStopWatch utility provides split timing and formatted performance summaries
Workgroup logic extracted tosrc/service/search/work_group.rs with RAII-style cleanup viaWorkGroupLock
Metrics queries now enforce workgroup concurrency limits (Enterprise usesWorkGroup::Short)
Flight search refactored to use modular workgroup with cleanerAsyncDefer pattern
Performance timing now tracked throughout search pipeline with detailed breakdowns in logs

Issues Found

Critical: MissingStopWatch import for OSS builds inwork_group.rs (line 18-19) - will cause compilation failure for non-enterprise builds

Confidence Score: 3/5

This PR has a critical compilation issue for OSS builds that must be fixed before merging
The refactoring is well-designed with good separation of concerns and proper RAII patterns, but the missing import will break OSS builds. Once fixed, the changes are sound.
src/service/search/work_group.rs requires immediate attention to fix the missing import

Important Files Changed

File Analysis

Filename	Score	Overview
src/config/src/utils/stopwatch.rs	5/5	New`StopWatch` utility for performance timing - well-implemented with copyright header
src/service/search/work_group.rs	3/5	New modular workgroup module with OSS/Enterprise support - has import issue for OSS builds
src/service/promql/search/mod.rs	4/5	Integrated workgroup control and stopwatch for metrics queries - good refactoring with timing breakdown
src/service/search/cluster/flight.rs	4/5	Refactored to use modular workgroup and stopwatch - cleaner implementation with RAII pattern

Sequence Diagram

sequenceDiagram    participant Client    participant SearchAPI as Search API    participant StopWatch    participant WorkGroup as WorkGroup Module    participant DistLock as Distributed Lock (OSS)    participant WGQueue as WorkGroup Queue (Enterprise)    participant Querier as Querier Nodes        Client->>SearchAPI: Search Request (metrics/logs)    SearchAPI->>StopWatch: new() - Start timing        alt OSS Build        SearchAPI->>WorkGroup: check_work_group(trace_id, org_id, timeout)        WorkGroup->>DistLock: lock_with_trace_id()        DistLock-->>WorkGroup: Lock acquired        WorkGroup->>StopWatch: record_split("queue_wait")        WorkGroup-->>SearchAPI: Return WorkGroupLock with _guard    else Enterprise Build        SearchAPI->>WorkGroup: check_work_group(trace_id, org_id, user_id, work_group)        WorkGroup->>DistLock: lock_with_trace_id() [temporary]        WorkGroup->>WGQueue: need_wait() [loop until slot available]        WGQueue-->>WorkGroup: Slot available        WorkGroup->>WGQueue: process(trace_id, user_id)        WorkGroup->>DistLock: unlock_with_trace_id()        WorkGroup->>StopWatch: record_split("queue_wait")        WorkGroup-->>SearchAPI: Return WorkGroupLock with _guard    end        SearchAPI->>Querier: Execute search queries    Querier-->>SearchAPI: Results    SearchAPI->>StopWatch: record_split("result")    SearchAPI->>StopWatch: get_summary() - Log timing breakdown        Note over WorkGroup,WGQueue: WorkGroupLock._guard dropped        alt OSS Build        WorkGroup->>DistLock: unlock_with_trace_id() [via AsyncDefer]    else Enterprise Build        WorkGroup->>WGQueue: done(trace_id, user_id) [via AsyncDefer]    end        SearchAPI-->>Client: Search Results