apache/apisixPublic

NotificationsYou must be signed in to change notification settings
Fork2.8k
Star16.2k

feat: add more spans to opentelemetry plugin#12686

Merged

nic-6443 merged 55 commits intoapache:masterfrom

nic-6443:nic/opentelemetry

Feb 7, 2026

Merged

feat: add more spans to opentelemetry plugin#12686
nic-6443 merged 55 commits intoapache:masterfrom
nic-6443:nic/opentelemetry

Conversation

Copy link

Member

nic-6443 commentedOct 19, 2025•
edited by AlinsRan
Loading

Description

Run jaeger in local byhttps://www.jaegertracing.io/docs/2.11/getting-started/#all-in-one

New Features
- Added comprehensive distributed tracing across request lifecycle (SSL/SNI, access, header/body filter, upstream, and logging).
- Enhanced OpenTelemetry integration with improved span propagation, context management, and per-request span lifecycle.
- Newtracing configuration option to enable detailed observability.
Tests
- Added OpenTelemetry plugin tracing test suite validating span generation and propagation.

Checklist

I have explained the need for this PR and the problem it solves
I have explained the changes or the new features added to this PR
I have added tests corresponding to this change
I have updated the documentation to reflect this change
I have verified that this change is backward compatible (If not, please discuss on theAPISIX mailing list first)

nic-6443 added3 commits

October 19, 2025 18:24

feat: add more spans to opentelemetry plugin

b5197a1

Signed-off-by: Nic <qianyong@api7.ai>

add todo

f6414fd

Signed-off-by: Nic <qianyong@api7.ai>

f

0317cf5

Signed-off-by: Nic <qianyong@api7.ai>

nic-6443 commented

Oct 20, 2025

View reviewed changes

apisix/core/response.luaShow resolvedHide resolved

nic-6443 commented

Oct 22, 2025

View reviewed changes

apisix/utils/span.lua OutdatedShow resolvedHide resolved

Revolyssup added6 commits

October 23, 2025 12:57

return span on newspan

7944e9f

fix lint

ec7adef

fix CI

bb28639

fix opentelemetry3

1b29310

add test

57cac44

add test

f8d5974

Revolyssup marked this pull request as ready for review

October 23, 2025 14:21

dosubotbot added size:XL

This PR changes 500-999 lines, ignoring generated files.

enhancementNew feature or request labels

Oct 23, 2025

revert

0c38fc7

dosubotbot added size:XXLThis PR changes 1000+ lines, ignoring generated files. and removed size:XLThis PR changes 500-999 lines, ignoring generated files. labels

Oct 23, 2025

revert

ce4277c

dosubotbot added size:LThis PR changes 100-499 lines, ignoring generated files. and removed size:XXLThis PR changes 1000+ lines, ignoring generated files. labels

Oct 23, 2025

Revolyssup added8 commits

October 23, 2025 19:55

revert

1d6c4e2

f

1f0cedb

add test

e6c31c0

add plugin phase test

234f9f7

fix test

c3f9ae9

add test

03f3906

f

12d2513

f

f6e92b8

dosubotbot added size:XLThis PR changes 500-999 lines, ignoring generated files. and removed size:LThis PR changes 100-499 lines, ignoring generated files. labels

Oct 24, 2025

bzp2010 reviewed

Jan 29, 2026

View reviewed changes

apisix/plugins/opentelemetry.lua OutdatedShow resolvedHide resolved

docs/en/latest/plugins/opentelemetry.md OutdatedShow resolvedHide resolved

docs/zh/latest/plugins/opentelemetry.md OutdatedShow resolvedHide resolved

apisix/ssl/router/radixtree_sni.lua OutdatedShow resolvedHide resolved

refactor v3

b9f3215

AlinsRan dismissedmembphis’sstale review viab9f3215

February 3, 2026 00:46

dosubotbot added size:XLThis PR changes 500-999 lines, ignoring generated files. and removed size:XXLThis PR changes 1000+ lines, ignoring generated files. labels

Feb 3, 2026

fix test

d3f9bd4

AlinsRan force-pushed thenic/opentelemetry branch fromdba7342 tod3f9bd4Compare

February 4, 2026 00:59

AlinsRan added2 commits

February 4, 2026 09:12

update doc

03e50dc

update doc

9616ff5

AlinsRan force-pushed thenic/opentelemetry branch from9e29643 to9616ff5Compare

February 4, 2026 06:02

fix lint

9a006ab

moonming requested a review fromCopilot

February 4, 2026 06:47

Copilotstarted reviewing on behalf ofmoonming

February 4, 2026 06:48

View session

fix finish span

460b316

CopilotAI reviewed

Feb 4, 2026

View reviewed changes

Copy link

CopilotAI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Pull request overview

Adds a new “comprehensive request lifecycle tracing” mode for the OpenTelemetry integration, instrumenting more APISIX internals and documenting/testing the resulting spans.

Changes:

Introduces a core tracing utility (apisix/tracer.lua) and span model (apisix/utils/span.lua) to capture APISIX-internal spans across phases.
Instruments multiple request lifecycle points (SSL/SNI matching, DNS resolve, plugin/global-rule execution, phases) and injects them into OpenTelemetry export.
Addsapisix.tracing config flag, updates EN/ZH docs, and adds a new tracing-focused test.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`t/plugin/opentelemetry6.t`	New test covering additional spans being emitted and exported.
`docs/zh/latest/plugins/opentelemetry.md`	Documents enabling comprehensive lifecycle tracing and shows updated collector output.
`docs/en/latest/plugins/opentelemetry.md`	Same as ZH docs, for English readers.
`conf/config.yaml.example`	Adds`apisix.tracing` example + explanation.
`apisix/cli/config.lua`	Adds default`apisix.tracing = false` in CLI default config.
`apisix/tracer.lua`	New internal tracer used to create/finish APISIX core spans.
`apisix/utils/span.lua`	New span container type used by the internal tracer.
`apisix/init.lua`	Starts/finishes internal spans across HTTP + SSL phases; finishes all spans in log phase and releases tracing state.
`apisix/plugin.lua`	Wraps plugin execution + global rules with spans.
`apisix/utils/upstream.lua`	Adds a`resolve_dns` span around domain resolution.
`apisix/ssl/router/radixtree_sni.lua`	Adds a`sni_radixtree_match` span around SNI routing.
`apisix/secret.lua`	Adds a`fetch_secret` span around secret retrieval.
`apisix/plugins/opentelemetry.lua`	Adds attributes + injects internal APISIX spans into OpenTelemetry export in log phase.
`apisix/core/response.lua`	Attempts to mark spans as error on`>= 400` exits.

Comments suppressed due to low confidence (1)

apisix/secret.lua:157

A span is started (span = tracer.start(...)) but on thepcall(require, ...) failure path the function returns without finishing the span, which will leak an unfinished span intongx.ctx.tracing. Finish the span with an error status before returning.

    local span = tracer.start(ngx.ctx, "fetch_secret", tracer.kind.client)    local ok, sm = pcall(require, "apisix.secret." .. opts.manager)    if not ok then        return nil, "no secret manager: " .. opts.manager    end

💡Add Copilot custom instructions for smarter, more guided reviews.Learn how to get started.

apisix/ssl/router/radixtree_sni.lua OutdatedShow resolvedHide resolved

apisix/secret.luaShow resolvedHide resolved

apisix/utils/span.lua OutdatedShow resolvedHide resolved

apisix/core/response.luaShow resolvedHide resolved

apisix/tracer.luaShow resolvedHide resolved

apisix/tracer.lua OutdatedShow resolvedHide resolved

apisix/plugins/opentelemetry.lua OutdatedShow resolvedHide resolved

apisix/plugins/opentelemetry.luaShow resolvedHide resolved

AlinsRan requested a review fromCopilot

February 4, 2026 07:14

Copilotstarted reviewing on behalf ofAlinsRan

February 4, 2026 07:14

View session

CopilotAI reviewed

Feb 4, 2026

View reviewed changes

Copy link

CopilotAI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

apisix/secret.lua:157

A span is started beforepcall(require, ...), but whenrequire fails this function returns without finishing the span. That leaves an open child span inngx.ctx.tracing and can skew parent/child tracking untilfinish_all runs. Finish the span on this error path as well (and set ERROR status/message).

    local span = tracer.start(ngx.ctx, "fetch_secret", tracer.kind.client)    local ok, sm = pcall(require, "apisix.secret." .. opts.manager)    if not ok then        return nil, "no secret manager: " .. opts.manager    end

💡Add Copilot custom instructions for smarter, more guided reviews.Learn how to get started.

apisix/plugins/opentelemetry.luaShow resolvedHide resolved

apisix/init.lua OutdatedShow resolvedHide resolved

apisix/utils/span.luaShow resolvedHide resolved

udpate comments

51ebadd

bzp2010 reviewed

Feb 6, 2026

View reviewed changes

Copy link

Contributor

bzp2010 left a comment•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I have no more problems to say.

I've already expressed my views twice; this would be the third time.

A span is an instance. Whatever it means to Lua—a metatable or anything else—I don't care.

We can always terminate a span using the following way:

localspan=tracer.start()span:finsih(api_ctx.ngx_ctxorngx.ctx,status,message)-- Even `api_ctx.ngx_ctx` can be omitted, since `ngx.ctx` is always available in any context where `api_ctx` can be accessed.

Simply attaching an identical metatable to the span table and obtaining the span instance from self is sufficient to achieve the same result internally as you are currently doing.
I don't understand why the API design persists in making function calls statelessly from the tracer. I've already repeated this three times, and I don't want to repeat it again.

Throughout the entire process, the PR never addressed my concerns regarding the API design.

I'm not sure how these design conflicts arose, as maintaining API compatibility with other popular implementations shouldn't be difficult. There is no external evidence to support this API design.

Since this API is unexpected for me, I won't approve this PR. However, I won't block the merge either—you'll need approval from other maintainers.