Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

AttentionOnnxAiLayer#27988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
nklskyoy wants to merge28 commits intoopencv:5.x
base:5.x
Choose a base branch
Loading
fromnklskyoy:attention-2-layer
Open

Conversation

@nklskyoy
Copy link

implementhttps://onnx.ai/onnx/operators/onnx__Attention.html#attention-23

Pull Request Readiness Checklist

See details athttps://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

asmorkalov reacted with hooray emojiasmorkalov reacted with rocket emoji
@asmorkalovasmorkalov added optimization feature pr: needs testNew functionality requires minimal tests set category: dnn (onnx)ONNX suport issues in DNN module labelsNov 10, 2025
@nklskyoy
Copy link
Author

nklskyoy commentedNov 12, 2025
edited
Loading

@vpisarev@asmorkalov , I noticed some issues with graph simplification (in particular attention subgraph - see the failing test cases).

Right now we have

  1. attention op fromcom.microsoft
  2. attention op fromai.onnx (thats what I implemented)

So currently, the graph simplifier takes subgraph consisting ofai.onnx nodes and simplifies this subgraph into a singlecom.microsoft attention operation. But at runtime, thedomain_dispatch_map includes only parsers forai.onnx layers. (So thecom.microsoft attention is wrongly interpreted as ai.onnx attention)

Is there some reason why the dispatch map does not include parsers for bothai.onnx andcom.microsoft by default? This would fix the problem here.

@asmorkalov
Copy link
Contributor

Is there some reason why the dispatch map does not include parsers for both ai.onnx and com.microsoft by default? This would fix the problem here.

I do not know about any intent here. Most probable reason - we did not have extensions before. You are welcome to fix the issue, but I propose to do it with another PR.

nklskyoy reacted with thumbs up emoji

@nklskyoynklskyoy mentioned this pull requestNov 17, 2025
6 tasks
asmorkalov pushed a commit that referenced this pull requestNov 18, 2025
Onnx importer2 dispatch map#28032in the new onnx_importer all domains in the dispatch map should be included per default. See#27988 (comment)### Pull Request Readiness ChecklistSee details athttps://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request- [x] I agree to contribute to the project under Apache 2 License.- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV- [ ] The PR is proposed to the proper branch- [ ] There is a reference to the original bug report and related work- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable      Patch to opencv_extra has the same branch name.- [ ] The feature is well documented and sample code can be built with the project CMake
@nklskyoynklskyoy marked this pull request as ready for reviewNovember 30, 2025 14:28
@nklskyoy
Copy link
Author

@vpisarev@asmorkalov , this is ready for review

@asmorkalovasmorkalov removed the pr: needs testNew functionality requires minimal tests set labelDec 5, 2025
@Shlok-Saxena
Copy link

Hi,

I was really interested in this Attention layer implementation, so I pulled the branch to test it locally on Linux. I noticed the CI was failing and encountered a few blockers during the build and test process.

I managed to fix all of them and get the tests passing 100% locally. Here is a summary of the findings that might help unblock this PR:

  1. Test Name Mismatch: TheAttention layer implementation seems to output the layer nameAttentionOnnxAi, buttest_graph_simplifier.cpp is still expectingAttention.

Fix: Updated the expected string in TEST_F(Test_Graph_Simplifier,AttentionSubgraph).

  1. Missing Test Data: The tests requireattention.onnx,attention_single_head.onnx, etc., which seem to be missing from the linkedopencv_extra PR (or at least weren't pulled in).

Workaround: I generated synthetic 4D ONNX models locally to verify the logic.

  1. Test Harness Input Shape Mismatch (The Crash): Intest_onnx_importer.cpp, the tests were using the generictestONNXModels("attention") helper. This helper defaults to feeding 1 input. However, the strict validation inAttentionOnnxAi::getMemoryShapes correctly demands 3 inputs (Query, Key, Value), causing an assertion failure: (-215:Assertion failed)nsuggestedShapes == ninputs

Fix: I replaced the generic call with a manual test block that initializes a 4D input(1, 2, 2, 2) and feeds it explicitly to all three ports (query,key,value).

With these changes, allAttention tests pass successfully on Linux.

I have a patch file ready if you'd like me to push it or share the snippets! Great work on the layer logic itself—it works perfectly once the harness is aligned.

@nklskyoy
Copy link
Author

nklskyoy commentedDec 14, 2025
edited
Loading

Hello@Shlok-Saxena , what do you mean by failing CI? Currently, all tests related to attention and dnn are passing.
Note that we have two layers: one attention layer from com.microsoft domain (labeled as attention), while the attentionOnnxAiLayer (labeled as attentionOnnxAiLayer) is from onnx.ai domain
The graph_simplifier produces the com.microsoft attention layer, so Test_Graph_Simplifier correctly expects the attention label.
Please note that I am currently still working on this PR.

@Shlok-Saxena
Copy link

Thanks for the detailed explanation,@nklskyoy !

That really clears up the architectural distinction between thecom.microsoft andonnx.ai implementations—I hadn't fully realized that context.

I mainly wanted to share these logs in case there is a platform-specific quirk on Linux (I'm building onUbuntu 22.04 / GCC 13) that might affect the CI later.

  1. Regarding the Simplifier Test: On my local build, theTest_Graph_Simplifier actually failed because it did output theonnx.ai layer name instead of the expected one. It seems my environment is resolving the graph differently than yours:
[ RUN      ] Test_Graph_Simplifier.AttentionSubgraph/modules/dnn/test/test_graph_simplifier.cpp:38: FailureExpected equality of these values:  layers    Which is: { "AttentionOnnxAi" }  expected_layers    Which is: { "Attention" }
  1. Regarding the Crash: You are totally right—the assertion failure(-215:Assertion failed) was indeed because I lacked the official test data (the.onnx files). I generated some dummy synthetic data to verify the logic, but I had to manually update the test harness to explicitly feed 3 inputs (Q, K, V) to get it to run without asserting.

Just wanted to document these behaviors from a fresh Linux build perspective in case it helps!

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@asmorkalovasmorkalovasmorkalov left review comments

Assignees

No one assigned

Labels

category: dnn (onnx)ONNX suport issues in DNN modulefeatureoptimization

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nklskyoy@asmorkalov@Shlok-Saxena

[8]ページ先頭

©2009-2025 Movatter.jp