Npu support mm bert - #306
Merged
zhaixuejun1993 merged 2 commits intoSep 8, 2026
Merged
Conversation
ravi9
force-pushed
the
dev_backend_openvino
branch
from
September 3, 2026 00:04
ec98377 to
fd9bc04
Compare
zhaixuejun1993
force-pushed
the
npu-support-mmBERT
branch
from
September 3, 2026 02:25
2228437 to
6f11671
Compare
ravi9
force-pushed
the
dev_backend_openvino
branch
from
September 3, 2026 21:10
fd9bc04 to
33237ab
Compare
Collaborator
Author
|
@copilot resolve the merge conflicts in this pull request |
Packed QKV views used by mmBERT were rejected by the ROPE support check. This split Q/K RoPE onto CPU, prevented cacheless attention detection, and sent fragmented encoder graphs through the decoder-oriented NPUW path.
Accept packed QKV RoPE views, detect cacheless attention from its mask, and run these models as a single full-sequence prefill without NPUW or a decode graph. Also provide static mask, output index, and mean-pooling shapes and inputs.
Replace the decomposed mean/variance normalization graph with an opset6 MVN operation. This preserves the GGML epsilon placement while allowing OpenVINO plugins to compile normalization as one operation with fewer intermediate tensors.
Cache RoPE sine and cosine outputs in the graph-wide tensor map. Build the cache key from all RoPE parameters and the optional frequency-factor input so compatible Q/K and layer nodes share one subgraph without mixing different RoPE configurations.
Expose NodeContext::put_shared() to publish translator-created outputs for graph-level reuse.
zhaixuejun1993
force-pushed
the
npu-support-mmBERT
branch
from
September 8, 2026 08:50
6f11671 to
f34fb3f
Compare
zhaixuejun1993
merged commit Sep 8, 2026
4a22842
into
ravi9:dev_backend_openvino
5 of 16 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
openvino: support cacheless encoder models on NPU
openvino: optimize norm and RoPE translation