Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[WIP] Deno browser WASM/WebGPU support#1344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
xenova wants to merge1 commit intomain
base:main
Choose a base branch
Loading
fromdeno-wasm-support

Conversation

xenova
Copy link
Collaborator

An experimental branch/PR to test running Transformers.js in Deno's WASM and WebGPU runtimes. This is a bit hacky, but it works... at least for now!

You can test this out by building this PR, and hosting the js files locally.

import{pipeline}from"http://localhost:8080/dist/transformers.js";constextractor=awaitpipeline("feature-extraction","Xenova/all-MiniLM-L6-v2",// { device: "webgpu" }, // <-- Enable WebGPU acceleration);// Compute sentence embeddingsconstsentences=["Hello world","This is an example sentence"];constoutput=awaitextractor(sentences,{pooling:"mean",normalize:true});console.log(output.tolist());

NOTE: This is different to running with the native EP (#1306), which will enable WebGPU support in Deno, via ONNXRuntime's Native WebGPU EP implementation.

There are some bugs/limitations with the current runtime (based onwgpu), e.g.,

An uncaught WebGPU validation error was raised: Shader 'Gather' parsing error: the `f16` enable-extension is not yet supported  ┌─ wgsl:1:8  │1 │ enable f16;  │        ^^^ this enable-extension specifies standard functionality which is not yet implemented in Naga  │  = note: Let Naga maintainers know that you ran into this at <https://github.com/gfx-rs/wgpu/issues/4384>, so they can prioritize it!: the `f16` enable-extension is not yet supportedAn uncaught WebGPU validation error was raised: ShaderModule with 'Gather' label is invalidAn uncaught WebGPU validation error was raised: ComputePipeline with 'Gather' label is invalidAn uncaught WebGPU validation error was raised: BindGroupLayout with '' label is invalidAn uncaught WebGPU validation error was raised: In a set_pipeline command: ComputePipeline with 'Gather' label is invalidAn uncaught WebGPU validation error was raised: In a set_bind_group command: BindGroup with 'Gather' label is invalidAn uncaught WebGPU validation error was raised: ComputePipeline with 'Gather' label is invalidAn uncaught WebGPU validation error was raised: BindGroupLayout with '' label is invalidAn uncaught WebGPU validation error was raised: In a set_bind_group command: BindGroup with 'Gather' label is invalid

but WASM works well 👍

cc@crowlKats for viz

@xenovaxenova marked this pull request as draftJune 20, 2025 03:21
@@ -142,7 +142,7 @@ export const env = {
remoteHost: 'https://huggingface.co/',
remotePathTemplate: '{model}/resolve/{revision}/',

allowLocalModels: !(IS_BROWSER_ENV || IS_WEBWORKER_ENV),
allowLocalModels: !(IS_BROWSER_ENV || IS_WEBWORKER_ENV || IS_DENO_RUNTIME),
Copy link
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Deno doesn't support relative URLs, so this is necessary... even if we have filesystem access

Comment on lines +63 to +70
.replace(
`"object"==typeof process&&"object"==typeof process.versions&&"string"==typeof process.versions.node&&"renderer"!=process.type`,
"false",
)
.replace(
`typeof globalThis.process?.versions?.node == 'string'`,
"false",
)
Copy link
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Very hacky, but is a current workaround, otherwise we get errors related toworker_threads not being prefixed withnode:

cc@fs-eire if you have a better idea.

@HuggingFaceDocBuilderDev

The docs for this PR livehere. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


} else if (apis.IS_NODE_ENV) {
ONNX = ONNX_NODE.default ?? ONNX_NODE;
} else if (apis.IS_NODE_ENV && (ONNX = ONNX_NODE.default ?? ONNX_NODE)?.InferenceSession) {
Copy link
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Handles an edge case where it is a node-like environment (e.g., Deno), but we are running in an environment where onnxruntime-node is not bundled (but onnxruntime-wasm is)

@xenova
Copy link
CollaboratorAuthor

Turns out, this is also necessary to ensure v0.dev operates correctly, otherwise we get this error:

Failed to load model: no available backend found. ERR: [webgpu] TypeError: Failed to resolve module specifier 'worker_threads'

@fs-eire
Copy link
Contributor

Turns out, this is also necessary to ensure v0.dev operates correctly, otherwise we get this error:

Failed to load model: no available backend found. ERR: [webgpu] TypeError: Failed to resolve module specifier 'worker_threads'

could you try replace "worker_threads" to "node:worker_threads"?

@xenova
Copy link
CollaboratorAuthor

Unfortunately, I've tried that and it doesn't fix the issue:

Deno:

error: Uncaught (in promise) Error: no available backend found. ERR: [wasm] Error: The argument 'filename' must be a file URL object, file URL string, or absolute path string. Received http://localhost:1923/dist/ort-wasm-simd-threaded.jsep.mjs

v0:

Access to script at 'node:module' from origin 'https://....lite.vusercontent.net' has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: chrome, chrome-extension, chrome-untrusted, data, http, https, isolated-app.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@xenova@HuggingFaceDocBuilderDev@fs-eire

[8]ページ先頭

©2009-2025 Movatter.jp