Wouldn't it be better to do the same thing as it is done in Onnx Runtime Web?

    if (apis.IS_WEBGPU_AVAILABLE) {        supportedDevices.push('webgpu');    }

Electron applications can have WebGPU enabled when terminal Node not. Alsoonnx-runtime-node provides only backers for native modules, whenonnx-runtime-web have bindings for WebGPU, so just adding supported devices will not work without switching runtime

If I remember it correctly, IS_WEBGPU_AVAILABLE is checked against nagivator.gpu, which is only available in browser.

For electron, the rendering process is actually a "web" environment instead of "node"

Yes, you are correct,IS_WEBGPU_AVAILABLE is just a simple check againstnavigation.gpu. In theory, it is possible to install a 3rd-party package for WebGPU support in Node, but it is a complicated topic. Still, my comment is valid; I copied my example from a few lines higher in the same source file.

I recently did tests. Unfortunately, transformers.js are not detecting Electron applications correctly and mark them as Node applications, so it provides only CPU. I had a lot of trouble getting it running in an Electron app. Mostly, it was picky aboutpath andfs packages. If I changed the target platform to Node, it generated other problems. I'm preparing a new issue report for developers with my findings.

I already did tests with your branch, and this simple change didn’t enable WebGPU in Electron apps.

Updated the version ofonnxruntime-node to 1.22.0-dev.20250418-c19a49615b. This version supports WebGPU on Windows and macOS.

Wow thanks@fs-eire! Very exciting!!! Does the browser packagehttps://www.npmjs.com/package/onnxruntime-web/v/1.22.0-dev.20250418-c19a49615b release also add anything of significance?

Wow thanks@fs-eire! Very exciting!!! Does the browser packagehttps://www.npmjs.com/package/onnxruntime-web/v/1.22.0-dev.20250418-c19a49615b release also add anything of significance?

No.

BTW for WebGPU EP support in onnxruntime-web : There are still some perf issue for using WebGPU EP in a WebAssembly build. If you want to do conformance test only for WebGPU EP (eg. check correctness but not latency), I can offer you a private build of onnxruntime-web with WebGPU EP.

That would be great! Feel free to send via slack perhaps? Eventually, we can hook this into the Transformers.js CI to ensure correctness across all supported architectures.

I've been testing the webgpu EP for some llama/qwen models, and running into a few correctness issues.

Here's some code to help test/debug:

import{pipeline,TextStreamer}from"@huggingface/transformers";// Create a text generation pipelineconstgenerator=awaitpipeline("text-generation","onnx-community/ZR1-1.5B-ONNX",{dtype:"q4f16",device:"webgpu"},// device="cpu" works fine);// Define the list of messagesconstmessages=[{role:"system",content:"You are a helpful assistant."},{role:"user",content:"Write me a poem about Machine Learning."},];// Generate a responseconstoutput=awaitgenerator(messages,{max_new_tokens:512,do_sample:false,streamer:newTextStreamer(generator.tokenizer,{skip_prompt:true,skip_special_tokens:true}),});console.log(output[0].generated_text.at(-1).content);

I can confirm that q4 (instead of q4f16) works correctly, so it looks to be an issue with the f16 implementation.

I can confirm that q4 (instead of q4f16) works correctly, so it looks to be an issue with the f16 implementation.

for webgpu-ep / DeepSeek-R1-Distill-Qwen-1.5B we know about some open issue when GQA takes the FA2 path.
Don't happen on all GPU's but I can reproduce it on nvidia.

If ZR1-1.5B-ONNX is similar to DeepSeek-R1-Distill-Qwen-1.5B, might be the same. Not tried DeepSeek-R1-Distill-Qwen-1.5B with fp32. Let me check on this.

looks like the same issue as deepseek when GQA uses FA2 with fp16. fp32 seems ok.
I'll put this high on my list to look at.

Great, thanks@guschmue!

I'm accumulating all these changes intohttps://github.com/huggingface/transformers.js/tree/ort-improvements to make development and testing a bit easier (many version bumps and ort-specific changes)