Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

A HuggingFace Transformer.js Demo Running Generative AI Model in Web Browser

License

NotificationsYou must be signed in to change notification settings

alankrantas/just-another-ai-assistant-huggingface-transformers-js

Repository files navigation

Try ithere.

The model may not run properly on your devices with insufficient RAM!

A simple demonstration modified from HuggingFace'sReact-translator example with TypeScript support.

The demo utilizesTransformers.js to load and run a smaller large language model (LLM) - or small language model (SLM) in the web browser. The app usesVite'sWorker to run the model in the background, hence this would have to be a React or Svelte app.

"Small" Large Language Models and Configuration

You can define themodels,tasks,device and other model parameters in/src/model/Config.json:

Notes:

  • The model loading time and memory usage mostly depends on model size. It is recommended to have at least 4-8 GB free memory on your device.
  • System roles may not work well on certain models.
  • Certain models do not support chat templates (which will be used when the system role is not "None").
  • Certain devices and dtype options may not work for certain models. "Audo" in dtype does not work for all models either.
    • For example,Gemma-3-1B-It does not work on WebGPU, andSmolLM2-1.7B-Instruct only works for dtype =int8,uint8,bnb4 orq4f16.
  • After loading a model, you must refresh the page to load a different one. There is no way to release the old model from the memory, and trying to load more than two models proved to be problematic.
{"models": {"SmolLM2-135M-Instruct":"HuggingFaceTB/SmolLM2-135M-Instruct","SmolLM2-360M-Instruct":"HuggingFaceTB/SmolLM2-360M-Instruct","Qwen2.5-0.5B-Instruct":"Mozilla/Qwen2.5-0.5B-Instruct","Qwen3-0.6B":"onnx-community/Qwen3-0.6B-ONNX","Gemma-3-1B-It":"onnx-community/gemma-3-1b-it-ONNX","Falcon3-1B-Instruct":"onnx-community/Falcon3-1B-Instruct","SmolLM2-1.7B-Instruct":"HuggingFaceTB/SmolLM2-1.7B-Instruct","Phi-3-mini-4k-Instruct (3.8B)":"Xenova/Phi-3-mini-4k-instruct"    },"system_roles": {"None (no chat template)":"","Helpful assistant":"You are a helpful, concise, and accurate assistant.","Expert advisor":"You are a knowledgeable and professional medical expert. Provide clear, evidence-based answers.","Socratic guide":"You are a Socratic tutor. Ask thoughtful questions to guide the user to their own conclusions.","Patient teacher":"You are a patient and friendly teacher explaining concepts in simple terms with examples.","Quiz generator":"You are a quiz master. Generate multiple-choice questions to test knowledge of a topic.","Code assistant":"You are a skilled software engineer. Help the user write clean, efficient code.","Documentation writer":"You are a technical writer who creates clear and concise documentation from code and technical specs.","Data analyst":"You are a data analyst. Interpret data clearly, with charts or summaries if needed.","Storyteller":"You are a creative and engaging storyteller. Write vivid and original fiction.","Poet":"You are a poetic wordsmith. Craft expressive and emotionally resonant poetry.","Motivational coach":"You are a motivational coach. Offer practical advice and encouragement.","Friendly companion":"You are a kind and empathetic companion. Listen and respond warmly."    },"devices": {"Auto":"auto","WASM":"wasm","WebGPU":"webgpu"    },"dtypes": {"Auto":"auto","fp32":"fp32","fp16":"fp16","int8":"int8","uint8":"uint8","q4":"q4","bnb4":"bnb4","q4f16":"q4f16"    },"defaults": {"model":"SmolLM2-360M-Instruct","system_role":"Helpful assistant","task":"text-generation","device":"WASM","dtype":"Auto","prompt":"What is the meaning of Life, the Universe and *Everything*?","config": {"max_new_tokens":1024,"temperature":0.2,"top_p":0.95,"top_k":30,"repetition_penalty":1.05,"do_sample":true        }    }}

Development

yarn

Install dependencies.

yarn start

Start the dev server.

yarn build

Build a production at./dist.

yarn serve

Serve and view the built production.

yarn commit

Commit changes.

Releases

No releases published

Packages

No packages published

Contributors2

  •  
  •  
[8]ページ先頭
©2009-2025 Movatter.jp