forked frommicrosoft/BitNet
- Notifications
You must be signed in to change notification settings - Fork0
A fork of official inference framework for 1-bit LLMs thats includes additional scripts to enhance the experience.
License
NotificationsYou must be signed in to change notification settings
ArrivedAditya/BitNet
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Kernels use in LLM model
- I2_S for x86, x86_64 cpus.
- TL1
- TL2
Model that is supported to use in this branch
- Bitnet-b1.58-2B-4T
- bitnet_b1_58-large
- Bitnet-b1.58-3B
- Llama3-8B-1.58-100B-tokens
- Falcon3 Family
- Falcon-E Family
- python>=3.9
- cmake>=3.22
- clang>=18
- conda (highly recommend) e.g., miniforge, miniconda.
For Windows users, its best use wsl for this thing and install linux distro like.
For Debian/Ubuntu or LTS Linux distro users
Packages respostory likellvm
`bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"`
- Clone the repo
git clone --recursive https://github.com/microsoft/BitNet.gitcd BitNet- Install the dependencies
# (Recommended) Create a new conda environmentconda create -n bitnet-cpp python=3.9conda activate bitnet-cpppip install -r requirements.txt- Build the project
# Manually download the model and run with local pathhuggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4Tpython setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s## UsageSimple, run run.py### CustomizeSimple edit run.py
About
A fork of official inference framework for 1-bit LLMs thats includes additional scripts to enhance the experience.
Resources
License
Code of conduct
Security policy
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Releases
No releases published
Packages0
No packages published
Languages
- C++53.0%
- Python46.1%
- Other0.9%