Repository files navigation

See in spanish/Ver en español

This Script takes a GGUF model and extracts it's Weight then converts it to NumPy and PyTorch Tensor.

First of all, you'll need a GGUF Model.
You can get one atHuggingFace or use other programs asLM Studio to get a Model.

Then, make sure you havePython installed and clone the .py Script of the repository.

You can setup the process in two ways:

• Go to thellama.cpp repos and eitherbuild or download thecompiled.
  To build it, you'll needCMAKE, and in case you use a NVIDIA GPU and you want to use it, make sure you have theCUDA Toolkit installed too.Follow the build guide that fits the best for your system.
  Install the necessary dependencies with the following command in a CMD:

cd llama.cpppip install -r requirements.txt

Or if it fails or you're using a newer version of Python:

cd llama.cpppy -m pip install -r requirements.txt

Then, place the Script and the GGUF Model inllama.cpp\gguf-py\gguf so it takes thegguf library reference necessary for the process.

• Just install these Python libraries in a CMD: torch, numpy, sentencepiece, pyyaml and gguf

pip install torch numpy sentencepiece pyyaml gguf

Or if it fails or you're using a newer version of Python:

py -m pip install torch numpy sentencepiece pyyaml gguf

Make sure the Script and the GGUF Model is in the same folder.

Run the Scriptgguftopytorch.py in the terminal using thepy orpython command in the CMD or in an IDE likeVisual Studio Code with thePython Extension to save GGUF Weigths to PyTorch Tensor.

If the Script runs succesfully, it will generate two files in the same folder:

• llama-weight.npy: NumPy file with the GGUF LlaMA Model's Weight stored.
• PyTorchTensor.pt: PyTorch Tensor file with the NumPy converted into a Tensor.

• Programming Language:Python (3.13.2)
• Libraries:
  • torch (2.6.0)
  • numpy (2.2.0)
  • sentencepiece (0.2.0)
  • pyyaml (6.0.2)
  • gguf (0.14.0)
• Other:
  • CMAKE (3.31.5)
  • CUDA Toolkit (12.8)
  • VS Code Python Extension
• Recommended IDE:VS Code