Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 115 Commits
examples		examples
include_shims		include_shims
llama.cpp @ edd1ab7		llama.cpp @ edd1ab7
src		src
.dockerignore		.dockerignore
.gitignore		.gitignore
.gitmodules		.gitmodules
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md
binding.cpp		binding.cpp
binding.h		binding.h
build.rs		build.rs

Repository files navigation

rust_llama.cpp

LLama.cpp rust bindings.

The rust bindings are mostly based onhttps://github.com/go-skynet/go-llama.cpp/

Building Locally

Note: This repository uses git submodules to keep track ofLLama.cpp.

Clone the repository locally:

git clone --recurse-submodules https://github.com/mdrokz/rust-llama.cpp

cargo build

Usage

[dependencies]llama_cpp_rs ="0.3.0"

use llama_cpp_rs::{    options::{ModelOptions,PredictOptions},LLama,};fnmain(){let model_options =ModelOptions::default();let llama =LLama::new("../wizard-vicuna-13B.ggmlv3.q4_0.bin".into(),&model_options,).unwrap();let predict_options =PredictOptions{token_callback:Some(Box::new(|token|{println!("token1: {}", token);true})),        ..Default::default()};    llama.predict("what are the national animals of india".into(),             predict_options,).unwrap();}