Commit74ce6ae

committed

add vllm docs

1 parentd017cd6 commit74ce6aeCopy full SHA for 74ce6ae

File tree

-1

lines changed

-1

lines changed

Lines changed: 2 additions & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -4,6 +4,8 @@ use serde_json::{json, Value};`
`4`	`4`
`5`	`5`	`usesuper::LLM;`
`6`	`6`
	`7`	`+/// Cache a single model per client process. vLLM does not allow multiple, simultaneous models to be loaded.`
	`8`	`+/// See GH issue, https://github.com/vllm-project/vllm/issues/565`
`7`	`9`	`staticMODEL:Mutex<Option<LLM>> =Mutex::new(None);`
`8`	`10`
`9`	`11`	`pubfnvllm_inference(task:&Value,inputs:&[&str]) ->PyResult<Value>{`

Lines changed: 1 addition & 1 deletion

Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		-//! Rust bindings to the Python package`vllm`.
	`1`	`+//! Rust bindings to the Python package[vLLM](https://vllm.readthedocs.io/en/latest/)`
`2`	`2`
`3`	`3`	`mod inference;`
`4`	`4`	`mod llm;`

Comments

(0)