|
1 | 1 | #Open Source Alternative for Building End-to-End Vector Search Applications without OpenAI & Pinecone |
2 | | -#How to use this crate |
3 | 2 |
|
4 | | -Here is a brief outline of how to use this crate and specifically add new Python classes. |
| 3 | +#Suported Languages |
5 | 4 |
|
6 | | -There are three main macros to know about: |
7 | | --`custom_derive` |
8 | | --`custom_methods` |
9 | | --`custom_into_py` |
| 5 | +We support a number of different languages: |
| 6 | +-[Python](/) |
| 7 | +-[JavaScript](/) |
| 8 | +-Rust |
10 | 9 |
|
11 | | -##custom_derive |
12 | | -`custom_derive` is used when defining a new struct that you want to be available as a Python class. This macro automatically creates a wrapper for the struct postfixing the name with`Python`. For example, the following code: |
13 | | -``` |
14 | | -#[derive(custom_derive, Debug, Clone)] |
15 | | -pub struct TestStruct { |
16 | | - pub name: String |
17 | | -} |
18 | | -``` |
| 10 | +Our SDK is written completely in Rust and translated by Rust to our other supported languages. See each individual language for an overview and specification on how to use the SDK. |
19 | 11 |
|
20 | | -Creates another struct: |
| 12 | +#Rust |
21 | 13 |
|
22 | | -``` |
23 | | -pub struct TestStructPython { |
24 | | - pub wrapped: TestStruct |
25 | | -} |
26 | | -``` |
27 | | - |
28 | | -You must currently implement`Debug` and`Clone` on the structs you use`custom_derive` on. |
29 | | - |
30 | | -##custom_methods |
31 | | -`custom_methods` is used on the impl block for a struct you want to be available as a Python class. This macro automatically creates methods that work seamlessly with pyO3. For example, the following code: |
32 | | -``` |
33 | | -#[custom_methods(new, get_name)] |
34 | | -impl TestStruct { |
35 | | - pub fn new(name: String) -> Self { |
36 | | - Self { name } |
37 | | - } |
38 | | - pub fn get_name(&self) -> String { |
39 | | - self.name.clone() |
40 | | - } |
41 | | -} |
42 | | -``` |
43 | | - |
44 | | -Produces similar code to the following: |
45 | | -``` |
46 | | -impl TestStruct { |
47 | | - pub fn new(name: String) -> Self { |
48 | | - Self { name } |
49 | | - } |
50 | | - pub fn get_name(&self) -> String { |
51 | | - self.name.clone() |
52 | | - } |
53 | | -} |
54 | | -
|
55 | | -impl TestStructPython { |
56 | | - pub fn new<'a>(name: String, py: Python<'a>) -> PyResult<Self> { |
57 | | - let x = TestStruct::new(name); |
58 | | - Ok(TestStructPython::from(x)) |
59 | | - } |
60 | | - pub fn get_name<'a>(&self, py: Python<'a>) -> PyResult<String> { |
61 | | - let x = self.wrapped.get_name(); |
62 | | - Ok(x) |
63 | | - } |
64 | | -} |
65 | | -``` |
66 | | - |
67 | | -Note that the macro only works on methods marked with`pub`; |
68 | | - |
69 | | -##custom_into_py |
70 | | -`custom_into_py` is used when we want to seamlessly return Rust structs as Python dictionaries. For example, let's say we have the following code: |
71 | | -``` |
72 | | -#[derive(custom_into_py, FromRow, Debug, Clone)] |
73 | | -pub struct Splitter { |
74 | | - pub id: i64, |
75 | | - pub created_at: DateTime<Utc>, |
76 | | - pub name: String, |
77 | | - pub parameters: Json<HashMap<String, String>>, |
78 | | -} |
79 | | -
|
80 | | -pub async fn get_text_splitters(&self) -> anyhow::Result<Vec<Splitter>> { |
81 | | - Ok(sqlx::query_as(&query_builder!( |
82 | | - "SELECT * from %s", |
83 | | - self.splitters_table_name |
84 | | - )) |
85 | | - .fetch_all(self.pool.borrow()) |
86 | | - .await?) |
87 | | -} |
88 | | -
|
89 | | -``` |
90 | | - |
91 | | -The`custom_into_py` macro automatically generates the following code for us: |
92 | | -``` |
93 | | -impl IntoPy<PyObject> for Splitter { |
94 | | - fn into_py(self, py: Python<'_>) -> PyObject { |
95 | | - let dict = PyDict::new(py); |
96 | | - dict.set_item("id", self.id) |
97 | | - .expect("Error setting python value in custom_into_py proc_macro"); |
98 | | - dict.set_item("created_at", self.created_at.timestamp()) |
99 | | - .expect("Error setting python value in custom_into_py proc_macro"); |
100 | | - dict.set_item("name", self.name) |
101 | | - .expect("Error setting python value in custom_into_py proc_macro"); |
102 | | - dict.set_item("parameters", self.parameters.0) |
103 | | - .expect("Error setting python value in custom_into_py proc_macro"); |
104 | | - dict.into() |
105 | | - } |
106 | | - } |
107 | | -``` |
108 | | - |
109 | | -Implementing`IntoPy` allows pyo3 to seamlessly convert between Rust and python. Note that Python users calling`get_text_splitters` will receive a list of dictionaries. |
110 | | - |
111 | | -##Other Noteworthy Things |
112 | | - |
113 | | -Be aware that the only pyo3 specific code in this crate is the`pymodule` invocation in`lib.rs`. Everything else is handled by`pgml-macros`. If you want to expose a Python Class directly on the Python module you have to add it in the`pymodule` invocation. For example, if you wanted to expose`TestStruct` so Python module users could access it directly on`pgml`, you could do the following: |
114 | | -``` |
115 | | -#[pymodule] |
116 | | -fn pgml(_py: Python, m: &PyModule) -> PyResult<()> { |
117 | | - m.add_class::<TestStructPython>()?; |
118 | | - Ok(()) |
119 | | -} |
120 | | -``` |
121 | | - |
122 | | -Now Python users can access it like so: |
123 | | -``` |
124 | | -import pgml |
125 | | -
|
126 | | -t = pgml.TestStruct("test") |
127 | | -print(t.get_name()) |
128 | | -
|
129 | | -``` |
130 | | - |
131 | | -For local development, install[maturin](https://github.com/PyO3/maturin) and run: |
132 | | -``` |
133 | | -maturin develop |
134 | | -``` |
135 | | - |
136 | | -You can now run the tests in`python/test.py`. |
| 14 | +More information about our methodologies and Rust SDK coming soon. |