You signed in with another tab or window.Reload to refresh your session.You signed out in another tab or window.Reload to refresh your session.You switched accounts on another tab or window.Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pgml-cms/docs/README.md
+6-4Lines changed: 6 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -21,16 +21,18 @@ PostgresML allows you to take advantage of the fundamental relationship between
21
21
22
22
<figure><imgsrc=".gitbook/assets/ml_system.svg"alt="Machine Learning Infrastructure (2.0) by a16z"><figcaptionclass="mt-2"><p>PostgresML handles all of the functions <ahref="https://a16z.com/emerging-architectures-for-modern-data-infrastructure/">described by a16z</a></p></figcaption></figure>
23
23
24
-
These capabilities are primarily provided by two open-source software projects, that may be used independently, but are designed to be used with the rest of the Postgres ecosystem:
24
+
These capabilities are primarily provided by two open-source software projects, that may be used independently, but are designed to be usedtogetherwith the rest of the Postgres ecosystem:
25
25
26
-
***pgml** - an open source extension for PostgreSQL. It adds support for GPUs and the latest ML & AI algorithms_inside_ the database with a SQL API and no additional infrastructure, networking latency, or reliability costs
27
-
***PgCat** - an open source pooler for PostgreSQL. It abstracts the scalability and reliability concerns of managing a distributed cluster of Postgres databases. Client applications connect only to the pooler, which handles load balancing, sharding, and failover, outside of any single database server.
26
+
*[**pgml**](/docs/api/sql-extension/) - an open source extension for PostgreSQL. It adds support for GPUs and the latest ML & AI algorithms_inside_ the database with a SQL API and no additional infrastructure, networking latency, or reliability costs.
27
+
*[**PgCat**](/docs/product/pgcat/) - an open source connection pooler for PostgreSQL. It abstracts the scalability and reliability concerns of managing a distributed cluster of Postgres databases. Client applications connect only to the pooler, which handles load balancing, sharding, and failover, outside of any single database server.
To learn more about how we designed PostgresML, take a look at our[architecture overview](/docs/resources/architecture/).
32
+
31
33
##Client SDK
32
34
33
-
The PostgresML team also provides[native language SDKs](https://github.com/postgresml/postgresml/tree/master/pgml-sdks/pgml) which implement best practices for common ML & AI applications. The JavaScript and Python SDKs are generated from the a core Rust library, which provides a uniform API, correctness and efficiency across all environments.
35
+
The PostgresML team also provides[native language SDKs](/docs/api/client-sdk/) which implement best practices for common ML & AI applications. The JavaScript and Python SDKs are generated from the a core Rust library, which provides a uniform API, correctness and efficiency across all environments.
34
36
35
37
While using the SDK is completely optional, SDK clients can perform advanced machine learning tasks in a single SQL request, without having to transfer additional data, models, hardware or dependencies to the client application.
Copy file name to clipboardExpand all lines: pgml-cms/docs/api/sql-extension/pgml.embed.md
+77-26Lines changed: 77 additions & 26 deletions
Original file line number
Diff line number
Diff line change
@@ -6,48 +6,99 @@ description: >-
6
6
7
7
#pgml.embed()
8
8
9
-
Embeddings are a numeric representation of text. They are used to represent words and sentences as vectors, an array of numbers. Embeddings can be used to find similar pieces of text, by comparing the similarity of the numeric vectors using a distance measure, or they can be used as input features for other machine learning models, since most algorithms can't use text directly.
10
-
11
-
Many pretrained LLMs can be used to generate embeddings from text within PostgresML. You can browse all the[models](https://huggingface.co/models?library=sentence-transformers) available to find the best solution on Hugging Face.
9
+
The`pgml.embed()` function generates[embeddings](/docs/use-cases/embeddings/) from text, using in-database models downloaded from Hugging Face. Thousands of[open-source models](https://huggingface.co/models?library=sentence-transformers) are available and new and better ones are being published regularly.
12
10
13
11
##API
14
12
15
13
```sql
16
14
pgml.embed(
17
-
transformerTEXT,-- huggingface sentence-transformer name
18
-
textTEXT,-- input to embed
19
-
kwargs JSON-- optional arguments (see below)
15
+
transformerTEXT,
16
+
"text"TEXT,
17
+
kwargs JSON
20
18
)
21
19
```
22
20
23
-
##Example
21
+
| Argument| Description| Example|
22
+
|----------|-------------|---------|
23
+
| transformer| The name of a Hugging Face embedding model.|`intfloat/e5-large-v2`|
24
+
| text| The text to embed. This can be a string or the name of a column from a PostgreSQL table.|`'I am your father, Luke'`|
25
+
| kwargs| Additional arguments that are passed to the model.||
24
26
25
-
Let's use the`pgml.embed` function to generate embeddings for tweets, so we can find similar ones. We will use the`distilbert-base-uncased` model from :hugging: HuggingFace. This model is a small version of the`bert-base-uncased` model. It is a good choice for short texts like tweets. To start, we'll load a dataset that provides tweets classified into different topics.
Creating an embedding from text is as simple as calling the function with the text you want to embed:
32
32
33
-
```sql
34
-
SELECT*
35
-
FROMpgml.tweet_eval
36
-
LIMIT10;
33
+
{% tabs %}
34
+
{% tab title="SQL" %}
35
+
36
+
```postgresql
37
+
SELECT * FROM pgml.embed(
38
+
'intfloat/e5-small',
39
+
'No, that''s not true, that''s impossible.'
40
+
) AS star_wars_embedding;
37
41
```
38
42
39
-
Get a preview of the embeddings for the first 10 tweets. This will also download the model and cache it for reuse, since it's the first time we've used it.
In this example, we're using[generated columns](https://www.postgresql.org/docs/current/ddl-generated-columns.html) to automatically create an embedding of the`quote` column every time the column value is updated.
73
+
74
+
####Using embeddings in queries
75
+
76
+
Once you have embeddings, you can use them in queries to find text with similar semantic meaning:
77
+
78
+
{% tabs %}
79
+
{% tab title="SQL" %}
80
+
81
+
```postgresql
82
+
SELECT
83
+
quote
84
+
FROM
85
+
star_wars_quotes
86
+
ORDER BY
87
+
pgml.embed(
88
+
'intfloat/e5-small',
89
+
'Feel the force!',
90
+
) <=> embedding
91
+
DESC
92
+
LIMIT 1;
53
93
```
94
+
95
+
{% endtab %}
96
+
{% endtabs %}
97
+
98
+
This query will return the quote with the most similar meaning to`'Feel the force!'` by generating an embedding of that quote and comparing it to all other embeddings in the table, using vector cosine similarity as the measure of distance.
99
+
100
+
##Performance
101
+
102
+
First time`pgml.embed()` is called with a new model, it is downloaded from Hugging Face and saved in the cache directory. Subsequent calls will use the cached model, which is faster, and if the connection to the database is kept open, the model will be reused across multiple queries without being unloaded from memory.
103
+
104
+
If a GPU is available, the model will be automatically loaded onto the GPU and the embedding generation will be even faster.
PostgresML integrates[🤗 Hugging Face Transformers](https://huggingface.co/transformers) to bring state-of-the-art models into the data layer. There are tens of thousands of pre-trainedmodels with pipelines to turn raw inputs into useful results. Many state of the art deep learning architectures have been publishedandmade available for download. You will wanttobrowse all the[models](https://huggingface.co/models) available to find the perfect solution foryour[dataset](https://huggingface.co/dataset) and[task](https://huggingface.co/tasks).
20
+
The`pgml.transform()` function isthe most powerful feature of PostgresML. It integrates open-source large languagemodels, like Llama, Mixtral,andmany more, which allowstoperform complex tasks onyourdata.
21
21
22
-
We'll demonstrate some of the tasks thatareimmediately available to users of your database upon installation:[translation](https://github.com/postgresml/postgresml/blob/v2.7.12/pgml-dashboard/content/docs/guides/transformers/pre\_trained\_models.md#translation),[sentiment analysis](https://github.com/postgresml/postgresml/blob/v2.7.12/pgml-dashboard/content/docs/guides/transformers/pre\_trained\_models.md#sentiment-analysis),[summarization](https://github.com/postgresml/postgresml/blob/v2.7.12/pgml-dashboard/content/docs/guides/transformers/pre\_trained\_models.md#summarization),[question answering](https://github.com/postgresml/postgresml/blob/v2.7.12/pgml-dashboard/content/docs/guides/transformers/pre\_trained\_models.md#question-answering) and[textgeneration](https://github.com/postgresml/postgresml/blob/v2.7.12/pgml-dashboard/content/docs/guides/transformers/pre\_trained\_models.md#text-generation).
22
+
The modelsaredownloaded from[🤗 Hugging Face](https://huggingface.co/transformers) which hosts tens of thousands ofpre-trained and fine-tuned models for various tasks like text generation,question answering, summarization,textclassification, and more.
23
23
24
-
###Examples
24
+
##API
25
25
26
-
All of the tasks and models demonstrated here can be customized by passing additional arguments to the`Pipeline` initializer or call. You'll find additional links to documentation in the examples below.
26
+
The`pgml.transform()` function comes in two flavors, task-based and model-based.
27
27
28
-
The Hugging Face[`Pipeline`](https://huggingface.co/docs/transformers/main\_classes/pipelines)API is exposed in Postgres via:
28
+
###Task-basedAPI
29
29
30
-
```sql
30
+
The task-based API automatically chooses a model to use based on the task:
31
+
32
+
```postgresql
33
+
pgml.transform(
34
+
task TEXT,
35
+
args JSONB,
36
+
inputs TEXT[]
37
+
)
38
+
```
39
+
40
+
| Argument| Description| Example|
41
+
|----------|-------------|---------|
42
+
| task| The name of a natural language processing task.|`text-generation`|
43
+
| args| Additional kwargs to pass to the pipeline.|`{"max_new_tokens": 50}`|
44
+
| inputs| Array of prompts to pass to the model for inference.|`['Once upon a time...']`|
45
+
46
+
####Example
47
+
48
+
{% tabs %}
49
+
{% tab title="SQL" %}
50
+
51
+
```postgresql
52
+
SELECT *
53
+
FROM pgml.transform (
54
+
'translation_en_to_fr',
55
+
'How do I say hello in French?',
56
+
);
57
+
```
58
+
59
+
{% endtab %}
60
+
{% endtabs %}
61
+
62
+
###Model-based API
63
+
64
+
The model-based API requires the name of the model and the task, passed as a JSON object, which allows it to be more generic:
65
+
66
+
```postgresql
31
67
pgml.transform(
32
-
taskTEXTORJSONB,-- task name or full pipeline initializer arguments
33
-
call JSONB,-- additional call arguments alongside the inputs
34
-
inputsTEXT[]ORBYTEA[]-- inputs for inference
68
+
modelJSONB,
69
+
args JSONB,
70
+
inputs TEXT[]
35
71
)
36
72
```
37
73
38
-
This is roughly equivalent to the following Python:
74
+
| Argument| Description| Example|
75
+
|----------|-------------|---------|
76
+
| task| Model configuration, including name and task.|`{"task": "text-generation", "model": "mistralai/Mixtral-8x7B-v0.1"}`|
77
+
| args| Additional kwargs to pass to the pipeline.|`{"max_new_tokens": 50}`|
78
+
| inputs| Array of prompts to pass to the model for inference.|`['Once upon a time...']`|
79
+
80
+
####Example
81
+
82
+
{% tabs %}
83
+
{% tab title="SQL" %}
84
+
85
+
```postgresql
86
+
SELECT pgml.transform(
87
+
task => '{
88
+
"task": "text-generation",
89
+
"model": "TheBloke/zephyr-7B-beta-GPTQ",
90
+
"model_type": "mistral",
91
+
"revision": "main",
92
+
}'::JSONB,
93
+
inputs => ['AI is going to change the world in the following ways:'],
['AI is going to change the world in the following ways:']
119
+
)
45
120
```
46
121
47
-
Most pipelines operate on`TEXT[]` inputs, but some require binary`BYTEA[]` data like audio classifiers.`inputs` can be`SELECT`ed from tables in the database, or they may be passed in directly with the query. The output of this call is a`JSONB` structure that is task specific. See the[Postgres JSON](https://www.postgresql.org/docs/14/functions-json.html) reference for ways to process this output dynamically.
122
+
{% endtab %}
123
+
{% endtabs %}
124
+
125
+
126
+
###Supported tasks
127
+
128
+
PostgresML currently supports most NLP tasks available on Hugging Face:
129
+
130
+
| Task| Name| Description|
131
+
|------|-------------|---------|
132
+
|[Fill mask](fill-mask)|`key-mask`| Fill in the blank in a sentence.|
133
+
|[Question answering](question-answering)|`question-answering`| Answer a question based on a context.|
134
+
|[Summarization](summarization)|`summarization`| Summarize a long text.|
135
+
|[Text classification](text-classification)|`text-classification`| Classify a text as positive or negative.|
136
+
|[Text generation](text-generation)|`text-generation`| Generate text based on a prompt.|
137
+
|[Text-to-text generation](text-to-text-generation)|`text-to-text-generation`| Generate text based on an instruction in the prompt.|
138
+
|[Token classification](token-classification)|`token-classification`| Classify tokens in a text.|
139
+
|[Translation](translation)|`translation`| Translate text from one language to another.|
140
+
|[Zero-shot classification](zero-shot-classification)|`zero-shot-classification`| Classify a text without training data.|
141
+
142
+
143
+
##Performance
48
144
49
-
!!! tip
145
+
Much like`pgml.embed()`, the models used in`pgml.transform()` are downloaded from Hugging Face and cached locally. If the connection to the database is kept open, the model remains in memory, which allows for faster inference on subsequent calls. If you want to free up memory, you can close the connection.
50
146
51
-
Models will be downloaded and stored locally on disk after the first call. They are also cached per connection to improve repeated calls in a single session. To free that memory, you'll need to close your connection. You may want to establish dedicated credentials and connection pools via[pgcat](https://github.com/levkk/pgcat) or[pgbouncer](https://www.pgbouncer.org/) for larger models that have billions of parameters. You may also pass`{"cache": false}` in the JSON`call` args to prevent this behavior.
147
+
##Additional resources
52
148
53
-
!!!
149
+
-[Hugging Face datasets](https://huggingface.co/datasets)
150
+
-[Hugging Face tasks](https://huggingface.co/tasks)