PostgresML is a complete MLOps platform built on PostgreSQL.

PostgresML is a complete MLOps platform built on PostgreSQL. 

The data for ML & AI systems is inherently larger and more dynamic than the models. It's more efficient, manageable and reliable to move the models to the database, rather than continuously moving the data to the models\_.\_ PostgresML allows you to take advantage of the fundamental relationship between data and models, by extending the database with the following capabilities and goals:

The data for ML & AI systems is inherently larger and more dynamic than the models. It's more efficient, manageable and reliable to move the models to the database, rather than continuously moving the data to the models. PostgresML allows you to take advantage of the fundamental relationship between data and models, by extending the database with the following capabilities and goals:

***Model Serving** -_**GPU accelerated**_ inference engine for interactive applications, with no additional networking latency or reliability costs.

***Model Store** - Download_**open-source**_ models including state of the art LLMs from HuggingFace, and track changes in performance between versions.

***Model Training** - Train models with_**your application data**_ using more than 50 algorithms for regression, classification or clustering tasks. Fine tune pre-trained models like LLaMA and BERT to improve performance.

***Feature Store** -_**Scalable**_ access to model inputs, including vector, text, categorical, and numeric data. Vector database, text search, knowledge graph and application data all in one_**low-latency**_ system.

***Feature Store** -_**Scalable**_ access to model inputs, including vector, text, categorical, and numeric data. Vector database, text search, knowledge graph and application data all in one_**low-latency**_ system. 

<figure><imgsrc=".gitbook/assets/ml_system.svg"alt="Machine Learning Infrastructure (2.0) by a16z"><figcaption><p>PostgresML handles all of the functions typically performed by a cacophony of services, <ahref="https://a16z.com/emerging-architectures-for-modern-data-infrastructure/">described by a16z</a></p></figcaption></figure>

These capabilities are primarily provided by two open-source software projects, that may be used independently, but are designed to be used with the rest of the Postgres ecosystem, including trusted extensions like pgvector and pg\_partman.

These capabilities are primarily provided by two open-source software projects, that may be used independently, but are designed to be used with the rest of the Postgres ecosystem, including trusted extensions like pgvector and pg\_partman.&#x20;

***pgml** is an open source extension for PostgreSQL. It adds support for GPUs and the latest ML & AI algorithms_**inside**_ the database with a SQL API and no additional infrastructure, networking latency, or reliability costs.

***PgCat** is an open source proxy pooler for PostgreSQL. It abstracts the scalability and reliability concerns of managing a distributed cluster of Postgres databases. Client applications connect only to the proxy, which handles load balancing and failover,_**outside**_ of any single database.

<figure><imgsrc=".gitbook/assets/architecture.png"alt="PostgresML architectural diagram"width="275"><figcaption><p>A PostgresML deployment at scale</p></figcaption></figure>

In addition, PostgresML provides[native language SDKs](https://github.com/postgresml/postgresml/tree/master/pgml-sdks/pgml) to implement best practices for common ML & AI applications. The JavaScript and Python SDKs are generated from the core Rust SDK, to provide the same API, correctness and efficiency across all application runtimes.

In addition, PostgresML provides[native language SDKs](https://github.com/postgresml/postgresml/tree/master/pgml-sdks/pgml) to implement best practices for common ML & AI applications. The JavaScript and Python SDKs are generated from the core Rust SDK, to provide the same API, correctness and efficiency across all application runtimes.&#x20;

SDK clients can perform advanced machine learning tasks in a single SQL request, without having to transfer additional data, models, hardware or dependencies to the client application. For example:

* Forecasting timeseries data for key metrics with complex metadata

* Fraud and anomaly detection with application data

Our goal is to provide access to Open Source AI for everyone. PostgresML is under continuous development to keep up with the rapidly evolving use cases for ML & AI, and we release non breaking changes with minor version updates in accordance with SemVer. We welcome contributions to our[open source code and documentation](https://github.com/postgresml).

Our goal is to provide access to Open Source AI for everyone. PostgresML is under continuous development to keep up with the rapidly evolving use cases for ML & AI, and we release non breaking changes with minor version updates in accordance with SemVer. We welcome contributions to our[open source code and documentation](https://github.com/postgresml).&#x20;

We can host your AI database in our cloud, or you can run our Docker image locally with PostgreSQL, pgml, pgvector and NVIDIA drivers included.

description:>-

There are 3 different deployment strategies available:

| Strategy| Description|

| -------------|------------------------------------------------------------------------------------------------|

| -------------|--------------------------------------------------------------------------------------------------|

|`most_recent`| The most recently trained model for this project is immediately deployed, regardless of metrics.|

|`best_score`| The model that achieved the best key metric score is immediately deployed.|

|`rollback`| The model that was deployed before to the current one is deployed.|

(1 row)

In case the new model isn't performing well in production, it's easy to rollback to the previous version. A rollback creates a new deployment for the old model. Multiple rollbacks in a row will oscillate between the two most recently deployed models, making rollbacks a safe and reversible operation.

In the case you need to deploy an exact model that is not the`most_recent` or`best_score`, you may deploy a model by id. Model id's can be found in the`pgml.models` table.

In the case you need to deploy an exact model that is not the`most_recent` or`best_score`, you may deploy a model by id. Model id's can be found in the`pgml.models` table.

description:>-

Embeddings are a numeric representation of text. They are used to represent words and sentences as vectors, an array of numbers. Embeddings can be used to find similar pieces of text, by comparing the similarity of the numeric vectors using a distance measure, or they can be used as input features for other machine learning models, since most algorithms can't use text directly.

description:>-

If you've already been through the[pgml.train](../pgml.train"mention") examples, you can see the predictive results of those models:

If you've already been through the[pgml.train](../pgml.train/"mention") examples, you can see the predictive results of those models:

description:>-

| Parameter| Example| Description|

| ---------------| -----------------------------------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

|`project_name`|`'Search Results Ranker'`| An easily recognizable identifier to organize your work.|

|`task`|`'regression'`| The objective of the experiment:`regression`,`classification` or`cluster`|

|`task`|`'regression'`| The objective of the experiment:`regression`,`classification` or`cluster`|

|`relation_name`|`'public.search_logs'`| The Postgres table or view where the training data is stored or defined.|

|`y_column_name`|`'clicked'`| The name of the label (aka "target" or "unknown") column in the training table.|

|`algorithm`|`'xgboost'`| <p>The algorithm to train on the dataset, see the task specific pages for available algorithms:<br><adata-mentionhref="regression.md">regression.md</a></p><p><adata-mentionhref="classification.md">classification.md</a><br><adata-mentionhref="clustering.md">clustering.md</a></p>|

description:>-

layout:

title:

visible:true

description:>-

Pre-trained models allow you to get up and running quickly, but you can likely improve performance on your dataset by fine tuning them. Normally, you'll bring your own data to the party, but for these examples we'll use datasets published on Hugging Face.

Pre-trained models allow you to get up and running quickly, but you can likely improve performance on your dataset by fine tuning them. Normally, you'll bring your own data to the party, but for these examples we'll use datasets published on Hugging Face.&#x20;

description:>-

Quantization allows PostgresML to fit larger models in less RAM. These algorithms perform inference significantly faster on NVIDIA, Apple and Intel hardware. Half-precision floating point and quantized optimizations are now available for your favorite LLMs downloaded from Huggingface.

_Discrete quantization is not a new idea. It's been used by both algorithms and artists for more than a hundred years._\\

_Discrete quantization is not a new idea. It's been used by both algorithms and artists for more than a hundred years._\

Going beyond 16-bit down to 8 or 4 bits is possible, but not with hardware accelerated floating point operations. If we want hardware acceleration for smaller types, we'll need to use small integers w/ vectorized instruction sets. This is the process of_quantization_. Quantization can be applied to existing models trained with 32-bit floats, by converting the weights to smaller integer primitives that will still benefit from hardware accelerated instruction sets like Intel's[AVX](https://en.wikipedia.org/wiki/Advanced\_Vector\_Extensions). A simple way to quantize a model can be done by first finding the maximum and minimum values of the weights, then dividing the range of values into the number of buckets available in your integer type, 256 for 8-bit, 16 for 4-bit. This is called_post-training quantization_, and it's the simplest way to quantize a model.

description:>-

Anyone who runs Postgres at scale knows that performance comes with trade offs. The typical playbook is to place a pooler like PgBouncer in front of your database and turn on transaction mode. This makes multiple clients reuse the same server connection, which allows thousands of clients to connect to your database without causing a fork bomb.

description:>-

The question "Does it Scale?" has become somewhat of a meme in software engineering. There is a good reason for it though, because most businesses plan for success. If your app, online store, or SaaS becomes popular, you want to be sure that the system powering it can serve all your new customers.