Jul 10, 2024 · Jun 27, 2024 · Jun 27, 2024 · Jul 2, 2024 · Jul 3, 2024 · Jul 8, 2024
diff --git a/pgml-cms/docs/SUMMARY.md b/pgml-cms/docs/SUMMARY.md
    * [Document Search](open-source/korvus/guides/document-search.md)
  * [Example Apps](open-source/korvus/example-apps/README.md)
    * [Semantic Search](open-source/korvus/example-apps/semantic-search.md)
 * [PgCat](open-source/pgcat/README.md)
  * [Features](open-source/pgcat/features.md)
  * [Installation](open-source/pgcat/installation.md)
  * [Configuration](open-source/pgcat/configuration.md)

 ## Guides

  * [Dedicated](product/cloud-database/dedicated.md)
  * [Enterprise](product/cloud-database/plans.md)
 * [Vector database](product/vector-database.md)
 * [PgCat pooler](product/pgcat/README.md)
  * [Features](product/pgcat/features.md)
  * [Installation](product/pgcat/installation.md)
  * [Configuration](product/pgcat/configuration.md)


 ## Resources

diff --git a/pgml-cms/docs/open-source/overview.md b/pgml-cms/docs/open-source/overview.md
 description: Overview of the PostgresML SQL API and SDK.
 ---

 #API overview
 #Open Source Overview

 PostgresML is a PostgreSQL extension which adds SQL functions to the database where it's installed. The functions work with modern machine learning algorithms and latest open source LLMs while maintaining a stable API signature. They can be used by any application that connects to the database.
 PostgresML maintins three open source projects:
 - [pgml](pgml/)
 - [Korvus](korvus/)
 - [pgcat](pgcat/)

 In addition to the SQL API, we built and maintain a client SDK for JavaScript, Python and Rust. The SDK uses the same extension functionality to implement common ML & AI use cases, like retrieval-augmented generation (RAG), chatbots, and semantic & hybrid search engines.
 ## PGML

 Using the SDK is optional, and you can implementthesame functionalitywithstandard SQL queries. If you feel more comfortable using a programming language, the SDK canhelp you to get started quickly.
 `pgml` is a PostgreSQL extension which adds SQL functions tothedatabase where it's installed. The functions workwithmodern machine learning algorithms and latest open source LLMs while maintaining a stable API signature. They canbe used by any application that connects to the database.

 ## [SQL extension](sql-extension/)
 See the [`pgml` docs](pgml/) for more information about `pgml`.

 The PostgreSQL extension provides all of the ML & AI functionality, like training models and inference, via SQL functions. The functions are designed for ML practitioners to use dozens of ML algorithms to train models, and run real time inference, on live application data. Additionally, the extension provides access to the latest Hugging Face transformers for a wide range of NLP tasks.
 ## Korvus

 ### Functions
 Korvus is an all-in-one, open-source RAG (Retrieval-Augmented Generation) pipeline built for Postgres. It combines LLMs, vector memory, embedding generation, reranking, summarization and custom models into a single query, maximizing performance and simplifying your search architecture.

 The following functions are implemented and maintained by the PostgresML extension:
 See the [Korvus docs](korvus/) for more information about Korvus.

 | Function    | Description                                                                                                                                                                                        |
 |------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | [pgml.embed()](sql-extension/pgml.embed)     | Generate embeddings inside the database using open source embedding models from Hugging Face.                                                                                                |
 | [pgml.transform()](sql-extension/pgml.transform/) | Download and run latest Hugging Face transformer models, like Llama, Mixtral, and many more to perform various NLP tasks like text generation, summarization, sentiment analysis and more. |
 | pgml.transform_stream() | Streaming version of [pgml.transform()](sql-extension/pgml.transform/). Retrieve tokens as they are generated by the LLM, decreasing time to first token. |
 | [pgml.train()](sql-extension/pgml.train/)     | Train a machine learning model on data from a Postgres table or view. Supports XGBoost, LightGBM, Catboost and all Scikit-learn algorithms.       |
 | [pgml.deploy()](sql-extension/pgml.deploy)    | Deploy a version of the model created with pgml.train(). |
 | [pgml.predict()](sql-extension/pgml.predict/) | Perform real time inference using a model trained with pgml.train() on live application data. |
 | [pgml.tune()](sql-extension/pgml.tune) | Run LoRA fine tuning on an open source model from Hugging Face using data from a Postgres table or view. |
 ## PgCat

 Together with standard database functionality provided by PostgreSQL, these functions allow to createandmanage the entire life cycle of a machine learning application.
 PgCat is PostgreSQL connection pooler and proxy which scales PostgreSQL (andPostgresML) databases beyond a single instance

 ## [Client SDK](client-sdk/)

 The client SDK implements best practices and common use cases, using the PostgresML SQL functions and standard PostgreSQL features to do it. The SDK core is written in Rust, which manages creating and running queries, connection pooling, and error handling.

 For each additional language we support (currently JavaScript and Python), we create and publish language-native bindings. This architecture ensures all programming languages we support have identical APIs and similar performance when interacting with PostgresML.

 ### Use cases

 The SDK currently implements the following use cases:

 | Use case | Description |
 |----------|---------|
 | [Collections](client-sdk/collections) | Manage documents, embeddings, full text and vector search indexes, and more, using one simple interface. |
 | [Pipelines](client-sdk/pipelines) | Easily build complex queries to interact with collections using a programmable interface. |
 | [Vector search](client-sdk/search) | Implement semantic search using in-database generated embeddings and ANN vector indexes. |
 | [Document search](client-sdk/document-search) | Implement hybrid full text search using in-database generated embeddings and PostgreSQL tsvector indexes. |
 See the [PgCat docs](pgcat/) for more information about PgCat.
diff --git a/pgml-cms/docs/product/pgcat/README.md → pgml-cms/docs/open-source/pgcat/README.md b/pgml-cms/docs/product/pgcat/README.md → pgml-cms/docs/open-source/pgcat/README.md
diff --git a/pgml-cms/docs/product/pgcat/configuration.md → ...s/docs/open-source/pgcat/configuration.md b/pgml-cms/docs/product/pgcat/configuration.md → ...s/docs/open-source/pgcat/configuration.md
diff --git a/pgml-cms/docs/product/pgcat/features.md → pgml-cms/docs/open-source/pgcat/features.md b/pgml-cms/docs/product/pgcat/features.md → pgml-cms/docs/open-source/pgcat/features.md
diff --git a/pgml-cms/docs/product/pgcat/installation.md → ...ms/docs/open-source/pgcat/installation.md b/pgml-cms/docs/product/pgcat/installation.md → ...ms/docs/open-source/pgcat/installation.md
Original file line number	Diff line number	Diff line change
Expand Up		@@ -55,6 +55,10 @@
		* [Document Search](open-source/korvus/guides/document-search.md)
		* [Example Apps](open-source/korvus/example-apps/README.md)
		* [Semantic Search](open-source/korvus/example-apps/semantic-search.md)
		* [PgCat](open-source/pgcat/README.md)
		* [Features](open-source/pgcat/features.md)
		* [Installation](open-source/pgcat/installation.md)
		* [Configuration](open-source/pgcat/configuration.md)

		## Guides

Expand All		@@ -79,11 +83,6 @@
		* [Dedicated](product/cloud-database/dedicated.md)
		* [Enterprise](product/cloud-database/plans.md)
		* [Vector database](product/vector-database.md)
		* [PgCat pooler](product/pgcat/README.md)
		* [Features](product/pgcat/features.md)
		* [Installation](product/pgcat/installation.md)
		* [Configuration](product/pgcat/configuration.md)


		## Resources

Expand Down
Original file line number	Diff line number	Diff line change
Expand Up		@@ -2,47 +2,27 @@
		description: Overview of the PostgresML SQL API and SDK.
		---

		#API overview
		#Open Source Overview

		PostgresML is a PostgreSQL extension which adds SQL functions to the database where it's installed. The functions work with modern machine learning algorithms and latest open source LLMs while maintaining a stable API signature. They can be used by any application that connects to the database.
		PostgresML maintins three open source projects:
		- [pgml](pgml/)
		- [Korvus](korvus/)
		- [pgcat](pgcat/)

		In addition to the SQL API, we built and maintain a client SDK for JavaScript, Python and Rust. The SDK uses the same extension functionality to implement common ML & AI use cases, like retrieval-augmented generation (RAG), chatbots, and semantic & hybrid search engines.
		## PGML

		Using the SDK is optional, and you can implementthesame functionalitywithstandard SQL queries. If you feel more comfortable using a programming language, the SDK canhelp you to get started quickly.
		`pgml` is a PostgreSQL extension which adds SQL functions tothedatabase where it's installed. The functions workwithmodern machine learning algorithms and latest open source LLMs while maintaining a stable API signature. They canbe used by any application that connects to the database.

		## [SQL extension](sql-extension/)
		See the [`pgml` docs](pgml/) for more information about `pgml`.

		The PostgreSQL extension provides all of the ML & AI functionality, like training models and inference, via SQL functions. The functions are designed for ML practitioners to use dozens of ML algorithms to train models, and run real time inference, on live application data. Additionally, the extension provides access to the latest Hugging Face transformers for a wide range of NLP tasks.
		## Korvus

		### Functions
		Korvus is an all-in-one, open-source RAG (Retrieval-Augmented Generation) pipeline built for Postgres. It combines LLMs, vector memory, embedding generation, reranking, summarization and custom models into a single query, maximizing performance and simplifying your search architecture.

		The following functions are implemented and maintained by the PostgresML extension:
		See the [Korvus docs](korvus/) for more information about Korvus.

		\| Function \| Description \|
		\|------------------\|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\|
		\| [pgml.embed()](sql-extension/pgml.embed) \| Generate embeddings inside the database using open source embedding models from Hugging Face. \|
		\| [pgml.transform()](sql-extension/pgml.transform/) \| Download and run latest Hugging Face transformer models, like Llama, Mixtral, and many more to perform various NLP tasks like text generation, summarization, sentiment analysis and more. \|
		\| pgml.transform_stream() \| Streaming version of [pgml.transform()](sql-extension/pgml.transform/). Retrieve tokens as they are generated by the LLM, decreasing time to first token. \|
		\| [pgml.train()](sql-extension/pgml.train/) \| Train a machine learning model on data from a Postgres table or view. Supports XGBoost, LightGBM, Catboost and all Scikit-learn algorithms. \|
		\| [pgml.deploy()](sql-extension/pgml.deploy) \| Deploy a version of the model created with pgml.train(). \|
		\| [pgml.predict()](sql-extension/pgml.predict/) \| Perform real time inference using a model trained with pgml.train() on live application data. \|
		\| [pgml.tune()](sql-extension/pgml.tune) \| Run LoRA fine tuning on an open source model from Hugging Face using data from a Postgres table or view. \|
		## PgCat

		Together with standard database functionality provided by PostgreSQL, these functions allow to createandmanage the entire life cycle of a machine learning application.
		PgCat is PostgreSQL connection pooler and proxy which scales PostgreSQL (andPostgresML) databases beyond a single instance

		## [Client SDK](client-sdk/)

		The client SDK implements best practices and common use cases, using the PostgresML SQL functions and standard PostgreSQL features to do it. The SDK core is written in Rust, which manages creating and running queries, connection pooling, and error handling.

		For each additional language we support (currently JavaScript and Python), we create and publish language-native bindings. This architecture ensures all programming languages we support have identical APIs and similar performance when interacting with PostgresML.

		### Use cases

		The SDK currently implements the following use cases:

		\| Use case \| Description \|
		\|----------\|---------\|
		\| [Collections](client-sdk/collections) \| Manage documents, embeddings, full text and vector search indexes, and more, using one simple interface. \|
		\| [Pipelines](client-sdk/pipelines) \| Easily build complex queries to interact with collections using a programmable interface. \|
		\| [Vector search](client-sdk/search) \| Implement semantic search using in-database generated embeddings and ANN vector indexes. \|
		\| [Document search](client-sdk/document-search) \| Implement hybrid full text search using in-database generated embeddings and PostgreSQL tsvector indexes. \|
		See the [PgCat docs](pgcat/) for more information about PgCat.