GoogleEl Carro Oracle Operatoroffers a way to run Oracle databases in Kubernetes as a portable, open source,community driven, no vendor lock-in container orchestration system. El Carroprovides a powerful declarative API for comprehensive and consistentconfiguration and deployment as well as for real-time operations andmonitoring.Extend your Oracle database's capabilities to build AI-powered experiencesby leveraging the El Carro Langchain integration.

This guide goes over how to use El Carro Langchain integration tosave, load and delete langchain documentswithElCarroLoader andElCarroDocumentSaver. This integration works for any Oracle database, regardless of where it is running.

Learn more about the package onGitHub.

Before You Begin​

Please completetheGetting Startedsection ofthe README to set up your El Carro Oracle database.

🦜🔗 Library Installation​

The integration lives in its ownlangchain-google-el-carro package, sowe need to install it.

%pip install--upgrade--quiet langchain-google-el-carro

Basic Usage​

Set Up Oracle Database Connection​

Fill out the following variable with your Oracle database connections details.

# @title Set Your Values Here { display-mode: "form" }
HOST="127.0.0.1"# @param {type: "string"}
PORT=3307# @param {type: "integer"}
DATABASE="my-database"# @param {type: "string"}
TABLE_NAME="message_store"# @param {type: "string"}
USER="my-user"# @param {type: "string"}
PASSWORD=input("Please provide a password to be used for the database user: ")

If you are using El Carro, you can find the hostname and port values in thestatus of the El Carro Kubernetes instance.Use the user password you created for your PDB.

Example Ouput:

kubectl get -w instances.oracle.db.anthosapis.com -n db
NAME   DB ENGINE   VERSION   EDITION      ENDPOINT      URL                DB NAMES   BACKUP ID   READYSTATUS   READYREASON        DBREADYSTATUS   DBREADYREASON

mydb   Oracle      18c       Express      mydb-svc.db   34.71.69.25:6021   ['pdbname']            TRUE          CreateComplete     True            CreateComplete

ElCarroEngine Connection Pool​

ElCarroEngine configures a connection pool to your Oracle database, enabling successful connections from your application and following industry best practices.

from langchain_google_el_carroimport ElCarroEngine

elcarro_engine= ElCarroEngine.from_instance(
    db_host=HOST,
    db_port=PORT,
    db_name=DATABASE,
    db_user=USER,
    db_password=PASSWORD,
)

Initialize a table​

Initialize a table of default schemaviaelcarro_engine.init_document_table(<table_name>). Table Columns:

• page_content (type: text)
• langchain_metadata (type: JSON)

elcarro_engine.drop_document_table(TABLE_NAME)
elcarro_engine.init_document_table(
    table_name=TABLE_NAME,
)

Save documents​

Save langchain documents withElCarroDocumentSaver.add_documents(<documents>).To initializeElCarroDocumentSaver class you need to provide 2 things:

1. elcarro_engine - An instance of aElCarroEngine engine.
2. table_name - The name of the table within the Oracle database to storelangchain documents.

from langchain_core.documentsimport Document
from langchain_google_el_carroimport ElCarroDocumentSaver

doc= Document(
    page_content="Banana",
    metadata={"type":"fruit","weight":100,"organic":1},
)

saver= ElCarroDocumentSaver(
    elcarro_engine=elcarro_engine,
    table_name=TABLE_NAME,
)
saver.add_documents([doc])

Load documents​

Load langchain documents withElCarroLoader.load()orElCarroLoader.lazy_load().lazy_load returns a generator that only queries database during the iteration.To initializeElCarroLoader class you need to provide:

1. elcarro_engine - An instance of aElCarroEngine engine.
2. table_name - The name of the table within the Oracle database to storelangchain documents.

from langchain_google_el_carroimport ElCarroLoader

loader= ElCarroLoader(elcarro_engine=elcarro_engine, table_name=TABLE_NAME)
docs= loader.lazy_load()
for docin docs:
print("Loaded documents:", doc)

Load documents via query​

Other than loading documents from a table, we can also choose to load documentsfrom a view generated from a SQL query. For example:

from langchain_google_el_carroimport ElCarroLoader

loader= ElCarroLoader(
    elcarro_engine=elcarro_engine,
    query=f"SELECT * FROM{TABLE_NAME} WHERE json_value(langchain_metadata, '$.organic') = '1'",
)
onedoc= loader.load()
print(onedoc)

The view generated from SQL query can have different schema than default table.In such cases, the behavior of ElCarroLoader is the same as loading from tablewith non-default schema. Please refer tosectionLoad documents with customized document page content & metadata.

Delete documents​

Delete a list of langchain documents from an Oracle tablewithElCarroDocumentSaver.delete(<documents>).

For a table with a default schema (page_content, langchain_metadata), thedeletion criteria is:

Arow should be deleted if there exists adocument in the list, such that

• document.page_content equalsrow[page_content]
• document.metadata equalsrow[langchain_metadata]

docs= loader.load()
print("Documents before delete:", docs)
saver.delete(onedoc)
print("Documents after delete:", loader.load())

Advanced Usage​

Load documents with customized document page content & metadata​

First we prepare an example table with non-default schema, and populate it withsome arbitrary data.

import sqlalchemy

create_table_query=f"""CREATE TABLE{TABLE_NAME} (
    fruit_id NUMBER GENERATED BY DEFAULT AS IDENTITY (START WITH 1),
    fruit_name VARCHAR2(100) NOT NULL,
    variety VARCHAR2(50),
    quantity_in_stock NUMBER(10) NOT NULL,
    price_per_unit NUMBER(6,2) NOT NULL,
    organic NUMBER(3) NOT NULL
)"""
elcarro_engine.drop_document_table(TABLE_NAME)

with elcarro_engine.connect()as conn:
    conn.execute(sqlalchemy.text(create_table_query))
    conn.commit()
    conn.execute(
        sqlalchemy.text(
f"""
            INSERT INTO{TABLE_NAME} (fruit_name, variety, quantity_in_stock, price_per_unit, organic)
            VALUES ('Apple', 'Granny Smith', 150, 0.99, 1)
            """
)
)
    conn.execute(
        sqlalchemy.text(
f"""
            INSERT INTO{TABLE_NAME} (fruit_name, variety, quantity_in_stock, price_per_unit, organic)
            VALUES ('Banana', 'Cavendish', 200, 0.59, 0)
            """
)
)
    conn.execute(
        sqlalchemy.text(
f"""
            INSERT INTO{TABLE_NAME} (fruit_name, variety, quantity_in_stock, price_per_unit, organic)
            VALUES ('Orange', 'Navel', 80, 1.29, 1)
            """
)
)
    conn.commit()

If we still load langchain documents with default parameters ofElCarroLoaderfrom this example table, thepage_content of loaded documents will be thefirst column of the table, andmetadata will be consisting of key-value pairsof all the other columns.

loader= ElCarroLoader(
    elcarro_engine=elcarro_engine,
    table_name=TABLE_NAME,
)
loaded_docs= loader.load()
print(f"Loaded Documents: [{loaded_docs}]")

We can specify the content and metadata we want to load by settingthecontent_columns andmetadata_columns when initializingtheElCarroLoader.

1. content_columns: The columns to write into thepage_content of thedocument.
2. metadata_columns: The columns to write into themetadata of the document.

For example here, the values of columns incontent_columns will be joinedtogether into a space-separated string, aspage_content of loaded documents,andmetadata of loaded documents will only contain key-value pairs of columnsspecified inmetadata_columns.

loader= ElCarroLoader(
    elcarro_engine=elcarro_engine,
    table_name=TABLE_NAME,
    content_columns=[
"variety",
"quantity_in_stock",
"price_per_unit",
"organic",
],
    metadata_columns=["fruit_id","fruit_name"],
)
loaded_docs= loader.load()
print(f"Loaded Documents: [{loaded_docs}]")

Save document with customized page content & metadata​

In order to save langchain document into table with customized metadata fieldswe need first create such a table viaElCarroEngine.init_document_table(), andspecify the list ofmetadata_columns we want it to have. In this example, thecreated table will have table columns:

• content (type: text): for storing fruit description.
• type (type VARCHAR2(200)): for storing fruit type.
• weight (type INT): for storing fruit weight.
• extra_json_metadata (type: JSON): for storing other metadata information of thefruit.

We can use the following parameterswithelcarro_engine.init_document_table() to create the table:

1. table_name: The name of the table within the Oracle database to storelangchain documents.
2. metadata_columns: A list ofsqlalchemy.Column indicating the list ofmetadata columns we need.
3. content_column: column name to storepage_content of langchaindocument. Default:"page_content", "VARCHAR2(4000)"
4. metadata_json_column: column name to store extraJSONmetadata of langchain document.Default:"langchain_metadata", "VARCHAR2(4000)".

elcarro_engine.drop_document_table(TABLE_NAME)
elcarro_engine.init_document_table(
    table_name=TABLE_NAME,
    metadata_columns=[
        sqlalchemy.Column("type", sqlalchemy.dialects.oracle.VARCHAR2(200)),
        sqlalchemy.Column("weight", sqlalchemy.INT),
],
    content_column="content",
    metadata_json_column="extra_json_metadata",
)

Save documents withElCarroDocumentSaver.add_documents(<documents>). As youcan see in this example,

• document.page_content will be saved intocontent column.
• document.metadata.type will be saved intotype column.
• document.metadata.weight will be saved intoweight column.
• document.metadata.organic will be saved intoextra_json_metadata column inJSON format.

doc= Document(
    page_content="Banana",
    metadata={"type":"fruit","weight":100,"organic":1},
)

print(f"Original Document: [{doc}]")

saver= ElCarroDocumentSaver(
    elcarro_engine=elcarro_engine,
    table_name=TABLE_NAME,
    content_column="content",
    metadata_json_column="extra_json_metadata",
)
saver.add_documents([doc])

loader= ElCarroLoader(
    elcarro_engine=elcarro_engine,
    table_name=TABLE_NAME,
    content_columns=["content"],
    metadata_columns=[
"type",
"weight",
],
    metadata_json_column="extra_json_metadata",
)

loaded_docs= loader.load()
print(f"Loaded Document: [{loaded_docs[0]}]")

Delete documents with customized page content & metadata​

We can also delete documents from table with customized metadata columnsviaElCarroDocumentSaver.delete(<documents>). The deletion criteria is:

Arow should be deleted if there exists adocument in the list, such that

• document.page_content equalsrow[page_content]
• For every metadata fieldk indocument.metadata
  • document.metadata[k] equalsrow[k] ordocument.metadata[k]equalsrow[langchain_metadata][k]
• There is no extra metadata field present inrow but notindocument.metadata.

loader= ElCarroLoader(elcarro_engine=elcarro_engine, table_name=TABLE_NAME)
saver.delete(loader.load())
print(f"Documents left:{len(loader.load())}")

More examples​

Please lookatdemo_doc_loader_basic.pyanddemo_doc_loader_advanced.pyforcomplete code examples.

Related​

• Document loaderconceptual guide
• Document loaderhow-to guides