Google Memorystore for Redis is a fully-managed service that is powered by the Redis in-memory data store to build application caches that provide sub-millisecond data access. Extend your database application to build AI-powered experiences leveraging Memorystore for Redis's Langchain integrations.

This notebook goes over how to useMemorystore for Redis tosave, load and delete langchain documents withMemorystoreDocumentLoader andMemorystoreDocumentSaver.

Learn more about the package onGitHub.

Before You Begin​

To run this notebook, you will need to do the following:

• Create a Google Cloud Project
• Enable the Memorystore for Redis API
• Create a Memorystore for Redis instance. Ensure that the version is greater than or equal to 5.0.

After confirmed access to database in the runtime environment of this notebook, filling the following values and run the cell before running example scripts.

# @markdown Please specify an endpoint associated with the instance and a key prefix for demo purpose.
ENDPOINT="redis://127.0.0.1:6379"# @param {type:"string"}
KEY_PREFIX="doc:"# @param {type:"string"}

🦜🔗 Library Installation​

The integration lives in its ownlangchain-google-memorystore-redis package, so we need to install it.

%pip install-upgrade--quiet langchain-google-memorystore-redis

Colab only: Uncomment the following cell to restart the kernel or use the button to restart the kernel. For Vertex AI Workbench you can restart the terminal using the button on top.

# # Automatically restart kernel after installs so that your environment can access the new packages
# import IPython

# app = IPython.Application.instance()
# app.kernel.do_shutdown(True)

☁ Set Your Google Cloud Project​

Set your Google Cloud project so that you can leverage Google Cloud resources within this notebook.

If you don't know your project ID, try the following:

• Rungcloud config list.
• Rungcloud projects list.
• See the support page:Locate the project ID.

# @markdown Please fill in the value below with your Google Cloud project ID and then run the cell.

PROJECT_ID="my-project-id"# @param {type:"string"}

# Set the project id
!gcloud configset project{PROJECT_ID}

🔐 Authentication​

Authenticate to Google Cloud as the IAM user logged into this notebook in order to access your Google Cloud Project.

• If you are using Colab to run this notebook, use the cell below and continue.
• If you are using Vertex AI Workbench, check out the setup instructionshere.

from google.colabimport auth

auth.authenticate_user()

Basic Usage​

Save documents​

Save langchain documents withMemorystoreDocumentSaver.add_documents(<documents>). To initializeMemorystoreDocumentSaver class you need to provide 2 things:

1. client - Aredis.Redis client object.
2. key_prefix - A prefix for the keys to store Documents in Redis.

The Documents will be stored into randomly generated keys with the specified prefix ofkey_prefix. Alternatively, you can designate the suffixes of the keys by specifyingids in theadd_documents method.

import redis
from langchain_core.documentsimport Document
from langchain_google_memorystore_redisimport MemorystoreDocumentSaver

test_docs=[
    Document(
        page_content="Apple Granny Smith 150 0.99 1",
        metadata={"fruit_id":1},
),
    Document(
        page_content="Banana Cavendish 200 0.59 0",
        metadata={"fruit_id":2},
),
    Document(
        page_content="Orange Navel 80 1.29 1",
        metadata={"fruit_id":3},
),
]
doc_ids=[f"{i}"for iinrange(len(test_docs))]

redis_client= redis.from_url(ENDPOINT)
saver= MemorystoreDocumentSaver(
    client=redis_client,
    key_prefix=KEY_PREFIX,
    content_field="page_content",
)
saver.add_documents(test_docs, ids=doc_ids)

Load documents​

Initialize a loader that loads all documents stored in the Memorystore for Redis instance with a specific prefix.

Load langchain documents withMemorystoreDocumentLoader.load() orMemorystoreDocumentLoader.lazy_load().lazy_load returns a generator that only queries database during the iteration. To initializeMemorystoreDocumentLoader class you need to provide:

1. client - Aredis.Redis client object.
2. key_prefix - A prefix for the keys to store Documents in Redis.

import redis
from langchain_google_memorystore_redisimport MemorystoreDocumentLoader

redis_client= redis.from_url(ENDPOINT)
loader= MemorystoreDocumentLoader(
    client=redis_client,
    key_prefix=KEY_PREFIX,
    content_fields=set(["page_content"]),
)
for docin loader.lazy_load():
print("Loaded documents:", doc)

Delete documents​

Delete all of keys with the specified prefix in the Memorystore for Redis instance withMemorystoreDocumentSaver.delete(). You can also specify the suffixes of the keys if you know.

docs= loader.load()
print("Documents before delete:", docs)

saver.delete(ids=[0])
print("Documents after delete:", loader.load())

saver.delete()
print("Documents after delete all:", loader.load())

Advanced Usage​

Customize Document Page Content & Metadata​

When initializing a loader with more than 1 content field, thepage_content of the loaded Documents will contain a JSON-encoded string with top level fields equal to the specified fields incontent_fields.

If themetadata_fields are specified, themetadata field of the loaded Documents will only have the top level fields equal to the specifiedmetadata_fields. If any of the values of the metadata fields is stored as a JSON-encoded string, it will be decoded prior to being loaded to the metadata fields.

loader= MemorystoreDocumentLoader(
    client=redis_client,
    key_prefix=KEY_PREFIX,
    content_fields=set(["content_field_1","content_field_2"]),
    metadata_fields=set(["title","author"]),
)

Related​

• Document loaderconceptual guide
• Document loaderhow-to guides