This will help you get started with Ollama embedding models using LangChain. For detailed documentation onOllamaEmbeddings features and configuration options, please refer to theAPI reference.

Overview​

Integration details​

Setup​

First, followthese instructions to set up and run a local Ollama instance:

• Download and install Ollama onto the available supported platforms (including Windows Subsystem for Linux aka WSL, macOS, and Linux)
  • macOS users can install via Homebrew withbrew install ollama and start withbrew services start ollama
• Fetch available LLM model viaollama pull <name-of-model>
  • View a list of available models via themodel library
  • e.g.,ollama pull llama3
• This will download the default tagged version of the model. Typically, the default points to the latest, smallest sized-parameter model.

On Mac, the models will be download to~/.ollama/models

On Linux (or WSL), the models will be stored at/usr/share/ollama/.ollama/models

• Specify the exact version of the model of interest as suchollama pull vicuna:13b-v1.5-16k-q4_0 (View thevarious tags for theVicuna model in this instance)
• To view all pulled models, useollama list
• To chat directly with a model from the command line, useollama run <name-of-model>
• View theOllama documentation for more commands. You can runollama help in the terminal to see available commands.

To enable automated tracing of your model calls, set yourLangSmith API key:

# os.environ["LANGSMITH_TRACING"] = "true"
# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")

Installation​

The LangChain Ollama integration lives in thelangchain-ollama package:

%pip install-qU langchain-ollama

Note: you may need to restart the kernel to use updated packages.

Instantiation​

Now we can instantiate our model object and generate embeddings:

from langchain_ollamaimport OllamaEmbeddings

embeddings= OllamaEmbeddings(
    model="llama3",
)

Indexing and Retrieval​

Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see ourRAG tutorials.

Below, see how to index and retrieve data using theembeddings object we initialized above. In this example, we will index and retrieve a sample document in theInMemoryVectorStore.

# Create a vector store with a sample text
from langchain_core.vectorstoresimport InMemoryVectorStore

text="LangChain is the framework for building context-aware reasoning applications"

vectorstore= InMemoryVectorStore.from_texts(
[text],
    embedding=embeddings,
)

# Use the vectorstore as a retriever
retriever= vectorstore.as_retriever()

# Retrieve the most similar text
retrieved_documents= retriever.invoke("What is LangChain?")

# Show the retrieved document's content
print(retrieved_documents[0].page_content)

LangChain is the framework for building context-aware reasoning applications

Direct Usage​

Under the hood, the vectorstore and retriever implementations are callingembeddings.embed_documents(...) andembeddings.embed_query(...) to create embeddings for the text(s) used infrom_texts and retrievalinvoke operations, respectively.

You can directly call these methods to get embeddings for your own use cases.

Embed single texts​

You can embed single texts or documents withembed_query:

single_vector= embeddings.embed_query(text)
print(str(single_vector)[:100])# Show the first 100 characters of the vector

[-0.0039849705, 0.023019705, -0.001768838, -0.0058736936, 0.00040999008, 0.017861595, -0.011274585,

Embed multiple texts​

You can embed multiple texts withembed_documents:

text2=(
"LangGraph is a library for building stateful, multi-actor applications with LLMs"
)
two_vectors= embeddings.embed_documents([text, text2])
for vectorin two_vectors:
print(str(vector)[:100])# Show the first 100 characters of the vector

[-0.0039849705, 0.023019705, -0.001768838, -0.0058736936, 0.00040999008, 0.017861595, -0.011274585,
[-0.0066985516, 0.009878328, 0.008019467, -0.009384944, -0.029560851, 0.025744654, 0.004872892, -0.0

API Reference​

For detailed documentation onOllamaEmbeddings features and configuration options, please refer to theAPI reference.

Related​

• Embedding modelconceptual guide
• Embedding modelhow-to guides