HippoRAG 2 is a powerful memory framework for LLMs that enhances their ability to recognize and utilize connections in new knowledge—mirroring a key function of human long-term memory.

Check out our papers to learn more:

Installation

conda create -n hipporag python=3.10conda activate hipporagpip install hipporag

export CUDA_VISIBLE_DEVICES=0,1,2,3export HF_HOME=<path to Huggingface home directory>export OPENAI_API_KEY=<your openai api key># if you want to use OpenAI modelconda activate hipporag

Quick Start

OpenAI Models

fromhipporagimportHippoRAG# Prepare datasets and evaluationdocs= ["Oliver Badman is a politician.","George Rankin is a politician.","Thomas Marwick is a politician.","Cinderella attended the royal ball.","The prince used the lost glass slipper to search the kingdom.","When the slipper fit perfectly, Cinderella was reunited with the prince.","Erik Hort's birthplace is Montebello.","Marina is bom in Minsk.","Montebello is a part of Rockland County."]save_dir='outputs'# Define save directory for HippoRAG objects (each LLM/Embedding model combination will create a new subdirectory)llm_model_name='gpt-4o-mini'# Any OpenAI model nameembedding_model_name='nvidia/NV-Embed-v2'# Embedding model name (NV-Embed, GritLM or Contriever for now)#Startup a HippoRAG instancehipporag=HippoRAG(save_dir=save_dir,llm_model_name=llm_model_name,embedding_model_name=embedding_model_name)#Run indexinghipporag.index(docs=docs)#Separate Retrieval & QAqueries= ["What is George Rankin's occupation?","How did Cinderella reach her happy ending?","What county is Erik Hort's birthplace a part of?"]retrieval_results=hipporag.retrieve(queries=queries,num_to_retrieve=2)qa_results=hipporag.rag_qa(retrieval_results)#Combined Retrieval & QArag_results=hipporag.rag_qa(queries=queries)#For Evaluationanswers= [    ["Politician"],    ["By going to the ball."],    ["Rockland County"]]gold_docs= [    ["George Rankin is a politician."],    ["Cinderella attended the royal ball.","The prince used the lost glass slipper to search the kingdom.","When the slipper fit perfectly, Cinderella was reunited with the prince."],    ["Erik Hort's birthplace is Montebello.","Montebello is a part of Rockland County."]]rag_results=hipporag.rag_qa(queries=queries,gold_docs=gold_docs,gold_answers=answers)

Local Deployment (vLLM)

export CUDA_VISIBLE_DEVICES=0,1export VLLM_WORKER_MULTIPROC_METHOD=spawnexport HF_HOME=<path to Huggingface home directory>conda activate hipporag# vllm should be in this environment# Tune gpu-memory-utilization or max_model_len to fit your GPU memory, if OOM occursvllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 2 --max_model_len 4096 --gpu-memory-utilization 0.95

save_dir='outputs'# Define save directory for HippoRAG objects (each LLM/Embedding model combination will create a new subdirectory)llm_model_name=# Any OpenAI model nameembedding_model_name=# Embedding model name (NV-Embed, GritLM or Contriever for now)llm_base_url=# Base url for your deployed LLM (i.e. http://localhost:8000/v1)hipporag=HippoRAG(save_dir=save_dir,llm_model_name=llm_model,embedding_model_name=embedding_model_name,llm_base_url=llm_base_url)# Same Indexing, Retrieval and QA as running OpenAI models above

Running Experiments

Data for Reproducibility

Running Indexing & QA

export CUDA_VISIBLE_DEVICES=0,1,2,3export HF_HOME=<path to Huggingface home directory>export OPENAI_API_KEY=<your openai api key># if you want to use OpenAI modelconda activate hipporag

Run with OpenAI Model

dataset=sample# or any other dataset under `reproduce/dataset`# Run OpenAI modelpython main.py --dataset$dataset --llm_base_url https://api.openai.com/v1 --llm_name gpt-4o-mini --embedding_name nvidia/NV-Embed-v2

Run with vLLM (Llama)

export CUDA_VISIBLE_DEVICES=0,1export VLLM_WORKER_MULTIPROC_METHOD=spawnexport HF_HOME=<path to Huggingface home directory>conda activate hipporag# vllm should be in this environment# Tune gpu-memory-utilization or max_model_len to fit your GPU memory, if OOM occursvllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 2 --max_model_len 4096 --gpu-memory-utilization 0.95

export CUDA_VISIBLE_DEVICES=2,3# set another GPUs while vLLM server is runningexport HF_HOME=<path to Huggingface home directory>dataset=samplepython main.py --dataset$dataset --llm_base_url http://localhost:8000/v1 --llm_name meta-llama/Llama-3.3-70B-Instruct --embedding_name nvidia/NV-Embed-v2

Advanced: Run with vLLM offline batch

export CUDA_VISIBLE_DEVICES=0,1,2,3# use all GPUs for faster offline indexingexport VLLM_WORKER_MULTIPROC_METHOD=spawnexport HF_HOME=<path to Huggingface home directory>export OPENAI_API_KEY=''dataset=samplepython main.py --dataset$dataset --llm_name meta-llama/Llama-3.3-70B-Instruct --openie_mode offline --skip_graph

Debugging Note

rm reproduce/dataset/openie_results/openie_sample_results_ner_meta-llama_Llama-3.3-70B-Instruct_3.jsonrm -rf outputs/sample/sample_meta-llama_Llama-3.3-70B-Instruct_nvidia_NV-Embed-v2

Custom Datasets

Retrieval Corpus JSON

[  {"title":"FIRST PASSAGE TITLE","text":"FIRST PASSAGE TEXT","idx":0  },  {"title":"SECOND PASSAGE TITLE","text":"SECOND PASSAGE TEXT","idx":1  }]

(Optional) Query JSON

[  {"id":"sample/question_1.json","question":"QUESTION","answer": ["ANSWER"    ],"answerable":true,"paragraphs": [      {"title":"{FIRST SUPPORTING PASSAGE TITLE}","text":"{FIRST SUPPORTING PASSAGE TEXT}","is_supporting":true,"idx":0      },      {"title":"{SECOND SUPPORTING PASSAGE TITLE}","text":"{SECOND SUPPORTING PASSAGE TEXT}","is_supporting":true,"idx":1      }    ]  }]

(Optional) Chunking Corpus

Code Structure

📦 .│-- 📂 src/hipporag│   ├── 📂 embedding_model          # Implementation of all embedding models│   │   ├── __init__.py             # Getter function for get specific embedding model classes|   |   ├── base.py                 # Base embedding model class `BaseEmbeddingModel` to inherit and `EmbeddingConfig`|   |   ├── NVEmbedV2.py            # Implementation of NV-Embed-v2 model|   |   ├── ...│   ├── 📂 evaluation               # Implementation of all evaluation metrics│   │   ├── __init__.py|   |   ├── base.py                 # Base evaluation metric class `BaseMetric` to inherit│   │   ├── qa_eval.py              # Eval metrics for QA│   │   ├── retrieval_eval.py       # Eval metrics for retrieval│   ├── 📂 information_extraction  # Implementation of all information extraction models│   │   ├── __init__.py|   |   ├── openie_openai_gpt.py    # Model for OpenIE with OpenAI GPT|   |   ├── openie_vllm_offline.py  # Model for OpenIE with LLMs deployed offline with vLLM│   ├── 📂 llm                      # Classes for inference with large language models│   │   ├── __init__.py             # Getter function|   |   ├── base.py                 # Config class for LLM inference and base LLM inference class to inherit|   |   ├── openai_gpt.py           # Class for inference with OpenAI GPT|   |   ├── vllm_llama.py           # Class for inference using a local vLLM server|   |   ├── vllm_offline.py         # Class for inference using the vLLM API directly│   ├── 📂 prompts                  # Prompt templates and prompt template manager class|   │   ├── 📂 dspy_prompts         # Prompts for filtering|   │   │   ├── ...|   │   ├── 📂 templates            # All prompt templates for template manager to load|   │   │   ├── README.md           # Documentations of usage of prompte template manager and prompt template files|   │   │   ├── __init__.py|   │   │   ├── triple_extraction.py|   │   │   ├── ...│   │   ├── __init__.py|   |   ├── linking.py              # Instruction for linking|   |   ├── prompt_template_manager.py  # Implementation of prompt template manager│   ├── 📂 utils                    # All utility functions used across this repo (the file name indicates its relevant usage)│   │   ├── config_utils.py         # We use only one config across all modules and its setup is specified here|   |   ├── ...│   ├── __init__.py│   ├── HippoRAG.py          # Highest level class for initiating retrieval, question answering, and evaluations│   ├── embedding_store.py   # Storage database to load, manage and save embeddings for passages, entities and facts.│   ├── rerank.py            # Reranking and filtering methods│-- 📂 examples│   ├── ...│   ├── ...│-- 📜 README.md│-- 📜 requirements.txt   # Dependencies list│-- 📜 .gitignore         # Files to exclude from Git

Contact

Citation

HippoRAG 2

@misc{gutiérrez2025ragmemorynonparametriccontinual,      title={From RAG to Memory: Non-Parametric Continual Learning for Large Language Models},       author={Bernal Jiménez Gutiérrez and Yiheng Shu and Weijian Qi and Sizhe Zhou and Yu Su},      year={2025},      eprint={2502.14802},      archivePrefix={arXiv},      primaryClass={cs.CL},      url={https://arxiv.org/abs/2502.14802}, }

HippoRAG

@inproceedings{gutiérrez2024hipporag,      title={HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models},       author={Bernal Jiménez Gutiérrez and Yiheng Shu and Yu Gu and Michihiro Yasunaga and Yu Su},      booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},      year={2024},      url={https://openreview.net/forum?id=hkujvAPVsg}

TODO: