Quickstart

Reranking is the process of reordering a list of items based on some criteria. In the context of search, reranking is used to reorder the search results returned by a search engine based on some criteria. This can be useful when the initial ranking of the search results is not satisfactory or when the user has provided additional information that can be used to improve the ranking of the search results.

LanceDB comes with some built-in rerankers. Some of the rerankers that are available in LanceDB are:

Using a Reranker

Using rerankers is optional for vector and FTS. However, for hybrid search, rerankers are required. To use a reranker, you need to create an instance of the reranker and pass it to thererank method of the query builder:

importlancedbfromlancedb.embeddingsimportget_registryfromlancedb.pydanticimportLanceModel,Vectorfromlancedb.rerankersimportCohereRerankerembedder=get_registry().get("sentence-transformers").create()db=lancedb.connect("~/.lancedb")classSchema(LanceModel):text:str=embedder.SourceField()vector:Vector(embedder.ndims())=embedder.VectorField()data=[{"text":"hello world"},{"text":"goodbye world"}]tbl=db.create_table("test",data)reranker=CohereReranker(api_key="your_api_key")# Run vector search with a rerankerresult=tbl.search("hello").rerank(reranker).to_list()# Run FTS search with a rerankerresult=tbl.search("hello",query_type="fts").rerank(reranker).to_list()# Run hybrid search with a rerankertbl.create_fts_index("text")result=tbl.search("hello",query_type="hybrid").rerank(reranker).to_list()

Multi-vector reranking

Most rerankers support reranking based on multiple vectors. To rerank based on multiple vectors, you can pass a list of vectors to thererank method. Here's an example of how to rerank based on multiple vector columns using theCrossEncoderReranker:

fromlancedb.rerankersimportCrossEncoderRerankerreranker=CrossEncoderReranker()query="hello"res1=table.search(query,vector_column_name="vector").limit(3)res2=table.search(query,vector_column_name="text_vector").limit(3)res3=table.search(query,vector_column_name="meta_vector").limit(3)reranked=reranker.rerank_multivector([res1,res2,res3],deduplicate=True)

Available Rerankers

LanceDB comes with the following built-in rerankers:

• Cohere Reranker
• Cross Encoder Reranker
• ColBERT Reranker
• OpenAI Reranker
• Linear Combination Reranker
• Jina Reranker
• AnswerDotAI Rerankers
• Reciprocal Rank Fusion Reranker
• VoyageAI Reranker

Creating Custom Rerankers

LanceDB also you to create custom rerankers by extending the baseReranker class. The custom reranker should implement thererank method that takes a list of search results and returns a reranked list of search results. This is covered in more detail in theCreating Custom Rerankers section.