Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

关于【构建向量索引数据库】示例的报错,提示:InvalidRequestError: This model's maximum context length is 4097 tokens, however you requested 23694 tokens (23438 in your prompt; 256 for the completion). Please reduce your prompt; or completion length. #58

Open
@yuanluochenfeng

Description

@yuanluochenfeng

我是小白一枚,感谢提供的langchain的中文文档,帮助我入门学习。

我是按照文档的案例逐个进行本地测试的

当【构建向量索引数据库】时,按照文中提示的

from langchain.vectorstores import Chroma# 持久化数据docsearch = Chroma.from_documents(documents, embeddings, persist_directory="D:/vector_store")docsearch.persist()# 加载数据docsearch = Chroma(persist_directory="D:/vector_store", embedding_function=embeddings)

然后再【构建本地知识库问答机器人】的示例代码的基础上进行修改的,以下为部分内容:

# 加载文件夹中的所有txt类型的文件loader = DirectoryLoader('/Users/ldx/Documents/data/document', glob='**/*.*')# 将数据转成 document 对象,每个文件会作为一个 documentdocuments = loader.load()# 初始化加载器text_splitter = CharacterTextSplitter(chunk_size=100, chunk_overlap=0)# 切割加载的 documentsplit_docs = text_splitter.split_documents(documents)# 持久化数据docsearch = Chroma.from_documents(documents, embeddings, persist_directory="/Users/ldx/Documents/data/chroma_data")docsearch.persist()# 加载数据docsearch = Chroma(persist_directory="/Users/ldx/Documents/data/chroma_data", embedding_function=OpenAIEmbeddings())# 创建问答对象qa = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type="stuff", retriever=docsearch.as_retriever(), return_source_documents=True)# 进行问答,问答内容省略result = qa({"query": ".............."})print(result)

然后就报错了!!!
调试了好久,也没成功,后来仔细对比了一下【构建本地知识库问答机器人】示例,发现
docsearch = Chroma.from_documents(split_docs, embeddings)
传入的是split_docs,切割后的document,而不是原始的document.
修改后,
docsearch = Chroma.from_documents(documents, embeddings, persist_directory="/Users/ldx/Documents/data/chroma_data")可以正常运行了。

看来还是自己粗心,记录一下,万一也有人遇到呢。

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions


      [8]ページ先頭

      ©2009-2025 Movatter.jp