技术领域Technical field
本公开涉及人工智能技术领域，具体为自然语言处理、深度学习、预训练模型技术领域，可应用于智慧城市、智慧政务等场景。本公开具体涉及一种文本检索方法、装置、电子设备、存储介质和计算机程序产品。This disclosure relates to the field of artificial intelligence technology, specifically the technical fields of natural language processing, deep learning, and pre-training models, and can be applied to scenarios such as smart cities and smart government affairs. The present disclosure specifically relates to a text retrieval method, device, electronic device, storage medium and computer program product.
背景技术Background technique
相关技术中通常会采用文本截断的方式进行文本检索，也即当检索内容长度超过某个限制后，仅取限制范围内的文本内容进行文本检索。但是，在利用包含复杂信息的长文本进行文本检索时，相关的关键信息可能分布在检索内容的各个位置。如果应用文本截断的方式来进行文本检索，会遗漏部分关键信息，导致检索结果不准确。In related technologies, text truncation is usually used for text retrieval, that is, when the length of the retrieval content exceeds a certain limit, only the text content within the limit is used for text retrieval. However, when using long texts containing complex information for text retrieval, relevant key information may be distributed in various locations of the retrieved content. If text truncation is used for text retrieval, some key information will be missed, resulting in inaccurate retrieval results.
发明内容Contents of the invention
本公开提供了一种文本检索方法、装置、电子设备、存储介质和计算机程序产品。The present disclosure provides a text retrieval method, device, electronic device, storage medium and computer program product.
根据本公开的一方面，提供了一种文本检索方法，包括：根据检索文本中的多个关键词，获取与检索文本相关联的多个候选文本；对检索文本进行解析处理，得到与检索文本对应的第一特征信息、第二特征信息和第三特征信息；分别对多个候选文本进行解析处理，得到与多个候选文本各自对应的候选特征信息；针对每个候选文本，根据第一特征信息、第二特征信息、第三特征信息和候选特征信息，确定候选文本与检索文本之间的匹配度；以及根据匹配度对多个候选文本进行排序，并基于排序结果，获得与检索文本对应的检索结果。According to one aspect of the present disclosure, a text retrieval method is provided, including: obtaining multiple candidate texts associated with the retrieved text based on multiple keywords in the retrieved text; parsing the retrieved text to obtain the retrieved text Corresponding first feature information, second feature information and third feature information; analyze and process multiple candidate texts respectively to obtain candidate feature information corresponding to the multiple candidate texts; for each candidate text, according to the first feature information, second feature information, third feature information and candidate feature information to determine the matching degree between the candidate text and the retrieved text; and sort multiple candidate texts according to the matching degree, and based on the sorting results, obtain the correspondence with the retrieved text search results.
根据本公开的另一方面，提供了一种文本检索装置，包括：获取模块，用于根据检索文本中的多个关键词，获取与检索文本相关联的多个候选文本；第一解析模块，用于对检索文本进行解析处理，得到与检索文本对应的第一特征信息、第二特征信息和第三特征信息；第二解析模块，用于分别对多个候选文本进行解析处理，得到与多个候选文本各自对应的候选特征信息；匹配模块，用于针对每个候选文本，根据第一特征信息、第二特征信息、第三特征信息和候选特征信息，确定候选文本与检索文本之间的匹配度；以及排序模块，用于根据匹配度对多个候选文本进行排序，并基于排序结果，获得与检索文本对应的检索结果。According to another aspect of the present disclosure, a text retrieval device is provided, including: an acquisition module, configured to acquire multiple candidate texts associated with the retrieval text based on multiple keywords in the retrieval text; a first parsing module, It is used to parse and process the retrieval text to obtain the first feature information, second feature information and third feature information corresponding to the retrieval text; the second parsing module is used to parse and process multiple candidate texts respectively to obtain multiple candidate texts. Candidate feature information corresponding to each candidate text; a matching module, for each candidate text, determining the relationship between the candidate text and the retrieved text based on the first feature information, the second feature information, the third feature information and the candidate feature information. Matching degree; and a sorting module, used to sort multiple candidate texts according to the matching degree, and obtain retrieval results corresponding to the retrieved text based on the sorting results.
根据本公开的另一方面，提供了一种电子设备，包括：至少一个处理器；以及与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行根据本公开提供的方法。According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are At least one processor executes to enable at least one processor to execute the method provided in accordance with the present disclosure.
根据本公开的另一方面，提供了一种存储有计算机指令的非瞬时计算机可读存储介质，该计算机指令用于使计算机执行根据本公开提供的方法。According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a method provided according to the present disclosure.
根据本公开的另一方面，提供了一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现根据本公开提供的方法。According to another aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a method provided in accordance with the present disclosure.
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.
附图说明Description of the drawings
附图用于更好地理解本方案，不构成对本公开的限定。其中：The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. in:
图1是根据本公开实施例的可以应用文本检索方法和装置的示例性系统架构示意图；Figure 1 is a schematic diagram of an exemplary system architecture to which text retrieval methods and devices can be applied according to embodiments of the present disclosure;
图2是根据本公开实施例的文本检索方法的流程图；Figure 2 is a flow chart of a text retrieval method according to an embodiment of the present disclosure;
图3A和图3B是根据本公开实施例的文本检索方法的示意图；3A and 3B are schematic diagrams of text retrieval methods according to embodiments of the present disclosure;
图4是根据本公开实施例的文本检索装置的框图；以及Figure 4 is a block diagram of a text retrieval device according to an embodiment of the present disclosure; and
图5是用来实现本公开实施例的文本检索方法的电子设备的框图。FIG. 5 is a block diagram of an electronic device used to implement a text retrieval method according to an embodiment of the present disclosure.
具体实施方式Detailed ways
以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered to be exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.
在此使用的术语仅仅是为了描述具体实施例，而并非意在限制本公开。在此使用的术语“包括”、“包含”等表明了所述特征、步骤、操作和/或部件的存在，但是并不排除存在或添加一个或多个其他特征、步骤、操作或部件。The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. The terms "comprising," "comprising," and the like, as used herein, indicate the presence of stated features, steps, operations, and/or components but do not exclude the presence or addition of one or more other features, steps, operations, or components.
在此使用的所有术语(包括技术和科学术语)具有本领域技术人员通常所理解的含义，除非另外定义。应注意，这里使用的术语应解释为具有与本说明书的上下文相一致的含义，而不应以理想化或过于刻板的方式来解释。All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used here should be interpreted to have meanings consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
在使用类似于“A、B和C等中至少一个”这样的表述的情况下，一般来说应该按照本领域技术人员通常理解该表述的含义来予以解释(例如，“具有A、B和C中至少一个的系统”应包括但不限于单独具有A、单独具有B、单独具有C、具有A和B、具有A和C、具有B和C、和/或具有A、B、C的系统等)。Where an expression similar to "at least one of A, B, C, etc." is used, it should generally be interpreted in accordance with the meaning that a person skilled in the art generally understands the expression to mean (e.g., "having A, B and C "A system with at least one of" shall include, but is not limited to, systems with A alone, B alone, C alone, A and B, A and C, B and C, and/or systems with A, B, C, etc. ).
图1是根据本公开的实施例的可以应用文本检索方法和装置的示例性系统架构示意图。需要注意的是，图1所示仅为可以应用本公开实施例的系统架构的示例，以帮助本领域技术人员理解本公开的技术内容，但并不意味着本公开实施例不可以用于其他设备、系统、环境或场景。FIG. 1 is a schematic diagram of an exemplary system architecture to which text retrieval methods and devices may be applied according to embodiments of the present disclosure. It should be noted that Figure 1 is only an example of a system architecture to which embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be used in other applications. Device, system, environment or scenario.
如图1所示，根据该实施例的系统架构100可以包括终端设备101、102、103，网络104和服务器105。网络104用以在终端设备101、102、103和服务器105之间提供通信链路的介质。网络104可以包括各种连接类型，例如有线和/或无线通信链路等等。As shown in Figure 1, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium used to provide communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired and/or wireless communication links, and the like.
用户可以使用终端设备101、102、103通过网络104与服务器105交互，以接收或发送消息等。终端设备101、102、103上可以安装有各种客户端应用。例如，知识阅读类应用、文档处理类应用、网页浏览器应用、搜索类应用、即时通信工具、邮箱客户端或社交平台软件等(仅为示例)。Users can use terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various client applications can be installed on the terminal devices 101, 102, and 103. For example, knowledge reading applications, document processing applications, web browser applications, search applications, instant messaging tools, email clients or social platform software, etc. (only examples).
终端设备101、102、103可以是具有显示屏并且支持网页浏览的各种电子设备，包括但不限于智能手机、平板电脑、膝上型便携计算机和台式计算机等等。The terminal devices 101, 102, and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
服务器105可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或分布式系统，还可以是提供云服务、云计算、网络服务、中间件服务等基础云计算服务的云服务器。The server 105 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud computing, network services, and middleware services.
服务器105可以是提供各种服务的服务器，例如对用户利用终端设备101、102、103所浏览的网站提供支持的后台管理服务器(仅为示例)。后台管理服务器可以对接收到的用户请求等数据进行分析等处理，并将处理结果(例如根据用户请求获取或生成的网页、信息、或数据等)反馈给终端设备。The server 105 may be a server that provides various services, such as a backend management server that provides support for websites browsed by users using the terminal devices 101, 102, and 103 (example only). The background management server can analyze and process the received user request and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device.
例如，服务器105可以通过网络104获取来自终端设备101、102、103的检索文本，并基于检索文本中的多个关键词，获取与检索文本相关联的多个候选文本。之后，对检索文本进行解析处理，得到与检索文本对应的第一特征信息、第二特征信息和第三特征信息。然后分别对多个候选文本进行解析处理，得到与多个候选文本各自对应的候选特征信息。之后，针对每个候选文本，根据第一特征信息、第二特征信息、第三特征信息和候选特征信息，确定候选文本与检索文本之间的匹配度，以及根据匹配度对多个候选文本进行排序，并基于排序结果，获得与检索文本对应的检索结果。在一些示例中，服务器105还可以将与检索文本对应的检索结果发送给终端设备101、102、103，以便用户获取针对检索文本的检索结果。For example, the server 105 can obtain the search text from the terminal devices 101, 102, and 103 through the network 104, and obtain multiple candidate texts associated with the search text based on multiple keywords in the search text. Afterwards, the retrieval text is parsed to obtain the first feature information, the second feature information and the third feature information corresponding to the retrieval text. Then the multiple candidate texts are parsed and processed respectively to obtain candidate feature information corresponding to the multiple candidate texts. Afterwards, for each candidate text, determine the matching degree between the candidate text and the retrieved text based on the first feature information, the second feature information, the third feature information and the candidate feature information, and conduct a search on the multiple candidate texts based on the matching degree. Sort, and based on the sorting results, obtain the search results corresponding to the retrieved text. In some examples, the server 105 can also send the retrieval results corresponding to the retrieval text to the terminal devices 101, 102, and 103, so that the user can obtain the retrieval results for the retrieval text.
需要说明的是，本公开实施例所提供的文本检索方法一般可以由服务器105执行。相应地，本公开实施例所提供的文本检索装置-般可以设置于服务器105中。It should be noted that the text retrieval method provided by the embodiment of the present disclosure can generally be executed by the server 105 . Correspondingly, the text retrieval device provided by the embodiment of the present disclosure can generally be provided in the server 105.
备选地，本公开实施例所提供的文本检索方法也可以由不同于服务器105且能够与终端设备101、102、103和/或服务器105通信的服务器或服务器集群执行。相应地，本公开实施例所提供的文本检索装置也可以设置于不同于服务器105且能够与终端设备101、102、103和/或服务器105通信的服务器或服务器集群中。Alternatively, the text retrieval method provided by the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and/or the server 105. Correspondingly, the text retrieval device provided by the embodiment of the present disclosure can also be provided in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and/or the server 105.
应该理解，图1中的终端设备、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备、网络和服务器。It should be understood that the number of terminal devices, networks and servers in Figure 1 is only illustrative. Depending on implementation needs, there can be any number of end devices, networks, and servers.
应注意，以下方法中各个操作的序号仅作为该操作的表示以便描述，而不应被看作表示该各个操作的执行顺序。除非明确指出，否则该方法不需要完全按照所示顺序来执行。It should be noted that the sequence number of each operation in the following method is only used as a representation of the operation for the purpose of description, and should not be regarded as indicating the execution order of the respective operations. Unless explicitly stated, the methods need not be performed in exactly the order shown.
图2是根据本公开的实施例的文本检索方法的流程图。Figure 2 is a flowchart of a text retrieval method according to an embodiment of the present disclosure.
如图2所示，文本检索方法200例如可以包括操作S210～S250。As shown in FIG. 2 , the text retrieval method 200 may include operations S210 to S250, for example.
在操作S210，根据检索文本中的多个关键词，获取与检索文本相关联的多个候选文本。In operation S210, multiple candidate texts associated with the retrieved text are obtained based on multiple keywords in the retrieved text.
在操作S220，对检索文本进行解析处理，得到与检索文本对应的第一特征信息、第二特征信息和第三特征信息。In operation S220, the retrieval text is parsed to obtain first feature information, second feature information, and third feature information corresponding to the retrieval text.
在操作S230，分别对多个候选文本进行解析处理，得到与多个候选文本各自对应的候选特征信息。In operation S230, multiple candidate texts are analyzed respectively to obtain candidate feature information corresponding to each of the multiple candidate texts.
在操作S240，针对每个候选文本，根据第一特征信息、第二特征信息、第三特征信息和候选特征信息，确定候选文本与检索文本之间的匹配度。In operation S240, for each candidate text, a matching degree between the candidate text and the retrieved text is determined based on the first feature information, the second feature information, the third feature information and the candidate feature information.
在操作S250，根据匹配度对多个候选文本进行排序，并基于排序结果，获得与检索文本对应的检索结果。In operation S250, multiple candidate texts are sorted according to the matching degree, and based on the sorting results, a retrieval result corresponding to the retrieved text is obtained.
根据本公开的实施例，检索文本例如可以是用户输入的文本内容，也可以是将用户输入的语音信息进行转换得到的文本内容，本公开对检索文本的获取方式不做限定。According to embodiments of the present disclosure, the search text may be, for example, text content input by the user, or text content obtained by converting voice information input by the user. The present disclosure does not limit the acquisition method of the search text.
检索文本例如可以包括多个关键词。该多个关键词可以用于表征检索文本中的关键信息，而这些关键信息与检索目的相关。The search text may include multiple keywords, for example. The multiple keywords can be used to characterize key information in the retrieval text, and these key information are related to the retrieval purpose.
在本公开的实施例中，根据检索文本中的多个关键词，可以检索得到与检索文本相关联的多个候选文本。由此，可以基于检索文本中的多个关键词，实现对检索文本的粗检索。In embodiments of the present disclosure, multiple candidate texts associated with the search text can be retrieved based on multiple keywords in the search text. Thus, a rough search of the search text can be implemented based on multiple keywords in the search text.
在上述粗检索的基础上，可以分别对检索文本和多个候选文本进行解析处理，以提取与检索文本对应的特征信息以及与多个候选文本各自对应的候选特征信息，以便基于检索文本对应的特征信息与各个候选文本对应的候选特征信息，对多个候选文本进行进一步匹配，从而进一步提高检索结果的精度和准确度。On the basis of the above rough retrieval, the search text and multiple candidate texts can be analyzed separately to extract feature information corresponding to the search text and candidate feature information corresponding to the multiple candidate texts, so that based on the search text corresponding The feature information is further matched with the candidate feature information corresponding to each candidate text, thereby further improving the precision and accuracy of the retrieval results.
例如，可以对检索文本进行解析处理，得到与检索文本对应的第一特征信息、第二特征信息和第三特征信息。其中，第一特征信息用于表征各个关键词的词性以及各个关键词对于检索文本的语义重要程度。第二特征信息用于表征检索文本对应的意图信息以及意图信息的置信度。第三特征信息用于表征检索文本中各个关键词的真实实体含义以及实体对于检索文本的语义重要程度。For example, the retrieval text can be parsed to obtain the first feature information, the second feature information, and the third feature information corresponding to the retrieval text. Among them, the first feature information is used to characterize the part of speech of each keyword and the semantic importance of each keyword to the retrieved text. The second feature information is used to characterize the intention information corresponding to the retrieved text and the confidence level of the intention information. The third feature information is used to characterize the true entity meaning of each keyword in the retrieval text and the semantic importance of the entity to the retrieval text.
例如，可以对各个候选文本进行解析处理，得到各个候选文本对应的候选特征信息。该候选特征信息用于表征候选文本中各个候选特征的真实实体含义以及实体对于候选文本的语义重要程度。For example, each candidate text can be parsed to obtain candidate feature information corresponding to each candidate text. The candidate feature information is used to characterize the true entity meaning of each candidate feature in the candidate text and the semantic importance of the entity to the candidate text.
接下来，针对每个候选文本，可以利用第一特征信息、第二特征信息和第三特征信息与候选特征信息进行匹配计算，得到候选文本与检索文本之间的匹配度。然后，根据匹配度对多个候选文本进行排序，并基于排序结果，获得与检索文本对应的检索结果。Next, for each candidate text, the first feature information, the second feature information, and the third feature information can be used to perform matching calculations with the candidate feature information to obtain the matching degree between the candidate text and the retrieved text. Then, multiple candidate texts are sorted according to the matching degree, and based on the sorting results, the retrieval results corresponding to the retrieved text are obtained.
需要说明的是，以上虽然以特定的顺序描述了方法的各个步骤，然而本公开的实施例不限于此，上述步骤可以根据需要以其他顺序执行。例如在一些实施例中，步骤S220可以在步骤S230之后执行，或者与步骤S230同时执行，本公开对此不作限制。It should be noted that although the various steps of the method are described above in a specific order, the embodiments of the present disclosure are not limited thereto, and the above steps can be performed in other orders as needed. For example, in some embodiments, step S220 may be performed after step S230, or simultaneously with step S230, which is not limited by the present disclosure.
在本公开的技术方案中，首先基于检索文本中的多个关键词，实现对检索文本的粗检索，得到多个候选文本。之后，通过利用第一特征信息、第二特征信息和第三特征信息与候选特征信息进行匹配计算来确定与检索文本对应的检索结果。由于在匹配计算过程中考虑了检索文本与候选文本之间的语义关联情况，并基于检索文本确定了检索倾向，因此，能够更加准确的确定检索文本与候选文本之间的匹配度，从而有利于提高文本检索的准确性。In the technical solution of the present disclosure, a rough search of the search text is first implemented based on multiple keywords in the search text, and multiple candidate texts are obtained. Afterwards, the retrieval result corresponding to the retrieval text is determined by performing matching calculations with the candidate feature information using the first feature information, the second feature information, and the third feature information. Since the semantic association between the search text and the candidate text is considered during the matching calculation process, and the search tendency is determined based on the search text, the matching degree between the search text and the candidate text can be determined more accurately, which is beneficial to Improve the accuracy of text retrieval.
根据本公开的实施例，例如可以通过对检索文本进行分词处理，得到检索文本中的多个关键词。在一些实施例中，还可以采用其他合适的方式来得到检索文本中的多个关键词，这里不做限定。According to embodiments of the present disclosure, for example, multiple keywords in the search text can be obtained by performing word segmentation processing on the search text. In some embodiments, other suitable methods can also be used to obtain multiple keywords in the search text, which are not limited here.
另外，在对检索文本进行分词处理时，还可以确定各个关键词的分词权重。分词权重用于指示各个关键词的分词准确率。In addition, when performing word segmentation processing on the search text, the word segmentation weight of each keyword can also be determined. The word segmentation weight is used to indicate the word segmentation accuracy of each keyword.
在一个示例中，可以将分词处理后的分词结果与专有词库进行匹配，以确定各个关键词的分词准确率。例如，如果能够在专有词库中匹配到与关键词对应的字词，则将该字词与关键词之间的相似度确定为分词准确率；如果未匹配到相应的字词，则将该关键词的分词权重设定为预设值。In one example, the word segmentation results after word segmentation processing can be matched with a proprietary thesaurus to determine the word segmentation accuracy of each keyword. For example, if a word corresponding to a keyword can be matched in the proprietary thesaurus, the similarity between the word and the keyword will be determined as the word segmentation accuracy; if the corresponding word is not matched, the similarity will be The word segmentation weight of this keyword is set to the default value.
根据本公开的实施例，可以采用以下方式确定上述第一特征信息、第二特征信息和第三特征信息。According to embodiments of the present disclosure, the above-mentioned first characteristic information, second characteristic information and third characteristic information may be determined in the following manner.
例如，可以对检索文本中的多个关键词进行词性识别，得到多个关键词的词性识别结果和关键词权重，并将多个关键词的词性识别结果和关键词权重确定为第一特征信息。For example, part-of-speech recognition can be performed on multiple keywords in the search text, and the part-of-speech recognition results and keyword weights of the multiple keywords can be obtained, and the part-of-speech recognition results and keyword weights of the multiple keywords can be determined as the first feature information. .
根据本公开的实施例，词性识别结果用于表征各个关键词的词性，例如名词、动词、形容词等等。关键词权重用于指示各个关键词对于检索文本的语义重要程度。According to embodiments of the present disclosure, the part-of-speech recognition results are used to characterize the part-of-speech of each keyword, such as nouns, verbs, adjectives, and so on. Keyword weight is used to indicate the semantic importance of each keyword to the retrieved text.
可以理解，名词、动词、形容词以及其他词性对于文本的语义重要程度通常是不同的。一般场景下，名词、动词等实词通常是关键词，也即对文本的语义重要程度较高。而诸如语气词、疑问词、虚词等往往是非关键词，即对文本的语义重要程度较低。因此，可以根据关键词的词性来确定它的关键词权重。It can be understood that nouns, verbs, adjectives and other parts of speech usually have different semantic importance to the text. In general scenarios, content words such as nouns and verbs are usually keywords, which means they are of high semantic importance to the text. However, modal particles, question words, function words, etc. are often non-keywords, that is, they are of low semantic importance to the text. Therefore, the keyword weight of the keyword can be determined based on its part of speech.
在一个示例中，可以根据词性识别结果中各个关键词对应的词性，查询预设的词权重对应表，以确定各个关键词对应的词权重，并将该词权重作为关键词权重。In one example, a preset word weight correspondence table can be queried according to the part of speech corresponding to each keyword in the part-of-speech recognition result, to determine the word weight corresponding to each keyword, and use the word weight as the keyword weight.
词权重对应表表征了词性与词权重之间的对应关系。示例性地，可以使用词权重标签“3”来表示与名词对应的词权重，使用词权重标签“2”来表示与动词、形容词对应的词权重，使用词权重标签“1”来表示与其他词性对应的词权重。当然，词性与词权重的对应关系还可以采用其他方式表示，具体可以根据实际设定，这里不做限定。The word weight correspondence table represents the correspondence between parts of speech and word weights. For example, the word weight label "3" can be used to represent the word weight corresponding to the noun, the word weight label "2" can be used to represent the word weight corresponding to the verb and adjective, and the word weight label "1" can be used to represent the word weight corresponding to the other words. The word weight corresponding to the part of speech. Of course, the corresponding relationship between part-of-speech and word weight can also be expressed in other ways, and the details can be set according to actual conditions, and are not limited here.
在另一个示例中，由于在对检索文本中的多个关键词进行词性识别之前，通常需要对检索文本进行分词处理，以便得到多个关键词。因此，在确定各个关键词的关键词权重时，还可以综合分词权重和词权重，例如，可以将分词权重和词权重相乘，得到关键词权重。由此提高关键词权重的准确性。In another example, before performing part-of-speech recognition on multiple keywords in the retrieved text, it is usually necessary to perform word segmentation processing on the retrieved text in order to obtain multiple keywords. Therefore, when determining the keyword weight of each keyword, the word segmentation weight and the word weight can also be integrated. For example, the word segmentation weight and the word weight can be multiplied to obtain the keyword weight. This improves the accuracy of keyword weighting.
根据本公开的实施例，可以对检索文本进行意图分类，得到检索文本对应的意图分类结果和意图置信度，并将意图分类结果和意图置信度确定为第二特征信息。According to embodiments of the present disclosure, the intent classification can be performed on the retrieval text, the intent classification result and the intent confidence level corresponding to the retrieval text can be obtained, and the intent classification result and the intent confidence level can be determined as the second feature information.
意图分类结果用于表征检索文本对应的意图信息，该意图信息用于指示用户的检索倾向，例如检索人名、物品名、地名、时间等要素，或者与这些要素相关的子要素，例如人物的高矮胖瘦、衣着等等。意图置信度用于指示对检索文本进行意图分类时，意图信息对应的置信度。The intent classification results are used to characterize the intent information corresponding to the retrieval text. This intent information is used to indicate the user's retrieval tendency, such as retrieving elements such as person names, item names, place names, time, or sub-elements related to these elements, such as the height of a person. Fat or thin, clothing, etc. Intent confidence is used to indicate the confidence corresponding to the intent information when classifying the intent of the retrieved text.
在一个示例中，可以利用预训练的意图分类模型来对检索文本进行意图分类。预训练的意图分类模型例如可以包括但不限于FastText分类器，或者其他的预训练模型，具体可以根据实际应用场景选择。In one example, a pretrained intent classification model can be utilized to classify retrieved text by intent. Pretrained intent classification models may include, for example, but are not limited to FastText classifier, or other pretrained models, which may be selected based on actual application scenarios.
根据本公开的实施例，可以对检索文本进行实体识别，得到第一实体识别结果以及与第一实体识别结果相关联的第一重要度识别结果，并将第一实体识别结果和第一重要度识别结果确定为第三特征信息。According to embodiments of the present disclosure, entity recognition can be performed on the retrieved text, the first entity recognition result and the first importance recognition result associated with the first entity recognition result are obtained, and the first entity recognition result and the first importance recognition result are obtained The recognition result is determined as the third characteristic information.
在本公开的实施例中，可以使用预训练的命名实体识别模型基于检索文本，得到第一实体识别结果。第一实体识别结果包括多个第一实体。多个第一实体分别对应检索文本中多个关键词的命名实体识别结果。其中，命名实体识别模型例如可以包括但不限于Bi-LSTM+CRF模型，具体可以根据实际需要选择。In embodiments of the present disclosure, a pre-trained named entity recognition model may be used to obtain the first entity recognition result based on the retrieved text. The first entity recognition result includes a plurality of first entities. The multiple first entities respectively correspond to the named entity recognition results of multiple keywords in the retrieved text. Among them, the named entity recognition model may include, for example, but is not limited to Bi-LSTM+CRF model, which may be selected according to actual needs.
第一重要度识别结果包括与多个第一实体分别对应的重要度。其中，每个第一实体的重要度表征了该第一实体对于检索文本的语义重要程度。The first importance recognition result includes importance respectively corresponding to multiple first entities. Among them, the importance of each first entity represents the semantic importance of the first entity to the retrieved text.
在一个示例中，可以根据各个第一实体，查询预设的实体重要度表来确定各个第一实体的重要度。In one example, a preset entity importance table can be queried according to each first entity to determine the importance of each first entity.
实体重要度表表征了实体与重要度之间的对应关系。示例性地，可以使用重要度标签1～9来标注与各个实体对应的重要度。其中，重要度标签对应的数值越大，表明使用该重要度标签标注的实体的重要度越大，反之亦然。如果两个实体的重要度相同，可以采用相同的重要度标签来标注这两个实体的重要度。需要说明的是，在一些实施例中，还可以采用其他方式表示实体与重要度之间的对应关系，具体可以根据实际设定，这里不做限定。The entity importance table represents the correspondence between entities and importance. For example, importance labels 1 to 9 can be used to mark the importance corresponding to each entity. Among them, the larger the value corresponding to the importance label, the greater the importance of the entity marked with the importance label, and vice versa. If the importance of two entities is the same, the same importance label can be used to mark the importance of the two entities. It should be noted that in some embodiments, other methods can be used to express the corresponding relationship between entities and importance, and the specific settings can be based on actual settings, which are not limited here.
根据本公开的实施例，可以采用以下方式确定上述候选特征信息。According to embodiments of the present disclosure, the above candidate feature information may be determined in the following manner.
例如，针对每个候选文本，对候选文本进行实体识别，得到第二实体识别结果以及与第二实体识别结果相关联的第二重要度识别结果，并将第二实体识别结果和第二重要度识别结果确定为候选特征信息。For example, for each candidate text, perform entity recognition on the candidate text, obtain the second entity recognition result and the second importance recognition result associated with the second entity recognition result, and combine the second entity recognition result and the second importance recognition result The recognition results are determined as candidate feature information.
第二实体识别结果包括多个第二实体。多个第二实体分别对应候选文本中多个关键词的命名实体识别结果。The second entity recognition result includes a plurality of second entities. The multiple second entities respectively correspond to the named entity recognition results of multiple keywords in the candidate text.
第二重要度识别结果包括与多个第二实体各自对应的重要度。其中，每个第二实体的重要度表征了该第二实体对于候选文本的语义重要程度。The second importance recognition result includes respective importance degrees corresponding to the plurality of second entities. The importance of each second entity represents the semantic importance of the second entity to the candidate text.
在本公开实施例中，第二实体识别结果和第二重要度识别结果分别与上述第一实体识别结果和第一重要度识别结果的获取方式相同或类似，这里不再赘述。In the embodiment of the present disclosure, the second entity recognition result and the second importance recognition result are obtained in the same or similar manner as the above-mentioned first entity recognition result and the first importance recognition result, respectively, and will not be described again here.
在获取上述第一特征信息、第二特征信息、第三特征信息和候选特征信息之后，可以根据这些特征信息来确定候选文本与检索文本之间的匹配度。After obtaining the above-mentioned first feature information, second feature information, third feature information and candidate feature information, the matching degree between the candidate text and the retrieved text can be determined based on these feature information.
图3A和图3B是根据本公开实施例的文本检索方法的示意图。下面参考图3A和图3B对确定候选文本与检索文本之间的匹配度的过程进行示例说明。3A and 3B are schematic diagrams of text retrieval methods according to embodiments of the present disclosure. The process of determining the matching degree between the candidate text and the retrieved text is illustrated below with reference to FIGS. 3A and 3B .
如图3A所示，可以对检索文本31进行分词处理，得到检索文本31中的多个关键词。之后，根据多个关键词，检索得到与检索文本31相关联的多个候选文本。As shown in FIG. 3A , word segmentation processing can be performed on the search text 31 to obtain multiple keywords in the search text 31 . Afterwards, multiple candidate texts associated with the search text 31 are retrieved based on multiple keywords.
接下来，可以对检索文本31进行解析处理，得到与检索文本31对应的多个关键词的词性识别结果和关键词权重311(即第一特征信息)、意图分类结果和意图置信度312(即第二特征信息)和第一实体识别结果和第一重要度识别结果313(即第三特征信息)。Next, the retrieval text 31 can be parsed to obtain part-of-speech recognition results and keyword weights 311 (i.e., first feature information), intent classification results, and intent confidence 312 (i.e., the plurality of keywords corresponding to the retrieval text 31 second characteristic information) and the first entity identification result and the first importance identification result 313 (ie, the third characteristic information).
另外，可以对每个候选文本32进行解析处理，得到与候选文本32对应的第二实体识别结果和第二重要度识别结果321，并将第二实体识别结果和第二重要度识别结果321确定为候选特征信息。In addition, each candidate text 32 can be parsed to obtain the second entity recognition result and the second importance recognition result 321 corresponding to the candidate text 32, and the second entity recognition result and the second importance recognition result 321 are determined. is candidate feature information.
接下来，可以根据多个关键词的词性识别结果和关键词权重311、意图分类结果和意图置信度312和第一实体识别结果、第一重要度识别结果313以及第二实体识别结果和第二重要度识别结果321，得到第一匹配度，并将第一匹配度确定为候选文本32与检索文本之间的匹配度3132。Next, the part-of-speech recognition results and keyword weights 311 of multiple keywords, the intent classification results and the intent confidence 312, the first entity recognition results, the first importance recognition results 313, the second entity recognition results and the second The importance recognition result 321 obtains the first matching degree, and determines the first matching degree as the matching degree 3132 between the candidate text 32 and the retrieved text.
接下来，根据候选文本32与检索文本31之间的匹配度3132，对多个候选文本进行排序，并基于排序结果，选择预设数量个候选文本作为与检索文本31对应的检索结果。Next, the multiple candidate texts are sorted according to the matching degree 3132 between the candidate text 32 and the search text 31, and based on the sorting results, a preset number of candidate texts are selected as search results corresponding to the search text 31.
在一些实施例中，还可以根据候选文本32的更新时间以及与候选文本32关联的文本的数量，确定属性特征信息322。然后，将属性特征信息322、第二实体识别结果和第二重要度识别结果321确定为候选特征信息。其中，与候选文本32关联的文本的数量是指检索数据库中与候选文本32具有关联关系的文本的数量。这里的关联关系例如包括与候选文本32具有相似的文本内容、与候选文本32具有引用关系等等。In some embodiments, the attribute feature information 322 may also be determined based on the update time of the candidate text 32 and the number of texts associated with the candidate text 32 . Then, the attribute feature information 322, the second entity recognition result and the second importance recognition result 321 are determined as candidate feature information. The number of texts associated with the candidate text 32 refers to the number of texts associated with the candidate text 32 in the retrieval database. The association relationship here includes, for example, having similar text content with the candidate text 32, having a reference relationship with the candidate text 32, and so on.
相应地，在确定候选文本32与检索文本31之间的匹配度3132时，还可以根据属性特征信息322，确定属性匹配度3221。之后，将属性匹配度3221与上述第一匹配度进行叠加，得到第二匹配度。然后，将第二匹配度确定为候选文本32与检索文本31之间的匹配度3132。Correspondingly, when determining the matching degree 3132 between the candidate text 32 and the search text 31 , the attribute matching degree 3221 may also be determined based on the attribute feature information 322 . Afterwards, the attribute matching degree 3221 is superposed with the above-mentioned first matching degree to obtain the second matching degree. Then, the second matching degree is determined as the matching degree 3132 between the candidate text 32 and the retrieved text 31 .
在本公开实施例中，可以采用如下方式确定属性匹配度3221。In this embodiment of the present disclosure, the attribute matching degree 3221 may be determined in the following manner.
例如，可以采用时间衰减函数基于候选文本32的更新时间以及检索时间，确定时效性匹配度。For example, a time decay function may be used to determine the timeliness matching degree based on the update time and retrieval time of the candidate text 32 .
在一个示例中，时效性匹配度满足如下关系。In one example, the timeliness matching degree satisfies the following relationship.
在公式(1)中，p表示时效性匹配度，表示时间衰减系数，tnow表示检索时间，tupdate表示候选文本的更新时间。In formula (1), p represents the timeliness matching degree, represents the time decay coefficient, tnow represents the retrieval time, and tupdate represents the update time of the candidate text.
接下来，可以根据与候选文本32关联的文本的数量，确定文本相关性匹配度。例如，可以根据与候选文本32关联的文本的数量除以检索数据库中所有文本的数量，得到上述文本相关性匹配度。Next, a text relevance match may be determined based on the number of texts associated with the candidate text 32 . For example, the above text correlation matching degree can be obtained based on the number of texts associated with the candidate text 32 divided by the number of all texts in the retrieval database.
接下来，根据时效性匹配度和文本相关性匹配度，确定属性匹配度3221。Next, determine the attribute matching degree 3221 based on the timeliness matching degree and the text relevance matching degree.
图3B示意性示出了确定第一匹配度的过程。如图3B所示，第一实体识别结果包括多个第一实体，例如第一实体1、第一实体2，...，第一实体n，n为正整数。第一重要度识别结果包括与多个第一实体分别对应的重要度，例如，与第一实体1至第一实体n各自对应的重要度。第一实体i(i＝1，2，...，n)的重要度表征了该第一实体i对于检索文本31的语义重要程度。Figure 3B schematically shows the process of determining the first matching degree. As shown in Figure 3B, the first entity recognition result includes multiple first entities, such as first entity 1, first entity 2, ..., first entity n, n is a positive integer. The first importance recognition result includes the importance respectively corresponding to the plurality of first entities, for example, the importance respectively corresponding to the first entity 1 to the first entity n. The importance of the first entity i (i=1, 2, ..., n) represents the semantic importance of the first entity i to the retrieval text 31.
第二实体识别结果包括多个第二实体，例如第二实体1、第二实体2，...，第二实体n。第二重要度识别结果包括与多个第二实体分别对应的重要度，例如，与第二实体1至第二实体n各自对应的重要度。第二实体i(i＝1，2，...，n)的重要度表征了该第二实体i对于候选文本32的语义重要程度。The second entity recognition result includes multiple second entities, such as second entity 1, second entity 2, ..., second entity n. The second importance recognition result includes the importance respectively corresponding to the plurality of second entities, for example, the importance respectively corresponding to the second entity 1 to the second entity n. The importance of the second entity i (i=1, 2, ..., n) represents the semantic importance of the second entity i to the candidate text 32.
针对每个第二实体i(例如第二实体1)，确定多个关键词中与第二实体i相匹配的目标关键词。之后，根据目标关键词，查询多个关键词的词性识别结果和关键词权重311，得到与目标关键词对应的目标关键词权重3111。目标关键词权重3111用于指示目标关键词对于检索文本31的语义重要程度。For each second entity i (for example, second entity 1), the target keyword matching the second entity i among the plurality of keywords is determined. After that, according to the target keyword, the part-of-speech recognition results and keyword weights 311 of multiple keywords are queried, and the target keyword weight 3111 corresponding to the target keyword is obtained. The target keyword weight 3111 is used to indicate the semantic importance of the target keyword to the retrieved text 31 .
另外，针对每个第二实体i(例如第二实体1)，将第二实体i与意图分类结果进行比对，以确定意图分类结果中与第二实体i相匹配的意图信息。例如，如果意图信息表征检索目的是人名，且第二实体i对应的命名实体也是人名，则说明该意图信息与第二实体i是匹配的；否则说明两者不匹配。之后，根据该意图信息，确定意图信息所对应的目标意图置信度3121。In addition, for each second entity i (for example, second entity 1), the second entity i is compared with the intent classification result to determine the intent information matching the second entity i in the intent classification result. For example, if the intention information indicates that the retrieval purpose is a person's name, and the named entity corresponding to the second entity i is also a person's name, it means that the intention information matches the second entity i; otherwise, it means that the two do not match. Afterwards, based on the intention information, the target intention confidence degree corresponding to the intention information is determined 3121.
接下来，根据第二实体i、第一实体识别结果中与第二实体i对应的第一实体i和目标关键词权重3111，确定第二实体i与对应的第一实体i之间的初始匹配度。Next, an initial match between the second entity i and the corresponding first entity i is determined based on the second entity i, the first entity i corresponding to the second entity i in the first entity recognition result, and the target keyword weight 3111. Spend.
接下来，根据第二实体i与对应的第一实体i之间的初始匹配度、目标意图置信度3121、第二实体i对应的重要度以及对应的第一实体i的重要度，确定第二实体i与对应的第一实体i之间的匹配度。Next, determine the second entity i according to the initial matching degree between the second entity i and the corresponding first entity i, the target intention confidence 3121, the corresponding importance of the second entity i, and the corresponding importance of the first entity i. The matching degree between entity i and the corresponding first entity i.
接下来，根据各个第二实体与对应的第一实体之间的匹配度，得到第一匹配度，并将第一匹配度确定为候选文本32与检索文本31之间的匹配度31_32。Next, according to the matching degree between each second entity and the corresponding first entity, the first matching degree is obtained, and the first matching degree is determined as the matching degree 31_32 between the candidate text 32 and the search text 31 .
在本公开的实施例中，一方面通过将检索文本和候选文本对应的多个实体信息进行匹配，实现了检索文本和候选文本中多维度的关键信息的比对，从而有利于提高检索结果的准确性。另一方面，在匹配过程中，利用关键词权重、意图置信度、第一实体的重要度和第二实体的重要度来调整检索文本和候选文本之间的匹配度，也即考虑了检索文本和候选文本之间的语义关联情况、检索目的等因素，从而提高了检索结果的准确性。In the embodiments of the present disclosure, on the one hand, by matching multiple entity information corresponding to the retrieval text and the candidate text, the comparison of multi-dimensional key information in the retrieval text and the candidate text is achieved, which is beneficial to improving the retrieval results. accuracy. On the other hand, during the matching process, the keyword weight, intent confidence, importance of the first entity and importance of the second entity are used to adjust the matching degree between the retrieved text and the candidate text, that is, the retrieved text is taken into account factors such as the semantic association between the text and the candidate text, the purpose of retrieval, etc., thus improving the accuracy of the retrieval results.
图4是根据本公开实施例的文本检索装置的框图。4 is a block diagram of a text retrieval device according to an embodiment of the present disclosure.
如图4所示，文本检索装置400包括：获取模块410、第一解析模块420、第二解析模块430、匹配模块440和排序模块450。As shown in FIG. 4 , the text retrieval device 400 includes: an acquisition module 410 , a first parsing module 420 , a second parsing module 430 , a matching module 440 and a sorting module 450 .
获取模块410用于根据检索文本中的多个关键词，获取与检索文本相关联的多个候选文本。The acquisition module 410 is configured to acquire multiple candidate texts associated with the search text based on multiple keywords in the search text.
第一解析模块420用于对检索文本进行解析处理，得到与检索文本对应的第一特征信息、第二特征信息和第三特征信息。The first parsing module 420 is used to parse the retrieval text to obtain first feature information, second feature information and third feature information corresponding to the retrieval text.
第二解析模块430用于分别对多个候选文本进行解析处理，得到与多个候选文本各自对应的候选特征信息。The second parsing module 430 is configured to parse multiple candidate texts respectively to obtain candidate feature information corresponding to the multiple candidate texts.
匹配模块440用于针对每个候选文本，根据第一特征信息、第二特征信息、第三特征信息和候选特征信息，确定候选文本与检索文本之间的匹配度。The matching module 440 is configured to, for each candidate text, determine the matching degree between the candidate text and the retrieved text based on the first feature information, the second feature information, the third feature information and the candidate feature information.
排序模块450用于根据匹配度对多个候选文本进行排序，并基于排序结果，获得与检索文本对应的检索结果。The sorting module 450 is used to sort multiple candidate texts according to matching degrees, and based on the sorting results, obtain retrieval results corresponding to the retrieved texts.
根据本公开的实施例，第一解析模块420包括：词性识别单元、意图分类单元和第一实体识别单元。词性识别单元用于对检索文本中的多个关键词进行词性识别，得到多个关键词的词性识别结果和关键词权重，并将多个关键词的词性识别结果和关键词权重确定为第一特征信息；意图分类单元用于对检索文本进行意图分类，得到检索文本对应的意图分类结果和意图置信度，并将意图分类结果和意图置信度确定为第二特征信息；以及第一实体识别单元用于对检索文本进行实体识别，得到第一实体识别结果以及与第一实体识别结果相关联的第一重要度识别结果，并将第一实体识别结果和第一重要度识别结果确定为第三特征信息；其中，第一重要度识别结果用于表征第一实体识别结果中每个第一实体的重要度。According to an embodiment of the present disclosure, the first parsing module 420 includes: a part-of-speech recognition unit, an intent classification unit, and a first entity recognition unit. The part-of-speech recognition unit is used to perform part-of-speech recognition on multiple keywords in the search text, obtain the part-of-speech recognition results and keyword weights of multiple keywords, and determine the part-of-speech recognition results and keyword weights of multiple keywords as the first Feature information; the intention classification unit is used to classify the intention of the retrieval text, obtain the intention classification result and the intention confidence level corresponding to the retrieval text, and determine the intention classification result and the intention confidence level as the second feature information; and the first entity recognition unit Used to perform entity recognition on the retrieved text, obtain the first entity recognition result and the first importance recognition result associated with the first entity recognition result, and determine the first entity recognition result and the first importance recognition result as the third Feature information; wherein, the first importance recognition result is used to characterize the importance of each first entity in the first entity recognition result.
根据本公开的实施例，第二解析模块430包括：第二实体识别单元和第一确定单元。第二实体识别单元用于针对每个候选文本，对候选文本进行实体识别，得到第二实体识别结果以及与第二实体识别结果相关联的第二重要度识别结果；其中，第二重要度识别结果用于表征第二实体识别结果中每个第二实体的重要度；以及第一确定单元用于将第二实体识别结果和第二重要度识别结果确定为候选特征信息。According to an embodiment of the present disclosure, the second parsing module 430 includes: a second entity identification unit and a first determination unit. The second entity recognition unit is used to perform entity recognition on each candidate text, and obtain a second entity recognition result and a second importance recognition result associated with the second entity recognition result; wherein, the second importance recognition result The result is used to characterize the importance of each second entity in the second entity recognition result; and the first determination unit is used to determine the second entity recognition result and the second importance recognition result as candidate feature information.
根据本公开的实施例，匹配模块440包括：第二确定单元、第三确定单元、第四确定单元和第五确定单元。第二确定单元用于针对每个第二实体，确定多个关键词中与第二实体相匹配的目标关键词所对应的目标关键词权重，以及意图分类结果中与第二实体相匹配的意图信息所对应的目标意图置信度；第三确定单元用于根据第二实体、第一实体识别结果中与第二实体对应的第一实体和目标关键词权重，确定第二实体与对应的第一实体之间的初始匹配度；第四确定单元用于根据第二实体与对应的第一实体之间的初始匹配度、目标意图置信度、第二实体对应的重要度以及对应的第一实体的重要度，确定第二实体与对应的第一实体之间的匹配度；以及第五确定单元用于根据各个第二实体与对应的第一实体之间的匹配度，确定候选文本与检索文本之间的匹配度。According to an embodiment of the present disclosure, the matching module 440 includes: a second determination unit, a third determination unit, a fourth determination unit and a fifth determination unit. The second determination unit is configured to determine, for each second entity, the target keyword weight corresponding to the target keyword matching the second entity among the plurality of keywords, and the intent matching the second entity in the intent classification result. The confidence of the target intention corresponding to the information; the third determination unit is used to determine the second entity and the corresponding first entity according to the weight of the first entity and the target keyword corresponding to the second entity in the recognition result of the second entity and the first entity. Initial matching degree between entities; the fourth determination unit is used to determine the initial matching degree between the second entity and the corresponding first entity, the target intention confidence degree, the corresponding importance degree of the second entity and the corresponding first entity degree. The degree of importance is used to determine the matching degree between the second entity and the corresponding first entity; and the fifth determination unit is used to determine the relationship between the candidate text and the retrieved text based on the matching degree between each second entity and the corresponding first entity. degree of matching between.
根据本公开的实施例，第二解析模块430还包括：第六确定单元、第七确定单元。第六确定单元用于针对每个候选文本，根据候选文本的更新时间以及与候选文本关联的文本的数量，确定属性特征信息；以及第七确定单元用于将属性特征信息、第二实体识别结果和第二重要度识别结果确定为候选特征信息。According to an embodiment of the present disclosure, the second parsing module 430 further includes: a sixth determination unit and a seventh determination unit. The sixth determination unit is used to determine the attribute feature information for each candidate text according to the update time of the candidate text and the number of texts associated with the candidate text; and the seventh determination unit is used to combine the attribute feature information and the second entity recognition result. and the second importance recognition result is determined as candidate feature information.
根据本公开的实施例，匹配模块440还包括：第八确定单元、第九确定单元。第八确定单元用于根据属性特征信息，确定属性匹配度；以及第九确定单元用于根据属性匹配度以及各个第二实体与对应的第一实体之间的匹配度，确定候选文本与检索文本之间的匹配度。According to an embodiment of the present disclosure, the matching module 440 further includes: an eighth determination unit and a ninth determination unit. The eighth determination unit is used to determine the attribute matching degree according to the attribute feature information; and the ninth determination unit is used to determine the candidate text and the search text according to the attribute matching degree and the matching degree between each second entity and the corresponding first entity. degree of matching between them.
根据本公开的实施例，文本检索装置400还包括：处理模块，处理模块用于对检索文本进行分词处理，得到检索文本中的多个关键词。According to an embodiment of the present disclosure, the text retrieval device 400 further includes: a processing module, which is configured to perform word segmentation processing on the retrieval text to obtain multiple keywords in the retrieval text.
需要说明的是，装置部分实施例中各模块/单元/子单元等的实施方式、解决的技术问题、实现的功能、以及达到的技术效果分别与方法部分实施例中各对应的步骤的实施方式、解决的技术问题、实现的功能、以及达到的技术效果相同或类似，在此不再赘述。It should be noted that the implementation of each module/unit/subunit, etc., the technical problems solved, the functions implemented, and the technical effects achieved in the device embodiment are respectively the same as the implementation of the corresponding steps in the method embodiment. , the technical problems solved, the functions implemented, and the technical effects achieved are the same or similar, and will not be described again here.
本公开的技术方案中，所涉及的数据(例如包括但不限于用户个人信息)的收集、存储、使用、加工、传输、提供、公开和应用等处理，均符合相关法律法规的规定，且不违背公序良俗。In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (for example, including but not limited to user personal information) are in compliance with relevant laws and regulations and are not Violate public order and good customs.
在本公开的技术方案中，在获取或采集相关数据之前，均获取了数据归属者的授权或同意。In the technical solution of the present disclosure, the authorization or consent of the data owner is obtained before obtaining or collecting relevant data.
根据本公开的实施例，本公开还提供了一种电子设备、一种可读存储介质和一种计算机程序产品。According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
根据本公开的实施例，一种电子设备，包括：至少一个处理器；以及与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行如本公开实施例的方法。According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are processed by at least one processor. The processor executes, so that at least one processor can execute the method according to the embodiment of the present disclosure.
根据本公开的实施例，一种存储有计算机指令的非瞬时计算机可读存储介质，其中，计算机指令用于使计算机执行如本公开实施例的方法。According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is used, wherein the computer instructions are used to cause a computer to perform a method according to an embodiment of the present disclosure.
根据本公开的实施例，一种计算机程序产品，包括计算机程序，计算机程序在被处理器执行时实现如本公开实施例的方法。According to an embodiment of the present disclosure, a computer program product includes a computer program, and when executed by a processor, the computer program implements a method according to an embodiment of the present disclosure.
图5是用来实现本公开实施例的文本检索方法的电子设备的框图。FIG. 5 is a block diagram of an electronic device used to implement a text retrieval method according to an embodiment of the present disclosure.
图5示出了可以用来实施本公开的实施例的示例电子设备500的示意性框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。Figure 5 shows a schematic block diagram of an example electronic device 500 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit implementations of the disclosure described and/or claimed herein.
如图5所示，设备500包括计算单元501，其可以根据存储在只读存储器(ROM)502中的计算机程序或者从存储单元508加载到随机访问存储器(RAM)503中的计算机程序，来执行各种适当的动作和处理。在RAM 503中，还可存储设备500操作所需的各种程序和数据。计算单元501、ROM 502以及RAM 503通过总线504彼此相连。输入/输出(I/O)接口505也连接至总线504。As shown in FIG. 5 , the device 500 includes a computing unit 501 that can execute according to a computer program stored in a read-only memory (ROM) 502 or loaded from a storage unit 508 into a random access memory (RAM) 503 Various appropriate actions and treatments. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. Computing unit 501, ROM 502 and RAM 503 are connected to each other via bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
设备500中的多个部件连接至I/O接口505，包括：输入单元506，例如键盘、鼠标等；输出单元507，例如各种类型的显示器、扬声器等；存储单元508，例如磁盘、光盘等；以及通信单元509，例如网卡、调制解调器、无线通信收发机等。通信单元509允许设备500通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。Multiple components in the device 500 are connected to the I/O interface 505, including: input unit 506, such as a keyboard, mouse, etc.; output unit 507, such as various types of displays, speakers, etc.; storage unit 508, such as a magnetic disk, optical disk, etc. ; and communication unit 509, such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices through computer networks such as the Internet and/or various telecommunications networks.
计算单元501可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元501的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元501执行上文所描述的各个方法和处理，例如文本检索方法。例如，在一些实施例中，文本检索方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元508。在一些实施例中，计算机程序的部分或者全部可以经由ROM 502和/或通信单元509而被载入和/或安装到设备500上。当计算机程序加载到RAM 503并由计算单元501执行时，可以执行上文描述的文本检索方法的一个或多个步骤。备选地，在其他实施例中，计算单元501可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行文本检索方法。Computing unit 501 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs various methods and processes described above, such as text retrieval methods. For example, in some embodiments, the text retrieval method may be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 500 via ROM 502 and/or communication unit 509 . When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the text retrieval method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the text retrieval method in any other suitable manner (eg, by means of firmware).
本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip implemented in a system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or a combination thereof. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions specified in the flowcharts and/or block diagrams/ The operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，也可以为分布式系统的服务器，或者是结合了区块链的服务器。Computer systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other. The server can be a cloud server, a distributed system server, or a server combined with a blockchain.
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in the present disclosure can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, there is no limitation here.
上述具体实施方式，并不构成对本公开保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本公开的精神和原则之内所作的修改、等同替换和改进等，均应包含在本公开保护范围之内。The above-mentioned specific embodiments do not constitute a limitation on the scope of the present disclosure. It will be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure shall be included in the protection scope of this disclosure.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310479153.8ACN116610782B (en) | 2023-04-28 | 2023-04-28 | Text retrieval method, device, electronic equipment and medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310479153.8ACN116610782B (en) | 2023-04-28 | 2023-04-28 | Text retrieval method, device, electronic equipment and medium |
| Publication Number | Publication Date |
|---|---|
| CN116610782A CN116610782A (en) | 2023-08-18 |
| CN116610782Btrue CN116610782B (en) | 2024-03-15 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310479153.8AActiveCN116610782B (en) | 2023-04-28 | 2023-04-28 | Text retrieval method, device, electronic equipment and medium |
| Country | Link |
|---|---|
| CN (1) | CN116610782B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117149956B (en)* | 2023-09-07 | 2025-09-16 | 中电金信软件(上海)有限公司 | Text retrieval method and device, electronic equipment and readable storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016179938A1 (en)* | 2015-05-14 | 2016-11-17 | 百度在线网络技术（北京）有限公司 | Method and device for question recommendation |
| CN111444320A (en)* | 2020-06-16 | 2020-07-24 | 太平金融科技服务（上海）有限公司 | Text retrieval method and device, computer equipment and storage medium |
| WO2021051521A1 (en)* | 2019-09-18 | 2021-03-25 | 平安科技（深圳）有限公司 | Response information obtaining method and apparatus, computer device, and storage medium |
| CN113326420A (en)* | 2021-06-15 | 2021-08-31 | 北京百度网讯科技有限公司 | Question retrieval method, device, electronic equipment and medium |
| CN115470313A (en)* | 2022-08-12 | 2022-12-13 | 北京百度网讯科技有限公司 | Information retrieval and model training method, device, equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598445B (en)* | 2013-11-01 | 2019-05-10 | 腾讯科技（深圳）有限公司 | Automatically request-answering system and method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016179938A1 (en)* | 2015-05-14 | 2016-11-17 | 百度在线网络技术（北京）有限公司 | Method and device for question recommendation |
| WO2021051521A1 (en)* | 2019-09-18 | 2021-03-25 | 平安科技（深圳）有限公司 | Response information obtaining method and apparatus, computer device, and storage medium |
| CN111444320A (en)* | 2020-06-16 | 2020-07-24 | 太平金融科技服务（上海）有限公司 | Text retrieval method and device, computer equipment and storage medium |
| CN113326420A (en)* | 2021-06-15 | 2021-08-31 | 北京百度网讯科技有限公司 | Question retrieval method, device, electronic equipment and medium |
| CN115470313A (en)* | 2022-08-12 | 2022-12-13 | 北京百度网讯科技有限公司 | Information retrieval and model training method, device, equipment and storage medium |
| Publication number | Publication date |
|---|---|
| CN116610782A (en) | 2023-08-18 |
| Publication | Publication Date | Title |
|---|---|---|
| EP3958145A1 (en) | Method and apparatus for semantic retrieval, device and storage medium | |
| US10586155B2 (en) | Clarification of submitted questions in a question and answer system | |
| CN111797214A (en) | Question screening method, device, computer equipment and medium based on FAQ database | |
| CN113326420B (en) | Problem retrieval method, device, electronic device and medium | |
| WO2021189951A1 (en) | Text search method and apparatus, and computer device and storage medium | |
| WO2019091026A1 (en) | Knowledge base document rapid search method, application server, and computer readable storage medium | |
| CN113988157B (en) | Semantic retrieval network training method, device, electronic equipment and storage medium | |
| CN112988969A (en) | Method, device, equipment and storage medium for text retrieval | |
| CN114116997A (en) | Knowledge question answering method, knowledge question answering device, electronic equipment and storage medium | |
| CN112926297B (en) | Method, apparatus, device and storage medium for processing information | |
| CN113609847B (en) | Information extraction method, device, electronic equipment and storage medium | |
| CN117112595A (en) | Information query method and device, electronic equipment and storage medium | |
| CN113836316B (en) | Processing method, training method, device, equipment and medium for ternary group data | |
| CN112989190A (en) | Commodity mounting method and device, electronic equipment and storage medium | |
| CN116150497A (en) | Text information recommendation method, device, electronic device and storage medium | |
| CN117216275A (en) | Text processing method, device, equipment and storage medium | |
| CN118797005A (en) | Intelligent question-answering method, device, electronic device, storage medium and product | |
| CN113378015A (en) | Search method, search apparatus, electronic device, storage medium, and program product | |
| CN116610782B (en) | Text retrieval method, device, electronic equipment and medium | |
| EP4535190A1 (en) | Method and apparatus of retrieving data, and method and apparatus of training deep learning model | |
| CN113407813B (en) | Method for determining candidate information, method for determining query result, device and equipment | |
| CN111737607B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN113919424A (en) | Training of text processing model, text processing method, device, equipment and medium | |
| CN112528644A (en) | Entity mounting method, device, equipment and storage medium | |
| CN114201607B (en) | Information processing method and device |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |