Disclosure of Invention
The present invention has been made in view of the above-described problems.
Therefore, the technical problems solved by the invention are as follows: the current knowledge graph construction method has the problems of complex construction, time consumption and labor consumption.
In order to solve the technical problems, the invention provides the following technical scheme: a knowledge graph construction method comprises the following steps: acquiring knowledge data from the multi-source data, preprocessing, performing semantic enhancement labeling, performing multi-mode entity identification, and extracting the relationship between entities by using a deep learning model; and constructing a knowledge graph, carrying out dynamic knowledge fusion, and updating and optimizing the knowledge graph regularly.
As a preferable scheme of the knowledge graph construction method, the invention comprises the following steps: the knowledge data includes acquiring relevant knowledge data, including text, image, audio and video forms, from the multi-source data using a data capture tool; for non-text data, text and knowledge in an image are extracted by using an image recognition technology, audio content is transcribed by using a voice recognition technology, the audio content is converted into an analyzable text format, and original data and the converted text data are corresponding and stored.
As a preferable scheme of the knowledge graph construction method, the invention comprises the following steps: the semantic enhancement labeling comprises the steps that a BERT model provides a confidence score for each labeling output, a threshold A is set according to the confidence score output by the BERT model, error-prone data types are set according to error historical data of the BERT model, and the BERT model is utilized to analyze preprocessed data; when the confidence scores of all labels in the data are larger than or equal to A and the data type does not belong to the error-prone data type, no manual review is performed; when the confidence coefficient score of the label in the data is smaller than A, the label with the confidence coefficient score smaller than A accounts for less than 50% of all labels, and the data type does not belong to the error prone data type, performing manual review on the label with the confidence coefficient score smaller than A; when the confidence coefficient score of the label in the data is smaller than A, and the label with the confidence coefficient score smaller than A accounts for more than 50% of all labels, manually reviewing all the data, and feeding back the result of the manual review to the BERT model for retraining and optimizing; and when the data type belongs to the error-prone data type, manually reviewing all the data, and feeding back the manually reviewed result to the BERT model for retraining and optimization.
As a preferable scheme of the knowledge graph construction method, the invention comprises the following steps: the multi-modal entity identification comprises the steps of integrating text data and non-text data by aligning entity boundaries in the text data with entity positions in the non-text data, extracting features of the integrated data, and extracting text features of the text data by using a pre-trained BERT model; for image and video data, a convolutional neural network is used for extracting image features, for audio data, a Mel frequency spectrum feature extraction method is used for extracting audio features, non-text features are integrated, and entities in text data are identified and marked according to the integrated non-text features.
As a preferable scheme of the knowledge graph construction method, the invention comprises the following steps: the deep learning model includes a model of the model,
Wherein R represents the relationship type between the entities; e represents an entity feature; c represents context information between entities; gi represents the gating value for relationship type i; wr represents a weight matrix of relationship type prediction; br represents a bias term for relationship type prediction; wu represents a context-encoded weight matrix; n represents the number of relationship types; m represents the dimension of the context feature; sigma represents a sigmoid activation function; alpha represents the adjustment parameter.
As a preferable scheme of the knowledge graph construction method, the invention comprises the following steps: setting a relation threshold Rh>Rl between entities according to historical data, setting a gating threshold Gh>Gl, adding the relation between the entities into the knowledge graph when R is more than or equal to Rh and G is more than or equal to Gh, marking the relation between the entities as a high-confidence relation, and providing priority for subsequent data analysis and inquiry; when Rl≤R<Rh and G is more than or equal to Gl among the entities, adding the relation into a queue to be verified, checking the relation in the queue to be verified according to the follow-up data, adding the relation into a knowledge graph if the follow-up data is checked to pass, and triggering manual review if the follow-up data is checked to fail; when R is more than or equal to Rl and Gl≤G<Gh among the entities, carrying out context analysis on the relationship, checking whether ambiguity and potential ambiguity exist, calling an additional data source to carry out cross-validation, adding the relationship into a map if the context analysis and the cross-validation are both passed, and transferring the relationship into a queue to be validated if the context analysis or the cross-validation is not passed; when R < Rl or G < Gl between entities, adding the relationship into an isolation queue, comparing the relationship with historical data through pattern analysis, checking whether misjudgment exists, if misjudgment exists, re-carrying out multi-mode entity identification and relationship extraction, and if misjudgment does not exist, marking the relationship as low confidence and carrying out manual review; assigning a unique identifier to each entity and the relation between the entities, identifying and merging repeated entity nodes, and when similar entities exist, selecting the entity with the most abundant and perfect information for reservation, and taking other similar entities as auxiliary merging; and merging the repeated relations pointing to the same entity pair, and normalizing the weights of the relations.
As a preferable scheme of the knowledge graph construction method, the invention comprises the following steps: the dynamic knowledge fusion comprises the steps of scanning new data through multi-mode entity identification, comparing the new entity with the existing entity when the new entity is identified, distributing a globally unique identifier for the new entity if the new entity is different from the existing entity, extracting the relation between the new entity and the existing entity according to the attribute of the new entity, and fusing the new entity into a knowledge graph; if the new entity is repeated with the existing entity, detecting attribute change of the existing entity, positioning the corresponding existing entity in the knowledge graph, updating the attribute of the entity, and creating a history record for the changed attribute; identifying new relations among the entities from the new data by using the deep learning model, if the new relations do not exist in the knowledge graph, carrying out normalization processing on the types and the attributes of the new relations, and fusing the new relations into the knowledge graph; if the new relationship exists in the knowledge graph, detecting the weight and attribute change of the existing relationship, positioning the corresponding existing relationship in the knowledge graph, updating the attribute and weight of the existing relationship, and updating the weight of the relationship to be expressed as,
Wherein Wu represents the updated relationship weight; wc represents the weight of the existing relationship; λ represents a learning rate of a step length controlling the weight update; Δw represents the magnitude of change in weight; τ represents a threshold value of the variation amplitude; sigma (gamma Gi) gamma denotes the confidence level calculated from gating; gamma denotes a parameter that adjusts the influence of confidence on the weight update.
The invention also provides a knowledge graph construction system, which comprises a data acquisition module, a semantic enhancement labeling recognition module and a deep learning model, wherein the data acquisition module acquires required knowledge data from multi-source data, converts non-text data into a text format, correspondingly stores the non-text data and the original non-text format data, carries out semantic enhancement labeling, carries out multi-mode entity recognition according to the knowledge data and the semantic enhancement labeling, and extracts the relationship between entities by utilizing the deep learning model to obtain the relationship between the entities in the data; the knowledge graph construction module is used for constructing a knowledge graph according to the relation between the entities, periodically executing a data grabbing task, detecting updated data, carrying out dynamic knowledge fusion, and periodically updating and optimizing the knowledge graph.
In a third aspect, the present invention also provides a computing device comprising: a memory and a processor;
The memory is configured to store computer-executable instructions that, when executed by the processor, implement the steps of the knowledge graph construction method.
In a fourth aspect, the present invention also provides a computer-readable storage medium storing computer-executable instructions which, when executed by a processor, implement the steps of the knowledge graph construction method.
The invention has the beneficial effects that: the method of the invention combines the BERT model and the manual review method, thereby remarkably improving the accuracy of semantic annotation and effectively extracting complex relations among entities through a deep learning technology. Dynamic knowledge fusion and regular updating of the atlas ensure timeliness of knowledge, enhance application value of the knowledge atlas, and provide accurate and comprehensive information retrieval, thereby supporting more intelligent decision and service.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
Example 1
Referring to fig. 1, for one embodiment of the present invention, a knowledge graph construction method is provided, including:
s1: knowledge data are collected from the multi-source data and preprocessed, and semantic enhancement labeling is carried out by combining the BERT model and manual review.
Further, sources of multi-source data collection including text files, online articles, databases, social media content, images, video and sound recordings, etc., are determined from the knowledge graph to be constructed, and data is automatically collected from these sources using data capture tools and APIs. For example, articles are collected from a web news source, records are derived from a database, or posts are crawled using a social media API.
The collected data includes text, image, audio and video forms, the collected data is cleaned, irrelevant or redundant information is removed, such as HTML tags, advertisements, irrelevant metadata and the like, then the data is converted into a consistent format, the date and time representation is unified, all the text is converted into the same language, the large data set is divided, the data is divided into smaller and manageable parts, and simultaneously, relevant data from different sources is integrated for unified processing in subsequent steps.
The text data is subjected to basic natural language processing such as word segmentation, part-of-speech tagging, named entity recognition and the like so as to facilitate deeper semantic analysis, text and knowledge in an image are extracted by using an image recognition technology for non-text data, audio content is transcribed by using a voice recognition technology, the audio content is converted into an analyzable text format, and original data and the converted text data are corresponding and stored.
Furthermore, in the knowledge graph construction process, accurate identification and classification of entities (such as names, places, organizations and the like) in the text are basic steps, semantic enhancement labeling can provide deeper text understanding, so that accuracy and richness of entity identification are improved, and meanwhile, relationships between the entities can be extracted and labeled more accurately, for example, whether the two entities are in cooperation, competition or other types of relationships can be determined.
And carrying out semantic enhancement labeling on the text formats of the text data and the non-text data after conversion by combining a BERT model and manual review, providing a confidence score for each labeling output by the BERT model, setting a threshold A according to the confidence score output by the BERT model, setting an error-prone data type according to error historical data of the BERT model, and analyzing the preprocessed data by using the BERT model.
When the confidence scores of all labels in the data are larger than or equal to A and the data type does not belong to the error-prone data type, no manual review is performed; when the confidence coefficient score of the label in the data is smaller than A, the label with the confidence coefficient score smaller than A accounts for less than 50% of all labels, and the data type does not belong to the error prone data type, performing manual review on the label with the confidence coefficient score smaller than A; when the confidence coefficient score of the label in the data is smaller than A, and the label with the confidence coefficient score smaller than A accounts for more than 50% of all labels, manually reviewing all the data, and feeding back the result of the manual review to the BERT model for retraining and optimizing; and when the data type belongs to the error-prone data type, manually reviewing all the data, and feeding back the manually reviewed result to the BERT model for retraining and optimization.
It should be noted that, even if the confidence of most data labeling is high, regular sampling review needs to be implemented to ensure the overall labeling quality, and an exception handling mechanism is set to cope with the problem that the model outputs an exception (such as that all labeling confidence is too low) or the problem that occurs in the data processing process.
By semantic enhancement labeling, it can be ensured that the information extracted from the source data has high accuracy, which is crucial for constructing accurate and reliable knowledge maps, and accurate labeling helps to better understand and classify entities and relationships in the data.
Moreover, the accurate labeling enables the data in the knowledge graph to be easier to search and retrieve, and in practical application, such as intelligent search engines, recommendation systems or data analysis, the efficiency and the relevance of the results can be remarkably improved.
S2: and carrying out multi-mode entity identification, and extracting the relation among the entities by using a deep learning model.
Furthermore, the multi-modal entity recognition refers to the process of recognizing and extracting entity information from different types (or 'modes') of data sources in the knowledge graph construction, and through integrating data from different modes, more comprehensive and deep information about a single entity can be obtained, and the data from different modes can be mutually verified and supplemented, so that the accuracy of entity recognition is improved.
And integrating the text data and the non-text data by aligning entity boundaries in the text data with entity positions in the non-text data, extracting features of the integrated data, and extracting text features of the text data by using a pre-trained BERT model.
For image and video data, a convolutional neural network is used for extracting image features, for audio data, a Mel frequency spectrum feature extraction method is used for extracting audio features, non-text features are integrated, and entities in text data are identified and marked according to the integrated non-text features.
Further, by identifying entities and representation of contexts between entities, establishing a context-encoding layer that integrates interactions between entities into the context while taking into account characteristics of individual entities, and then introducing a dynamic relationship gating mechanism by which the degree of interest of the model in relation to different types of entities is dynamically adjusted,
Wherein R represents the relationship type between the entities; e represents an entity feature; c represents context information between entities; gi represents the gating value for relationship type i; wr represents a weight matrix of relationship type prediction; br represents a bias term for relationship type prediction; wu represents a context-encoded weight matrix; n represents the number of relationship types; m represents the dimension of the context feature; sigma represents a sigmoid activation function; alpha represents the adjustment parameter.
By identifying entities in text and relationships between them, connections between entities in a knowledge graph are constructed, and relationships between entities may represent various relationships between entities, such as collaborative relationships between people, associations between places and events, and so on.
By continuously extracting entity relations in the text, the content of the knowledge graph can be gradually enriched, so that the knowledge graph has more information value, a more comprehensive and detailed knowledge graph can be constructed, and knowledge reasoning can be performed according to the entity relations. For example, by knowing that a person is related to an event, other relevant information, such as the time, place, etc. of the event, can be inferred.
S3: and constructing a knowledge graph, carrying out dynamic knowledge fusion, and updating and optimizing the knowledge graph regularly.
Further, topics and scope of the knowledge graph are defined, and core entity types and attributes, and relationships among the core entity types and attributes are determined. Combining the identified entities with the extracted relationships to construct a preliminary structure of the knowledge graph, distributing a unique identifier to each entity and the relationships between the entities, identifying and merging repeated entity nodes, and selecting the entity with the most abundant and complete information for reservation and other similar entities as auxiliary merging when similar entities exist; and merging the repeated relations pointing to the same entity pair, and normalizing the weights of the relations.
Further, constructing the knowledge graph includes setting a relation threshold Rh>Rl between the entities according to the historical data, setting a gating threshold Gh>Gl, adding the relation between the entities to the knowledge graph when R is more than or equal to Rh and G is more than or equal to Gh, marking the relation between the entities as a high confidence relation, and providing priority for subsequent data analysis and query.
And when Rl≤R<Rh and G is more than or equal to Gl among the entities, adding the relation into a queue to be verified, checking the relation in the queue to be verified according to the follow-up data, adding the relation into a knowledge graph if the follow-up data is checked to pass, and triggering manual review if the follow-up data is checked to fail.
When R is more than or equal to Rl and Gl≤G<Gh among the entities, carrying out context analysis on the relationship, checking whether ambiguity and potential ambiguity exist, calling an additional data source to carry out cross-validation, adding the relationship into a map if the context analysis and the cross-validation are both passed, and transferring the relationship into a queue to be validated if the context analysis or the cross-validation is not passed.
When R < Rl or G < Gl between entities, adding the relationship into an isolation queue, comparing the relationship with historical data through pattern analysis, checking whether misjudgment exists, if misjudgment exists, re-carrying out multi-mode entity identification and relationship extraction, and if misjudgment does not exist, marking the relationship as low confidence and carrying out manual review.
The dynamic knowledge fusion comprises the steps of periodically fusing new entities and relations into an existing map, scanning new data through multi-mode entity identification, comparing the new entities with the existing entity when the new entities are identified, distributing a globally unique identifier for the new entities if the new entities are different from the existing entity, extracting the relations between the new entities and the existing entity according to the attributes of the new entities, and fusing the new entities into the knowledge map; if the new entity is repeated with the existing entity, detecting attribute change of the existing entity, positioning the corresponding existing entity in the knowledge graph, updating the attribute of the entity, and creating a history record for the changed attribute.
Identifying new relations among the entities from the new data by using the deep learning model, if the new relations do not exist in the knowledge graph, carrying out normalization processing on the types and the attributes of the new relations, and fusing the new relations into the knowledge graph; if the new relationship exists in the knowledge graph, detecting the weight and attribute change of the existing relationship, positioning the corresponding existing relationship in the knowledge graph, updating the attribute and weight of the existing relationship, and updating the weight of the relationship to be expressed as,
Wherein Wu represents the updated relationship weight; wc represents the weight of the existing relationship; λ represents a learning rate of a step length controlling the weight update; Δw represents the magnitude of change in weight; τ represents a threshold value of the variation amplitude; sigma (gamma Gi) gamma denotes the confidence level calculated from gating; gamma denotes a parameter that adjusts the influence of confidence on the weight update.
It should be noted that, the amount of weight update depends not only on the difference between the new weight and the old weight, but also on the change amplitude of the weight and the confidence of the relationship, if the confidence of a relationship is high and the difference between the new weight and the old weight is large, the step size of the weight update will be larger. The model can respond faster to those relationships that have high confidence and large weight changes, while taking more careful updating steps for those relationships that have lower confidence.
The embodiment also provides a knowledge graph construction system, which comprises a data acquisition module, a semantic enhancement labeling recognition module and a depth learning model, wherein the data acquisition module acquires required knowledge data from multi-source data, converts non-text data into a text format, correspondingly stores the non-text data and the original non-text format data, carries out multi-modal entity recognition according to the knowledge data and the semantic enhancement labeling, and extracts the relationship between entities by using the depth learning model to obtain the relationship between the entities in the data; the knowledge graph construction module is used for constructing a knowledge graph according to the relation between the entities, periodically executing a data grabbing task, detecting updated data, carrying out dynamic knowledge fusion, and periodically updating and optimizing the knowledge graph.
The present embodiment also provides a computing device comprising, a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the knowledge graph construction method according to the foregoing embodiment.
The present embodiment also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the knowledge graph construction method as set forth in the above embodiments.
The storage medium proposed in the present embodiment belongs to the same inventive concept as the knowledge graph construction method proposed in the above embodiment, and technical details not described in detail in the present embodiment can be seen in the above embodiment, and the present embodiment has the same beneficial effects as the above embodiment.
From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a floppy disk, a read only Memory (ReadOnly, a Memory, a ROM), a random access Memory (RandomAccessMemory, RAM), a FLASH Memory (FLASH), a hard disk, or an optical disk of a computer, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method of the embodiments of the present invention.
Example 2
In order to verify the beneficial effects of the invention, a scientific demonstration is carried out through a comparison experiment.
In order to prove that the method can improve accuracy, richness and dynamic updating efficiency compared with the traditional method in the aspect of constructing a knowledge graph, the same multisource data set is selected, the comparison of the two methods is fair, the same hardware and software environments are respectively prepared for the traditional method and the method, wherein the traditional method is a method based on rules and heuristic algorithms, entity identification uses a dictionary-based method to identify entities in a text, and relation extraction relies on a predefined mode and keywords to identify the relation between the entities.
The method and the device respectively construct two knowledge maps by using the traditional method and the method, record the completion time and accuracy of each step in the construction process, and evaluate the performance of the two knowledge maps, including entity identification accuracy, relation extraction accuracy, map updating speed and data richness, wherein specific experimental data are shown in the following table.
Table 1 comparative test data table
The method and the traditional method are further evaluated through the accuracy and the recall rate, and in the aspect of entity identification, the traditional method comprises the following steps:
The method comprises the following steps:
In relation extraction, the conventional method is as follows:
The method comprises the following steps:
It can be seen that the accuracy and recall rate of the method of the invention in terms of entity identification and relationship extraction are higher than those of the conventional method. The method is more accurate and reliable in the aspects of entity identification and extraction, valuable information can be extracted from multi-source data more effectively, and a richer and more accurate knowledge graph is constructed.
The method of the invention combines the BERT model and the manual review method, thereby remarkably improving the accuracy of semantic annotation and effectively extracting complex relations among entities through a deep learning technology. Dynamic knowledge fusion and regular updating of the atlas ensure timeliness of knowledge, enhance application value of the knowledge atlas, and provide accurate and comprehensive information retrieval, thereby supporting more intelligent decision and service.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.