Detailed Description
The embodiment of the application provides a training method, a training device, training equipment and a storage medium for a similar text matching model, which are used for shortening the distance between a positive sample and a similar sample through a triplet loss function and pushing away the distance between the positive sample and a different sample so as to enable similar text vectors to form clusters in a feature space, improve the learning capability of the similar text matching model on the similarity between the text vectors, enable the similar text matching model to be better fitted, and further improve the recall rate of a target similar text matching model on the similar text.
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented, for example, in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "includes" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
With the rapid development of information, cloud technology (Cloud technology) is also gradually moving into the aspects of people's life. The cloud technology is a generic term of network technology, information technology, integration technology, management platform technology, application technology and the like based on cloud computing business model application, can form a resource pool, and is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.
Cloud Security (Cloud Security) refers to a generic term of Security software, hardware, users, institutions, and Security Cloud platforms based on Cloud computing business model application. Cloud security fuses emerging technologies and concepts such as parallel processing, grid computing, unknown virus behavior judgment and the like, acquires the latest information of Trojan horse and malicious programs in the Internet through abnormal monitoring of a large number of network clients on software behaviors, sends the latest information to a server for automatic analysis and processing, and distributes solutions of viruses and Trojan horse to each client. The training test method of the similar text matching model provided by the embodiment of the application can be realized through a cloud computing technology and a cloud security technology.
It should be understood that the training test method of the similar text matching model provided by the application can be applied to the fields of cloud technology, artificial intelligence, intelligent traffic and the like, and is used for completing the pushing or putting of the similar text to the target object through the matching of the similar text, for example, more matched advertisements can be recommended to the target object through the matching of the similar text to advertisement text, for example, more matched commodities can be recommended to the target object through the matching of the similar text to commodity text, for example, more matched books or documents can be recommended to the target object through the matching of the similar text to book text, in the various scenes, the model is usually judged based on deep learning to realize the matching of the similar text, but a large number of manual labeling training samples are needed to enable the model to be better fitted and generalized, however, the text semantics are rich and the standards of different labeling personnel about the text similarity are very different and are very difficult to unify, and therefore, great difficulty is brought to labeling the training samples to the training data, the training samples are required to be used for training, optimization and iteration, the model is difficult to be matched, and the model is difficult to be better, and the model is called, and the model is difficult to be called.
In order to solve the above-mentioned problems, the present application provides a training test method of a similar text matching model, which is applied to a text data control system shown in fig. 1, referring to fig. 1, fig. 1 is a schematic diagram of a structure of the text data control system in an embodiment of the present application, as shown in fig. 1, a server respectively inputs a first batch of positive examples and a first batch of negative examples in a first batch of sample sets to an original similar text matching model to perform vector conversion operation, so as to obtain a first batch of positive example sentence vectors and a first batch of negative example sentence vectors, performs a triplet construction operation on the first batch of positive example sentence vectors, so as to obtain a plurality of first batch triples, further performs a loss calculation operation on the plurality of first batch triples, obtains a first batch loss function corresponding to a first batch of sample set, and performs a parameter adjustment operation on the original similar text matching model according to the first batch loss function, so as to obtain an intermediate similar text matching model, and then repeatedly obtains a second batch of similar text matching model corresponding to the target scene, and performs a parameter adjustment operation on the first batch of sample sets, so as to obtain a target text matching operation loss. Through the method, the triplet loss function can be obtained by constructing the triplet through the first batch of positive example sentence vectors and the first batch of negative example sentence vectors, the distance between the positive example sample and the similar sample can be shortened through the triplet loss function, the distance between the positive example sample and the similar sample can be pushed away, so that similar text vectors can form clusters in a feature space, the learning capacity of the similar text matching model on the similarity between the text vectors is improved, the similar text matching model can be better fitted, and the recall rate of the target similar text matching model on the similar text is improved.
It should be understood that only one terminal device is shown in fig. 1, and in an actual scenario, a greater variety of terminal devices may participate in the data processing process, where the terminal devices include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, intelligent home appliances, vehicle terminals, etc., and the specific number and variety are determined by the actual scenario, and the specific number and variety are not limited herein. In addition, one server is shown in fig. 1, but in an actual scenario, there may also be a plurality of servers involved, especially in a scenario of multi-model training interaction, the number of servers depends on the actual scenario, and the present application is not limited thereto.
It should be noted that in this embodiment, the server may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content distribution network (content delivery network, CDN), and basic cloud computing services such as big data and an artificial intelligence platform. The terminal device and the server may be directly or indirectly connected through wired or wireless communication, and the terminal device and the server may be connected to form a blockchain network, which is not limited herein.
In order to solve the above-mentioned problems, the present application proposes a training method of a similar text matching model, which is generally executed by a server or a terminal device, and accordingly, a training apparatus applied to the similar text matching model is generally provided in the server or the terminal device.
It is to be understood that the training method, apparatus, device and storage medium of the similar text matching model disclosed in the present application, wherein a plurality of servers or terminal devices may be formed into a blockchain, and the servers or terminal devices are nodes on the blockchain. In practical applications, data sharing between nodes may be required in a blockchain, and text data or the like may be stored on each node.
Referring to fig. 2 and fig. 6, an embodiment of a training method for a similar text matching model in an embodiment of the present application includes:
In step S101, a first batch sample set corresponding to a target scene is obtained, where the first batch sample set includes a first batch positive sample and a first batch negative sample;
In this embodiment, before a matching request or a search request sent by a target terminal device is obtained, for example, before a text to be matched or a text to be searched is obtained, a first batch of sample sets corresponding to a target scene may be obtained, so that an original similar text matching model may be trained by a first batch of positive examples samples and a first batch of negative examples in the first batch of sample sets, so as to optimize the original similar text matching model.
Specifically, before obtaining the text to be matched or the text to be retrieved, the target scene may be specifically represented as an advertisement scene, a news information scene, a book management scene, or the like, or may be other target scenes, which are not specifically limited herein, and further, a first batch of sample sets corresponding to the target scene may be obtained, where the first batch of sample sets may be a batch of sample data including a positive sample set and a negative sample set that are randomly extracted from the sample sets, the first positive sample set is advertisement text under a known advertisement scene, and the first negative sample set and the first positive sample are text data subjected to ES matching, where, as shown in table 1, one positive sample may correspond to one or more negative sample.
TABLE 1
For example, a first batch sample set may be 128-dimensional sample data, i.e. include 12 pieces of sample data, which may include 10 positive samples, i.e. first batch positive samples, and 118 pieces of text data that are ES-matched with 10 positive samples, i.e. first batch negative samples, respectively.
In step S102, the first batch of positive examples and the first batch of negative examples are respectively input to the original similar text matching model for vector conversion operation, so as to obtain a first batch of positive examples and a first batch of negative examples;
In this embodiment, after the first batch of positive examples and the first batch of negative examples are obtained, the first batch of positive examples and the first batch of negative examples may be input to a plurality of original similar text matching models respectively for vector conversion, so as to obtain a first batch of positive example sentence vectors and a first batch of negative example sentence vectors, so that the distance between each positive example and each negative example can be better calculated by the first batch of positive example sentence vectors and the first batch of negative example sentence vectors, and the similarity between the positive examples and the negative examples can be better represented by the distance between the positive examples and the negative examples.
Specifically, as shown in fig. 6, the original similar text matching model may be specifically represented by a Bert model combined with a plurality of fully connected layers and a pooling layer, and other text processing models may also be used, which are not particularly limited herein. After the first batch of positive examples and the first batch of negative examples are obtained, as shown in fig. 6, the first batch of positive examples and the first batch of negative examples can be respectively input into a plurality of original similar text matching models to perform vector conversion, for example, the first batch of positive examples and the first batch of negative examples are compiled by the Bert model respectively, at least two word vectors corresponding to the first batch of positive examples and at least two word vectors corresponding to the first batch of negative examples can be obtained, then at least two word vectors corresponding to the first batch of positive examples and at least two word vectors corresponding to the first batch of negative examples are respectively passed through a plurality of full-connection layers and a pooling layer, and the first batch of positive examples and the first batch of negative examples can be obtained.
In step S103, performing a triplet construction operation on the first lot normal sentence vector to obtain a plurality of first lot triples, where each first lot triplet includes a first lot normal sentence vector, a first lot similar sentence vector, and a first lot heterogeneous sentence vector, and the first lot similar sentence vector and the first lot heterogeneous sentence vector are derived from the first lot negative example sentence vector;
In this embodiment, after the first lot of positive example sentence vectors and the first lot of negative example sentence vectors are obtained, for each first lot of positive example sentence vectors, one first lot of positive example sentence vector, one first lot of similar sentence vector and one first lot of heterogeneous sentence vector may be randomly combined into one triplet, where one first lot of positive example sentence vector may correspond to one or more triples, so that a plurality of first lot triples may be obtained.
The first lot of similar sentence vectors can be understood as negative example samples with higher similarity to the first lot of positive example sentence vectors, and can be specifically expressed as sentence vectors corresponding to the negative example samples with the matching score larger than 0.5, and the first lot of heterogeneous sentence vectors can be understood as negative example samples with lower similarity to the first lot of positive example sentence vectors, and can be specifically expressed as sentence vectors corresponding to the negative example samples with the matching score smaller than 0.5.
Specifically, after the first lot of positive example sentence vectors and the first lot of negative example sentence vectors are obtained, for example, 10 first lot of positive example samples and 118 first lot of negative example samples are obtained, and for one first lot of positive example samples, the first lot of negative example samples matched with the first lot of positive example samples through ES are assumed to be 3, wherein 2 first lot of similar sentence vectors and 1 first lot of alien sentence vectors corresponding to the 3 first lot of negative example samples are randomly extracted, and one first lot of positive example sentence vectors, one first lot of similar sentence vectors and one first lot of alien sentence vectors are randomly extracted to form one triplet, so that 2 triples corresponding to the first lot of positive example samples can be obtained.
In step S104, performing a loss calculation operation on the first lot triples to obtain a first lot loss function corresponding to the first lot sample set;
In this embodiment, after obtaining the first batches of triples, a loss function may be calculated for each triplet, and then the obtained first batches of loss functions are integrated into a loss function, that is, the first batch of loss functions corresponding to the first batch of sample sets, so that the triples loss function can be constructed based on the triples, the distances between the positive samples and the similar samples can be shortened, and the distances between the positive samples and the similar samples can be pushed away, so that similar text vectors can form clusters in the feature space, and the purpose of text matching is achieved.
Further, the similarity between text sentences can be regressed through the triple loss function, so that the original similar text model is represented by the embedded vectors (Embedding) obtained after learning, namely the similarity between the sentence vectors, and the normalized matching score is as close as possible.
The first batch loss function may be expressed as a triplet loss function, which may be specifically shown as follows:
Wherein L is a first batch loss function, ES (a, p) represents a normalized matching score of a positive example sample a corresponding to a positive example sentence vector and a negative example sample p corresponding to a similar sentence vector, d (p, n) represents cosine similarity between the similar negative example sentence vector corresponding to the negative example sample p and a heterogeneous sentence vector corresponding to the negative example sample n, i.e., d (a, p) =1-cosine (a, p).
In step S105, according to the first batch loss function, performing parameter adjustment operation on the original similar text matching model to obtain an intermediate similar text matching model;
Specifically, after the first batch of loss functions are obtained, parameter adjustment operation may be performed on the original similar text matching model, specifically, a reverse gradient descent algorithm may be adopted to update the model parameters in bert until convergence, so that an intermediate similar text matching model may be obtained.
In step S106, based on the intermediate similar text matching model, a second batch of sample sets corresponding to the target scene is repeatedly acquired, and a vector conversion operation, a triplet construction operation, a loss calculation operation, and a parameter adjustment operation are performed, so as to obtain the target similar text matching model.
In this embodiment, after the intermediate similar text matching model is obtained, a second batch of sample sets corresponding to the target scene may be repeatedly obtained, and based on the obtained second batch of sample sets, the vector conversion operation, the triplet construction operation, the loss calculation operation, and the parameter adjustment operation similar to those of steps S102 to S105 may be repeatedly performed until model parameters of the intermediate similar text matching model tend to be stable, and the intermediate similar text matching model may be used as the target similar text matching model.
According to the training method of the similar text matching model, the triples can be constructed through the first batch of positive example sentence vectors and the first batch of negative example sentence vectors to obtain the triples loss function, the distances between the positive example samples and the similar samples can be shortened through the triples loss function, the distances between the positive example samples and the similar samples are pushed away, so that the similar text vectors can form clusters in the feature space, the learning capacity of the similar text matching model on the similarity between the text vectors is improved, the similar text matching model can be better fitted, and the recall rate of the target similar text matching model on the similar text is improved.
Optionally, on the basis of the embodiment corresponding to fig. 2, in another optional embodiment of the training method for a similar text matching model provided by the embodiment of the present application, obtaining a first batch of sample sets corresponding to a target scene includes:
Acquiring a target text data set corresponding to a target scene, wherein the target text data set at least comprises a first batch of positive examples samples and source text data corresponding to the target scene;
Retrieving N first matching texts corresponding to the first batch of positive examples from the target text data set as N first batch of negative examples, wherein N is an integer greater than 1;
Calculating matching scores between the first batch of positive examples and each first batch of negative examples to obtain N first matching scores;
respectively carrying out normalization operation on the N first matching scores to obtain N sample matching scores;
and constructing a first batch sample set according to the first batch positive example sample, the first batch negative example sample and the sample matching score.
In this embodiment, as shown in fig. 5, before performing model training, a corresponding target text dataset may be obtained according to a target scene, N first matching texts corresponding to a first batch of positive examples may be retrieved from the target text dataset to obtain a first batch of negative examples, then, normalization operations may be performed on N first matching scores respectively to obtain N sample matching scores to obtain a first batch of sample sets corresponding to the target scene, and the target text dataset may be constructed by a search engine (ELASTICSEARCH, ES), where ES is a text search engine supporting efficient and multiple scoring strategies, and a self-supervision training sample, such as the first batch of sample sets, may be obtained by the target text dataset, without spending a lot of time to make text similarity standards, without performing tedious manual labeling, and the target text dataset may be replaced and adjusted according to different target scenes and requirements, so that a sample set suitable for the target scene may be obtained better and more accurately, and the construction of the sample set may be more flexible and stronger.
Specifically, before the target text data set corresponding to the target scene is obtained, a search library corresponding to the target scene may be established through a search engine, that is, source text data corresponding to the target scene is obtained, where the source text data may be specifically represented by an advertisement text, a description or a commodity text, etc. under the advertisement scene, or may be represented by other text data, where no specific limitation is made, the source text data may be specifically obtained by obtaining initial text data corresponding to the target scene through the search engine, further, since the initial text data, such as an advertisement text, has more text with the same meaning, for example, only punctuation difference, only individual text is different, so, in order to enhance diversity of the obtained initial text data as much as possible, for example, an edit distance between each two texts or a length of a longest text in each two texts is calculated, if the edit distance or a length difference between each two texts is smaller than a preset distance threshold, a text with a shorter length may be used as a sample with the same meaning, then an ES may be filtered, and a text with a different length may be used as a text segment, and then a text segment may be stored in an index unit (an IKER) may be used, and a text segment may be extracted from a text segment unit may be used as a memory word.
Further, as shown in fig. 5, after the source text data is acquired, a positive example sample (query) collected according to the target scene may be acquired, and the target text data may be composed with the acquired source text data, for example, assuming that six thousand positive example samples in the advertisement scene are collected, two million six thousand target text data may be composed with two million pieces of source text data in the advertisement scene, and then, when one target text data is stored by the ES, the ES may extract word units from the target text data using the word segmenter to establish an index of the target text data.
Further, as shown in fig. 5, the retrieval may be performed in the target text data for each positive sample, for example, six positive samples are retrieved in two million and six thousand target text data, and the matching of ESs may be performed during the retrieval, specifically, may be one or more of matching methods based on a bag-of-words (bag-of-words) or a sentence matching method that generates a sentence vector from a word vector, for example, as shown in fig. 3, assuming that one positive sample is "beijing Tiananmen", a set of word units such as "beijing", "Tiananmen" may be obtained after passing through the word splitter, if one source text data is "i'm, i.e." i "," beijing ", and" Tiananmen "after dividing words, and based on the matching of keywords of the bag-of-words, that is" beijing "and" Tiananmen ", the relevance score between the positive sample and the source text data may be calculated according to the word units that are hit, and the matching score may be similarly arranged, and the matching score may be the N-th score may be obtained after dividing words into" i.e. "beijing", and the matching score may be ranked and the first text data is similar to the first score and the first text data.
For example, as shown in fig. 4, assuming that a positive sample is a "electric drill" screwdriver which is in a fire-explosion country for 1 minute and is easy to punch out a first piece of | 47 pieces of material, 500 kinds of screws | 1-fold | can be used for matching ES with target text data containing the positive sample, if the text matched with the positive sample is also the positive sample, the maximum matching score can be obtained as "133.11464", or the positive sample can be matched with a source text data such as "electric drill" which is in a fire-explosion country for 1 minute 2500 rotations, the maximum 3-second punching | 47 pieces of material can be used for applying 500 kinds of screws | 1-fold | ", a matching score can be obtained as" 105.32658", or the positive sample can be matched with another source text data such as" the electric drill is in a fire | 1 minute 250 rotations, the first piece of material is easy to punch out | 47 pieces of material, the maximum matching score can be obtained as "133.11464", or the positive sample can be matched with 500 kinds of screws | 1-fold | can be easily used for not be used for driving the positive sample, the positive sample can be obtained as "102.89305", and the electric drive out tool can be matched with a second piece of material for example 1-fold 1 minute can be obtained, and the first piece of material can be easily punched out 1-fold 1 piece of material.
Further, after N first matching scores are obtained, since the matching score does not have an explicit range, for example, the highest matching score may vary from tens to hundreds according to different positive samples, if the matching score is directly fitted to the deep learning model, the model training process is difficult to converge, so, as shown in fig. 5, each matching score may be normalized by normalizing the matching score to obtain a sample matching score, which is used for solving the comparability between the matching scores, so that each matching score is in the same order of magnitude, and thus may be better fitted to the deep learning model, so that the model training process may be better and easier to converge, and then, a first batch sample set may be better constructed according to the first batch of positive samples, the first batch of negative samples, and the sample matching score, where the normalization for each matching score may be specifically obtained by the following formula (1):
where q is a positive example sample, d is source text data, k is the total number of target text data, es_ normed (q, d) sample match scores, score (q, d) is a match score,The maximum matching score is obtained by matching the positive sample with the positive sample, wherein the positive sample is the best matching with the positive sample.
Wherein score (q, d) can be obtained by the following formula (2):
score(q,d)=coord(q,d)*queryNorm(q)*∑t∈d(tf(t∈d)*idf(t)2*
boost(t)*norm(t,d))(2)
Wherein, chord (q, d) is a coordination factor, queryNorm (q) is a query norm, tf (t E d) is a word frequency, idf (t)2 is an inverse document frequency, boost (t) is a term weight, norm (t, d) is a length norm.
Optionally, in another optional embodiment of the training method for a similar text matching model provided by the embodiment of the present application based on the embodiment corresponding to fig. 2, performing a triplet construction operation on the first batch of normal sentence vectors to obtain a plurality of first batch triples, including:
Dividing the first batch of negative example sentence vectors according to the sample matching score to obtain a similar sentence vector set and a heterogeneous sentence vector set;
extracting any similar sentence vector from the similar sentence vector set to obtain similar sentence vectors of a first batch;
any heterogeneous sentence vector is extracted from the heterogeneous sentence vector set, and a first batch of heterogeneous sentence vectors are obtained.
In this embodiment, after the first lot of positive example sentence vectors and the first lot of negative example sentence vectors are obtained, the first lot of negative example sentence vectors may be divided into a similar sentence vector set and a heterogeneous sentence vector set according to the sample matching score of the first lot of sample sets, then any similar sentence vector may be extracted from the similar sentence vector set to obtain the first lot of similar sentence vectors, and any heterogeneous sentence vector may be extracted from the heterogeneous sentence vector set to obtain the first lot of heterogeneous sentence vectors, so that a triplet may be constructed for the first lot of positive example sentence vectors based on the first lot of similar sentence vectors and the first lot of heterogeneous sentence vectors, so that the distance between the similar text and the dissimilar text may be better represented by the triplet.
Specifically, the dividing operation is performed on the first batch of negative example sentence vectors according to the sample matching score, which may specifically be that sentence vectors with the sample matching score greater than 0.5 are used as a similar sentence vector set, sentence vectors with the sample matching score less than 0.5 are used as a different sentence vector set, then any similar sentence vector can be randomly extracted from the similar sentence vector set to obtain the first batch of similar sentence vectors, and similarly, any different sentence vector is randomly extracted from the different sentence vector set to obtain the first batch of different sentence vectors.
TABLE 2
For example, as shown in table 2, the matching score corresponding to the negative example sample may be obtained, the positive example sample "the electric screwdriver is too much | 1 minute 250 turns, the | 47 pieces of screwdriver head can be easily punched, 500 kinds of screws | can be screwed well, the negative example sample 1 and the negative example sample 2 corresponding to the inexpensive | 1 are converted into the negative example sentence vector to obtain the negative example sentence vector 1 and the negative example sentence vector 2, and then the negative example sentence vector 1 may be used as the similar sentence vector and the negative example sentence vector 2 may be used as the heterogeneous sentence vector according to the sample matching score 0.89 corresponding to the negative example sentence vector 1 and the sample matching score 0.18 corresponding to the negative example sentence vector 2.
Optionally, in another optional embodiment of the training method for a similar text matching model provided by the embodiment of the present application based on the embodiment corresponding to fig. 2, performing a loss calculation operation on a plurality of first batch triples, to obtain a first batch loss function corresponding to a first batch sample set, including:
Respectively carrying out loss calculation operation on the first batch of normal sentence vectors, the first batch of similar sentence vectors and the first batch of heterogeneous sentence vectors to obtain loss functions corresponding to a plurality of first batch triples;
And carrying out weighted calculation operation on the loss functions corresponding to the first batch of triples to obtain the first batch of loss functions.
In this embodiment, after obtaining a plurality of first-batch triples constructed based on the first-batch normal sentence vector, loss calculation may be performed on each triplet, that is, loss calculation may be performed on the first-batch normal sentence vector, the first-batch similar sentence vector, and the first-batch heterogeneous sentence vector, so as to obtain a loss function corresponding to each first-batch triplet, and then, weighting calculation may be performed on the loss functions corresponding to the plurality of first-batch triples according to a preset weight, so as to obtain a first-batch loss function, and the triples loss function may be constructed based on the triples, so that distances between the normal sample and the similar sample may be pulled, and distances between the normal sample and the heterogeneous sample may be pushed, so that similar text vectors may form clusters in a feature space, thereby achieving the purpose of text matching.
Specifically, after obtaining the first plurality of triples, the loss function may be calculated for each triplet, specifically, the first lot normal sentence vector, the first lot similar sentence vector, and the first lot heterogeneous sentence vector may be substituted into a function expression of the triplet loss function in step S104 to perform calculation, so that a loss function corresponding to a triplet may be obtained, and then the obtained first plurality of loss functions may be integrated into one loss function, specifically, the loss functions corresponding to the first plurality of triples may be subjected to a weighted calculation operation through a preset weight, so as to obtain the first lot loss function corresponding to the first lot sample set, where the preset weight is set according to an actual application requirement, and the specific limitation is not imposed herein.
Optionally, in another optional embodiment of the training method for a similar text matching model provided in the embodiment of the present application based on the embodiment corresponding to fig. 2, a second batch of sample sets corresponding to the target scene is repeatedly obtained based on the intermediate similar text matching model, and a vector conversion operation, a triplet configuration operation, a loss calculation operation and a parameter adjustment operation are performed to obtain the target similar text matching model, which includes:
Acquiring a second batch of sample sets corresponding to the target scene, and executing vector conversion operation, ternary structure construction operation and loss calculation operation according to the second batch of sample sets to obtain a second loss function;
And if the second loss function is smaller than the first threshold value, taking the current intermediate similar text matching model as a target similar text matching model.
In this embodiment, after the intermediate similar text matching model is obtained, vector conversion operation, ternary combination operation and loss calculation operation may be performed continuously on a second batch of sample sets corresponding to the target scene, so as to obtain a second loss function, and reverse iterative training is performed on the intermediate similar text matching model through the second loss function, when the second loss function is smaller than the first threshold, the current intermediate similar text matching model may be used as the target similar text matching model, so that the model may fully learn the similarity between text vectors, and the target similar text matching model may be better fitted, so that the recall rate of the target similar text matching model to the similar text is improved to a certain extent.
Specifically, after the second batch of sample sets corresponding to the target scene is obtained, where the second batch of sample sets is used to refer broadly to sample sets of other batches extracted from the sample sets different from the first batch of sample sets, specifically may be represented by three batches, four batches or N batches of sample sets, and then operations similar to the vector conversion operation, the three-group construction operation and the loss calculation operation in steps S102 to S104 may be performed, which are not described herein, to obtain a second loss function, where the second loss function may be used to refer broadly to a loss function corresponding to each batch of sample sets.
Further, after the second loss function is obtained, it may be understood that the smaller the second loss function is, the better the fitting of the model is, and thus, the second loss function may be compared with a first threshold, where the first threshold may specifically be represented by a smaller value, such as 0.18, and the first threshold is set according to the actual application requirement, and is not specifically limited herein, and then, when the second loss function is smaller than the first threshold, it may be understood that when the second loss function is already small enough, the current intermediate similar text matching model tends to be stable, and the current intermediate similar text matching model has converged, and then the current intermediate similar text matching model may be regarded as the target similar text matching model.
Optionally, in another optional embodiment of the training method for a similar text matching model provided in the embodiment of the present application based on the embodiment corresponding to fig. 2, a second batch of sample sets corresponding to the target scene is repeatedly obtained based on the intermediate similar text matching model, and a vector conversion operation, a triplet configuration operation, a loss calculation operation and a parameter adjustment operation are performed to obtain the target similar text matching model, which includes:
obtaining intermediate model parameters of an intermediate similar text matching model;
obtaining a second batch of sample sets corresponding to the target scene, and performing vector conversion operation, triple construction operation and parameter adjustment operation to obtain a current similar text matching model, wherein the current similar text matching model comprises current model parameters;
And if the difference value between the intermediate model parameter and the current model parameter meets a second threshold value, taking the current intermediate similar text matching model as a target similar text matching model.
In this embodiment, after the intermediate similar text matching model is obtained, vector conversion operation, ternary combination operation and loss calculation operation may be performed continuously on a second batch of sample sets corresponding to the target scene, so as to obtain a second loss function, and reverse iterative training is performed on the intermediate similar text matching model through the second loss function, when the difference between the intermediate model parameter and the current model parameter meets a second threshold, the current intermediate similar text matching model may be used as the target similar text matching model, so that the model may learn the similarity between text vectors sufficiently, and the target similar text matching model may be better fitted, so that the recall rate of the target similar text matching model to the similar text may be improved to a certain extent, where the second threshold may be specifically expressed as a smaller value, and the second threshold is set according to the actual application requirement, and is not particularly limited herein.
Specifically, after the intermediate similar text matching model is obtained, the intermediate model parameters may be extracted, further, a sample set of the next batch corresponding to the target scene may be obtained, and operations similar to the vector conversion operation, the triplet construction operation and the parameter adjustment operation in the steps S102 to S105 are repeatedly executed, which are not described herein, so that the current similar text matching model can be obtained, and the model parameters of the current similar text matching model are extracted, so as to obtain the current model parameters.
Further, since the intermediate similar text matching model tends to be stable, the convergence of the intermediate similar text matching model may be represented by the stability of the model parameters, and thus, the difference between the intermediate model parameters and the current model parameters may be calculated, and if the difference between the intermediate model parameters and the current model parameters is smaller than the second threshold, it may be understood that the difference between the intermediate model parameters and the current model parameters is sufficiently small, that is, the model parameters tend to be stable, that is, the intermediate similar text matching model tends to be stable, and then the current intermediate similar text matching model may be used as the target similar text matching model.
Optionally, in another optional embodiment of the training method for a similar text matching model provided by the embodiment of the present application based on the embodiment corresponding to fig. 2, based on the intermediate similar text matching model, a second batch of sample sets corresponding to the target scene are repeatedly obtained, and a vector conversion operation, a triplet configuration operation, a loss calculation operation and a parameter adjustment operation are performed, so that after the target similar text matching model is obtained, the method further includes:
receiving a text to be matched;
Respectively passing the text to be matched and the target text data set through a target similar text matching model to obtain sentence vectors to be matched and a plurality of original sentence vectors;
Calculating the similarity between the sentence vector to be matched and each original sentence vector to obtain a plurality of similarity scores;
And determining a target similar text according to the plurality of similar scores, and pushing the target similar text to the target terminal equipment.
In this embodiment, after the target similar text matching model is obtained, the target similar text matching model may be applied, by receiving the text to be matched and obtaining a target text data set corresponding to the text to be matched, then, the text to be matched and the target text data set are respectively input into the target similar text matching model, a sentence vector to be matched and a plurality of original sentence vectors are obtained through the target similar text model, a plurality of similarity scores may be obtained by calculating the similarity between the sentence vector to be matched and each original sentence vector, and the target similar text may be determined according to the plurality of similarity scores, and the target similar text may be pushed to the target terminal device, so that the matched target similar text may be recommended for the target object better and more accurately.
Specifically, the text to be matched may be specifically represented by advertisement text, commodity keywords, and the like, and may also be other text, which is not particularly limited herein. After the target similar text matching model is obtained, if a text to be matched such as an advertisement document a sent by a target object through target terminal equipment is received, a target scene such as an advertisement retrieval scene to which the text to be matched belongs can be determined first, a corresponding target similar text matching model and a target text data set such as an advertisement document retrieval library can be determined according to the target scene, then the obtained text to be matched and the target text data set can be respectively input into the target similar text matching model, a sentence vector to be matched and a plurality of original sentence vectors can be obtained through the target similar text matching model, further, the sentence vector to be matched and each original sentence vector can be subjected to pairwise matching, and the similarity between the sentence vector to be matched and each original sentence vector can be calculated, so that the similarity score between the sentence vector to be matched and each original sentence vector can be obtained, wherein the calculation mode of the similarity can be specifically through the euclidean distance or cosine similarity, and other similarity calculation modes can be also used, and the calculation modes of the similarity are not limited specifically.
Further, after obtaining the similarity score between the sentence vector to be matched and each original sentence vector, the similarity scores may be ranked according to the principle that the higher the similarity score is, and the similarity score may be ranked from high to low, and the original sentence vectors corresponding to the first ten or hundred similarity scores may be selected according to requirements of the target scene, for example, then, texts corresponding to the ten or hundred selected original sentence vectors are determined to be target similar texts, and pushed to the target terminal device.
When pushing the target similar text to the target terminal device, the pushing may be specifically performed according to the type of the target similar text, for example, the target similar text may be directly pushed to the target terminal device on the assumption that the target similar text is represented by a commodity image or a commodity link, or the target similar text may be further determined to be more matched with the target object from the target similar text on the assumption that the target similar text is represented by a video advertisement or hot spot information, or the like, according to the historical click rate and conversion rate of the target object, and the like, and then pushed to the target terminal device.
It can be understood that the target similar text matching model can also be used for a plurality of links such as commodity retrieval, advertisement material retrieval, advertisement estimation model characteristics, advertisement analysis and diagnosis and the like, and the overall advertisement putting effect of the whole link can be improved. For example, the commodity or advertisement material is usually provided with a corresponding document, and matching precision and recall rate can be greatly improved by combining text keyword search and target similar text matching model search, so that more matched commodity or advertisement is recommended for a target object. Or the method can be applied to a plurality of links such as recall, coarse-ranking or fine-ranking model estimation, strategy adjustment, analysis and diagnosis of the advertising effect and the like related to the advertising process. For example, in the rough ranking step, similar texts and picture videos in advertisements are acquired through the target similar text matching model so as to filter the similar advertisements and increase the diversity of the advertisements, or the advertisements can be better understood and the generalization performance of the models can be increased through the combination of the rough ranking or fine ranking estimated model and the target similar text matching model.
It will be appreciated that the target similar text matching model may also be applied to other text retrieval scenarios, such as common general knowledge, virtual or physical object retrieval, book retrieval in fine-grained fields, legal document retrieval, etc., without specific limitation.
Referring to fig. 7, fig. 7 is a schematic diagram showing an embodiment of a training apparatus for a similar text matching model according to an embodiment of the present application, and the training apparatus 20 for a similar text matching model includes:
An obtaining unit 201, configured to obtain a first batch sample set corresponding to a target scene, where the first batch sample set includes a first batch positive sample and a first batch negative sample;
The processing unit 202 is configured to input a first batch of positive example samples and a first batch of negative example samples to the original similar text matching model to perform vector conversion operation, so as to obtain a first batch of positive example sentence vectors and a first batch of negative example sentence vectors;
The processing unit 202 is further configured to perform a triplet construction operation on the first lot normal sentence vector to obtain a plurality of first lot triples, where each first lot triplet includes a first lot normal sentence vector, a first lot similar sentence vector, and a first lot heterogeneous sentence vector, and the first lot similar sentence vector and the first lot heterogeneous sentence vector are derived from the first lot negative example sentence vector;
the processing unit 202 is further configured to perform a loss calculation operation on the plurality of first batch triples, and obtain a first batch loss function corresponding to the first batch sample set;
The processing unit 202 is further configured to perform parameter adjustment operation on the original similar text matching model according to the first batch loss function, so as to obtain an intermediate similar text matching model;
The processing unit 202 is further configured to repeatedly obtain a second batch of sample sets corresponding to the target scene based on the intermediate similar text matching model, and perform vector conversion operation, ternary combination operation, loss calculation operation, and parameter adjustment operation to obtain the target similar text matching model.
Alternatively, in another embodiment of the training device for a similar text matching model provided in the embodiment of the present application based on the embodiment corresponding to fig. 7, the obtaining unit 201 may specifically be configured to:
Acquiring a target text data set corresponding to a target scene, wherein the target text data set at least comprises a first batch of positive examples samples and source text data corresponding to the target scene;
retrieving N first matching texts corresponding to the first batch of positive examples from the target text data set as N first batch of negative examples;
Calculating matching scores between the first batch of positive examples and each first batch of negative examples to obtain N first matching scores;
respectively carrying out normalization operation on the N first matching scores to obtain N sample matching scores;
And constructing the first batch sample set according to the first batch positive sample, the first batch negative sample and the sample matching score.
Optionally, in another embodiment of the training device for a similar text matching model provided in the embodiment of the present application based on the embodiment corresponding to fig. 7, the processing unit 202 may specifically be configured to:
Dividing the first batch of negative example sentence vectors according to the sample matching score to obtain a similar sentence vector set and a heterogeneous sentence vector set;
extracting any similar sentence vector from the similar sentence vector set to obtain similar sentence vectors of a first batch;
any heterogeneous sentence vector is extracted from the heterogeneous sentence vector set, and a first batch of heterogeneous sentence vectors are obtained.
Optionally, in another embodiment of the training device for a similar text matching model provided in the embodiment of the present application based on the embodiment corresponding to fig. 7, the processing unit 202 may specifically be configured to:
Respectively carrying out loss calculation operation on the first batch of normal sentence vectors, the first batch of similar sentence vectors and the first batch of heterogeneous sentence vectors to obtain loss functions corresponding to a plurality of first batch triples;
And carrying out weighted calculation operation on the loss functions corresponding to the first batch of triples to obtain the first batch of loss functions.
Optionally, in another embodiment of the training device for a similar text matching model provided in the embodiment of the present application based on the embodiment corresponding to fig. 7, the processing unit 202 may specifically be configured to:
Acquiring a second batch of sample sets corresponding to the target scene, and executing vector conversion operation, ternary structure construction operation and loss calculation operation according to the second batch of sample sets to obtain a second loss function;
And if the second loss function is smaller than the first threshold value, taking the current intermediate similar text matching model as a target similar text matching model.
Optionally, in another embodiment of the training device for a similar text matching model provided in the embodiment of the present application based on the embodiment corresponding to fig. 7, the processing unit 202 may specifically be configured to:
obtaining intermediate model parameters of an intermediate similar text matching model;
When a second batch of sample sets corresponding to the target scene are obtained, vector conversion operation, triple construction operation and parameter adjustment operation are carried out, a current similar text matching model is obtained, and the current similar text matching model comprises current model parameters;
And if the difference value between the intermediate model parameter and the current model parameter meets a second threshold value, taking the current intermediate similar text matching model as a target similar text matching model.
Alternatively, based on the embodiment corresponding to fig. 7, in another embodiment of the training device for a similar text matching model provided in the embodiment of the present application,
The obtaining unit 201 is further configured to receive a text to be matched;
The processing unit 202 is further configured to obtain a sentence vector to be matched and a plurality of original sentence vectors by respectively passing the text to be matched and the target text dataset through a target similar text matching model;
the processing unit 202 is further configured to calculate a similarity between the sentence vector to be matched and each original sentence vector, so as to obtain a plurality of similarity scores;
And the determining unit 203 is configured to determine a target similar text according to the plurality of similar scores, and push the target similar text to the target terminal device.
Another aspect of the present application provides another schematic diagram of a computer device, as shown in fig. 8, where fig. 8 is a schematic diagram of a computer device structure provided in an embodiment of the present application, where the computer device 300 may have a relatively large difference due to different configurations or performances, and may include one or more central processing units (central processing units, CPUs) 310 (e.g., one or more processors) and a memory 320, one or more storage mediums 330 (e.g., one or more mass storage devices) storing application programs 331 or data 332. Wherein memory 320 and storage medium 330 may be transitory or persistent storage. The program stored on the storage medium 330 may include one or more modules (not shown), each of which may include a series of instruction operations in the computer device 300. Still further, the central processor 310 may be configured to communicate with the storage medium 330 and execute a series of instruction operations in the storage medium 330 on the computer device 300.
The computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input/output interfaces 360, and/or one or more operating systems 333, such as a Windows ServerTM,Mac OS XTM,UnixTM,LinuxTM,FreeBSDTM, or the like.
The computer device 300 described above is also used to perform the steps in the corresponding embodiment as in fig. 2.
Another aspect of the application provides a computer readable storage medium having instructions stored therein which, when run on a computer, cause the computer to perform the steps of the method described in the embodiment shown in fig. 2.
Another aspect of the application provides a computer program product comprising instructions which, when run on a computer or processor, cause the computer or processor to perform the steps in the method described in the embodiment shown in fig. 2.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. The storage medium includes a U disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.