WO2022095376A1

Movatterモバイル変換

Info

Publication number: WO2022095376A1
Application number: PCT/CN2021/091198
Authority: WO
Inventors: 刘剑; 杨海钦; 姚晓远
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2020-11-06
Filing date: 2021-04-29
Publication date: 2022-05-12
Anticipated expiration: 2023-05-06
Also published as: CN112347787A

Abstract

An aspect-based sentiment classification method and apparatus, a device, and a readable storage medium. The method comprises: obtaining a text to be classified, and transforming keywords contained in said text into tokens to form a token sequence, the token sequence comprising T tokens (S101); inputting the token sequence into a preset token processing model to obtain a probability matrix, the probability matrix being a matrix of T columns×(T+1) rows (S102); configuring tokens of which values of the probability that the tokens belong to aspect terms are greater than a preset threshold as target tokens, and forming the values of the probability at second to (T+1)-th rows corresponding to the target tokens into a sentiment token probability sequence (S103); and on the basis of the sentiment token probability sequence, determining, by using preset Transformer model and classifier, a sentiment type corresponding to said text (S104). The method can achieve cross-field aspect-based sentiment analysis.

Description

Translated fromChinese

方面级别情感分类方法、装置、设备及可读存储介质Aspect-level sentiment classification method, apparatus, device, and readable storage medium

相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS

本申请申明享有2020年11月06日递交的申请号为202011227233.7、名称为“方面级别情感分类方法、装置、设备及可读存储介质”的中国专利申请的优先权，该中国专利申请的整体内容以参考的方式结合在本申请中。This application declares that it enjoys the priority of the Chinese patent application with the application number 202011227233.7 and the title of "Aspect-level emotion classification method, device, equipment and readable storage medium" filed on November 06, 2020. The overall content of the Chinese patent application Incorporated herein by reference.

技术领域technical field

本申请涉及人工智能技术领域，特别涉及一种方面级别情感分类方法、装置、设备及可读存储介质。The present application relates to the technical field of artificial intelligence, and in particular, to an aspect-level emotion classification method, apparatus, device, and readable storage medium.

背景技术Background technique

随着网络社交媒体的崛起，互联网上产生了大量的用户评论信息，这些用户评论信息中表达了各种各样的情感色彩和情感倾向性，通过对用户评论信息进行情感分析，可以了解大众舆论对于某一时事件或产品的看法。With the rise of online social media, a large amount of user comment information has been generated on the Internet. These user comment information expresses a variety of emotional colors and emotional tendencies. By analyzing user comment information, we can understand public opinion. An opinion about an event or product at a time.

情感分析是一个对带有情感色彩的主观性文本进行分析、处理、归纳和推理的过程，为了充分的获取某文本中各个方面的情感倾向，提出方面级别的情感分析，将分析的粒度细化到方面的级别；发明人意识到，现有技术中大多通过监督学习的方式训练神经网络模型，以获取用于方面级别情感分析的情感分类器，不可避免的，以监督学习的方式训练神经网络模型的过程中需要大量的被标记的训练样本，缺乏被标记的训练样本成为获取情感分类器的主要障碍。Sentiment analysis is a process of analyzing, processing, summarizing and reasoning on subjective texts with emotional color. In order to fully obtain the emotional tendencies of various aspects in a text, an aspect-level sentiment analysis is proposed to refine the granularity of the analysis. to the aspect level; the inventor realizes that in the prior art, most of the neural network models are trained by means of supervised learning to obtain sentiment classifiers for sentiment analysis at the aspect level, and inevitably, the neural network is trained by means of supervised learning. A large number of labeled training samples are required in the process of modeling, and the lack of labeled training samples has become the main obstacle to obtaining sentiment classifiers.

因此，如何避免获取情感分类器无足够被标记的训练样本的障碍，如何利用预训练模型的强大特征表征能力，实现跨领域的方面级别情感分析，是本领域技术人员需要解决的问题。Therefore, how to avoid the obstacle of obtaining enough labeled training samples for the sentiment classifier, and how to utilize the powerful feature representation ability of the pre-trained model to achieve cross-domain aspect-level sentiment analysis are problems that those skilled in the art need to solve.

发明内容SUMMARY OF THE INVENTION

本申请的目的在于提供一种方面级别情感分类方法、装置、设备及可读存储介质，能够实现跨领域的方面级别情感分析。The purpose of the present application is to provide an aspect-level sentiment classification method, apparatus, device, and readable storage medium, which can realize cross-domain aspect-level sentiment analysis.

根据本申请的一个方面，提供了一种方面级别情感分类方法，所述方法包括：According to one aspect of the present application, an aspect-level sentiment classification method is provided, the method comprising:

获取待分类文本，并将包含在所述待分类文本中的关键词转化为令牌，以形成令牌序列；其中，所述令牌序列包括T个令牌；Obtain the text to be classified, and convert the keywords contained in the text to be classified into tokens to form a token sequence; wherein, the token sequence includes T tokens;

将所述令牌序列输入到预设的令牌处理模型中，以得到概率矩阵；其中，所述概率矩阵为T列、(T+1)行的矩阵，所述概率矩阵的每一列表征一种令牌，所述概率矩阵的第一行表征各个令牌属于方面术语的概率值、第二行至第(T+1)行表征每个令牌属于与对应方面术语相映射的情感令牌的概率值；Input the token sequence into a preset token processing model to obtain a probability matrix; wherein, the probability matrix is a matrix of T columns and (T+1) rows, and each column of the probability matrix represents A token, the first row of the probability matrix represents the probability value of each token belonging to the aspect term, and the second row to (T+1)th row represent that each token belongs to the emotional order mapped to the corresponding aspect term. the probability value of the card;

将所述属于方面术语的概率值大于预设阈值的令牌设置为目标令牌，并将与所述目标令牌对应的第二行至第(T+1)行的概率值形成情感令牌概率序列；Set the token whose probability value belonging to the aspect term is greater than the preset threshold as the target token, and form the emotion token from the probability values of the second row to the (T+1)th row corresponding to the target token probability sequence;

基于所述情感令牌概率序列，利用预设的Transformer模型和分类器，确定出与所述待分类文本对应的情感类型。Based on the emotion token probability sequence, a preset Transformer model and a classifier are used to determine the emotion type corresponding to the text to be classified.

为了实现上述目的，本申请还提供一种方面级别情感分类装置，所述装置包括：In order to achieve the above object, the present application also provides an aspect-level emotion classification device, the device comprising:

获取模块，用于获取待分类文本，并将包含在所述待分类文本中的关键词转化为令牌，以形成令牌序列；其中，所述令牌序列包括T个令牌；an acquisition module, configured to acquire the text to be classified, and convert the keywords contained in the text to be classified into tokens to form a token sequence; wherein the token sequence includes T tokens;

输入模块，用于将所述令牌序列输入到预设的令牌处理模型中，以得到概率矩阵；其中，所述概率矩阵为T列、(T+1)行的矩阵，所述概率矩阵的每一列表征一种令牌，所述概率矩阵的第一行表征各个令牌属于方面术语的概率值、第二行至第(T+1)行表征每个令牌属于与对应方面术语相映射的情感令牌的概率值；an input module, configured to input the token sequence into a preset token processing model to obtain a probability matrix; wherein the probability matrix is a matrix of T columns and (T+1) rows, and the probability matrix Each column of the probability matrix represents a token, the first row of the probability matrix represents the probability value of each token belonging to the aspect term, the second row to the (T+1)th row represent that each token belongs to the corresponding aspect term The probability value of the mapped sentiment token;

处理模块，用于将所述属于方面术语的概率值大于预设阈值的令牌设置为目标令牌，并将与所述目标令牌对应的第二行至第(T+1)行的概率值形成情感令牌概率序列；The processing module is configured to set the token whose probability value belonging to the aspect term is greater than the preset threshold as the target token, and set the probability of the second row to the (T+1)th row corresponding to the target token values form a sequence of sentiment token probabilities;

确定模块，基于所述情感令牌概率序列，利用预设的Transformer模型和分类器，确定出与所述待分类文本对应的情感类型。The determining module determines the emotion type corresponding to the text to be classified by using a preset Transformer model and a classifier based on the emotion token probability sequence.

为了实现上述目的，本申请还提供一种计算机设备，该计算机设备具体包括：存储器、处理器以及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现以下步骤：In order to achieve the above object, the present application also provides a computer device, the computer device specifically includes: a memory, a processor, and a computer program stored in the memory and running on the processor, the processor executes the computer program. The following steps are implemented when the computer program is described:

为了实现上述目的，本申请还提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现以下步骤：In order to achieve the above purpose, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program implements the following steps when executed by a processor:

本申请提供的方面级别情感分类方法、装置、设备及可读存储介质，通过利用预训练模型强大的特征表征能力，提出了一种基于预训练模型的跨领域方面级别情感分析方法，且预训练模型的结构比较简单，能够广泛应用到其他相似任务中；本申请通过设置阈值，使得大于阈值的方面术语才能参与后续的方面级别情感分析的任务，从而有效减少了计算量，能够进一步提升模型的性能。The aspect-level sentiment classification method, device, device and readable storage medium provided by this application, by using the powerful feature representation capability of the pre-training model, propose a cross-domain aspect-level sentiment analysis method based on the pre-training model, and the pre-training The structure of the model is relatively simple and can be widely applied to other similar tasks; this application sets a threshold so that aspect terms larger than the threshold can participate in the subsequent aspect-level sentiment analysis tasks, thereby effectively reducing the amount of computation and further improving the model’s performance. performance.

附图说明Description of drawings

通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本申请的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustrating preferred embodiments only and are not to be considered limiting of the application. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

图1为实施例一提供的方面级别情感分类方法的一种可选的流程示意图；FIG. 1 is an optional schematic flowchart of the aspect-level sentiment classification method provided in Embodiment 1;

图2为实施例一提供的情感分类模型的示意图；2 is a schematic diagram of an emotion classification model provided by Embodiment 1;

图3为实施例三提供的方面级别情感分类装置的一种可选的组成结构示意图；3 is a schematic diagram of an optional composition structure of the aspect-level emotion classification device provided in Embodiment 3;

图4为实施例四提供的计算机设备的一种可选的硬件架构示意图。FIG. 4 is a schematic diagram of an optional hardware architecture of the computer device provided in the fourth embodiment.

具体实施方式Detailed ways

为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

实施例一Example 1

本申请实施例提供了一种方面级别情感分类方法，如图1所示，该方法具体包括以下步骤：The embodiment of the present application provides an aspect-level emotion classification method, as shown in FIG. 1 , the method specifically includes the following steps:

步骤S101：获取待分类文本，并将包含在所述待分类文本中的关键词转化为令牌，以形成令牌序列；其中，所述令牌序列包括T个令牌。Step S101: Acquire the text to be classified, and convert the keywords contained in the text to be classified into tokens to form a token sequence; wherein the token sequence includes T tokens.

具体的，步骤S101，包括：Specifically, step S101 includes:

步骤A1：对所述待分类文本进行分词处理，以得到包含在所述待分类文本中的T个关键词；Step A1: performing word segmentation processing on the text to be classified to obtain T keywords contained in the text to be classified;

步骤A2：分别对每个关键词进行编码，以得到每个关键词的令牌；Step A2: encode each keyword separately to obtain the token of each keyword;

步骤A3：将所有令牌组成所述令牌序列。Step A3: Composing all tokens into the token sequence.

例如，获取用户输入的待分类文本：“这家餐馆的服务很好，并且饭菜也很好”；对所述待分类文本进行分词处理，以得到关键词：这家、餐馆、的、服务、很好、并且、饭菜、也、很好；利用One-Hot编码分别将每个关键词编码为对应的令牌，从而根据所有关键词的令牌形成令牌序列X＝{x₁,x₂,…,x_T}。For example, obtain the text to be classified entered by the user: "The service of this restaurant is very good, and the food is also very good"; perform word segmentation on the text to be classified to obtain the keywords: this, restaurant, of, service, Very good, and, meal, also, very good; use One-Hot encoding to encode each keyword into a corresponding token, so as to form a token sequence X={x₁ , x₂ according to the tokens of all keywords ,…,x_T }.

步骤S102：将所述令牌序列输入到预设的令牌处理模型中，以得到概率矩阵；其中，所述概率矩阵为T列、(T+1)行的矩阵，所述概率矩阵的每一列表征一种令牌，所述概率矩阵的第一行表征各个令牌属于方面术语的概率值、第二行至第(T+1)行表征每个令牌属于与对应方面术语相映射的情感令牌的概率值。Step S102: Input the token sequence into a preset token processing model to obtain a probability matrix; wherein, the probability matrix is a matrix of T columns and (T+1) rows, and each probability matrix is One column represents a token, the first row of the probability matrix represents the probability value of each token belonging to the aspect term, the second row to (T+1)th row represent that each token belongs to the mapping with the corresponding aspect term The probability value of the sentiment token.

其中，所述令牌处理模型包括：BERT模型、全连接层和softmax函数。Wherein, the token processing model includes: a BERT model, a fully connected layer and a softmax function.

具体的，步骤S102，包括：Specifically, step S102 includes:

步骤B1：将所述令牌序列输入到所述令牌处理模型中的BERT(Bidirectional Encoder Representation from Transformers)模型中，得到每个令牌对应的融合了前后令牌信息的特征表征，并将所有所述特征表征组成特征表征序列；Step B1: Input the token sequence into the BERT (Bidirectional Encoder Representation from Transformers) model in the token processing model, obtain the corresponding feature representation of each token that integrates the information of the front and back tokens, and convert all the tokens into the BERT (Bidirectional Encoder Representation from Transformers) model The feature representation constitutes a feature representation sequence;

例如，将令牌序列X＝{x₁,x₂,…,x_T}输入至由BERT模型构成的预训练词嵌入层中，从而输出了一个融合了上下文的特征表征序列

其中，

为令牌x_i对应的特征表征，i∈[1,T]。本实施例引入BERT模块作为预训练词嵌入层，该BERT模块是一个通过大规模数据集进行预训练获得的能够进行上下文感知的预训练模型。For example, the token sequence X={x₁ ,x₂ ,...,x_T } is input into the pre-trained word embedding layer composed of the BERT model, thereby outputting a context-fused feature representation sequence

in,

is the feature representation corresponding to token x_i , i∈[1,T]. This embodiment introduces a BERT module as a pre-training word embedding layer, and the BERT module is a pre-training model capable of context awareness obtained by pre-training on a large-scale data set.

步骤B2：将所述特征表征序列输入到所述令牌处理模型中的全连接层中，并通过softmax函数对所述全连接层的输出进行归一化处理，以得到所述概率矩阵；Step B2: Input the feature representation sequence into the fully connected layer in the token processing model, and normalize the output of the fully connected layer through a softmax function to obtain the probability matrix;

例如，将上述特征表征序列H^L输入至特定于任务的神经体系结构层，优选的，该神经体系结构层为全连接层；此外，该神经体系结构层还包括softmax函数模型，用于进行概率的归一化操作，即输出每一个令牌x_i对应的T+1个维度，从而形成一个T×(T+1)的概率矩阵；其中，概率矩阵的第一行表示每个令牌x_i属于方面术语的概率；针对目标令牌x_i，概率矩阵的第二行至第T+1行则分别表示令牌x₁,x₂,…,x_T属于与目标令牌x_i的方面术语对应的情感令牌的概率，从而利用上述T+1个维度对令牌序列中每个令牌x_i进行概率标记。For example, the above feature representation sequence^HL is input to a task-specific neural architecture layer, preferably, the neural architecture layer is a fully connected layer; in addition, the neural architecture layer also includes a softmax function model for probabilistic The normalization operation of , that is, output T+1 dimensions corresponding to each token x_i , thus forming a T×(T+1) probability matrix; where the first row of the probability matrix represents each token x The probability that_i belongs to the aspect term; for the target token x_i , the second row to the T+1th row of the probability matrix indicates that the tokens x₁ , x₂ ,...,x_T belong to the aspect with the target token x_i , respectively The probability of the emotional token corresponding to the term, so that each token_xi in the token sequence is probabilistically marked using the above T+1 dimensions.

步骤S103：将所述属于方面术语的概率值大于预设阈值的令牌设置为目标令牌，并将与所述目标令牌对应的第二行至第(T+1)行的概率值形成情感令牌概率序列。Step S103: Set the token whose probability value belonging to the aspect term is greater than the preset threshold as the target token, and form the probability value of the second row to the (T+1)th row corresponding to the target token. Sentiment token probability sequence.

在令牌序列中，筛选出属于方面术语的概率值大于预设阈值的目标令牌，若任一令牌的概率标记中显示其属于方面术语的概率大于预设阈值时，则证明该令牌为方面术语；例如，一个令牌序列为{这家，餐馆，的，服务，很好，并且，饭菜，也，很好}，预设的概率阈值为0.5，而在当前令牌序列中，只有令牌“服务”和“饭菜”属于方面术语的概率大于预设概率阈值0.5，那么输出的目标令牌所对应的方面术语为“服务”和“饭菜”。In the token sequence, filter out target tokens whose probability value belonging to an aspect term is greater than a preset threshold. If the probability of any token belonging to an aspect term is greater than the preset threshold, it is proved that the token has a probability of belonging to an aspect term. is an aspect term; for example, a token sequence of {this, restaurant, of, service, good, and, meal, also, good}, the preset probability threshold is 0.5, while in the current token sequence, Only the probability that the tokens "service" and "meal" belong to aspect terms is greater than the preset probability threshold of 0.5, then the aspect terms corresponding to the output target token are "service" and "meal".

步骤S104：基于所述情感令牌概率序列，利用预设的Transformer模型和分类器，确定出与所述待分类文本对应的情感类型。Step S104: Based on the emotion token probability sequence, using a preset Transformer model and a classifier, determine the emotion type corresponding to the text to be classified.

具体的，步骤S104，包括：Specifically, step S104 includes:

步骤C1：将所述特征表征序列与所述情感令牌概率序列进行向量级别的元素相乘，以得到表征结果；Step C1: Multiplying the feature representation sequence and the emotion token probability sequence by elements at the vector level to obtain a representation result;

步骤C2：将所述表征结果依次输入所述Transformer模型和分类器中，以得到与所述待分类文本对应的情感类型。Step C2: Input the characterization results into the Transformer model and the classifier in turn to obtain the emotion type corresponding to the text to be classified.

在从令牌序列中确定出目标令牌之后，根据该目标令牌的情感令牌概率序列去做情感分析任务。具体的，将融合了上下文的特征表征序列

与目标令牌的情感令牌概率序列进行向量级别的元素相乘运算，获取最终的表征结果，将该表征结果利用Transformer模型进行特征转换，将最终的表征结果转换为融合句子语义的目标表征序列，进一步的，利用全连接层将转换后的目标表征序列进行整合，获得能够表征所述令牌序列的一个目标表征，最后，将目标表征输入至分类器，筛选获得该令牌序列对应的情感类别。After the target token is determined from the token sequence, the sentiment analysis task is performed according to the sentiment token probability sequence of the target token. Specifically, the feature representation sequence incorporating the context is

Perform element multiplication operation at the vector level with the emotional token probability sequence of the target token to obtain the final representation result, use the Transformer model to perform feature transformation, and convert the final representation result into the target representation sequence fused with sentence semantics , further, using the fully connected layer to integrate the converted target representation sequence to obtain a target representation that can characterize the token sequence, and finally, input the target representation to the classifier, and filter to obtain the emotion corresponding to the token sequence category.

进一步的，所述方法还包括：Further, the method also includes:

步骤D1：获取样本文本集；其中，所述样本文本集中的每个样本文本均标注有对应的方面术语和情感类型；Step D1: obtaining a sample text set; wherein, each sample text in the sample text set is marked with a corresponding aspect term and emotion type;

步骤D2：基于所述样本文本集对初始神经网络模型进行训练，以对所述初始神经网络模型中的各个参数进行修正，得到情感分类模型；其中，所述情感分类模型包括：令牌处理模型、Transformer模型和分类器。Step D2: Train the initial neural network model based on the sample text set, so as to modify each parameter in the initial neural network model to obtain a sentiment classification model; wherein, the sentiment classification model includes: a token processing model , Transformer models and classifiers.

优选的，如图2所示，为情感分类模型的示意图，由图2可知，根据待分类文本形成的令牌序列X先输入到令牌处理模型中的BERT模型中，以得到融合了前后令牌信息的特征表征序列H^L；再将所述特征表征序列H^L先后输入到令牌处理模型中的全连接层(FC Layer)和softmax函数(Softmax Layer)中，以得到概率矩阵，并根据所述概率矩阵得到情感令牌概率序列P；再将所述特征表征序列H^L与所述情感令牌概率序列P进行向量级别的元素相乘，以得到表征结果；最后将所述表征结果先后输入到Transformer模型(Transformer Block)和分类器(Classifier Layer)中，以得到与所述待分类文本对应的情感类型。其中，BERT模型包括：Token Embedding(令牌嵌入)、Segment Embedding(分片嵌入)、Position Embedding(位置嵌入)、以及L个Transformer Block。Preferably, as shown in FIG. 2, it is a schematic diagram of the sentiment classification model. It can be seen from FIG. 2 that the token sequence X formed according to the text to be classified is firstly input into the BERT model in the token processing model, so as to obtain a fusion of the front and rear commands. The feature representation sequence HL of the card information; Then the feature representation sequence^HL is successively input into the fully connected layer (FC Layer) and the^softmax function (Softmax Layer) in the token processing model, to obtain the probability matrix, and according to The probability matrix obtains the emotional token probability sequence P; then the feature representation sequence^HL and the emotional token probability sequence P are multiplied by the elements of the vector level to obtain the characterization result; finally, the characterization results are sequentially Input into the Transformer model (Transformer Block) and the classifier (Classifier Layer) to obtain the emotion type corresponding to the text to be classified. The BERT model includes: Token Embedding, Segment Embedding, Position Embedding, and L Transformer Blocks.

实施例二Embodiment 2

本申请实施例提供了一种方面级别情感分类方法，该方法具体包括以下步骤：The embodiment of the present application provides an aspect-level emotion classification method, and the method specifically includes the following steps:

步骤S1，获取样本文本集，并利用所述样本文本集对神经网络模型进行训练。In step S1, a sample text set is obtained, and the neural network model is trained by using the sample text set.

在本实施例中，神经网络模型由预训练词嵌入层和神经体系结构层组成；首先将样本文本拆分为若干令牌，即若干词语，并将若干令牌组成令牌序列X＝{x₁,x₂,…,x_T}输入至预训练词嵌入层；其中，本实施例引入BERT模块作为预训练词嵌入层，该BERT模块是一个通过大规模数据集进行预训练获得的能够进行上下文感知的预训练模型。In this embodiment, the neural network model consists of a pre-trained word embedding layer and a neural architecture layer; first, the sample text is split into several tokens, that is, several words, and several tokens are formed into a token sequence X={x₁ ,x₂ ,...,x_T } is input to the pre-training word embedding layer; wherein, this embodiment introduces the BERT module as the pre-training word embedding layer, and the BERT module is a pre-training obtained through large-scale data sets. Context-aware pretrained models.

进一步的，在BERT模块接收了令牌序列X后，输出一个融合了上下文的特征表征序列

其中，

为令牌x_i对应的特征表征，i∈[1,T]。Further, after the BERT module receives the token sequence X, it outputs a feature representation sequence fused with context

in,

is the feature representation corresponding to token x_i , i∈[1,T].

进一步的，将特征表征序列H^L输入至特定于任务的神经体系结构层，该层包括softmax函数模型，用于进行概率的归一化操作，即输出每一个令牌x_i对应的T+1个维度；其中，第一个维度表示该令牌x_i属于方面术语的概率，第二至第T+1个维度则表示令牌序列X中的每一个令牌x_i属于该方面术语对应的情感令牌的概率。Further, the feature representation sequence HL is input to the task-specific neural architecture layer, which includes a^softmax function model for normalizing the probability, that is, outputting the T+1 corresponding to each token_xi . The first dimension represents the probability that the token_xi belongs to the aspect term, and the second to T+1th dimensions represent that each token_xi in the token sequence X belongs to the aspect term corresponding to the Probability of sentiment tokens.

进一步的，冻结神经网络模型的全部参数，完成用于方面级别情感分析的神经网络模型的微调，以得到令牌处理模型。Further, all parameters of the neural network model are frozen, and fine-tuning of the neural network model for aspect-level sentiment analysis is completed to obtain a token processing model.

步骤S2，将所述令牌序列输入至所述令牌处理模型中，生成每个令牌对应的概率标记；Step S2, inputting the token sequence into the token processing model, and generating a probability mark corresponding to each token;

将令牌序列X＝{x₁,x₂,…,x_T}输入至步骤S1微调后的令牌处理模型中，利用BERT模型，输出一个融合了上下文的特征表征序列

其中，

为令牌x_i对应的特征表征，i∈[1,T]；将特征表征序列H^L输入至特定于任务的神经体系结构层，该层包括softmax函数模型，用于进行概率的归一化操作，即输出每一个令牌xⁱ对应的T+1个维度，即一个T×(T+1)的矩阵；其中，矩阵的第一行表示每个令牌x_i属于方面术语的概率，第二行至第T+1行则分别表示令牌序列X中的每一个令牌x_i属于该方面术语对应的情感令牌的概率，并利用上述T+1个维度对令牌序列中每个令牌x_i进行概率标记。Input the token sequence X={x₁ , x₂ ,...,x_T } into the token processing model fine-tuned in step S1, and use the BERT model to output a feature representation sequence fused with context

in,

is the feature representation corresponding to the token x_i , i∈[1,T]; the feature representation sequence^HL is input to the task-specific neural architecture layer, which includes a softmax function model for probability normalization operation, that is, output T+1 dimensions corresponding to each token^xi , that is, a T×(T+1) matrix; where the first row of the matrix represents the probability that each token_xi belongs to the aspect term, The second row to the T+1th row respectively represent the probability that each token_xi in the token sequence X belongs to the emotional token corresponding to the term in this aspect, and use the above T+1 dimensions to evaluate each token in the token sequence. The tokens_xi are marked with probability.

步骤S3，在所述令牌序列中筛选出目标令牌，对所述目标令牌进行情感分析，完成对所述令牌序列对应领域的Transformer模型和分类器的训练。In step S3, a target token is selected from the token sequence, and sentiment analysis is performed on the target token to complete the training of the Transformer model and the classifier in the field corresponding to the token sequence.

预先设定一个概率阈值，在令牌序列中，筛选出属于方面术语的概率大于该预设的概率阈值的目标令牌，若任一令牌的概率标记中显示其属于方面术语的概率大于预设阈值时，则证明该令牌为方面术语；例如，一个令牌序列为{这家，餐馆，的，服务，很好，并且，饭菜，也，很好}，则其对应的方面术语令牌为“服务”和“饭菜”，自然本实施例中神经网络模型输出的该令牌对应的属于方面术语的概率大于预设概率阈值。A probability threshold is preset, and in the token sequence, the target token whose probability of belonging to an aspect term is greater than the preset probability threshold is screened out. When the threshold is set, it is proved that the token is an aspect term; for example, a token sequence of {this, restaurant, , service, good, and, meal, also, good}, its corresponding aspect term makes The tokens are "service" and "meal". Naturally, the probability that the token corresponding to the token belongs to the aspect term output by the neural network model in this embodiment is greater than the preset probability threshold.

进一步的，在选择完目标令牌后，根据该目标令牌的概率标记中每个令牌x_i对应的属于该方面术语的情感令牌的概率进行运算。具体的，将步骤S2中获得融合了上下文的特征表征序列

与该目标令牌的概率标记中每个令牌x_i对应的属于该方面术语的情感令牌的概率进行向量级别的元素相乘运算，获取最终的特征表征，将该特征表征序列利用Transformer模块进行特征转换，将最终的特征表征序列转换为融合句子语义的特征表征序列，进一步的，利用全连接层将特征表征序列进行整合，获得能够表征令牌序列的一个特征表征，最后，将特征表征输入至分类器，筛选获得该令牌序列对应的情感分类。Further, after the target token is selected, calculation is performed according to the probability of the emotion token belonging to the aspect term corresponding to each token_xi in the probability token of the target token. Specifically, the feature representation sequence fused with the context obtained in step S2

The probability of the sentiment token belonging to the aspect term corresponding to each token_xi in the probability token of the target token is multiplied by the elements of the vector level to obtain the final feature representation, and the feature representation sequence is used by the Transformer module. Perform feature transformation to convert the final feature representation sequence into a feature representation sequence fused with sentence semantics. Further, use the fully connected layer to integrate the feature representation sequence to obtain a feature representation that can characterize the token sequence. Finally, the feature representation Input to the classifier, and filter to obtain the sentiment classification corresponding to the token sequence.

进一步的，根据上述步骤，完成了针对该令牌序列对应领域的方面级别情感分析的情感分类模型的训练。Further, according to the above steps, the training of the sentiment classification model for the aspect-level sentiment analysis in the field corresponding to the token sequence is completed.

步骤S4，获取待分类文本，利用所述情感分类模型进行情感分类，筛选所述待分类文本对应的情感分类类别。In step S4, the text to be classified is acquired, the sentiment classification model is used to perform sentiment classification, and the sentiment classification category corresponding to the text to be classified is screened.

获取用户输入的待分类文本，并将该待分类文本拆分为由若干令牌构成的令牌序列X＝{x₁,x₂,…,x_T}；例如，客户输入的待分类文本为“这家餐馆的服务很好，并且饭菜也很好”，则将该待分类文本拆分为令牌序列{这家，餐馆，的，服务，很好，并且，饭菜，也，很好}。Obtain the text to be classified entered by the user, and split the text to be classified into a token sequence X={x₁ , x₂ ,..., x_T } composed of several tokens; for example, the text to be classified entered by the customer is "The service at this restaurant is good, and the meal is good", then splits the text to be classified into a sequence of tokens {this, restaurant, of, service, good, and, meal, also, good} .

进一步的，将上述令牌序列输入至由BERT模型构成的令牌处理模型中，输出一个融合了上下文的特征表征序列

其中，

为令牌x_i对应的特征表征，i∈[1,T]。Further, the above token sequence is input into the token processing model composed of the BERT model, and a feature representation sequence that integrates the context is output.

in,

is the feature representation corresponding to token x_i , i∈[1,T].

进一步的，将上述特征表征序列H^L输入至特定于任务的神经体系结构层，该层包括softmax函数模型，用于进行概率的归一化操作，即输出每一个令牌x_i对应的T+1个维度，即一个T×(T+1)的矩阵；其中，矩阵的第一行表示表示每个令牌x_i属于方面术语的概率，第二行至第T+1行则分别表示令牌序列X中的每一个令牌x_i属于该方面术语对应的情感令牌的概率，并利用上述T+1个维度对令牌序列中每个令牌x_i进行概率标记。Further, the above-mentioned feature representation sequence HL is input to the task-specific neural architecture layer, which includes a^softmax function model for normalizing the probability, that is, outputting the T+ corresponding to each token_xi . 1 dimension, that is, a T×(T+1) matrix; the first row of the matrix represents the probability that each token x_i belongs to the aspect term, and the second to T+1th rows represent the order The probability that each token_xi in the card sequence X belongs to the emotion token corresponding to the term in this aspect, and the above T+1 dimension is used to mark each token_xi in the token sequence with probability.

进一步的，在令牌序列中，筛选出属于方面术语的概率大于该预设的概率阈值的目标令牌，若该令牌的概率标记中显示其属于方面术语的概率大于预设阈值时，则证明该令牌为方面术语；例如，一个令牌序列为{这家，餐馆，的，服务，很好，并且，饭菜，也，很好}，预设的概率阈值为0.5，而在当前令牌序列中，只有令牌“服务”和“饭菜”属于方面术语的概率大于预设概率阈值0.5，那么输出的方面术语令牌为“服务”和“饭菜”。Further, in the token sequence, a target token whose probability of belonging to an aspect term is greater than the preset probability threshold is selected, and if the probability mark of the token shows that the probability of belonging to an aspect term is greater than the preset threshold, then Prove that the token is an aspect term; for example, a token sequence of {this, restaurant, of, service, good, and, meal, also, good} with a preset probability threshold of 0.5, and in the current order In the card sequence, only the probability of tokens "service" and "meal" belonging to aspect terms is greater than the preset probability threshold of 0.5, then the output aspect term tokens are "service" and "meal".

进一步的，在选择完目标令牌后，根据该目标令牌的概率标记中令牌序列X中的每一个令牌x_i对应的属于该方面术语的情感令牌的概率进行运算。具体的，将获得融合了上下文的特征表征序列

与该目标令牌的概率标记中每个令牌x_i对应的属于该方面术语的情感令牌的概率进行向量级别的元素相乘运算，获取最终的特征表征，将该特征表征序列利用Transformer模型进行特征转换，将最终的特征表征序列转换为融合句子语义的特征表征序列，进一步的，利用全连接层将转换后的特征表征序列进行整合，获得能够表征令牌序列的一个特征表征，最后，将特征表征输入至分类器，筛选获得该令牌序列对应的情感分类类别。Further, after the target token is selected, calculation is performed according to the probability of the emotion token belonging to the aspect term corresponding to each token_xi in the token sequence X in the probability token of the target token. Specifically, the feature representation sequence fused with the context will be obtained

The probability of the emotional token belonging to the aspect term corresponding to each token_xi in the probability token of the target token is multiplied by the elements of the vector level to obtain the final feature representation, and the feature representation sequence is used by the Transformer model. Perform feature transformation to convert the final feature representation sequence into a feature representation sequence that fuses sentence semantics. Further, the fully connected layer is used to integrate the converted feature representation sequence to obtain a feature representation that can characterize the token sequence. Finally, The feature representation is input to the classifier, and the sentiment classification category corresponding to the token sequence is obtained by screening.

实施例三Embodiment 3

本申请实施例提供了一种方面级别情感分类装置，如图3所示，该装置具体包括以下组成部分：The embodiment of the present application provides an aspect-level emotion classification device, as shown in FIG. 3 , the device specifically includes the following components:

获取模块301，用于获取待分类文本，并将包含在所述待分类文本中的关键词转化为令牌，以形成令牌序列；其中，所述令牌序列包括T个令牌；The obtaining module 301 is configured to obtain the text to be classified, and convert the keywords contained in the text to be classified into tokens to form a token sequence; wherein, the token sequence includes T tokens;

输入模块302，用于将所述令牌序列输入到预设的令牌处理模型中，以得到概率矩阵；其中，所述概率矩阵为T列、(T+1)行的矩阵，所述概率矩阵的每一列表征一种令牌，所述概率矩阵的第一行表征各个令牌属于方面术语的概率值、第二行至第(T+1)行表征每个令牌属于与对应方面术语相映射的情感令牌的概率值；The input module 302 is configured to input the token sequence into a preset token processing model to obtain a probability matrix; wherein, the probability matrix is a matrix of T columns and (T+1) rows, and the probability Each column of the matrix represents a token, the first row of the probability matrix represents the probability value of each token belonging to the aspect term, and the second to (T+1)th rows represent that each token belongs to the corresponding aspect. The probability value of the sentiment token to which the term is mapped;

处理模块303，用于将所述属于方面术语的概率值大于预设阈值的令牌设置为目标令牌，并将与所述目标令牌对应的第二行至第(T+1)行的概率值形成情感令牌概率序列；The processing module 303 is configured to set the token whose probability value belonging to the aspect term is greater than the preset threshold as the target token, and set the tokens corresponding to the target token in the second row to the (T+1)th row. The probability values form the emotional token probability sequence;

确定模块304，用于基于所述情感令牌概率序列，利用预设的Transformer模型和分类器，确定出与所述待分类文本对应的情感类型。A determination module 304 is configured to determine an emotion type corresponding to the text to be classified by using a preset Transformer model and a classifier based on the emotion token probability sequence.

具体的，获取模块301，用于：Specifically, the acquisition module 301 is used for:

对所述待分类文本进行分词处理，以得到包含在所述待分类文本中的T个关键词；Perform word segmentation processing on the text to be classified to obtain T keywords contained in the text to be classified;

分别对每个关键词进行编码，以得到每个关键词的令牌；Encode each keyword separately to get the token of each keyword;

将所有令牌组成所述令牌序列。All tokens are formed into the token sequence.

进一步的，输入模块302，用于：Further, the input module 302 is used for:

将所述令牌序列输入到所述令牌处理模型中的BERT模型中，得到每个令牌对应的融合了前后令牌信息的特征表征，并将所有所述特征表征组成特征表征序列；Inputting the token sequence into the BERT model in the token processing model, to obtain a feature representation corresponding to each token that fuses the information of the preceding and following tokens, and combining all the feature representations into a feature representation sequence;

将所述特征表征序列输入到所述令牌处理模型中的全连接层中，并通过softmax函数对所述全连接层的输出进行归一化处理，以得到所述概率矩阵。The feature representation sequence is input into the fully connected layer in the token processing model, and the output of the fully connected layer is normalized by a softmax function to obtain the probability matrix.

进一步的，确定模块304，用于：Further, the determining module 304 is used for:

将所述特征表征序列与所述情感令牌概率序列进行向量级别的元素相乘，以得到表征结果；Multiplying the feature representation sequence and the emotion token probability sequence by elements at the vector level to obtain a representation result;

将所述表征结果依次输入所述Transformer模型和分类器中，以得到与所述待分类文本对应的情感类型。The characterization results are sequentially input into the Transformer model and the classifier to obtain the emotion type corresponding to the text to be classified.

更进一步的，所述装置还包括：Further, the device also includes:

训练模块，用于training module for

获取样本文本集；其中，所述样本文本集中的每个样本文本均标注有对应的方面术语和情感类型；Obtaining a sample text set; wherein, each sample text in the sample text set is marked with a corresponding aspect term and sentiment type;

基于所述样本文本集对初始神经网络模型进行训练，以对所述初始神经网络模型中的各个参数进行修正，得到情感分类模型；其中，所述情感分类模型包括：令牌处理模型、Transformer模型和分类器。The initial neural network model is trained based on the sample text set, so as to modify each parameter in the initial neural network model to obtain a sentiment classification model; wherein, the sentiment classification model includes: a token processing model, a Transformer model and classifier.

实施例四Embodiment 4

本实施例还提供一种计算机设备，如可以执行程序的智能手机、平板电脑、笔记本电脑、台式计算机、机架式服务器、刀片式服务器、塔式服务器或机柜式服务器(包括独立的服务器，或者多个服务器所组成的服务器集群)等。如图4所示，本实施例的计算机设备40至少包括但不限于：可通过系统总线相互通信连接的存储器401、处理器402。需要指出的是，图4仅示出了具有组件401-402的计算机设备40，但是应理解的是，并不要求实施所有示出的组件，可以替代的实施更多或者更少的组件。This embodiment also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including independent servers, or A server cluster composed of multiple servers), etc. As shown in FIG. 4 , the computer device 40 in this embodiment at least includes but is not limited to: amemory 401 and a processor 402 that can be communicatively connected to each other through a system bus. It should be noted that FIG. 4 only shows the computer device 40 having components 401-402, but it should be understood that implementation of all of the illustrated components is not required, and more or fewer components may be implemented instead.

本实施例中，存储器401(即可读存储介质)包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等。在一些实施例中，存储器401可以是计算机设备40的内部存储单元，例如该计算机设备40的硬盘或内存。在另一些实施例中，存储器401也可以是计算机设备40的外部存储设备，例如该计算机设备40上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。当然，存储器401还可以既包括计算机设备40的内部存储单元也包括其外部存储设备。在本实施例中，存储器401通常用于存储安装于计算机设备40的操作系统和各类应用软件。此外，存储器401还可以用于暂时地存储已经输出或者将要输出的各类数据。In this embodiment, the memory 401 (that is, a readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (eg, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disk, etc. In some embodiments, thememory 401 may be an internal storage unit of the computer device 40 , such as a hard disk or a memory of the computer device 40 . In other embodiments, thememory 401 may also be an external storage device of the computer device 40, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Of course, thememory 401 may also include both the internal storage unit of the computer device 40 and its external storage device. In this embodiment, thememory 401 is generally used to store the operating system and various application software installed on the computer device 40 . In addition, thememory 401 can also be used to temporarily store various types of data that have been output or will be output.

处理器402在一些实施例中可以是中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器、或其他数据处理芯片。该处理器402通常用于控制计算机设备40的总体操作。The processor 402 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 402 is typically used to control the overall operation of the computer device 40 .

具体的，在本实施例中，处理器402用于执行处理器402中存储的方面级别情感分类方法的程序，所述方面级别情感分类方法的程序被执行时实现如下步骤：Specifically, in this embodiment, the processor 402 is configured to execute the program of the aspect-level sentiment classification method stored in the processor 402, and the following steps are implemented when the program of the aspect-level sentiment classification method is executed:

上述方法步骤的具体实施例过程可参见第一实施例，本实施例在此不再重复赘述。For the specific embodiment process of the above method steps, reference may be made to the first embodiment, which will not be repeated in this embodiment.

实施例五Embodiment 5

本实施例还提供一种计算机可读存储介质，如闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘、服务器、App应用商城等等，所述计算机可读存储介质可以是非易失性，也可以是易失性，其上存储有计算机程序，所述计算机程序被处理器执行时实现如下方法步骤：This embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), only Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disc, Server, App Store, etc., the computer-readable storage medium It can be non-volatile or volatile, and a computer program is stored thereon, and when the computer program is executed by the processor, the following method steps are implemented:

需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course hardware can also be used, but in many cases the former is better implementation.

以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。The above are only the preferred embodiments of the present application, and are not intended to limit the scope of the patent of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present application, or directly or indirectly applied in other related technical fields , are similarly included within the scope of patent protection of this application.