CN112862007B - Commodity sequence recommendation method and system based on user interest editing - Google Patents

Abstract

The present disclosure provides a commodity sequence recommendation method and system based on user interest editing, including: acquiring a commodity history sequence generating interactive behaviors with a user; inputting the commodity historical sequence into a pre-trained sequence prediction model, and outputting a recommended commodity; the training process adopts an interest editing strategy to enable the sequence prediction model to learn the commonalities and the peculiarities among different commodity historical sequences to obtain a recombined sequence representation, and the recombined sequence is used for training the sequence prediction model. The scheme solves the problem of user interest extraction and representation in sequence recommendation through an automatic supervision technology, and forces a sequence recommendation model to be capable of distinguishing the commonality and the particularity of different sequences of a user in interaction with a recommendation system based on an interest editing strategy, so that more accurate user interest is obtained, and the accuracy of sequence recommendation is improved.

Description

Translated fromChinese

基于用户兴趣编辑的商品序列推荐方法及系统Product sequence recommendation method and system based on user interest editing

技术领域technical field

本公开属于商品序列推荐技术领域，尤其涉及一种基于用户兴趣编辑的商品序列推荐方法及系统。The disclosure belongs to the technical field of product sequence recommendation, and in particular relates to a method and system for product sequence recommendation based on user interest editing.

背景技术Background technique

本部分的陈述仅仅是提供了与本公开相关的背景技术信息，不必然构成在先技术。The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

序列推荐是根据用户在某一时间段内的浏览或者购买记录来捕捉用户短期或者长期的兴趣爱好以此来对用户作出推荐的方法。序列推荐在推荐系统中占有非常重要的地位，它通过对用户的购买或浏览行为记录来建模，以此学习出用户的兴趣表示和变化，从而能够对用户的下一次点击进行预测和推荐。此外，利用这些技术的相关产品如今已被广泛应用至各个领域，比如淘宝会推荐符合用户兴趣的商品、网易云音乐会推荐符合用户心情的音乐、美团会推荐适合用户喜好的外卖和餐厅等，这极大的便利了人们的生活也促进了销售行业的收益。与此同时，随着大数据时代到来，用户数量以及用户记录激增，为了充分挖掘数据背后的关联，学术界和工业界对此广为关注。Sequence recommendation is a method that captures the user's short-term or long-term interests and hobbies based on the user's browsing or purchase records within a certain period of time to make recommendations to the user. Sequence recommendation occupies a very important position in the recommendation system. It learns the user's interest representation and changes by modeling the user's purchase or browsing behavior records, so that it can predict and recommend the user's next click. In addition, related products using these technologies have been widely used in various fields. For example, Taobao will recommend products that meet the user's interests, NetEase Cloud Concert will recommend music that suits the user's mood, and Meituan will recommend takeaways and restaurants that suit the user's preferences, etc. , which greatly facilitates people's lives and also promotes the income of the sales industry. At the same time, with the advent of the era of big data, the number of users and user records have surged. In order to fully mine the correlation behind the data, academia and industry have paid extensive attention to this.

发明人发现，在序列推荐中，已有的方法基于卷积神经网络(CNN)，循环神经网络(RNN)以及转换器(Transformer)等技术大多都假设用户的兴趣是集中且混合的，他们通常会认为在一个序列中会存在一个主要的用户兴趣，因此他们通常会用一个混合的向量来表示用户的整体兴趣并据此去做推荐，这并没有区分用户的不同兴趣并分别表示出来，导致推荐结果准确性不高且对于推荐的可解释性没有帮助。同时，已有方法通常依赖用户交互序列中的下一项点击记录来监督模型的学习过程，这些方法虽然对于用户兴趣的捕捉很有效，但是他们忽视了数据内在的其他关联。此前一些应用自监督技术的方法也只是简单在序列内部寻找表层的自监督信号，并没有扩展到序列之间来找寻更深层的关联关系。The inventors found that in sequence recommendation, most of the existing methods based on convolutional neural network (CNN), recurrent neural network (RNN) and converter (Transformer) assume that users' interests are concentrated and mixed, and they usually will think that there will be a main user interest in a sequence, so they usually use a mixed vector to represent the user's overall interest and make recommendations based on it, which does not distinguish the user's different interests and express them separately, resulting in Recommendation results are not accurate and do not contribute to the interpretability of recommendations. At the same time, existing methods usually rely on the next click record in the user interaction sequence to supervise the learning process of the model. Although these methods are effective in capturing user interests, they ignore other inherent associations in the data. Some previous methods of applying self-supervision technology simply searched for superficial self-supervised signals within the sequences, and did not extend to sequences to find deeper correlations.

发明内容Contents of the invention

本公开为了解决上述问题，提供了一种基于用户兴趣编辑的商品序列推荐方法及系统，所述方案通过自监督技术解决了序列推荐中用户兴趣抽取和表示的问题，并基于兴趣编辑策略，迫使序列推荐模型能够分辨出用户在与推荐系统交互中的不同序列之间公有性和特有性，从而获取更加准确的用户兴趣，提高序列推荐的准确性。In order to solve the above problems, the present disclosure provides a product sequence recommendation method and system based on user interest editing. The sequence recommendation model can distinguish the publicity and specificity between different sequences in the user's interaction with the recommendation system, so as to obtain more accurate user interests and improve the accuracy of sequence recommendation.

根据本公开实施例的第一个方面，提供了一种基于用户兴趣编辑的商品序列推荐方法，包括：According to the first aspect of the embodiments of the present disclosure, there is provided a method for recommending product sequences edited based on user interests, including:

获取与用户产生交互行为的商品历史序列；Obtain the historical sequence of commodities that have interacted with users;

将所述商品历史序列输入预训练的序列预测模型，输出推荐的商品；Inputting the historical sequence of commodities into a pre-trained sequence prediction model, and outputting recommended commodities;

其中，所述序列预测模型包括序列编码器、兴趣分辨器以及序列解码器，其训练过程采用兴趣编辑策略，使所述序列预测模型学习不同商品历史序列之间的公有性和特有性，获得重新组合后的序列表示，利用重新组合的序列对所述序列预测模型进行训练。Wherein, the sequence prediction model includes a sequence encoder, an interest discriminator, and a sequence decoder, and its training process adopts an interest editing strategy, so that the sequence prediction model learns the publicity and specificity between different commodity historical sequences, and obtains a new The combined sequence representation is used to train the sequence prediction model.

进一步的，所述兴趣编辑策略包括兴趣分离和兴趣交换两种操作，通过兴趣分辨器获得不同序列的多兴趣表示，然后通过兴趣分离操作迫使所述序列预测模型学习序列间的公有性和独特性，所述兴趣交换操作交换序列各自的公有性表示部分，对每一条序列生成重新组合之后的序列表示。Further, the interest editing strategy includes two operations of interest separation and interest exchange, and the multi-interest representation of different sequences is obtained through the interest distinguisher, and then the sequence prediction model is forced to learn the commonality and uniqueness between sequences through the interest separation operation , the interest exchange operation exchanges the respective public representation parts of the sequences, and generates a recombined sequence representation for each sequence.

进一步的，在所述序列编码器中，对于每个商品序列均拼接若干特殊标记，其中，每个标记表示用户特殊的兴趣，并且通过所述序列编码器将标记后的商品序列编码成隐状态表示。Further, in the sequence encoder, a number of special tags are spliced for each commodity sequence, wherein each tag represents the user's special interest, and the tagged commodity sequence is encoded into a hidden state through the sequence encoder express.

进一步的，在所述兴趣分辨器中，计算每一个特殊标记对于序列中的所有商品的注意力分布，同时，引入了兴趣覆盖机制，避免不同的特殊标记都关注相同的商品。Further, in the interest discriminator, the attention distribution of each special tag for all commodities in the sequence is calculated, and at the same time, an interest coverage mechanism is introduced to prevent different special tags from focusing on the same commodity.

根据本公开实施例的第二个方面，提供了一种基于用户兴趣编辑的商品序列推荐系统，包括：According to the second aspect of the embodiments of the present disclosure, there is provided a product sequence recommendation system edited based on user interests, including:

数据获取单元，其用于获取与用户产生交互行为的商品历史序列；A data acquisition unit, which is used to acquire a historical sequence of commodities that interact with users;

商品推荐单元，其用于将所述商品历史序列输入预训练的序列预测模型，输出推荐的商品；Commodity recommendation unit, which is used to input the commodity history sequence into the pre-trained sequence prediction model, and output the recommended commodity;

其中，所述序列预测模型包括序列编码器、兴趣分辨器以及序列解码器，其训练过程采用兴趣编辑策略，使所述序列预测模型学习不同商品历史序列之间的公有性和特有性，获得重新组合后的序列表示，利用重新组合的序列对所述序列预测模型进行训练。Wherein, the sequence prediction model includes a sequence encoder, an interest discriminator, and a sequence decoder, and its training process adopts an interest editing strategy, so that the sequence prediction model learns the publicity and specificity between different commodity historical sequences, and obtains a new The combined sequence representation is used to train the sequence prediction model.

根据本公开实施例的第三个方面，提供了一种电子设备，包括存储器、处理器及存储在存储器上运行的计算机程序，所述处理器执行所述程序时实现所述的一种基于用户兴趣编辑的商品序列推荐方法。According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory, and the processor implements the user-based Item sequence recommendation method for editors of interest.

根据本公开实施例的第四个方面，提供了一种非暂态计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现所述的一种基于用户兴趣编辑的商品序列推荐方法。According to a fourth aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned commodity edited based on user interests is realized. Sequence recommendation method.

与现有技术相比，本公开的有益效果是：Compared with the prior art, the beneficial effects of the present disclosure are:

(1)本公开所述方案通过自监督技术解决了序列推荐中用户兴趣抽取和表示的问题，并基于兴趣编辑策略，迫使序列推荐模型能够分辨出用户在与推荐系统交互中的不同序列之间公有性和特有性，从而获取更加准确的用户兴趣，提高序列推荐的准确性；(1) The solution described in this disclosure solves the problem of user interest extraction and representation in sequence recommendation through self-supervision technology, and based on the interest editing strategy, the sequence recommendation model is forced to distinguish between different sequences of users interacting with the recommendation system. Publicity and uniqueness, so as to obtain more accurate user interests and improve the accuracy of sequence recommendation;

(2)本公开所述方案创造性的提出了全新的自监督损失函数来增强学习过程，同时还分开建模了用户序列中的隐藏的多种兴趣表示。所述方案相对于已有的序列推荐方法，由于充分挖掘了序列之间的关联关系，从而在电商推荐领域的多个数据集上的多个评测指标上都取得了不错的提升；(2) The scheme in this disclosure creatively proposes a brand-new self-supervised loss function to enhance the learning process, and also separately models hidden multiple interest representations in user sequences. Compared with the existing sequence recommendation method, the described scheme has achieved a good improvement in multiple evaluation indicators on multiple data sets in the field of e-commerce recommendation due to fully mining the correlation between sequences;

(3)本公开所述方案可以应用至多个领域的推荐场景下，能够更加准确的判断和捕捉用户兴趣，让推荐系统更细致精准，以此能够提升用户的使用体验，也会提高电商运营的收益。(3) The solution described in this disclosure can be applied to recommendation scenarios in multiple fields, and can more accurately judge and capture user interests, making the recommendation system more detailed and accurate, thereby improving user experience and improving e-commerce operations income.

本公开附加方面的优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本公开的实践了解到。Advantages of additional aspects of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure.

附图说明Description of drawings

构成本公开的一部分的说明书附图用来提供对本公开的进一步理解，本公开的示意性实施例及其说明用于解释本公开，并不构成对本公开的不当限定。The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure, and the exemplary embodiments and descriptions thereof are used to explain the present disclosure, and do not constitute undue limitations on the present disclosure.

图1为本公开实施例一中所述的基于用户兴趣编辑的商品序列推荐方法的流程图。FIG. 1 is a flowchart of a method for recommending product sequences based on user interest editing described in Embodiment 1 of the present disclosure.

具体实施方式detailed description

下面结合附图与实施例对本公开做进一步说明。The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

应该指出，以下详细说明都是例示性的，旨在对本公开提供进一步的说明。除非另有指明，本文使用的所有技术和科学术语具有与本公开所属技术领域的普通技术人员通常理解的相同含义。It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

需要注意的是，这里所使用的术语仅是为了描述具体实施方式，而非意图限制根据本公开的示例性实施方式。如在这里所使用的，除非上下文另外明确指出，否则单数形式也意图包括复数形式，此外，还应当理解的是，当在本说明书中使用术语“包含”和/或“包括”时，其指明存在特征、步骤、操作、器件、组件和/或它们的组合。It should be noted that the terminology used herein is only for describing specific embodiments, and is not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural, and it should also be understood that when the terms "comprising" and/or "comprising" are used in this specification, they mean There are features, steps, operations, means, components and/or combinations thereof.

在不冲突的情况下，本公开中的实施例及实施例中的特征可以相互组合。In the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

实施例一：Embodiment one:

本实施例的目的是提供一种基于用户兴趣编辑的商品序列推荐方法。The purpose of this embodiment is to provide a product sequence recommendation method edited based on user interests.

一种基于用户兴趣编辑的商品序列推荐方法，包括：A product sequence recommendation method based on user interest editing, including:

获取与用户产生交互行为的商品历史序列；Obtain the historical sequence of commodities that have interacted with users;

将所述商品历史序列输入预训练的序列预测模型，输出推荐的商品；Inputting the historical sequence of commodities into a pre-trained sequence prediction model, and outputting recommended commodities;

其中，所述序列预测模型包括序列编码器、兴趣分辨器以及序列解码器，其训练过程采用兴趣编辑策略，使所述序列预测模型学习不同商品历史序列之间的公有性和特有性，获得重新组合后的序列表示，利用重新组合的序列对所述序列预测模型进行训练。Wherein, the sequence prediction model includes a sequence encoder, an interest discriminator, and a sequence decoder, and its training process adopts an interest editing strategy, so that the sequence prediction model learns the publicity and specificity between different commodity historical sequences, and obtains a new The combined sequence representation is used to train the sequence prediction model.

具体的，为了便于理解，以下结合附图1对本实施例所述方案进行详细说明：Specifically, for ease of understanding, the scheme described in this embodiment will be described in detail below in conjunction with accompanying drawing 1:

本公开的基本模型采用了当前流行的基于深度学习的编码器-解码器框架，名称为Multi-preference Transformer(简称为MrTransformer)，模型中包含了三个模块，分别为：序列编码器、兴趣分辨器以及序列解码器。在模型的学习过程中，采用兴趣编辑策略(Preference Editing)指导模型训练，其中包含两个操作：兴趣分离和兴趣交换。他们迫使模型学习到不同序列之间的公有性和特有性，从而产生全新的自监督损失来指导模型训练。基本模型(MrTransformer)和兴趣编辑(Preference Editing)组成了本公开的整体架构，称之为MrTransformer(PE)，其工作流程如图1所示。The basic model of this disclosure adopts the currently popular encoder-decoder framework based on deep learning, named Multi-preference Transformer (referred to as MrTransformer), and the model contains three modules, namely: sequence encoder, interest discrimination and sequence decoders. During the learning process of the model, the Preference Editing strategy (Preference Editing) is used to guide the model training, which includes two operations: interest separation and interest exchange. They force the model to learn the commonality and specificity between different sequences, resulting in a novel self-supervised loss to guide model training. The basic model (MrTransformer) and interest editing (Preference Editing) constitute the overall architecture of the present disclosure, called MrTransformer (PE), and its workflow is shown in Figure 1 .

接下来形式化定义MrTransformer(PE)的输入和输出。我们让I＝{i₁,i₂,…,i_τ,…,i_t}表示商品集合，让S＝{S₁,S₂,…S_S}表示序列集合。给定任意一个序列S＝{i₁,i₂,…,i_τ,…,i_t}，其中i_τ代表在τ时刻用户交互的商品，模型旨在建模此序列中隐含的用户兴趣表示，以此来预测下一个时刻用户将会交互的商品，该过程可用形式化定义如下：Next, define the input and output of MrTransformer (PE) formally. We let I={i₁ ,i₂ ,...,i_τ ,...,i_t } denote a commodity set, and let S={S₁ ,S₂ ,...S_S } denote a sequence set. Given any sequence S={i₁ ,i₂ ,…,i_τ ,…,i_t }, where i_τ represents the item that the user interacts with at time τ, the model aims to model the implicit user interest in this sequence Indicates that in order to predict the products that the user will interact with at the next moment, the process can be formally defined as follows:

P(it+1|S)～f(S) (1)P(it+1|S)～f(S) (1)

其中，P(i_t+1|S)代表推荐下一个商品的概率计算，f(S)是建模这个概率的函数。Among them, P(i_t+1 |S) represents the calculation of the probability of recommending the next product, and f(S) is a function that models this probability.

以下对MrTransformer(PE)的各个部分进行详细说明：The following describes each part of MrTransformer (PE) in detail:

(1)序列编码器(1) Sequence encoder

对于任意一条序列，和已有的基于循环神经网络(RNN)以及转换器(Transformer)的工作不同的是，本公开所述方案在序列前面拼接上K个特殊的标记([P1],[P₂],…,[P_K])，其中每一个标记都代表特殊的用户兴趣表示(用户在该段交互序列中，可能会反映不同的兴趣，比如在读书的过程中，他有可能既喜欢科学类型的，也喜欢浪漫类型的，还喜欢武侠类型的，而每种兴趣的表示我们用前面提到的特殊标记[Pi]来代表)，K代表对于整个数据集的兴趣个数，并非针对某一条特殊的序列。本公开所述方案定义处理之后的序列如下：For any sequence, different from the existing work based on recurrent neural network (RNN) and converter (Transformer), the scheme in this disclosure splices K special tags ([P1], [P₂ ],…,[P_K ]), each of which represents a special user interest expression (the user may reflect different interests in this interaction sequence, for example, in the process of reading, he may like both The scientific type, the romantic type, and the martial arts type, and the expression of each interest is represented by the special mark [Pi] mentioned above), K represents the number of interests in the entire data set, not for a particular sequence. The sequence after the protocol definition process described in this disclosure is as follows:

S′＝{[P₁],[P₂],…,[P_K],i₁,i₂,…i_τ,…i_t} (2)S′＝{[P₁ ],[P₂ ],…,[P_K ],i₁ ,i₂ ,…i_τ ,…i_t } (2)

在该模块中，将处理之后的序列编码成隐状态表示，具体的：首先，本公开所述方案分别初始化该序列的表示和位置表示为E和P，然后将位置表示加到序列表示上作为整体的序列表示：E＝E+P。之后，将这些序列表示输入到L层堆叠着的双向转换器层中，每一层通过掩码矩阵在某些特定的位置上交换信息来迭代的修改所有位置上的表示，该过程可以被定义为：In this module, the sequence after processing is encoded into a hidden state representation. Specifically: first, the scheme described in this disclosure initializes the representation and position representation of the sequence as E and P respectively, and then adds the position representation to the sequence representation as The overall sequence representation: E=E+P. Afterwards, these sequence representations are input into the bidirectional converter layer stacked by L layers, and each layer iteratively modifies the representations at all positions by exchanging information at certain positions through the mask matrix. This process can be defined for:

E^l＝Trm(E^l-1,Mask_e) (3)E^l ＝Trm(E^l-1 ,Mask_e ) (3)

其中，Trm代表转换器层，E^l是第l层的序列表示矩阵，Mask_e是掩码矩阵。具体来说，对于每一个特殊的标记[P_k]，他都可以从所有位置获取到信息，因为它旨在通过观察整条序列来捕捉用户特殊的兴趣表示，所以观察域为整条序列，其掩码向量是全一。对于序列中的每一个商品，它能接受的信息只限于商品之间，故掩码向量由K个零和t个一组成。最后，可以得到最上层的E^l作为整体的序列表示。Among them, Trm represents the transformer layer, E^l is the sequence representation matrix of layer l, and Mask_e is the mask matrix. Specifically, for each special marker [P_k ], he can obtain information from all positions, because it aims to capture the user's special interest representation by observing the entire sequence, so the observation domain is the entire sequence, Its mask vector is all ones. For each item in the sequence, the information it can accept is limited to items, so the mask vector consists of K zeros and t ones. Finally, the sequence representation of the topmost E^l as a whole can be obtained.

(2)兴趣分辨器(2) Interest Discriminator

在此模块中计算每一个特殊标记对于序列中的所有商品的注意力分布，表示如下：In this module, the attention distribution of each special tag for all items in the sequence is calculated, expressed as follows:

P,A＝Ident(E^l,Mask_I) (4)P,A＝Ident(E^l ,Mask_I ) (4)

其中，Ident由转化层实现，Mask_I是针对兴趣辨别的掩码矩阵。与编码器中的掩码矩阵不同，每一个特殊标记只能对商品计算兴趣分布，故需要去除其他标记的信息。对于每一个商品，掩码向量与编码器中商品的掩码向量相同。P代表着前K个特殊标记的多兴趣表示矩阵，A是这些标记对商品产生的兴趣分布矩阵。Among them, Ident is implemented by the conversion layer, and Mask_I is a mask matrix for interest identification. Unlike the mask matrix in the encoder, each special marker can only calculate the interest distribution for the product, so the information of other markers needs to be removed. For each item, the mask vector is the same as the item's mask vector in the encoder. P represents the multi-interest representation matrix of the first K special markers, and A is the interest distribution matrix generated by these markers for commodities.

同时，为了避免不同的特殊标记都关注相同的商品，本公开所述方案引入了一种兴趣覆盖机制，具体是维护K个覆盖向量

向量

记录的是在特殊标记[P_k]之前的所有标记对序列中所有商品的注意力分布之和，这代表了这些商品从注意力机制中得到的覆盖程度，其计算如下：At the same time, in order to prevent different special tags from focusing on the same product, the scheme described in this disclosure introduces an interest coverage mechanism, specifically maintaining K coverage vectors

vector

What is recorded is the sum of the attention distributions of all tokens before the special token [P_k ] to all items in the sequence, which represents the degree of coverage of these items from the attention mechanism, which is calculated as follows:

其中，a^k是由[P_k]对所有商品产生的注意力分布向量表示并且

是零向量这代表着在第一个时间步下，没有商品被覆盖到。where a^k is represented by the attention distribution vector generated by [P_k ] for all items and

is a zero vector which means that at the first time step, no items are covered.

(3)序列解码器(3) Sequence decoder

为了使用已经得到的用户多兴趣表示来做推荐，本实施例通过如下公式进行计算：In order to use the user's multi-interest representation that has been obtained to make recommendations, this embodiment calculates by the following formula:

P(i_t+1|S)＝softmax(pW+b) (6)P(i_t+1 |S)＝softmax(pW+b) (6)

其中，p是多兴趣表示矩阵P求和得到的整体向量表示，W是所有商品的嵌入表示矩阵，b是偏差项。Among them, p is the overall vector representation obtained by summing the multi-interest representation matrix P, W is the embedded representation matrix of all commodities, and b is the bias term.

(4)模型训练目标函数(4) Model training objective function

和大多数序列推荐方法一致，本公开所述方案的首要目标是对输入序列的每一个位置去预测下一项商品。所述方案使用负对数似然作为推荐损失函数，其计算如下：Consistent with most sequence recommendation methods, the primary goal of the proposed scheme is to predict the next item for each position in the input sequence. The proposed scheme uses the negative log-likelihood as the recommendation loss function, which is calculated as follows:

其中，θ是MrTransformer的所有参数。除此之外，本公开所述方案还定义一个覆盖损失来惩罚不同兴趣关注相同商品的情况：where θ are all parameters of MrTransformer. In addition, the scheme described in this disclosure also defines a coverage loss to penalize the case where different interests focus on the same item:

最后，将覆盖损失函数乘上超参α，加上推荐损失函数作为整体的损失函数来指导整个学习过程：Finally, the coverage loss function is multiplied by the hyperparameter α, and the recommendation loss function is added as the overall loss function to guide the entire learning process:

L(θ)＝L_rec(θ)+α·L_cov(θ) (9)L(θ)＝L_rec (θ)+α·L_cov (θ) (9)

其中，α控制着覆盖损失函数的比例。where α controls the scale of the coverage loss function.

(5)兴趣编辑学习策略(5) Interest editing learning strategy

该学习策略旨在挖掘不同交互序列之间商品之间的关联关系。This learning strategy aims to mine the associative relationship between items among different interaction sequences.

如图1所示，兴趣编辑包含两种操作：兴趣分离和兴趣交换。首先需要从序列集合中采样两条序列，这需要保证这两条序列有一定的重合程度和各自的独特性。通过基本模型中的兴趣分辨器，可以得到不同序列的多兴趣表示。然后兴趣分离操作迫使模型学习到序列之间的公有性和独特性，兴趣交换操作交换各自的公有性表示部分以此对原先的每一条序列生成重新组合之后的序列表示。As shown in Figure 1, interest editing consists of two operations: interest separation and interest exchange. First, two sequences need to be sampled from the sequence set, which needs to ensure that the two sequences have a certain degree of coincidence and their respective uniqueness. Through the interest discriminator in the basic model, multiple interest representations for different sequences can be obtained. Then the interest separation operation forces the model to learn the commonness and uniqueness between the sequences, and the interest exchange operation exchanges the respective public representation parts to generate a recombined sequence representation for each original sequence.

兴趣分离操作：对于每一对序列S_x和S_y,为了衡量他们的相似程度，通过计算相似度矩阵来表示他们的相关程度，每一个元素I_ij可以按照如下计算：Interest separation operation: For each pair of sequences S_x and S_y , in order to measure their similarity, calculate the similarity matrix to represent their degree of correlation, and each element I_ij can be calculated as follows:

其中，p_i和p_j分别是多兴趣表示矩阵P_x和P_y的向量，

代表元素相乘操作。基于该相似度矩阵，可以计算注意力矩阵A_x和B_y，他们反映了一个序列的兴趣表示对另一个序列的兴趣表示的注意力分布，计算如下：where p_i and p_j are vectors of multi-interest representation matrices P_x and P_y , respectively,

Represents an element-wise multiplication operation. Based on this similarity matrix, the attention matrices A_x and By_y can be calculated, which reflect the attention distribution of the interest representation of one sequence to the interest representation of another sequence, calculated as follows:

A_x＝softmax_row(I) (11)A_x = softmax_row (I) (11)

B_y＝softmax_col(I)B_y = softmax_col (I)

其中，softmax_row和softmax_col分别代表按照行和列计算softmax函数。随后，每条序列公有的和特有的兴趣表示可按如下计算：Among them, softmax_row and softmax_col represent calculating the softmax function by row and column respectively. Subsequently, the public and specific interest representations for each sequence can be computed as follows:

其中，C_x和C_y代表每条序列的公有性兴趣表示，U_x和U_y代表序列各自的独特性兴趣表示。Among them, C_x and C_y represent the public interest representations of each sequence, and U_x and U_y represent the unique interest representations of each sequence.

兴趣交换操作：对于两条序列，通过交换此前得到的公有性兴趣表示以此来得到重新组合之后的序列表示，交换过程如下：Interest exchange operation: For two sequences, the recombined sequence representation is obtained by exchanging the previously obtained public interest representations. The exchange process is as follows:

基于重新组合的新的序列表示，本公开所述方案定义了两种自监督信号用于训练：Based on recombined new sequence representations, the scheme described in this disclosure defines two self-supervised signals for training:

L_SSL(θ)＝L_pred(θ)+L_app(θ) (14)L_SSL (θ) = L_pred (θ) + L_app (θ) (14)

其中，L_pred(θ)基于重新组合的序列表示P_x′和P_y′来预测下一个商品，计算采用基本模型中的负对数似然函数：Among them, L_pred (θ) is based on the recombined sequence representations P_x ′ and P_y ′ to predict the next commodity, and the calculation uses the negative logarithmic likelihood function in the basic model:

L_pred(θ)＝L^x_rec+L^y_rec (15)L_pred (θ) = L^x_rec +L^y_rec (15)

另一项是正则化项，它包含三个部分：The other term is the regularization term, which consists of three parts:

其中，L_app(θ)^x和L_app(θ)^y保证重组合之后的序列表示无限接近原始序列表示，L_app(θ)^c确保两条序列之间的公有性表示部分足够接近。Among them, L_app (θ)^x and L_app (θ)^y ensure that the sequence representation after recombination is infinitely close to the original sequence representation, and L_app (θ)^c ensures that the common representation between the two sequences is close enough.

最后，MrTransformer(PE)的训练损失函数计算如下：Finally, the training loss function of MrTransformer(PE) is calculated as follows:

L_all(θ)＝L(θ)+L_SSL(θ) (17)L_all (θ)＝L(θ)+L_SSL (θ) (17)

其中，L(θ)用于兴趣抽取和表示，L_SSL(θ)用于兴趣编辑。Among them, L(θ) is used for interest extraction and representation, and L_SSL (θ) is used for interest editing.

本公开所述方案旨在创新性的应用自监督学习理论至序列推荐领域，创造性的提出了全新的自监督损失函数来增强学习过程，同时还分开建模了用户序列中的隐藏的多种兴趣表示。本公开所述方案相对于已有的序列推荐方法，由于充分挖掘了序列之间的关联关系，从而在电商推荐领域的多个数据集上的多个评测指标上都取得了不错的提升。同时，本公开所述方案可以应用至多个领域的推荐场景下，能够更加准确的判断和捕捉用户兴趣，让推荐系统更细致精准，以此能够提升用户的使用体验，也会提高电商运营的收益。The scheme described in this disclosure aims to innovatively apply self-supervised learning theory to the field of sequence recommendation, creatively propose a new self-supervised loss function to enhance the learning process, and also separately model the hidden multiple interests in user sequences express. Compared with the existing sequence recommendation method, the solution described in the present disclosure has fully exploited the relationship between the sequences, thus achieving a good improvement in multiple evaluation indicators on multiple data sets in the field of e-commerce recommendation. At the same time, the solution described in this disclosure can be applied to recommendation scenarios in multiple fields, and can more accurately judge and capture user interests, making the recommendation system more detailed and accurate, thereby improving the user experience and improving the efficiency of e-commerce operations. income.

实施例二：Embodiment two:

本实施例的目的是提供一种基于用户兴趣编辑的商品序列推荐系统。The purpose of this embodiment is to provide a product sequence recommendation system edited based on user interests.

一种基于用户兴趣编辑的商品序列推荐系统，包括：A product sequence recommendation system edited based on user interests, including:

数据获取单元，其用于获取与用户产生交互行为的商品历史序列；A data acquisition unit, which is used to acquire a historical sequence of commodities that interact with users;

商品推荐单元，其用于将所述商品历史序列输入预训练的序列预测模型，输出推荐的商品；Commodity recommendation unit, which is used to input the commodity history sequence into the pre-trained sequence prediction model, and output the recommended commodity;

其中，所述序列预测模型包括序列编码器、兴趣分辨器以及序列解码器，其训练过程采用兴趣编辑策略，使所述序列预测模型学习不同商品历史序列之间的公有性和特有性，获得重新组合后的序列表示，利用重新组合的序列对所述序列预测模型进行训练。Wherein, the sequence prediction model includes a sequence encoder, an interest discriminator, and a sequence decoder, and its training process adopts an interest editing strategy, so that the sequence prediction model learns the publicity and specificity between different commodity historical sequences, and obtains a new The combined sequence representation is used to train the sequence prediction model.

在更多实施例中，还提供：In further embodiments, there is also provided:

一种电子设备，包括存储器和处理器以及存储在存储器上并在处理器上运行的计算机指令，所述计算机指令被处理器运行时，完成实施例一中所述的方法。为了简洁，在此不再赘述。An electronic device includes a memory, a processor, and computer instructions stored in the memory and run on the processor. When the computer instructions are run by the processor, the method described in Embodiment 1 is completed. For the sake of brevity, details are not repeated here.

应理解，本实施例中，处理器可以是中央处理单元CPU，处理器还可以是其他通用处理器、数字信号处理器DSP、专用集成电路ASIC，现成可编程门阵列FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。It should be understood that in this embodiment, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate array FPGA or other programmable logic devices , discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

存储器可以包括只读存储器和随机存取存储器，并向处理器提供指令和数据、存储器的一部分还可以包括非易失性随机存储器。例如，存储器还可以存储设备类型的信息。The memory may include read-only memory and random access memory, and provides instructions and data to the processor, and a part of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

一种计算机可读存储介质，用于存储计算机指令，所述计算机指令被处理器执行时，完成实施例一中所述的方法。A computer-readable storage medium is used for storing computer instructions, and when the computer instructions are executed by a processor, the method described in the first embodiment is completed.

实施例一中的方法可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器、闪存、只读存储器、可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor. The software module may be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware. To avoid repetition, no detailed description is given here.

本领域普通技术人员可以意识到，结合本实施例描述的各示例的单元即算法步骤，能够以电子硬件或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。Those skilled in the art can appreciate that the units of the examples described in this embodiment, that is, the algorithm steps, can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementation should not be considered beyond the scope of the present disclosure.

上述实施例提供的一种基于用户兴趣编辑的商品序列推荐方法及系统可以实现，具有广阔的应用前景。The method and system for recommending product sequences based on user interest editing provided by the above embodiments can be implemented and have broad application prospects.

以上所述仅为本公开的优选实施例而已，并不用于限制本公开，对于本领域的技术人员来说，本公开可以有各种更改和变化。凡在本公开的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

上述虽然结合附图对本公开的具体实施方式进行了描述，但并非对本公开保护范围的限制，所属领域技术人员应该明白，在本公开的技术方案的基础上，本领域技术人员不需要付出创造性劳动即可做出的各种修改或变形仍在本公开的保护范围以内。Although the specific implementation of the present disclosure has been described above in conjunction with the accompanying drawings, it does not limit the protection scope of the present disclosure. Those skilled in the art should understand that on the basis of the technical solutions of the present disclosure, those skilled in the art do not need to pay creative work Various modifications or variations that can be made are still within the protection scope of the present disclosure.

Claims (5)

Translated fromChinese

1.一种基于用户兴趣编辑的商品序列推荐方法，其特征在于，包括：1. A commodity sequence recommendation method edited based on user interests, characterized in that it comprises:获取与用户产生交互行为的商品历史序列；Obtain the historical sequence of commodities that have interacted with users;将所述商品历史序列输入预训练的序列预测模型，输出推荐的商品；Inputting the historical sequence of commodities into a pre-trained sequence prediction model, and outputting recommended commodities;其中，所述序列预测模型包括序列编码器、兴趣分辨器以及序列解码器，其训练过程采用兴趣编辑策略，使所述序列预测模型学习不同商品历史序列之间的公有性和特有性，获得重新组合后的序列表示，利用重新组合的序列对所述序列预测模型进行训练；Wherein, the sequence prediction model includes a sequence encoder, an interest discriminator, and a sequence decoder, and its training process adopts an interest editing strategy, so that the sequence prediction model learns the publicity and specificity between different commodity historical sequences, and obtains a new the combined sequence representation, using the recombined sequences to train the sequence prediction model;在所述序列编码器中，对于每个商品序列均拼接若干特殊标记，其中，每个标记表示用户特殊的兴趣，并且通过所述序列编码器将标记后的商品序列编码成隐状态表示；In the sequence encoder, a number of special marks are spliced for each commodity sequence, wherein each mark represents the user's special interest, and the marked commodity sequence is encoded into a hidden state representation through the sequence encoder;在所述兴趣分辨器中，计算每一个特殊标记对于序列中的所有商品的注意力分布，同时，引入了兴趣覆盖机制，避免不同的特殊标记都关注相同的商品；In the interest distinguisher, calculate the attention distribution of each special tag for all commodities in the sequence, and at the same time, introduce an interest coverage mechanism to avoid different special tags all paying attention to the same commodity;所述兴趣覆盖机制，具体是维护

个覆盖向量

，向量

记录的是在特殊标记

之前的所有标记对序列中所有商品的注意力分布之和，这代表了这些商品从注意力机制中得到的覆盖程度，其计算如下：The interest coverage mechanism, specifically maintaining

coverage vector

,vector

Records are marked in special

The sum of the attention distributions of all previous tokens to all items in the sequence, which represents the degree of coverage these items get from the attention mechanism, is calculated as follows:

其中，

是由

对所有商品产生的注意力分布向量表示并且

，

是零向量这代表着在第一个时间步下，没有商品被覆盖到；in,

By

A vector representation of the attention distribution generated over all items and

,

is a zero vector which means that at the first time step, no items are covered;在所述序列解码器中，使用已经得到的用户多兴趣表示来做推荐；In the sequence decoder, use the obtained multi-interest representation of the user to make recommendations;所述兴趣编辑策略包括兴趣分离和兴趣交换两种操作，通过兴趣分辨器获得不同序列的多兴趣表示，然后通过兴趣分离操作迫使所述序列预测模型学习序列间的公有性和特有性，所述兴趣交换操作交换各自的公有性表示部分以此对之前的每一条序列生成重新组合之后的序列表示；The interest editing strategy includes two operations of interest separation and interest exchange. The multi-interest representation of different sequences is obtained through the interest discriminator, and then the sequence prediction model is forced to learn the commonality and specificity between sequences through the interest separation operation. The interest exchange operation exchanges the respective public representation parts to generate a recombined sequence representation for each previous sequence;所述兴趣分离操作具体为：对于每一对序列

和

,通过计算相似度矩阵来表示它们的相关程度；基于该相似度矩阵，计算注意力矩阵

和

，根据所述注意力矩阵获得每条序列具有公有性和特有性的表示部分；The interest separation operation is specifically: for each pair of sequences

with

, by calculating the similarity matrix to represent their degree of correlation; based on the similarity matrix, calculate the attention matrix

with

, according to the attention matrix, obtain the common and unique representation part of each sequence;

其中，

和

代表每条序列的公有性兴趣表示，

和

代表序列各自的特有性兴趣表示，

和

代表注意力矩阵，

和

代表多兴趣表示矩阵，

代表元素相乘操作。in,

with

represents the public interest representation of each sequence,

with

Representing the respective idiosyncratic interest representations of the sequences,

with

represents the attention matrix,

with

represents the multi-interest representation matrix,

Represents an element-wise multiplication operation.2.如权利要求1所述的一种基于用户兴趣编辑的商品序列推荐方法，其特征在于，所述序列预测模型训练采用的目标函数为推荐损失函数和覆盖损失函数的组合，其中，所述推荐损失函数采用负对数似然估计，所述覆盖损失函数用于惩罚不同兴趣关注相同商品的情况；2. A method for recommending commodity sequences edited based on user interests as claimed in claim 1, wherein the objective function used in the training of the sequence prediction model is a combination of a recommendation loss function and a coverage loss function, wherein the The recommendation loss function uses negative logarithmic likelihood estimation, and the coverage loss function is used to punish the situation where different interests pay attention to the same product;首要目标是对输入序列的每一个位置去预测下一项商品，使用负对数似然作为推荐损失函数，其计算如下：The primary goal is to predict the next item for each position of the input sequence, using the negative log-likelihood as the recommendation loss function, which is calculated as follows:

其中，

是基于深度学习的编码器-解码器框架Multi-preference Transformer的所有参数；除此之外，还定义一个覆盖损失来惩罚不同兴趣关注相同商品的情况：in,

is all the parameters of the deep learning-based encoder-decoder framework Multi-preference Transformer; in addition, a coverage loss is defined to punish the situation where different interests focus on the same product:

最后，将覆盖损失函数乘上超参

，加上推荐损失函数作为整体的损失函数来指导整个学习过程：Finally, the coverage loss function is multiplied by the hyperparameter

, plus the recommended loss function as the overall loss function to guide the entire learning process:

其中，

控制着覆盖损失函数的比例。in,

Controls the scaling of the coverage loss function.3.一种基于用户兴趣编辑的商品序列推荐系统，其特征在于，包括：3. A product sequence recommendation system edited based on user interests, characterized in that it comprises:数据获取单元，其用于获取与用户产生交互行为的商品历史序列；A data acquisition unit, which is used to acquire a historical sequence of commodities that interact with users;商品推荐单元，其用于将所述商品历史序列输入预训练的序列预测模型，输出推荐的商品；Commodity recommendation unit, which is used to input the commodity history sequence into the pre-trained sequence prediction model, and output the recommended commodity;其中，所述序列预测模型包括序列编码器、兴趣分辨器以及序列解码器，其训练过程采用兴趣编辑策略，使所述序列预测模型学习不同商品历史序列之间的公有性和特有性，获得重新组合后的序列表示，利用重新组合的序列对所述序列预测模型进行训练；Wherein, the sequence prediction model includes a sequence encoder, an interest discriminator, and a sequence decoder, and its training process adopts an interest editing strategy, so that the sequence prediction model learns the publicity and specificity between different commodity historical sequences, and obtains a new the combined sequence representation, using the recombined sequences to train the sequence prediction model;在所述序列编码器中，对于每个商品序列均拼接若干特殊标记，其中，每个标记表示用户特殊的兴趣，并且通过所述序列编码器将标记后的商品序列编码成隐状态表示；In the sequence encoder, a number of special marks are spliced for each commodity sequence, wherein each mark represents the user's special interest, and the marked commodity sequence is encoded into a hidden state representation through the sequence encoder;在所述兴趣分辨器中，计算每一个特殊标记对于序列中的所有商品的注意力分布，同时，引入了兴趣覆盖机制，避免不同的特殊标记都关注相同的商品；In the interest discriminator, calculate the attention distribution of each special tag for all commodities in the sequence, and at the same time, introduce an interest coverage mechanism to avoid different special tags all paying attention to the same commodity;所述兴趣覆盖机制，具体是维护

个覆盖向量

，向量

记录的是在特殊标记

之前的所有标记对序列中所有商品的注意力分布之和，这代表了这些商品从注意力机制中得到的覆盖程度，其计算如下：The interest coverage mechanism, specifically maintaining

coverage vector

,vector

Records are marked in special

The sum of the attention distributions of all previous tokens to all items in the sequence, which represents the degree of coverage these items get from the attention mechanism, is calculated as follows:

其中，

是由

对所有商品产生的注意力分布向量表示并且

，

是零向量这代表着在第一个时间步下，没有商品被覆盖到；in,

By

A vector representation of the attention distribution generated over all items and

,

is a zero vector which means that at the first time step, no items are covered;在所述序列解码器中，使用已经得到的用户多兴趣表示来做推荐；In the sequence decoder, use the obtained multi-interest representation of the user to make recommendations;所述兴趣编辑策略包括兴趣分离和兴趣交换两种操作，通过兴趣分辨器获得不同序列的多兴趣表示，然后通过兴趣分离操作迫使所述序列预测模型学习序列间的公有性和特有性，所述兴趣交换操作交换各自的公有性表示部分以此对之前的每一条序列生成重新组合之后的序列表示；The interest editing strategy includes two operations of interest separation and interest exchange. The multi-interest representation of different sequences is obtained through the interest discriminator, and then the sequence prediction model is forced to learn the commonality and specificity between sequences through the interest separation operation. The interest exchange operation exchanges the respective public representation parts to generate a recombined sequence representation for each previous sequence;所述兴趣分离操作具体为：对于每一对序列

和

,通过计算相似度矩阵来表示它们的相关程度；基于该相似度矩阵，计算注意力矩阵

和

，根据所述注意力矩阵获得每条序列具有公有性和特有性的表示部分；The interest separation operation is specifically: for each pair of sequences

with

, by calculating the similarity matrix to represent their degree of correlation; based on the similarity matrix, calculate the attention matrix

with

, according to the attention matrix, obtain the common and unique representation part of each sequence;

其中，

和

代表每条序列的公有性兴趣表示，

和

代表序列各自的特有性兴趣表示，

和

代表注意力矩阵，

和

代表多兴趣表示矩阵，

代表元素相乘操作。in,

with

represents the public interest representation of each sequence,

with

Representing the respective idiosyncratic interest representations of the sequences,

with

represents the attention matrix,

with

represents the multi-interest representation matrix,

Represents an element-wise multiplication operation.4.一种电子设备，包括存储器、处理器及存储在存储器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现如权利要求1-2任一项所述的一种基于用户兴趣编辑的商品序列推荐方法。4. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, wherein the processor implements the one described in any one of claims 1-2 when executing the program. Product sequence recommendation method based on user interest editing.5.一种非暂态计算机可读存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时实现如权利要求1-2任一项所述的一种基于用户兴趣编辑的商品序列推荐方法。5. A non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that, when the program is executed by a processor, the editing method based on user interests as described in any one of claims 1-2 is realized. Product sequence recommendation method.

Images