CN111881363B - Recommendation method based on graph interaction network - Google Patents

Abstract

A recommendation method based on a graph interaction network is applied to the field of personalized recommendation of users. The rapid development of the internet industry and the continuous increase of network data volume, the traditional recommendation method and the deep learning method are difficult to meet the complex application environment, and have the defects in accuracy and space complexity. Therefore, the recommendation method based on the graph interaction network, provided by the invention, has the advantages that the personalized recommendation accuracy can be ensured, the model space complexity is reduced, and the application prospect is wide.

Description

Recommendation method based on graph interaction network

Technical Field

The invention is applied to the field of recommendation systems based on U-I relations, and particularly relates to data mining and deep learning technologies such as a graph depth network, an attention mechanism, user preference information and item attribute information feature extraction, U-I interaction information modeling and the like.

Background

Personalized recommendation is a comprehensive analysis task, and is widely applied to the fields of social networks, music stations, electronic commerce, personalized advertisements, movies, video websites and the like, so that the personalized recommendation is paid attention to. With the rapid development of the internet industry and the continuous increase of network data volume, recommendation systems face increasingly complex recommendation tasks and application environments. Particularly, since the Web 2.0 era, with the advent of the dismilitary of social networking media, the internet people are consumers of network information and producers of network content, and the amount of information in the internet has increased exponentially. Because of the limited discrimination capability of users, users often feel unproductive when faced with huge and complex internet information, so that the cost of searching useful information in the internet is huge, and the information overload problem arises. Thus, it is a very important and challenging matter for users how to quickly and accurately locate their own desired content in exponentially growing resources. It is also a very difficult matter for merchants to present the appropriate items to users in time, thereby promoting increases in transaction volume and economy. The advent of recommendation systems has greatly alleviated this difficulty.

Collaborative filtering algorithm plays an important role in the early recommended scenario as the most classical traditional recommendation algorithm. The collaborative filtering algorithm can naturally model the higher-order relation between the user and the object, and has low model complexity and easy deployment. These advantages make it the most widely used recommended method to date. However, with the explosive growth of internet users, the variety and properties of the articles are more and more complete, and the increasing richness of internet resources makes the preference information of users progress toward diversification and refinement. Under such an environment, the recommendation accuracy of the algorithm has a great disadvantage, and the conventional recommendation algorithm represented by the collaborative filtering algorithm is difficult to meet the personalized requirements of users.

In recent years, deep learning techniques have been rapidly developed. Deep learning is increasingly used in the field of computer vision, natural language processing and recommendation systems. The deep learning technology is used for deep mining of user preference information and article attribute information by constructing a deep network. The model architecture of the recommendation system is innovated, and various defects of the traditional model are overcome. The recommendation accuracy is greatly improved, and a more effective solution is provided for the problems of system cold start and the like, so that the recommendation accuracy is widely paid attention to. Conventional deep learning techniques do not naturally combine user and item information. Generally, the method for stacking multiple models only extracts better quality characteristics of users and articles by utilizing a deep network according to the interaction relation between the users and the articles, and predicts the preference degree of the users to the articles to be recommended through other models or another deep network.

In recent years, the analysis and processing of non-European data becomes a hot topic in academia and industry, and a graph depth network can naturally integrate interaction relationship between users and articles. The method effectively extracts the graph data characteristics and the node characteristics, naturally mines the user preference and the article attribute information expression, and provides a brand-new direction for personalized recommendation tasks. In recent years, the research of the graph depth network has greatly developed, thomas Kpif in 2017 proposes the concept of the graph convolution network, which provides a brand-new idea for processing graph structure data and applies the convolutional neural network in the deep learning to the graph structure data. The graph convolution method based on the spatial domain takes any node as a convolution object, and gathers information of neighbor nodes to construct a graph convolution layer, so that the graph convolution method based on the spatial domain is more widely applied compared with the graph convolution method based on the frequency domain due to flexibility and high efficiency. Meanwhile, hamilton provides a generalized learning mode suitable for a large-scale network, so that node characteristics can be quickly generated for newly added nodes without additional training, and the problem of expandability of the graph convolution neural network is greatly relieved. The rapid development of graph depth networks indicates directions for the construction of personalized recommendation systems.

Disclosure of Invention

In order to realize a personalized recommendation system of a user, a personalized recommendation scheme based on a graph interaction network is provided. The process flow is shown in fig. 1. According to the method, a U-I interaction relation diagram is used as input data, all modules finish feature extraction and feature analysis processing on user preference and object attributes, and finally, the prediction score of a user on a target object is output. Specifically, the method firstly carries out graph structure modeling on the interactive relation data of the user and the article, and the data sets are all from the public data sets in the industry. After the graph structuring process is completed, the characteristic distribution of the user and the object is optimized by stacking a plurality of average convolution layers. After the U-I feature optimization is completed, there may be a greater similarity between the user features and the item features that meet the user's preferences. And then, utilizing an attention mechanism, fusing the node characteristics of the target user and the node characteristics of the object to be recommended to obtain an implicit characteristic expression of the graph interaction network, namely an interaction vector of the U-I relation pair, and finally, modeling a high-order nonlinear relation among U-I through a multi-layer graph full-connection layer network learning interaction characteristic vector distribution rule to obtain a prediction score of the user on the target object. And finally, realizing personalized recommendation tasks for the user through a top-N recommendation mechanism. The overall block diagram of the method is shown in fig. 1.

The invention content of each main module of the method is as follows:

1. User preference information and item attribute information optimization

The first module is a user preference and article attribute feature optimization module, and the purpose of user preference information modeling is to obtain accurate user feature representation according to user interaction behaviors so as to fully mine the interest and hobby information of a user. The object attribute information modeling is used for obtaining perfect relation attributes and content characteristics, further accurately finding out audience groups of objects, and completing personalized recommendation of the objects. User and item feature expressions are quickly obtained by using two mean convolution layers. Where the characteristics between U-I, where the user preferences and item attributes are consistent, are more similar. The user and article feature extraction is an important component for constructing a recommendation system, and plays a key role in personalized recommendation.

Firstly, carrying out graph structural modeling on the U-I interaction data, and establishing a connection relationship between user items with interaction behaviors. After the structured modeling of the graph is completed, initializing user preference characteristics and item attribute characteristics, and then accurately modeling the user item characteristics, so that user preference information and item attribute information are fully mined. The method optimizes the characteristics of users and articles through two average convolution layers, wherein each layer of target node characteristic learning is obtained by carrying out averaging treatment on the basis of one layer of neighbor node characteristics and the node characteristics on the node. After the modeling of the user and the article features is finished, the user features and the article features have stronger distribution regularity, and then the user preference and the article attribute information are preliminarily extracted. The mean convolution processing greatly reduces the space complexity of the model while ensuring the feature accuracy. The module flow framework is shown in fig. 2.

U-I interaction feature extraction module

The second module is a U-I interaction feature extraction module, and the function of this module is to extract the interaction feature expression between the user item pairs. The U-I interactive feature extraction module is used for directly fusing user features and article features on the graphic neural network to learn interactive feature expression of the user features and the article features after the user preference information and the article attribute information are extracted. The U-I interaction characteristics are input characteristics of the interaction speculation module.

Firstly, splicing user characteristics and first-order neighbor node characteristics of a user, splicing characteristics of articles to be recommended and first-order neighbor node characteristics of the articles to be recommended, and inputting the spliced characteristics into a self-attention network to obtain attention coefficients of the module, wherein the self-attention network adopts multi-layer full-connection layer network modeling. And finally, aggregating the U-I target node characteristics and the neighbor node characteristics thereof through the attention coefficients obtained through learning to obtain the final interaction characteristic expression. The specific flow of the module is shown in fig. 3.

3. Interactive speculation module

The third module is an interaction presumption module, and the function of the module is to learn the distribution rule of the interaction characteristics after the U-I interaction characteristic extraction module extracts the interaction characteristics between the user and the target object, so as to obtain the presumption score of the user on the target object. The interactive speculation module is an indispensable step for building recommendation, and the design method of the interactive speculation module is quite many, such as traditional feature inner products, logistic regression algorithms and the like, but the traditional algorithms cannot well model the high-dimensional feature relation between the user and the object, so that the invention adopts the classical DNN algorithm to fuse the feature information of the user and the object and obtain a more accurate speculation result.

After the interactive feature expression of the user and the target object is obtained, the interactive vector is directly input into the DNN network, and the preliminary prediction of the model is obtained. And then, carrying out normalization processing on the module predicted value through a sigmoid function, and modeling the predicted score of the target object by the user into a preference probability expression. The flow of this module is shown in fig. 4.

Top-N recommendation module

The last module of the invention is a top-N recommendation module, and a top-N recommendation mechanism is also the most commonly used mechanism for constructing a recommendation system. And after the evaluation value prediction of all the articles of the list to be recommended by the target user is obtained. And sorting all the articles in a descending order according to the scores, recommending the first N articles to the user, and realizing personalized recommendation of the user.

Drawings

FIG. 1 is a general block diagram of an interaction graph based neural network;

FIG. 2 is a user preference information and item attribute information modeling module framework;

FIG. 3 is a U-I interaction feature extraction module framework

FIG. 4 is an interactive speculation module framework

Detailed Description

The invention provides a personalized recommendation method based on an interaction graph neural network. The specific implementation steps of the invention are as follows:

Step one: selecting a public recommended data set, arranging serial numbers for all users and articles, randomly selecting 90% of the articles interacted by each user as a training set, and remaining 10% of the articles as a test set. Each training set and test set consists of three parts: user, article, label. The item with interactive behavior with the user has a label of 1, otherwise the label is 0. And carrying out undirected graph structural expression through all pieces of data with the labels of 1 in the training set, and establishing a connection relationship for users with interactive behaviors and articles.

Step two: after the training set undirected graph is structured, randomly initializing high-dimensional feature expression of all user nodes and object nodes in the graph, namely randomly initializing user preference information and object attribute information. And then, building two layers of mean value convolution layer networks according to the node connection relation of the undirected graph, wherein all node characteristics in each layer of mean value convolution layer network are obtained by polymerizing the node characteristics and first-order neighbor node characteristics in the previous layer of network, the polymerization mode is an averaging treatment, and the mathematical expression is as follows:

Wherein the method comprises the steps ofFeatures of user node u representing the K-th mean convolution layer. N (u) represents a first-order item neighbor node of the user node u. N (v) represents a first-order user neighbor node of item node v. /(I)Features of the item node v representing the K-th mean convolution layer. MEAN represents the averaging process, i.e. averaging the relevant U-I features for each dimension. After the multi-layer mean convolution layer processing, all node characteristics in the undirected graph have larger distribution regularity, and user nodes with similar preference and attribute and object node characteristics are similar. User preference information and item attribute information are initially modeled.

Step three: after the modeling of the user object features is completed, aiming at a target user and an object node to be recommended in the undirected graph, a graph attention mechanism is fused, and the interaction feature expression between the U-I pair is obtained by aggregating the user and the first-order neighbor features thereof and the object and the first-order neighbor features thereof. And splicing the user characteristics and the first-order neighbor node characteristics of the user, splicing the characteristics of the articles to be recommended and the first-order neighbor node characteristics thereof, inputting the spliced characteristics into the attention network, and obtaining attention coefficients corresponding to the user and the article characteristics. And carrying out softmax normalization treatment on the obtained attention coefficients. The method for the graph annotation force network adopts two full-connection layers for modeling. The advantage of constructing the self-attention mechanism network by adopting the multi-layer full-connection layer is obvious, the number of the first-order neighbor nodes of the undirected graph node can be self-adapted, and the importance of the first-order neighbors of the node to the node can be effectively modeled. And thus a more accurate attention coefficient. Wherein attention coefficient modeling mathematical expression is:

Wherein W₁,W₂ represents the parameter matrix of the first layer and the second layer of the two-layer attention network, b₁,b₂ represents the deviation coefficient of the first layer and the second layer of the two-layer attention network, sigma represents a nonlinear activation function, and Relu activation function is adopted in the invention.Represented is a node characteristic of the target user u_i in the undirected graph. h_a denotes the characteristics of node a, which is any node in the set of user u_i and its first-order neighbors N (i). N (i) represents the first-order item neighbor set for user u_i.Representing the stitching process. /(I)Represented is a node characteristic of the item v_j to be recommended. h_b is a feature of node b, which is any node in the set of item v_j and its first order neighbors N (j). N (j) represents a first-order set of user neighbors of item v_j. /(I)AndAnd (5) representing the weight coefficient which is preliminarily obtained by the characteristics obtained by splicing through attention networks. And then carrying out softmax normalization processing to obtain attention coefficients alpha_ia of the target user and the first-order neighbors thereof and attention coefficients beta_jb of the articles to be recommended and the first-order neighbors thereof.

And after attention coefficients of the user, the first-order neighbors of the user and the target object and the first-order neighbors of the target object are obtained, the attention coefficient weighting target node characteristics are fused to obtain the final interaction characteristic expression. The mathematical expression is as follows:

z_ij is the interactive feature expression of the obtained target user u_i and the item v_j to be recommended.

Step four: after the interactive feature expression of the user and the target object is obtained, the model learns the distribution rule of the interactive feature through the interactive presumption module, and further the accurate object scoring of the user on the object is obtained. Interaction speculation module the method uses classical DNN networks. The DNN network can effectively learn the characteristic distribution and model the nonlinear relationship between the user and the article. The interaction presumption module directly inputs the interaction characteristics into the DNN network to obtain the predictive score of the user on the object, and then models the predictive score of the model into the probability expression of the user on the target object through sigmoid normalization processing. The mathematical expression is as follows:

g₁＝z_ij (8)

g₂＝σ(W₁·g₁+b₁) (9)

g₃＝σ(W₂·g₂+b₂) (10)

r′_ij＝sigmoid(W₃·g₃) (11)

Wherein W₁,W₂,W₃ represents a parameter matrix of the DNN network, b₁,b₂ represents a deviation coefficient in the DNN network, sigma represents a nonlinear activation function, and Relu activation function is adopted in the method. g₁,g₂,g₃ is the interaction vector expression output by each layer of the DNN network. r'_ij is the final probability prediction evaluation value expression obtained after sigmoid normalization.

Step five: to optimize the parameters of the model and the user item node feature expression. According to the method, the fitting degree of the cross entropy loss function modeling model is constructed, and the loss function value is minimized through a random gradient descent algorithm, so that the effect of optimizing the model parameters and node characteristics is achieved. Wherein, the mathematical expression of the cross entropy loss function is:

Where |O| represents all user item node pairs extracted from the undirected graph during model training, and r_ij represents the tag values of the target user u_i and the item v_j to be recommended, and the value range is {0,1}. A label of 0 indicates that the item v_j attribute information does not conform to the preference information of the user u_i as a negative sample of model training. A label of 1 indicates that user u_i has interactive behavior with item v_j, which is positive sample data for model training. r'_ij represents the predictive score of the model. The difference between r'_ij and the actual label r_ij is minimized through a random gradient descent algorithm, so that the loss function value is optimized, the parameter matrix of the model is optimized, and the user preference information and the article attribute information are effectively extracted.

Step six: after model training is completed, the algorithm of the present invention was tested on dataset huaban and Amazon-Book in order to verify the effectiveness of the present invention. After model training is completed, the predictive rating of the negative-sample participation model is randomly acquired at a ratio of 1:100 for the test set of each data set. Meanwhile, in order to ensure the effectiveness of model evaluation, the negative samples collected by the test set do not participate in training in the training set. After the predictive evaluation values of the model on all the articles of each user are obtained, sorting the articles participating in the evaluation from large to small according to the output value of the model, and recommending the top N articles after sorting to the target user through a top-N recommendation machine. And the effectiveness of the method is compared through effective evaluation indexes. Tables 1 and 2 show the algorithm of the present invention in comparison to the partial front recommendation algorithm recall, which can be seen to be superior to the other recommendation algorithms shown.

Table 1: performance comparison on Amazon-book dataset

Table 2: performance contrast on huaban dataset

Claims (1)

1. The recommendation method based on the graph interaction network is characterized in that the following modules are designed and applied: the system comprises a user preference information and article attribute information optimizing module, a U-I interaction characteristic extracting module, an interaction conjecture module and a Top-N recommending module;

Firstly, carrying out graph structural modeling on user and article relation data; the user preference information and article attribute information optimizing module extracts user characteristics and article characteristic expression by stacking two average convolution layers; the U-I interaction feature extraction module utilizes a graph attention mechanism to fuse target user features and to-be-recommended object features to obtain implicit feature expression on a graph interaction network, namely interaction vectors of the U-I relation pairs; the interaction presumption module learns the distribution rule of the interaction feature vector through the DNN network, models the nonlinear relation among U-I, and obtains the prediction score of the user on the target object; finally, sorting the target objects through a top-N recommendation module to realize personalized recommendation tasks for users;

1) User preference information and item attribute information optimization

The first module is a user preference and item attribute feature optimization module, and the purpose of user preference information modeling is to obtain accurate user preference feature representation according to user interaction behaviors; obtaining user and article characteristic expression through two average convolution layers; wherein the distribution of the user preference and the object attribute are more similar in the feature space;

firstly, carrying out graph structural modeling on the U-I interaction data, and establishing a connection relationship between user items with interaction behaviors; then initializing user preference characteristics and article attribute characteristics, wherein the user characteristics and the article characteristics are high-dimensional vectors of 128 dimensions; optimizing feature representation of a user and an object through a mean value convolution layer, wherein each layer of target node feature learning is obtained by carrying out averaging processing on the basis of a layer of neighbor node features on the node and the node features, and the node feature dimensions of each layer of mean value convolution layer are the same;

2) U-I interaction feature extraction module

The second module is a U-I interaction characteristic extraction module for extracting interaction characteristic expression between the target user and the object to be recommended; the U-I interaction feature extraction module directly utilizes an attention mechanism network on a graph interaction network, and fuses the target user feature, the first-order neighbor feature thereof, the item feature to be recommended and the first-order neighbor feature thereof to obtain the interaction feature expression thereof;

firstly, splicing user characteristics and user first-order neighbor node characteristics, and splicing article characteristics to be recommended and first-order neighbor node characteristics thereof; inputting the spliced characteristic into a self-attention network to obtain attention coefficients of the module, wherein the self-attention network adopts two-layer full-connection layer network modeling; finally, the corresponding U-I target node characteristics and the neighbor node characteristics thereof are aggregated through the attention weight coefficients obtained through learning to obtain a final interaction characteristic vector;

3) Interactive speculation module

The third module is an interaction presumption module, which learns the information distribution of the U-I interaction characteristics and calculates the presumption score of the user on the target object; fusing characteristic information of a user and an article through a classical DNN network to obtain a presumption result;

After the interactive feature expression of the user and the target object is obtained, directly inputting the interactive vector into a DNN network to obtain the preliminary prediction of the model; carrying out normalization processing on the module predicted value through a sigmoid function, and modeling the predicted score of the target object by the user into a preference probability expression;

4) Top-N recommendation module

The last module is a top-N recommendation module, and after the evaluation values of all the articles in the list to be recommended by the target user are predicted; sorting all the articles in descending order according to the scores, recommending the first N articles to the user, and realizing personalized recommendation of the user;

The specific implementation steps are as follows:

firstly, carrying out graph structural modeling according to U-I interaction data; establishing a connection relationship between users and articles with interaction records, wherein each user and article exist in the form of nodes of an undirected graph;

1) User preference information and item attribute information modeling

After the structural modeling of the graph is completed, randomly initializing the node characteristics of the U-I relation graph, wherein any node characteristic is a 128-dimensional high-dimensional vector; according to the connection relation of the nodes of the undirected graph, two layers of mean value convolution layer networks are built, all node characteristics in each layer of mean value convolution layer network are obtained by polymerizing the node characteristics and first-order neighbor node characteristics in the previous layer of network, wherein the polymerization mode is an averaging treatment, and the mathematical expression of the polymerization mode is as follows:

Wherein the method comprises the steps ofFeatures representing user node u of the K-th mean convolution layer; n (u) represents a first-order item neighbor node of the user node u; n (v) represents a first-order set of user neighbors of item node v; /(I)Features representing item node v of the K-th mean convolution layer; MEAN represents the averaging process, i.e., the average value of each dimension of the relevant U-I feature is calculated;

2) U-I interaction feature extraction module

After the modeling of the user object features is completed, aiming at a target user and an object node to be recommended in the undirected graph, a graph attention mechanism is fused, and the user object features and first-order neighbor features thereof are aggregated to obtain interactive feature expression;

Firstly, splicing user characteristics and user first-order neighbor node characteristics, splicing article characteristics to be recommended and first-order neighbor node characteristics thereof, inputting the spliced characteristics into a self-attention network to obtain attention coefficients corresponding to the user and article node characteristics, and carrying out softmax normalization processing on the parameters; the drawing force network adopts two fully-connected layers for modeling; attention coefficient modeling mathematical expression is:

Wherein W₁,W₂ represents the parameter matrix of the first layer and the second layer of the two-layer attention network, b₁,b₂ represents the deviation coefficient of the first layer and the second layer of the two-layer attention network, sigma represents a nonlinear activation function, and Relu activation function is adopted; the node characteristics of a target user u_i in the undirected graph are shown; h_a is a feature of a node a, where a node a is any node in the set of the user u_i and its first-order neighbor N (i); n (i) represents a first-order set of article neighbors for user u_i; /(I)Representing a splicing process; /(I)Representing the node characteristics of the item v_j to be recommended; h_b is the feature of node b, which is any node in the set of item v_j and its first-order neighbor N (j); n (j) represents a first-order set of user neighbors of item v_j; /(I)AndRepresenting a weight coefficient preliminarily obtained by the characteristics obtained by splicing through attention networks; then carrying out softmax normalization processing to obtain attention coefficients alpha_ia of the target user and the first-order neighbors thereof and attention coefficients beta_jb of the articles to be recommended and the first-order neighbors thereof;

After attention coefficients of the user, the first-order neighbors of the user and the target object and the first-order neighbors of the target object are obtained, the attention coefficient weighted target node characteristics are fused to obtain final interactive characteristic expression; the mathematical expression is as follows:

z_ij is the interactive feature expression of the obtained target user u_i and the object v_j to be recommended;

3) Interactive speculation module

Inputting the interaction characteristics into a DNN (digital network) to obtain a predictive score of the object by a user, and modeling the predictive score of the model into a probability expression of the object by the user through sigmoid normalization processing; the mathematical expression is as follows:

g₁＝z_ij (8)

g₂＝σ(W₁·g₁+b₁) (9)

g₃＝σ(W₂·g₂+b₂) (10)

r_ij'＝sigmoid(W₃·g₃) (11)

Wherein W₁,W₂,W₃ represents a parameter matrix of the DNN network, b₁,b₂ represents a deviation coefficient in the DNN network, sigma represents a nonlinear activation function, and Relu activation function is adopted; g₁,g₂,g₃ is the interactive vector expression output by each layer of DNN network; r_ij' is the final probability prediction evaluation value expression obtained after sigmoid normalization;

4) Top-N recommendation module

After the predictive evaluation values of the model on all the articles of each user are obtained, sorting the articles participating in the evaluation from large to small according to the output value of the model, recommending the first N articles after sorting to the target user through a top-N recommendation machine, and further completing personalized recommendation of the user.