Disclosure of Invention
The invention provides a big data-based hand-tour safety platform construction method which comprises the steps of collecting hand-tour client data, hand-tour server data and hand-tour crawler data, storing the collected data into a distributed file system, and training the collected data by adopting a tree integration model and a neural network model to respectively generate hand-tour safety models for identifying abnormal behaviors in games.
Further, the step of collecting the hand-tour client data, the hand-tour server data and the hand-tour crawler data comprises the steps of integrating a hand-tour security SDK in the hand-tour client to collect the client data, and collecting the server data by adopting a direct flow protocol analysis or mirror image tool mode for the hand-tour server. And monitoring the game forum and the security technology website forum in real time by utilizing a crawler technology, and analyzing the hand-tour security data. The collected data is collected by a distributed log collection system, and is transmitted and stored in a distributed file system in an encrypted or compressed form. The distributed log data collection system is characterized in that a plurality of nodes are deployed in the distributed file system, so that the data throughput efficiency is improved and the single-point problem is solved.
The training sample is manually marked as an abnormal behavior sample and a normal behavior sample according to experience of operation and maintenance personnel, the training tree integrated model or the neural network model is trained according to the behavior sample after the manual marking, the trained model is evaluated according to a corresponding preset model measurement index, the evaluation is passed and is used as a hand game safety recognition model, and the hand game safety recognition model carries out game behavior abnormal recognition on the standardized behavior characteristics extracted from hand game client data, hand game server data and hand game crawler data acquired in real time by adopting the characteristic extraction module.
The construction method further comprises the steps of correlating the safety result of the specific game behaviors identified by the game safety identification model with corresponding standardized behavior characteristics, storing the safety result in an identification behavior sample library, and periodically retraining the tree integration model and the neural network model according to newly-added sample data in the identification behavior sample library to generate a new game safety model for identifying abnormal behaviors in the game, and iterating the old game safety model by using the new game safety model.
Further, the tree integration model includes one or more of random forest, xgbt, catgboost, and lightgb models. Grid search is employed for some of the main parameters in the tree integration model, such as n_ estimators, max_depth, and min_sample of files, to explore the super-parametric space. In particular, the scope of the exploration space needs to be adjusted according to the specific game. The neural network model is preferably a deep neural network model using a fully connected network, wherein the number of layers (depth) of the network, the number of unit cells of the hidden layer (network layer scale), are all adjusted according to the specific game.
The invention further provides a hand-tour safety platform system based on big data, which comprises a data acquisition module, a data storage module, a model training module and a model online module, wherein the data acquisition module is used for acquiring hand-tour client data, hand-tour server data and hand-tour crawler data, the data storage module is used for transmitting and storing the data acquired by the data acquisition module to a distributed file system, the model training module is used for training a tree integrated model and a neural network by adopting the acquired data stored in the distributed file system to respectively generate hand-tour safety models for identifying abnormal behaviors in games, and the model online module is used for setting the hand-tour safety identification models on a wind control server to operate and identifying the abnormal behaviors in the games in real time.
The specific implementation details of the hand-tour safety platform system correspond to the details in the hand-tour safety platform construction method based on big data.
The beneficial effects of the invention include:
1. Compared with the existing hand-tour safety recognition system, the hand-tour safety recognition system sets rules according to experience, performs operation analysis by collecting multidimensional data such as client data, server data, hand-tour safety crawler data and the like, synthesizes related information in different fields, considers interaction among different features, builds a hand-tour safety intelligent recognition model through machine learning, can recognize the plug-in of an unknown mode, and provides early warning for hand-tour safety problems or anomalies;
2. the invention adopts big data technology stack technology, improves the processing capacity of burst big data volume, and effectively solves the problems of reliability, expandability and the like;
3. the method can realize the feature classification and pretreatment of the data of different channels, can carry out the normalization and integration operation on the data with non-uniform standards, solves the problem of data island, and enhances the multiplexing of heterogeneous data;
4. The invention adopts a distributed file system to disperse access pressure to each node, has no obvious transmission bottleneck, supports online capacity expansion, avoids the problems of shutdown maintenance and the like, and solves the problems of single-point failure and expansion.
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects solved by the invention more clear, the invention is further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
FIG. 1 illustrates a method of constructing a hand-tour security platform of the present invention. The method for constructing the hand-tour safety platform comprises the following steps:
step 100, collecting hand tour client data, hand tour server data and hand tour security crawler data.
Integrating a hand game safety SDK in a hand game client to collect client data. The client data includes client hardware information (e.g., client IP address, MAC address, etc.), hand-in-hand information, sensor status information, network information, etc. The client data to be collected may notify the user in the game privacy protocol.
And collecting server data by adopting a direct flow protocol analysis or mirror image tool mode for the hand-stream server. Direct streaming protocol communication analysis may be performed by either end-of-traffic mirroring tools at the hand-streaming server or integrating the relevant SDKs at the hand-streaming server. The protocol communication data can be subjected to flow calculation analysis in real time by adopting a flow analysis tool and imported into a hand-tour database. Wherein data already stored in the hand-tour database may also be imported for analysis in real time.
And (3) monitoring related game forums, safety technology website forums and the like in real time by utilizing a crawler technology, analyzing hand-tour safety data, and importing the hand-tour safety data into a hand-tour client big data platform.
Step 200, the collected data is transmitted and stored to a distributed file system. The collected data can be collected and transmitted by adopting a distributed log collection system (such as a Flume and the like), so that high-reliability collection of data of a large number of different data sources is realized.
The distributed log collection system may include a proxy point layer, a collection node layer, and a storage layer, each of which may be horizontally scalable. The agent point and the collection node are uniformly managed by the main node, so that the system is easier to monitor and maintain. Alternatively, multiple master nodes may be allowed and set up, avoiding single point failure problems. Optionally, the data is transmitted in a compressed manner, for example, gzip, bzip2, and the like, so as to reduce network traffic. In addition, the data can be transmitted in an encrypted mode, so that sensitive data is prevented from being intercepted or tampered.
The collected data is transferred and stored to a hand-tour secure big data platform distributed file system (e.g., HDFS, etc.). The distributed file system adopts a master-slave structure model, and a cluster consists of a name node (NameNode) and a plurality of data nodes (DataNode). The name node serves as a main server and manages the name space of the file system and the access operation of the client to the file. The data nodes in the cluster manage the stored data. The distributed file system provides high fault tolerance and high throughput data access, providing high reliability underlying storage support.
The data stored in the distributed file system may also include firewall data or firewall related data. Some or all of the hand-tour client data, hand-tour server-side data, hand-tour database data, firewall data, security intelligence data (e.g., hand-tour security crawler data), etc. are stored in the distributed file system after formatting.
And 300, training data by using a tree integration model and a neural network, and evaluating and determining a hand-tour safety model. The method comprises the steps of collecting data, utilizing a feature extraction module to extract behavior features from the collected data and standardizing the behavior features to obtain training samples, manually marking the training samples into abnormal behavior samples and normal behavior samples according to experience of game operators, training a training tree integrated model or a neural network model according to the behavior samples after manual marking, evaluating the trained model according to a corresponding preset model measurement index, and taking the evaluated model as a hand game safety recognition model to conduct online, wherein the hand game safety recognition model carries out abnormal game behavior recognition on the standardized behavior features extracted from hand game client data, hand game server data and hand game crawler data which are collected in real time by the feature extraction module.
Wherein the hand-tour client data may include client IP addresses, MAC addresses, etc. The hand-tour database data includes identification data, character base data, spatiotemporal data, and game behavior data. The identification data includes account name, account ID, zone ID, game character name, and the like. The character basic data includes player level, occupation, player meeting name, and the like. The spatiotemporal data includes a location map ID, coordinate information, a time stamp, and the like. Game behavior data includes character unique identifiers, behavior types (e.g., login, attack, exit, etc.), time stamps, etc., as shown in table 1 below.
| Id | Action_type | Time_stamp |
| A | Log_in | 2020-04-16 12:20 |
| B | attack | 2020-04-16 12:21 |
| A | Log_out | 2020-04-16 12:22 |
Table 1 game behavior data
And then, extracting behavior characteristics from the acquired data by utilizing a characteristic extraction module, and normalizing the behavior characteristics to obtain a training sample. A time window is set to count the data. For example, a daily window statistic of 5 minutes per unit time may be set. The time window may be set according to the desired service. According to the game service, the data will be divided into character basic data, game client data and character behavior data. The character basic data and the game client data can be directly used within a set time window. The character behavior data needs to count each feature of the character in the basic unit time, and then obtain descriptive statistics of each feature according to the count. Descriptive statistics of the feature include aggregate, mean, quartile, and maximum. Statistics of these features and role base data and client data make up a feature space for training the model. According to different game services, different fields, feature types and corresponding feature processing modes are set. The feature type can be set as continuous feature or classified feature, and the pretreatment mode of the feature can be set as no-treatment, digital sequential coding, single-heat coding and standardized pretreatment. Preferably, in the tree model, the continuous characteristic data adopts a non-processing mode, the classified characteristic data adopts digital sequential coding, and in the neural network model, the continuous characteristic data adopts standardized processing, and the classified characteristic data adopts independent thermal coding.
The data preprocessing scheme under an exemplary tree model is shown as follows:
Character basic data
| Feature names | Feature type | Feature processing |
| Game level | Continuous type | Without any means for |
| Occupation of | Classification | Digital sequential encoding |
| Whether to join a family | Classification | Digital sequential encoding |
| Maximum time for single login | Continuous type | Without any means for |
| Total duration of activity | Continuous type | Without any means for |
| Number of logins | Continuous type | Without any means for |
Character behavior data
Game client data
| Feature names | Feature type | Feature processing |
| Different ip numbers of clients | Continuous type | Without any means for |
| Number of different mac addresses at client | Continuous type | Without any means for |
The data preprocessing scheme under an exemplary neural network model is shown as follows:
Character basic data
| Feature names | Feature type | Feature processing |
| Game level | Continuous type | Normalization |
| Occupation of | Classification | Single hot coding |
| Whether to join a family | Classification | Single hot coding |
| Maximum time for single login | Continuous type | Normalization |
| Total duration of activity | Continuous type | Normalization |
| Number of logins | Continuous type | Normalization |
Character behavior data
Game client data
| Feature names | Feature type | Feature processing |
| Different ip numbers of clients | Continuous type | Normalization |
| Number of different mac addresses at client | Continuous type | Normalization |
Wherein, the data is normalized according to the following formula (1):
Wherein xinitial is the current value of the variable, xmean is the mean value of the variable, and xstd is the standard deviation of the variable.
Step 303, training the tree integration model and the neural network model, and checking and determining the hand-tour safety recognition model by using model measurement indexes.
Training the tree integration model requires training random forest, xgbt, catgboost, and lightgb models, respectively, and then weighted averaging the individual model results as an output tree integration model. The main parameters in the tree integration model (e.g., n_ estimators, max_depth, and min_sample of files, etc.) may employ a grid search method to explore the super-parametric space. In particular, the exploration space range may be adjusted according to a particular game. The neural network model uses a fully connected neural network model. The number of layers (depth) of the neural network, the number of unit elements of the hidden layer (network layer scale) can be adjusted according to the specific game.
And determining a final hand-tour safety recognition model according to the model measurement indexes. Model metrics include performance scores, precision, recall, etc. Roc _ auc _score may be used to measure the performance of the model. And the final model is determined by comprehensively considering the precision and recall rate according to the service requirement. The generated hand-tour safety recognition model can recognize hand-tour plug-in, a gold stamping working room, DDOS attack, hand-tour account safety, hand-tour information safety and the like.
And periodically retraining the tree integration model and the neural network model according to newly-added sample data in the recognition behavior sample library to generate a new hand game safety model for recognizing abnormal behaviors in the game, and iterating the old hand game safety model by using the new hand game safety model. In general, player behavior data varies with version, i.e., the data distribution changes with version changes. The data for model training is time-efficient. The model is trained by default using data in the last half year, updated every month. When the model is updated in an iteration mode, the sample label is updated in time according to the model prediction result and manual verification so as to improve the performance of the model.
The method for constructing the hand-tour safety platform further comprises the step of displaying alarm information and alarming. The audience needing game safety data can register and subscribe relevant alarm information, and relevant data such as prediction data is acquired in real time, so that game service prospects are predicted, and a game service module is adjusted according to the prediction information to better serve game services.
The invention also provides a hand tour safety recognition system. As shown in fig. 2, the system includes:
The game machine comprises a data acquisition module, a data storage module, a model training module and a model online module, wherein the data acquisition module is used for acquiring hand game client data, hand game server data and hand game crawler data, the data acquisition data storage module is used for transmitting and storing the data acquired by the data acquisition module to a distributed file system, the model training module is used for training a tree integrated model and a neural network by adopting the acquired data stored in the distributed file system to respectively generate hand game safety models for identifying abnormal behaviors in a game, and the model online module is used for setting the hand game safety identification models on a wind control server to operate and identifying the abnormal behaviors in the game in real time.
The data acquisition module comprises a client data acquisition module, a server data acquisition module and a hand-tour safety crawler data acquisition module, wherein the client data acquisition module is used for acquiring hand-tour client data by using a hand-tour SDK, the server data acquisition module is used for acquiring hand-tour server data in a direct flow protocol analysis or mirror image tool mode, and the hand-tour safety crawler data acquisition module is used for acquiring data related to hand-tour safety business on the Internet by using a crawler technology. The data collection and storage module collects data by adopting a distributed log collection system, encrypts and/or compresses the collected data, and then transmits and stores the data into the distributed file system, wherein the distributed log data collection system and the distributed file system are provided with a plurality of nodes for solving the single-point problem and providing data throughput efficiency.
Further, the system also comprises a recognition result database and a model iteration module. The recognition behavior sample library is used for carrying out association storage on results generated by abnormal behavior recognition on the data acquired by the data acquisition module in real time by the hand tour safety recognition model and corresponding standardized behavior characteristics, wherein before the abnormal recognition is carried out on the data acquired by the data acquisition module by the hand tour safety recognition model, the behavior characteristics of the data acquired by the data acquisition module are extracted and standardized by the characteristic extraction module; the model iteration module is used for retraining the tree integration model and the neural network model according to the newly added sample data in the recognition behavior sample library at regular intervals to generate a new hand game safety model for recognizing abnormal behaviors in the game, and the old hand game safety model is iteratively updated by using the new hand game safety model.
The specific implementation details of the hand-tour safety platform system correspond to the details in the hand-tour safety platform construction method based on big data.