Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
It should be noted that, in the embodiments of the present application, some existing solutions in the industry such as software, components, models, etc. may be mentioned, and they should be regarded as exemplary, which is only for illustrating the feasibility of implementing the technical solution of the present application, but does not mean that the present disclosure has or must use the solution.
In the technical solution of the present disclosure, the related user information (including, but not limited to, user personal information, user image information, user equipment information, such as location information, etc.) and data (including, but not limited to, data for analysis, stored data, displayed data, etc.) are information and data authorized by the user or sufficiently authorized by each party, and the related data is collected, stored, used, processed, transmitted, provided, disclosed, applied, etc. in compliance with relevant laws and regulations and standards, necessary security measures are taken, no prejudice to the public order colloquia is provided, and corresponding operation entries are provided for the user to select authorization or rejection.
In the scenario of using personal information to make an automated decision, the method, the device and the system provided by the embodiment of the disclosure provide corresponding operation inlets for users to choose to agree or reject the automated decision, and enter an expert decision flow if the users choose to reject. The expression "automated decision" here refers to an activity of automatically analyzing, assessing the behavioral habits, hobbies or economic, health, credit status of an individual, etc. by means of a computer program, and making a decision. The expression "expert decision" here refers to an activity of making a decision by a person who is specializing in a certain field of work, has specialized experience, knowledge and skills and reaches a certain level of expertise.
In the related art, the alarm rule used is a static threshold alarm, an engineer or an operation and maintenance personnel sets a fixed threshold value for each index to define the normal water level of the index, and when the actual value of the index exceeds the threshold value, the corresponding engineer receives the alarm. Or storing the data corresponding to the acquired index into a database or a cloud storage space, generating an abnormal event according to the configured threshold static rule, matching the configured alarm strategy based on the event level, generating an alarm notification based on the alarm strategy, and sending an alarm notification object according to the configured alarm notification rule.
The early warning method in the related art cannot cope with the normal data fluctuation in the business peak under the complex and dynamic data change environment, and rationalizes suggestions based on the historical trend and the future trend of the data. Because the setting of the static threshold value makes the traditional safety pre-warning method not suitable for the novel and unknown system environment, a large number of false positives and false negatives are easy to generate.
In order to solve the problem, the embodiment of the disclosure provides a safety early warning method, which comprises the steps of extracting information from equipment state data, communication state data and service data to obtain a knowledge graph representing a communication relationship between target equipment and a plurality of other equipment, carrying out time sequence analysis on the equipment state data and the communication state data to obtain sequence data representing the change of the performance of at least one target equipment along with time, carrying out format conversion on the service data to obtain text data representing a service processing process, carrying out feature extraction on the knowledge graph, the sequence data and the text data respectively according to a feature extraction method matched with a data type to obtain feature sets, carrying out feature fusion on the feature sets to obtain target features, and carrying out safety early warning analysis on the target features to obtain an early warning analysis result.
Fig. 1 schematically illustrates an application scenario diagram of a security early warning method, apparatus, device, medium and program product according to an embodiment of the disclosure.
As shown in fig. 1, an application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium used to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages etc. Various communication client applications, such as a shopping class application, a web browser application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.
The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the security pre-warning method provided by the embodiments of the present disclosure may be generally executed by the server 105. Accordingly, the security early warning device provided in the embodiments of the present disclosure may be generally disposed in the server 105. The security early warning method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105. Accordingly, the security early warning apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The safety warning method of the disclosed embodiment will be described in detail with reference to fig. 2 to 4 based on the scenario described in fig. 1.
Fig. 2 schematically illustrates a flow chart of a security early warning method according to an embodiment of the disclosure.
As shown in FIG. 2, the full warning method of this embodiment includes operations S210-S260.
In operation S210, information extraction is performed on the device status data, the communication status data, and the service data, so as to obtain a knowledge graph of the communication relationship between the target device and the plurality of other devices.
In operation S220, time sequence analysis is performed on the device status data and the communication status data to obtain time-varying sequence data of the performance of the target device.
In operation S230, format conversion is performed on the service data to obtain text data representing the service processing procedure.
In operation S240, feature extraction is performed on the knowledge graph, the sequence data, and the text data according to a feature extraction method that matches the data type, respectively, to obtain a feature set.
In operation S250, feature fusion is performed on the feature set to obtain a target feature.
In operation S260, a security early warning analysis is performed on the target feature, so as to obtain an early warning analysis result.
According to embodiments of the present disclosure, device status data, communication status data, and traffic data may be collected by a data collection interface and a sensor network pre-deployed on a target device. The target device is a device to be subjected to early warning analysis, and the target device may be a hardware device of an IT system of a load financial institution. The other devices are devices that have a communication relationship with the target device during the processing of the business by the financial institution.
According to embodiments of the present disclosure, the device state data may include a plurality of hardware metrics, which may include processor occupancy, memory occupancy, disk read-write rate, and the like. The communication status data may include a connection status, a communication delay, a communication bandwidth, etc. of each communication between the target device and the other device. The business data is business processing records recorded on the target equipment in the business processing process, and the business data can comprise transaction data and system logs.
According to the embodiment of the disclosure, information extraction is performed on the collected device state data, communication state data and service data to obtain a communication relationship among a plurality of devices in a service system, and the communication relationship is represented in the form of a knowledge graph. And carrying out time sequence analysis on the equipment state data and the communication state data to acquire sequence data including data such as processor occupancy rate, communication bandwidth and the like in the equipment state data and the communication state data. Because the information stored in the system such as the transaction log is recorded in the service data, the text data related to the transaction log stored in the system is acquired by performing format conversion on the service data, so that the characteristics related to the data are conveniently extracted for safety early warning analysis in the subsequent data processing process.
According to the embodiments of the present disclosure, device status data, communication status data, and service data may be acquired through a data detection and acquisition tool previously set on a data acquisition point or data contact, for example, a target device such as a terminal server, in order to acquire device status data, communication status data between the target device and other devices, and service data remaining in a storage medium of the target device.
According to an embodiment of the present disclosure, the sequence data is data characterized in a time sequence form, for example, data between the time node a and the time node B, such as a memory occupancy rate, a disk read-write rate, and a communication bandwidth, which may be represented by the time sequence. The text data is data in a text format, for example, text content is recorded in a system log. Knowledge-graph characterization data in the form of graphs.
According to the embodiment of the disclosure, the feature extractors corresponding to the knowledge graph, the sequence data and the text data are trained respectively, and feature extraction is carried out respectively by adopting feature extraction methods respectively matched with the sequence data, the text data and the knowledge graph to obtain the corresponding feature set.
According to the embodiment of the disclosure, a plurality of features in the feature set are fused to obtain the target feature comprising a plurality of feature dimensions, so that the environmental state of the target device is represented by the target feature. The early warning analysis results are used for early warning of events which may be generated in the future and which threaten the safety of equipment, such as long-term overheat of a processor, slow response of an IT system and the like. The machine learning technology can be utilized to perform safety early warning analysis on the target characteristics, so that early warning analysis results of target equipment are obtained, and safety events possibly generated can be conveniently handled by operation and maintenance personnel based on the early warning analysis results. The early warning analysis result may include at least one of a plurality of information such as prediction data, an early warning type, etc. for each index.
According to the embodiment of the disclosure, information extraction is performed on equipment state data, communication state data and service data to obtain a knowledge graph representing a communication relationship among a plurality of pieces of equipment, time sequence analysis is performed on the equipment state data and the communication state data to obtain sequence data representing the performance change of at least one target equipment along with time, format conversion is performed on the service data to obtain text data representing a service processing process, the equipment state data, the communication state data and the service data are converted into three types of data including the knowledge graph, the sequence data and the text data, and multidimensional representation and multidimensional analysis are facilitated on the equipment state data, the communication state data and the service data so as to improve the richness of information and the accuracy of early warning results. And capturing the characteristics of the data in multiple dimensions from the three data types according to the characteristic extraction method matched with the data types by the obtained sequence data, text data and knowledge graph to obtain a characteristic set. And carrying out feature fusion on the feature set to obtain target features representing the current environment state, and carrying out safety early warning analysis by utilizing a plurality of dimension features included in the target features to obtain early warning analysis results of target equipment, so that the early warning accuracy is improved, and the false alarm rate are reduced.
According to the embodiment of the disclosure, information extraction is performed on equipment state data, communication state data and service data to obtain a knowledge graph representing communication relations among a plurality of pieces of equipment, wherein the knowledge graph comprises obtaining nodes in the knowledge graph based on equipment identifiers in the equipment state data, and obtaining association relations among the plurality of pieces of equipment as edges in the knowledge graph based on equipment links used for processing services in the communication state data and the service data.
Fig. 3 schematically illustrates a flow chart of determining a knowledge-graph, according to an embodiment of the disclosure.
As shown in fig. 3, the device state data 301 includes a plurality of device identifiers, and the device identifiers in the device state data are used as nodes 302 of a knowledge graph. Determining whether a connection relationship exists between a node corresponding to other devices and a node corresponding to a target device according to the connection state or communication bandwidth and other data between any two devices or interfaces of the two devices in the communication state data 303, and determining the communication direction of the node with the connection relationship in the knowledge graph according to the device link for processing the service included in the service data 304, thereby obtaining the association relationship between a plurality of devices to determine the edge 305 of the knowledge graph. Knowledge graph 306 composed of directed edges is obtained based on nodes 302 of the knowledge graph and edges 305 of the knowledge graph, and the knowledge graph composed of the directed edges can be used for extracting data transmission directions between devices.
According to the embodiment of the disclosure, in the knowledge graph 306 formed by the directed edges, the communication direction between the nodes represents the data circulation direction in the service processing process, and the characteristics related to the service processing sequence of the nodes can be extracted from the knowledge graph 306 formed by the directed edges, so that the characteristic set is enriched, and the target characteristics are enriched, so that the accuracy of the safety early warning is improved.
According to the embodiments of the present disclosure, the communication status data is used only to record communication data between other devices that communicate with the target device, for example, communication bandwidth between a personal computer for business transaction and a self-service terminal or the like in an IT system of a financial institution. The service data records the device links of all the nodes involved in the processed transaction of the target device in the transaction service processing process.
According to the embodiment of the disclosure, even if no device link capable of representing the connection relationship between two devices exists in service data, that is, the transaction service processing relationship may not be related between the two devices, other data transmission may exist between the two devices, so that a certain communication relationship exists, therefore, a part of edges in a knowledge graph can be determined based on communication state data, another part of edges in the knowledge graph can be determined based on the device link used for processing services in service data, and the edges of the knowledge graph are supplemented to obtain the knowledge graph formed by undirected edges. The knowledge graph formed by undirected edges can be used for extracting characteristics of connection relation between the target equipment and other equipment in the service link.
According to the embodiment of the disclosure, the association relationship between the target device and the plurality of other devices is obtained based on the communication state data and the device link for processing the service in the service data. The side attribute information of the knowledge graph formed by the undirected graph can be determined based on the device links for processing the service in the service data, wherein the side attribute information is used for representing the service processing times between nodes or the communication times generated by the service processing in a preset period. And the subsequent extraction of the characteristics about the side attribute information is facilitated, and the subsequent characteristic fusion and safety early warning are carried out, so that the accuracy of the safety early warning is improved.
According to the embodiment of the disclosure, the knowledge graph is constructed through the equipment state data, the communication state data and the service data, the association relation among a plurality of pieces of equipment is reserved by utilizing the characteristics of the knowledge graph, and a new feature extraction dimension is provided for safety early warning, so that the accuracy of the safety early warning is improved.
According to the embodiment of the disclosure, the time sequence analysis is performed on the device state data and the communication state data to obtain the sequence data representing the time-varying performance of at least one target device, and the method further comprises the step of respectively arranging the device state data and the communication state data according to the time sequence to obtain the sequence data about the device and the sequence data about the communication.
According to an embodiment of the present disclosure, in order to save resources, a data acquisition task is generally performed periodically, and based on the acquisition time of data in the device status data and the communication status data, the device status data and the communication status data are respectively ordered to obtain sequence data about the device, and sequence data about the communication, so as to analyze the sequence data about the device and the sequence data about the communication in a time sequence, where the sequence data about the device may be values of indicators such as a processor occupancy rate, a memory occupancy rate, and the like at a plurality of time nodes in a time period a, and the sequence data about the communication may be values of indicators such as a communication bandwidth, a communication latency, and the like at a plurality of time nodes in a time period a.
According to the embodiment of the disclosure, when the service data comprises various forms of data, such as structured JavaScript object representation (JavaScript Object Notation, JSON) data, xml data and the like, the structured data is subjected to format conversion to obtain text data with unified format, so that the service data is conveniently analyzed, characteristics related to service processing are acquired based on text content corresponding to the service data, the data analysis dimension of the safety early warning is improved, and the accuracy of the safety early warning is further improved.
According to the embodiment of the disclosure, the data with time relation in the equipment state data and the communication state data are arranged according to the time sequence so as to reflect the time change and trend of the equipment state data and the communication state data, and the analyzable dimension of the equipment state data and the communication state data is increased, so that the accuracy of safety early warning is improved.
According to the embodiment of the disclosure, for the collected device state data, communication state data and service data, sensitive data and non-sensitive data can be distinguished, sensitive data processing is reserved in a local private cloud, so that data security is ensured, management requirements are met, and non-sensitive data processing and analysis tasks are deployed to a public cloud platform, so that computing efficiency and expandability are improved.
Specifically, classifying the data, distinguishing the sensitive data from the non-sensitive data, the sensitive data may include business data, device status data, communication status data, etc., while the non-sensitive data may include a disclosed feature fusion algorithm, etc., retaining the sensitive data in a local private cloud, which means that both data storage and processing occur within the financial institution's own data center, the private cloud provides a higher level of security control, as it allows the financial institution to manage the data center's physical security, network security, and data access rights itself, while the non-sensitive data processing and analysis tasks are deployed to the public cloud platform and provide computing resources and services as needed.
According to an embodiment of the present disclosure, for sensitive data stored in a private cloud, the sensitive data may be data desensitized by operation S210 as in fig. 2. The data stored in the private cloud and the public cloud can be combined to obtain a data set corresponding to the service data, the device state data and the communication state data. In order to further improve the security of the data, the feature extraction may be performed on the obtained data set through operation S220, so as to obtain the feature set after the feature extraction, and reduce the probability of leakage of the sensitive data.
According to embodiments of the present disclosure, by such a hybrid cloud strategy, a financial institution may utilize the flexibility and cost effectiveness of public clouds to process non-sensitive data while maintaining sensitive data security and compliance, optimizing the configuration of technical resources.
According to the embodiment of the disclosure, the safety precaution method further comprises the step of carrying out data complementation on sub-sequence data adjacent to the missing data in the sequence data by utilizing an Auto REGRESSIVE INTEGRATED Moving Average (ARIMA) model under the condition that the missing data exists in the sequence data.
According to embodiments of the present disclosure, for periodically acquired data that is missing data over a continuous time period, where missing data is present for a period of time defined by the sequence data, the auto-regressive sum moving average model may be utilized to perform data completion on the missing data based on the sub-sequence data that has been determined to be adjacent to the missing data. The integrity of the data can be ensured by complementing the missing data, and the data with multidimensional characteristics, such as the integral change characteristics or the change trend of the sequence data, is provided for subsequent characteristic extraction, so that the accuracy of the safety pre-warning is improved.
According to the embodiment of the disclosure, according to a feature extraction method matched with a data type, feature extraction is respectively carried out on a knowledge graph, sequence data and text data to obtain a feature set, wherein the feature extraction method comprises the steps of carrying out feature extraction on the sequence data by using a visual encoder, carrying out feature extraction on the text data by using a coding decoder, and carrying out feature extraction on the knowledge graph by using a graph neural network.
According to embodiments of the present disclosure, the sequence data may be represented in a graph form, which is feature extracted using a visual encoder to identify a change feature of the sequence data over time. The visual encoder can extract the change rate of the memory occupancy rate and the number of extreme points and the change interval thereof from the sequence data, wherein the visual encoder can be obtained based on network training such as a convolutional neural network or a long-short-period memory network, so that a plurality of characteristics for early warning analysis are obtained from the sequence data, and the long-period memory network can be used for better processing the sequence data.
According to the embodiment of the disclosure, the text data comprises text type data representing a business processing process, and the text data is subjected to feature extraction through a coder-decoder to obtain features which can be used for representing the business processing, wherein the coder-decoder can be a coder-decoder of transducer architecture. For example, an error code for recording a task execution failure corresponding to a service is included in the text data, and feature extraction may be performed on the text data by a codec to obtain features corresponding to the text data that may include the error code.
According to the embodiment of the disclosure, the distribution condition of the nodes in the knowledge graph can be determined by extracting the characteristics of the knowledge graph through a graph neural network, wherein the distribution condition comprises the association condition between the nodes, and the graph neural network can be a graph rolling network (Graph Convolutional Network, GCN). For situations where communications between multiple nodes are more frequent than other nodes in the knowledge graph, it is easier to generate events that threaten network or transaction security. And extracting features of the knowledge graph through the graph neural network to extract the communication condition of the nodes in the knowledge graph, and taking the communication condition of the nodes as the features of the safety early warning.
According to the embodiment of the disclosure, for the knowledge graph including the side attribute information, the association relationship between the nodes of the knowledge graph and the variable attribute information between the nodes of the knowledge graph can be extracted through the graph neural network, so that the communication intensity of different devices and other devices is determined, the communication intensity between a plurality of devices and the target device is used as the characteristic of safety early warning, for example, when a certain device is in communication intensity with the target device and corresponding service between each communication fails, the safety risk of the communication with the device can be considered to be larger.
According to the embodiment of the disclosure, according to the feature extraction method matched with the data type, the data in the data set are respectively subjected to feature extraction to obtain the features of multiple dimensions from the data set, so that the feature set for safety early warning is enriched, and the accuracy of the model for the safety early warning is improved.
According to the embodiment of the disclosure, the data in the data set are respectively subjected to feature extraction by using the visual encoder, the encoder and the decoder and the graph neural network which are obtained through training, so that the feature set is obtained, the feature corresponding to the data which is more representative to the early warning analysis result in the data set is obtained, and the data volume of model processing is reduced. According to embodiments of the present disclosure, the graph neural network may be trained using community discovery algorithms by which to identify communication patterns in the knowledge-graph, such as abnormal network connection patterns, which help reveal potential security threats.
Fig. 4 schematically illustrates a flow chart of determining target features according to an embodiment of the disclosure.
As shown in fig. 4, the device state data 401 and the communication state data 402 are respectively processed to obtain the sequence data 403, where the sequence data 403 may be stored in multiple forms, and in order to extract data features of more dimensions as possible, the sequence data 403 may be sorted in a graph form, so as to preserve features of the sequence data 403 in multiple dimensions.
Data in the form of text in the business data 404 is acquired to determine text data 405. A knowledge-graph 406 characterizing communication relationships between the plurality of devices and the target device is determined based on the device state data 401, the communication state data 402, and the traffic data 404. The knowledge graph 406 is extracted by using the graph neural network 407, the sequence data 403 is extracted by using the visual encoder 408, the text data 405 is extracted by using the codec 409, so as to obtain a feature set 410, and a plurality of features of the feature set 410 obtained after the feature extraction are fused by using the features to determine the target feature 411.
The device state data 401, the communication state data 402 and the service data 404 are utilized to carry out multidimensional feature extraction, so that a feature set 410 is obtained, and the target device is subjected to omnibearing safety early warning based on target features 411 obtained by feature fusion of a plurality of features of the feature set 410, so that the early warning accuracy is improved.
According to the embodiment of the disclosure, feature fusion is performed on the feature set to obtain target features, wherein the feature fusion comprises the step of splicing a plurality of features in the feature set to obtain the target features. Different dimensionalities and modal characteristics can be spliced in a concat mode, so that the target characteristics are enriched, and meanwhile, the expression capability of each characteristic in the characteristic set is reserved as much as possible, and the accuracy of safety early warning is improved.
According to the embodiment of the disclosure, the security early warning analysis is performed on the target features to obtain early warning analysis results, wherein the security early warning analysis comprises at least one of performing security early warning analysis on the user behavior on the target features to obtain the user early warning analysis results, performing equipment security early warning analysis on the target features to obtain the equipment early warning analysis results, and performing network security early warning analysis on the target features to obtain the network early warning analysis results.
According to embodiments of the present disclosure, the target features may be analyzed by a pre-trained machine learning model, which may be a neural network or random forest. The machine learning model is trained by utilizing historical target features obtained from historical data, wherein the historical data is equipment state data, communication state data and business data in a historical time period. The user early warning analysis result, the equipment early warning analysis result and the network early warning analysis result are analysis results of risks possibly generated in a future time period, wherein the duration of the future time period can be preset. The pre-trained machine learning model can be utilized to perform user behavior safety early warning analysis, equipment safety early warning analysis and network safety early warning analysis on the target characteristics, and early warning analysis results are obtained.
According to the embodiment of the disclosure, for the user behavior safety precaution analysis, the equipment safety precaution analysis and the network safety precaution analysis, the safety precaution analysis can be respectively carried out by training three different machine learning models in advance. And under the condition that one or more early warning results are obtained in the early warning analysis results of the user, the equipment or the network, determining the obtained one or more early warning results as early warning analysis results.
According to the embodiment of the disclosure, the user early warning analysis result is used for early warning the threat IT system caused by the user operation. And carrying out safety early warning analysis on the user behavior by utilizing the target characteristics to obtain a user early warning analysis result, wherein the user early warning analysis result can be the probability that the operation of the user threatens an IT system.
According to the embodiment of the disclosure, the device early warning analysis result is used for early warning of a hardware device fault event which may occur, wherein the hardware device fault event may include, for example, long-term overheating of a processor caused by a long-term too high occupancy rate of the processor, and the processor fault is caused. And carrying out equipment safety early warning analysis by utilizing the target characteristics to obtain equipment early warning analysis results, wherein the equipment early warning analysis results can be the probability of threat to the operation of an IT system due to the failure of the target equipment caused by hardware.
According to the embodiment of the disclosure, the network early warning analysis result is used for early warning of possible communication link faults or communication link communication efficiency reduction. And carrying out network security early warning analysis by utilizing the target characteristics to obtain a network early warning analysis result, wherein the network early warning analysis result can be the probability of threat to an IT system caused by communication network paralysis.
According to the embodiment of the disclosure, the machine learning or related algorithm is utilized to learn the mode and the behavior from the historical data, the detection strategy is automatically adjusted to identify the abnormal behavior, the safety early warning analysis is carried out on a plurality of layers of user safety, equipment safety and network safety, a more comprehensive safety early warning view angle is provided for the system operation, the safety early warning accuracy of an IT system of a financial institution is improved, the false alarm rate is reduced to reduce the interference of the legal transaction, the customer satisfaction is improved, meanwhile, the working pressure of a safety team is reduced, the threat detection speed is improved, the time window for attack discovery can be shortened, and the potential loss is reduced.
In accordance with embodiments of the present disclosure, in training a machine learning model, the machine learning model may be fine-tuned by a cross-validation method. And verifying the performance of the machine learning model by a multi-fold cross verification method, and fine-tuning the machine learning model based on the early warning accuracy and the early warning false alarm rate of the machine learning model obtained by verification.
Specifically, the historical data for model training is divided into K subsets, each subset should represent the distribution of the whole data set as much as possible, where K is a positive integer. For each iteration, selecting one subset as a verification set, the rest K-1 subsets as training sets, training a machine learning model by using the training sets, evaluating the performance of the model on the verification set, and recording the accuracy and false alarm rate of the model. The training and validation process described above is repeated K times, each time a different subset is selected as the validation set, to ensure that each subset has the opportunity to participate in the model evaluation as the validation set. Taking the average value of the performance indexes in the K iterative processes as the final performance evaluation result of the model. And selecting a model with optimal performance as a machine learning model finally used for safety pre-warning according to the comprehensive evaluation result of the cross validation.
In accordance with embodiments of the present disclosure, in training a machine learning model, the accuracy and robustness of the machine learning model may also be verified by an a/B test method. And determining the machine learning model with good model effect based on the obtained indexes such as the false alarm rate and the response time by using the indexes of the machine learning model in the aspects of the false alarm rate and the response time.
Specifically, two machine learning models, respectively referred to as a version a model and a version B model, are trained using the same historical data and the same training method. And recording the performances of the two models in an actual running environment, wherein the performances comprise key performance indexes such as detection rate, false alarm rate, response time and the like. Statistical methods are used to analyze the performance differences of version a and version B models to ensure that any observed performance improvement is statistically significant. If the version B model shows significant performance improvement over the version A, the version B model is considered to be put into the production environment, and if the performance is not significantly improved or is inferior to the version A, the version B model is continuously optimized or a new model scheme is explored.
According to the embodiment of the disclosure, for the machine learning model for safety precaution, a meta learning algorithm can be introduced to realize the capability of the model to quickly adapt to a new environment, and a reinforcement learning simulation decision process is used to optimize a response strategy.
According to the embodiment of the disclosure, the accuracy and the robustness of the machine learning model are verified through the cross-validation and A/B test method, prediction errors caused by over-fitting or under-fitting of the model are reduced, more reliable and more effective guarantee can be provided for the IT system of the financial institution, potential security threats can be timely dealt with, a user can conveniently fine-tune model parameters and algorithms according to actual conditions, and the model is more suitable for the characteristics and requirements of the IT system of the financial institution.
According to embodiments of the present disclosure, an explanatory artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) module is built in a machine learning model and used to provide a clear explanation for each early warning, in conjunction with visualization tools, including but not limited to display screens, smart devices, to present detection data and early warning information, enabling operators to understand the decision making process of the model.
According to the embodiment of the disclosure, an online learning mechanism is adopted, a machine learning model is continuously updated to reflect the latest system state and threat information, system performance is periodically evaluated, detection strategies and model parameters are automatically adjusted according to results, and accuracy and robustness of the system can be continuously improved through feedback loops by using user feedback and real results. Specifically, the machine learning model can be continuously updated by adopting an online learning mechanism, the system performance is periodically evaluated, the detection strategy and model parameters are automatically adjusted according to the result, and the accuracy and the robustness of the system are continuously improved by utilizing the feedback of the user and the real result.
According to the embodiment of the disclosure, an online learning mechanism allows a machine learning model to continuously learn from new data in actual operation, so that the latest state and threat information of a system are reflected timely, the system can regularly evaluate the performance of the system and automatically adjust the detection strategy and model parameters according to evaluation results to ensure that the system is always in an optimal state, and a feedback loop enables the system to continuously perfect the system by using user feedback and real results to form a feedback loop, so that the accuracy and the robustness of a safety early warning method are continuously improved. In a word, through online learning mechanism and feedback cycle, a continuously improved safety early warning scheme is provided for the IT system of the financial institution, so that the accuracy and robustness of the system are ensured, and the user satisfaction is improved.
Based on the safety early warning method, the disclosure further provides a safety early warning device. The device will be described in detail below in connection with fig. 5.
Fig. 5 schematically illustrates a block diagram of a security early warning device according to an embodiment of the present disclosure.
As shown in fig. 5, the security early warning device 500 of this embodiment includes an information extraction module 510, a timing analysis module 520, a format conversion module 530, a feature extraction module 540, a feature fusion module 550, and an early warning analysis module 560.
The information extraction module 510 performs information extraction on the device state data, the communication state data, and the service data, and obtains a knowledge graph that characterizes a communication relationship between the target device and a plurality of other devices. The information extraction module 510 may be configured to perform the operation S210 described above, which is not described herein.
The time sequence analysis module 520 is configured to perform time sequence analysis on the device status data and the communication status data, so as to obtain time-varying sequence data of the performance of the target device. The information extraction module 520 may be used to perform the operation S220 described above, and will not be described herein.
The format conversion module 530 is configured to perform format conversion on the service data, so as to obtain text data representing the service processing procedure. The format conversion module 530 may be used to perform the operation S230 described above, and will not be described herein.
The feature extraction module 540 is configured to perform feature extraction on the knowledge graph, the sequence data and the text data according to a feature extraction method matched with the data type, so as to obtain a feature set. The feature extraction module 520 may be configured to perform the operation S240 described above, which is not described herein.
The feature fusion module 550 is configured to perform feature fusion on the feature set to obtain a target feature. The feature fusion module 550 may be configured to perform the operation S250 described above, which is not described herein.
And the early warning analysis module 560 is used for carrying out safety early warning analysis on the target characteristics to obtain early warning analysis results. The early warning analysis module 560 may be configured to perform the operation S260 described above, which is not described herein.
According to an embodiment of the present disclosure, the data processing module 510 includes a node determining unit, an edge determining unit.
And the node determining unit is used for obtaining the nodes in the knowledge graph based on the equipment identification in the equipment state data.
And the edge determining unit is used for obtaining the association relation among the plurality of devices based on the communication state data and the device links for processing the service in the service data, and taking the association relation as an edge in the knowledge graph, thereby obtaining the knowledge graph.
According to an embodiment of the present disclosure, the timing analysis module 520 includes a data arrangement unit.
And the data arrangement unit is used for arranging the equipment state data and the communication state data according to time sequence respectively to obtain the sequence data about the equipment and the sequence data about the communication.
According to an embodiment of the present disclosure, the security early warning device 500 further includes a data complement module.
And the data complement module is used for carrying out data complement on the subsequence data adjacent to the missing data in the sequence data by utilizing the autoregressive sum moving average model under the condition that the missing data exists in the sequence data.
According to an embodiment of the present disclosure, the feature extraction module 520 includes a first extraction unit, a second extraction unit, and a third extraction unit.
And the first extraction unit is used for carrying out feature extraction on the sequence data by utilizing the visual encoder.
And the second extraction unit is used for carrying out feature extraction on the text data by utilizing the coder-decoder.
And the third extraction unit is used for extracting the characteristics of the knowledge graph by using the graph neural network.
According to an embodiment of the present disclosure, the feature fusion module 530 includes a feature stitching unit.
And the characteristic splicing unit is used for splicing a plurality of characteristics in the characteristic set to obtain target characteristics.
According to an embodiment of the present disclosure, the early warning analysis module 540 includes at least one of a first early warning unit, a second early warning unit, and a third early warning unit.
And the first early warning unit is used for carrying out user behavior safety early warning analysis on the target characteristics to obtain user early warning analysis results.
And the second early warning unit is used for carrying out equipment safety early warning analysis on the target characteristics to obtain equipment early warning analysis results.
And the third early warning unit is used for carrying out network security early warning analysis on the target characteristics to obtain a network early warning analysis result.
According to an embodiment of the present disclosure, any of the information extraction module 510, the timing analysis module 520, the format conversion module 530, the feature extraction module 540, the feature fusion module 550, and the early warning analysis module 560 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules. At least one of the information extraction module 510, the timing analysis module 520, the format conversion module 530, the feature extraction module 540, the feature fusion module 550, and the pre-alarm analysis module 560 may be implemented, at least in part, as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or as hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or as any one of or a suitable combination of three of software, hardware, and firmware. Or at least one of the information extraction module 510, the timing analysis module 520, the format conversion module 530, the feature extraction module 540, the feature fusion module 550, and the pre-alarm analysis module 560 may be at least partially implemented as a computer program module, which, when executed, may perform the corresponding functions.
Fig. 6 schematically illustrates a block diagram of an electronic device adapted to implement a security early warning method according to an embodiment of the disclosure.
As shown in fig. 6, an electronic device 600 according to an embodiment of the present disclosure includes a processor 601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. The processor 601 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 601 may also include on-board memory for caching purposes. The processor 601 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.
In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The processor 601 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 602 and/or the RAM 603. Note that the program may be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.
According to an embodiment of the present disclosure, the electronic device 600 may also include an input/output (I/O) interface 605, the input/output (I/O) interface 605 also being connected to the bus 604. The electronic device 600 may also include one or more of an input portion 606 including a keyboard, a mouse, etc., an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc., a storage portion 608 including a hard disk, etc., and a communication portion 609 including a network interface card such as a LAN card, a modem, etc., connected to an input/output (I/O) interface 605. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to an input/output (I/O) interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
The present disclosure also provides a computer-readable storage medium that may be included in the apparatus/device/system described in the above embodiments, or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 602 and/or RAM 603 and/or one or more memories other than ROM 602 and RAM 603 described above.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code, when executed in a computer system, causes the computer system to implement the security pre-warning method provided by embodiments of the present disclosure.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and/or installed from the removable medium 611. The computer program may comprise program code that is transmitted using any appropriate network medium, including but not limited to wireless, wireline, etc., or any suitable combination of the preceding.
In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure may be combined and/or combined in various combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, features recited in various embodiments of the present disclosure may be combined and/or combined in various ways without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.