CN111597550A - A kind of log information analysis method and related device - Google Patents
A kind of log information analysis method and related deviceDownload PDFInfo
- Publication number
- CN111597550A CN111597550ACN202010407002.8ACN202010407002ACN111597550ACN 111597550 ACN111597550 ACN 111597550ACN 202010407002 ACN202010407002 ACN 202010407002ACN 111597550 ACN111597550 ACN 111597550A
- Authority
- CN
- China
- Prior art keywords
- log
- analysis
- event
- data
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/552—Detecting local intrusion or implementing counter-measures involving long-term monitoring or reporting
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本申请涉及计算机技术领域，特别涉及一种日志信息分析方法、日志信息分析装置、服务器以及计算机可读存储介质。The present application relates to the field of computer technology, and in particular, to a log information analysis method, a log information analysis device, a server, and a computer-readable storage medium.
背景技术Background technique
随着信息技术的不断发展，计算机设备在运行过程中产生的日志数据量越来越多，日志数据种类越来越多。因此，对日志数据进行分析的难度越来越高。同时，对日志数据进行分析需求也越来越多，不仅仅是技术人员需要对日志数据进行分析，非技术人员也需要根据需求对日志数据进行分析。With the continuous development of information technology, the amount of log data generated by computer equipment during the operation process is increasing, and the types of log data are increasing. As a result, it becomes increasingly difficult to analyze log data. At the same time, there are more and more demands for analyzing log data. Not only technical personnel need to analyze log data, but non-technical personnel also need to analyze log data according to their needs.
目前，现有技术中使用的SIEM(Security Information and Event Management安全信息和事件管理)对日志数据进行分析。其中，SIEM是一种软件解决方案，可以汇总和分析整个IT基础架构中不同资源的活动，从网络设备、服务器、域控制器等收集安全数据，同时对数据进行存储，规范化，汇总和分析，以发现趋势，检测威胁等。但是，采用现有的技术方案对日志数据进行分析的过程，需要技术人员熟悉代码语言，将对日志数据进行分析的需求编写为对应的可执行语句。但是，此技术方案进行分析过滤的方式门槛过高，不够灵活，造成日志分析的效率下降，日志分析的灵活性下降。Currently, SIEM (Security Information and Event Management) used in the prior art analyzes log data. Among them, SIEM is a software solution that can aggregate and analyze the activities of different resources throughout the IT infrastructure, collect security data from network devices, servers, domain controllers, etc., while storing, normalizing, aggregating and analyzing the data, to spot trends, detect threats, and more. However, in the process of analyzing the log data by using the existing technical solution, the technicians are required to be familiar with the code language, and the requirements for analyzing the log data are written into corresponding executable statements. However, the threshold for analyzing and filtering in this technical solution is too high, and the method is not flexible enough, resulting in a decrease in the efficiency of log analysis and a decrease in the flexibility of log analysis.
因此，如何提高对日志数据进行分析的灵活性是本领域技术人员关注的重点问题。Therefore, how to improve the flexibility of analyzing log data is a key issue concerned by those skilled in the art.
发明内容SUMMARY OF THE INVENTION
本申请的目的是提供一种日志信息分析方法、日志信息分析装置、服务器以及计算机可读存储介质，通过接收到的模式选择信息选择出对应的日志分析模式对接收到的条件信息进行编码生成处理，得到分析规则并对日志数据进行分析，得到目标日志事件，实现对日志信息进行分析，由于可以选择不同的日志分析模式对日志数据进行分析，提高了进行分析的灵活性。The purpose of this application is to provide a log information analysis method, a log information analysis device, a server, and a computer-readable storage medium, which select a corresponding log analysis mode through the received mode selection information to encode and generate the received condition information. , obtain analysis rules and analyze log data, obtain target log events, and analyze log information. Since different log analysis modes can be selected to analyze log data, the flexibility of analysis is improved.
为解决上述技术问题，本申请提供一种日志信息分析方法，包括：In order to solve the above-mentioned technical problems, the present application provides a method for analyzing log information, including:
根据接收到的模式选择信息选择对应的日志分析模式；Select the corresponding log analysis mode according to the received mode selection information;
根据所述日志分析模式对接收到的条件信息进行编码生成处理，得到所述日志分析模式对应的分析规则；Perform coding and generation processing on the received condition information according to the log analysis mode, to obtain an analysis rule corresponding to the log analysis mode;
根据所述分析规则对待分析的日志数据进行分析，得到该日志数据对应的目标日志事件。The log data to be analyzed is analyzed according to the analysis rules to obtain a target log event corresponding to the log data.
可选的，所述日志分析模式包括统计规则分析模式；Optionally, the log analysis mode includes a statistical rule analysis mode;
相应的，所述根据所述日志分析模式对接收到的条件信息进行编码生成处理，得到所述日志分析模式对应的分析规则，包括：Correspondingly, performing coding and generating processing on the received condition information according to the log analysis mode to obtain analysis rules corresponding to the log analysis mode, including:
从所述接收到的条件信息中提取出单一事件条件数据；其中，所述单一事件条件数据包括过滤条件和统计数据；Extracting single event condition data from the received condition information; wherein, the single event condition data includes filter conditions and statistical data;
根据所述单一事件条件数据生成所述统计规则分析模式对应的分析规则。An analysis rule corresponding to the statistical rule analysis mode is generated according to the single event condition data.
可选的，所述根据所述分析规则对待分析的日志数据进行分析，得到该日志数据对应的目标日志事件，包括：Optionally, the log data to be analyzed is analyzed according to the analysis rules to obtain a target log event corresponding to the log data, including:
根据所述分析规则中的过滤条件对所述待分析的日志数据进行过滤，得到初级日志数据；Filter the log data to be analyzed according to the filter conditions in the analysis rule to obtain primary log data;
对所述初级日志数据中的预设字段进行统计得到统计数据，将所述统计数据与所述分析规则中的统计标准进行对比，根据得到的对比结果确定所述目标日志事件。Statistics are performed on preset fields in the primary log data to obtain statistical data, the statistical data is compared with the statistical standard in the analysis rule, and the target log event is determined according to the obtained comparison result.
可选的，所述日志分析模式包括序列规则分析模式；Optionally, the log analysis mode includes a sequence rule analysis mode;
所述根据所述日志分析模式对接收到的条件信息进行编码生成处理，得到所述日志分析模式对应的分析规则，包括：The encoding and generating processing is performed on the received condition information according to the log analysis mode, and the analysis rule corresponding to the log analysis mode is obtained, including:
从所述接收到的条件信息中提取出序列事件条件数据；其中，所述序列事件条件数据包括多个事件的事件过滤条件和事件统计数据；Extract sequence event condition data from the received condition information; wherein, the sequence event condition data includes event filter conditions and event statistics data of multiple events;
根据所述序列事件条件数据及多个事件之间的关系生成所述序列规则分析模式对应的分析规则。An analysis rule corresponding to the sequence rule analysis mode is generated according to the sequence event condition data and the relationship between multiple events.
可选的，所述根据所述分析规则对待分析的日志数据进行分析，得到该日志数据对应的目标日志事件，包括：Optionally, the log data to be analyzed is analyzed according to the analysis rules to obtain a target log event corresponding to the log data, including:
根据所述分析规则中的事件过滤条件对所述待分析的日志数据中的事件进行过滤，得到初级日志事件；Filter the events in the log data to be analyzed according to the event filter conditions in the analysis rules to obtain primary log events;
对所述初级日志事件进行统计得到事件统计数据；Counting the primary log events to obtain event statistics;
根据所述初级日志事件、所述事件统计数据及各初级日志事件的关系进行统计分析，得到所述目标日志事件。Statistical analysis is performed according to the primary log event, the event statistical data, and the relationship between each primary log event to obtain the target log event.
可选的，还包括：Optionally, also include:
根据事件属性信息对获取到的多个目标日志事件进行关联分析，得到目标关联事件。Correlation analysis is performed on the acquired multiple target log events according to the event attribute information, and target correlation events are obtained.
可选的，还包括：Optionally, also include:
根据配置数据判断所述目标日志事件是否为告警事件；Determine whether the target log event is an alarm event according to the configuration data;
若是，则生成告警信息。If so, generate an alarm message.
可选的，还包括：Optionally, also include:
根据告警事件特征库对多个告警事件进行关联分析，得到关联告警事件；Correlation analysis is performed on multiple alarm events according to the alarm event feature library, and the associated alarm events are obtained;
根据所述关联告警事件生成关联告警信息。Generate associated alarm information according to the associated alarm event.
本申请还提供一种日志信息分析装置，包括：The application also provides a log information analysis device, including:
分析模式选择模块，用于根据接收到的模式选择信息选择对应的日志分析模式；The analysis mode selection module is used to select the corresponding log analysis mode according to the received mode selection information;
分析规则生成模块，用于根据所述日志分析模式对接收到的条件信息进行编码生成处理，得到所述日志分析模式对应的分析规则；an analysis rule generation module, configured to perform coding and generation processing on the received condition information according to the log analysis mode to obtain an analysis rule corresponding to the log analysis mode;
目标事件分析模块，用于根据所述分析规则对待分析的日志数据进行分析，得到该日志数据对应的目标日志事件。A target event analysis module, configured to analyze the log data to be analyzed according to the analysis rules, and obtain a target log event corresponding to the log data.
本申请还提供一种服务器，包括：The application also provides a server, including:
存储器，用于存储计算机程序；memory for storing computer programs;
处理器，用于执行所述计算机程序时实现如上所述的日志信息分析方法的步骤。The processor is configured to implement the steps of the log information analysis method described above when executing the computer program.
本申请还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如上所述的日志信息分析方法的步骤。The present application also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the log information analysis method described above are implemented.
本申请提供的一种日志信息分析方法，包括：根据接收到的模式选择信息选择对应的日志分析模式；根据所述日志分析模式对接收到的条件信息进行编码生成处理，得到所述日志分析模式对应的分析规则；根据所述分析规则对待分析的日志数据进行分析，得到该日志数据对应的目标日志事件。A method for analyzing log information provided by the present application includes: selecting a corresponding log analysis mode according to received mode selection information; encoding and generating the received condition information according to the log analysis mode to obtain the log analysis mode Corresponding analysis rules; analyze the log data to be analyzed according to the analysis rules, and obtain target log events corresponding to the log data.
首先根据接收到的模式选择信息选择对应的日志分析模式，接着根据日志分析模式对接收到的条件信息进行对应的编码生成处理，得到该日志分析模式对应的分析规则，也就是根据接收到的条件信息即可得到用于进行日志分析的分析规则，最后根据该分析规则对日志数据进行分析，分析出目标日志事件，可见，通过输入的条件信息即可得到对应的分析规则，实现对日志数据的分析操作，而不用技术人员输入复杂些繁复的分析规则，避免了主观错误，降低了日志分析的门槛，提高了日志分析操作的灵活性。First, select the corresponding log analysis mode according to the received mode selection information, and then perform the corresponding code generation processing on the received condition information according to the log analysis mode, and obtain the analysis rule corresponding to the log analysis mode, that is, according to the received conditions The analysis rules for log analysis can be obtained from the information, and finally the log data is analyzed according to the analysis rules, and the target log events are analyzed. Analysis operations do not require technicians to input complex analysis rules, which avoids subjective errors, lowers the threshold for log analysis, and improves the flexibility of log analysis operations.
本申请还提供一种日志信息分析装置、服务器以及计算机可读存储介质，具有以上有益效果，在此不作赘述。The present application also provides a log information analysis device, a server, and a computer-readable storage medium, which have the above beneficial effects, and will not be repeated here.
附图说明Description of drawings
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is an embodiment of the present application. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without any creative effort.
图1为本申请实施例所提供的第一种日志信息分析方法的流程图；1 is a flowchart of a first log information analysis method provided by an embodiment of the present application;
图2为本申请实施例所提供的第二种日志信息分析方法的流程图；2 is a flowchart of a second log information analysis method provided by an embodiment of the present application;
图3为本申请实施例所提供的第三种日志信息分析方法的流程图；3 is a flowchart of a third log information analysis method provided by an embodiment of the present application;
图4为本申请实施例所提供的一种日志信息分析装置的结构示意图。FIG. 4 is a schematic structural diagram of an apparatus for analyzing log information provided by an embodiment of the present application.
具体实施方式Detailed ways
本申请的核心是提供一种日志信息分析方法、日志信息分析装置、服务器以及计算机可读存储介质，通过接收到的模式选择信息选择出对应的日志分析模式对接收到的条件信息进行编码生成处理，得到分析规则并对日志数据进行分析，得到目标日志事件，实现对日志信息进行分析，由于可以选择不同的日志分析模式对日志数据进行分析，提高了进行分析的灵活性。The core of the present application is to provide a log information analysis method, a log information analysis device, a server, and a computer-readable storage medium, and select a corresponding log analysis mode through the received mode selection information to encode and generate the received condition information. , obtain analysis rules and analyze log data, obtain target log events, and analyze log information. Since different log analysis modes can be selected to analyze log data, the flexibility of analysis is improved.
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.
现有技术中使用的SIEM(Security Information and Event Management安全信息和事件管理)对日志数据进行分析。其中，SIEM是一种软件解决方案，可以汇总和分析整个IT基础架构中不同资源的活动，从网络设备、服务器、域控制器等收集安全数据，同时对数据进行存储，规范化，汇总和分析，以发现趋势，检测威胁等。但是，采用现有的技术方案对日志数据进行分析的过程，需要技术人员熟悉代码语言，将对日志数据进行分析的需求编写为对应的可执行语句。但是，此技术方案进行分析过滤的方式门槛过高，不够灵活，不能随时根据用户实际场景定制关联的分析规则，同时也不够易用，非技术人员不能简单的通过界面自定义规则。造成日志分析的效率下降，日志分析的灵活性下降。SIEM (Security Information and Event Management) used in the prior art analyzes log data. Among them, SIEM is a software solution that can aggregate and analyze the activities of different resources throughout the IT infrastructure, collect security data from network devices, servers, domain controllers, etc., while storing, normalizing, aggregating and analyzing the data, to spot trends, detect threats, and more. However, in the process of analyzing the log data by using the existing technical solution, the technicians are required to be familiar with the code language, and the requirements for analyzing the log data are written into corresponding executable statements. However, this technical solution has a high threshold for analysis and filtering, and is not flexible enough to customize the associated analysis rules according to the user's actual scenario at any time. At the same time, it is not easy to use, and non-technical personnel cannot simply customize the rules through the interface. This reduces the efficiency of log analysis and the flexibility of log analysis.
因此，本申请提供一种日志信息分析方法，首先根据接收到的模式选择信息选择对应的日志分析模式，接着根据日志分析模式对接收到的条件信息进行对应的编码生成处理，得到该日志分析模式对应的分析规则，也就是根据接收到的条件信息即可得到用于进行日志分析的分析规则，最后根据该分析规则对日志数据进行分析，分析出目标日志事件，可见，通过输入的条件信息即可得到对应的分析规则，实现对日志数据的分析操作，而不用技术人员输入复杂些繁复的分析规则，避免了主观错误，降低了日志分析的门槛，提高了日志分析操作的灵活性。Therefore, the present application provides a method for analyzing log information. First, a corresponding log analysis mode is selected according to the received mode selection information, and then corresponding code generation processing is performed on the received condition information according to the log analysis mode to obtain the log analysis mode. The corresponding analysis rules, that is, the analysis rules for log analysis can be obtained according to the received condition information, and finally the log data is analyzed according to the analysis rules, and the target log events are analyzed. It can be seen that the input condition information is Corresponding analysis rules can be obtained to realize the analysis operation of log data, without the need for technicians to input complex analysis rules, which avoids subjective errors, lowers the threshold of log analysis, and improves the flexibility of log analysis operations.
以下通过一个实施例，对本申请提供的一种日志信息分析方法做进一步说明。A method for analyzing log information provided by the present application will be further described below through an embodiment.
请参考图1，图1为本申请实施例所提供的第一种日志信息分析方法的流程图。Please refer to FIG. 1 , which is a flowchart of a first log information analysis method provided by an embodiment of the present application.
本实施例，该方法可以包括：In this embodiment, the method may include:
S101，根据接收到的模式选择信息选择对应的日志分析模式；S101, select a corresponding log analysis mode according to the received mode selection information;
本步骤旨在根据接收到的模式选择信息确定到对应的日志分析模式。也就是，本实施例中不仅可以针对不同的过滤条件对日志数据进行对应的过滤分析操作，还可以选择不同的日志分析模式。在不同的模式下根据模式选择信息执行不同的日志分析方法，提高了本实施例中日志信息分析方法进行日志分析的灵活性。This step aims to determine the corresponding log analysis mode according to the received mode selection information. That is, in this embodiment, not only can the corresponding filtering and analysis operations be performed on the log data according to different filtering conditions, but also different log analysis modes can be selected. In different modes, different log analysis methods are executed according to the mode selection information, which improves the flexibility of log analysis by the log information analysis method in this embodiment.
现有技术中，对日志信息进行分析的过程主要是技术人员对计算机装置直接输入可以用于直接对日志信息进行分析的编码。但是，该编码一般是由技术人员根据不同的需求付出一定的脑力劳动后编写出的可以被执行的编码。并且，该脑力劳动一般需要进行学习才可以使技术人员具备。因此，现有技术的日志分析过程具有相当的门槛。当需要对日志分析过程随时进行调整时，需要耗费较高的人力成本，灵活性较低。In the prior art, the process of analyzing log information is mainly that a technician directly inputs a code to a computer device that can be used to directly analyze the log information. However, the code is generally a code that can be executed after the technicians pay a certain amount of mental work according to different needs. In addition, this mental work generally needs to be learned before the technicians can have it. Therefore, the log analysis process in the prior art has a considerable threshold. When the log analysis process needs to be adjusted at any time, it requires high labor costs and low flexibility.
而在更进一步的现有技术中，当针对不同的分析模式时需要技术人员编写不同逻辑的过滤编码。不仅需要将日志分析需求编写为过滤编码，还需要将不同分析模式之间的逻辑进行区分，以便适应不同逻辑的分析模式。可见，难度大门槛高。In the further prior art, when aiming at different analysis modes, technicians are required to write filtering codes of different logics. It is not only necessary to write log analysis requirements as filtering codes, but also to distinguish the logic between different analysis modes so as to adapt to the analysis modes of different logics. It can be seen that the difficulty threshold is high.
因此，本步骤中直接通过接收到的模式选择信息确定对应的日志分析模式，以便后续步骤中根据该日志分析模式实现对应的编码生成方式。并且，相比于现有技术，本申请实施例直接通过本步骤接收到模式选择信息，而不是接收到技术人员直接输入对应分析模式的分析规则。降低了对计算机输入相关信息的门槛和难度。进一步，通过后续的步骤实现自动生成对应的分析规则。Therefore, in this step, the corresponding log analysis mode is directly determined by the received mode selection information, so that the corresponding code generation mode can be implemented according to the log analysis mode in the subsequent steps. Moreover, compared with the prior art, the embodiment of the present application directly receives the mode selection information through this step, instead of receiving the analysis rule corresponding to the analysis mode directly input by the technician. The threshold and difficulty of inputting relevant information to the computer are lowered. Further, the corresponding analysis rules are automatically generated through subsequent steps.
其中，模式选择信息主要是日志信息分析过程中用于确定分析模式的信息，即用于确定规则类型的信息。在更具体的实施例中，可以是用户输入的选择规则类型，也可以是用户在界面中点选的不同的分析模式后得到的模式选择信息，还可以是通过预设路径获取到的模式选择信息，如通过网络方式从远端客户端获取到的模式选择信息。可见，在不同的具体实施例方式中获取模型选择信息的方式并不唯一，但是诸如此类的技术方案获取到的信息均是用于确定分析模式的信息，即用于确定规则类型的信息，均可视作本申请技术方案中的模式选择信息。The mode selection information is mainly the information used to determine the analysis mode in the log information analysis process, that is, the information used to determine the type of the rule. In a more specific embodiment, it may be the selection rule type input by the user, or the mode selection information obtained after the user clicks on different analysis modes in the interface, or the mode selection obtained through a preset path. Information, such as the mode selection information obtained from the remote client through the network. It can be seen that the way to obtain model selection information is not unique in different specific embodiments, but the information obtained by such technical solutions is all information used to determine the analysis mode, that is, the information used to determine the type of rules, can be It is regarded as the mode selection information in the technical solution of the present application.
举例来说，在本实施例中可以对日志数据进行两种分析模式。包括统计规则分析模式和序列规则分析模式。根据这两种分析模式，分别可以对日志数据进行基于统计规则的分析方式和基于序列规则的分析方式。For example, in this embodiment, two analysis modes can be performed on the log data. Including statistical rule analysis mode and sequence rule analysis mode. According to these two analysis modes, the log data can be analyzed by statistical rule-based analysis and sequence-based analysis.
此外，可以想到的是，本实施例中还可以同时采用多种分析模式。例如，确定此时的日志分析模式为统计规则分析模式和序列规则分析模式，也就是同时采用统计规则分析模式和序列规则分析模式对日志数据进行分析处理。In addition, it is conceivable that in this embodiment, multiple analysis modes may be simultaneously adopted. For example, it is determined that the log analysis mode at this time is the statistical rule analysis mode and the sequence rule analysis mode, that is, the log data is analyzed and processed by using the statistical rule analysis mode and the sequence rule analysis mode at the same time.
S102，根据日志分析模式对接收到的条件信息进行编码生成处理，得到日志分析模式对应的分析规则；S102, performing coding and generation processing on the received condition information according to the log analysis mode, to obtain an analysis rule corresponding to the log analysis mode;
在S101的基础上，本步骤旨在根据该日志分析模式对接收到的条件信息进行编码生成处理，得到分析规则。也就是，在选择出的日志分析模式下对接收到的条件信息进行进一步的解析处理，最后生成对应的分析规则。On the basis of S101, this step aims to encode and generate the received condition information according to the log analysis mode to obtain analysis rules. That is, in the selected log analysis mode, further analysis processing is performed on the received condition information, and a corresponding analysis rule is finally generated.
可见，相对于现有技术，本步骤通过针对不同的日志分析模式，可以对接收到的条件信息生成出不同的分析规则，以便适应不同的场景的日志信息分析的需求，以便提高日志信息分析的灵活性。It can be seen that, compared with the prior art, this step can generate different analysis rules for the received condition information by aiming at different log analysis modes, so as to adapt to the needs of log information analysis in different scenarios, so as to improve the efficiency of log information analysis. flexibility.
其中，条件信息主要是日志信息分析过程中用于确定分析规则的信息，即用于确定日志信息分析过程的信息。在更具体的实施例中，该条件信息可以是用户直接输入的规则信息，也可以是用户在界面中点选的不同的分析过程后得到的条件信息，还可以是通过预设路径获取到的条件信息，如通过网络方式从远端客户端获取到的条件信息。可见，在不同的具体实施例方式中条件信息在计算机设备中存在方式并不唯一，但是诸如此类的技术方案获取到的条件信息均是用于确定分析规则的信息，即用于确定日志信息分析过程的信息，均可视作本申请技术方案中的条件信息。The condition information is mainly information used to determine analysis rules in the log information analysis process, that is, information used to determine the log information analysis process. In a more specific embodiment, the condition information may be rule information directly input by the user, or condition information obtained after different analysis processes selected by the user in the interface, or obtained through a preset path Condition information, such as the condition information obtained from the remote client through the network. It can be seen that the existence of condition information in the computer device is not unique in different specific embodiments, but the condition information obtained by such technical solutions is the information used to determine the analysis rules, that is, used to determine the log information analysis process. The information can be regarded as the condition information in the technical solution of this application.
其中，本步骤获取到的分析规则主要是指对日志分析过程进行控制的规则数据，形如对数据进行判断的条件大小，对数据进行判断阈值数据，对数据进行判断的过滤条件等均可对日志分析过程进行控制，因此，诸如此类的信息数据均可视为本实施例中所采用的分析规则。需要注意的是，此处仅仅是简化说明分析规则包含条件大小、阈值数据以及过滤条件等元信息，并不表示本实施例中的分析规则是通过以上少数几个元信息组合得到的。实际来说，本实施例中是通过大量复杂的元信息等的组合才可以得到本步骤中的分析规则。因此，如采用人工的方式对设备输入对应的分析规则则会耗费极大的人力成本，大量的采用人工规则编写编写而分析规则还会引入不同程度的主观错误，进而导致日志分析过程出错以及失败。所以，本实施例中根据对应的日志分析模型对接收到的条件信息进行编码生成处理，得到对应的分析规则，进而实现分析规则的自动编写处理，降低人工成本，减少主观错误，提高日志分析过程的可靠性和稳定程度。The analysis rules obtained in this step mainly refer to the rule data for controlling the log analysis process, such as the size of the conditions for judging the data, the threshold data for judging the data, and the filter conditions for judging the data, etc. The log analysis process is controlled, therefore, such information data can be regarded as the analysis rules adopted in this embodiment. It should be noted that, here is only a simplified description that the analysis rule includes meta-information such as condition size, threshold data, and filter condition, and does not mean that the analysis rule in this embodiment is obtained by combining the above few meta-information. Actually, in this embodiment, the analysis rules in this step can be obtained only through the combination of a large number of complex meta-information and the like. Therefore, entering the corresponding analysis rules into the device manually will consume a lot of labor costs. A large number of manual rules are written and the analysis rules will introduce different degrees of subjective errors, which will lead to errors and failures in the log analysis process. . Therefore, in this embodiment, the received condition information is encoded and generated according to the corresponding log analysis model to obtain corresponding analysis rules, thereby realizing automatic writing processing of analysis rules, reducing labor costs, reducing subjective errors, and improving the log analysis process. reliability and stability.
举例来说，本实施例中可以通过不同的日志分析模式编码生成不同的分析规则。例如，包括统计规则分析模式和序列规则分析模式，对应的编码生成处理后得到的就是基于统计规则的分析规则以及基于序列规则的分析规则。For example, in this embodiment, different analysis rules can be generated by encoding different log analysis modes. For example, including the statistical rule analysis mode and the sequence rule analysis mode, the corresponding code generation processing obtains the analysis rules based on the statistical rules and the analysis rules based on the sequence rules.
还可以想到的是，本实施例中还可以得到采用混合分析模式对应的分析规则。例如，确定此时的日志分析模式为统计规则分析模式和序列规则分析模式的混合分析模式，也就是同时采用统计规则分析模式和序列规则分析模式对日志数据进行分析处理。因此，编码生成处理后得到的就是混合模式下的分析规则。It is also conceivable that in this embodiment, an analysis rule corresponding to the mixed analysis mode can also be obtained. For example, it is determined that the log analysis mode at this time is a mixed analysis mode of the statistical rule analysis mode and the sequence rule analysis mode, that is, the log data is analyzed and processed by using the statistical rule analysis mode and the sequence rule analysis mode at the same time. Therefore, what is obtained after the code generation process is the analysis rules in the mixed mode.
进一步的，本步骤中采用的编码生成处理主要是将条件信息中的元信息进行解析分析，并根据该日志分析模式对应的模式规则将解析出的元信息组合成可以使用的分析规则。Further, the code generation processing adopted in this step is mainly to parse and analyze the meta-information in the condition information, and combine the parsed meta-information into usable analysis rules according to the mode rules corresponding to the log analysis mode.
该分析规则的形式可以是用于控制分析过程的执行代码，也可以是在具体软件的中的语言代码，例如FlinkSQL(Apache Flink Structured Query Language Flink中的结构化查询语言)或CEP SQL(Complex Event Processing Structured Query Language复杂事件处理结构化查询语言)语句。可见，在不同的执行环境中该分析规则的形式也存在不同的差异，均作为本实施例中的分析规则。具体的，可以根据不同的执行环境选择对应的存在形态，在此不做具体限定。The analysis rules can be in the form of execution codes used to control the analysis process, or language codes in specific software, such as FlinkSQL (Apache Flink Structured Query Language Flink) or CEP SQL (Complex Event Processing Structured Query Language complex event processing Structured Query Language) statement. It can be seen that there are differences in the form of the analysis rules in different execution environments, and they are all used as the analysis rules in this embodiment. Specifically, a corresponding existence form may be selected according to different execution environments, which is not specifically limited here.
此外，针对S101与S102中接收到的模式选择信息和接收到的条件信息的获取方式再做说明。其中，模式选择信息和条件信息可以是用户直接对计算机设备输入的，也可以是用户编写完成的数据通过互联网发送至本地的，还可以是根据通告文件数据文本解析后得到的。可见，本实施例中获取模式选择信息和条件信息的方式并不唯一，只要能获取到模式选择信息和条件信息即可，在此不做具体限定。In addition, the mode selection information received in S101 and S102 and the acquisition manner of the received condition information will be described again. The mode selection information and condition information may be directly input by the user to the computer device, or may be data written by the user and sent to the local through the Internet, or may be obtained by parsing the text of the notification file data. It can be seen that the manner of acquiring the mode selection information and the condition information in this embodiment is not unique, as long as the mode selection information and the condition information can be acquired, which is not specifically limited here.
S103，根据分析规则对待分析的日志数据进行分析，得到该日志数据对应的目标日志事件。S103, analyze the log data to be analyzed according to the analysis rule, and obtain a target log event corresponding to the log data.
在S102的基础上，本步骤旨在执行该分析规则直接对日志数据进行处理，得到目标日志事件。也就是，对客观的日志数据进行区别分类得到的客观的目标日志事件。On the basis of S102, this step aims to execute the analysis rule to directly process the log data to obtain the target log event. That is, objective target log events obtained by distinguishing and classifying objective log data.
其中，目标日志事件指的是该日志分析配置数据所分析出特定数据规律对应的事件。例如，如果分析规则的目的是为了分析出危险事件，那么目标日志事件即为危险事件。如果分析配置数据的目的是为了分析出告警事件，那么目标日志事件即为告警事件。The target log event refers to an event corresponding to a specific data rule analyzed by the log analysis configuration data. For example, if the purpose of the analysis rule is to analyze a dangerous event, then the target log event is a dangerous event. If the purpose of analyzing configuration data is to analyze alarm events, the target log events are alarm events.
可选的，本实施例还可以包括：Optionally, this embodiment may further include:
根据事件属性信息对获取到的多个目标日志事件进行关联分析，得到目标关联事件。Correlation analysis is performed on the acquired multiple target log events according to the event attribute information, and target correlation events are obtained.
本可选方案主要是将获取到的多个目标日志事件之间进行关联分析。可以是，分析多个目标日志事件之间的相同之处，例如，相同的进程、相同的路径、相同的周期。还可以是分析多个目标日志事件之间的交集、也可以是分析多个目标日志事件之间的相关联信息。通过本可选方案中的关联分析，使得在多个目标日志事件中关联出一个或少数几个的目标关联事件，也就是对分散的独立事件进行整体关联分析，找出存在一定联系的事件，从而更有效的发现需要判定的目标关联事件，提高日志分析的效果。This optional solution is mainly to perform correlation analysis among the acquired multiple target log events. It can be to analyze the similarities between multiple target log events, for example, the same process, the same path, the same cycle. It may also be to analyze the intersection between multiple target log events, or to analyze the correlation information between multiple target log events. Through the correlation analysis in this optional solution, one or a few target correlation events are associated with multiple target log events, that is, the overall correlation analysis is performed on the scattered independent events to find out the events that have a certain relationship. Therefore, the target-related events that need to be determined can be found more effectively, and the effect of log analysis can be improved.
可选的，本实施例还可以包括：Optionally, this embodiment may further include:
根据配置数据判断目标日志事件是否为告警事件；Determine whether the target log event is an alarm event according to the configuration data;
若是，则生成告警信息。If so, generate an alarm message.
本可选方案中，根据配置数据进行判断从而确定是否进行告警操作，对于特定的目标日志事件进行及时的告警提醒，避免遗漏特定事件，提高运行维护的效率。In this optional solution, judgment is made according to the configuration data to determine whether to perform an alarm operation, and a timely alarm reminder is performed for a specific target log event, so as to avoid omission of specific events and improve the efficiency of operation and maintenance.
可选的，本实施例还可以包括：Optionally, this embodiment may further include:
步骤1，根据告警事件特征库对多个告警事件进行关联分析，得到关联告警事件；Step 1, perform correlation analysis on a plurality of alarm events according to the alarm event feature library to obtain correlated alarm events;
步骤2，根据关联告警事件生成关联告警信息。Step 2, generating associated alarm information according to the associated alarm event.
可见，本可选方案中主要是在判定出告警事件的基础上，根据告警事件特征库关联分析出关联告警事件，并进行告警操作。其中，告警事件特征库主要包括多个告警事件之间的共性特征，以便判断出对应的关联告警事件。例如，采用病毒特征库，那么就根据该病毒特征库对多个告警事件进行关联分析，得到关联告警事件，进而生成关联告警信息，实现对病毒事件的告警操作。很显然，在告警事件的基础上进一步通过告警事件特征库进行关联分析，可以关联分出存在深层联系的关联告警事件，进一步提高对日志数据进行特定事件分析的准确性和精度。It can be seen that, in this optional solution, on the basis of determining the alarm event, the related alarm events are correlated and analyzed according to the alarm event feature database, and the alarm operation is performed. The alarm event feature library mainly includes common features among multiple alarm events, so as to determine the corresponding related alarm events. For example, if a virus signature database is used, correlation analysis is performed on multiple alarm events according to the virus signature database to obtain associated alarm events, and then associated alarm information is generated to realize alarm operations on virus events. Obviously, on the basis of alarm events, further correlation analysis is carried out through the alarm event feature database, which can correlate and separate out the correlated alarm events with deep connections, and further improve the accuracy and precision of specific event analysis on log data.
综上，本实施例首先根据接收到的模式选择信息选择对应的日志分析模式，接着根据日志分析模式对接收到的条件信息进行对应的编码生成处理，得到该日志分析模式对应的分析规则，也就是根据接收到的条件信息即可得到用于进行日志分析的分析规则，最后根据该分析规则对日志数据进行分析，分析出目标日志事件，可见，通过输入的条件信息即可得到对应的分析规则，实现对日志数据的分析操作，而不用技术人员输入复杂些繁复的分析规则，避免了主观错误，降低了日志分析的门槛，提高了日志分析操作的灵活性。To sum up, this embodiment first selects a corresponding log analysis mode according to the received mode selection information, and then performs corresponding code generation processing on the received condition information according to the log analysis mode to obtain an analysis rule corresponding to the log analysis mode, and also That is, according to the received condition information, the analysis rules for log analysis can be obtained, and finally the log data is analyzed according to the analysis rules, and the target log events are analyzed. It can be seen that the corresponding analysis rules can be obtained through the input condition information. , to realize the analysis operation of log data, without the need for technicians to input complex analysis rules, avoid subjective errors, lower the threshold of log analysis, and improve the flexibility of log analysis operations.
以下通过另一具体的实施例，对本申请提供的一种日志信息分析方法做进一步说明。A method for analyzing log information provided by the present application will be further described below through another specific embodiment.
请参考图2，图2为本申请实施例所提供的第二种日志信息分析方法的流程图。Please refer to FIG. 2 , which is a flowchart of a second log information analysis method provided by an embodiment of the present application.
本实施例中，该方法可以包括：In this embodiment, the method may include:
S201，根据接收到的模式选择信息选择对应的日志分析模式，日志分析模式包括统计规则分析模式；S201, select a corresponding log analysis mode according to the received mode selection information, where the log analysis mode includes a statistical rule analysis mode;
S202，从接收到的条件信息中提取出单一事件条件数据；其中，单一事件条件数据包括过滤条件和统计数据；S202, extracting single event condition data from the received condition information; wherein, the single event condition data includes filter conditions and statistical data;
S203，根据单一事件条件数据生成统计规则分析模式对应的分析规则；S203, generating an analysis rule corresponding to the statistical rule analysis mode according to the single event condition data;
S204，根据分析规则中的过滤条件对待分析的日志数据进行过滤，得到初级日志数据；S204, filter the log data to be analyzed according to the filter conditions in the analysis rule to obtain primary log data;
S205，对初级日志数据中的预设字段进行统计得到统计数据，将统计数据与分析规则中的统计标准进行对比，根据得到的对比结果确定目标日志事件。S205 , perform statistics on preset fields in the primary log data to obtain statistical data, compare the statistical data with the statistical standards in the analysis rules, and determine a target log event according to the obtained comparison result.
可见，本实施例中主要是针对在统计规则分析模式下对日志数据实现对应的日志分析操作。其中，统计规则分析模式主要是采用日志信息中的统计信息对日志数据进行分析操作。It can be seen that this embodiment mainly aims at implementing the corresponding log analysis operation on the log data in the statistical rule analysis mode. Among them, the statistical rule analysis mode mainly uses the statistical information in the log information to analyze the log data.
具体的，当选择日志分析模式为统计规则分析模式后，从条件信息中提取出单一事件条件数据，该单一事件条件数据主要所用是针对单一事事件进行条件判定的数据。具体包括过滤条件和统计数据。其中，统计数据包括聚合统计后得到的统计数据和阈值。其中，聚合统计后得到的统计数据可以是平均值、最大值等。由于本实施例中是采用统计规则进行日志分析，该统计规则所指就是对单一事件进行统计数据分析的规则。因此，本实施例中获取到单一事件条件数据即可实现统计规则模式下的日志分析操作，采用的数据量较少，可以有效的保持日志分析的效率。然后，根据该过滤条件和统计数据生成对应的分析规则。Specifically, when the log analysis mode is selected as the statistical rule analysis mode, single event condition data is extracted from the condition information, and the single event condition data is mainly used for condition determination data for a single event. Specifically, filter conditions and statistics are included. The statistical data includes statistical data and thresholds obtained after aggregating statistics. The statistical data obtained after aggregating the statistics may be an average value, a maximum value, and the like. Since the log analysis is performed by using a statistical rule in this embodiment, the statistical rule refers to a rule for performing statistical data analysis on a single event. Therefore, in this embodiment, the log analysis operation in the statistical rule mode can be realized by acquiring the single event condition data, the amount of data used is small, and the efficiency of log analysis can be effectively maintained. Then, the corresponding analysis rules are generated according to the filtering conditions and the statistical data.
最后，通过数据库系统或实时计算引擎使用该分析规则对日志数据进行处理。具体如S204至S205所示。具体的，首先根据分析规则中的过滤条件对待分析的日志数据进行过滤，得到初级日志数据；也就是对日志数据进行初步的过滤处理。然后，再对初级日志数据中的预设字段进行统计得到统计数据，将统计数据与分析规则中的统计标准进行对比，根据得到的对比结果确定目标日志事件。也就是将统计数据与统计标准进行对比，以便判断是否符合对应的事件标准。Finally, the log data is processed by the database system or real-time computing engine using the analysis rules. The details are shown in S204 to S205. Specifically, firstly, the log data to be analyzed is filtered according to the filter conditions in the analysis rule to obtain primary log data; that is, the log data is subjected to preliminary filtering processing. Then, the preset fields in the primary log data are counted to obtain statistical data, the statistical data is compared with the statistical standard in the analysis rule, and the target log event is determined according to the obtained comparison result. That is, the statistical data is compared with the statistical standard in order to judge whether the corresponding event standard is met.
可见，本实施例中，首先根据接收到的模式选择信息选择对应的日志分析模式，该日志分析模式包括统计规则分析模式，接着根据日志分析模式对接收到的条件信息进行对应的编码生成处理，得到该统计规则分析模式对应的分析规则，也就是根据接收到的条件信息即可得到用于进行日志分析的分析规则，最后根据该分析规则对日志数据进行基于统计规则的分析，分析出目标日志事件，可见，通过输入的条件信息即可得到基于统计规则的分析规则，实现对日志数据进行基于统计规则分析操作，而不用技术人员输入复杂些繁复的统计规则，避免了主观错误，降低了日志分析的门槛，提高了日志分析操作的灵活性。并且，本实施中主要是在统计数据的角度进行目标日志事件的筛选操作，其中，主要采用过滤条件的判断操作，有效的提高了日志数据进行分析的效率。It can be seen that in this embodiment, first select a corresponding log analysis mode according to the received mode selection information, the log analysis mode includes a statistical rule analysis mode, and then perform corresponding code generation processing on the received condition information according to the log analysis mode, Obtain the analysis rule corresponding to the analysis mode of the statistical rule, that is, the analysis rule for log analysis can be obtained according to the received condition information, and finally analyze the log data based on the statistical rule according to the analysis rule, and analyze the target log. It can be seen that the analysis rules based on statistical rules can be obtained through the input condition information, which realizes the analysis and operation of log data based on statistical rules, without the need for technicians to input complicated and complicated statistical rules, which avoids subjective errors and reduces the number of logs. The threshold for analysis improves the flexibility of log analysis operations. Moreover, in this implementation, the filtering operation of target log events is mainly performed from the perspective of statistical data, and the judgment operation of filtering conditions is mainly used, which effectively improves the efficiency of log data analysis.
以下通过另一具体的实施例，对本申请提供的一种日志信息分析方法做进一步说明。A method for analyzing log information provided by the present application will be further described below through another specific embodiment.
请参考图3，图3为本申请实施例所提供的第三种日志信息分析方法的流程图。Please refer to FIG. 3 , which is a flowchart of a third method for analyzing log information provided by an embodiment of the present application.
本实施例中，该方法可以包括：In this embodiment, the method may include:
S301，根据接收到的模式选择信息选择对应的日志分析模式，日志分析模式包括序列规则分析模式；S301, select a corresponding log analysis mode according to the received mode selection information, where the log analysis mode includes a sequence rule analysis mode;
S302，从接收到的条件信息中提取出序列事件条件数据；其中，序列事件条件数据包括多个事件的事件过滤条件和事件统计数据；S302, extracting sequence event condition data from the received condition information; wherein, the sequence event condition data includes event filter conditions and event statistics data of multiple events;
S303，根据序列事件条件数据及多个事件之间的关系生成序列规则分析模式对应的分析规则；S303, generating an analysis rule corresponding to the sequence rule analysis mode according to the sequence event condition data and the relationship between multiple events;
S304，根据分析规则中的事件过滤条件对待分析的日志数据中的事件进行过滤，得到初级日志事件；S304, filter the events in the log data to be analyzed according to the event filter condition in the analysis rule to obtain primary log events;
S305，对初级日志事件进行统计得到事件统计数据；S305, perform statistics on primary log events to obtain event statistics;
S306，根据初级日志事件、事件统计数据及各初级日志事件的关系进行统计分析，得到目标日志事件。S306, perform statistical analysis according to the primary log events, the event statistical data, and the relationship of each primary log event to obtain a target log event.
可见，本实施例中是确定了采用序列规则分析模式后对日志数据基于序列规则进行相应的数据分析操作。其中，序列规则指的是针对动态的事件发生规则进行分析的规则，分析规则主要是基于事件的时序发生特征。例如，某一特定事件发生的次数、发生的周期，或多个特定事件发生的次数、发生的周期。It can be seen that, in this embodiment, it is determined that the sequence rule analysis mode is adopted to perform a corresponding data analysis operation on the log data based on the sequence rule. Among them, the sequence rules refer to the rules for analyzing the dynamic event occurrence rules, and the analysis rules are mainly based on the time series occurrence characteristics of the events. For example, the number of occurrences of a specific event, the period of occurrence, or the number of occurrences of multiple specific events, the period of occurrence.
具体的，本实施例中确定了序列规则分析模式后，从条件信息提取出序列事件条件数据，该序列事件条件数据主要作用是对在时间序列上特定事件进行判定的条件数据。该序列事件条件数据包括多个事件的事件过滤条件和事件统计数据。其中，事件过滤条件用于将特定事件从日志数据中过滤出来，事件统计数据用于对特征时间在时间序列上进行特征分析，以便判断该特定事件的时序特征是否符合要求。Specifically, after the sequence rule analysis mode is determined in this embodiment, sequence event condition data is extracted from the condition information, and the sequence event condition data is mainly used as condition data for judging specific events in the time series. The sequence event condition data includes event filter conditions and event statistics for multiple events. The event filter condition is used to filter out specific events from the log data, and the event statistics data is used to perform feature analysis on the time series of the feature time, so as to determine whether the time series feature of the specific event meets the requirements.
最后，根据序列事件条件数据及多个事件之间的关系生成序列规则分析模式对应的分析规则。其中，多个事件之间的关系可以是多个事件在时间序列上的时间相对关系，例如，时间前后关系。也可以是多个事件在时间序列上的发生次数之间的关系，例如，总是在一定时间内发生两次A事件和一次B事件。还可以是多个事件在时间序列上的时间相对关系和发生次数时间的混合关系。例如，先发生1次B事件然后发生3次A事件。可见，本实施例中根据序列事件条件数据和多个事件之间的关系准确的确定不同类型的序列事件，有利于提高序列事件的判断准确性。Finally, an analysis rule corresponding to the sequence rule analysis mode is generated according to the sequence event condition data and the relationship between multiple events. The relationship between the multiple events may be a time relative relationship between the multiple events in the time series, for example, a time-before-and-after relationship. It can also be the relationship between the number of occurrences of multiple events in the time series, for example, event A and event B always occur twice within a certain period of time. It can also be the time relative relationship of multiple events in the time series and the mixed relationship of occurrence times and time. For example, event B occurs 1 time and then event A occurs 3 times. It can be seen that, in this embodiment, different types of sequence events are accurately determined according to the sequence event condition data and the relationship between multiple events, which is beneficial to improve the judgment accuracy of the sequence events.
进一步的，采用该分析规则对日志数据进行分析处理，得到目标日志事件。具体的，请参考S305至S306。具体的，根据分析规则中的事件过滤条件对待分析的日志数据中的事件进行过滤，得到初级日志事件；对初级日志事件进行统计得到事件统计数据；也就是，对日志数据中的单个事件进行统计操作，得到对应的时间统计数据。然后，根据初级日志事件、事件统计数据及各初级日志事件的关系进行统计分析，得到目标日志事件。实现通过基于序列规则的分析规则对日志数据进行分析操作，由于采用序列规则进行分析，可以进行更加复杂的分析操作，提高了日志分析的准确性和精度。Further, the analysis rule is used to analyze and process the log data to obtain the target log event. For details, please refer to S305 to S306. Specifically, filter the events in the log data to be analyzed according to the event filtering conditions in the analysis rules to obtain primary log events; perform statistics on the primary log events to obtain event statistics; that is, perform statistics on a single event in the log data Operation to get the corresponding time statistics. Then, statistical analysis is performed according to the primary log events, the event statistical data, and the relationship of each primary log event, so as to obtain the target log event. It realizes the analysis operation of log data through analysis rules based on sequence rules. Due to the use of sequence rules for analysis, more complex analysis operations can be performed, which improves the accuracy and precision of log analysis.
可见，本实施例中，首先根据接收到的模式选择信息选择对应的日志分析模式，该日志分析模式包括序列规则分析模式，接着根据日志分析模式对接收到的条件信息进行对应的编码生成处理，得到该序列规则分析模式对应的分析规则，也就是根据接收到的条件信息即可得到用于进行日志分析的分析规则，最后根据该分析规则对日志数据进行基于序列规则的分析，分析出目标日志事件，可见，通过输入的条件信息即可得到基于序列规则的分析规则，实现对日志数据进行基于序列规则分析操作，而不用技术人员输入复杂些繁复的序列规则，避免了主观错误，降低了日志分析的门槛，提高了日志分析操作的灵活性。并且，由于基于序列规则主要是在时间序列中的对发生的特定事件之间的关系进行判断，有效的提高了目标日志事件判定的准确性和精度，避免出现误判情况。It can be seen that in this embodiment, first select a corresponding log analysis mode according to the received mode selection information, the log analysis mode includes a sequence rule analysis mode, and then perform corresponding code generation processing on the received condition information according to the log analysis mode, The analysis rules corresponding to the sequence rule analysis mode are obtained, that is, the analysis rules for log analysis can be obtained according to the received condition information, and finally the log data is analyzed based on the sequence rules according to the analysis rules, and the target log is analyzed. Event, it can be seen that the analysis rules based on sequence rules can be obtained through the input condition information, and the analysis operation based on sequence rules can be realized on the log data, without the need for technicians to input complicated sequence rules, which avoids subjective errors and reduces the number of logs. The threshold for analysis improves the flexibility of log analysis operations. In addition, because the relationship between specific events that occur is mainly judged based on the sequence rules in the time series, the accuracy and precision of the target log event judgment are effectively improved, and misjudgment is avoided.
以下通过一个具体的实施例，对本申请提供的一种日志信息分析方法做进一步说明。A method for analyzing log information provided by the present application will be further described below through a specific embodiment.
本实施例中，用户首先打开页面选择添加规则类型，进行规则选择，得到的规则选择也就是前述实施例中的日志分析模式。其中，规则类型包含两种，一种是统计规则，对单一事件进行聚合统计、阈值比较。例如主机在1分钟内登录失败30次则告警；另一种是序列规则，定义多个事件并且顺序发生，例如主机在1分钟内多次登录失败后登录成功则告警。In this embodiment, the user first opens the page and selects the type of added rule, selects the rule, and the obtained rule selection is the log analysis mode in the foregoing embodiment. Among them, there are two types of rules, one is a statistical rule, which performs aggregate statistics and threshold comparison on a single event. For example, if the host fails to log in 30 times within 1 minute, an alarm will be generated; the other is a sequence rule, which defines multiple events and occurs in sequence.
对于统计规则，本实施例根据用户配置的过滤条件、聚合字段和阈值等动态生成基于统计聚合的SQL(Structured Query Language结构化查询语言)。而在具体实施例中技术人员只需要根据自身的实际需求，确定自身需要的数据的过滤条件、聚合字段和阈值在界面配置即可，而不用耗费大量时间确定具体的操作语言。安全设备或者操作系统等日志通过Flink(Apache Flink是一个分布式流批一体化的开源平台)实时计算先根据输入的过滤条件过滤，然后使用聚合字段进行数据统计，判断是否符合配置的阈值，如果符合即生成告警。本实施例通过接收到的统计规则，可以将如登录失败事件进行告警，并判定为暴力破解攻击事件。在统计规则中的规则信息，包括：名称、说明、介绍信息、描述信息；聚合统计数据包括：聚合字段(聚合分组后每一组的平均值、最大值)、阈值。For statistical rules, this embodiment dynamically generates SQL (Structured Query Language) based on statistical aggregation according to filter conditions, aggregation fields, and thresholds configured by the user. In the specific embodiment, the technical personnel only need to determine the filter conditions, aggregation fields and thresholds of the data they need to configure on the interface according to their actual needs, instead of spending a lot of time determining the specific operation language. Logs such as security devices or operating systems are calculated in real time through Flink (Apache Flink is an open source platform integrating distributed stream and batch). First, filter according to the input filter conditions, and then use the aggregation field for data statistics to determine whether it meets the configured threshold. An alert is generated if it matches. In this embodiment, through the received statistical rules, an event such as a login failure can be alerted and determined as a brute force attack event. The rule information in the statistical rule includes: name, description, introduction information, and description information; the aggregated statistical data includes: aggregation field (average and maximum value of each group after aggregation grouping), and threshold.
对于序列规则，本实施例根据用户输入的配置数据的事件及各个事件的过滤条件和事件发生次数等动态生成基于事件时序发生的CEP(Complex Event Processing复杂事件处理)。在技术实施例的过程中，只需要技术人员根据自身的实际需求，确认需要定义几个事件及各个事件的过滤条件和事件发生次数在界面配置即可，降低门槛，提高灵活程度。安全设备或者操作系统等日志通过Flink实时计算先根据用户配置的事件条件判断各个事件是否符合，然后判断各个事件是否符合时序关系，如果符合即生成告警。本实施例通过技术人员设置的序列规则，可以将如发生登录失败事件后发生登录成功事件进行告警，并判定为暴力破解攻击成功事件。For sequence rules, this embodiment dynamically generates CEP (Complex Event Processing) based on event sequence occurrence according to the events of the configuration data input by the user, the filter conditions of each event, and the number of occurrences of the events. In the process of the technical embodiment, the technician only needs to confirm that several events need to be defined and the filter conditions of each event and the number of occurrences of the event can be configured on the interface according to their actual needs, so as to lower the threshold and improve the flexibility. The logs of security devices or operating systems are calculated in real time by Flink to first determine whether each event conforms to the event conditions configured by the user, and then determine whether each event conforms to the time sequence relationship. If it conforms, an alarm is generated. In this embodiment, through the sequence rules set by the technician, an alarm can be given for a successful login event after a login failure event occurs, and it is determined as a successful brute force attack event.
此外，还可以将多个告警之间进行关联，比如某种病毒在主机上存在几种行为(目标日志事件)，可以对单个行为进行告警，同时结合多个行为将此告警关联为特定病毒的事件。In addition, multiple alarms can also be correlated. For example, a virus has several behaviors on the host (target log events), and a single behavior can be alarmed, and the alarm can be associated with a specific virus in combination with multiple behaviors. event.
可见，本实施例中首先根据接收到的模式选择信息选择对应的日志分析模式，包括基于统计规则的分析模式和基于序列规则的分析模式，接着根据日志分析模式对接收到的条件信息进行对应的编码生成处理，得到该日志分析模式对应的分析规则，也就是根据接收到的条件信息即可得到用于进行日志分析的分析规则，最后根据该分析规则对日志数据进行分析，分析出目标日志事件，可见，通过输入的条件信息即可得到对应的分析规则，实现对日志数据的分析操作，而不用技术人员输入复杂些繁复的分析规则，避免了主观错误，降低了日志分析的门槛，提高了日志分析操作的灵活性。It can be seen that, in this embodiment, the corresponding log analysis mode is first selected according to the received mode selection information, including the analysis mode based on statistical rules and the analysis mode based on sequence rules, and then corresponding to the received condition information according to the log analysis mode. Code generation and processing to obtain the analysis rules corresponding to the log analysis mode, that is, according to the received condition information, the analysis rules for log analysis can be obtained, and finally the log data is analyzed according to the analysis rules to analyze the target log events. , it can be seen that the corresponding analysis rules can be obtained through the input condition information, and the analysis operation of log data can be realized without the need for technicians to input complicated and complicated analysis rules, which avoids subjective errors, lowers the threshold of log analysis, and improves the efficiency of log analysis. Flexibility of log analysis operations.
下面对本申请实施例提供的一种日志信息分析装置进行介绍，下文描述的一种日志信息分析装置与上文描述的一种日志信息分析方法可相互对应参照。An apparatus for analyzing log information provided by an embodiment of the present application is introduced below. The apparatus for analyzing log information described below and the method for analyzing log information described above may refer to each other correspondingly.
请参考图4，图4为本申请实施例所提供的一种日志信息分析装置的结构示意图。Please refer to FIG. 4 , which is a schematic structural diagram of a log information analysis apparatus provided by an embodiment of the present application.
本实施例中，该装置可以包括：In this embodiment, the device may include:
分析模式选择模块100，用于根据接收到的模式选择信息选择对应的日志分析模式；An analysis
分析规则生成模块200，用于根据日志分析模式对接收到的条件信息进行编码生成处理，得到日志分析模式对应的分析规则；The analysis
目标事件分析模块300，用于根据分析规则对待分析的日志数据进行分析，得到该日志数据对应的目标日志事件。The target
本实施例中该提供一种服务器，包括：In this embodiment, a server should be provided, including:
存储器，用于存储计算机程序；memory for storing computer programs;
处理器，用于执行计算机程序时实现如以上实施例的日志信息分析方法的步骤。The processor is configured to implement the steps of the log information analysis method according to the above embodiment when executing the computer program.
本实施例中该提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如以上实施例所述的日志信息分析方法的步骤。In this embodiment, a computer-readable storage medium is provided, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the log information analysis method described in the above embodiment are implemented. .
说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.
专业人员还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。Professionals may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the possibilities of hardware and software. Interchangeability, the above description has generally described the components and steps of each example in terms of functionality. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.
以上对本申请所提供的一种日志信息分析方法、日志信息分析装置、服务器以及计算机可读存储介质进行了详细介绍。本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想。应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以对本申请进行若干改进和修饰，这些改进和修饰也落入本申请权利要求的保护范围内。A log information analysis method, a log information analysis device, a server, and a computer-readable storage medium provided by the present application have been described in detail above. Specific examples are used herein to illustrate the principles and implementations of the present application, and the descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of the present application, several improvements and modifications can also be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.
Claims (11)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010407002.8ACN111597550A (en) | 2020-05-14 | 2020-05-14 | A kind of log information analysis method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010407002.8ACN111597550A (en) | 2020-05-14 | 2020-05-14 | A kind of log information analysis method and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111597550Atrue CN111597550A (en) | 2020-08-28 |
Family
ID=72187349
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010407002.8APendingCN111597550A (en) | 2020-05-14 | 2020-05-14 | A kind of log information analysis method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111597550A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113259358A (en)* | 2021-05-21 | 2021-08-13 | 杭州海康威视系统技术有限公司 | Data anomaly identification method and device |
| CN113342564A (en)* | 2021-06-25 | 2021-09-03 | 阿波罗智联(北京)科技有限公司 | Log auditing method and device, electronic equipment and medium |
| CN115292062A (en)* | 2022-07-29 | 2022-11-04 | 成都智元汇信息技术股份有限公司 | Method, system and device for realizing product sequence confirmation based on stream architecture |
| CN115297166A (en)* | 2022-07-29 | 2022-11-04 | 成都智元汇信息技术股份有限公司 | Rule engine architecture, system and method of flow architecture |
| CN115296913A (en)* | 2022-08-05 | 2022-11-04 | 武汉思普崚技术有限公司 | Rapid arranging system suitable for flink operation rule |
| CN115309716A (en)* | 2022-10-10 | 2022-11-08 | 杭州中电安科现代科技有限公司 | Log analysis method, device, equipment and medium |
| JPWO2022259558A1 (en)* | 2021-06-11 | 2022-12-15 |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101939742A (en)* | 2007-10-02 | 2011-01-05 | 洛格逻辑公司 | Searching log data for associated events |
| CN107864056A (en)* | 2017-11-04 | 2018-03-30 | 公安部第三研究所 | A kind of distributed event acquisition probe, distributed event high speed acquisition system and method |
| CN108108466A (en)* | 2017-12-29 | 2018-06-01 | 咪咕文化科技有限公司 | Distributed system log query analysis method and device |
| CN109324996A (en)* | 2018-10-12 | 2019-02-12 | 平安科技（深圳）有限公司 | Journal file processing method, device, computer equipment and storage medium |
| CN109634818A (en)* | 2018-10-24 | 2019-04-16 | 中国平安人寿保险股份有限公司 | Log analysis method, system, terminal and computer readable storage medium |
| CN109947715A (en)* | 2018-09-07 | 2019-06-28 | 网联清算有限公司 | Log alarm method and device |
| CN110427307A (en)* | 2019-06-21 | 2019-11-08 | 平安科技（深圳）有限公司 | Log analytic method, device, computer equipment and storage medium |
- 2020
- 2020-05-14CNCN202010407002.8Apatent/CN111597550A/enactivePending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101939742A (en)* | 2007-10-02 | 2011-01-05 | 洛格逻辑公司 | Searching log data for associated events |
| CN107864056A (en)* | 2017-11-04 | 2018-03-30 | 公安部第三研究所 | A kind of distributed event acquisition probe, distributed event high speed acquisition system and method |
| CN108108466A (en)* | 2017-12-29 | 2018-06-01 | 咪咕文化科技有限公司 | Distributed system log query analysis method and device |
| CN109947715A (en)* | 2018-09-07 | 2019-06-28 | 网联清算有限公司 | Log alarm method and device |
| CN109324996A (en)* | 2018-10-12 | 2019-02-12 | 平安科技（深圳）有限公司 | Journal file processing method, device, computer equipment and storage medium |
| CN109634818A (en)* | 2018-10-24 | 2019-04-16 | 中国平安人寿保险股份有限公司 | Log analysis method, system, terminal and computer readable storage medium |
| CN110427307A (en)* | 2019-06-21 | 2019-11-08 | 平安科技（深圳）有限公司 | Log analytic method, device, computer equipment and storage medium |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113259358A (en)* | 2021-05-21 | 2021-08-13 | 杭州海康威视系统技术有限公司 | Data anomaly identification method and device |
| JPWO2022259558A1 (en)* | 2021-06-11 | 2022-12-15 | ||
| JP7605310B2 (en) | 2021-06-11 | 2024-12-24 | 日本電信電話株式会社 | Determination device, determination method, and determination program |
| CN113342564A (en)* | 2021-06-25 | 2021-09-03 | 阿波罗智联(北京)科技有限公司 | Log auditing method and device, electronic equipment and medium |
| CN113342564B (en)* | 2021-06-25 | 2023-12-12 | 阿波罗智联(北京)科技有限公司 | Log auditing method and device, electronic equipment and medium |
| CN115292062A (en)* | 2022-07-29 | 2022-11-04 | 成都智元汇信息技术股份有限公司 | Method, system and device for realizing product sequence confirmation based on stream architecture |
| CN115297166A (en)* | 2022-07-29 | 2022-11-04 | 成都智元汇信息技术股份有限公司 | Rule engine architecture, system and method of flow architecture |
| CN115296913A (en)* | 2022-08-05 | 2022-11-04 | 武汉思普崚技术有限公司 | Rapid arranging system suitable for flink operation rule |
| CN115309716A (en)* | 2022-10-10 | 2022-11-08 | 杭州中电安科现代科技有限公司 | Log analysis method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111597550A (en) | A kind of log information analysis method and related device | |
| US11909753B2 (en) | Virtual private cloud flow log event fingerprinting and aggregation | |
| CN110321371B (en) | Log data anomaly detection method, device, terminal and medium | |
| CN111813960B (en) | Knowledge graph-based data security audit model device, method and terminal equipment | |
| US20190250971A1 (en) | Tuning context-aware rule engine for anomaly detection | |
| CN111309539A (en) | Abnormity monitoring method and device and electronic equipment | |
| CN110362473A (en) | Test optimization method and device, storage medium, the terminal of environment | |
| CN112988509B (en) | Alarm message filtering method and device, electronic equipment and storage medium | |
| CN115296913B (en) | Quick arrangement system adapting to flink operation rules | |
| CN110598959A (en) | Asset risk assessment method and device, electronic equipment and storage medium | |
| CN117112339A (en) | Abnormality detection method, abnormality detection device, electronic device, and computer program product | |
| CN115102836A (en) | Network equipment failure analysis method, device and storage medium | |
| US10909242B2 (en) | System and method for detecting security risks in a computer system | |
| CN116227916A (en) | Real-time wind control system and method based on rule engine | |
| CN119718733A (en) | Log data generation method, device, equipment and medium based on support vector machine | |
| CN119051957A (en) | Network defense rule validity detection method, work order platform and electronic equipment | |
| CN114064757A (en) | Application program optimization method, device, equipment and medium | |
| CN113656314A (en) | Pressure test processing method and device | |
| CN117828515A (en) | An intelligent log anomaly diagnosis system and method based on a low-code platform | |
| CN111111211A (en) | Method, device, system, equipment and storage medium for reporting game data | |
| CN117407245A (en) | Model training task anomaly detection method and system, electronic equipment and storage medium | |
| CN118820026A (en) | Cloud service cluster status monitoring method, device, equipment and storage medium | |
| CN116132103A (en) | A network security situation monitoring method, device, electronic equipment and storage medium | |
| CN102611714A (en) | Link discovery technique based network intrusion prediction method | |
| CN113190844A (en) | Detection method, related method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20200828 |