技术领域technical field
本发明涉及日志管理技术领域，尤其涉及一种日志管理方法、装置、设备及计算机可读存储介质。The present invention relates to the technical field of log management, and in particular, to a log management method, apparatus, device and computer-readable storage medium.
背景技术Background technique
Windows网络操作设备都设计有各种各样的日志文件，如应用程序日志，安全日志、设备日志、Scheduler(调度程序)服务日志、FTP(File Transfer Protocol，文件传输协议)日志、WWW(World Wide Web，万维网)日志、DNS(Domain Name System，域名系统)服务器日志等等，这些根据你的设备开启的服务的不同而有所不同。我们在设备上进行一些操作时，这些日志文件通常会记录下我们操作的一些相关内容，这些内容对设备安全工作人员相当有用。比如说有人对设备进行了IPC(Inter-Process Communication，进程间通信)探测，设备就会在安全日志里迅速地记下探测者探测时所用的IP地址(Internet ProtocolAddress，网际协议地址)、时间、用户名等，用FTP探测后，就会在FTP日志中记下IP、时间、探测所用的用户名等。Windows network operating devices are designed with various log files, such as application logs, security logs, device logs, Scheduler (scheduler) service logs, FTP (File Transfer Protocol, file transfer protocol) logs, WWW (World Wide Web, World Wide Web) logs, DNS (Domain Name System, Domain Name System) server logs, etc., these vary depending on the services your device has turned on. When we perform some operations on the device, these log files usually record some related content of our operation, which is quite useful to the device security staff. For example, if someone performs IPC (Inter-Process Communication) detection on the device, the device will quickly record the IP address (Internet ProtocolAddress, Internet Protocol Address), time, User name, etc. After using FTP detection, the IP, time, and user name used for detection will be recorded in the FTP log.
电脑里的日志是指日志数据，可以是有价值的信息宝库，也可以是毫无价值的数据泥潭。要保护和提高网络安全，由各种操作设备、应用程序、设备和安全产品的日志数据能够帮助提前发现和避开灾难，并且找到安全事件的根本原因。The log in the computer refers to the log data, which can be a treasure trove of valuable information or a quagmire of worthless data. To protect and improve network security, log data from various operational devices, applications, devices, and security products can help detect and avoid disasters in advance and find the root cause of security incidents.
目前日志的归档是将目标服务器上面的日志文档上传到一台归档服务器中缓存，当日志缓存超过限定时间时，将日志存入磁盘中进行归档，这过程中需要定制化的脚本才可以实现，因此在平时对日志进行归档时，需要熟悉脚本的人来管理，对于不懂脚本的新人来说，不知道该怎么处理，也不敢随便修改脚本处理，导致日志的归档很不方便。The current log archive is to upload the log files on the target server to an archive server for caching. When the log cache exceeds the limit time, the log is stored in the disk for archiving. In this process, a customized script can be implemented. Therefore, when archiving logs at ordinary times, people who are familiar with scripts are required to manage them. For newcomers who do not understand scripts, they do not know how to deal with them, and they do not dare to modify script processing casually, which makes log archiving very inconvenient.
发明内容SUMMARY OF THE INVENTION
本发明的主要目的在于提供一种日志管理方法、装置、设备及计算机可读存储介质，旨在解决现有的日志归档管理需要懂脚本的人才行，且耗时不方便的技术问题。The main purpose of the present invention is to provide a log management method, device, device and computer-readable storage medium, aiming to solve the technical problem that the existing log archive management requires people who understand scripts and is time-consuming and inconvenient.
为实现上述目的，本发明提供一种日志管理方法，所述日志管理方法包括以下步骤：In order to achieve the above object, the present invention provides a log management method, which includes the following steps:
当接收到输入的实例名时，基于所述实例名，获取对应的IP地址和日志路径；When receiving the input instance name, obtain the corresponding IP address and log path based on the instance name;
基于所述IP地址和所述日志路径，获取对应的待归档日志；Obtain the corresponding log to be archived based on the IP address and the log path;
确定所述待归档日志的大小，并基于所述待归档日志的大小，确定对应的归档规则，所述待归档日志的大小为所占内存的大小；Determine the size of the log to be archived, and determine the corresponding filing rule based on the size of the log to be archived, where the size of the log to be archived is the size of the occupied memory;
根据所述归档规则，对所述待归档日志进行归档。According to the archiving rule, the log to be archived is archived.
可选地，所述根据所述归档规则，对所述待归档日志进行归档的步骤包括：Optionally, the step of archiving the log to be archived according to the archiving rule includes:
确定所述待归档日志的日志类型，并根据所述日志类型确定所述待归档日志对应的归档位置；Determine the log type of the log to be archived, and determine the archive location corresponding to the log to be archived according to the log type;
根据所述归档规则，将所述待归档日志归档至所述归档位置。According to the archiving rule, the log to be archived is archived to the archive location.
可选地，所述根据所述归档规则，对所述待归档日志进行归档的步骤包括：Optionally, the step of archiving the log to be archived according to the archiving rule includes:
监测所述待归档日志对应的归档任务是否出现异常状态，若所述归档任务出现异常状态，则确定所述异常状态对应的补救策略；Monitoring whether the archiving task corresponding to the log to be archived has an abnormal state, and if the archiving task has an abnormal state, determining a remedy strategy corresponding to the abnormal state;
根据所述归档规则和所述补救策略，对所述待归档日志进行归档。The log to be archived is archived according to the archiving rule and the remediation policy.
可选地，所述监测所述待归档日志对应的归档任务是否出现异常状态，若所述归档任务出现异常状态，则确定所述异常状态对应的补救策略的步骤包括：Optionally, the monitoring of whether an archiving task corresponding to the log to be archived has an abnormal state, and if the archiving task has an abnormal state, the step of determining a remedy policy corresponding to the abnormal state includes:
获取所述待归档日志对应的归档状态码，并根据所述归档状态码，监测所述归档任务是否出现异常状态。The archiving status code corresponding to the log to be archived is acquired, and according to the archiving status code, whether an abnormal state occurs in the archiving task is monitored.
若所述归档任务出现异常状态，则根据所述归档状态码，确定所述异常状态对应的异常类型；If an abnormal state occurs in the archiving task, determining the abnormal type corresponding to the abnormal state according to the archiving state code;
确定所述异常类型对应的补救策略。Determine the remediation strategy corresponding to the exception type.
可选地，所述当接收到输入的实例名时，基于所述实例名，获取对应的IP地址和日志路径的步骤包括：Optionally, when the input instance name is received, based on the instance name, the steps of obtaining the corresponding IP address and log path include:
当接收到输入的实例名时，判断所述实例名是否有效；When receiving the input instance name, determine whether the instance name is valid;
若有效，则获取所述实例名对应的归档链接；If valid, obtain the archive link corresponding to the instance name;
获取所述归档链接对应的IP地址和所述归档链接对应的日志路径。Obtain the IP address corresponding to the archive link and the log path corresponding to the archive link.
可选地，所述根据所述归档规则，对所述待归档日志进行归档的步骤之后，所述方法还包括：Optionally, after the step of archiving the log to be archived according to the archiving rule, the method further includes:
当接收到历史日志的查询指令时，获取所述查询指令对应的查询IP地址；When receiving the query command of the historical log, obtain the query IP address corresponding to the query command;
获取并显示所述查询IP地址对应的归档日志；Obtain and display the archive log corresponding to the query IP address;
当接收到基于所述归档日志的定位关键字时，将所述定位关键字对应的归档日志以亮色显示。When a positioning keyword based on the archived log is received, the archived log corresponding to the positioning keyword is displayed in a bright color.
所述归档规则包括限流归档，所述根据所述归档规则，对所述待归档日志进行归档的步骤包括：The archiving rule includes current-limited archiving, and the step of archiving the log to be archived according to the archiving rule includes:
若所述归档规则为限流归档，则将所述待归档日志发送至所述待归档日志对应的缓存队列；If the archiving rule is current-limited archiving, sending the log to be archived to the cache queue corresponding to the log to be archived;
按照预设批次大小，对所述待归档日志进行分割，以得到至少两个分割日志；According to the preset batch size, the log to be archived is divided to obtain at least two divided logs;
获取所述分割日志中包含的关键字，并基于所述关键字赋予所述分割日志不同的优先级；Acquiring keywords contained in the segmented logs, and assigning different priorities to the segmented logs based on the keywords;
基于所述缓存队列和所述优先级，依次对所述分割日志进行归档。The split logs are sequentially archived based on the cache queue and the priority.
此外，为实现上述目的，本发明还提供一种日志管理装置，所述日志管理装置包括：In addition, in order to achieve the above object, the present invention also provides a log management device, and the log management device includes:
第一获取模块，用于当接收到输入的实例名时，基于所述实例名，获取对应的IP地址和日志路径；The first obtaining module is used to obtain the corresponding IP address and log path based on the instance name when receiving the input instance name;
第二获取模块，用于基于所述IP地址和所述日志路径，获取对应的待归档日志；A second obtaining module, configured to obtain the corresponding log to be archived based on the IP address and the log path;
选取模块，用于确定所述待归档日志的大小，并基于所述待归档日志的大小，确定对应的归档规则，所述待归档日志的大小为所占内存的大小；A selection module, configured to determine the size of the log to be archived, and determine a corresponding filing rule based on the size of the log to be archived, where the size of the log to be archived is the size of the occupied memory;
归档模块，用于根据所述归档规则，对所述待归档日志进行归档。An archiving module, configured to archive the log to be archived according to the archiving rule.
此外，为实现上述目的，本发明还提供一种日志管理设备，所述日志管理设备包括处理器、存储器、以及存储在所述存储器上并可被所述处理器执行的日志管理程序，其中所述日志管理程序被所述处理器执行时，实现如上述的日志管理方法的步骤。In addition, in order to achieve the above object, the present invention also provides a log management device, the log management device includes a processor, a memory, and a log management program stored on the memory and executable by the processor, wherein the When the log management program is executed by the processor, the steps of the log management method described above are implemented.
此外，为实现上述目的，本发明还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有日志管理程序，其中所述日志管理程序被处理器执行时，实现如上述的日志管理方法的步骤。In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium, where a log management program is stored on the computer-readable storage medium, wherein when the log management program is executed by a processor, the above-mentioned log is realized. The steps of the management method.
本发明提供一种日志管理方法，当接收到输入的实例名时，基于所述实例名，获取对应的IP地址和日志路径；基于所述IP地址和所述日志路径，获取对应的待归档日志；确定所述待归档日志的大小，并基于所述待归档日志的大小，确定对应的归档规则；根据所述归档规则，对所述待归档日志进行归档。本发明通过IP地址和日志路径，获取目标待归档日志，并根据待归档日志的大小确定归档规则，最后根据归档规则对待归档日志进行归档，无需专门定制脚本，也无需懂脚本的专门人员来处理，实现日志的智能管理，提高了日志的归档速度和归档的简易程度。The present invention provides a log management method. When an input instance name is received, a corresponding IP address and a log path are obtained based on the instance name; and a corresponding log to be archived is obtained based on the IP address and the log path. ; determine the size of the log to be archived, and determine a corresponding archiving rule based on the size of the log to be archived; and archive the log to be archived according to the archiving rule. The present invention obtains the target log to be archived through the IP address and the log path, determines the archiving rule according to the size of the log to be archived, and finally archives the log to be archived according to the archiving rule, and does not need to customize a script, nor does it need special personnel who understand the script to process , realizes the intelligent management of logs, improves the log filing speed and the simplicity of filing.
附图说明Description of drawings
图1为本发明实施例方案中涉及的日志管理设备的硬件结构示意图；FIG. 1 is a schematic diagram of a hardware structure of a log management device involved in an embodiment of the present invention;
图2为本发明日志管理方法第一实施例的流程示意图；2 is a schematic flowchart of a first embodiment of a log management method according to the present invention;
图3为本发明日志管理装置第一实施例的功能模块示意图。FIG. 3 is a schematic diagram of functional modules of the first embodiment of the log management apparatus according to the present invention.
本发明目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.
具体实施方式Detailed ways
应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.
本发明实施例涉及的日志管理方法主要应用于日志管理设备，该日志管理设备可以是PC、便携计算机、移动终端等具有显示和处理功能的设备。The log management method involved in the embodiment of the present invention is mainly applied to a log management device, and the log management device may be a device with display and processing functions, such as a PC, a portable computer, and a mobile terminal.
参照图1，图1为本发明实施例方案中涉及的日志管理设备的硬件结构示意图。本发明实施例中，日志管理设备可以包括处理器1001(例如CPU)，通信总线1002，用户接口1003，网络接口1004，存储器1005。其中，通信总线1002用于实现这些组件之间的连接通信；用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard)；网络接口1004可选的可以包括标准的有线接口、无线接口(如WI-FI接口)；存储器1005可以是高速RAM存储器，也可以是稳定的存储器(non-volatile memory)，例如磁盘存储器，存储器1005可选的还可以是独立于前述处理器1001的存储装置。Referring to FIG. 1 , FIG. 1 is a schematic diagram of a hardware structure of a log management device involved in an embodiment of the present invention. In this embodiment of the present invention, the log management device may include a processor 1001 (for example, a CPU), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Wherein, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface); the memory 1005 can be a high-speed RAM memory, or a non-volatile memory, such as a disk memory, and the memory 1005 can optionally be a storage device independent of the aforementioned processor 1001 .
本领域技术人员可以理解，图1中示出的硬件结构并不构成对日志管理设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。Those skilled in the art can understand that the hardware structure shown in FIG. 1 does not constitute a limitation on the log management device, and may include more or less components than the one shown, or combine some components, or arrange different components.
继续参照图1，图1中作为一种计算机可读存储介质的存储器1005可以包括操作系统、网络通信模块以及日志管理程序。Continuing to refer to FIG. 1 , the memory 1005 as a computer-readable storage medium in FIG. 1 may include an operating system, a network communication module, and a log management program.
在图1中，网络通信模块主要用于连接服务器，与服务器进行数据通信；而处理器1001可以调用存储器1005中存储的日志管理程序，并执行本发明实施例提供的日志管理方法。In FIG. 1 , the network communication module is mainly used to connect to the server and perform data communication with the server; and the processor 1001 can call the log management program stored in the memory 1005 and execute the log management method provided by the embodiment of the present invention.
本发明实施例提供了一种日志管理方法，该方法可运用在日志管理设备中，日志管理设备以下简称管理设备。An embodiment of the present invention provides a log management method, which can be applied to a log management device, and the log management device is hereinafter referred to as a management device.
参照图2，图2为本发明日志管理方法第一实施例的流程示意图。Referring to FIG. 2, FIG. 2 is a schematic flowchart of a first embodiment of a log management method according to the present invention.
本实施例中，所述日志管理方法包括以下步骤：In this embodiment, the log management method includes the following steps:
步骤S10，当接收到输入的实例名时，基于所述实例名，获取对应的IP地址和日志路径；Step S10, when receiving the input instance name, obtain the corresponding IP address and log path based on the instance name;
步骤S20，基于所述IP地址和所述日志路径，获取对应的待归档日志；Step S20, based on the IP address and the log path, obtain the corresponding log to be archived;
步骤S30，确定所述待归档日志的大小，并基于所述待归档日志的大小，确定对应的归档规则，所述待归档日志的大小为所占内存的大小；Step S30, determine the size of the log to be archived, and determine the corresponding filing rule based on the size of the log to be archived, where the size of the log to be archived is the size of the occupied memory;
步骤S40，根据所述归档规则，对所述待归档日志进行归档。Step S40: Archive the log to be archived according to the archiving rule.
本实施例通过IP地址和日志路径，获取待归档日志，并根据待归档日志的大小确定归档规则，最后根据归档规则对待归档日志进行归档，无需专门定制脚本，也无需懂脚本的专门人员来处理，实现日志的智能管理，提高了日志的归档速度和归档的简易程度。In this embodiment, the logs to be archived are obtained through the IP address and the log path, and the archiving rules are determined according to the size of the logs to be archived, and finally the logs to be archived are archived according to the archiving rules. There is no need to customize scripts, and no special personnel who understand scripts are required to process them. , realizes the intelligent management of logs, improves the log filing speed and the simplicity of filing.
以下将对各个步骤进行详细的说明：Each step will be explained in detail below:
步骤S10，当接收到输入的实例名时，基于所述实例名，获取对应的IP地址和日志路径。Step S10, when the input instance name is received, obtain the corresponding IP address and log path based on the instance name.
本实施例中，用户在对某个服务器的日志进行归档的时候，只需要在管理系统对应的显示界面输入实例名，管理系统即可根据接收到的实例名获取对应的IP地址和日志路径，其中，实例名指的是用户对每个服务器的主机所起的名字，即管理设备连接多台服务器，并且每一个服务器的主机都有用户赋予的实例名。如，用户想要获取A服务器对应的日志信息，只需在管理系统中输入A服务器的实例名A，管理系统即去获取A服务器对应的IP地址和日志路径。In this embodiment, when the user archives the log of a certain server, he only needs to input the instance name on the display interface corresponding to the management system, and the management system can obtain the corresponding IP address and log path according to the received instance name. The instance name refers to the name given by the user to the host of each server, that is, the management device is connected to multiple servers, and the host of each server has an instance name assigned by the user. For example, if the user wants to obtain the log information corresponding to the A server, he only needs to enter the instance name A of the A server in the management system, and the management system will obtain the IP address and log path corresponding to the A server.
进一步的，步骤S10包括：Further, step S10 includes:
步骤S11，当接收到输入的实例名时，判断所述实例名是否有效；Step S11, when receiving the input instance name, determine whether the instance name is valid;
步骤S12，若有效，则获取所述实例名对应的归档链接；Step S12, if valid, obtain the archive link corresponding to the instance name;
步骤S13，获取所述归档链接对应的IP地址和所述归档链接对应的日志路径。Step S13: Obtain the IP address corresponding to the archive link and the log path corresponding to the archive link.
当接收到输入的实例名时，获取所述实例名对应的归档链接，并基于所述归档链接，获取对应的IP地址和日志路径。When the input instance name is received, the archive link corresponding to the instance name is obtained, and based on the archive link, the corresponding IP address and log path are obtained.
可以理解的，管理设备连接有多台服务器，可对每台服务器进行管理，每台服务器发生事件时，都各自记录事件对应的日志信息，管理设备可事先创建一个数据库，用于储存用户在管理设备对应的操作界面输入的实例名、IP地址和日志路径，并基于SQL语音(Structured Query Language，结构化查询语言)将将该实例名、IP地址和日志路径进行关联，生成归档链接。当管理系统接收到输入的实例名时，即可获取到该实例名对应的归档链接，并基于该归档链接获得对应的IP地址和日志路径。It can be understood that the management device is connected to multiple servers and can manage each server. When an event occurs on each server, it records the log information corresponding to the event. The management device can create a database in advance to store the user's management The instance name, IP address and log path entered in the corresponding operation interface of the device will be associated with the instance name, IP address and log path based on SQL voice (Structured Query Language, Structured Query Language) to generate an archive link. When the management system receives the input instance name, it can obtain the archive link corresponding to the instance name, and obtain the corresponding IP address and log path based on the archive link.
可以理解的，在接收到输入的实例名时，还包括对该实例名进行验证，判断管理系统中是否存在该实例名，即在接收到输入的实例名时，判断该实例名是否有效，如管理设备管理有A服务器和B服务器，那么当实例名为C时，由于管理设备中不存在C，那么管理设备输出错误提示，提示用户不存在该实例名。It can be understood that when receiving the input instance name, it also includes verifying the instance name, and judging whether the instance name exists in the management system, that is, when receiving the input instance name, judging whether the instance name is valid, such as The management device manages the A server and the B server. When the instance name is C, since C does not exist in the management device, the management device outputs an error prompt, prompting the user that the instance name does not exist.
步骤S20，基于所述IP地址和所述日志路径，获取对应的待归档日志。Step S20, based on the IP address and the log path, obtain the corresponding log to be archived.
本实施例中，管理设备根据IP地址和日志路径，确定哪一服务器中的哪一位置是待归档日志，具体的，通过IP地址确定IP地址对应的服务器，在该服务器中，通过日志路径确定对应位置的日志为待归档日志。In this embodiment, the management device determines which server and which location is the log to be archived according to the IP address and the log path. Specifically, the server corresponding to the IP address is determined by the IP address, and in the server, the log path is determined by the log path. The log in the corresponding location is the log to be archived.
可以理解的，在服务器中日志信息有多种多样，并且这些日志信息缓存在服务器对应的位置，有些日志信息是用户需要的，有些日志信息是用户不需要的，因此，管理系统需先通过IP地址，确定对应的服务器，再在该服务器中，确定日志路径对应的日志信息是用户需要的待归档日志。It is understandable that there are various log information in the server, and the log information is cached in the corresponding location of the server. Some log information is required by the user, and some log information is not required by the user. Therefore, the management system needs to pass the IP address first. address, determine the corresponding server, and then in the server, determine that the log information corresponding to the log path is the log to be archived required by the user.
若实例名对应的IP地址有一个，日志路径有多个，即归档链接中的IP地址对应的日志路径有多个，则在该IP地址对应的服务器中获取多个日志路径对应的待归档日志；若实例名对应的IP地址有多个，日志路径有一个，即归档链接中有多个IP地址对应同一日志路径，则在多个IP地址对应的服务器中获取该日志路径对应的待归档日志；若实例名对应的IP地址有多个，日志路径有多个，即归档链接中有多个IP地址对应多个日志路径，则在多个IP地址对应的服务器中获取多个日志路径对应的待归档日志。If there is one IP address corresponding to the instance name and multiple log paths, that is, there are multiple log paths corresponding to the IP address in the archive link, the logs to be archived corresponding to the multiple log paths are obtained from the server corresponding to the IP address. ;If there are multiple IP addresses corresponding to the instance name and one log path, that is, if there are multiple IP addresses in the archive link corresponding to the same log path, the log to be archived corresponding to the log path is obtained from the servers corresponding to the multiple IP addresses. ;If there are multiple IP addresses corresponding to the instance name and multiple log paths, that is, there are multiple IP addresses corresponding to multiple log paths in the archive link, then the corresponding log paths are obtained from the servers corresponding to multiple IP addresses. Logs to be archived.
进一步的，在获取待归档日志的过程中还包括：Further, the process of obtaining the log to be archived also includes:
基于所述IP地址和所述日志路径，确定是目标服务器否存在对应的待归档日志，若存在，则获取所述待归档日志。Based on the IP address and the log path, it is determined whether the target server has a corresponding log to be archived, and if so, the log to be archived is acquired.
即在获取待归档日志之前，管理设备需判断目标服务器是否存在对应的待归档日志，具体的，判断实例名对应的IP地址是否有对应的服务器，若无，则确定该实例名无效；若有，则进一步判断实例名对应的日志路径在该服务器中是否存在，若无，则确定目标服务器不存在对应的待归档日志，若有，则执行获取待归档日志。That is, before obtaining the log to be archived, the management device needs to determine whether the target server has the corresponding log to be archived. Specifically, it determines whether the IP address corresponding to the instance name has a corresponding server. If not, the instance name is determined to be invalid; , then it is further judged whether the log path corresponding to the instance name exists in the server, if not, it is determined that the target server does not have the corresponding log to be archived, and if so, execute the acquisition of the log to be archived.
可以理解的，管理设备管理的服务器是有限的，对于不在管理设备管辖下的服务器，管理设备无权获取其日志信息，并且对于虽然在管理设备的管辖下，但是没有实例名对应的日志路径的服务器，管理系统也无法获取到对应的待归档日志。只有在管理系统管辖下的服务器，且存在对应的日志路径，管理系统才可获取到对应的待归档日志。It is understandable that the servers managed by the management device are limited. For servers that are not under the jurisdiction of the management device, the management device has no right to obtain their log information, and for servers that are under the jurisdiction of the management device but do not have the log path corresponding to the instance name. The server and management system cannot obtain the corresponding log to be archived. The management system can obtain the corresponding log to be archived only if the server is under the jurisdiction of the management system and there is a corresponding log path.
步骤S30，确定所述待归档日志的大小，并基于所述待归档日志的大小，确定对应的归档规则，所述待归档日志的大小为所占内存的大小。Step S30: Determine the size of the log to be archived, and determine a corresponding archiving rule based on the size of the log to be archived, where the size of the log to be archived is the size of the memory occupied.
本实施例中，根据获取到的待归档日志，确定待归档日志的大小，并基于待归档日志的大小，确定对应的归档规则，其中，待归档日志的大小为待归档日志所占内存的大小。In this embodiment, the size of the log to be archived is determined according to the obtained log to be archived, and the corresponding archiving rule is determined based on the size of the log to be archived, wherein the size of the log to be archived is the size of the memory occupied by the log to be archived .
具体的，确定待归档日志的大小，并将待归档日志的大小与预设阈值进行比较，若待归档日志的大小小于预设阈值，则确定对应的归档规则为直接归档；若待归档日志的大小等于或者大于预设阈值，则确定对应的归档规则为限流归档，其中，限流归档具体表现为将待归档日志加入缓存队列，待归档日志将根据缓存队列分批归档，具体预设一个批次大小，根据批次大小，将待归档日志分割成若干批次，依次将分成若干批次的待归档日志进行归档，此时管理设备不会因为数据一次性写入过多而崩溃。Specifically, the size of the log to be archived is determined, and the size of the log to be archived is compared with a preset threshold. If the size of the log to be archived is smaller than the preset threshold, the corresponding archiving rule is determined to be direct archive; If the size is equal to or greater than the preset threshold, the corresponding archiving rule is determined to be current-limited archiving. The current-limited archiving is embodied by adding the logs to be archived to the cache queue, and the logs to be archived will be archived in batches according to the cache queue. Batch size. According to the batch size, the log to be archived is divided into several batches, and the batches of logs to be archived are archived in turn. At this time, the management device will not crash due to too much data being written at one time.
步骤S40，根据所述归档规则，对所述待归档日志进行归档。Step S40: Archive the log to be archived according to the archiving rule.
本实施例中，在确定了归档规则后，根据归档规则对待归档日志进行归档，具体的，可通过复制或者剪切的方式，获取待归档日志，并依据确定归档规则对待归档日志进行归档。In this embodiment, after the archiving rule is determined, the log to be archived is archived according to the archiving rule. Specifically, the log to be archived can be obtained by copying or cutting, and the log to be archived can be archived according to the determined archiving rule.
进一步地，当归档规则为限流归档时，步骤S40包括：Further, when the filing rule is current-limited filing, step S40 includes:
若所述归档规则为限流归档，则将所述待归档日志发送至所述待归档日志对应的缓存队列；If the archiving rule is current-limited archiving, sending the log to be archived to the cache queue corresponding to the log to be archived;
在该步骤中，若归档规则为限流归档，即待归档日志的大小等于或者大于预设阈值，为避免管理一次性写入过多的数据而导致对应的系统崩溃，则将待归档日志先发送至缓存队列中进行分割处理，而不是直接归档，可以理解的，缓存队列中存在当前待归档日志的同时，也可能存在其他待归档日志，即管理设备在将待归档日志发送至缓存队列后，进行分割，并将分割所得的分割日志进行排队等候，以便后续能依次对分割日志进行归档。In this step, if the archiving rule is current-limited archiving, that is, the size of the log to be archived is equal to or greater than the preset threshold, in order to avoid the management of writing too much data at one time and causing the corresponding system to crash, the log to be archived is first stored. It is sent to the cache queue for split processing instead of being archived directly. It is understandable that there may be other logs to be archived while the current log to be archived exists in the cache queue, that is, after the management device sends the log to be archived to the cache queue , perform segmentation, and queue up the segmented logs obtained from the segmentation, so that the segmented logs can be archived in sequence.
按照预设批次大小，对所述待归档日志进行分割，以得到至少两个分割日志；According to the preset batch size, the log to be archived is divided to obtain at least two divided logs;
在该步骤中，管理设备根据预设批次大小，对待归档日志进行分割，以得到至少两个分割日志，如当前待归档日志的大小为2G(Gigabyte，吉咖字节或京字节或十亿字节或戟)，预设批次大小为1G，则管理设备将当前待归档日志分割为两个1G大小的分割日志。In this step, the management device divides the log to be archived according to the preset batch size to obtain at least two divided logs. 100 million bytes or halberd), and the preset batch size is 1G, then the management device divides the current log to be archived into two 1G split logs.
获取所述分割日志中包含的关键字，并基于所述关键字赋予所述分割日志不同的优先级；Acquiring keywords contained in the segmented logs, and assigning different priorities to the segmented logs based on the keywords;
在该步骤中，管理设备获取各个分割日志中包含的关键字，并根据关键字对分割日志进行优先级区分，关键字可指待归档日志的属性，如待归档日志为安全日志，则其对应的关键字为安全；待归档日志为调度日志时，其对应的关键字为调度等。In this step, the management device obtains keywords contained in each segmented log, and prioritizes the segmented logs according to the keywords. The keywords may refer to the attributes of the logs to be archived. If the logs to be archived are security logs, the corresponding The keyword is security; when the log to be archived is a scheduling log, the corresponding keyword is scheduling, etc.
事先对各类日志的重要程度以阿拉伯数字1、2、3......进行分级，1级最高，依次降低，如安全日志的优先级为1级，调度日子为3级，设备日志为2级等，管理设备在将当前待归档日志进行分割后，对分割日志日志进行优先级赋予。The importance of various types of logs is graded in advance with Arabic numerals 1, 2, 3..., with 1 being the highest and decreasing in turn. For example, the priority of security logs is level 1, scheduling days is level 3, and equipment logs are level 3. For level 2, the management device assigns priority to the split log after splitting the current log to be archived.
基于所述缓存队列和所述优先级，依次对所述分割日志进行归档。The split logs are sequentially archived based on the cache queue and the priority.
在该步骤中，管理设备根据缓存队列的排队情况，以及确定的优先级，以优先级从高到低的顺序，依次对分割日志进行归档，直至所有的分割日志归档成功，此时，完成待归档日志的归档。In this step, according to the queuing situation of the cache queue and the determined priority, the management device archives the split logs in order from high to low until all the split logs are successfully archived. Archive of archived logs.
需要说明的是，在根据优先级进行依次归档的过程中，由于缓存队列可能存在其他的待归档日志，因此，分割日志的优先级优选针对分割日志所对应的同一待归档日志，如当前缓存队列中有待归档日志G和待归档日志H，其中待归档日志G被分割为优先级为1的分割日志f和优先级为2的分割日志g，而待归档日志H被分割为优先级为5的分割日志h和优先级为2的分割日志i，但待归档日志H在缓存队列中排在待归档日志G之前，因此，正确的归档顺序为分割日志i、分割日志h，分割日志f和分割日志g，即不管分割日志f的优先级有多高，其也要等排在前面的待归档日志H归档完成才进行归档。It should be noted that in the process of archiving sequentially according to the priority, since there may be other logs to be archived in the cache queue, the priority of the split log is preferably the same log to be archived corresponding to the split log, such as the current cache queue. There are log G to be archived and log H to be archived, wherein log G to be archived is split into split log f with priority 1 and split log g with priority 2, while log H to be archived is split into log with priority 5 Split log h and split log i with priority 2, but log H to be archived is queued before log G to be archived in the cache queue. Therefore, the correct archiving order is split log i, split log h, split log f and split log The log g, that is, no matter how high the priority of the split log f is, is to be archived after the archiving of the log H to be archived in front is completed.
进一步的，步骤S40包括：Further, step S40 includes:
步骤S41，确定所述待归档日志的日志类型，并根据所述日志类型确定所述待归档日志对应的归档位置。Step S41: Determine the log type of the log to be archived, and determine the archive location corresponding to the log to be archived according to the log type.
在获取到待归档日志后，通过判断待归档日志的日志类型，确定待归档日志的归档位置，其中，日志类型包括应用程序日志、安全日志、Scheduler服务日志、FTP日志、WWW日志、DNS服务器日志等，管理设备根据待归档日志的日志类型，确定待归档日志的归档位置。After obtaining the log to be archived, determine the archive location of the log to be archived by judging the log type of the log to be archived, wherein the log type includes application log, security log, Scheduler service log, FTP log, WWW log, DNS server log and so on, the management device determines the archive location of the log to be archived according to the log type of the log to be archived.
步骤S42，根据所述归档规则，将所述待归档日志归档至所述归档位置。Step S42: Archive the log to be archived to the archive location according to the archive rule.
在确定了待归档日志的归档位置后，根据归档规则，将待归档日志归档至该归档位置。即每一个待归档日志都有对应的归档位置与之对应。After the archiving position of the log to be archived is determined, according to the archiving rule, the log to be archived is archived to the archiving position. That is, each log to be archived has a corresponding archive location corresponding to it.
需要说明的是，归档位置还可通过归档链接中的日志路径确定，具体可根据日志路径，在管理设备中新建一个对应的归档路径，将获取到的待归档日志归档至归档路径对应的归档位置。It should be noted that the archive location can also be determined by the log path in the archive link. Specifically, a corresponding archive path can be created in the management device according to the log path, and the obtained logs to be archived can be archived to the archive location corresponding to the archive path. .
进一步的，在将待归档日志进行归档的过程中，还可预设一个归档周期，管理设备根据归档周期定时对待归档日志进行归档。Further, in the process of archiving the logs to be archived, an archiving period may be preset, and the management device regularly archives the logs to be archived according to the archiving period.
本实施例当接收到输入的实例名时，基于所述实例名，获取对应的IP地址和日志路径；基于所述IP地址和所述日志路径，获取对应的待归档日志；确定所述待归档日志的大小，并基于所述待归档日志的大小，确定对应的归档规则；根据所述归档规则，对所述待归档日志进行归档。本发明通过IP地址和日志路径，获取目标待归档日志，并根据待归档日志的大小确定归档规则，最后根据归档规则对待归档日志进行归档，无需专门定制脚本，也无需懂脚本的专门人员来处理，并且不会因为一次性写入过多数据而导致系统崩溃，实现日志的智能管理，提高了日志的归档速度、归档的简易程度和管理设备的系统稳定性。In this embodiment, when the input instance name is received, the corresponding IP address and log path are obtained based on the instance name; the corresponding log to be archived is obtained based on the IP address and the log path; the log to be archived is determined The size of the log is determined, and based on the size of the log to be archived, a corresponding archiving rule is determined; according to the archiving rule, the log to be archived is archived. The present invention obtains the target log to be archived through the IP address and the log path, determines the archiving rule according to the size of the log to be archived, and finally archives the log to be archived according to the archiving rule, and does not need to customize a script, nor does it need special personnel who understand the script to process , and will not cause the system to crash due to writing too much data at one time, realize the intelligent management of logs, improve the log filing speed, the simplicity of filing and the system stability of the management device.
进一步地，基于第一实施例提出本发明日志管理方法的第二实施例。日志管理方法的第二实施例与日志管理方法的第一实施例的区别在于，步骤S40包括：Further, based on the first embodiment, a second embodiment of the log management method of the present invention is proposed. The difference between the second embodiment of the log management method and the first embodiment of the log management method is that step S40 includes:
步骤S43，监测所述待归档日志对应的归档任务是否出现异常状态；Step S43, monitoring whether the archiving task corresponding to the log to be archived has an abnormal state;
步骤S44，若所述归档任务出现异常状态，则确定所述异常状态对应的补救策略；Step S44, if an abnormal state occurs in the archiving task, then determine a remedy strategy corresponding to the abnormal state;
步骤S45，根据所述归档规则和所述补救策略，对所述待归档日志进行归档。Step S45: Archive the log to be archived according to the archiving rule and the remediation policy.
本实施例中，在对待归档日志进行归档的过程中，管理设备对应创建归档任务，当监测到归档任务出现异常时，获取对应的补救策略，通过补救策略和归档规则对待归档日志进行归档。In this embodiment, in the process of archiving to-be-archived logs, the management device correspondingly creates an archiving task, and when an abnormality is detected in the archiving task, it obtains a corresponding remediation policy, and archives the to-be-archived logs through the remediation policy and archiving rules.
以下将对各个步骤进行详细说明：The individual steps are explained in detail below:
步骤S43，监测所述待归档日志对应的归档任务是否出现异常状态。Step S43, monitoring whether the archiving task corresponding to the log to be archived has an abnormal state.
本实施例中，管理设备在对待归档日志进行归档时，创建有对应的归档任务，通过归档任务可及时获取待归档日志的归档情况，管理设备实时监测归档任务的进程，判断其是否出现异常状态。In this embodiment, when archiving logs to be archived, the management device creates a corresponding archiving task, and through the archiving task, the archiving status of the log to be archived can be obtained in time, and the management device monitors the progress of the archiving task in real time to determine whether an abnormal state occurs. .
进一步的，步骤S43包括：Further, step S43 includes:
步骤a，获取所述待归档日志对应的归档状态码；Step a, obtaining the archiving status code corresponding to the log to be archived;
步骤b，根据所述归档状态码，监测所述归档任务是否出现异常状态。Step b, according to the archiving status code, monitor whether the archiving task has an abnormal state.
在管理设备对待归档日志进行归档的过程中，实时监控归档任务的归档状态，具体监控方式可以是将获取到的待归档日志与管理设备中待归档日志对应的已归档日志进行比较，如将待归档日志的大小与已归档日志的大小进行比较，若待归档日志等于已归档日志，则判定待归档日志处于归档成功的状态；若待归档日志小于已归档日志，则判定待归档日志处于归档中的状态；在此基础上，若在预设时间内，已归档日志未增加，则判定待归档日志处于归档中断的状态；若管理设备与待归档日志对应的服务器连接中断，则判定待归档日志处于归档失败的状态，其中，确定管理设备与待归档日志对应的服务器之间连接是否中断的方式可以是：管理设备定时向待归档日志对应的服务器发送网络存活检测包，以检测两者的连接是否中断。将日志的归档状态(包括归档成功、归档中、归档中断和归档失败等)与归档状态码(如f_log_archive表)关联记录在数据库中，如archive_code＝0，表示归档成功；archive_code＝1，表示归档中；archive_code＝-1，表示归档中断；archive_code＝-2，表示归档失败。During the process of archiving the log to be archived by the management device, the archive status of the archiving task is monitored in real time. The specific monitoring method may be to compare the obtained log to be archived with the archived log corresponding to the log to be archived in the management device. The size of the archived log is compared with the size of the archived log. If the to-be-archived log is equal to the archived log, it is determined that the to-be-archived log is in a successful archiving state; if the to-be-archived log is smaller than the archived log, it is determined that the to-be-archived log is being archived On this basis, if the archived log does not increase within the preset time, it is determined that the log to be archived is in the state of archive interruption; if the connection between the management device and the server corresponding to the log to be archived is interrupted, it is determined that the log to be archived In a state of archiving failure, the method of determining whether the connection between the management device and the server corresponding to the log to be archived is interrupted may be: the management device periodically sends a network survival detection packet to the server corresponding to the log to be archived to detect the connection between the two whether to interrupt. Record the archive status of the log (including archive success, archive in progress, archive interruption, and archive failure, etc.) and the archive status code (such as the f_log_archive table) in the database. For example, archive_code=0, indicating successful archive; archive_code=1, indicating archive Medium; archive_code=-1, indicating that the archive is interrupted; archive_code=-2, indicating that the archive has failed.
管理设备通过获取待归档日志对应的归档状态码，并根据归档状态码确定归档任务是否出现异常，其中，归档成功和归档中表示归档任务正常，归档中断和归档失败表示归档任务异常。The management device obtains the archiving status code corresponding to the log to be archived, and determines whether the archiving task is abnormal according to the archiving status code.
步骤S44，若所述归档任务出现异常状态，则确定所述异常状态对应的补救策略。Step S44, if an abnormal state occurs in the archiving task, determine a remedy strategy corresponding to the abnormal state.
若管理设备确定归档任务出现异常，则根据异常状态确定对应的补救策略，可以理解的，在管理设备中，预设有各个异常状态对应的补救策略。If the management device determines that the archiving task is abnormal, it determines a corresponding remediation strategy according to the abnormal state. It can be understood that the management device is preset with a remedial strategy corresponding to each abnormal state.
具体的，若在归档过程中出现异常，如服务器关机，断点等，则采取对应的补救策略，如在归档过程中遇到服务器关机时，等服务器恢复时，管理设备会继续对之前未归档完的待归档日志进行归档，具体的，当检测到日志数据中断时，记录当前已归档日志的位置标记，并在服务器恢复时，根据该位置标记，定位待归档日志的中断位置，继续获取未归档完的待归档日志。Specifically, if an exception occurs during the archiving process, such as server shutdown, breakpoints, etc., corresponding remedial strategies are adopted. For example, when the server is shut down during the archiving process, when the server is restored, the management device will continue to record the files that have not been archived before. The completed log to be archived is archived. Specifically, when the log data interruption is detected, the position mark of the currently archived log is recorded, and when the server recovers, according to the position mark, locate the interruption position of the log to be archived, and continue to obtain the undocumented log. Archived logs to be archived.
补救策略还包括对待归档日志的重新获取以及重新归档，即在归档过程中若出现归档失败，则放弃本次归档，删除已归档的日志，并重新根据IP地址和日志路径获取待归档日志，对待归档日志进行重新归档等。The remediation strategy also includes re-acquiring and re-archiving the logs to be archived, that is, if the archiving fails during the archiving process, the archiving will be abandoned, the archived logs will be deleted, and the logs to be archived will be re-acquired according to the IP address and log path. Archive logs for re-archiving, etc.
进一步的，步骤S44包括：Further, step S44 includes:
步骤c，若所述归档任务出现异常状态，则根据所述归档状态码，确定所述异常状态对应的异常类型。In step c, if an abnormal state occurs in the archiving task, the abnormal type corresponding to the abnormal state is determined according to the archiving state code.
在本实施例中，将所有的归档过程中出现的所有异常状态分为若干异常类型，管理设备在确定归档任务出现异常状态后，根据归档状态码，确定对应的异常类型，如上所述，异常类型可包括archive_code＝-1，归档中断；archive_code＝-2，归档失败两种。In this embodiment, all abnormal states that occur in all archiving processes are divided into several abnormal types. After determining that an abnormal state occurs in the archiving task, the management device determines the corresponding abnormal type according to the archiving state code. The types can include archive_code=-1, archive interruption; archive_code=-2, archive failure.
步骤d，确定所述异常类型对应的补救策略。Step d, determining a remedy strategy corresponding to the exception type.
根据异常类型，管理设备采取不同的补救策略，具体的，当异常类型为归档中断时，对当前已归档日志的中断位置进行标记，并记录，并检测传输路径是否通畅，具体可通过想目标服务器发送检测包测试传输路径是否通畅，当检测到传输路径通畅时，根据标记，定位待归档日志未归档完成的位置，继续获取未归档完成的待归档日志并进行归档；当异常类型为归档失败时，删除待归档日志中已归档的部分，重新根据IP地址和日志路径获取待归档日志并进行归档。According to the abnormal type, the management device adopts different remedial strategies. Specifically, when the abnormal type is archive interruption, it will mark and record the interruption position of the currently archived log, and detect whether the transmission path is smooth. Send a detection packet to test whether the transmission path is smooth. When it is detected that the transmission path is smooth, locate the position where the log to be archived has not been archived according to the mark, and continue to obtain the log to be archived that has not been archived and archive it; when the exception type is archiving failure , delete the archived part of the log to be archived, obtain and archive the log to be archived again according to the IP address and log path.
步骤S45，根据所述归档规则和所述补救策略，对所述待归档日志进行归档；Step S45, archiving the log to be archived according to the archiving rule and the remedial strategy;
在确定了补救策略后，即可根据归档规则和补救策略对待归档日志进行归档。After the remediation strategy is determined, the to-be-archived logs can be archived according to the archiving rules and the remediation strategy.
本实施例在对待归档日志进行归档的过程中，管理设备对应创建归档任务，当监测到归档任务出现异常时，获取对应的补救策略，通过补救策略和归档规则对待归档日志进行归档，实现日志的智能管理，提高日志的归档成功率。In the process of archiving the log to be archived in this embodiment, the management device creates an archiving task correspondingly, and when an abnormality of the archiving task is detected, a corresponding remedial policy is obtained, and the to-be-archived log is archived through the remediation policy and the archiving rule, so that the log can be archived. Intelligent management improves the success rate of log archiving.
进一步的，基于第一实施例和第二实施例提出本发明日志管理方法的第三实施例。日志管理方法的第三实施例与日志管理方法的第一实施例和第二实施例的区别在于，所述方法还包括：Further, based on the first embodiment and the second embodiment, a third embodiment of the log management method of the present invention is proposed. The difference between the third embodiment of the log management method and the first and second embodiments of the log management method is that the method further includes:
步骤S50，当接收到历史日志的查询指令时，获取所述查询指令对应的查询IP地址；Step S50, when receiving the query command of the history log, obtain the query IP address corresponding to the query command;
步骤S60，获取并显示所述查询IP地址对应的归档日志；Step S60, acquiring and displaying the archive log corresponding to the query IP address;
步骤S70，当接收到基于所述归档日志的定位关键字时，将所述定位关键字对应的归档日志以亮色显示。Step S70, when receiving the locating keyword based on the archived log, display the archived log corresponding to the locating keyword in a bright color.
本实施例中，在将日志归档之后，若接收到历史日志的查询指令，则可直接在显示界面显示对应的日志，并且可根据用户输入的关键字迅速定位用户想要找的日志。In this embodiment, after the log is archived, if a query command of the history log is received, the corresponding log can be displayed directly on the display interface, and the log that the user wants to find can be quickly located according to the keyword input by the user.
以下将对各个步骤进行详细说明：The individual steps are explained in detail below:
步骤S50，当接收到历史日志的查询指令时，获取所述查询指令对应的查询IP地址。Step S50, when a query command of the history log is received, obtain the query IP address corresponding to the query command.
在本实施例中，在对待归档日志进行归档之后，若用户想要查看相关的日志信息，只需在对应的查询界面输入对应的IP地址，管理设备在接收到查询指令时，即可获取到对应的IP地址。可以理解的，由于本方案引入实例名的概念，因此，用户在实际操作中即使不知道想要查看的服务器的IP地址，也可以通过输入实例名进行查看。In this embodiment, after the log to be archived is archived, if the user wants to view the relevant log information, he only needs to enter the corresponding IP address in the corresponding query interface, and the management device can obtain the query instruction when receiving the query instruction. the corresponding IP address. It is understandable that since this solution introduces the concept of instance name, users can view by entering the instance name even if they do not know the IP address of the server they want to view in actual operation.
步骤S60，获取并显示所述查询IP地址对应的归档日志。Step S60, acquiring and displaying the archive log corresponding to the query IP address.
在本实施例中，管理设备在确定查询IP地址后，获取并显示对应的归档日志，可以理解的，此时显示的归档日志是该查询IP地址对应的服务器的全部日志信息。In this embodiment, after determining the query IP address, the management device obtains and displays the corresponding archive log. It can be understood that the archive log displayed at this time is all log information of the server corresponding to the query IP address.
步骤S70，当接收到基于所述归档日志的定位关键字时，将所述定位关键字对应的归档日志以亮色显示。Step S70, when receiving the locating keyword based on the archive log, display the archive log corresponding to the locating keyword in a bright color.
在本实施例中，管理设备还具备定位功能，用户只需在相关显示界面输入定位关键字，管理设备在接收到定位关键字后，即可快速定位对应的归档日志，并且，为了方便用户查看，将定位的归档日志以亮色显示。In this embodiment, the management device also has a positioning function. The user only needs to input the positioning keyword on the relevant display interface. After receiving the positioning keyword, the management device can quickly locate the corresponding archived log, and for the convenience of the user to view to display the located archive logs in a bright color.
本实施例在将日志归档之后，若接收到历史日志的查询指令，则可直接在显示界面显示对应的日志，并且可根据用户输入的关键字迅速定位用户想要找的日志，实现日志的快速查看。In this embodiment, after the log is archived, if a query command of the historical log is received, the corresponding log can be displayed directly on the display interface, and the log that the user wants to find can be quickly located according to the keyword input by the user, so as to realize the rapidity of the log. Check.
此外，本发明实施例还提供一种日志管理装置。In addition, an embodiment of the present invention also provides a log management apparatus.
参照图3，图3为本发明日志管理装置第一实施例的功能模块示意图。Referring to FIG. 3 , FIG. 3 is a schematic diagram of functional modules of the first embodiment of the log management apparatus of the present invention.
本实施例中，所述日志管理装置包括：In this embodiment, the log management device includes:
第一获取模块10，用于当接收到输入的实例名时，基于所述实例名，获取对应的IP地址和日志路径；The first obtaining module 10 is used to obtain the corresponding IP address and log path based on the instance name when receiving the input instance name;
第二获取模块20，用于基于所述IP地址和所述日志路径，获取对应的待归档日志；The second obtaining module 20 is configured to obtain the corresponding log to be archived based on the IP address and the log path;
选取模块30，用于确定所述待归档日志的大小，并基于所述待归档日志的大小，确定对应的归档规则，所述待归档日志的大小为所占内存的大小；The selection module 30 is used to determine the size of the log to be archived, and based on the size of the log to be archived, determine the corresponding filing rule, and the size of the log to be archived is the size of the occupied memory;
归档模块40，用于根据所述归档规则，对所述待归档日志进行归档。The archiving module 40 is configured to archive the log to be archived according to the archiving rule.
进一步地，所述归档模块40还用于：Further, the filing module 40 is also used for:
确定所述待归档日志的日志类型，并根据所述日志类型确定所述待归档日志对应的归档位置；Determine the log type of the log to be archived, and determine the archive location corresponding to the log to be archived according to the log type;
根据所述归档规则，将所述待归档日志归档至所述归档位置。According to the archiving rule, the log to be archived is archived to the archive location.
进一步地，所述归档模块40还用于：Further, the filing module 40 is also used for:
监测所述待归档日志对应的归档任务是否出现异常状态，若所述归档任务出现异常状态，则确定所述异常状态对应的补救策略；Monitoring whether the archiving task corresponding to the log to be archived has an abnormal state, and if the archiving task has an abnormal state, determining a remedy strategy corresponding to the abnormal state;
根据所述归档规则和所述补救策略，对所述待归档日志进行归档。The log to be archived is archived according to the archiving rule and the remediation policy.
进一步地，所述归档模块40还用于：Further, the filing module 40 is also used for:
获取所述待归档日志对应的归档状态码，并根据所述归档状态码，监测所述归档任务是否出现异常状态；Acquiring the archiving status code corresponding to the log to be archived, and monitoring whether an abnormal state occurs in the archiving task according to the archiving status code;
若所述归档任务出现异常状态，则根据所述归档状态码，确定所述异常状态对应的异常类型；If an abnormal state occurs in the archiving task, determining the abnormal type corresponding to the abnormal state according to the archiving state code;
确定所述异常类型对应的补救策略。Determine the remediation strategy corresponding to the exception type.
进一步地，所述第一获取模块10还用于：Further, the first acquisition module 10 is also used for:
当接收到输入的实例名时，判断所述实例名是否有效；When receiving the input instance name, determine whether the instance name is valid;
若有效，则获取所述实例名对应的归档链接；If valid, obtain the archive link corresponding to the instance name;
获取所述归档链接对应的IP地址和所述归档链接对应的日志路径。Obtain the IP address corresponding to the archive link and the log path corresponding to the archive link.
进一步地，所述日志管理装置还包括：Further, the log management device also includes:
接收模块，用于当接收到历史日志的查询指令时，获取所述查询指令对应的查询IP地址；a receiving module, configured to obtain the query IP address corresponding to the query command when receiving the query command of the historical log;
第三获取模块，用于获取并显示所述查询IP地址对应的归档日志；The third acquisition module is used to acquire and display the archive log corresponding to the query IP address;
定位模块，用于当接收到基于所述归档日志的定位关键字时，将所述定位关键字对应的归档日志以亮色显示。The locating module is configured to display the archive log corresponding to the locating keyword in bright color when the locating keyword based on the archive log is received.
进一步地，所述归档规则包括限流归档，所述归档模块40还用于：Further, the filing rule includes current-limiting filing, and the filing module 40 is also used for:
若所述归档规则为限流归档，则将所述待归档日志发送至所述待归档日志对应的缓存队列；If the archiving rule is current-limited archiving, sending the log to be archived to the cache queue corresponding to the log to be archived;
按照预设批次大小，对所述待归档日志进行分割，以得到至少两个分割日志；According to the preset batch size, the log to be archived is divided to obtain at least two divided logs;
获取所述分割日志中包含的关键字，并基于所述关键字赋予所述分割日志不同的优先级；Acquiring keywords contained in the segmented logs, and assigning different priorities to the segmented logs based on the keywords;
基于所述缓存队列和所述优先级，依次对所述分割日志进行归档。The split logs are sequentially archived based on the cache queue and the priority.
其中，上述日志管理装置中各个模块和单元与上述日志管理方法实施例中各步骤相对应，其功能和实现过程在此处不再一一赘述。Wherein, each module and unit in the above log management device corresponds to each step in the above log management method embodiment, and the functions and implementation process thereof will not be repeated here.
此外，本发明实施例还提供一种计算机可读存储介质。In addition, an embodiment of the present invention further provides a computer-readable storage medium.
本发明计算机可读存储介质上存储有日志管理程序，其中所述日志管理程序被处理器执行时，实现如上述的日志管理方法的步骤。A log management program is stored on the computer-readable storage medium of the present invention, wherein when the log management program is executed by the processor, the steps of the log management method described above are implemented.
其中，日志管理程序被执行时所实现的方法可参照本发明日志管理方法的各个实施例，此处不再赘述。For the method implemented when the log management program is executed, reference may be made to the various embodiments of the log management method of the present invention, which will not be repeated here.
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.
上述本发明实施例序号仅仅为了描述，不代表实施例的优劣。The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本发明各个实施例所述的方法。From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course hardware can also be used, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art. The computer software products are stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present invention.
以上仅为本发明的优选实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910539988.1ACN110377481B (en) | 2019-06-19 | 2019-06-19 | Log management method, device, equipment and storage medium |
| PCT/CN2019/122073WO2020253125A1 (en) | 2019-06-19 | 2019-11-29 | Log management method, apparatus, and device, and storage medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910539988.1ACN110377481B (en) | 2019-06-19 | 2019-06-19 | Log management method, device, equipment and storage medium |
| Publication Number | Publication Date |
|---|---|
| CN110377481Atrue CN110377481A (en) | 2019-10-25 |
| CN110377481B CN110377481B (en) | 2022-06-28 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910539988.1AExpired - Fee RelatedCN110377481B (en) | 2019-06-19 | 2019-06-19 | Log management method, device, equipment and storage medium |
| Country | Link |
|---|---|
| CN (1) | CN110377481B (en) |
| WO (1) | WO2020253125A1 (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110990335A (en)* | 2019-12-06 | 2020-04-10 | 深圳前海微众银行股份有限公司 | Log archiving method, apparatus, device, and computer-readable storage medium |
| WO2020253125A1 (en)* | 2019-06-19 | 2020-12-24 | 深圳壹账通智能科技有限公司 | Log management method, apparatus, and device, and storage medium |
| CN112463571A (en)* | 2020-12-17 | 2021-03-09 | 未来电视有限公司 | Log processing method, device and equipment |
| CN113656358A (en)* | 2020-05-12 | 2021-11-16 | 网联清算有限公司 | Method and system for processing database log files |
| WO2022237507A1 (en)* | 2021-05-12 | 2022-11-17 | 康键信息技术（深圳）有限公司 | Intelligent server fault pushing method, apparatus, and device, and storage medium |
| CN115827678A (en)* | 2023-02-15 | 2023-03-21 | 零犀(北京)科技有限公司 | Method, device, medium and electronic equipment for acquiring service data |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113391995B (en)* | 2021-04-26 | 2024-10-18 | 北京沃东天骏信息技术有限公司 | Log processing method and device, equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080320257A1 (en)* | 2007-06-20 | 2008-12-25 | Cowham Adrian W | Network message logging and archival |
| CN105577445A (en)* | 2015-12-30 | 2016-05-11 | 北京京东尚科信息技术有限公司 | Method and device for collecting and reporting logs |
| CN107092552A (en)* | 2017-03-10 | 2017-08-25 | 武汉斗鱼网络科技有限公司 | A kind of blog management method and device |
| CN107342888A (en)* | 2016-12-02 | 2017-11-10 | 杭州迪普科技股份有限公司 | The storage method and device of daily record message |
| CN107426023A (en)* | 2017-07-21 | 2017-12-01 | 携程旅游信息技术（上海）有限公司 | Cloud platform log collection and retransmission method, system, equipment and storage medium |
| CN108989471A (en)* | 2018-09-05 | 2018-12-11 | 郑州云海信息技术有限公司 | The management method and device of log in network system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8880478B2 (en)* | 2006-12-28 | 2014-11-04 | International Business Machines Corporation | Scan-free archiving |
| CN105005528B (en)* | 2015-06-26 | 2018-07-24 | 浪潮(北京)电子信息产业有限公司 | A kind of log information extracting method and device |
| CN107870842B (en)* | 2016-09-28 | 2021-05-04 | 平安科技（深圳）有限公司 | Log management method and system |
| CN107451034A (en)* | 2017-08-17 | 2017-12-08 | 浪潮软件股份有限公司 | A kind of big data cluster log management apparatus, method and system |
| CN107819616A (en)* | 2017-10-30 | 2018-03-20 | 杭州安恒信息技术有限公司 | Automatically extract the method, apparatus and system of daily record |
| CN110377481B (en)* | 2019-06-19 | 2022-06-28 | 深圳壹账通智能科技有限公司 | Log management method, device, equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080320257A1 (en)* | 2007-06-20 | 2008-12-25 | Cowham Adrian W | Network message logging and archival |
| CN105577445A (en)* | 2015-12-30 | 2016-05-11 | 北京京东尚科信息技术有限公司 | Method and device for collecting and reporting logs |
| CN107342888A (en)* | 2016-12-02 | 2017-11-10 | 杭州迪普科技股份有限公司 | The storage method and device of daily record message |
| CN107092552A (en)* | 2017-03-10 | 2017-08-25 | 武汉斗鱼网络科技有限公司 | A kind of blog management method and device |
| CN107426023A (en)* | 2017-07-21 | 2017-12-01 | 携程旅游信息技术（上海）有限公司 | Cloud platform log collection and retransmission method, system, equipment and storage medium |
| CN108989471A (en)* | 2018-09-05 | 2018-12-11 | 郑州云海信息技术有限公司 | The management method and device of log in network system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020253125A1 (en)* | 2019-06-19 | 2020-12-24 | 深圳壹账通智能科技有限公司 | Log management method, apparatus, and device, and storage medium |
| CN110990335A (en)* | 2019-12-06 | 2020-04-10 | 深圳前海微众银行股份有限公司 | Log archiving method, apparatus, device, and computer-readable storage medium |
| CN113656358A (en)* | 2020-05-12 | 2021-11-16 | 网联清算有限公司 | Method and system for processing database log files |
| CN112463571A (en)* | 2020-12-17 | 2021-03-09 | 未来电视有限公司 | Log processing method, device and equipment |
| WO2022237507A1 (en)* | 2021-05-12 | 2022-11-17 | 康键信息技术（深圳）有限公司 | Intelligent server fault pushing method, apparatus, and device, and storage medium |
| CN115827678A (en)* | 2023-02-15 | 2023-03-21 | 零犀(北京)科技有限公司 | Method, device, medium and electronic equipment for acquiring service data |
| Publication number | Publication date |
|---|---|
| CN110377481B (en) | 2022-06-28 |
| WO2020253125A1 (en) | 2020-12-24 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110377481B (en) | Log management method, device, equipment and storage medium | |
| US8181071B2 (en) | Automatically managing system downtime in a computer network | |
| US9727625B2 (en) | Parallel transaction messages for database replication | |
| US10120924B2 (en) | Quarantine and repair of replicas in a quorum-based data storage system | |
| US6418469B1 (en) | Managing conditions in a network | |
| US20020188706A1 (en) | Secure computer support system | |
| US11329869B2 (en) | Self-monitoring | |
| US10033796B2 (en) | SAAS network-based backup system | |
| EP3407240A1 (en) | Data protection method and associated apparatus | |
| CN113791943A (en) | Website real-time monitoring method, system, equipment and storage medium | |
| WO2019019457A1 (en) | Control center device, business system processing method and system, and storage medium | |
| EP2674868A1 (en) | Database update notification method | |
| CN112800410A (en) | Multi-product login management method, device, equipment and storage medium | |
| JP2003233512A (en) | Client monitoring system with maintenance function, monitoring server, program, and client monitoring/ maintaining method | |
| US8380729B2 (en) | Systems and methods for first data capture through generic message monitoring | |
| US11550692B2 (en) | Integrated event processing and policy enforcement | |
| CN104462106A (en) | Data updating method and system | |
| JP4532946B2 (en) | Application replacement method and program | |
| WO2001035599A2 (en) | Secure communication system | |
| US9009546B2 (en) | Heuristic failure prevention in software as a service (SAAS) systems | |
| JP6568232B2 (en) | Computer system and device management method | |
| CN110727898B (en) | OTA website event assisted processing method, system, equipment and storage medium | |
| CN117938956B (en) | Optimization method, device, equipment and storage medium for cloud computing data caching strategy | |
| US11822438B1 (en) | Multi-computer system for application recovery following application programming interface failure | |
| CN117078211A (en) | Data processing method, device and server for backup file |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20220628 |