Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The embodiment of the application relates to a log (or called a log file), wherein the log is a text object formed by unordered key value pairs, for example, the log can comprise, but is not limited to, a JSON object, any data row in any data table in a database (such as a relational database), data generated by recording related operations (such as access operations, deletion operations, browsing operations and the like) generated during the running process of a platform (or an operating system, an application program, a network device and the like), and the like. Each key-value pair in the log is also referred to as a tag, where the key is referred to as a tag name and the value is referred to as a tag value, and the representation of the tag value may include, but is not limited to, text, numbers, objects, arrays (e.g., arrays including text, numbers, or objects). It should be noted that the log does not necessarily show the label of each label, i.e. the labels of the labels contained in the log may be identical, and the label name may be displayed only at one location of the log file, for example, the log records the related operation of any data table in the relational database, where the label of each label may be implicitly specified by the header of any data table.
The above-mentioned platforms may include, but are not limited to, application platforms, network platforms, machine learning platforms, etc., where the machine learning platform is also referred to as an intelligent machine learning platform (or intelligent titanium platform), a learning platform that provides full-flow training capabilities for machine learning (MACHINE LEARNING, ML). Machine learning is a multi-field interdisciplinary, and relates to multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like, and a computer is specially researched how to simulate or realize the learning behavior of human beings so as to acquire new knowledge or skills, and the existing knowledge structure is reorganized to continuously improve the performance of the machine learning. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, confidence networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.
The platform often comprises different services (or called service systems), each service can correspond to one or more log sets, any log set comprises logs related to the service, for example, when a target service (such as any service) is accessed to 5 data sources, if a registrar wants to expose 3 data sources in the 5 data sources, 3 log sets can be created, that is, the target service corresponds to 3 log sets, and the number of the log sets corresponding to the service is not limited in the embodiment of the present application. For example, training management services (or referred to as training services) that include network models (e.g., neural network models), management services for model reasoning (or referred to as reasoning services), rights management services, platform management services, and so forth, in the machine learning platform. The log set corresponding to the training service may include a log for recording error information in the training process of the network model. The log set corresponding to the reasoning service can comprise a log for recording interface call information in the reasoning process of the network model. The log set corresponding to the platform management service may include a log recording operations (e.g., creation operations, deletion operations, change operations, etc. on the items) of the user.
Currently, a query service developed by a developer for each service separately may be used to query logs under different services. For example, if the platform contains 3 services, then 3 independent query services need to be developed for the 3 services, and when a query party wants to query the log under a target service (such as any service), the query service developed for the target service can be invoked for query. It is not difficult to find that, because the query authority requirements (such as query conditions, authentication requirements and the like) and the processing logic (such as post-processing requirements and the like) of each service on the log are different, and the logic difference is large, the query services (or query modules) developed for each service are difficult to multiplex, and the query logic provided by the query services developed for each service is fixed, so that the flexibility of log query is lower, and the efficiency of log query is reduced. Based on the above, the embodiment of the application provides a log processing scheme, which is used for realizing that a set of total log query service is provided for the log query under different services by decoupling the log query from the two parts of setting the access right and the processing requirement of the log query, wherein the log query service supports the log under different services, and the registries of different services only need to specify or realize how the access right and the processing requirement of the log are acquired, so that the registries of different services can be prevented from repeatedly developing the log query service only because of different logics of the access right and the processing requirement, and the efficiency of the log query can be improved by adopting the set of total log query service to query the log under different services.
The log query flow of the log processing scheme provided by the embodiment of the application generally comprises the steps of receiving a log query request sent by a query party, wherein the log query request comprises a log set identifier of a target log set requesting query, the target log set corresponds to a service system (or service), searching registration information of the target log set according to the log set identifier, wherein the registration information is obtained by registering a registration party of the target log set, obtaining a target query condition according to the log query request and the registration information of the target log set, obtaining at least one candidate log by adopting target query condition query, determining a registration processing requirement based on the log query request and the registration processing information of the target log set, and processing at least one candidate log by adopting the registration processing requirement, so as to obtain a log query result. In the scheme, the target query condition for querying the candidate logs is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the flexibility of managing the access right and the processing requirement in the log query process is improved, the requirements of dynamically screening and processing the candidate logs according to different query requests can be met, and the query efficiency of the logs is improved.
In order to better understand the log processing scheme provided by the embodiment of the present application, the log query scenario is described in detail below in conjunction with the log processing system shown in fig. 1, where, as shown in fig. 1, the log processing system may include a query terminal 101, a computer device 102, and a data source 103, and the embodiment of the present application does not limit the names and numbers of the devices (such as the query terminal 101, the computer device 102, and the data source device 103) included in the log processing system. The devices included in the log processing system may be directly or indirectly connected in a communication manner through a limited or wireless manner, and the embodiment of the present application is not limited to a communication manner between devices, for example, a communication manner between devices may include, but is not limited to, a manner of using HTTP requests, remote procedure calls (Remote Procedure Call), sockets, and sharing content. A brief description of the various devices contained in the log processing system follows, in which:
The query terminal 101 may be a terminal device for querying a log by a query party, where the terminal device may include, but is not limited to, smart devices such as smart phones (e.g. Android Mobile phones, iOS Mobile phones, etc.), tablet computers, portable personal computers, mobile internet devices (Mobile INTERNET DEVICES, abbreviated as MID), smart televisions, vehicle devices, head-mounted devices, and the like. Alternatively, if the log of the desired program is an application log, an application having a function of querying the application log is deployed in the query terminal 101, and when the application is opened and used, a log query request may be generated for requesting the query log.
The computer device 102 is a device for executing log query, specifically, a log query service is deployed in the computer device, and the log query service can be used to implement log query. The computer devices may include, but are not limited to, terminal devices such as personal computers, smart phones, tablet computers, laptop computers, desktop computers, edge computing devices, embedded devices, or service devices such as data processing servers, web servers, application servers, and the like. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing service, where the server may also be a node server on a blockchain.
Data sources 103 are devices for storing logs and data sources 103 may include, but are not limited to, database management systems (e.g., relational database management systems, non-relational database management systems), full text search engines, text files, cloud storage, cloud databases, blockchain-based data storage, and the like. The cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and the distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that provides data storage and service access functions for the outside through aggregation of a large number of storage devices (storage devices are also referred to as storage nodes) of different types in a network through application software or application interfaces by means of functions such as cluster application, grid technology, and distributed storage file systems. The cloud Database (Database), which can be considered as an electronic filing cabinet, namely a place for storing electronic files, can be used for performing operations such as adding, inquiring, updating, deleting and the like on the data in the files. A "database" is a collection of data stored together in a manner that can be shared with multiple users, with as little redundancy as possible, independent of the application.
Taking the example of the log processing scheme performed by the query terminal 101, the computer device 102 and the data source 103 together, a brief description will be given of a log query flow, where, first, a query party having a log query requirement may send a log query request from the query device 101 to the computer device 102 through a network connection. And after the computer device 102 acquires the log query request, the computer device 102 can search the registration information of the target log set from the log set storage space based on the log query request, wherein the registration information is obtained by registering a registrant of the target log set, and comprises access right information and registration processing information, and the computer device 102 determines target query conditions based on the access right information of the target log set and the log query request and sends the target query conditions to the data source 103 so as to pull at least one candidate log matched with the target log query conditions from the data source 103. Next, the computer device 103 determines a registration processing requirement based on the registration processing information and the log query request, and then processes at least one candidate log according to the registration processing requirement to obtain a log query result. Of course, if the log query request includes a candidate processing requirement, the computer device may further process the log query result according to the candidate processing requirement, to obtain a log query response result. Finally, the computer device 102 returns the log query result (or log query response result) to the query device 101. Of course, if the log query request is initiated locally by the computer device 102, the computer device 102 displays the log query results (or log query response results) in a display screen.
It should be noted that, the log set storage space shown in ① in fig. 1 is located in the computer device 102, that is, the registration information of the log set (e.g., the target log set) is stored in the computer device 102, but it should be understood that the registration information of the log set may also be stored in another storage space independent of the computer device 102, such as, the registration information of the log set is stored in a cloud database, a blockchain-based database, a database, or the like. ② The number of data sources 103 shown in fig. 1 is 1, that is, log sets under different services are all stored in the data sources 103, but it is to be understood that the number of data sources 103 may be multiple, each data source 103 may be used to store log sets under one or more services, and the storage manner of the log sets is not limited in the embodiment of the present application. ③ The computer device 102 and the query device 101 may be the same device, in which implementation, the log query request received by the computer device 102 is initiated locally, and the computer device 102 and the data source 103 may be the same device, in which implementation, the computer device 102 may query at least one candidate log by means of local access. ④ The computer device 102 may also receive registration, modification, deletion, etc. of the log set from local or through network access to enable registration and update of registration information for the log set.
Based on the above-described log processing scheme, the embodiment of the present application proposes a more detailed log processing method, and the log processing method proposed by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.
Fig. 2 shows a flow chart of a log processing method according to an exemplary embodiment of the present application, which may be performed by the above-mentioned computer device, and may include, but is not limited to, steps S201-S204:
S201, receiving a log query request.
As described above, embodiments of the present application support providing a set of overall log query services to query logs under different services, and then receiving log query requests may include invoking a log query service interface to receive log query requests. The log query service interface is an interface (such as HTTP interface, remote system call interface, message queue, shared memory, cloud function, etc.) provided by the log query service and adapted to a plurality of service systems (or services), in other words, the log query service interface of the log query service may be used to receive a query request for a log set corresponding to any service system included in the platform. Compared with developing a log query service for each service system, the embodiment of the application supports providing a unified log query service interface for front-end pages, application programs and the like with the log query function, thereby avoiding redundant interface adaptation work and reducing the workload of interface adaptation.
The log query request may include a log set identifier of a target log set for requesting the query, where the log set identifier of the target log set may be used to uniquely identify the target log set, and the log set identifier may include, but is not limited to, a name of a service (or service system) corresponding to the target log set, so that the target log set may be found according to the log set identifier. Of course, the log query request may include ① request meta information besides the log set identifier of the target log set, for example, the request meta information may include type information of the application program, an authentication key, identity information (such as identity level information) of the querying party, authority information, and an IP address. For example, the target application (such as IM (INSTANT MESSAGING, instant messaging) application, content interaction application, etc.) has the function of log query, and then the query party can open and use the target application to initiate the log query request, so that the type information of the target application can include a target application of web page version, a target application of mobile phone version, etc. according to the different running environments of the running target application. The types of applications are distinguished because the content permissions that can be queried are not the same when the same application belongs to different categories, e.g., a target application of a mobile phone version may support returning logs recorded for social dynamic information (e.g., circles of friends), but a target application of a web page version may not support returning logs recorded for social dynamic information. The mode of distinguishing the application program types can improve the security of log inquiry, ensure that the log is not leaked and improve the storage security of the log.
② The candidate query condition is a filtering condition for filtering the logs in the target log set. For example, the candidate query condition is that the last 30 minutes indicates that the querying party wants to query the log within a 30 minute period from the time of initiating the log query request. ③ Query processing requirements, a processing requirement for processing the queried log (i.e., the candidate log mentioned above) or information of the log. For example, the query processing requires that the tag value of the tag signature "detailed XX" in the log is kept at most 20 characters, and after the log containing the tag name "tag XX" is queried, the log needs to be processed, so that the characters contained in the tag value corresponding to the tag name of the log are deleted to at most 20 characters. ④ The method comprises the steps of obtaining a log from a query log, obtaining an aggregation condition for aggregating the queried log, for example, calculating an average value of label values under a certain label in the queried log, and if the queried log comprises a label zone, aggregating the queried log can be counting the number of logs with label values of X zone in the label zone, and the like. A schematic of the content contained in an exemplary log query request can be seen in fig. 3.
S202, searching registration information of the target log set based on the log set identification of the target log set.
The registration information of the target log set includes access authority information and registration processing information of the target log set, and of course, the registration information of the target log set may also include meta information of the target log set (such as log set identifier of the target log set, etc.) and access information of the data source (such as data source address information, secret key, etc.) that stores the log of the target log set. An exemplary schematic diagram of registration information of a target log set may be referred to in fig. 4, and as shown in fig. 4, the registration information of the target log set includes access right information 401, registration processing information 402, meta information 403 of the target log set, and access information 404 of a data source.
Registration information of the target log set may be stored in a log set storage space, which may be a local storage space of the computer device, or may be a storage space for storing registration information of the log set independent of the computer device. The log set storage space stores registration information of a plurality of log sets each for recording a log generated in one service system. In other words, the log set storage space stores the registration information of the log set corresponding to all or part of the service systems under the platform, so that the registration information of the log set corresponding to any service system under the platform can be pulled from the log set storage space through the unified log query service interface.
S203, determining target query conditions according to the log query request and access right information of the target log set, and acquiring at least one candidate log from the target log set according to the target query conditions.
The access right information of the target log set is information for determining a filtering condition (or referred to as a query condition) for filtering the target log set to obtain candidate logs. The access right information comprises at least one access right, authorized user information corresponding to each access right information and query condition information under each access right. One exemplary access right information may include 1 that the log query request includes a valid authentication key issued by a target service (e.g., any service), then all logs in the target log set may be queried, and 2 that an administrator of item X may view all logs with a tag name of "tag name X1" and a tag value of "tag value X2". Then the access authority information of the target log set is determined to comprise 2 access authorities, wherein the access authority 1 is '1, the log inquiry request comprises a valid authentication key issued by a target service (such as any service), all logs in the target log set can be inquired', the authorized user information corresponding to the access authority 1 is 'an inquirer with a valid authentication key issued by the target service (such as any service)' and the inquiry condition information under the access authority 1 is 'all logs in the target log set'. The administrator with the access authority 2 being 2 can check all the logs with the tag names of ' tag name X1 ' and the tag value of ' tag value X2 ', the authorized user information corresponding to the access authority 2 is ' administrator with the item X ', and the query condition information under the access authority 2 is ' check all the logs with the tag names of ' tag name X1 ' and the tag value of ' tag value X2 '.
The candidate query condition is a filtering condition set by the querying party for filtering the logs in the target log set to query the candidate logs that the querying party wants to query. The implementation process of determining the target query condition according to the log query request and the access right information of the target log set is described briefly below with reference to fig. 5, wherein ① determines the target access right of the querying party according to the identity information of the querying party and the authorized user information corresponding to each access right. The target log query request includes the identity information of the querying party, which may include, but is not limited to, a user nickname, user class information, an authentication key, etc., based on step S201. Continuing to refer to the above example, assuming that the identity information in the log query request sent by the querying party indicates that the querying party is an administrator of the item X, based on the identity information of the querying party and the 2 access rights corresponding to the target log set, it is known that the target access rights possessed by the querying party are access rights 2. ② And generating registration query conditions based on the query condition information under the target access rights. The registration query condition is a filtering condition for filtering the logs in the target log set to obtain at least one candidate log, and the mode of generating the registration query condition based on the query condition information is different according to the content contained in the query condition information. For example, the query condition information comprises the rule shown in the example, the rule is directly determined to be the registration query condition, and further, if the query condition information comprises a code segment, the result obtained by inputting the log query request to the code segment can be used as the registration query condition, the mode of defining the registration query condition by the code segment can simplify the operation of defining the complex registration query condition, and if the query condition information also comprises an interface, the result obtained by calling the interface to respond to the log query request can be used as the registration query condition, so that the log query request is sent to a registrar of a target log set through the interface, the registration query condition is determined by the registrar, and the registrar is given customized and complicated rights for setting the query condition to a certain extent. ③ The registration query condition and the candidate query condition are determined as target query conditions, wherein the target log query request contains the candidate query log based on the step S201. In other words, the target query condition for filtering the logs in the target log set may be a log query request transmitted from the inquirer and registration information of the target log set registered by the registrar, which implements a process that can dynamically control filtering of the logs according to identity information of the inquirer to some extent.
The process of generating the registration query condition in step ② is different according to the content included in the query condition information, and the following description describes a specific implementation manner of generating the registration query condition based on the query condition information by taking the query condition information including a rule (i.e., a query rule), a code segment (i.e., a first code segment), and an interface (i.e., a condition management interface) as examples, respectively, where:
(1) The query condition information under the target access right defines a query rule. In such an implementation, generating the registration query condition based on the query condition information may include determining a query rule in the query condition information as the registration query condition. In other words, if the query condition information includes a series of query rules defined by a registrar of the target log set, the query rules defined by the registrar may be determined as the registration query condition. For example, it is assumed that the access right information of the target log set includes access right 1 and access right 2, the authorized user information corresponding to the access right 1 is "administrator of platform", the query condition information under the access right 1 is "all logs in the queriable target log set", the authorized user information corresponding to the access right 2 is "normal user", and the query condition information under the access right 2 is "log with queriable label name" XX "and label value" XX ". If the identity information contained in the log query request is "normal user", the identity information "normal user" of the querying party is the same as the authorized user information "normal user" of the access right 2, and it is determined that the log query request is matched with the access right 2, that is, the querying party can query the log indicated by the query condition information under the access right 2, that is, the log with the registered query condition being "queriable tag name" XX "and the tag value being" XX ".
(2) The query condition information under the target access right includes a first code fragment, and then the first code fragment may define a query function for the query log. In such an implementation, the implementation of generating the registration query condition based on the query condition information may include determining the target query request as input information for the query function, and invoking the first code segment to run the query function to process the log query request to obtain the registration query condition. It is easy to understand that the code segments have the advantages of easy understanding, simple logic writing and the like, so that the embodiment of the application supports the adoption of the code segments to define the query function, and can improve the flexibility of defining the query function.
(3) The query condition information under the target access authority comprises a condition management interface, in particular a condition management interface provided by a registry of the target log set. In such an implementation, the implementation of generating the registration query condition based on the query condition information may include invoking a condition management interface to send a log query request to a registrar of the target log set, and receiving the registration query condition returned by the registrar through the condition management interface. The registration inquiry conditions are received through the condition management interface provided by the registration party, so that the complicated and customized authority control requirements of the registration party on the registration inquiry conditions can be met, and the acquisition mode of the registration inquiry conditions is enriched.
Based on the above operation, a registration query condition for querying the candidate logs is obtained, and then, by using the registration query condition and the candidate query condition included in the log query request, a query operation is performed in the target log set, so that at least one candidate log can be obtained. Referring to fig. 6, the process of querying at least one candidate log based on the target query condition is described, as shown in fig. 6, the log query request includes the identity information of the query party, "common member of item X", the user name "Zhang san", the meta information "no key", the candidate query condition "latest 30 minutes", the registration information of the target log set includes the authentication key in the validity period included in the access authority 1 "log query request, all logs in the target log set can be viewed, the administrator of the access authority 2" item X can view the label name "item number", the log with the label value "X", the other members of the access authority 3 "item X can view the label name" item number ", the label value" X "and the label name" creator ", the registration information of the user name" of the label value "member" is known, the registration information of the query party and the target log set included in the log query request is based on the identity information of the query party and the registration information of the target log set, the obtained registration query condition is the label name "item number" label value "X" and the label value "label name" latest 30 ", the label name of the member of the candidate log set is further created based on the label value" latest 30 ", and the label name" label "of the candidate log set" is further included in the label value "label name" latest 30 ".
S204, determining a registration processing requirement according to the log query request and registration processing information of the target log set, and processing at least one candidate log according to the registration processing requirement to obtain a log query result.
The registration processing information is information for determining processing of a query result obtained based on a target query condition query. The registration processing information includes at least one kind of registration processing authority, authorized user information corresponding to each kind of registration processing authority, and processing information under each kind of registration processing authority. An exemplary registration process information may include 1, removing the key (i.e., tag value) of tag "XX" when the log query request is for external network access, and 2, not doing any processing when the log query request is for local area network access. The registration processing information can be determined to include two registration processing authorities, wherein when the registration processing authority 1 is "1" and the log query request is external network access, the key value (i.e. the tag value) of the tag label "XX" is removed, the authorized user information corresponding to the registration processing authority 1 includes that the log query request is external network access, and the processing information under the registration processing authority 1 is that the key value of the tag label "XX" is removed. The registration processing authority 2 is ' 2', and no processing is performed when the log inquiry request is local area network access ', wherein the authorized user information corresponding to the registration processing authority 2 is that the log inquiry request is local area network access, and the processing information under the registration processing authority 2 is that no processing is performed.
The implementation manner of determining the registration processing requirement according to the log query request and the registration processing information of the target log set is described briefly below with reference to fig. 7, in which first, the target registration processing authority of the query party is determined according to the identity information of the query party and the authorized user information corresponding to each registration processing authority, and then the registration processing requirement is generated based on the processing information under the target registration processing authority. Similar to the access right information, the implementation of determining the registration processing requirement is different depending on the content contained in the processing information under the target registration processing right. The following describes an implementation manner of generating a registration processing requirement based on the processing information under the target registration processing authority, taking a rule (i.e., a registration processing rule), a code segment (i.e., a second code segment), and an interface (i.e., a registration processing interface) as an example, where:
(1) The processing information under the target registration processing authority defines the registration processing rule. In such an implementation, an implementation of generating a registration processing requirement based on the processing information under the target registration processing authority may include determining a registration processing requirement from a registration processing rule defined by the processing information. In other words, if the processing information under the target registration processing authority includes a series of registration processing rules defined by the registrar of the target log set, the registration processing rules defined by the registrar may be determined as registration processing requirements.
(2) The processing information under the target registration processing authority comprises a second code segment which defines a registration processing function, namely, the registration processing requirement is acquired by means of the code segment. In such an implementation, generating the registration processing requirement based on the processing information under the target registration processing authority may include determining the log query request as input information of a registration processing function, calling a second code segment to run the registration processing function, and processing the log query request to obtain the registration processing requirement. It should be noted that, the input information of the registration processing function may include, in addition to the log query request, the target information of at least one candidate log, so that the registration processing function may determine the registration processing requirement based on the target information of at least one candidate log and the log query request as the input information. The target information of the at least one candidate log may include, but is not limited to, a total number of candidate logs, a duration of querying the candidate logs, a number of candidate logs failing to return, etc., and the target information of the at least one candidate log is obtained during querying of the at least one candidate log from the data source based on a target query condition.
(3) The processing information under the target registration processing authority includes a registration processing interface provided by a registrar of the target log set. In such an implementation, generating registration processing requirements based on processing information under the target registration processing permissions may include invoking a registration processing interface to send a log query request to a registrar of the target log set and receiving a registration requirement returned by the registrar. It should be noted that, in addition to calling the registration processing interface to send the log query request to the registrar of the target log set, the target information of at least one candidate log may be sent as a call parameter to the registrar of the target log set, so that the registrar determines the registration processing requirement based on the target information of at least one candidate log and the log query request together. The registration processing requirements provided by the registrar are acquired in an interface mode, so that a log query service for log query does not need to care about the specific implementation mode and the running condition of the interface, decoupling of log query and registration processing requirement setting is realized, and flexibility of setting the registration processing requirements is improved.
Based on the above operation, a registration processing requirement is obtained, and then the registration processing requirement is adopted to process at least one candidate log, so that a log query result can be obtained, wherein the log query result comprises at least one log after the at least one candidate log is processed. An exemplary description of a process for processing at least one candidate log according to registration processing requirements is described below with reference to fig. 8, where, as shown in fig. 8, the log query request is assumed to include "the querying party accesses the computer device (or log query service) through the external network" as to the identity information of the querying party. The registration information of the target log set comprises that the registration processing authority 1 is not processed when the local area network is accessed, and the registration processing authority 2 is removed from the label of the label "IP address" and the sensitive information in the log is hidden when the external network is accessed. Based on the log query request and the registration information of the target log set, the identity information of the query party, namely 'the query party accesses the computer equipment (or log query service) through the external network', is the same as the authorized user information corresponding to the registration processing authority 2, the target registration processing authority is determined to be the registration processing authority 2, and the registration processing requirement comprises the steps of removing the label of the IP address and hiding sensitive information in the log. Assuming that the target log (e.g., any one of at least one candidate log) obtained by filtering based on the target query condition includes a tag with a tag name of "tag 1", a tag with a tag name of "tag 2" and a tag with a tag name of "IP address", then the updated target log may be obtained after the tag with the tag name of "IP address" in the target log is removed according to the obtained registration processing requirement, where the updated target log includes the tag with the tag name of "tag 1" and the tag with the tag name of "tag 2". Secondly, processing sensitive information in the hidden target log is carried out on the updated target log, and a log query result is obtained.
It is to be appreciated that the sensitive information in the log may be dynamically changing, e.g., the type and number of domestic sensitive words may be different for both the current time period (e.g., the time period during which the log query is currently being performed) and the historical time period (e.g., the time period prior to the current time). In order to more accurately detect sensitive information in a log, the embodiment of the application supports the registration processing requirement (such as HTTP interface, remote system call interface, message queue, shared memory, cloud function, etc.) acquired based on the log query request and the registration processing information of the target log set. In such an implementation, processing of any log may include making an interface call with the log as a parameter, returning the result of the interface as a processed result (or log query result). The method for hiding the sensitive information by calling the service specially processing the sensitive information through the interface ensures that a log query service or a registry of a target log set does not need to care about the implementation process of hiding the sensitive information, and the decoupling of 'hiding the sensitive information', 'maintaining the target log set' and 'querying the target log set' is realized. Of course, in addition to the requirement of the registration processing of the interface type, there are other cases where the requirement of the registration processing of the interface type is required, and the removal of the sensitive information in the log is described by way of example and not limitation of the embodiment of the present application.
According to the embodiment of the application, the target query condition can be determined based on the log query request sent by the query party and the registration information of the target log set registered by the registration party, at least one candidate log is queried by adopting the log query request, the registration processing requirement is determined based on the log query request and the registration processing information, and at least one candidate log is processed according to the registration processing requirement, so that a log query result is obtained. In the scheme, the target query condition for querying the candidate logs is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the flexibility of managing the access right and the processing requirement in the log query process is improved, the requirements of dynamically screening and processing the candidate logs according to different query requests can be met, and the query efficiency of the logs is improved.
Referring to fig. 9, fig. 9 is a flow chart illustrating a log processing method according to an exemplary embodiment of the present application, where the log processing method may be performed by the above-mentioned computer device, and the log processing method may include, but is not limited to, steps S901-S910:
s901, a registration request for a target log set is received.
S902, generating registration information of a target log set in response to the registration request.
In steps S901 to S902, the registration request is initiated by the registration party registering the target log set, so as to implement the customization of the registration information of the target log set, and improve the flexibility of the management of the access right information and the registration processing information of the target log set. When registering the registration information of the target log set, the registration party mainly comprises the registration of the access right information and the registration processing information of the target log set. Taking registration processing information of a target log set by a registrar as an example for description, assuming that the registrar wants that only an administrator can access all logs of the target log set, no processing operation exists when the registrar accesses in a local area network, and when external network accesses, a label named as XXX in the log is removed first, and all sensitive information in the log needs to be hidden. The registration party can set registration processing information when registering the target log set, wherein the registration processing information comprises registration processing information P1 and registration processing information P2 if the source of the log query request is a local area network, and otherwise, the registration processing information comprises registration processing information P1 and registration processing information P2, wherein the registration processing information P1 is a registration processing rule, a label with a label name of XXX in the log is removed, and the registration processing information P2 is a registration processing interface (such as HTTP calling a given address). Wherein the registration processing interface is provided by a registrar and functions to hide sensitive information.
The embodiment of the application supports the definition of the access right information of the target log set by adopting a log Boolean function, particularly the definition of the query condition information under each access right by adopting the log Boolean function, and the definition of the registration processing information of the target log set by adopting a log processing function, particularly the definition of the processing information under each registration processing right by adopting the log processing function. For convenience of subsequent explanation, a general relational expression between the log and the label included in the log is given below, where in the embodiment of the present application, a log is represented by a character R, and the label value of the label with the label name of "R" is equal to the character V "R [ K ] =v". Similarly, the number of the devices to be used in the system, the embodiment of the application supports the adoption of symbols of 'greater than, less than, not equal to, greater than or equal to, less than or equal to, and less than or equal to' respectively to represent the label value. Similarly, the embodiment of the application records that the label value of the label with the label name of the R is the character K and satisfies the condition C, the label value of the label with the label name of the R is the R [ K ] and satisfies the condition C, the label with the label name of the R is the R [ K ] and exists/does not exist, the label value of the label with the label name of the R is the R [ K ] and conforms to a regular expression, and the label value of the label with the label name of the R is the R [ K ] and conforms to a regular expression. The following describes the concepts and terms related to the log boolean function and log processing function, wherein:
(1) A log boolean function may refer to a function that inputs a log R and outputs a boolean value (i.e., true)/false). For example, the method is described as that the input is recorded as R, the output R [ K ] meets the condition C and is a log Boolean function, so that the log can be filtered by adopting the log Boolean function to obtain candidate logs meeting the query condition, that is, the log Boolean function is equivalent to the query condition (such as the registered query condition and the candidate query condition) and the log Boolean function and the query condition are not distinguished, and the description is given herein. If the output value of any one of the Boolean functions is affected by at most one of the tag values in its inputs, then that one of the Boolean functions may be referred to as an atomic Boolean function, e.g. "input is R, output R [ project number ] >1" is an atomic Boolean function (temporary Fa), "input is R, output R has a tag number greater than 10" and an atomic Boolean function (temporary Fb). An atomic log boolean function may be a triplet of tag names or attribute names (e.g., "item numbers", "tag numbers"), relationships (e.g., "greater than", "containing") and values (e.g., "2", "error", "null"). For example, an atomic log boolean function, with an "input noted R, an output R [ ID ] present, may be described by a triplet (ID, null).
The plurality of atomic log Boolean functions adopt a Boolean function formed by operators as one log Boolean function, wherein the operators can comprise but are not limited to logical operators (or 'V', AND 'V', NOT)) Operators such as "at least/at most", and "Fa∨Fb" is a log Boolean function. The data structure of the log boolean function may be a tree structure (e.g., a tree), the leaf nodes are atomic log boolean functions, and the non-leaf nodes are operators, e.g., the log boolean function is "Fa∨Fb", then the root node in the tree is "or operator", and the two leaf nodes are (item number, >, 1) and (tag number, >, 10), respectively. When a log R is given as the input of the log Boolean function, the tree defining the log Boolean function can be traversed in sequence, each non-leaf node is judged according to the Boolean value of its child node, the leaf node is judged according to the triplet defining the atomic log Boolean function and the log R, and finally the result of the log Boolean function is output according to the Boolean value of the root node.
(2) The log processing function may be a function of inputting one log R and outputting 0, 1 log or a plurality of logs. For example, describing that "input is denoted as R, if the length of the R detailed information exceeds 20, the first 20 characters are reserved for the R detailed information and output" is a log processing function, so that the log processing function may be used to process at least one candidate log queried, that is, the log processing function corresponds to a post-processing requirement (such as the registration processing requirement described above), and the log processing function and the registration processing requirement (or referred to as the post-processing requirement) are not distinguished in the embodiment of the present application, which is described herein.
It should be noted that, the embodiment of the present application supports the registrar to define the acquisition mode of acquiring the log boolean function or the log processing function by using one or more of the following three modes, including but not limited to ① defining the rule of acquiring the log boolean function or the log processing function, specifically, the registrar acquires the log boolean function or the log processing function by customizing some rules. ② The log Boolean function or the log processing function is obtained by defining a code segment for obtaining the log Boolean function or the log processing function, specifically, a registry can obtain the log Boolean function or the log processing function by defining a function (such as a query function) through the code segment, for example, the log query request is used as input information of the query function, and then the output result of the function can be used as the log Boolean function. ③ The method comprises the steps of obtaining a log Boolean function or a log processing function by defining an interface for obtaining the log Boolean function or the log processing function, specifically, providing an interface by a registry to obtain the log Boolean function or the log processing function, for example, providing a condition management interface by the registry, and taking a result returned by the condition management interface as the log Boolean function when a log query request is taken as a call parameter.
Based on the three ways of defining the acquisition of the log boolean function or the log processing function described above, a more specific implementation of the acquisition of the log boolean function or the log processing function in three ways is given below, in which:
(1) A log boolean function is obtained. The access authority information contained in the registration information can be used for filtering the logs in the target log set to obtain the logs meeting the query conditions, and when the process of filtering the logs is described by adopting the log Boolean function, the process of filtering the logs in the target log set to obtain the query conditions can be simply and abstracted as given a log query request Q and obtaining the log Boolean function F, if one or more logs enable the log Boolean function F to return to be correct (true) for each log in the target log set, the one or more logs can be accessed, namely the one or more logs are determined to be candidate logs.
The access right information (or the query condition information under the access right included in the access right information) of the target log set may be defined by a log boolean function, where f=f1∨F2∨F3, F1 is an atomic log boolean function obtained by defining a query rule by the registrar, F2 is an atomic log boolean function obtained by calling a code fragment provided by the registrar, and F3 is an atomic log boolean function obtained by calling a condition management interface provided by the registrar. Of course, if the registrar does not define an atomic log boolean function F1, an atomic log boolean function F2, or an atomic log boolean function F3, the default defined log boolean function is an atomic log boolean function that always returns false. The following briefly describes the acquisition method of F1、F2 and F3:
The acquisition mode of F1 is that the registry defines a series of query rules-when the log query request for the target log set meets a certain target query condition, a certain log Boolean function is returned. For example, the log query request for the target log set may contain information such as the user group of the user of the log query request, the IP address of the log query request, etc., then the registrar may specify two query rules, query rule 1, a log Boolean function (denoted as FA) is returned if the user group of the log query request user is a "platform administrator," the log Boolean function FA always returns true (i.e., any log within the target log set causes the log Boolean function FA to return true, i.e., "the platform administrator may access all logs under the target log set"). Query rule 2. If the prefix of the IP address of the log query request is "192.168" and the USER name of the USER of the log query request is not null (this USER name is denoted USER), a log boolean function (denoted FB) is returned, which is described as "input denoted R, output as R creator" is USER "(i.e." the USER in the IP section can access the log where the creator is his own USER name "). If the log query request meets the two query rules, the obtained log Boolean function (or the atomic log Boolean function) is marked as FA、FB respectively, and the log query request is the log Boolean function F1=FA∨FB obtained by the query rules. Of course, the above-mentioned registration party defines two kinds of query rules, and no limitation is made to the embodiment of the present application, for example, the number of query rules may be three, four, three, etc.
F2, the registry defines a query function through a code segment (such as a first code segment), wherein the input of the query function is a log query request for a target log set, and the output of the query function is a log Boolean function. In a specific implementation, when the log query service (or the computer device) receives a log query request for a target log set, a query function defined by the code segment is called, the log query request is used as input information, and the output of the query function is used as a log boolean function F2.
And F3, calling a condition management interface provided by a registry, taking the log query request as a calling parameter, and taking a returned log Boolean function as F3. This manner of acquisition may allow the target log set to possess complex, customized access rights logic. For example, the registrar may use this way of obtaining to obtain a log boolean function in the hope of verifying his own issued authentication key. In a specific implementation, when the registry of the target log set receives a log query request, the log query request may be used to call a condition management interface provided by the registry, so that the registry verifies the authentication key, and returns a log boolean function F3 according to the authentication result.
In summary, the registry of the target log set defines the access right information of the target log set by defining the query rule for obtaining the log boolean function, obtaining the first code segment of the log boolean function, and obtaining the condition management interface of the log boolean function.
(2) A log processing function is obtained. The registration processing requirement of the target log set can be defined as a series of log processing functions P1、P2、P3、……、Pn, etc., n is a positive integer, that is, at least one candidate log obtained according to the log boolean function query is processed by P1, the obtained processing result is processed by P2, and the like, and the result obtained after the processing of Pn is used as the log query result. Of course, the registration processing requirement of the target log set may be null, i.e. at least one candidate log obtained by the query is not processed. Similar to the log boolean function, the log processing function is obtained in three ways, including:
① The log processing function is obtained by a registration processing rule defined by the registrar. The acquisition mode is mainly suitable for some simpler log processing functions, wherein the log processing functions can be defined in the following mode that input is recorded as R, if R enables a log Boolean function F to return true, operation O is carried out on the R to obtain a log query result, and if the operation O fails, P (R) is returned to serve as the log query result. The log processing function can be defined by only giving F, O, P definitions, wherein F can be obtained in the manner of F1, O is some operation on the log (for example, removing R item number), P is a fixed log processing function, for example, P is "input is R, return is null" (removing the log), R is "input is R, and R item number is set as default value 1" (default value is set).
For example, assuming that the processing logic that the registrar wishes to process the candidate log is such that when the IP from which the log query request originates is from a segment of "192.168..x.," if the candidate log contains a tag "timestamp", the tag value of the tag "timestamp" is converted to a string in RFC3339 format, and if the conversion fails, the candidate log is removed; otherwise, no processing is performed. The acquisition mode of F may be defined as a query rule mode, and if the IP from which the log query request originates is a network segment from 192.168., the log boolean function F "given the input R, returns whether R contains an R timestamp" or not, and otherwise returns the log boolean function whose output is false. O can be defined as "input is denoted as R, R [ timestamp ] is converted into a character string in RFC3339 format and output". The log processing function P may be defined as "input is noted R, return to null".
② The log processing function is obtained by a code fragment (e.g., a second code fragment) provided by the registrar. The registrar defines a registration processing function through the code segment, the input of the registration processing function is the log query request for the target log set and the target information of at least one candidate log, and the output of the registration processing function is a log processing function. In a specific implementation, when the log query service (or the computer device) receives a log query request for a target log set, a registration processing function defined by the code segment is called, the log query request is used as input information, and the output of the registration processing function is used as a log processing function.
③ The log processing function is obtained through a registration processing interface provided by the registrar. And calling a registration processing interface provided by a registration party, taking a log query request and target information of at least one candidate log as call parameters, and taking a returned call result as a log processing function. The acquisition mode can allow the target log set to have complex and customized processing logic, so that the log query request aiming at each time can be completely realized, the processing logic can be provided for each candidate log in a targeted way, and even a registrar can acquire some other information in a mode of providing a registration processing interface, such as whether each query aiming at the target log set is successful, the time-consuming condition of log query and the like.
It is particularly noted that the obtained log processing function may be an interface (such as an HTTP interface, a remote system call interface, a message queue, a shared memory, a cloud function, etc.), and in this implementation, the process of processing any log by using the log processing function in the form of the interface may include performing an interface call with the log R as a parameter, and taking a result returned by the interface as a result after processing the log R by using the log processing function. The log is processed by adopting the log processing function in the form of an interface, so that the registrar can dynamically change the processing result (such as the log query result) according to the state of the registrar, and the requirement of the registrar on customized processing logic is met.
For example, if the registrar wants to "when the log query request contains the authentication key, no post-processing is performed, otherwise, the access interface of the sensitive word replacement service is returned as the log processing function according to the source of the log query request", the registrar of the target log set defines that the acquisition mode of the log processing function is to call the registration processing interface, and the registration processing interface returns the log processing function of the registration processing interface which does not perform any processing according to whether the log query request contains the legal authentication key and the source of the log query request, respectively.
In summary, the registry of the target log set may define the log processing logic by defining the log processing functions P1、P2、P3、……、Pn and the like of the series.
In addition, the embodiment of the application supports the registrant to edit the registration information of the registered target log set. In a specific implementation, a log editing request sent by a registry of a target log set can be received, the log editing request is used for requesting to edit registration information of the target log set, an editing operation is performed on the registration information of the target log set in response to the log editing request, so as to obtain updated registration information of the target log set, wherein the editing operation can comprise one or a combination of more of a new operation (such as adding a log Boolean function or a log processing function in the registration information), a modification operation (such as modifying the log Boolean function or the log processing function in the registration information) and a deletion operation (such as deleting the log Boolean function or the log processing function in the registration information). Of course, for the computer device (or the log query service), the computer device may receive the log editing request from the local, or may receive the registration editing request sent by other devices through the network, and the manner of obtaining the registration editing request by the computer device is not limited in this embodiment of the present application, and is described herein.
S903, receiving a log query request.
S904, searching registration information of the target log set based on the log set identification of the target log set.
S905, determining target query conditions according to the target query request and the access right information of the target log set.
S906, at least one candidate log is obtained from the target log set according to the target query condition.
S907, determining registration processing requirements according to the log query request and the registration processing information of the target log set.
S908, processing at least one candidate log according to the registration processing requirement to obtain a log query result.
It should be noted that, the specific implementation manner shown in steps S903-S908 may refer to the related description of the specific implementation manner shown in steps S201-S204 in the embodiment shown in fig. 2, which is not described herein.
S909, processing the log query result based on the query processing requirement to obtain a log query response result.
S910, returning the log query response result to the querying party.
In steps S909-S910, the log query request mentioned in the embodiment of the present application may further include a query processing requirement, where the query processing requirement is a requirement defined by a querying party for processing the log query result. In this implementation, the process of determining the target query condition and the target processing requirement (e.g., including the registration processing requirement and the query processing requirement) based on the log query request and the registration information of the target log set may be referred to in fig. 10, and fig. 10 illustrates a schematic diagram for determining the target query condition and the target processing requirement according to an exemplary embodiment of the present application. As shown in FIG. 10, the target query condition can be obtained based on the access right information contained in the registration information of the target log set and the log query request, wherein the target query condition can comprise one or two of the registration query condition and the candidate query condition, so that at least one candidate log can be obtained based on the target query condition, and the target information (such as query time-consuming information, number information of the candidate logs and the like) of the at least one candidate log, and similarly, the target processing requirement can be obtained based on the registration processing information contained in the registration information of the target log set and the log query request, wherein the target processing requirement can comprise one or two of the registration processing requirement and the query processing requirement, specifically, the target query result of the queried at least one candidate log can be processed by adopting the registration processing requirement, the log query result can be processed by adopting the query processing requirement to obtain the log query response result, and the log query response result can be returned to the query party to realize the query of the log in the target log set. Wherein the log query response result includes at least one log.
In addition, the embodiment of the application also supports the aggregation processing of the log query response results and outputs the aggregation result after the aggregation processing. The aggregation process may include, but is not limited to, counting the number of logs containing a target tag (e.g., any tag) in the log contained in the log query result (or log query response result), calculating an average value of tag values of the tag name "XXXX" in the log contained in the log query response result, counting the number of logs containing the tag name "XXX" in the log contained in the log query response result, and the like. In the specific implementation, an aggregation request sent by a query party can be received, the aggregation request carries an aggregation condition, at least one log contained in a log query response result is subjected to aggregation processing by adopting the aggregation condition in response to the aggregation request, an aggregation result is obtained, and the aggregation result is returned to the query party. The method directly requests the computer equipment (or the storage engine) to aggregate the log query response results by sending the aggregation request, so that the calculation cost can be reduced. Of course, the aggregation condition for aggregating the log query response request may also be carried in the log query request, and the embodiment of the present application does not limit the manner of obtaining the aggregate request, which is described herein.
In addition, referring to the schematic diagram of the registration information of the target log set shown in fig. 3, the registration information of the target log set further carries information such as paging conditions and sorting conditions, so that after the log query response result (or log query result) is obtained, the embodiment of the application supports that one or two of the paging conditions and the sorting conditions are adopted to perform paging processing, sorting processing or paging processing and sorting processing on a plurality of logs contained in the log query response result, so that the plurality of logs output to the querying party can perform paging display or sorting display according to requirements, and the readability of the logs is improved. Taking the example of paging display of a plurality of logs (such as 200 logs) contained in the log query response result according to the paging condition, if the paging condition is that 50 logs are displayed for each page, the 200 logs can be respectively displayed on 5 pages, which improves the readability of the logs and is convenient for a query party to check and search the logs. Taking the example of sorting and displaying the plurality of logs contained in the log query response result according to the sorting condition, if the sorting condition is that the logs are sorted in the order from near to far according to the log generation time, if the generation time of the log 1 is 12:00 of 5 months and 8 days, the generation time of the log 2 is 16:00 of 5 months and 8 days, and the generation time of the log 3 is 00:00 of 5 months and 9 days, the sorting order of the log 1, the log 2 and the log 3 can be determined to be log 1- >, log 2- >, and log 3. An exemplary schematic diagram of displaying a plurality of logs in a sequence according to the log generation time can be seen in fig. 11, as shown in fig. 11, the label numbers of N logs all have the implicit direction of the header of the data table, N is a positive integer, and N logs are displayed in a sequence according to the generation time from near to far. The method for displaying the logs in the ordering way is beneficial to the query party to check and inquire the logs, and improves the query experience of the query party. The above description describes the display of the plurality of logs in two modes of paging and sorting, but in an actual application scenario, the plurality of logs may be displayed based on other display conditions, and the display mode of the plurality of logs is not limited in this embodiment of the present application.
In summary, the embodiment of the application decouples the log query from the registration information of the target log set, that is, decouples the log query from the access right information and the registration processing information, so that a set of log query services can be implemented to query logs under different service systems, and further, repeated development of the log query services is avoided. In addition, the mode of setting the access right information and the registration processing information by the registrant can improve the flexibility of the access right and the processing logic, and further, the acquisition mode of the access right information and the registration processing information can comprise code fragments and interfaces, so that the flexibility of the access right and the processing logic is improved, and the requirement of the registrant for updating the registration information at any time is met. In addition, the target query condition for querying the candidate logs is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the requirements of dynamically screening and processing the candidate logs according to different query requests can be met, the flexibility of log query is further improved, and the query efficiency of the logs is improved.
The foregoing details of the method of embodiments of the present application are provided for the purpose of better implementing the foregoing aspects of embodiments of the present application, and accordingly, the following provides an apparatus of embodiments of the present application.
Fig. 12 shows a schematic diagram of a log processing apparatus according to an exemplary embodiment of the present application, which may be used as a computer program (including program code) running in a computer device, and which may be used to perform some or all of the steps in the method embodiments shown in fig. 2 and 9. Referring to fig. 12, the log processing apparatus includes the following units:
an obtaining unit 1201, configured to receive a log query request, where the log query request includes a log set identifier of a target log set for requesting a query;
a processing unit 1202, configured to search registration information of a target log set based on a log set identifier of the target log set, where the registration information of the target log set includes access right information and registration processing information of the target log set;
The processing unit 1202 is further configured to determine a target query condition according to the log query request and access right information of the target log set, and obtain at least one candidate log from the target log set according to the target query condition;
The processing unit 1202 is further configured to determine a registration processing requirement according to the log query request and registration processing information of the target log set, and process at least one candidate log according to the registration processing requirement to obtain a log query result.
In one implementation, the log query request further includes a query processing requirement, and the processing unit 1202 is further configured to:
processing the log query result according to the query processing requirement to obtain a log query response result, and returning the log query response result to the query party.
In one implementation, the access right information includes at least one access right, authorized user information corresponding to each access right, and query condition information under each access right, where the log query request further includes identity information of a query party and candidate query conditions, and the processing unit 1202 is configured to determine, according to the log query request and the access right information of the target log set, a target query condition, where the determining is specifically configured to:
Determining target access rights of the inquirer according to the identity information of the inquirer and the authorized user information corresponding to each access right;
generating registration query conditions based on the query condition information under the target access rights;
The registration query condition and the candidate query condition are determined as target query conditions.
In one implementation, the query condition information under the target access right defines a query rule, and the processing unit 1202 is configured to, when generating the registration query condition based on the query condition information under the target access right, specifically:
The query rule in the query condition information is determined as a registered query condition.
In one implementation, the query condition information under the target access right includes a first code segment defining a query function, and the processing unit 1202 is configured to, when generating the registration query condition based on the query condition information under the target access right, specifically:
determining a log query request as input information of a query function;
And calling the first code fragment to run a query function, and processing the log query request to obtain a registration query condition.
In one implementation, the query condition information under the target access right includes a condition management interface provided by a registrar of the target log set, and the processing unit 1202 is specifically configured to, when generating the registration query condition based on the query condition information under the target access right:
Calling a condition management interface to send a log query request to a registry of a target log set;
and receiving a registration inquiry condition returned by the registrar through the condition management interface.
In one implementation, the registration processing information includes at least one registration processing permission, authorized user information corresponding to each registration processing permission, and processing information under each registration processing permission, where the log query request further includes identity information of the querying party, and the processing unit 1202 is configured to determine, according to the log query request and the registration processing information of the target log set, a registration processing requirement, where the registration processing requirement is specifically:
determining target registration processing authorities of the inquirer according to the identity information of the inquirer and authorized user information corresponding to each registration processing authority;
a registration processing requirement is generated based on the processing information under the target registration processing authority.
In one implementation, the processing unit 1202 is configured to, when generating a registration processing requirement based on the processing information under the target registration processing authority, specifically:
the registration processing rule defined by the processing information is determined as a registration processing requirement.
In one implementation, the processing information under the target registration processing authority includes a second code segment defining a registration processing function, and the processing unit 1202 is configured to, when generating the registration processing requirement based on the processing information under the target registration processing authority, specifically:
determining the log query request as input information of a registration processing function;
And calling a second code segment to run a registration processing function, and processing the log query request to obtain registration processing requirements.
In one implementation, the processing information under the target registration processing authority includes a registration processing interface provided by a registrar of the target log set, and the processing unit 1202 is configured to, when generating a registration processing requirement based on the processing information under the target registration processing authority, specifically:
Calling a registration processing interface to send a log query request to a registrar of a target log set;
And receiving a registration processing requirement returned by the registrar.
In one implementation, the processing unit 1202 is further configured to:
Receiving a log editing request sent by a registry of a target log set, wherein the log editing request is used for requesting editing of registration information of the target log set;
responding to the log editing request to execute editing operation on the registration information of the target log set to obtain updated registration information of the target log set;
The editing operation comprises one or more of an adding operation, a modifying operation and a deleting operation.
In one implementation, the registration information of the target log set is stored in a log set storage space, where registration information of a plurality of log sets are stored, each log set is used for recording a log generated in a service system, and the obtaining unit 1201 is specifically configured to, when receiving a log query request:
Calling a log query service interface to receive a log query request;
The log query service interface is adapted to a plurality of service systems, and is used for receiving a query request of a log set corresponding to any one of the service systems.
According to an embodiment of the present application, each unit in the log processing apparatus shown in fig. 12 may be separately or completely combined into one or several other units, or some unit(s) thereof may be further split into a plurality of units with smaller functions, which may achieve the same operation without affecting the implementation of the technical effects of the embodiment of the present application. The above units are divided based on logic functions, and in practical applications, the functions of one unit may be implemented by a plurality of units, or the functions of a plurality of units may be implemented by one unit. In other embodiments of the present application, the log processing device may also include other units, and in practical applications, these functions may also be implemented with assistance from other units, and may be implemented by cooperation of multiple units. According to another embodiment of the present application, a log processing apparatus as shown in fig. 12 may be constructed by running a computer program (including program code) capable of executing the steps involved in the respective methods as shown in fig. 2 and 9 on a general-purpose computing device such as a computer including a processing element such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read only storage medium (ROM), and the like, and a storage element, and the log processing method of the embodiment of the present application is implemented. The computer program may be recorded on, for example, a computer-readable recording medium, and loaded into and run in the above-described computing device through the computer-readable recording medium.
In the embodiment of the application, the processing unit 1202 can determine the target query condition based on the log query request sent by the query party and the registration information of the target log set registered by the registration party, query at least one candidate log by adopting the log query request, determine the registration processing requirement based on the log query request and the registration processing information, and process at least one candidate log according to the registration processing requirement to obtain the log query result. In the above scheme, the target query condition for querying at least one candidate log is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the flexibility of managing the access right and the processing requirement in the log query process is improved, the requirements of dynamically screening and processing the candidate log according to different query requests can be met, and the query efficiency of the log is improved.
Fig. 13 is a schematic diagram showing a structure of a log processing apparatus according to an exemplary embodiment of the present application. Referring to fig. 13, the log processing apparatus includes a processor 1201, a communication interface 1202, and a computer-readable storage medium 1203. Wherein the processor 1201, the communication interface 1202, and the computer readable storage medium 1203 may be connected by a bus or other means. Wherein the communication interface 1302 is for receiving and transmitting data. The computer readable storage medium 1303 may be stored in a memory of the log processing apparatus, the computer readable storage medium 1303 storing a computer program including program instructions, and the processor 1301 executing the program instructions stored in the computer readable storage medium 1303. Processor 1301, or CPU (Central Processing Unit )), is a computing core and control core of the log processing device, which is adapted to implement one or more instructions, in particular to load and execute one or more instructions to implement a corresponding method flow or a corresponding function.
The embodiment of the application also provides a computer readable storage medium (Memory), which is a Memory device in the log processing device and is used for storing programs and data. It will be appreciated that the computer readable storage medium herein may include both built-in storage media in the log processing device and extended storage media supported by the log processing device. The computer readable storage medium provides a storage space that stores a processing system of the log processing device. Also stored in this memory space are one or more instructions, which may be one or more computer programs (including program code), adapted to be loaded and executed by processor 1301. The computer readable storage medium may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory, or may alternatively be at least one computer readable storage medium located remotely from the processor.
In one embodiment, the log processing device may be a computer device mentioned in the foregoing embodiment, the computer readable storage medium has one or more instructions stored therein, the processor 1301 loads and executes the one or more instructions stored in the computer readable storage medium to implement the corresponding steps in the foregoing log processing method embodiment, and in a specific implementation, the one or more instructions in the computer readable storage medium are loaded by the processor 1301 and execute the following steps:
Receiving a log query request, wherein the log query request comprises a log set identifier of a target log set for requesting query;
Searching registration information of a target log set based on log set identification of the target log set, wherein the registration information of the target log set comprises access right information and registration processing information of the target log set;
Determining target query conditions according to the log query request and access right information of the target log set, and acquiring at least one candidate log from the target log set according to the target query conditions;
Determining a registration processing requirement according to the log query request and the registration processing information of the target log set, and processing at least one candidate log according to the registration processing requirement to obtain a log query result.
In one implementation, the log query request further includes query processing requirements, and one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and further perform the steps of:
processing the log query result according to the query processing requirement to obtain a log query response result;
And returning the log query response result to the querying party.
In one implementation, when the access right information includes at least one access right, authorized user information corresponding to each access right, and query condition information under each access right, the log query request further includes identity information of a query party and candidate query conditions, and one or more instructions in the computer readable storage medium are loaded by the processor 1301 and when executing the access right information according to the log query request and the target log set, the method is specifically used for executing the following steps:
Determining target access rights of the inquirer according to the identity information of the inquirer and the authorized user information corresponding to each access right;
generating registration query conditions based on the query condition information under the target access rights;
The registration query condition and the candidate query condition are determined as target query conditions.
In one implementation, the query condition information under the target access rights defines a query rule, and one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and are specifically configured to, when executed, generate a registration query condition based on the query condition information under the target access rights, perform the steps of:
The query rule in the query condition information is determined as a registered query condition.
In one implementation, the query condition information under the target access right includes a first code segment defining a query function, and one or more instructions in the computer readable storage medium are loaded by the processor 1301 and are specifically configured to perform the following steps when executing the generation of the registered query condition based on the query condition information under the target access right:
determining a log query request as input information of a query function;
And calling the first code fragment to run a query function, and processing the log query request to obtain a registration query condition.
In one implementation, the query condition information under the target access right includes a condition management interface provided by a registrar of the target log set, and one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and are specifically configured to perform the following steps when executing the generation of the registration query condition based on the query condition information under the target access right:
Calling a condition management interface to send a log query request to a registry of a target log set;
and receiving a registration inquiry condition returned by the registrar through the condition management interface.
In one implementation, the registration processing information includes at least one registration processing authority, authorized user information corresponding to each registration processing authority, and processing information under each registration processing authority, the log query request further includes identity information of the querying party, and one or more instructions in the computer readable storage medium are loaded by the processor 1301 and executed to determine a registration processing requirement according to the log query request and the registration processing information of the target log set, and are specifically configured to perform the following steps:
determining target registration processing authorities of the inquirer according to the identity information of the inquirer and authorized user information corresponding to each registration processing authority;
a registration processing requirement is generated based on the processing information under the target registration processing authority.
In one implementation, the processing information under the target registration processing authority defines a registration processing rule, and one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and are specifically configured to perform the following steps when executing a registration processing requirement generated based on the processing information under the target registration processing authority:
the registration processing rule defined by the processing information is determined as a registration processing requirement.
In one implementation, the processing information under the target registration processing authority includes a second code segment defining a registration processing function, and one or more instructions in the computer readable storage medium are loaded by the processor 1301 and are specifically configured to perform the following steps when executing the generation of the registration processing requirement based on the processing information under the target registration processing authority:
determining the log query request as input information of a registration processing function;
And calling a second code segment to run a registration processing function, and processing the log query request to obtain registration processing requirements.
In one implementation, the processing information under the target registration processing authority includes a registration processing interface provided by a registrar of the target log set, and one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and are specifically configured to perform the following steps when executing a registration processing requirement generated based on the processing information under the target registration processing authority:
Calling a registration processing interface to send a log query request to a registrar of a target log set;
And receiving a registration processing requirement returned by the registrar.
In one implementation, the loading of one or more instructions in the computer-readable storage medium by the processor 1301 further performs the steps of:
Receiving a log editing request sent by a registry of a target log set, wherein the log editing request is used for requesting editing of registration information of the target log set;
responding to the log editing request to execute editing operation on the registration information of the target log set to obtain updated registration information of the target log set;
The editing operation comprises one or more of an adding operation, a modifying operation and a deleting operation.
In one implementation, the registration information of the target log set is stored in a log set storage space, and the log set storage space stores registration information of a plurality of log sets, each log set is used for recording a log generated in a service system, and one or more instructions in the computer readable storage medium are loaded by the processor 1301 and are specifically used for executing the following steps when executing the received log query request:
Calling a log query service interface to receive a log query request;
The log query service interface is adapted to a plurality of service systems, and is used for receiving a query request of a log set corresponding to any one of the service systems.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the log processing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the log processing device performs a log processing method.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable devices. The computer instructions may be stored in or transmitted across a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). Computer readable storage media can be any available media that can be accessed by a computer or data storage devices, such as servers, data centers, etc., that contain an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (Solid STATE DISK, SSD)), or the like.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.