Database Full-automatic monitoring system based on mpm modelsTechnical field
The invention belongs to database monitoring technical field, internet industry relational database and internal storage data are mainly used inThe data base automatic monitoring of storehouse type.
Background technology
For Internet firm, it is ensured that database service is critically important, in order to be able to find service fault in time, is neededWant a very powerful database monitoring system.Many database monitoring schemes are currently, there are, these schemes generally there are as followsShortcoming：
(1) manual maintenance cost is high：Monitoring for adding or deleting some database, almost all needs behaviour by handMake, inefficiency.
(2) complex operation：Addition monitoring is, it is necessary to much operating procedures, and it is big risk of errors occur every time.
(3) feasibility is low：Deployment monitoring is, it is necessary to much workloads, and it is high to occur omission monitoring risk.
(4) inconvenient management on large：For large scale database cluster, it is necessary to high human cost, it is difficult to meetThe demand of production environment.
The content of the invention
In view of the deficiencies of the prior art, the present invention provides a kind of database Full-automatic monitoring system based on mpm models, baseIn mpm models, perfect design is compared to database monitoring, it is possible to achieve Full-automatic monitoring database.The system can be withSupport the state and performance monitoring of extensive relational database and memory database.
The purpose of the present invention is achieved through the following technical solutions：A kind of database based on mpm models is full-automaticMonitoring system, the system includes management node, Zabbix service ends, four parts of mpm nodes and database node：
(1) management node
Lost including database host management module, monitoring template management module, monitoring configuration management module and inspection monitoringThe IP of database service on stripping pattern block, all database nodes of acquisition and port, then call Zabbix server ends itselfInterface, the corresponding monitoring template of automatic registration database main frame, carry and the corresponding monitoring configuration of establishment；In addition, it is optional, it is fixedPhase scans all database nodes and database host, checks the database node for lacking monitoring.
Agent management ends are run on the node, for sending instructions to agent clients, institute on database node are obtainedSome database service information.The instruction is to look on database node the presence of which database service process.
(2) Zabbix service ends
According to different types of database, data storage storehouse main frame is formulated corresponding database monitoring template, disposably ledEnter to Zabbix service ends, so that management node is called.
Database host represents some specific database service, and form is as follows：
<type>_<port>_<ip>
Type is the mark of type of database, and port is database service port, and IP is database where database serviceThe IP of node.
(3) mpm nodes
Agent clients are run on the node, the instruction of agent management ends is received and perform.
Including monitoring data collection module, scheduling acquisition module and monitoring data sending module, automatically generate monitoring configuration,Acquisition monitoring data and transmission monitoring data to Zabbix service ends.Wherein, it is not necessary to log on database node, remotely accessDatabase node, acquisition monitoring data, very convenient monitoring data storehouse.
(4) database node
Run on the node on database service, including various types of database services, same database node,All database port is unique, IP and port as a database service unique mark.
Agent clients are run on each database node, the instruction for receiving and performing the transmission of agent management ends.
Zabbix service ends store all database hosts and related monitoring Template Information, meanwhile, receive and storeMonitoring data, according to monitoring template, analyzes these monitoring datas, then produces alarm or monitoring tendency chart.
Monitoring template is the configuration file for defining collection item, monitored item and tendency chart, and different types of database monitors mouldPlate is different, and database host needs the corresponding monitoring template of carry.
It is the corresponding information of Value Data storehouse main frame to monitor configuration, generally comprises database host, and its IP and port, dataStorehouse type and monitoring template type, database user name password.
Agent management ends refer to the service for managing agent clients, can distribute instruction and arrive agent client executings, fromAnd the purpose of remote management database is completed, agent management ends are deployed in management node.
Agent clients refer to operate in the service on database node, and its function is reception and performs agent management endsInstruction, and return result to agent management ends.All database nodes are required for disposing agent clients.
Further, the submodule that each module is specifically included is as follows：
(1) database host management module
The agent management ends distribution instruction of management node, which is arrived on all agent clients, scan database node, ownsDatabase service, then calls Zabbix service end its own interfaces, if there is no corresponding database host, automatically creates pairThe database host answered, while calling monitoring template management module and monitoring configuration management module, automatic carry is monitored accordinglyTemplate, the corresponding monitoring configuration of establishment；If there is corresponding database host, it is not necessary to do any operation.
Some database host information is deleted if desired, for the sake of security, manual deletion is set to here.
Some database host is closed or opened if desired, is directly operated in Zabbix service ends.
(2) template management module is monitored
The incoming parameter is needed to be：Database host
First, the monitoring template of all types of databases is disposably imported in Zabbix service ends, is easy to this module to adjustWith.According to the incoming database host, according to the form of database host, type of database can be clearly resolved to, soAfterwards, it is the corresponding monitoring template of the database host carry.
(3) configuration management module is monitored
The module needs incoming parameter：Database host and action type；
Action type includes creating and deleting two types.
When creating database host, the module, incoming data storehouse main frame and establishment type, module parsing database are calledMain frame, obtains database server IP, port and type of database, and according to type of database, corresponding monitoring is got automaticallyAccount, then adds the configuration information of the database host in monitoring configuration；
When deleting some database host, the module receives database host and deletes type, according to database hostObtain and delete corresponding monitoring configuration information.
(4) monitoring data collection module
The module needs incoming parameter：Database host, monitoring type, account monitoring, database server IP and endMouthful；Above parameter is the monitoring configuration of some database service.
The module includes the collection and monitoring configuration of various database types, different for obtaining required monitoring dataThe database of type, collection item is different.According to database monitoring demand, collection item is set.
The module remotely accesses the database service according to incoming parameter, according to database its own interfaces, obtains dataThe status information in storehouse, is then set, filtering obtains the gathered data needed, is preserved together with database host according to collection itemTo local, referred to as gathered data file.
(5) acquisition module is dispatched
Need incoming parameter：Monitoring configuration
Module scans monitoring configuration, according to type of database, calls monitoring data collection module, each gathered data knotShu Hou, empties gathered data file.
Dispatching cycle can be formulated according to self-demand, then within the cycle of specifying, call the module.
After the completion of each periodic scheduling, the module calls monitoring data sending module, is sent to Zabbix service ends.
(6) monitoring data sending module
Need incoming parameter：Gathered data file
The module reads the information of gathered data file, and parsing obtains database host and corresponding collection and collectionMonitoring data, is then inserted into described in the Zabbix service ends in the corresponding record of database host by value.
If there is multiple gathered data files, the module performs multiple.
(7) check that module is omitted in monitoring
The module is optional.
Then the module compares all database hosts by all database services of agent periodic scannings, if depositedLack the database host of monitoring, calling database host management module, automatically create corresponding database host；
The module periodically calls Zabbix to service end interface, obtains all database hosts, checks the database hostWhether carry monitors template and monitoring configuration accordingly, if lacking corresponding monitoring template, then call monitoring Template Manager mouldBlock, automatic loading is corresponding to monitor template；If lacking corresponding monitoring configuration, then call monitoring configuration management module, automaticallyCreate monitoring configuration.
The beneficial effects of the invention are as follows：
(1) automation addition monitoring：As long as deploying database service, monitoring system automatically generates relevant information, is not required toArtificial treatment is wanted, maintenance cost is reduced, can avoid omitting the risk brought again.With little need for manual intervention, backup procedureRealize automation.
(2) it is ingenious in design, it is easy to safeguard：The system uses mpm models, long-range to obtain database positioning information, deployment sideJust, maintenance cost is low.
(3) simple to operate, feasibility is high：The system operating procedure fixes simple, substantially reduces risk of error, feasibility is veryIt is high.
(4) large scale database cluster can be supported：It is simple to operate because the system maintenance cost is low, it is ingenious in design, it is non-Often it is adapted to support large scale database cluster monitoring.
(5) solve the problems, such as to monitor missing well：Monitoring is added and checked in an automated fashion due to adopting, it is easy to sent outNow with solve the problems, such as monitor missing, it is ensured that the reliability of database monitoring, thus the very first time find service fault askTopic, reduction loss.
Brief description of the drawings
Fig. 1 is database Full-automatic monitoring system general frame figure of the present invention；
Fig. 2 is database Full-automatic monitoring system modules flow chart of the present invention.
Embodiment
The present invention is described in further detail with specific embodiment below in conjunction with the accompanying drawings.
Fig. 1 is present system general frame figure, and present system includes management node, mpm nodes, Zabbix service endsAnd database node, wherein, database section has many, and each database node runs an agent client, management nodeUpper operation agent management ends, mpm nodes operation agent clients.Management node obtains all databases using agent modesThe IP and port information of node, then, in Zabbix service end registration database main frames and the corresponding monitoring template of carry, togetherWhen, mpm nodes are signed in using agent modes, corresponding monitoring configuration is created；The regular remote collection database nodes of mpmStatus information, is subsequently sent to Zabbix service ends；Zabbix service end analysis monitoring data, can show the tendency chart of correlationAnd alarm, so as to accomplish the state of real-time monitoring data storehouse node.
Fig. 2 is present system modules flow chart, and database host monitoring module uses agent forms, by sweepingAll database services are retouched, Zabbix service end its own interfaces, automatic interpolation data storehouse host information, while calling monitoring are calledTemplate management module and monitoring configuration management module, are that each corresponding monitoring template of database host addition and monitoring are configured.The monitoring configuration of acquisition module periodic scanning is dispatched, and calls monitoring data collection module, is each database host gathered data,Then monitoring data sending module is called, Zabbix service ends are sent to.
Except this thinks, in order to ensure not lack monitoring, check that monitoring is omitted module and inspected periodically, find and add monitoring,Ensure the reliability of monitoring.
The implementation process of each module is described in detail below：
(1) management node
Lost including database host management module, monitoring template management module, monitoring configuration management module and inspection monitoringThe IP of database service on stripping pattern block, all database nodes of acquisition and port, then call Zabbix server ends itselfInterface, the corresponding monitoring template of automatic registration database main frame, carry and the corresponding monitoring configuration of establishment；In addition, it is optional, it is fixedPhase scans all database nodes and database host, checks the database node for lacking monitoring.
Agent management ends are run on the node, for sending instructions to agent clients, institute on database node are obtainedSome database service information.The instruction is to look on database node the presence of which database service process.
(1.1) database host management module
The agent management ends distribution instruction of management node, which is arrived on all agent clients, scan database node, ownsDatabase service, then calls Zabbix service end its own interfaces, if there is no corresponding database host, automatically creates pairThe database host answered, while calling monitoring template management module and monitoring configuration management module, automatic carry is monitored accordinglyTemplate, the corresponding monitoring configuration of establishment；If there is corresponding database host, it is not necessary to do any operation.
Some database host information is deleted if desired, for the sake of security, manual deletion is set to here.
Some database host is closed or opened if desired, is directly operated in Zabbix service ends.
(1.2) template management module is monitored
The incoming parameter is needed to be：Database host
First, the monitoring template of all types of databases is disposably imported in Zabbix service ends, is easy to this module to adjustWith.According to the incoming database host, according to the form of database host, type of database can be clearly resolved to, soAfterwards, it is the corresponding monitoring template of the database host carry.
(1.3) configuration management module is monitored
The module needs incoming parameter：Database host and action type；
Action type includes creating and deleting two types.
When creating database host, the module, incoming data storehouse main frame and establishment type, module parsing database are calledMain frame, obtains database server IP, port and type of database, and according to type of database, corresponding monitoring is got automaticallyAccount, then adds the configuration information of the database host in monitoring configuration；
When deleting some database host, the module receives database host and deletes type, according to database hostObtain and delete corresponding monitoring configuration information.
(1.4) check that module is omitted in monitoring
The module is optional.
Then the module compares all database hosts by all database services of agent periodic scannings, if depositedLack the database host of monitoring, calling database host management module, automatically create corresponding database host；
The module periodically calls Zabbix to service end interface, obtains all database hosts, checks the database hostWhether carry monitors template and monitoring configuration accordingly, if lacking corresponding monitoring template, then call monitoring Template Manager mouldBlock, automatic loading is corresponding to monitor template；If lacking corresponding monitoring configuration, then call monitoring configuration management module, automaticallyCreate monitoring configuration.
(2) Zabbix service ends
According to different types of database, data storage storehouse main frame is formulated corresponding database monitoring template, disposably ledEnter to Zabbix service ends, so that management node is called.
Database host represents some specific database service, and form is as follows：
<type>_<port>_<ip>
Type is the mark of type of database, and port is database service port, and IP is database where database serviceThe IP of node.
(3) mpm nodes
Agent clients are run on the node, the instruction of agent management ends is received and perform.
Including monitoring data collection module, scheduling acquisition module and monitoring data sending module, automatically generate monitoring configuration,Acquisition monitoring data and transmission monitoring data to Zabbix service ends.Wherein, it is not necessary to log on database node, remotely accessDatabase node, acquisition monitoring data, very convenient monitoring data storehouse.
(3.1) monitoring data collection module
The module needs incoming parameter：Database host, monitoring type, account monitoring, database server IP and endMouthful；Above parameter is the monitoring configuration of some database service.
The module includes the collection and monitoring configuration of various database types, different for obtaining required monitoring dataThe database of type, collection item is different.According to database monitoring demand, collection item is set.
The module remotely accesses the database service according to incoming parameter, according to database its own interfaces, obtains dataThe status information in storehouse, is then set, filtering obtains the gathered data needed, is preserved together with database host according to collection itemTo local, referred to as gathered data file.
(3.2) acquisition module is dispatched
Need incoming parameter：Monitoring configuration
Module scans monitoring configuration, according to type of database, calls monitoring data collection module, each gathered data knotShu Hou, empties gathered data file.
Dispatching cycle can be formulated according to self-demand, then within the cycle of specifying, call the module.
After the completion of each periodic scheduling, the module calls monitoring data sending module, is sent to Zabbix service ends.
(3.3) monitoring data sending module
Need incoming parameter：Gathered data file
The module reads the information of gathered data file, and parsing obtains database host and corresponding collection and collectionMonitoring data, is then inserted into described in the Zabbix service ends in the corresponding record of database host by value.
If there is multiple gathered data files, the module performs multiple.
(4) database node
Run on the node on database service, including various types of database services, same database node,All database port is unique, IP and port as a database service unique mark.
Agent clients are run on each database node, the instruction for receiving and performing the transmission of agent management ends.
Zabbix service ends store all database hosts and related monitoring Template Information, meanwhile, receive and storeMonitoring data, according to monitoring template, analyzes these monitoring datas, then produces alarm or monitoring tendency chart.
Monitoring template is the configuration file for defining collection item, monitored item and tendency chart, and different types of database monitors mouldPlate is different, and database host needs the corresponding monitoring template of carry.
It is the corresponding information of Value Data storehouse main frame to monitor configuration, generally comprises database host, and its IP and port, dataStorehouse type and monitoring template type, database user name password.
Agent management ends refer to the service for managing agent clients, can distribute instruction and arrive agent client executings, fromAnd the purpose of remote management database is completed, agent management ends are deployed in management node.
Agent clients refer to operate in the service on database node, and its function is reception and performs agent management endsInstruction, and return result to agent management ends.All database nodes are required for disposing agent clients.