Universal software deployment monitoring system and methodTechnical Field
The invention belongs to the technical field of big data, and relates to a general software deployment monitoring system and a general software deployment monitoring method.
Background
The novel infrastructure represented by 5G, artificial intelligence, industrial internet and Internet of things is essentially an informationized and digitalized infrastructure, and a software system is an important component of the novel infrastructure (new infrastructure). In addition, in the field of software development and software delivery, continuous integration and continuous delivery have become one of the hottest innovative directions in entrepreneurship. Software system deployment and monitoring is the most important part of the software delivery process, and the efficiency of the software system deployment and monitoring often determines the cost and profit of a software provider; efficient software deployment monitoring systems are becoming an on-demand for software vendors, particularly enterprise software providers.
In the aspect of deployment and monitoring of a software system, an open-source solution such as infrastructure, Jenkins, Puppet and the like exists in a community, and the open-source solution has a side-weight point such as infrastructure side-weight automatic operation and maintenance, although a simple Web interaction page exists, a certain technical threshold exists in practical use; jenkins focuses on automatic flow work, such as automatic compiling, automatic packaging, automatic deployment and the like, and Jenkins plays an important role in continuous integration and continuous delivery directions.
The open source scheme has the advantages of flexibility and the disadvantages of high technical requirements on implementers, need to invest in research and study, needs to integrate the open source system deployment scheme and the open source system monitoring scheme and has few open source schemes which take system deployment and system monitoring into consideration. The opposite surface of the open source scheme is a commercial deployment system, the commercial deployment monitoring system has low use threshold and low requirement on the technical capability of implementation personnel, but the purchasing cost is always considered. In addition, large-scale enterprises can choose to self-build and deploy the monitoring system, but the self-building has higher technical requirements on the enterprises and higher cost.
Disclosure of Invention
The invention aims to provide a general software deployment monitoring system and a general software deployment monitoring method, which solve the technical problem that software deployment and monitoring can be completed through simple page operation.
In order to achieve the purpose, the invention adopts the following technical scheme:
a general software deployment monitoring system comprises a server cluster and a client, wherein the server cluster and the client adopt a C/S system architecture;
a gPC mechanism is adopted between the server cluster and the client for communication;
establishing a host resource management module, a product package deployment management module, a post-deployment monitoring and warning module and a software upgrading management module in a server cluster;
the host resource management module is used for providing a gPC communication mechanism for all the servers in the server cluster and monitoring the operation condition of the servers in real time;
the product package management module is used for defining the rule of the software product package and compressing the software product package into an independent binary product package which can be copied and transmitted independently according to the rule;
the product package deployment management module is used for providing a product package management operation interface and establishing a butt joint interface between an operation item on the product package management operation interface and the product package management module;
the deployed monitoring warning module is used for automatically acquiring running indexes of the software according to rules of the software product package and supporting monitoring warning during the running of the software by setting warning rules.
The software upgrading management module is used for strictly controlling the upgrading of the software product package according to the version number of the software product package.
Preferably, the host resource management module is provided with a compatible interface supporting the client to custom install the plug-in or agent lifecycle management;
the host resource management module is provided with a performance monitoring tool and is used for automatically acquiring the performance data of the server and making the performance data into a monitoring chart.
Preferably, the software product package comprises 6 elements: the software product package comprises a program package, a starting command, a configuration file, an operation log file, a data directory and an operation index acquisition mode, wherein the rule for defining the software product package is a rule for formulating 6 elements of the software product package.
Preferably, the alarm rule includes a short message alarm or a mail alarm.
A general software deployment monitoring method comprises the following steps:
step 1: selecting one server in the Hadoop server cluster as a main server, setting other servers as hosts, and establishing the general software deployment monitoring system in the main server;
step 2: the host resource management module provides a gPC communication mechanism for all hosts accessed to the main server to realize communication between the main server and all hosts;
and step 3: the main server initializes the software environment of all the accessed hosts;
and 4, step 4: installing an access host performance monitoring tool in the main server, automatically acquiring performance data of all hosts, making the performance data into a monitoring chart, and acquiring the resource use condition of the access host through the monitoring chart;
and 5: the product package management module makes a rule according to 6 elements of the software product package, and the steps are as follows:
step S1: the software product package comprises 6 elements: a program package, a starting command, a configuration file, an operation log file, a data directory and an operation index acquisition mode;
step S2: defining the version number of the software product package in a package description file;
step S3: specifying a starting mode and a starting command of the software product package in the starting command;
step S4: listing all file names of the configuration files included in the software product in the configuration files, and making a configuration file list;
step S5: enumerating paths of all running log files of the software product package in the running log file;
step S6: the data directory specifies a data read-write directory of the software;
step S7: the monitoring port level and the maximum deployment copy number of the software product package are specified in a runtime index acquisition mode;
step 6: the product package deployment management module provides a management operation interface of the product package, the management operation interface is displayed on the client in a WEB page mode, and a user performs uploading, editing, unloading, starting, stopping, updating and executing operations of the software product package on the management operation interface and generates operation instruction information;
and 7: the product package deployment management module sends the operation instruction information to the product package management module, and the product package management module carries out actual operation on the software product package according to the operation instruction information;
inputting 6 elements of the software product package by a user through a management operation interface;
the host resource management module prepares host resources according to actual deployment requirements of software, accesses the host and carries out environment initialization on the accessed host;
and 8: after the user uploads the software product package, the product package management module decompresses the software product package into an independent binary product package which can be copied and transmitted independently according to the method in the step 5;
and step 9: the monitoring and warning module starts the collection of monitoring indexes when the software deployment is finished after the deployment, automatically collects the running indexes of the software and sends out monitoring and warning information when errors exist, and the collection of the monitoring indexes is carried out according to the running index collection mode;
step 10: and the monitoring and warning module sends a warning notice to the client according to a preset warning rule after deployment.
Step 11: the software upgrading management module dynamically checks the arranged host resources when the software product package is deployed, and performs capacity expansion and capacity reduction operations, and the software upgrading management module performs upgrading control on the software product package according to the version number of the software product package and controls the rolling stop and start of the software product package in the upgrading, capacity expansion and capacity reduction processes.
The universal software deployment monitoring system and the universal software deployment monitoring method solve the technical problem that the deployment and monitoring of software can be completed through simple page operation, standardize and streamline the deployment, monitoring and operation and maintenance processes of the software, are simple to operate, reduce the operation and maintenance implementation cost, improve the deployment, delivery and operation and maintenance efficiency, have wide applicability and high reproducibility, provide a software deployment monitoring scheme for reference for enterprises, and reduce the investment of the enterprises in implementing software system delivery.
Drawings
FIG. 1 is a flow chart illustrating host management and control according to the present invention;
fig. 2 is a schematic diagram of the system architecture of the present invention.
Detailed Description
Example 1:
a general software deployment monitoring system shown in fig. 1-2 includes a server cluster and a client, where the server cluster and the client use a C/S system architecture;
a gPC mechanism is adopted between the server cluster and the client for communication;
establishing a host resource management module, a product package deployment management module, a post-deployment monitoring and warning module and a software upgrading management module in a server cluster;
the host resource management module is used for providing a gPC communication mechanism for all the servers in the server cluster and monitoring the operation condition of the servers in real time;
in fig. 2, the host cluster is a server cluster, the visual WEB is a client, and the backend service is a host resource management module, a product package deployment management module, a post-deployment monitoring and warning module, and a software upgrade management module established in the server cluster.
The product package management module is used for defining the rule of the software product package and compressing the software product package into an independent binary product package which can be copied and transmitted independently according to the rule;
the product package deployment management module is used for providing a product package management operation interface and establishing a butt joint interface between an operation item on the product package management operation interface and the product package management module;
the deployed monitoring warning module is used for automatically acquiring running indexes of the software according to rules of the software product package and supporting monitoring warning during the running of the software by setting warning rules.
The software upgrading management module is used for strictly controlling the upgrading of the software product package according to the version number of the software product package.
Preferably, the host resource management module is provided with a compatible interface supporting the client to custom install the plug-in or agent lifecycle management;
the host resource management module is provided with a performance monitoring tool and is used for automatically acquiring the performance data of the server and making the performance data into a monitoring chart.
Preferably, the software product package comprises 6 elements: the software product package comprises a program package, a starting command, a configuration file, an operation log file, a data directory and an operation index acquisition mode, wherein the rule for defining the software product package is a rule for formulating 6 elements of the software product package.
Preferably, the alarm rule includes a short message alarm or a mail alarm.
Example 2:
as shown in fig. 1 and fig. 2, the method for monitoring deployment of generic software according to embodiment 2 is implemented on the basis of the system for monitoring deployment of generic software according to embodiment 1, and includes the following steps:
step 1: selecting one server in the Hadoop server cluster as a main server, setting other servers as hosts, and establishing the general software deployment monitoring system in the main server;
in fig. 1, the Host Server is a main Server, and the Host Cl is a Host.
Step 2: the host resource management module provides a gPC communication mechanism for all hosts accessed to the main server to realize communication between the main server and all hosts;
the embodiment supports the installation of the custom plug-ins or agent life cycle management on the host end as required, meets the functional requirements of the host end for custom performance data, log acquisition and the like, is more real-time than the traditional SSH mode for controlling the host, and has higher on-demand expansibility.
And step 3: the main server initializes the software environment of all the accessed hosts;
the embodiment supports executable scripts such as an operating system shell or a bash and the like, supports the python advanced predictive host initialization script, and can be flexibly customized and disassembled.
And 4, step 4: installing an access host performance monitoring tool in the main server, automatically acquiring performance data of all hosts, making the performance data into a monitoring chart, and acquiring the resource use condition of the access host through the monitoring chart;
and 5: the product package management module makes a rule according to 6 elements of the software product package, and the steps are as follows:
step S1: the software product package comprises 6 elements: a program package, a starting command, a configuration file, an operation log file, a data directory and an operation index acquisition mode;
step S2: defining the version number of the software product package in a package description file;
step S3: specifying a starting mode and a starting command of the software product package in the starting command;
step S4: listing all file names of the configuration files included in the software product in the configuration files, and making a configuration file list;
step S5: enumerating paths of all running log files of the software product package in the running log file;
step S6: the data directory specifies a data read-write directory of the software;
step S7: the monitoring port level and the maximum deployment copy number of the software product package are specified in a runtime index acquisition mode; such as prometheus class monitor port.
The invention abstracts six elements of software deployment, describes key information required by the software deployment in a specification mode, is suitable for various stateless software systems and stateful software systems, and has flexible definition and wide applicability.
Step 6: the product package deployment management module provides a management operation interface of the product package, the management operation interface is displayed on the client in a WEB page mode, and a user performs uploading, editing, unloading, starting, stopping, updating and executing operations of the software product package on the management operation interface and generates operation instruction information;
and 7: the product package deployment management module sends the operation instruction information to the product package management module, and the product package management module carries out actual operation on the software product package according to the operation instruction information;
inputting 6 elements of the software product package by a user through a management operation interface;
the host resource management module prepares host resources according to actual deployment requirements of software, accesses the host and carries out environment initialization on the accessed host;
and 8: after the user uploads the software product package, the product package management module decompresses the software product package into an independent binary product package which can be copied and transmitted independently according to the method in the step 5;
the user can also perform health check on the software product package on the WEB page and query the host resources.
And step 9: the monitoring and warning module starts the collection of monitoring indexes when the software deployment is finished after the deployment, automatically collects the running indexes of the software and sends out monitoring and warning information when errors exist, and the collection of the monitoring indexes is carried out according to the running index collection mode;
step 10: and the monitoring and warning module sends a warning notice to the client according to a preset warning rule after deployment.
Step 11: the software upgrading management module dynamically checks the arranged host resources when the software product package is deployed, and performs capacity expansion and capacity reduction operations, and the software upgrading management module performs upgrading control on the software product package according to the version number of the software product package and controls the rolling stop and start of the software product package in the upgrading, capacity expansion and capacity reduction processes.
The universal software deployment monitoring system and the universal software deployment monitoring method solve the technical problem that the deployment and monitoring of software can be completed through simple page operation, standardize and streamline the deployment, monitoring and operation and maintenance processes of the software, are simple to operate, reduce the operation and maintenance implementation cost, improve the deployment, delivery and operation and maintenance efficiency, have wide applicability and high reproducibility, provide a software deployment monitoring scheme for reference for enterprises, and reduce the investment of the enterprises in implementing software system delivery.