技术领域technical field
本发明属于信息处理领域，尤其涉及一种多层信息处理方法、系统及集群管理节点。The invention belongs to the field of information processing, and in particular relates to a multi-layer information processing method, system and cluster management node.
背景技术Background technique
现有技术中集群系统管理软件往往仅能获取集群本身信息，例如集群节点状态信息、集群节点存储容量信息；其它诸如集群节点的CPU信息、集群节点的内存信息，则需要其它第三方管理软件在额外的Server服务器上去获取，其中，获取信息命令从Server服务器下发到各个集群节点，然后各个集群节点将信息再返回到Server服务器上，这样会造成大量的等待时间。In the prior art, the cluster system management software can only obtain the information of the cluster itself, such as cluster node status information, cluster node storage capacity information; other such as cluster node CPU information, cluster node memory information, you need other third-party management software in the Additional Servers are used to obtain information, wherein the command to obtain information is sent from the Server server to each cluster node, and then each cluster node returns the information to the Server server, which will cause a lot of waiting time.
图1所示为现有技术的多层信息处理方法流程图，包括客户端、Server服务器、通信服务器、多个集群节点(包括集群节点1、集群节点2…集群节点n)；其中，所述客户端通过所述Server服务器与所述通信服务器相连；所述Server服务器通过所述通信服务器与所述多个集群节点相连。Fig. 1 shows the flowchart of the multi-layer information processing method in the prior art, including client, Server server, communication server, multiple cluster nodes (comprising cluster node 1, cluster node 2 ... cluster node n); wherein, the The client is connected to the communication server through the server; the server is connected to the plurality of cluster nodes through the communication server.
上述方案通信原理如下：The communication principle of the above scheme is as follows:
客户端向所述Server服务器发送信息获取请求，其中，所述信息获取请求中携带了集群节点名称信息、集群节点地址信息；The client sends an information acquisition request to the Server server, wherein the information acquisition request carries cluster node name information and cluster node address information;
所述Server服务器接收所述信息获取请求后，通过通信服务器向目标集群节点发送获取信息命令；After the Server server receives the information acquisition request, it sends an information acquisition command to the target cluster node through the communication server;
所述目标集群节点接收所述获取信息命令后，向所述Server服务器反馈响应消息，其中，所述响应消息中携带了集群节点监控信息。The target cluster node feeds back a response message to the Server after receiving the information acquisition command, wherein the response message carries cluster node monitoring information.
所述Server服务器获取集群节点监控信息后，通知所述客户端获取；After the Server server obtains the cluster node monitoring information, it notifies the client to obtain it;
所述客户端接收所述通知后，从所述Server服务器中获取所述集群节点监控信息。After receiving the notification, the client acquires the cluster node monitoring information from the Server.
上述方案中，获取信息命令从Server服务器下发到各个集群节点，然后各个集群节点将集群节点监控信息再返回到Server服务器上，这样会造成大量的等待时间，严重影响了用户体验。In the above solution, the command to obtain information is sent from the server to each cluster node, and then each cluster node returns the monitoring information of the cluster node to the server, which will cause a lot of waiting time and seriously affect the user experience.
发明内容Contents of the invention
本发明提供一种多层信息处理方法、系统及集群管理节点，以解决上述问题。The present invention provides a multi-layer information processing method, system and cluster management node to solve the above problems.
本发明提供一种多层信息处理方法。上述方法包括以下步骤：The invention provides a multi-layer information processing method. The above method comprises the following steps:
各个集群节点分别将获取的集群节点监控信息发送至主集群管理节点并由所述主集群管理节点存储；Each cluster node sends the acquired cluster node monitoring information to the main cluster management node and is stored by the main cluster management node;
通过访问主集群管理节点，获取各个集群节点监控信息。Obtain the monitoring information of each cluster node by accessing the main cluster management node.
本发明还提供了一种多层信息处理系统，包括客户端、通信服务器、多个集群节点，其中，所述客户端通过所述通信服务器与所述多个集群节点相连；The present invention also provides a multi-layer information processing system, including a client, a communication server, and multiple cluster nodes, wherein the client is connected to the multiple cluster nodes through the communication server;
各个集群节点，用于分别将获取的集群节点监控信息发送至主集群管理节点并由所述主集群管理节点存储；其中，所述主集群管理节点从所述多个集群节点中选定；Each cluster node is configured to respectively send the acquired cluster node monitoring information to the main cluster management node and be stored by the main cluster management node; wherein, the main cluster management node is selected from the plurality of cluster nodes;
所述客户端通过通信服务器访问所述主集群管理节点，获取各个集群节点监控信息。The client accesses the main cluster management node through the communication server to acquire monitoring information of each cluster node.
本发明还提供了一种集群管理节点，包括代理收集模块、服务模块、数据库模块；其中，所述代理收集模块通过所述服务模块与所述数据库模块相连；The present invention also provides a cluster management node, including an agent collection module, a service module, and a database module; wherein, the agent collection module is connected to the database module through the service module;
所述代理收集模块，用于将收集的集群节点监控信息发送至所述服务模块；The proxy collection module is configured to send the collected cluster node monitoring information to the service module;
所述服务模块，用于将获取的各个集群节点的监控信息写入所述数据库模块；The service module is configured to write the acquired monitoring information of each cluster node into the database module;
所述数据库模块，用于存储各个集群节点的监控信息。The database module is used to store monitoring information of each cluster node.
通过以下方案：各个集群节点分别将获取的集群节点监控信息发送至主集群管理节点并由所述主集群管理节点存储；通过访问主集群管理节点，获取各个集群节点监控信息；实现了客户端通过主集群管理节点获取各个集群节点监控信息，提高了数据处理速度，减少响应时间，提高了用户体验。Through the following scheme: each cluster node sends the obtained cluster node monitoring information to the main cluster management node and stored by the main cluster management node; by accessing the main cluster management node, the monitoring information of each cluster node is obtained; the client is realized through The main cluster management node obtains the monitoring information of each cluster node, which improves the data processing speed, reduces the response time, and improves the user experience.
通过以下方案：若为两个集群管理节点，则分为主集群管理节点、备份集群管理节点，所述主集群管理节点实时或定时将数据库模块中的监控信息同步至与所述备份集群管理节点中的数据库模块，使得通信安全及数据可靠性大大增强。Through the following scheme: if it is two cluster management nodes, it is divided into a main cluster management node and a backup cluster management node, and the main cluster management node synchronizes the monitoring information in the database module to the backup cluster management node in real time or regularly The database module in the system greatly enhances the communication security and data reliability.
附图说明Description of drawings
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：The accompanying drawings described here are used to provide a further understanding of the present invention and constitute a part of the application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations to the present invention. In the attached picture:
图1所示为现有技术的多层信息处理方法流程图；Fig. 1 shows the flowchart of the multi-layer information processing method in the prior art;
图2所示为本发明实施例1的多层信息处理方法流程图；FIG. 2 is a flowchart of a multi-layer information processing method in Embodiment 1 of the present invention;
图3所示为本发明实施例2的多层信息处理系统结构图；FIG. 3 is a structural diagram of a multi-layer information processing system according to Embodiment 2 of the present invention;
图4所示为本发明实施例3的集群管理节点结构图。FIG. 4 is a structural diagram of a cluster management node in Embodiment 3 of the present invention.
具体实施方式Detailed ways
下文中将参考附图并结合实施例来详细说明本发明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。Hereinafter, the present invention will be described in detail with reference to the drawings and examples. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.
图2所示为本发明实施例1的多层信息处理方法流程图，包括以下步骤：FIG. 2 is a flow chart of a multi-layer information processing method in Embodiment 1 of the present invention, including the following steps:
步骤201：各个集群节点分别将获取的集群节点监控信息发送至主集群管理节点；Step 201: each cluster node sends the acquired cluster node monitoring information to the main cluster management node;
进一步地，各个集群节点的代理收集模块分别将获取的集群节点监控信息发送至主集群管理节点。Further, the agent collection modules of each cluster node respectively send the acquired cluster node monitoring information to the main cluster management node.
进一步地，各个集群节点的代理收集模块定时或实时通过广播方式，将获取的集群节点监控信息发送至主集群管理节点。Further, the proxy collection module of each cluster node sends the acquired cluster node monitoring information to the main cluster management node by broadcasting at regular intervals or in real time.
进一步地，所述集群节点监控信息包括集群节点状态信息、集群节点存储容量信息、集群节点中的文件系统服务状态信息、集群节点CPU信息、集群节点内存信息。Further, the cluster node monitoring information includes cluster node status information, cluster node storage capacity information, file system service status information in the cluster node, cluster node CPU information, and cluster node memory information.
进一步地，集群节点的代理收集模块通过集群命令，获取集群节点状态信息、集群节点存储容量信息、集群节点中的文件系统服务状态信息；集群节点的代理收集模块通过操作系统命令，获取集群节点CPU信息、集群节点内存信息。Further, the proxy collection module of the cluster node obtains cluster node status information, cluster node storage capacity information, and file system service status information in the cluster node through the cluster command; the proxy collection module of the cluster node obtains the cluster node CPU through the operating system command. Information, cluster node memory information.
进一步地，预先在各个集群节点中设置代理收集模块即Agent模块及在多个集群管理节点中设置Agent模块、服务模块即Server模块、数据库模块；Further, an agent collection module, namely the Agent module, is set in each cluster node in advance, and an Agent module, a service module, namely the Server module, and a database module are set in a plurality of cluster management nodes;
进一步地，所述多个集群管理节点互为主备。例如：若为两个集群管理节点，则分为主集群管理节点、备份集群管理节点。Further, the multiple cluster management nodes are active and standby for each other. For example: if there are two cluster management nodes, they are divided into active cluster management nodes and backup cluster management nodes.
进一步地，所述多个集群管理节点从各个集群节点中选定。Further, the multiple cluster management nodes are selected from each cluster node.
步骤202：所述主集群管理节点中的服务模块即Server模块将所述集群节点监控信息写入数据库模块。Step 202: the server module in the master cluster management node writes the cluster node monitoring information into a database module.
进一步地，所述主集群管理节点实时或定时将数据库模块中的监控信息同步至与所述备份集群管理节点中的数据库模块。Further, the master cluster management node synchronizes the monitoring information in the database module with the database module in the backup cluster management node in real time or periodically.
步骤203：通过访问主集群管理节点中的数据库模块，获取各个集群节点监控信息。Step 203: Obtain monitoring information of each cluster node by accessing the database module in the master cluster management node.
进一步地，客户端通过通信服务器访问主集群管理节点中的数据库模块，获取各个集群节点监控信息。Further, the client accesses the database module in the main cluster management node through the communication server to obtain monitoring information of each cluster node.
图3所示为本发明实施例2的多层信息处理系统结构图，包括客户端、通信服务器、多个集群节点(集群节点1、集群节点2…集群节点n)；其中，所述客户端通过所述通信服务器与所述多个集群节点相连；Fig. 3 shows the structure diagram of the multi-layer information processing system of Embodiment 2 of the present invention, including a client, a communication server, and a plurality of cluster nodes (cluster node 1, cluster node 2...cluster node n); wherein, the client connected to the plurality of cluster nodes through the communication server;
各个集群节点，用于分别将获取的集群节点监控信息发送至主集群管理节点并由所述主集群管理节点存储；其中，所述主集群管理节点从所述多个集群节点中选定(例如：选择集群节点1为主集群管理节点)；Each cluster node is configured to respectively send the obtained cluster node monitoring information to the master cluster management node and store it by the master cluster management node; wherein, the master cluster management node is selected from the plurality of cluster nodes (for example : select cluster node 1 as the master cluster management node);
所述客户端通过通信服务器访问所述主集群管理节点，获取各个集群节点监控信息。The client accesses the main cluster management node through the communication server to acquire monitoring information of each cluster node.
图4所示为本发明实施例3的集群管理节点结构图，包括代理收集模块、服务模块、数据库模块；其中，所述代理收集模块通过所述服务模块与所述数据库模块相连；FIG. 4 is a structural diagram of a cluster management node according to Embodiment 3 of the present invention, including an agent collection module, a service module, and a database module; wherein, the agent collection module is connected to the database module through the service module;
所述代理收集模块，用于将收集的集群节点监控信息发送至所述服务模块；The proxy collection module is configured to send the collected cluster node monitoring information to the service module;
所述服务模块，用于将获取的各个集群节点的监控信息写入所述数据库模块；The service module is configured to write the acquired monitoring information of each cluster node into the database module;
所述数据库模块，用于存储各个集群节点的监控信息。The database module is used to store monitoring information of each cluster node.
通过以下方案：各个集群节点分别将获取的集群节点监控信息发送至主集群管理节点并由所述主集群管理节点存储；通过访问主集群管理节点，获取各个集群节点监控信息；实现了客户端通过主集群管理节点获取各个集群节点监控信息，提高了数据处理速度，减少响应时间，提高了用户体验。Through the following scheme: each cluster node sends the obtained cluster node monitoring information to the main cluster management node and stored by the main cluster management node; by accessing the main cluster management node, the monitoring information of each cluster node is obtained; the client is realized through The main cluster management node obtains the monitoring information of each cluster node, which improves the data processing speed, reduces the response time, and improves the user experience.
通过以下方案：若为两个集群管理节点，则分为主集群管理节点、备份集群管理节点，所述主集群管理节点实时或定时将数据库模块中的监控信息同步至与所述备份集群管理节点中的数据库模块，使得通信安全及数据可靠性大大增强。Through the following scheme: if it is two cluster management nodes, it is divided into a main cluster management node and a backup cluster management node, and the main cluster management node synchronizes the monitoring information in the database module to the backup cluster management node in real time or regularly The database module in the system greatly enhances the communication security and data reliability.
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510512640.5ACN105007193A (en) | 2015-08-19 | 2015-08-19 | Multi-layer information processing method, system thereof and cluster management node |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510512640.5ACN105007193A (en) | 2015-08-19 | 2015-08-19 | Multi-layer information processing method, system thereof and cluster management node |
| Publication Number | Publication Date |
|---|---|
| CN105007193Atrue CN105007193A (en) | 2015-10-28 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510512640.5APendingCN105007193A (en) | 2015-08-19 | 2015-08-19 | Multi-layer information processing method, system thereof and cluster management node |
| Country | Link |
|---|---|
| CN (1) | CN105007193A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106027623A (en)* | 2016-03-14 | 2016-10-12 | 中国科学院计算技术研究所 | Distributed cluster state management method and system thereof |
| CN106161090A (en)* | 2016-07-12 | 2016-11-23 | 许继集团有限公司 | The monitoring method of a kind of subregion group system and device |
| CN106301895A (en)* | 2016-08-03 | 2017-01-04 | 浪潮(北京)电子信息产业有限公司 | A kind of disaster recovery method obtaining cluster monitoring data and device |
| CN106453377A (en)* | 2016-10-28 | 2017-02-22 | 中金云金融（北京）大数据科技股份有限公司 | Intelligent monitoring system and method for distributed network based on block chain |
| CN106506203A (en)* | 2016-10-25 | 2017-03-15 | 杭州云象网络技术有限公司 | A kind of monitoring nodes system for being applied to block chain |
| CN106790723A (en)* | 2017-03-23 | 2017-05-31 | 泰康保险集团股份有限公司 | The monitoring method of application service, load-balancing device and monitoring device |
| CN107395443A (en)* | 2017-08-31 | 2017-11-24 | 郑州云海信息技术有限公司 | A kind of distributed type assemblies management method, apparatus and system |
| CN111130920A (en)* | 2019-11-26 | 2020-05-08 | 网宿科技股份有限公司 | Hardware information acquisition method, device, server and storage medium |
| CN112311596A (en)* | 2020-10-22 | 2021-02-02 | 深圳前海微众银行股份有限公司 | Data management method, apparatus, device and computer storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101667034A (en)* | 2009-09-21 | 2010-03-10 | 北京航空航天大学 | Scalable monitoring system supporting hybrid clusters |
| CN103607297A (en)* | 2013-11-07 | 2014-02-26 | 上海爱数软件有限公司 | Fault processing method of computer cluster system |
| CN103701661A (en)* | 2013-12-23 | 2014-04-02 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing node monitoring |
| US20140164452A1 (en)* | 2012-12-06 | 2014-06-12 | Empire Technology Development Llc | Decentralizing a hadoop cluster |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101667034A (en)* | 2009-09-21 | 2010-03-10 | 北京航空航天大学 | Scalable monitoring system supporting hybrid clusters |
| US20140164452A1 (en)* | 2012-12-06 | 2014-06-12 | Empire Technology Development Llc | Decentralizing a hadoop cluster |
| CN103607297A (en)* | 2013-11-07 | 2014-02-26 | 上海爱数软件有限公司 | Fault processing method of computer cluster system |
| CN103701661A (en)* | 2013-12-23 | 2014-04-02 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing node monitoring |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106027623A (en)* | 2016-03-14 | 2016-10-12 | 中国科学院计算技术研究所 | Distributed cluster state management method and system thereof |
| CN106161090A (en)* | 2016-07-12 | 2016-11-23 | 许继集团有限公司 | The monitoring method of a kind of subregion group system and device |
| CN106301895A (en)* | 2016-08-03 | 2017-01-04 | 浪潮(北京)电子信息产业有限公司 | A kind of disaster recovery method obtaining cluster monitoring data and device |
| CN106506203A (en)* | 2016-10-25 | 2017-03-15 | 杭州云象网络技术有限公司 | A kind of monitoring nodes system for being applied to block chain |
| CN106506203B (en)* | 2016-10-25 | 2019-12-10 | 杭州云象网络技术有限公司 | Node monitoring system applied to block chain |
| CN106453377B (en)* | 2016-10-28 | 2021-03-02 | 中金云金融（北京）大数据科技股份有限公司 | Block chain based distributed network intelligent monitoring system and method |
| CN106453377A (en)* | 2016-10-28 | 2017-02-22 | 中金云金融（北京）大数据科技股份有限公司 | Intelligent monitoring system and method for distributed network based on block chain |
| CN106790723A (en)* | 2017-03-23 | 2017-05-31 | 泰康保险集团股份有限公司 | The monitoring method of application service, load-balancing device and monitoring device |
| CN106790723B (en)* | 2017-03-23 | 2020-03-13 | 泰康保险集团股份有限公司 | Application service monitoring method, load balancing equipment and monitoring equipment |
| CN107395443A (en)* | 2017-08-31 | 2017-11-24 | 郑州云海信息技术有限公司 | A kind of distributed type assemblies management method, apparatus and system |
| CN111130920A (en)* | 2019-11-26 | 2020-05-08 | 网宿科技股份有限公司 | Hardware information acquisition method, device, server and storage medium |
| CN112311596A (en)* | 2020-10-22 | 2021-02-02 | 深圳前海微众银行股份有限公司 | Data management method, apparatus, device and computer storage medium |
| CN112311596B (en)* | 2020-10-22 | 2023-05-12 | 深圳前海微众银行股份有限公司 | Data management method, device, equipment and computer storage medium |
| Publication | Publication Date | Title |
|---|---|---|
| CN105007193A (en) | Multi-layer information processing method, system thereof and cluster management node | |
| US10817195B2 (en) | Key-value based message oriented middleware | |
| US10609150B2 (en) | Lock management method in cluster, lock server, and client | |
| CN106662983B (en) | The methods, devices and systems of data reconstruction in distributed memory system | |
| CN102148850B (en) | Cluster system and service processing method thereof | |
| CN105493474B (en) | System and method for supporting partition level logging for synchronizing data in a distributed data grid | |
| CN109407977B (en) | A method and system for distributed storage management of big data | |
| CN103530362B (en) | A kind of computer data reading/writing method for many copies distributed system | |
| US9699276B2 (en) | Data distribution method and system and data receiving apparatus | |
| WO2015192661A1 (en) | Method, device, and system for data synchronization in distributed storage system | |
| CN103138912B (en) | Method of data synchronization and system | |
| CN104202387B (en) | A kind of metadata restoration methods and relevant apparatus | |
| JP5686034B2 (en) | Cluster system, synchronization control method, server device, and synchronization control program | |
| CN111552701B (en) | Method for determining data consistency in distributed cluster and distributed data system | |
| CN104935654A (en) | A cache method, write point client, and read client in a server cluster system | |
| CN102130959A (en) | A system and method for implementing cloud storage resource scheduling | |
| CN106210147A (en) | A kind of load-balancing method based on poll and device | |
| CN105338078A (en) | Data storage method and device used for storing system | |
| CN113010549A (en) | Data processing method based on remote multi-active system, related equipment and storage medium | |
| CN107888666A (en) | A kind of cross-region data-storage system and method for data synchronization and device | |
| CN105554143A (en) | High-availability cache server and data processing method and system thereof | |
| CN105610947A (en) | Method, device and system for realizing high-available distributed queue service | |
| WO2023207492A1 (en) | Data processing method and apparatus, device, and readable storage medium | |
| JP2010044553A (en) | Data processing method, cluster system, and data processing program | |
| CN104410531A (en) | Redundant system architecture approach |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20151028 |