





技术领域technical field
本发明涉及通讯技术领域,尤其涉及一种中间件性能的监控方法和设备。 The invention relates to the technical field of communication, in particular to a method and equipment for monitoring middleware performance. the
背景技术Background technique
中间件是介于客户机与服务器之间的夹层,它突破了二层C/S(Client/Server,客户端/服务器)结构的局限性,为构建大规模、高性能、分布式C/S应用程序提供了通信、事务、安全、容错等基础服务,它屏蔽了底层技术细节,使应用程序开发不必再从底层做起,以自身的复杂性换来了应用程序开发的简单。 Middleware is the mezzanine between the client and the server, which breaks through the limitations of the two-tier C/S (Client/Server, client/server) structure, and provides a solution for building large-scale, high-performance, distributed C/S The application program provides basic services such as communication, transaction, security, and fault tolerance. It shields the underlying technical details, so that application development does not have to start from the bottom layer, and the complexity of the application is exchanged for the simplicity of application development. the
目前常用的中间件系统包括TUXDEO(Transaction for UNIX has beenExtended for Distributed Operation,,被分布式操作扩展之后的UNIX事务系统)、MQSeries系统等。以TUXEDO系统为例,TUXEDO是在企业、Internet这样的分布式运算环境中开发和管理三层结构的C/S型关键任务应用系统的强有力工具。它具备分布式事务处理和应用通信功能,并提供完善的各种服务来建立、运行和管理关键任务应用系统。开发人员能够通过其建立跨多个硬件平台、数据库和操作系统的可互操作的应用系统。 Currently commonly used middleware systems include TUXDEO (Transaction for UNIX has been Extended for Distributed Operation, UNIX transaction system extended by distributed operation), MQSeries system, etc. Taking TUXEDO system as an example, TUXEDO is a powerful tool for developing and managing three-tiered C/S-type mission-critical application systems in distributed computing environments such as enterprises and the Internet. It has distributed transaction processing and application communication functions, and provides a variety of comprehensive services to build, run and manage mission-critical application systems. Developers can use it to build interoperable application systems across multiple hardware platforms, databases and operating systems. the
目前基于中间件的企业应用环境中,系统是7天*24小时运行,不能间断,系统运行压力比较大,中间件的性能监控对于保障系统的稳定性很重要。 In the current middleware-based enterprise application environment, the system runs 24 hours a day, 7 days a day without interruption, and the pressure on the system is relatively high. The performance monitoring of middleware is very important to ensure the stability of the system. the
现有的中间件监控技术大多基于系统日志和命令行方式实现,如对于目前常用的中间件TUXDEO系统的管理监控方法主要包括以下两种,一种就是常用的命令行方式,用tmadmin,dmadmin,tmconfig等命令来查看TUXEDO系统状况,例如通过tmadmin和tmconfig两个命令行工具实现不同级别的TUXEDO配置修改时,tmadmin的作用主要是监控TUXEDO的运行,可以修改服务的参数;而通过tmconfig可以在线修改当前TUXEDO的系统UBB (Ultimate Bulletin Board,一种BBS程序)配置,即通过MIB(ManagementInformation Base,管理信息库)修改用UBBCONFIG生成的二进制TUXCONFIG在内存的拷贝;另一种就是通过设置服务的日志方式来记录执行情况。 Most of the existing middleware monitoring technologies are implemented based on system logs and command lines. For example, the management and monitoring methods of the commonly used middleware TUXDEO system mainly include the following two methods. One is the commonly used command line method, using tmadmin, dmadmin, tmconfig and other commands to view the TUXEDO system status, for example, when using tmadmin and tmconfig two command line tools to implement different levels of TUXEDO configuration modification, the function of tmadmin is mainly to monitor the operation of TUXEDO and modify the parameters of the service; and through tmconfig can be modified online The current TUXEDO system UBB (Ultimate Bulletin Board, a BBS program) configuration, that is, through MIB (Management Information Base, management information base) to modify the copy of the binary TUXCONFIG generated by UBBCONFIG in memory; the other is to set the log method of the service to record the execution. the
发明人在实现本发明的过程中,发现现有技术中存在的问题在于: In the process of realizing the present invention, the inventor finds that the problems in the prior art are:
现有实现中间件监控通过命令或日志方式方法,首先在技术架构上,不具有开放性和可扩展性,操作较复杂;而且基于命令行方式依赖于中间件系统本身所提供的命令集合,具有一定的管理宽度局限性。 Existing middleware monitoring methods through commands or logs, first of all, in terms of technical architecture, do not have openness and scalability, and the operation is more complicated; and based on the command line, it depends on the set of commands provided by the middleware system itself, which has Certain management width limitations. the
发明内容Contents of the invention
本发明的实施例提供一种中间件性能的监控方法和设备,用于简便高效的实现对中间件性能数据的监控和管理。 Embodiments of the present invention provide a method and device for monitoring middleware performance, which are used to monitor and manage middleware performance data simply and efficiently. the
本发明的实施例提供了一种中间件性能的监控方法,包括: Embodiments of the present invention provide a monitoring method for middleware performance, including:
获取配置文件,根据所述配置文件的配置从外部数据库中获取SNMP操作任务列表,其中,所述配置文件中包括获取中间件的性能参数数据、以及对采集到的性能参数数据进行合并处理的相关配置; Obtain the configuration file, and obtain the SNMP operation task list from the external database according to the configuration of the configuration file, wherein the configuration file includes obtaining the performance parameter data of the middleware and the related information for merging the collected performance parameter data configuration;
根据所述SNMP操作任务列表,获取中间件的性能参数数据; Obtain the performance parameter data of the middleware according to the SNMP operation task list;
对采集到的性能参数数据进行合并处理并保存。 Combine and save the collected performance parameter data. the
其中,还包括: Among them, also include:
接收用户发送的系统消息,根据所述系统消息的内容对所述获取配置文件、获取中间件的性能参数数据以及对采集到的性能参数数据进行合并处理并保存的步骤进行相应的控制。 The system message sent by the user is received, and the steps of acquiring the configuration file, acquiring the performance parameter data of the middleware, and merging and saving the collected performance parameter data are controlled accordingly according to the content of the system message. the
其中,还包括: Among them, also include:
对所述获取配置文件、获取中间件的性能参数数据以及对采集到的性能参数数据进行合并处理并保存的步骤,生成日志文件并存储。 A log file is generated and stored for the steps of acquiring configuration files, acquiring performance parameter data of the middleware, and merging and storing the collected performance parameter data. the
其中,所述根据所述SNMP操作任务列表,获取中间件的性能参数数据,包括: Wherein, according to the SNMP operation task list, the performance parameter data of the middleware is obtained, including:
定时检查所述SNMP操作任务列表,发现有符合条件的SNMP操作任务时,则将其加入线程池并运行;所述线程池中没有空闲线程时,则等待有空闲线程存在后运行;所述线程池为一先入先出的线程池队列。 Regularly check the SNMP operation task list, and when finding a qualified SNMP operation task, it is added to the thread pool and runs; when there is no idle thread in the thread pool, it waits for an idle thread to exist and then runs; the thread The pool is a first-in-first-out thread pool queue. the
其中,所述对采集到的性能参数数据进行合并处理并保存,包括: Wherein, the combined processing and preservation of the collected performance parameter data includes:
对采集到的性能参数数据进行分析,针对同一类别的数据进行合并; Analyze the collected performance parameter data and merge data of the same category;
将合并后得到的数据文件通过队列消息发送给外部的invoker程序进行入库操作。 Send the merged data files to the external invoker program through the queue message for storage operation. the
本发明的实施例还提供了一种网络设备,用于对中间件性能进行监控,包括: Embodiments of the present invention also provide a network device for monitoring middleware performance, including:
主控模块,用于调度配置更新模块获取配置文件,并接收所述配置更新模块发送的SNMP操作任务列表; The main control module is used to schedule the configuration update module to obtain the configuration file, and receive the SNMP operation task list sent by the configuration update module;
配置更新模块,用于在所述主控模块的控制下获取配置文件,根据所述配置文件的配置从外部数据库中获取SNMP操作任务列表,并通知所述主控模块,其中,所述配置文件中包括获取中间件的性能参数数据、以及对采集到的性能参数数据进行合并处理的相关配置; A configuration update module, configured to obtain a configuration file under the control of the main control module, obtain an SNMP operation task list from an external database according to the configuration of the configuration file, and notify the main control module, wherein the configuration file Including obtaining the performance parameter data of the middleware and related configurations for merging the collected performance parameter data;
调度模块,用于根据所述主控模块中维护的SNMP操作任务列表,获取中间件的性能参数数据; Scheduling module, for obtaining the performance parameter data of the middleware according to the SNMP operation task list maintained in the main control module;
数据整理模块,用于周期性的检查所述调度模块采集到的性能参数数据,对采集到的性能参数数据进行合并处理并保存。 The data sorting module is used to periodically check the performance parameter data collected by the scheduling module, and merge and store the collected performance parameter data. the
其中,还包括: Among them, also include:
系统消息处理模块,用于接收用户发送的系统消息,根据所述系统消息的内容向所述主控模块发送相应的控制消息; A system message processing module, configured to receive a system message sent by a user, and send a corresponding control message to the main control module according to the content of the system message;
所述主控模块,还用于根据所述系统消息处理模块发送的控制消息,对所述配置更新模块、调度模块以及数据整理模块进行相应的控制。 The main control module is further configured to control the configuration update module, scheduling module and data sorting module according to the control message sent by the system message processing module. the
其中,还包括: Among them, also include:
日志模块,用于为所述主控模块、配置更新模块、调度模块、数据整理模块以及系统消息处理模块提供公用的日志功能,生成日志文件并存储。 The log module is used to provide a common log function for the main control module, the configuration update module, the scheduling module, the data sorting module and the system message processing module, and generate and store log files. the
其中,所述配置更新模块,还用于当所述配置文件中还包括获取中间件的性能参数数据、以及对采集到的性能参数数据进行合并处理的相关配置时,将相关配置提供给所述调度模块和数据整理模块。 Wherein, the configuration update module is further configured to provide relevant configurations to the Scheduling module and data collation module. the
其中,所述调度模块具体用于: Wherein, the scheduling module is specifically used for:
定时检查所述主控模块中的SNMP操作任务列表,发现有符合条件的SNMP操作任务时,则将其加入线程池并运行;所述线程池中没有空闲线程时,则等待有空闲线程存在后运行;所述线程池为一先入先出的线程池队列。 Regularly check the SNMP operation task list in the main control module, and when finding qualified SNMP operation tasks, it will be added to the thread pool and run; when there is no idle thread in the thread pool, then wait for an idle thread to exist Running; the thread pool is a first-in-first-out thread pool queue. the
其中,所述数据整理模块具体用于: Wherein, the data collation module is specifically used for:
对采集到的性能参数数据进行分析,针对同一类别的数据进行合并; Analyze the collected performance parameter data and merge data of the same category;
将合并后得到的数据文件通过队列消息发送给外部的invoker程序进行入库操作。 Send the merged data files to the external invoker program through the queue message for storage operation. the
与现有技术相比,本发明的实施例具有以下优点: Compared with the prior art, embodiments of the present invention have the following advantages:
基于SNMP操作任务列表的配置,获取中间件的性能参数数据并对采集到的性能参数数据进行合并处理并保存。与现有的使用命令行方式的中间件监控方式相比,采用了可配置的基于SNMP协议的分布式网络管理模型,与传统的网络管理模型相比,具有更好的灵活性和可扩展性。 Based on the configuration of the SNMP operation task list, the performance parameter data of the middleware is obtained, and the collected performance parameter data is merged, processed and saved. Compared with the existing middleware monitoring method using the command line, it adopts a configurable distributed network management model based on the SNMP protocol, which has better flexibility and scalability than the traditional network management model . the
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。 In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort. the
图1是本发明的实施例中提供的中间件性能的监控方法流程图; Fig. 1 is the monitoring method flowchart of the middleware performance provided in the embodiment of the present invention;
图2是本发明的实施例中提供的SNMPDau所应用的系统结构示意图; Fig. 2 is a schematic structural diagram of the applied system of SNMPDau provided in the embodiments of the present invention;
图3是本发明的实施例中提供的SNMPDau与其他系统的连接示意图; Fig. 3 is the connection schematic diagram of SNMPDau and other systems provided in the embodiment of the present invention;
图4是本发明的实施例中提供的网络设备的结构示意图; Fig. 4 is a schematic structural diagram of a network device provided in an embodiment of the present invention;
图5是本发明的实施例中提供的网络设备的应用环境示意图; Fig. 5 is the application environment schematic diagram of the network equipment provided in the embodiment of the present invention;
图6是本发明的实施例中提供的设备以及设备间交互示意图; Fig. 6 is a schematic diagram of devices and interactions between devices provided in an embodiment of the present invention;
图7是本发明的实施例中提供的另一设备以及设备间交互示意图。 Fig. 7 is a schematic diagram of another device and interaction between devices provided in the embodiment of the present invention. the
具体实施方式Detailed ways
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。 The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention. the
本发明实施例提供了一种中间件性能的监控方法,如图1所示,包括: The embodiment of the present invention provides a monitoring method for middleware performance, as shown in Figure 1, including:
步骤s101、获取配置文件,根据所述配置文件的配置从外部数据库中获取SNMP操作任务列表; Step s101, obtaining the configuration file, and obtaining the SNMP operation task list from the external database according to the configuration of the configuration file;
步骤s102、根据所述SNMP操作任务列表,获取中间件的性能参数数据; Step s102, according to the SNMP operation task list, obtain the performance parameter data of the middleware;
具体的,该步骤可以进一步包括:定时检查所述SNMP操作任务列表,发现有符合条件的SNMP操作任务时,则将其加入线程池并运行;所述线程池中没有空闲线程时,则等待有空闲线程存在后运行;所述线程池为一先入先出的线程池队列。 Concrete, this step can further comprise: regularly check described SNMP operation task list, when finding that there is qualified SNMP operation task, then add it to thread pool and run; When there is no idle thread in described thread pool, then wait for available Run after the idle thread exists; the thread pool is a first-in-first-out thread pool queue. the
步骤s103、对采集到的性能参数数据进行合并处理并保存。 Step s103, combining and saving the collected performance parameter data. the
具体的,该步骤可以进一步包括:对采集到的性能参数数据进行分析,针对同一类别的数据进行合并;将合并后得到的数据文件通过posix消息队列消息发送给外部的invoker程序进行入库操作。 Specifically, this step may further include: analyzing the collected performance parameter data, and merging data of the same category; sending the merged data files to an external invoker program through a posix message queue message for storage operation. the
另外,还包括接收到用户发送的系统消息时,根据所述系统消息的内容对所述获取配置文件、获取中间件的性能参数数据以及对采集到的性能参数数据进行合并处理并保存的步骤进行相应的控制。 In addition, when a system message sent by a user is received, the steps of acquiring the configuration file, acquiring the performance parameter data of the middleware, and merging and saving the collected performance parameter data are carried out according to the content of the system message Control accordingly. the
另外,配置文件中还包括获取中间件的性能参数数据、以及对采集到的性能参数数据进行合并处理的相关配置时,步骤s102和s103中的操作需要根据配置进行。 In addition, when the configuration file also includes related configurations for acquiring performance parameter data of the middleware and merging the collected performance parameter data, the operations in steps s102 and s103 need to be performed according to the configuration. the
另外,还可以对步骤s101中获取配置文件、步骤s102中获取中间件的性 能参数数据、以及步骤s103中对采集到的性能参数数据进行合并处理并保存的流程,分别生成日志文件并存储。 In addition, log files can also be generated and stored for the processes of obtaining the configuration file in step s101, obtaining the performance parameter data of the middleware in step s102, and merging and storing the collected performance parameter data in step s103. the
以下结合具体的应用场景对发明的具体实施方式进行介绍。 The specific implementation manner of the invention will be introduced below in conjunction with specific application scenarios. the
本发明实施例提供的中间件性能的监控方法中,采用比较流行和成熟的网络管理模型如SNMP(Simple Network Management Protocol,简单网络管理协议)模型,实现基于SNMP的中间件性能监控。本发明实施例中,将能够实施本发明实施例提供的方法的装置称为SNMPDau模块,使用标准的SNMP协议,根据接收到的配置,获取被监控设备的属性值。然后存储到设定的文件中,最后,通过invoker程序,读取文件中的值并进行入库操作。 In the monitoring method of middleware performance that the embodiment of the present invention provides, adopt more popular and mature network management model such as SNMP (Simple Network Management Protocol, Simple Network Management Protocol) model, realize the middleware performance monitoring based on SNMP. In the embodiment of the present invention, the device capable of implementing the method provided by the embodiment of the present invention is called the SNMPDau module, which uses the standard SNMP protocol to obtain the attribute value of the monitored device according to the received configuration. Then store it in the set file, and finally, through the invoker program, read the value in the file and perform storage operation. the
本发明的实施例中,SNMPDau模块作为基于SNMP协议的中间件性能参数采集程序,在网管系统中的位置如图2所示, In an embodiment of the present invention, the SNMPDau module is as a middleware performance parameter acquisition program based on the SNMP protocol, and its position in the network management system is as shown in Figure 2,
其中,网管系统的结构包括:展现层、业务处理层、基础数据处理层以及采集层。其中: Among them, the structure of the network management system includes: presentation layer, business processing layer, basic data processing layer and collection layer. in:
展现层,主要用于图形化展现采集来的监控数据;与用户互动,响应用户的操作与设定;以及集成第三方监控产品等; The display layer is mainly used to graphically display the collected monitoring data; interact with users, respond to user operations and settings; and integrate third-party monitoring products, etc.;
业务处理层和基础数据处理层,主要用于处理采集层采集到的监控数据,以供展现层调用监控数据;同时也用于处理展现层传递的用户操作与设定; The business processing layer and the basic data processing layer are mainly used to process the monitoring data collected by the acquisition layer for the display layer to call the monitoring data; at the same time, they are also used to process user operations and settings passed by the display layer;
采集层,主要用于采集被监控资源各项指标的数据,并上传至业务处理层和基础数据处理层。 The collection layer is mainly used to collect the data of various indicators of the monitored resources and upload them to the business processing layer and the basic data processing layer. the
本发明实施例中提供的SNMPDau模块位于网管系统最底层的采集层中,主要完成基于SNMP协议的中间件性能参数数据采集、处理等功能。SNMPDau模块启动后,SNMPDau获取预先配置的SNMP操作任务列表,完成对中间件的SNMP原始数据的采集、处理等功能,并将采集后的原始数据保存为文件形式,通过Invoker程序进行入库。 The SNMPDau module provided in the embodiment of the present invention is located in the collection layer at the bottom of the network management system, and mainly completes functions such as data collection and processing of middleware performance parameters based on the SNMP protocol. After the SNMPDau module is started, SNMPDau obtains the pre-configured SNMP operation task list, completes the collection and processing of the SNMP raw data of the middleware, and saves the collected raw data in the form of a file, which is stored through the Invoker program. the
以下对图2中涉及的系统示意图中SNMPDau模块与其他系统的具体连接关系进行说明。如图3所示,为SNMPDau模块与通过不同的接口与其他系统连接的结构示意图。 The specific connection relationship between the SNMPDau module and other systems in the system schematic diagram involved in FIG. 2 will be described below. As shown in Figure 3, it is a schematic diagram of the structure of the SNMPDau module connected to other systems through different interfaces. the
以下对每个接口的分别进行以下详细说明: The following is a detailed description of each interface:
d1:SNMPDau模块与数据库(database)间的访问接口,SNMPDau模块可以通过该接口向SNMPDau模块提供SNMP操作任务列表,该d1接口为单向只读接口,SNMPDau模块不能通过该接口修改数据库中的任何内容。SNMP操作任务列表中包括被监控设备的某属性的OID(Object identifier,目标标识), d1: The access interface between the SNMPDau module and the database (database). The SNMPDau module can provide the SNMP operation task list to the SNMPDau module through this interface. The d1 interface is a one-way read-only interface. The SNMPDau module cannot modify any data in the database through this interface. content. The SNMP operation task list includes the OID (Object identifier) of an attribute of the monitored device,
f1:采集入库数据接口,为SNMPDau模块与invoker程序间的数据接口。SNMPDau模块记录所有采集到的原始数据并保存在文件中后,SNMPDau模块可以通过该f1接口将文件传送给invoker程序,进而由invoker程序统一完成入库操作。 f1: The interface for collecting and storing data, which is the data interface between the SNMPDau module and the invoker program. After the SNMPDau module records all the collected raw data and saves them in the file, the SNMPDau module can transfer the file to the invoker program through the f1 interface, and then the invoker program completes the storage operation in a unified manner. the
i1:采集入库接口,为SNMPDau模块与invoker程序间的控制接口。当SNMPDau模块中的数据文件准备好后,通过该i1接口向invoker程序发送消息,以便invoker程序得悉需要进行入库的文件。该消息接口i1可以采用IPC(Inter Process Communication,进程间通信)消息队列,以posix消息队列实现,队列名称可配置。 i1: Collection and storage interface, which is the control interface between the SNMPDau module and the invoker program. When the data files in the SNMPDau module are ready, send a message to the invoker program through the i1 interface, so that the invoker program can learn the files that need to be stored. The message interface i1 can use IPC (Inter Process Communication, inter-process communication) message queue, and implement it with posix message queue, and the queue name is configurable. the
s1、s2、…、sn:这些sn(n=1,…,N)接口为SNMP接口,为SNMPDau模块与各代理(如网络设备代理、主机代理以及第三方代理等),既是控制通道,也是数据通道。SNMPDau模块通过标准的SNMP协议,可以通过这些接口,实现对于各个代理的中间件性能参数数据进行采集(SNMPget/SNMPwalk消息)以及对代理端的配置或者管理(SNMPset消息)。例如, s1, s2, ..., sn: these sn (n=1, ..., N) interfaces are SNMP interfaces, which are both control channels and data channel. Through the standard SNMP protocol, the SNMPDau module can collect the middleware performance parameter data of each agent (SNMPget/SNMPwalk message) and configure or manage the agent (SNMPset message) through these interfaces. For example,
通过SNMPwalk,从代理返回所有的SNMP对象所组成的数组,错误则返回FALSE; Through SNMPwalk, return an array of all SNMP objects from the agent, and return FALSE if an error occurs;
通过SNMPget,从代理获取一个SNMP对象,成功则返回SNMP的对象值,失败则返回FALSE; Obtain an SNMP object from the agent through SNMPget, and return the SNMP object value if successful, and return FALSE if it fails;
通过SNMPset,设置指定的SNMP对象的值,成功返回TRUE,错误则返回FALSE。 Through SNMPset, set the value of the specified SNMP object, return TRUE if successful, and return FALSE if it fails. the
其中图3中还显示了一个SNMPDau.xml的配置文件,用户可以根据该配 置文件实现对该文件的配置,用于初始化安装后定制好一些数据库联接、线程池设置等不轻易修改的设置,以控制SNMPDau模块的运行。SNMPDau.xml中配置的修改可以通过WEB界面的集中配置来实现,SNMPDau模块会定时更新配置。 Figure 3 also shows a SNMPDau.xml configuration file. Users can configure the file according to the configuration file, which is used to customize some settings that are not easy to modify, such as database connections and thread pool settings, after initialization and installation. To control the operation of the SNMPDau module. The modification of the configuration in SNMPDau.xml can be realized through the centralized configuration of the WEB interface, and the SNMPDau module will update the configuration regularly. the
本发明实施例提供的上述方法中,基于SNMP操作任务列表的配置,获取中间件的性能参数数据并对采集到的性能参数数据进行合并处理并保存。与现有的使用命令行方式的中间件监控方式相比,采用了可配置的基于SNMP协议的分布式网络管理模型,与传统的网络管理模型相比,具有更好的灵活性和可扩展性。使用这种系统管理技术和SNMP协议相结合的技术,能灵活获取想要的设备信息,而且安全性高,扩展能力很强。 In the above method provided by the embodiment of the present invention, based on the configuration of the SNMP operation task list, the performance parameter data of the middleware is obtained, and the collected performance parameter data is merged and stored. Compared with the existing middleware monitoring method using the command line, it adopts a configurable distributed network management model based on the SNMP protocol, which has better flexibility and scalability than the traditional network management model . Using this combination of system management technology and SNMP protocol, you can flexibly obtain the desired device information, and it has high security and strong expansion capabilities. the
本发明的实施例还提供了一种网络设备,如图4所示,用于对中间件性能进行监控,该网络设备可以为SNMPDau模块。具体的,该网络设备包括: The embodiment of the present invention also provides a network device, as shown in FIG. 4 , for monitoring the middleware performance. The network device may be an SNMPDau module. Specifically, the network equipment includes:
主控模块10,用于调度配置更新模块10获取配置文件,并接收所述配置更新模块20发送的SNMP操作任务列表; The
配置更新模块20,用于在所述主控模块10的控制下获取配置文件,根据所述配置文件的配置从外部数据库中获取SNMP操作任务列表,并通知所述主控模块10; The
调度模块30,用于根据所述主控模块10中维护的SNMP操作任务列表,获取中间件的性能参数数据;具体的,调度模块30对采集到的性能参数数据进行分析,针对同一类别的数据进行合并;将合并后得到的数据文件通过posix消息队列消息发送给外部的invoker程序进行入库操作。 The
数据整理模块40,用于周期性的检查所述调度模块30采集到的性能参数数据,对采集到的性能参数数据进行合并处理并保存。 The
另外,该网络设备中还包括: In addition, the network equipment also includes:
系统消息处理模块50,用于接收用户发送的系统消息,根据所述系统消息的内容向所述主控模块10发送相应的控制消息; A system
所述主控模块10,还用于根据所述系统消息处理模块50发送的控制消息, 对所述配置更新模块20、调度模块30以及数据整理模块40进行相应的控制。 The
日志模块60,用于为所述主控模块10、配置更新模块20、调度模块30、数据整理模块40以及系统消息处理模块50提供公用的日志功能,生成日志文件并存储。 The
配置更新模块20,还用于当所述配置文件中还包括获取中间件的性能参数数据、以及对采集到的性能参数数据进行合并处理的相关配置时,将相关配置提供给所述调度模块30和数据整理模块40。 The
调度模块10具体用于:
定时检查所述主控模块10中的SNMP操作任务列表,发现有符合条件的SNMP操作任务时,则将其加入线程池并运行;所述线程池中没有空闲线程时,则等待有空闲线程存在后运行;所述线程池为一先入先出的线程池队列。 Regularly check the SNMP operation task list in the
本发明的另一实施例中,由本发明实施例提供的方法中描述的系统功能及接口情况,结合细化SNMPDau内部实现,可以得到SNMPDau中各部分相互间的连接关系、以及与外部设备的连接关系如下图5所示。以下对各个模块的功能分别进行介绍。 In another embodiment of the present invention, the system function and interface situation described in the method provided by the embodiment of the present invention, combined with the internal implementation of refining SNMPDau, can obtain the connection relationship between each part in SNMPDau and the connection with external equipment The relationship is shown in Figure 5 below. The functions of each module are introduced separately below. the
(1)主控模块 (1) Main control module
主控模块为其它所有模块的控制模块,在系统启动后对其它模块进行控制。主控模块作为主线程,在生成各个模块实例后,周期性调度配置更新模块从数据库中获取新的SNMP操作任务列表,并接收配置更新模块发送的SNMP操作任务列表。 The main control module is the control module of all other modules, and controls other modules after the system starts. The main control module acts as the main thread, after generating each module instance, periodically schedules the configuration update module to obtain a new SNMP operation task list from the database, and receives the SNMP operation task list sent by the configuration update module. the
(2)配置更新模块 (2) Configuration update module
配置更新模块根据配置文件中的配置获取数据库(database)的连接方式及采集入库配置文件的位置等信息,然后通过d1接口连接数据库,获取所有可用的采集任务列表。同时,从配置文件中获取所有的配置信息,包括供采集,供入库通知等的配置,并将获取到的配置均保存在内存中,供其它模块通过只读的方式获取这些配置。配置模块可以不是一个独立的线程,而是由主控模块周期性调度。 The configuration update module obtains information such as the connection mode of the database and the location of the configuration file collected into the warehouse according to the configuration in the configuration file, and then connects to the database through the d1 interface to obtain a list of all available collection tasks. At the same time, all configuration information is obtained from the configuration file, including configurations for collection and storage notification, and the obtained configurations are stored in the memory for other modules to obtain these configurations in a read-only manner. The configuration module may not be an independent thread, but is periodically scheduled by the main control module. the
(3)调度模块 (3) Scheduling module
调度模块启动后作为一个单独的调度线程存在,调度模块定时(如每分钟)检查主控模块中维护的SNMP操作任务列表,如果有符合条件的任务,则将其置入自身内部的线程池中,该线程池为一先入先出的线程池队列,如果线程池队列中没有空闲线程,则将新获取到的符合条件的任务放在调度队列中存放,一旦线程池队列中有空闲线程则立即运行。调度模块根据SNMP操作任务列表,向各个代理发送SNMP指令,获取中间件的性能参数数据。之后,对采集到的性能参数数据进行分析,针对同一类别的数据进行合并;将合并后得到的数据文件发送给外部的invoker程序进行入库操作。 After the scheduling module is started, it exists as a separate scheduling thread. The scheduling module regularly (such as every minute) checks the SNMP operation task list maintained in the main control module. If there is a qualified task, it will be placed in its own internal thread pool. , the thread pool is a first-in-first-out thread pool queue. If there is no idle thread in the thread pool queue, the newly acquired eligible tasks will be stored in the scheduling queue. Once there are idle threads in the thread pool queue, they will be immediately run. The scheduling module sends SNMP commands to each agent according to the SNMP operation task list to obtain the performance parameter data of the middleware. Afterwards, the collected performance parameter data is analyzed, and the data of the same category are merged; the merged data files are sent to the external invoker program for storage operation. the
(4)数据整理模块 (4) Data sorting module
数据整理模块作为独立运行的线程,周期性的检查调度模块采集到的原始数据,为提高效率,针对同一类别的数据进行合并后交给invoker程序入库程序处理。此处有两个操作,一是收集原始数据合并成大文件,二是发送posix消息队列消息给invoker程序的常驻模块。数据整理模块可以通过i1接口向invoker程序发送IPC消息,以便invoker程序得悉需要进行入库的文件;并通过f1接口将文件传送给invoker程序,进而由invoker程序统一完成入库操作。 As an independent thread, the data sorting module periodically checks the raw data collected by the scheduling module. In order to improve efficiency, the data of the same category is merged and then handed over to the invoker program for processing. There are two operations here, one is to collect raw data and merge it into a large file, and the other is to send posix message queue messages to the resident module of the invoker program. The data sorting module can send IPC messages to the invoker program through the i1 interface, so that the invoker program can know the files that need to be stored; and transmit the files to the invoker program through the f1 interface, and then the invoker program can complete the storage operation in a unified manner. the
(5)日志模块 (5) Log module
日志模块是系统中所有模块使用的公用服务,该日志可提供本地循环文件组日志,以达到具有最大系统资源占用的目的。另外为了防止日志操作阻塞调用它的线程,该日志服务也作为单独的线程存在,其调用仅仅将日志内容压入内部缓存中,由单独的线程进行写磁盘操作。 The log module is a common service used by all modules in the system. The log can provide a local circular file group log to achieve the purpose of maximizing system resource usage. In addition, in order to prevent the log operation from blocking the thread that calls it, the log service also exists as a separate thread. Its call only pushes the log content into the internal cache, and the separate thread performs the disk operation. the
(6)系统消息处理模块 (6) System message processing module
系统消息处理模块用于处理系统信号,接收用户发送的系统消息,根据所述系统消息的内容向主控模块发送相应的控制消息。如系统TERM信号表示系统要退出了,则通知主控模块需要停止采集程序,该模块作为单独的线程运行,普通情况下均处于阻塞状态。 The system message processing module is used to process system signals, receive system messages sent by users, and send corresponding control messages to the main control module according to the content of the system messages. If the system TERM signal indicates that the system is about to exit, it will notify the main control module that it needs to stop the collection program. This module runs as a separate thread and is in a blocked state under normal circumstances. the
对于图5所示的结构,各个模块间以及与外部设备间交互的控制流的流向示意图如图6中的虚线箭头所示;调度模块与各个代理间交互SNMP流的流向示意图如图6中的实线箭头所示。其中: For the structure shown in Figure 5, the schematic diagram of the flow direction of the control flow between each module and the interaction with external devices is shown in the dotted arrow in Figure 6; the flow diagram of the interactive SNMP flow between the scheduling module and each agent is shown in Figure 6 Indicated by the solid arrow. in:
控制流主要包括:(1)主控模块向配置更新模块、数据整理模块、调度模块、系统消息处理模块以及日志模块发送的控制消息;(2)系统消息处理模块在接收到用户发送的消息时,根据该消息的内容向主控模块发送相应的控制消息;(3)配置更新模块从配置文件中获取到包括配置信息的控制消息。 The control flow mainly includes: (1) the control message sent by the main control module to the configuration update module, data sorting module, scheduling module, system message processing module and log module; (2) when the system message processing module receives the message sent by the user , and send a corresponding control message to the main control module according to the content of the message; (3) The configuration update module obtains the control message including configuration information from the configuration file. the
SNMP流主要包括:调度模块根据获取到的SNMP操作任务列表,向各个代理发送SNMP指令,以获取中间件的性能参数数据。 The SNMP flow mainly includes: the scheduling module sends SNMP commands to each agent according to the obtained SNMP operation task list, so as to obtain the performance parameter data of the middleware. the
对于图5所示的结构,各个模块间以及与外部设备间交互的数据流的流向示意图如图7中的虚线箭头所示;各个模块与外部设备间交互的IPC流的流向示意图如图7中的实线箭头所示;外部设备间交互的外部流的流向示意图如图7中的宽箭头所示。其中: For the structure shown in Figure 5, the schematic diagram of the flow direction of the data flow between each module and the interaction with the external device is shown in the dotted arrow in Figure 7; the flow diagram of the IPC flow between each module and the external device is shown in Figure 7 As shown by the solid line arrows; the schematic diagram of the flow direction of the external flow interacting between external devices is shown by the wide arrows in FIG. 7 . in:
数据流主要包括: The data flow mainly includes:
(1)配置更新模块根据配置文件中的配置通过接口连接数据库,获取所有可用的采集任务列表; (1) The configuration update module connects to the database through the interface according to the configuration in the configuration file, and obtains a list of all available collection tasks;
(2)主控模块从配置更新模块获取SNMP操作任务列表; (2) The main control module obtains the SNMP operation task list from the configuration update module;
(3)调度模块获取主控模块中维护的SNMP操作任务列表;并从各个代理获取获取中间件的性能参数数据; (3) The scheduling module obtains the SNMP operation task list maintained in the main control module; and obtains the performance parameter data of the middleware from each agent;
(4)数据整理模块周期性从调度模块获取由调度模块采集到的原始性能参数数据;并将收集到的原始性能参数数据进行合并,得到数据文件; (4) The data sorting module periodically obtains the original performance parameter data collected by the scheduling module from the scheduling module; and merges the collected original performance parameter data to obtain the data file;
(5)日志模块将日志内容存储到日志文件。 (5) The log module stores the log content to a log file. the
IPC流主要包括: The IPC flow mainly includes:
(1)用户向系统消息处理模块发送的IPC消息,用于对SNMPDau进行控制消息; (1) The IPC message sent by the user to the system message processing module is used to control the SNMPDau message;
(2)数据整理模块向invoker程序发送的IPC消息,以便invoker程序得悉需要进行入库的文件。 (2) The IPC message sent by the data sorting module to the invoker program, so that the invoker program can know the files that need to be stored. the
外部流包括: External streams include:
(1)invoker程序根据数据整理模块发送的IPC消息,从数据文件获取数据; (1) The invoker program obtains data from the data file according to the IPC message sent by the data sorting module;
(2)invoker程序将从数据整理模块获取的数据进行入库操作。 (2) The invoker program puts the data obtained from the data sorting module into the database. the
本发明实施例提供的上述设备中,基于SNMP操作任务列表的配置,获取中间件的性能参数数据并对采集到的性能参数数据进行合并处理并保存。与现有的使用命令行方式的中间件监控方式相比,采用了可配置的基于SNMP协议的分布式网络管理模型,与传统的网络管理模型相比,具有更好的灵活性和可扩展性。使用这种系统管理技术和SNMP协议相结合的技术,能灵活获取想要的设备信息,而且安全性高,扩展能力很强。 In the above-mentioned device provided by the embodiment of the present invention, based on the configuration of the SNMP operation task list, the performance parameter data of the middleware is obtained, and the collected performance parameter data is combined and stored. Compared with the existing middleware monitoring method using the command line, it adopts a configurable distributed network management model based on the SNMP protocol, which has better flexibility and scalability than the traditional network management model . Using this combination of system management technology and SNMP protocol, you can flexibly obtain the desired device information, and it has high security and strong expansion capabilities. the
通过使用本发明的实施例提供的设备,基于SNMP操作任务列表的配置,获取中间件的性能参数数据并对采集到的性能参数数据进行合并处理并保存。与现有的使用命令行方式的中间件监控方式相比,采用了可配置的基于SNMP协议的分布式网络管理模型,与传统的网络管理模型相比,具有更好的灵活性和可扩展性。 By using the device provided by the embodiment of the present invention, based on the configuration of the SNMP operation task list, the performance parameter data of the middleware is obtained, and the collected performance parameter data is combined, processed and saved. Compared with the existing middleware monitoring method using the command line, it adopts a configurable distributed network management model based on the SNMP protocol, which has better flexibility and scalability than the traditional network management model . the
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到本发明可以通过硬件实现,也可以借助软件加必要的通用硬件平台的方式来实现。基于这样的理解,本发明的技术方案可以以软件产品的形式体现出来,该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM,U盘,移动硬盘等)中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本发明各个实施例所述的方法。 Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be realized by hardware, or by software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.), including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present invention. the
本领域技术人员可以理解附图只是一个优选实施例的示意图,附图中的单元或流程并不一定是实施本发明所必须的。 Those skilled in the art can understand that the drawing is only a schematic diagram of a preferred embodiment, and the units or processes in the drawing are not necessarily necessary for implementing the present invention. the
本领域技术人员可以理解实施例中的装置中的单元可以按照实施例描述进行分布于实施例的装置中,也可以进行相应变化位于不同于本实施例的一个或多个装置中。上述实施例的单元可以合并为一个单元,也可以进一步拆分成多个子单元。 Those skilled in the art can understand that the units in the device in the embodiment can be distributed in the device in the embodiment according to the description in the embodiment, and can also be changed and located in one or more devices different from the device in the embodiment. The units in the above embodiments can be combined into one unit, and can also be further divided into multiple sub-units. the
上述本发明实施例序号仅仅为了描述,不代表实施例的优劣。 The serial numbers of the above embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments. the
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910249888CN101719852B (en) | 2009-11-30 | 2009-11-30 | Method and device for monitoring performance of middleware |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910249888CN101719852B (en) | 2009-11-30 | 2009-11-30 | Method and device for monitoring performance of middleware |
| Publication Number | Publication Date |
|---|---|
| CN101719852A CN101719852A (en) | 2010-06-02 |
| CN101719852Btrue CN101719852B (en) | 2012-11-07 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910249888ActiveCN101719852B (en) | 2009-11-30 | 2009-11-30 | Method and device for monitoring performance of middleware |
| Country | Link |
|---|---|
| CN (1) | CN101719852B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102347869B (en)* | 2010-08-02 | 2014-10-08 | 阿里巴巴集团控股有限公司 | Method, device and system for monitoring equipment performance |
| CN103209081A (en)* | 2012-01-11 | 2013-07-17 | 国家电网公司 | Information checking method and device for net-connected computer |
| CN106326249B (en)* | 2015-06-23 | 2021-08-03 | 中兴通讯股份有限公司 | Data integration processing method and device |
| CN107360026B (en)* | 2017-07-07 | 2020-05-19 | 西安电子科技大学 | Performance prediction and modeling method of distributed message middleware |
| CN108536766A (en)* | 2018-03-21 | 2018-09-14 | 四川斐讯信息技术有限公司 | A kind of data storage method and system based on clock timer |
| CN108920322A (en)* | 2018-06-04 | 2018-11-30 | 郑州云海信息技术有限公司 | A kind of method, apparatus and computer storage medium of middleware operational monitoring |
| CN109510678B (en)* | 2018-09-30 | 2020-10-27 | 武汉武钢绿色城市技术发展有限公司 | Method for realizing IP voice broadcast control through middleware |
| CN110768832B (en)* | 2019-10-24 | 2022-07-26 | 中国计量大学 | A method for monitoring information security domain of industrial control system |
| CN113098734B (en)* | 2021-03-31 | 2022-07-08 | 科来网络技术股份有限公司 | Method for dynamically monitoring performance index of network equipment |
| CN114217867A (en)* | 2021-11-26 | 2022-03-22 | 新华三大数据技术有限公司 | Automatic operation and maintenance agent device, equipment and storage medium |
| CN119728379A (en)* | 2024-03-12 | 2025-03-28 | 淮安市新创安智能化工程有限公司 | A network management system based on intelligent evaluation |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1983962A (en)* | 2006-06-08 | 2007-06-20 | 华为技术有限公司 | Extended network managing system and method |
| CN1992635A (en)* | 2005-12-27 | 2007-07-04 | 中兴通讯股份有限公司 | Method for Simulating SNMP Network Element and Using the Network Element to Test Network Management System |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1992635A (en)* | 2005-12-27 | 2007-07-04 | 中兴通讯股份有限公司 | Method for Simulating SNMP Network Element and Using the Network Element to Test Network Management System |
| CN1983962A (en)* | 2006-06-08 | 2007-06-20 | 华为技术有限公司 | Extended network managing system and method |
| Title |
|---|
| 周泽华.消息中间件管理研究.《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》.2002,(第01期),* |
| Publication number | Publication date |
|---|---|
| CN101719852A (en) | 2010-06-02 |
| Publication | Publication Date | Title |
|---|---|---|
| CN101719852B (en) | Method and device for monitoring performance of middleware | |
| US10613903B1 (en) | Stream processing at scale | |
| CN109889575B (en) | A collaborative computing platform system and method in an edge environment | |
| US11110601B2 (en) | Scheduling robots for robotic process automation | |
| CN104618693B (en) | A kind of monitor video based on cloud computing handles task management method and system online | |
| CN111506412A (en) | Distributed asynchronous task construction and scheduling system and method based on Airflow | |
| CN109327351B (en) | Real-time collection method, device, electronic device and storage medium of log data | |
| CN109857558A (en) | A kind of data flow processing method and system | |
| CN113515356B (en) | Lightweight distributed resource management and task scheduler and method | |
| CN106844198A (en) | A distributed scheduling automation test platform and method | |
| US11818152B2 (en) | Modeling topic-based message-oriented middleware within a security system | |
| CN102739452A (en) | Method and system for monitoring resources | |
| US10498817B1 (en) | Performance tuning in distributed computing systems | |
| CN1805379A (en) | Method of constructing cluster management and monitoring system in resilient system architecture | |
| CN113485812B (en) | Partition parallel processing method and system based on large-data-volume task | |
| CN110895486A (en) | Distributed Task Scheduling System | |
| CN115115329A (en) | Manufacturing middleware and cloud manufacturing framework for intelligent production line | |
| CN120186142B (en) | MCP protocol-based cross-environment AI task collaborative execution method, device and storage medium | |
| US10122602B1 (en) | Distributed system infrastructure testing | |
| CN113515363A (en) | Special-shaped task high-concurrency multi-level data processing system dynamic scheduling platform | |
| CN117056403A (en) | Data persistence method and device | |
| EP4024761A1 (en) | Communication method and apparatus for multiple management domains | |
| CN118245331B (en) | Data acquisition method and electronic equipment | |
| CN113672452A (en) | Method and system for monitoring operation of data acquisition task | |
| CN117076508B (en) | Method for supporting batch data processing by stream data processing system |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |