CN115242879A - Data sharing system and method - Google Patents

Abstract

The invention provides a data sharing system and a method, wherein the system comprises: the system comprises a sharing interface module, a data warehouse management module, a data processing module and a visualization management module; the sharing interface module is used for determining a corresponding sharing service interface according to the requirement of the upper application; the data warehouse management module is used for sending an access request of a service requester based on an interaction mode of various custom standardized interfaces and a shared service interface corresponding to an upper application; the data processing module is used for selecting a target server cluster based on a load balancing algorithm and cluster performance weights of different server clusters and returning response data; and providing the visual interface display response data to the service requester. The invention manages the access mode of the system in a unified way through different standardized interfaces, adopts the server cluster, selects the server cluster with matched performance for the access requests of different service requesters with different service types, and improves the expansion performance of the data sharing service.

Description

Translated fromChinese

数据共享系统及方法Data sharing system and method

技术领域technical field

本发明涉及数据共享技术领域，尤其涉及一种数据共享系统及方法。The present invention relates to the technical field of data sharing, in particular to a data sharing system and method.

背景技术Background technique

随着互联网及信息化技术的飞速发展，特别是在大数据背景下，各行各业都有自己的信息应用平台或者大数据平台，数据的汇总和共享需求非常大。With the rapid development of the Internet and information technology, especially in the context of big data, all walks of life have their own information application platforms or big data platforms, and the demand for data aggregation and sharing is very large.

鉴于系统应用的客户对于数据共享的需求多样性及需求变更的频繁性，需要频繁的修改共享方式和共享接口，对于系统平台开发人员，特别是对于运维人员来说来具有一定的难度。In view of the variety of data sharing requirements and frequent changes of requirements of customers of system applications, it is necessary to frequently modify the sharing method and sharing interface, which is difficult for system platform developers, especially for operation and maintenance personnel.

如何有效利用数据共享平台，并帮助运维人员高效地解决客户对于平台在数据高并发共享方面的需求具有十分重要的意义。How to effectively use the data sharing platform and help the operation and maintenance personnel to efficiently solve the customer's demand for high concurrent data sharing on the platform is of great significance.

发明内容Contents of the invention

本发明提供一种数据共享系统及方法，用以解决现有技术中客户对于数据共享的需求多样性及需求变更的频繁性，需要频繁的修改共享方式和共享接口，操作和管理不方便，不能即时按照对应的需求提供合适的数据共享服务。The present invention provides a data sharing system and method, which are used to solve the diversity of customer needs for data sharing and the frequency of demand changes in the prior art. It is necessary to frequently modify the sharing mode and sharing interface, which is inconvenient to operate and manage. Immediately provide appropriate data sharing services according to corresponding needs.

现有的信息应用平台或者数据共享平台，对不同应用的客户的数据共享需求具有的多样性以及需求变更的频繁性，不能提供灵活的支持，需要频繁的修改共享方式和共享接口，实际操作和运维中存在一定的操作难度，而且容易出错。Existing information application platforms or data sharing platforms cannot provide flexible support for the diversity of data sharing requirements of customers with different applications and the frequency of demand changes, and frequent modification of sharing methods and sharing interfaces is required. There is a certain degree of operational difficulty in operation and maintenance, and it is easy to make mistakes.

本发明提供一种数据共享系统，包括：共享接口模块、数据仓库管理模块、数据处理模块和可视化管理模块；The invention provides a data sharing system, comprising: a sharing interface module, a data warehouse management module, a data processing module and a visual management module;

其中，所述共享接口模块用于根据上层应用的需求，确定每个所述上层应用对应的共享服务接口；Wherein, the shared interface module is configured to determine the shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer application;

所述数据仓库管理模块用于基于多种自定义标准化接口的交互方式以及所述上层应用对应的所述共享服务接口，发送服务请求方的访问请求；所述多种自定义标准化接口的交互方式包括消息交互方式、发布订阅方式和数据库访问方式；The data warehouse management module is used to send the access request of the service requester based on the interaction modes of multiple self-defined standardized interfaces and the shared service interface corresponding to the upper layer application; the interaction mode of the various self-defined standardized interfaces Including message interaction method, publish and subscribe method and database access method;

所述数据处理模块用于基于负载均衡算法，以及确定的服务器集群中不同服务器集群的集群性能权值，选择用于处理所述服务请求方的所述访问请求的目标服务器集群，并由所述目标服务器集群返回对应的响应数据；The data processing module is configured to select a target server cluster for processing the access request of the service requester based on a load balancing algorithm and determined cluster performance weights of different server clusters in the server cluster, and the The target server cluster returns the corresponding response data;

所述可视化管理模块用于基于所述数据处理模块获得的所述响应数据，向所述服务请求方提供可视化界面展示所述响应数据。The visualization management module is configured to provide the service requester with a visualization interface to display the response data based on the response data obtained by the data processing module.

根据本发明提供的一种数据共享系统，所述共享接口模块用于根据上层应用的需求，确定每个所述上层应用对应的共享服务接口，包括以下至少一项：According to a data sharing system provided by the present invention, the shared interface module is configured to determine the shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer application, including at least one of the following:

基于第三方应用对应的接入权限，控制所述第三方应用是否接入所述数据共享系统，并提供通道加密机制；Based on the access authority corresponding to the third-party application, control whether the third-party application accesses the data sharing system, and provide a channel encryption mechanism;

或，基于多租户的访问权限以及访问量需求，确定所述多租户中每个租户的可用接口；Or, based on the multi-tenant's access rights and traffic requirements, determine the available interfaces of each tenant in the multi-tenant;

或，基于第三方应用的访问频度，确定是否对所述第三方应用的访问进行限制；Or, based on the access frequency of the third-party application, determine whether to restrict the access of the third-party application;

或，基于访问队列中访问请求的优先级次序以及所述访问请求的连接状态，调整不同优先级的所述访问请求的处理优先级。Or, based on the priority order of the access requests in the access queue and the connection status of the access requests, the processing priorities of the access requests with different priorities are adjusted.

根据本发明提供的一种数据共享系统，所述基于消息交互方式、发布订阅方式和数据库方式中任意一种或多种方式的组合，发送所述访问请求至所述数据处理模块，包括：According to a data sharing system provided by the present invention, sending the access request to the data processing module based on a combination of any one or more of the message interaction mode, publish-subscribe mode and database mode includes:

若确定所述多种自定义标准化接口的交互方式为消息交互方式，则基于所述服务请求方的所述访问请求的数据量、响应时间需求以及对应响应性能指标，确定所述消息交互方式中对应的响应方式为同步响应或异步响应；If it is determined that the interaction mode of the various self-defined standardized interfaces is a message interaction mode, then determine the message interaction mode based on the data volume of the access request of the service requester, the response time requirement, and the corresponding response performance index. The corresponding response mode is synchronous response or asynchronous response;

若确定所述多种自定义标准化接口的交互方式为发布订阅方式，则基于服务请求方的注册请求，获取所述注册请求对应的响应数据；If it is determined that the interaction mode of the various self-defined standardized interfaces is a publish-subscribe mode, then based on the registration request of the service requester, obtain the response data corresponding to the registration request;

若确定所述多种自定义标准化接口的交互方式为数据库方式，则基于ODBC或JDBC访问方式，确定外部系统对数据库的访问接口。If it is determined that the interaction mode of the various self-defined standardized interfaces is the database mode, then the access interface of the external system to the database is determined based on the ODBC or JDBC access mode.

根据本发明提供的一种数据共享系统，所述若确定所述多种自定义标准化接口的交互方式为消息交互方式，则基于所述服务请求方的所述访问请求的数据量、响应时间需求以及对应响应性能指标，确定所述消息交互方式中对应的响应方式为同步响应或异步响应之后，包括：According to a data sharing system provided by the present invention, if it is determined that the interaction mode of the multiple custom standardized interfaces is a message interaction mode, the data volume and response time requirements of the access request of the service requester are determined based on the service requester. and corresponding response performance indicators, after determining that the corresponding response mode in the message interaction mode is a synchronous response or an asynchronous response, including:

若所述消息交互方式中对应的响应方式为同步响应，则所述服务请求方进入阻塞状态等待服务提供方的响应；If the corresponding response mode in the message interaction mode is a synchronous response, the service requester enters a blocking state and waits for a response from the service provider;

若所述消息交互方式中对应的响应方式为异步响应，则所述服务请求方通过轮询或者事件通知确定所述响应数据是否准备完毕。If the corresponding response mode in the message interaction mode is an asynchronous response, the service requester determines whether the response data is ready through polling or event notification.

根据本发明提供的一种数据共享系统，所述若确定所述多种自定义标准化接口的交互方式为发布订阅方式，则基于服务请求方的注册请求，获取所述注册请求对应的响应数据，包括：According to a data sharing system provided by the present invention, if it is determined that the interaction mode of the multiple self-defined standardized interfaces is a publish-subscribe mode, based on the registration request of the service requester, the response data corresponding to the registration request is obtained, include:

所述服务提供方确定所述响应数据的准备状态；The service provider determines the readiness status of the response data;

若服务提供方确定所述响应数据的准备状态为成功，则按照所述注册请求中包括的反馈数据的方式以及反馈周期，周期性地反馈响应数据给所述服务请求方。If the service provider determines that the preparation status of the response data is successful, it periodically feeds back the response data to the service requester according to the feedback data mode and feedback period included in the registration request.

根据本发明提供的一种数据共享系统，所述若所述访问请求用于数据查询，则基于KPI数据查询、数据模型结构查询和DQI数据查询中任意一种，发送所述访问请求至所述数据处理模块，包括：According to a data sharing system provided by the present invention, if the access request is used for data query, the access request is sent to the Data processing module, including:

基于KPI数据查询中的维度属性和查询指标，以及网页服务Web Service/简单对象访问协议SOAP技术，确定与所述KPI数据查询对应数据查询结果；Based on the dimension attributes and query indicators in the KPI data query, and the web service Web Service/Simple Object Access Protocol SOAP technology, determine the data query result corresponding to the KPI data query;

或，基于数据模型结构查询中的数据结构名称，维度指标，以及Web Service/SOAP技术，确定与所述数据模型结构查询对应的数据查询结果；Or, based on the data structure name, dimension index, and Web Service/SOAP technology in the data model structure query, determine the data query result corresponding to the data model structure query;

或，基于DQI数据查询中的维度属性和查询指标，以及Web Service/SOAP技术，确定与所述DQI数据查询对应的数据查询结果。Or, a data query result corresponding to the DQI data query is determined based on the dimension attributes and query indicators in the DQI data query, and the Web Service/SOAP technology.

根据本发明提供的一种数据共享系统，所述数据处理模块用于基于负载均衡算法，以及确定的服务器集群中不同服务器集群的集群性能权值，选择用于处理所述服务请求方的所述访问请求的目标服务器集群，并由所述目标服务器集群返回对应的响应数据，包括：According to a data sharing system provided by the present invention, the data processing module is configured to select the server for processing the service requester based on the load balancing algorithm and the determined cluster performance weights of different server clusters in the server cluster. Access the requested target server cluster, and the target server cluster returns corresponding response data, including:

基于所述服务请求方的访问请求对应的业务类型、数据量或响应性能需求，在所有所述服务集群中，确定一个或多个匹配的待定服务器集群；Based on the service type, data volume or response performance requirements corresponding to the access request of the service requester, among all the service clusters, determine one or more matching pending server clusters;

基于负载均衡算法，以及每个所述待定服务器集群的集群性能权值，选择目标服务集群处理所述服务请求方的访问请求；Based on the load balancing algorithm and the cluster performance weight of each pending server cluster, select a target service cluster to process the access request of the service requester;

发送所述目标服务器集群返回的对应的响应数据至所述服务请求方。sending the corresponding response data returned by the target server cluster to the service requester.

根据本发明提供的一种数据共享系统，所述系统还包括配置管理模块，用于对可视化管理模块中包括的所有可显示单元进行客制化配置；其中所述所有可显示单元包括服务地图、订阅管理、服务管理、审核管理和共享日志管理。According to a data sharing system provided by the present invention, the system further includes a configuration management module for customizing the configuration of all displayable units included in the visualization management module; wherein all displayable units include a service map, Subscription management, service management, audit management, and shared log management.

本发明提供一种数据共享方法，包括：The present invention provides a data sharing method, comprising:

根据上层应用的需求，确定每个所述上层应用对应的共享服务接口；Determine the shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer application;

基于多种自定义标准化接口的交互方式以及所述上层应用对应的所述共享服务接口，发送服务请求方的访问请求；所述多种自定义标准化接口的交互方式包括消息交互方式、发布订阅方式和数据库访问方式；Based on the interaction modes of multiple self-defined standardized interfaces and the shared service interface corresponding to the upper-layer application, the access request of the service requester is sent; the interaction modes of the various self-defined standardized interfaces include message interaction mode, publish and subscribe mode and database access methods;

基于负载均衡算法，以及确定的服务器集群中不同服务器集群的集群性能权值，选择用于处理所述服务请求方的所述访问请求的目标服务器集群，并由所述目标服务器集群返回对应的响应数据；Based on the load balancing algorithm and the determined cluster performance weights of different server clusters in the server cluster, select a target server cluster for processing the access request of the service requester, and return a corresponding response by the target server cluster data;

基于所述数据处理模块获得的所述响应数据，向所述服务请求方提供可视化界面展示所述响应数据。Based on the response data obtained by the data processing module, a visual interface is provided to the service requester to display the response data.

本发明提供的数据共享系统及方法，通过不同的标准化接口，对外界系统访问该数据共享系统的方式进行统一的管理，采用服务器集群，对不同业务类型的不同服务请求方的访问请求选择性能匹配的服务器集群，提高本数据共享系统的数据共享服务的扩展性能。在保障数据安全和可靠性的前提下，更灵活的实现对数据的共享。The data sharing system and method provided by the present invention, through different standardized interfaces, conducts unified management on the ways in which external systems access the data sharing system, adopts server clusters, and selects performance matching for access requests of different service requesters of different business types The server cluster improves the expansion performance of the data sharing service of the data sharing system. On the premise of ensuring data security and reliability, more flexible realization of data sharing.

附图说明Description of drawings

为了更清楚地说明本发明或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the present invention or the technical solutions in the prior art, the accompanying drawings that need to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are the present invention. For some embodiments of the invention, those skilled in the art can also obtain other drawings based on these drawings without creative effort.

图1是本发明提供的数据共享系统的结构示意图；Fig. 1 is a schematic structural diagram of a data sharing system provided by the present invention;

图2是本发明提供的在线请求与同步响应方式交互示意图；Fig. 2 is a schematic diagram of online request and synchronous response mode interaction provided by the present invention;

图3是本发明提供的在线请求与异步响应方式交互示意图之一；Fig. 3 is one of the interactive schematic diagrams of online request and asynchronous response provided by the present invention;

图4是本发明提供的在线请求与异步响应方式交互示意图之二；Fig. 4 is the second schematic diagram of the interaction between online request and asynchronous response provided by the present invention;

图5是本发明提供的发布订阅方式交互示意图；Fig. 5 is the interactive schematic diagram of the publish-subscribe mode provided by the present invention;

图6是本发明提供的数据共享方法的流程示意图。Fig. 6 is a schematic flowchart of the data sharing method provided by the present invention.

具体实施方式Detailed ways

为使本发明的目的、技术方案和优点更加清楚，下面将结合本发明中的附图，对本发明中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention , but not all examples. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

下面结合图1-图6描述本发明的数据共享系统及方法。The data sharing system and method of the present invention will be described below with reference to FIGS. 1-6 .

图1是本发明提供的数据共享系统的结构示意图；如图1所示，所述数据共享系统100包括：共享接口模块110、数据仓库管理模块120、数据处理模块130和可视化管理模块140；1 is a schematic structural diagram of a data sharing system provided by the present invention; as shown in FIG. 1 , thedata sharing system 100 includes: a sharinginterface module 110, a datawarehouse management module 120, adata processing module 130 and avisualization management module 140;

其中，所述共享接口模块用于根据上层应用的需求，确定每个所述上层应用对应的共享服务接口；Wherein, the shared interface module is configured to determine the shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer application;

所述数据仓库管理模块用于基于多种自定义标准化接口的交互方式以及所述上层应用对应的所述共享服务接口，发送服务请求方的访问请求；所述多种自定义标准化接口的交互方式包括消息交互方式、发布订阅方式和数据库访问方式；The data warehouse management module is used to send the access request of the service requester based on the interaction modes of multiple self-defined standardized interfaces and the shared service interface corresponding to the upper layer application; the interaction mode of the various self-defined standardized interfaces Including message interaction method, publish and subscribe method and database access method;

所述数据处理模块用于基于负载均衡算法，以及确定的服务器集群中不同服务器集群的集群性能权值，选择用于处理所述服务请求方的所述访问请求的目标服务器集群，并由所述目标服务器集群返回对应的响应数据；The data processing module is configured to select a target server cluster for processing the access request of the service requester based on a load balancing algorithm and determined cluster performance weights of different server clusters in the server cluster, and the The target server cluster returns the corresponding response data;

所述可视化管理模块基于所述数据处理模块获得的所述响应数据，向所述服务请求方提供可视化界面展示所述响应数据。The visualization management module provides the service requester with a visualization interface to display the response data based on the response data obtained by the data processing module.

具体来说，本发明提供的数据共享系统，可面向不同场景，提供差异化的数据共享。针对不同厂商提供的多种应用，主要可通过三类差异化共享方式为上层应用提供数据支撑。上述三类差异化共享方式主要包括：实时KPI计算类，实时细节查询类，分析类非实时统计。其中，实时KPI计算类主要是通过实时信令KPI，数据质量预警和准时统计三方面对上层应用提供数据支撑；实时细节查询类主要是通过号码查询，用户记录查询，投诉故障查询三方面对上层应用提供数据支撑；分析类非实时统计主要是通过指标获取呈现，细节数据获取，使用数据共享系统计算/存储资源的应用对上层应用提供数据支撑。Specifically, the data sharing system provided by the present invention can provide differentiated data sharing for different scenarios. For various applications provided by different manufacturers, data support can be provided for upper-layer applications mainly through three types of differentiated sharing methods. The above three types of differentiated sharing methods mainly include: real-time KPI calculation, real-time detail query, and analysis non-real-time statistics. Among them, the real-time KPI calculation class mainly provides data support for upper-layer applications through real-time signaling KPI, data quality early warning and punctual statistics; the real-time detail query class mainly provides data support for upper-layer applications through number query, user record query, and complaint fault query. Applications provide data support; analytical non-real-time statistics are mainly presented through the acquisition of indicators, detailed data acquisition, and applications that use computing/storage resources of the data sharing system to provide data support for upper-level applications.

且通过实时KPI计算提供数据支撑的业务请求或上层应用需求，主要具有如下特点：对数据实时性要求较高，共享数据量较小，交互过程通常为单向，数据获取频率为实时。Moreover, business requests or upper-layer application requirements that provide data support through real-time KPI calculations mainly have the following characteristics: high requirements for real-time data, small amount of shared data, usually one-way interaction process, and real-time data acquisition frequency.

通过实时细节查询类提供数据支撑的业务请求或上层应用需求，主要具有如下特点：对数据实时性要求较低，共享数据量较小，交互过程通常为双向，数据获取频率为按需。The business requests or upper-layer application requirements that provide data support through the real-time detailed query class mainly have the following characteristics: low requirements for real-time data, small amount of shared data, usually two-way interaction process, and data acquisition frequency on demand.

通过分析类非实时统计提供数据支撑的业务请求或上层应用需求，主要具有如下特点：对数据实时性要求较低，共享数据量大小均可，交互过程通常为双向，数据获取频率为定时。Business requests or upper-layer application requirements that provide data support through analytical non-real-time statistics mainly have the following characteristics: low requirements for real-time data, the amount of shared data can be large or small, the interaction process is usually two-way, and the frequency of data acquisition is regular.

而上述三类差异化共享方式，则是通过数据共享系统中对应的共享服务接口模块根据上层应用的需求确定的，具体的上层应用的需求可以是满足上述时KPI计算类，实时细节查询类，分析类非实时统计提供数据支撑的业务请求或上层应用需求的特点中的任意一类；确定对应的共享服务接口。The above three types of differentiated sharing methods are determined through the corresponding sharing service interface module in the data sharing system according to the requirements of upper-layer applications. The specific requirements of upper-layer applications can be to meet the above-mentioned KPI calculation category, real-time detail query category, Analytical non-real-time statistics provide any type of data-supported business requests or characteristics of upper-layer application requirements; determine the corresponding shared service interface.

可选的，所述共享接口模块用于根据上层应用的需求，确定每个所述上层应用对应的共享服务接口，包括以下至少一项：Optionally, the shared interface module is configured to determine a shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer application, including at least one of the following:

基于第三方应用对应的接入权限，控制所述第三方应用是否接入所述数据共享系统，并提供通道加密机制；Based on the access authority corresponding to the third-party application, control whether the third-party application accesses the data sharing system, and provide a channel encryption mechanism;

或，基于多租户的访问权限以及访问量需求，确定所述多租户中每个租户的可用接口；Or, based on the multi-tenant's access rights and traffic requirements, determine the available interfaces of each tenant in the multi-tenant;

或，基于第三方应用的访问频度，确定是否对所述第三方应用的访问进行限制；Or, based on the access frequency of the third-party application, determine whether to restrict the access of the third-party application;

或，基于访问队列中访问请求的优先级次序以及所述访问请求的连接状态，调整不同优先级的所述访问请求的处理优先级。Or, based on the priority order of the access requests in the access queue and the connection status of the access requests, the processing priorities of the access requests with different priorities are adjusted.

具体地，通过对上层应用需求的特点，确定对应的共享服务接口后，还需要确定该上层应用需求对应的权限、访问频度以及访问量等，确定是否可通过确定的共享服务接口对共享数据进行访问，进而对上述共享服务接口统一管理。Specifically, after determining the corresponding shared service interface based on the characteristics of the upper-layer application requirements, it is also necessary to determine the permissions, access frequency, and access volume corresponding to the upper-layer application requirements, and determine whether the shared data can be accessed through the determined shared service interface. access, and then manage the above-mentioned shared service interface in a unified manner.

比如，获取第三方应用对应的接入权限，确定该第三应用是否具备访问该数据共享系统的权限，从而实现允许具备接入权限的第三方应用才能访问该数据共享系统，而不具备访问该数据共享系统的接入权限的第三方应用，则不允许接入。并根据不同的权值设定不同的加密机制，灵活的实现对数据的保密性。For example, obtain the access permission corresponding to the third-party application, and determine whether the third application has the permission to access the data sharing system, so as to allow the third-party application with the access permission to access the data sharing system, and not have access to the data sharing system. Third-party applications that have access rights to the data sharing system are not allowed to access. And set different encryption mechanisms according to different weights to flexibly realize the confidentiality of data.

或者，存在多个租户租用该数据共享系统中部分服务器，这些租户对不同的服务器具有不同的访问权限，通过对每个租户的访问权限动态设置，可以独立启动或停止某个租户的服务，相互不影响。而且可以按照租户(上层应用)的需求动态分配和回收所需资源，包括CPU计算资源、I/O带宽、内存、并行度、空间大小、时间窗口；其中I/O带宽、CPU、内存的管理能力尤为重要。还可以根据各租户的服务级别协议(Service Level Agreement，SLA)进行资源的动态供给。这里的SLA级别可以包括：不同时间，不同应用，不同作业，不同用户等。Or, there are multiple tenants renting some servers in the data sharing system. These tenants have different access rights to different servers. By dynamically setting the access rights of each tenant, a tenant's service can be started or stopped independently, and mutual does not affect. Moreover, the required resources can be dynamically allocated and reclaimed according to the needs of tenants (upper-layer applications), including CPU computing resources, I/O bandwidth, memory, parallelism, space size, and time window; among them, the management of I/O bandwidth, CPU, and memory Ability is especially important. Resources can also be dynamically provisioned according to a service level agreement (Service Level Agreement, SLA) of each tenant. The SLA level here can include: different time, different applications, different jobs, different users, etc.

或者，根据第三方应用的访问频度，确定是否为恶意攻击，如果确定为恶意攻击，则阻止该第三方应用对数据共享系统的访问，确保数据仓库以及服务器的性能和安全。比如说访问频度大于预设阈值，则判断为恶意攻击，预设阈值可以根据实际应用中的统计值得到，或者根据经验设定。Or, according to the access frequency of the third-party application, determine whether it is a malicious attack, and if it is determined to be a malicious attack, prevent the third-party application from accessing the data sharing system to ensure the performance and security of the data warehouse and the server. For example, if the access frequency is greater than a preset threshold, it is determined as a malicious attack, and the preset threshold can be obtained according to statistical values in practical applications, or set according to experience.

或者，通过设备不同的访问队列，保存不同优先级的访问请求，并监控每个访问队列中访问请求的连接状态，保证数据共享系统在高负荷下，满足对应的访问请求的处理时延要求。若某个访问请求连接超时，则释放对应的服务器资源。Or, store access requests with different priorities through different access queues of the device, and monitor the connection status of access requests in each access queue to ensure that the data sharing system meets the processing delay requirements of corresponding access requests under high load. If an access request connection times out, the corresponding server resources are released.

访问量需求是单位时间内并发的访问请求的数量，可以支持至少50个并发处理；在采用文件方式进行交互时，对于数据量较大的文件(比如大于10M)可以进行压缩后再传送。The access requirement is the number of concurrent access requests per unit time, which can support at least 50 concurrent processing; when using file mode for interaction, files with large data volume (for example, greater than 10M) can be compressed and then transmitted.

可选地，所述数据仓库管理模块用于基于多种自定义标准化接口的交互方式以及所述上层应用对应的所述共享服务接口，发送服务请求方的访问请求，包括：Optionally, the data warehouse management module is configured to send the access request of the service requester based on the interaction modes of various self-defined standardized interfaces and the shared service interface corresponding to the upper-layer application, including:

确定所述访问请求用于数据共享或数据查询；determine that the access request is for data sharing or data query;

若所述访问请求用于数据共享，则基于消息交互方式、发布订阅方式和数据库方式中任意一种或多种方式的组合，发送所述访问请求至所述数据处理模块；If the access request is for data sharing, send the access request to the data processing module based on any one or a combination of message interaction, publish-subscribe, and database;

若所述访问请求用于数据查询，则基于KPI数据查询、数据模型结构查询和DQI数据查询中任意一种，发送所述访问请求至所述数据处理模块。If the access request is used for data query, the access request is sent to the data processing module based on any one of KPI data query, data model structure query and DQI data query.

具体地，本发明提供的数据共享系统，为了保证数据共享服务及接口的通用性、易用性、一致性与稳定性等要求。通过自定义标准化接口和接口协议，保证任何需要获取数据的外部应用，只需要遵循数据共享接口规范要求，均能通过数据共享系统方便获取数据。Specifically, the data sharing system provided by the present invention is to ensure the universality, ease of use, consistency and stability of data sharing services and interfaces. By customizing the standardized interface and interface protocol, it is guaranteed that any external application that needs to obtain data can easily obtain data through the data sharing system only by following the requirements of the data sharing interface specification.

其中，自定义标准化接口和接口协议包括：消息队列、网页服务Web Service、文件传输协议(File Transfer Protocol，FTP)、数据库访问接口和套接字Socket等。消息队列可分为主消息队列，多个从消息队列，当前正在处理的消息队列为主消息队列，其中包括的访问请求的优先级更高。且主消息队列和从消息队列之间可以相互转化。也就是当前时刻的主消息队列可能是下个时刻的从消息队列。Among them, the self-defined standardized interface and interface protocol include: message queue, web service Web Service, file transfer protocol (File Transfer Protocol, FTP), database access interface and socket socket, etc. The message queue can be divided into a main message queue and multiple slave message queues. The message queue currently being processed is the main message queue, and the priority of the access request included in it is higher. And the main message queue and the slave message queue can be converted to each other. That is, the main message queue at the current moment may be the slave message queue at the next moment.

并对应的接口协议满足如下条件：And the corresponding interface protocol meets the following conditions:

整个接口消息基于XML语言，符合XML V1.0(2nd Edition or Higher)规范和XMLSchema(1st Edition or Higher)规范。The entire interface message is based on the XML language, conforming to the XML V1.0 (2nd Edition or Higher) specification and the XMLSchema (1st Edition or Higher) specification.

接口实现符合简单对象访问协议(Simple Object Access Protocol，SOAP)V1.1版本规范，高版本SOAP协议保证与1.1版本的兼容性，接口描述使用网页服务描述语言(WebServices Description Language，WSDL)。The interface implementation conforms to the Simple Object Access Protocol (Simple Object Access Protocol, SOAP) V1.1 version specification, and the high-version SOAP protocol guarantees compatibility with version 1.1. The interface description uses Web Services Description Language (WSDL).

接口实现至少支持SOAP在超文本传输协议(Hyper Text Transfer Protocol，HTTP)V1.0协议上的传输。The interface implementation supports at least the transmission of SOAP over the Hyper Text Transfer Protocol (HTTP) V1.0 protocol.

接口实现符合WSDL语言V1.1版本规范，高版本WSDL语言保证与1.1版本的兼容性。The interface implementation conforms to the WSDL language V1.1 version specification, and the higher version WSDL language guarantees compatibility with version 1.1.

接口描述支持WSDL在SOAP V1.1协议上的绑定。The interface description supports the binding of WSDL on the SOAP V1.1 protocol.

而针对自定义标准化接口的交互方式，本发明提出了三种形式：消息交互方式、发布订阅方式和数据库方式。For the interaction mode of the self-defined standardized interface, the present invention proposes three forms: a message interaction mode, a publish-subscribe mode and a database mode.

消息交互方式是服务请求方与服务提供方进行数据交互的方式。消息交互方式按照数据交互的方式分为“在线请求与同步响应方式”、“在线请求与异步响应方式”两种，分别对应不同的数据量和响应性能指标。通常用于单次获取共享数据。The message interaction mode is the way of data interaction between the service requester and the service provider. The message interaction mode is divided into two types according to the data interaction mode: "online request and synchronous response mode" and "online request and asynchronous response mode", which correspond to different data volumes and response performance indicators. Typically used for a single fetch of shared data.

发布订阅方式主要是服务请求方可以周期性的获取订阅数据。The publish-subscribe method mainly enables the service requester to obtain subscription data periodically.

数据库方式只要是针对不同类型的数据库，可能采用不同的数据库连接协议，而本发明中通过通用的连接协议，避免了直接暴露数据库的物理模型，提高数据共享系统的耦合性。As long as the database mode is aimed at different types of databases, different database connection protocols may be used, and the general connection protocol in the present invention avoids directly exposing the physical model of the database and improves the coupling of the data sharing system.

上述说明，主要是通过上述三种标准化接口的交互方式，对外部系统提供数据共享，当然也可以对服务请求方的访问请求数据进行采集并存储，分析等操作。The above description mainly provides data sharing to external systems through the interaction of the above three standardized interfaces. Of course, it is also possible to collect, store, and analyze the access request data of the service requester.

任何需要共享的系统内部数据均可在不影响系统数据架构的前提下实现数据共享，共享数据与系统内部数据应采用的松耦合方式设计。系统应具备完善的数据抽象和封装能力，内部数据可被封装为面向应用的数据共享服务。共享数据与内部数据完全分离，共享数据模型的变更不会引起的内部数据模型变更。Any system internal data that needs to be shared can realize data sharing without affecting the system data architecture. The shared data and the internal data of the system should be designed in a loosely coupled manner. The system should have perfect data abstraction and encapsulation capabilities, and internal data can be encapsulated as application-oriented data sharing services. Shared data is completely separated from internal data, and changes to the shared data model will not cause changes to the internal data model.

此外，数据共享系统中的数据仓库管理模块还可以对外部系统提供数据查询功能。具体地，本发明中数据仓库管理模块采用基于Web Service/简单对象访问协议(SimpleObject Access Protocol，SOAP)技术实现外部系统数据查询需求。In addition, the data warehouse management module in the data sharing system can also provide data query functions to external systems. Specifically, in the present invention, the data warehouse management module adopts the technology based on Web Service/Simple Object Access Protocol (SOAP) to realize the data query requirement of the external system.

并确定元数据，也就是最小的数据单元，通过统一的最小的数据单元，有利于在不同的平台或者应用之间采用统一的方式进行处理。通过元数据对共享数据的格式进行定义。实现上层应用和第三方应用的服务请求方在不清楚数据共享系统中数据组织结构的情况下，能够根据数据内容及结构的描述信息访问数据。And determine the metadata, that is, the smallest data unit, through the unified smallest data unit, it is beneficial to use a unified way to process between different platforms or applications. The format of shared data is defined by metadata. The service requester implementing the upper-layer application and the third-party application can access the data according to the description information of the data content and structure without knowing the data organization structure in the data sharing system.

通过预设的三种查询方式，提供数据的查询结果。该预设的三种查询方式为：关键绩效指标(Key Performance Indicator，KPI)数据查询、数据模型结构查询和设计质量指标(Design Quality Indicator，DQI)数据查询。Provide data query results through three preset query methods. The preset three query methods are: key performance indicator (Key Performance Indicator, KPI) data query, data model structure query and design quality indicator (Design Quality Indicator, DQI) data query.

可选地，所述基于消息交互方式、发布订阅方式和数据库方式中任意一种或多种方式的组合，发送所述访问请求至所述数据处理模块，包括：Optionally, sending the access request to the data processing module based on any one or a combination of message interaction, publish-subscribe, and database methods includes:

若确定所述多种自定义标准化接口为消息交互方式，则基于所述服务请求方的所述访问请求的数据量、响应时间需求以及对应响应性能指标，确定所述消息交互方式中对应的响应方式为同步响应或异步响应；If it is determined that the multiple self-defined standardized interfaces are the message interaction mode, then determine the corresponding response in the message interaction mode based on the data volume of the access request of the service requester, the response time requirement and the corresponding response performance index The method is synchronous response or asynchronous response;

若确定所述多种自定义标准化接口为发布订阅方式，则基于服务请求方的注册请求，获取所述注册请求对应的响应数据；If it is determined that the multiple custom standardized interfaces are published and subscribed, then based on the registration request of the service requester, the response data corresponding to the registration request is obtained;

若确定所述多种自定义标准化接口为数据库方式，则基于ODBC或JDBC访问方式，确定外部系统对数据库的访问接口。If it is determined that the multiple self-defined standardized interfaces are database modes, the access interfaces of the external system to the database are determined based on the ODBC or JDBC access modes.

具体地，上述多种自定义标准化接口的交互方式包括消息交互方式、发布订阅方式和数据库方式等，对应的不同的自定义标准化接口，对应不同的处理流程。Specifically, the interaction modes of the above-mentioned various custom standardized interfaces include message interaction mode, publish-subscribe mode, database mode, etc., corresponding to different custom standardized interfaces, corresponding to different processing flows.

比如多种自定义标准化接口的交互方式为消息交互方式，则基于所述服务请求方的所述访问请求的数据量、响应时间需求以及对应响应性能指标，确定所述消息交互方式中对应的响应方式为同步响应或异步响应。比如访问请求的数据量小，响应时间需求较短，以及响应性能指标比较高，一般采用同步响应的方式；若访问请求的数据量大，响应性能指标比较低，则一般采用异步响应的方式。For example, the interaction mode of multiple self-defined standardized interfaces is a message interaction mode, and the corresponding response in the message interaction mode is determined based on the data volume of the access request of the service requester, the response time requirement, and the corresponding response performance index The mode is synchronous response or asynchronous response. For example, if the data volume of the access request is small, the response time requirement is short, and the response performance index is relatively high, the synchronous response method is generally adopted; if the data volume of the access request is large and the response performance index is relatively low, the asynchronous response method is generally adopted.

或，确定所述多种自定义标准化接口的交互方式为发布订阅方式，那么基于服务请求方的注册请求，获取所述注册请求对应的响应数据；比如所述注册请求中包括反馈数据的方式，则服务提供方确认自身是否支持该反馈数据的方式，如果支持，则按照所述注册请求，返回满足上述反馈数据的方式的响应数据。Or, if it is determined that the interaction mode of the various self-defined standardized interfaces is a publish-subscribe mode, then based on the registration request of the service requester, the response data corresponding to the registration request is obtained; for example, the registration request includes feedback data, Then the service provider confirms whether it supports the way of the feedback data, and if so, returns the response data satisfying the way of the feedback data according to the registration request.

若确定所述多种自定义标准化接口的交互方式为数据库方式，则基于开放数据库连接(Open Database Connectivity，ODBC)或Java数据库连接(Java DatabaseConnectivity，JDBC)访问方式，确定外部系统对数据库的访问接口。If it is determined that the interactive mode of the multiple self-defined standardized interfaces is a database mode, then based on an open database connection (Open Database Connectivity, ODBC) or a Java database connection (Java Database Connectivity, JDBC) access mode, determine the access interface of the external system to the database .

确定服务请求方和服务提供方的标准化接口的交互方式为数据库方式后，可通过标准的数据库连接协议ODBC或JDBC，对外部系统提供统一的访问方式，避免将数据库的物理模型暴露，造成第三方的攻击和篡改。因为数据库物理模型的变化会对上层应用造成直接影响，采用本发明提供的标准数据库连接协议ODBC或JDBC，可以提高数据共享系统的耦合度。After determining that the interaction mode of the standardized interface between the service requester and the service provider is the database mode, the standard database connection protocol ODBC or JDBC can be used to provide a unified access mode to the external system, so as to avoid exposing the physical model of the database and causing third parties attack and tampering. Because the change of the physical model of the database will directly affect the upper-layer application, using the standard database connection protocol ODBC or JDBC provided by the present invention can improve the coupling degree of the data sharing system.

可选地，所述若确定所述多种自定义标准化接口为消息交互方式，则基于所述服务请求方的所述访问请求的数据量、响应时间需求以及对应响应性能指标，确定所述消息交互方式中对应的响应方式为同步响应或异步响应之后，包括：Optionally, if it is determined that the various self-defined standardized interfaces are message interaction methods, then the message is determined based on the data volume of the access request of the service requester, the response time requirement, and the corresponding response performance index. The corresponding response mode in the interactive mode is after synchronous response or asynchronous response, including:

若所述消息交互方式中对应的响应方式为同步响应，则所述服务请求方进入阻塞状态等待服务提供方的响应；If the corresponding response mode in the message interaction mode is a synchronous response, the service requester enters a blocking state and waits for a response from the service provider;

若所述消息交互方式中对应的响应方式为异步响应，则所述服务请求方通过轮询或者事件通知确定所述响应数据是否准备完毕。If the corresponding response mode in the message interaction mode is an asynchronous response, the service requester determines whether the response data is ready through polling or event notification.

具体地，在服务请求方和服务提供方的标准化接口的交互方式为消息交互方式的情况下，需要确定消息交互方式为“在线请求与同步响应方式”或“在线请求与异步响应方式”。Specifically, when the interaction mode of the standardized interface between the service requester and the service provider is the message interaction mode, it is necessary to determine the message interaction mode as "online request and synchronous response mode" or "online request and asynchronous response mode".

通常根据服务请求方的访问请求的数据量、响应时间需求以及对应响应性能指标，确定所述消息交互方式中对应的响应方式为同步响应或异步响应。其中，在线请求与同步响应方式适用于获取响应数据较快、数据量较小且响应性能较高的数据共享服务。在线请求与异步响应方式适用于准备响应数据较慢、数据量巨大或响应性能较低的数据共享服务。Generally, according to the data volume of the access request of the service requester, the response time requirement, and the corresponding response performance index, it is determined that the corresponding response mode in the message interaction mode is a synchronous response or an asynchronous response. Among them, the online request and synchronous response methods are suitable for data sharing services that acquire response data quickly, have a small amount of data, and have high response performance. The online request and asynchronous response methods are suitable for data sharing services that are slow to prepare response data, have a huge amount of data, or have low response performance.

图2是本发明提供的在线请求与同步响应方式交互示意图，如图2所示，服务请求方向服务提供方发送数据请求，并进入阻塞状态，等待服务提供方的响应数据返回。服务提供方在接收到数据请求后，进行一系列的业务处理获取响应数据，并将响应数据回复给服务请求方。服务请求方在接收到响应数据后，中止阻塞状态继续运行。Fig. 2 is a schematic diagram of the interaction between online request and synchronous response provided by the present invention. As shown in Fig. 2, the service requester sends a data request to the service provider, enters a blocking state, and waits for the response data from the service provider to return. After receiving the data request, the service provider performs a series of business processes to obtain the response data, and replies the response data to the service requester. After receiving the response data, the service requester stops the blocking state and continues to run.

在线请求与同步响应方式适用于获取响应数据较快、数据量较小且响应性能较高的数据共享服务。由于发送数据请求的服务请求方需要阻塞运行，因此如果服务提供方获取数据的速度比较慢或者响应的数据量巨大，会造成服务请求方长时间处于阻塞状态，影响其正常运行。The online request and synchronous response methods are suitable for data sharing services that acquire response data quickly, with a small amount of data and high response performance. Since the service requester that sends the data request needs to block the operation, if the service provider obtains data slowly or responds with a large amount of data, the service requester will be blocked for a long time, affecting its normal operation.

虽然可以通过服务响应超时机制来控制长时间等待响应问题，但是依然应该避免在获取数据缓慢或返回数据量巨大的数据共享服务中使用在线请求与同步响应方式来进行数据交互。Although the problem of waiting for a response for a long time can be controlled by the service response timeout mechanism, it is still necessary to avoid using online request and synchronous response for data interaction in data sharing services that acquire data slowly or return a large amount of data.

图3是本发明提供的在线请求与异步响应方式交互示意图之一，如图3所示，服务请求方调用服务提供方的“准备数据方法”，发送数据请求。服务提供方在接收到数据请求后，返回请求成功标志、数据准备的预估处理时间、建议定期查询数据准备情况的时间间隔和数据的获取方式等消息。服务请求方在获得上述消息后可继续运行，不需要长时间处于阻塞状态。FIG. 3 is one of the schematic diagrams of the interaction between the online request and the asynchronous response provided by the present invention. As shown in FIG. 3 , the service requester invokes the "prepare data method" of the service provider to send a data request. After receiving the data request, the service provider returns messages such as the request success flag, the estimated processing time for data preparation, the recommended time interval for regularly querying the data preparation status, and the data acquisition method. The service requester can continue to run after obtaining the above message, and does not need to be in a blocking state for a long time.

服务提供方在返回上述消息后进行一系列的业务处理以准备数据。After returning the above message, the service provider performs a series of business processes to prepare data.

服务请求方在数据准备的预估处理时间到达后，按照建议的定期查询数据准备情况的时间间隔，向服务提供方的“查询数据准备情况”发送查询数据准备情况的要求。如果数据未准备完毕，服务提供方返回数据未准备完毕的消息，服务请求方则在建议的时间间隔后继续查询；如果数据准备完毕，服务提供方返回包含何时何地如何获取数据的消息。After the estimated processing time for data preparation arrives, the service requester sends a request for querying the data preparation status to the service provider's "Query Data Preparation Status" according to the recommended time interval for regular query data preparation status. If the data is not ready, the service provider returns a message that the data is not ready, and the service requester continues to query after the suggested time interval; if the data is ready, the service provider returns a message containing when and how to obtain the data.

服务请求方即可以按照给定的时间到给定的地点(如FTP或HTTP地址)在给定的有效期内自行决定何时使用给定的方式(如FTP或HTTP协议)取得数据。The service requester can then go to a given location (such as FTP or HTTP address) at a given time and within a given validity period to decide when to use a given method (such as FTP or HTTP protocol) to obtain data.

图4是本发明提供的在线请求与异步响应方式交互示意图之二，如图4所示，服务请求方调用服务提供方的“准备数据方法”，发送数据请求。服务提供方在接收到数据请求后，返回请求成功标志。Fig. 4 is the second schematic diagram of interaction between online request and asynchronous response provided by the present invention. As shown in Fig. 4, the service requester invokes the "data preparation method" of the service provider to send a data request. After receiving the data request, the service provider returns the request success flag.

服务请求方在获得上述消息后可继续运行，不需要长时间处于阻塞状态。The service requester can continue to run after obtaining the above message, and does not need to be in a blocking state for a long time.

服务提供方在返回上述消息后进行一系列的业务处理以准备数据。After returning the above message, the service provider performs a series of business processes to prepare the data.

服务提供方在数据准备好后，通知服务请求方。The service provider notifies the service requester after the data is ready.

服务请求方收到数据准备好的通知后，向服务提供方发起数据获取请求，并进一步获取数据。After receiving the notification that the data is ready, the service requester initiates a data acquisition request to the service provider and further acquires the data.

在线请求与异步响应方式适用于准备响应数据较慢、数据量巨大或响应性能较低的数据共享服务。使用在线请求与异步响应方式，服务请求方不需要长时间的等待与阻塞，可正常运行其他功能。对于大数据量的数据交互，服务提供方可指定专用的文件传输协议，如FTP或HTTP协议进行传输。The online request and asynchronous response methods are suitable for data sharing services with slow response data, huge data volume, or low response performance. Using the online request and asynchronous response method, the service requester does not need to wait and block for a long time, and can run other functions normally. For data interaction with a large amount of data, the service provider can specify a dedicated file transfer protocol, such as FTP or HTTP protocol for transmission.

数据请求应答类接口工作方式为：服务提供方需在接收到请求数据文件后提供应答文件的时间不应超过24小时。该方式主要适用于服务请求方具有具体的数据筛选要求，服务提供方根据筛选要求提供数据的情况。The working mode of the data request response interface is as follows: the time for the service provider to provide the response file after receiving the request data file should not exceed 24 hours. This method is mainly applicable to the situation where the service requester has specific data screening requirements and the service provider provides data according to the screening requirements.

可选地，所述若确定所述多种自定义标准化接口的交互方式为发布订阅方式，则基于服务请求方的注册请求，获取所述注册请求对应的响应数据，包括：Optionally, if it is determined that the interaction mode of the multiple custom standardized interfaces is a publish-subscribe mode, then based on the registration request of the service requester, the response data corresponding to the registration request is obtained, including:

所述服务提供方确定所述响应数据的准备状态；The service provider determines the readiness status of the response data;

若服务提供方确定所述响应数据的准备状态为成功，则按照所述注册请求中包括的反馈数据的方式以及反馈周期，周期性地反馈响应数据给所述服务请求方。If the service provider determines that the preparation status of the response data is successful, it periodically feeds back the response data to the service requester according to the feedback data included in the registration request and the feedback period.

具体地，在服务请求方和服务提供方的标准化接口的交互方式为发布订阅方式的情况下，服务请求方可以周期性的获得订阅数据。Specifically, in the case that the interaction mode of the standardized interface between the service requester and the service provider is a publish-subscribe mode, the service requester can obtain subscription data periodically.

图5是本发明提供的发布订阅方式交互示意图，如图5所示，服务请求方向服务提供方的“注册方法”注册请求数据的要求和通知服务。服务提供方在接到服务请求方的注册消息后，返回请求成功标志。服务请求方在获得上述消息后可以继续运行而不需要长时间处于阻塞状态。Fig. 5 is a schematic diagram of publishing and subscribing interaction provided by the present invention. As shown in Fig. 5, the service requester registers request data requirements and notification services with the "registration method" of the service provider. After receiving the registration message from the service requester, the service provider returns the request success flag. After receiving the above message, the service requester can continue to run without being blocked for a long time.

服务提供方在返回上述消息后进行一系列的业务处理以准备数据。After returning the above message, the service provider performs a series of business processes to prepare the data.

当数据准备完毕，服务提供方调用服务请求方注册的通知服务，告知服务请求方数据准备情况和相关的获取信息。服务请求方回复成功或失败的标志。服务请求方即可以按照给定的时间到给定的地点(如FTP或HTTP地址)在给定的有效期内自行决定何时使用给定的方式(如FTP或HTTP协议)取得数据。从而实现数据的触发式交互。When the data preparation is completed, the service provider calls the notification service registered by the service requester to inform the service requester of the data preparation status and related acquisition information. The service requester replies with a success or failure flag. The service requester can then go to a given location (such as FTP or HTTP address) at a given time and within a given validity period to decide when to use a given method (such as FTP or HTTP protocol) to obtain data. In order to realize the triggered interaction of data.

发布订阅方式适用于触发性传输各种数据量的数据共享服务。由于发布订阅方式本质上是异步响应方式，因此服务请求方不需要长时间的等待与阻塞，可以正常运行其他功能。The publish-subscribe method is suitable for data sharing services that trigger the transmission of various data volumes. Since the publish-subscribe method is essentially an asynchronous response method, the service requester does not need to wait and block for a long time, and can run other functions normally.

可选地，所述若所述访问请求用于数据查询，则基于KPI数据查询、数据模型结构查询和DQI数据查询中任意一种，发送所述访问请求至所述数据处理模块，包括：Optionally, if the access request is used for data query, sending the access request to the data processing module based on any one of KPI data query, data model structure query and DQI data query, including:

基于KPI数据查询中的维度属性和查询指标，以及Web Service/SOAP技术，确定与所述KPI数据查询对应数据查询结果；Based on the dimension attribute and query index in the KPI data query, and the Web Service/SOAP technology, determine the data query result corresponding to the KPI data query;

或，基于数据模型结构查询中的数据结构名称，维度指标，以及Web Service/SOAP技术，确定与所述数据模型结构查询对应的数据查询结果；Or, based on the data structure name, dimension index, and Web Service/SOAP technology in the data model structure query, determine the data query result corresponding to the data model structure query;

或，基于DQI数据查询中的维度属性和查询指标，以及Web Service/SOAP技术，确定与所述DQI数据查询对应的数据查询结果。Or, based on the dimension attribute and query index in the DQI data query, and the Web Service/SOAP technology, determine the data query result corresponding to the DQI data query.

具体地，本发明基于Web Service/SOAP技术实现外部系统数据查询需求，同时为保证Web Service/SOAP技术的规范性和互联互通性，还支持如下相关技术标准：Specifically, the present invention implements external system data query requirements based on Web Service/SOAP technology, and at the same time, in order to ensure the standardization and interconnectivity of Web Service/SOAP technology, the following related technical standards are also supported:

整个接口消息基于可扩展标记语言(Extensible Markup Language，XML)语言，符合XML V1.0(2nd Edition or Higher)规范和XML Schema(1st Edition or Higher)规范。The entire interface message is based on the Extensible Markup Language (XML) language, which conforms to the XML V1.0 (2nd Edition or Higher) specification and the XML Schema (1st Edition or Higher) specification.

接口实现符合SOAP协议V1.1版本规范，高版本SOAP协议保证与1.1版本的兼容性，接口描述使用网页服务描述语言(Web Services Description Language，WSDL)。The implementation of the interface conforms to the SOAP protocol V1.1 specification, and the high-version SOAP protocol guarantees compatibility with version 1.1. The interface description uses Web Services Description Language (WSDL).

接口实现至少支持SOAP在HTTP V1.0协议上的传输。The interface implementation at least supports the transmission of SOAP on the HTTP V1.0 protocol.

接口实现符合WSDL语言V1.1版本规范，高版本WSDL语言保证与1.1版本的兼容性。The interface implementation conforms to the WSDL language V1.1 version specification, and the higher version WSDL language guarantees the compatibility with the 1.1 version.

接口描述支持WSDL在SOAP V1.1协议上的绑定。The interface description supports the binding of WSDL over the SOAP V1.1 protocol.

在应用层采用UTF-8编码方式。UTF-8 encoding is used at the application layer.

在数据查询采用KPI数据查询时，基于元数据，以及KPI查询中维度属性和查询指标作为查询条件，查询符合条件的数据并返回。具体包括：根据维度属性查询指标、根据指标值范围查询过滤数据，同维度多指标合并查询。When the data query adopts KPI data query, based on the metadata, and the dimension attributes and query indicators in the KPI query as query conditions, the qualified data is queried and returned. Specifically, it includes: querying indicators according to dimension attributes, querying and filtering data according to the range of indicator values, and combining queries of multiple indicators in the same dimension.

在数据查询采用数据模型结构查询时，基于元数据，以及数据模型结构查询中的数据结构名称，维度指标作为查询条件，查询符合条件的数据并返回。具体包括：根据数据结构名称返回数据属性、根据维度查询纬度属性、根据维度查询数据模型的上级维度、根据维度查询指标，根据指标查询维度。When the data query uses the data model structure query, based on the metadata, the data structure name in the data model structure query, and the dimension index as the query condition, query and return the qualified data. Specifically, it includes: returning data attributes according to the data structure name, querying the latitude attribute according to the dimension, querying the upper-level dimension of the data model according to the dimension, querying the index according to the dimension, and querying the dimension according to the index.

在数据查询采用DQI数据查询时，基于元数据，以及DQI数据查询中维度属性和查询指标作为查询条件，查询符合条件的数据并返回。具体包括：根据维度属性查询指标、根据指标值范围查询过滤数据、同维度多指标合并查询以及根据维度以及查询指标，查询DQI。When the data query adopts DQI data query, based on the metadata, and the dimension attributes and query indicators in the DQI data query as query conditions, the qualified data is queried and returned. Specifically, it includes: querying indicators according to dimension attributes, querying and filtering data according to the range of indicator values, combining queries of multiple indicators in the same dimension, and querying DQIs according to dimensions and query indicators.

此外，上述查询结果，还可以通过可视化管理模块将查询结果通过可视化界面展示，方便服务请求方获取更直观的查询结果。In addition, the above query results can also be displayed through a visual interface through the visual management module, so that the service requester can obtain more intuitive query results.

通过上述不同的数据查询方式，可以灵活的为上层应用提供对应的数据查询业务，并可通过可视化界面展示对应的查询结果。Through the above-mentioned different data query methods, corresponding data query services can be flexibly provided for upper-layer applications, and corresponding query results can be displayed through a visual interface.

可选地，所述数据处理模块用于基于负载均衡算法，以及确定的服务器集群中不同服务器集群的集群性能权值，选择用于处理所述服务请求方的所述访问请求的目标服务器集群，并由所述目标服务器集群返回对应的响应数据，包括：Optionally, the data processing module is configured to select a target server cluster for processing the access request of the service requester based on a load balancing algorithm and determined cluster performance weights of different server clusters in the server cluster, And the corresponding response data is returned by the target server cluster, including:

基于所述服务请求方的访问请求对应的业务类型、数据量或响应性能需求，在所有所述服务集群中，确定一个或多个匹配的待定服务器集群；Based on the service type, data volume or response performance requirements corresponding to the access request of the service requester, among all the service clusters, determine one or more matching pending server clusters;

基于负载均衡算法，以及每个所述待定服务器集群的集群性能权值，选择目标服务集群处理所述服务请求方的访问请求；Based on the load balancing algorithm and the cluster performance weight of each pending server cluster, select a target service cluster to process the access request of the service requester;

发送所述目标服务器集群返回的对应的响应数据至所述服务请求方。sending the corresponding response data returned by the target server cluster to the service requester.

具体地，本发明提供的数据共享系统中还包括大量的服务器集群，为上述数据共享，数据采集，数据查询等中对应的处理流程进行计算。而每个服务器可能因为配置不同，硬件的差异，对应的性能可能存在差异，将性能相近的若干个服务器组成一个服务器集群，有利于有效的利用各个服务器的处理能力，并使得服务器集群的效益最大化。同样的，因为不同性能的服务器被划分为不同的服务器集群，那么不同的服务器集群也对应着不同的数据处理能力，也就是不同的服务器集群的性能不同。Specifically, the data sharing system provided by the present invention also includes a large number of server clusters to perform calculations for the above-mentioned corresponding processing procedures in data sharing, data collection, and data query. Each server may have different performance due to different configurations and hardware differences. Composing several servers with similar performance into a server cluster is conducive to effectively utilizing the processing capabilities of each server and maximizing the benefits of the server cluster. change. Similarly, because servers with different performances are divided into different server clusters, different server clusters also correspond to different data processing capabilities, that is, different server clusters have different performances.

将所有的服务器集群按照每个集群对应的数据处理能力，负载能力等，进行排序，并依据上述排序结果，确定每个服务器集群对应的集群性能权值。All server clusters are sorted according to the data processing capacity and load capacity corresponding to each cluster, and the cluster performance weight corresponding to each server cluster is determined according to the above sorting results.

当存在高并发访问请求(数据共享请求)时，对该数据共享请求按照业务类型进行分类后，结合数据量和响应性能需求，在所有服务集群中，筛选出匹配的待定服务器集群。这里筛选出的待定服务器集群可能包括一个或多个。在待定服务器集群包括多个的情况下，采用负载均衡算法，结合每个待定服务器集群的集群性能权值，选择目标服务集群处理上述数据共享请求。具体的，可以是待定服务器集群的集群性能权值越高，那么被选择作为目标服务集群的可能性越大。但是还需要结合该集群性能权值较高的代理服务器集群的负载量是否满足上述数据共享请求中对数据量的需求。When there is a high concurrent access request (data sharing request), after classifying the data sharing request according to the business type, combined with the data volume and response performance requirements, among all service clusters, the matching pending server clusters are screened out. The pending server clusters screened here may include one or more. In the case that there are multiple pending server clusters, a load balancing algorithm is used, combined with the cluster performance weight of each pending server cluster, to select a target service cluster to process the above data sharing request. Specifically, it may be that the higher the cluster performance weight of the undetermined server cluster is, the higher the possibility of being selected as the target service cluster is. However, it also needs to be combined with whether the load of the proxy server cluster with a higher cluster performance weight meets the data volume requirement in the above data sharing request.

可选地，所述系统还包括配置管理模块，用于对可视化管理模块中包括的所有可显示单元进行客制化配置；其中所述所有可显示单元包括服务地图、订阅管理、服务管理、审核管理和共享日志管理。Optionally, the system further includes a configuration management module, configured to customize all displayable units included in the visual management module; wherein all displayable units include service maps, subscription management, service management, auditing Management and shared log management.

具体地，本发明提供的数据共享系统，还包括配置管理模块，可用于对可视化管理模块中包括的服务地图、订阅管理、服务管理、审核管理和共享日志管理等可显示单元进行客制化配置。Specifically, the data sharing system provided by the present invention also includes a configuration management module, which can be used to customize the display units included in the visual management module, such as service map, subscription management, service management, audit management, and shared log management. .

其中，服务地图是外部系统中服务请求方的入口，为数据共享系统的第三方应用者提供直观的服务导航的功能，提供按照服务的类型、服务名称、服务提供系统等进行服务的查询，并直观展示各类服务的详细信息。Among them, the service map is the entrance of the service requester in the external system, providing intuitive service navigation functions for third-party users of the data sharing system, providing service query according to service type, service name, service provider system, etc., and Visually display the detailed information of various services.

上述服务的类型具体包括：FTP类服务和WS类的服务。The types of the above services specifically include: FTP services and WS services.

1)FTP类服务：1) FTP service:

服务的名称、服务的数据集、服务的提供系统、数据调度周期、数据保存的路径等信息。Information such as the name of the service, the data set of the service, the system for providing the service, the data scheduling cycle, and the path for saving the data.

2)网页服务(Web Service，WS)类的服务2) Web service (Web Service, WS) type of service

服务的名称、服务的数据集、服务的提供系统、服务的参数描述、服务返回数据格式等。Service name, service data set, service providing system, service parameter description, service return data format, etc.

订阅管理，为外部系统的服务请求方提供对共享服务订阅的管理，外部系统的服务请求方可通过服务地图查找需要的数据共享服务，对已发布的共享服务进行订阅。Subscription management, which provides the service requester of the external system with the management of the shared service subscription. The service requester of the external system can find the required data sharing service through the service map and subscribe to the published shared service.

服务管理，提供对数据共享服务的管理，数据共享服务既支持第三方应用系统提供，也支持数据共享系统提供。主要包含服务注册、服务变更、服务注销。Service management, providing management of data sharing services, data sharing services not only support the provision of third-party application systems, but also support the provision of data sharing systems. It mainly includes service registration, service modification, and service cancellation.

服务注册，用于发布服务，提供给服务请求方使用。服务查找是服务的请求方在服务注册信息中查找到相应的服务，并根据查找到的服务绑定信息，调用该服务。Service registration, which is used to publish services for use by service requesters. Service lookup means that the requester of the service finds the corresponding service in the service registration information, and invokes the service according to the found service binding information.

服务变更，对已发布的服务提供变更的功能，通过服务变更对发布的服务内容进行修改。Service change, providing the function of change to the published service, and modifying the content of the published service through service change.

服务注销，对已经不使用的数据服务可通过注销的功能将其从系统内注销，服务注销审批通过后。Service deregistration. Data services that are no longer in use can be deregistered from the system through the deregistration function. After the service deregistration is approved.

审核管理，在数据共享服务全生命周期内提供审核管理能力，主要包含服务注册审批、服务订阅审批、服务注销审批。Audit management, which provides audit management capabilities in the entire life cycle of data sharing services, mainly including service registration approval, service subscription approval, and service cancellation approval.

服务注册审批，对新注册的服务提供审批的功能，审批通过的服务在服务地图内才能查询到，供第三方系统使用。Service registration approval, which provides the approval function for newly registered services, and the approved services can only be queried in the service map for use by third-party systems.

服务订阅审批，订阅服务时需要经过审批，经过审批后服务方可调用。Approval for service subscription. Approval is required when subscribing to a service, and the service can only be called after approval.

服务注销审批，对于注销的服务，经审核没有使用者后，方可真正注销。Service cancellation approval, for the canceled service, the real cancellation can only be done after there are no users after verification.

共享日志管理，日志管理记录服务运行过程中的重要事件，需要长期保留并定期备份。日志主要包含如下内容：Shared log management, log management records important events during service operation, which need to be kept for a long time and backed up regularly. The log mainly includes the following contents:

服务的注册信息包括：服务注册者信息、服务标识、服务地址、服务描述、服务名称和注册时间等；The registration information of the service includes: service registrant information, service identification, service address, service description, service name and registration time, etc.;

服务访问信息包括：服务请求方身份信息、服务标识、服务地址、服务名称、数据范围等。Service access information includes: service requester identity information, service identification, service address, service name, data range, etc.

此外，本发明提供的数据共享系统，核心硬件设备需采用持分布式系统架构或热系统备份等冗余设计，避免单点故障对系统造成的风险。In addition, in the data sharing system provided by the present invention, the core hardware equipment needs to adopt redundant design such as distributed system architecture or hot system backup, so as to avoid the risk caused by single point failure to the system.

支持面向应用的存储及计算资源的配置和管控，实现应用之间的有效隔离，消除其相互的影响；具备应用自动切换能力，数据共享出现异常后，系统能够捕捉异常信息，并自动切换至备用节点。7*24小时不间断的数据共享及接口服务能力，对于共享过程中丢失的数据，提供回补功能。It supports the configuration and management of application-oriented storage and computing resources, realizes effective isolation between applications, and eliminates their mutual influence; it has the ability to automatically switch applications, and when data sharing is abnormal, the system can capture abnormal information and automatically switch to standby node. 7*24 hours of uninterrupted data sharing and interface service capabilities, and provide a recovery function for data lost during the sharing process.

自动化的异常处理能力，服务请求方(数据消费者)在接口服务调用上存在异常时，数据仓库管理模块能够俘获异常信息并终止此次调用，以便维持数据共享服务及接口的稳定性与正常工作，不会影响到其他服务请求方。对俘获的异常信息能够提交人工进行分析，发现异常产生的根源并予以解决。Automatic exception handling capability. When the service requester (data consumer) has an exception in the interface service call, the data warehouse management module can capture the abnormal information and terminate the call, so as to maintain the stability and normal operation of the data sharing service and interface. , will not affect other service requesters. The captured abnormal information can be submitted for manual analysis, and the root cause of the abnormality can be found and solved.

本发明提供的数据共享系统，还支持不同的共享模式，具体包括：The data sharing system provided by the present invention also supports different sharing modes, specifically including:

A模式：共享的数据完全由本数据共享系统提供，服务请求方直接从该数据共享系统中获取子集所需要的数据；Mode A: The shared data is completely provided by the data sharing system, and the service requester directly obtains the data required by the subset from the data sharing system;

B模式：各应用开发厂商(服务请求方)基于本数据共享系统提供的元数据，自定义符合业务需要的指标，并通过共享接口获取数据；Mode B: Each application developer (service requester) customizes indicators that meet business needs based on the metadata provided by this data sharing system, and obtains data through the shared interface;

C模式：各应用开发厂商(服务请求方)基于本数据共享系统提供的元数据，对数据结构进行自定义，获取对应的结果数据并使用。Mode C: Each application developer (service requester) customizes the data structure based on the metadata provided by the data sharing system, obtains the corresponding result data and uses it.

通过上述不同的模式，对不同开发厂商或者第三方应用提供更灵活的数据共享方式，兼容性较强。Through the above different modes, it provides more flexible data sharing methods for different developers or third-party applications, and has strong compatibility.

本发明提供的数据共享系统，通过不同的标准化接口，对外界系统访问该数据共享系统的方式进行统一的管理，采用服务器集群，对不同业务类型的不同服务请求方的访问请求选择性能匹配的服务器集群，提高本数据共享系统的数据共享服务的扩展性能。在保障数据安全和可靠性的前提下，更灵活的实现对数据的共享。The data sharing system provided by the present invention conducts unified management on the ways in which external systems access the data sharing system through different standardized interfaces, adopts server clusters, and selects servers with matching performance for the access requests of different service requesters of different business types The cluster improves the expansion performance of the data sharing service of the data sharing system. On the premise of ensuring data security and reliability, more flexible realization of data sharing.

图6是本发明提供的数据共享方法的流程示意图，如图6所示，该方法包括：FIG. 6 is a schematic flowchart of a data sharing method provided by the present invention. As shown in FIG. 6 , the method includes:

步骤601、根据上层应用的需求，确定每个所述上层应用对应的共享服务接口；Step 601: Determine the shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer application;

步骤602、基于多种自定义标准化接口的交互方式以及所述上层应用对应的所述共享服务接口，发送服务请求方的访问请求；所述多种自定义标准化接口的交互方式包括消息交互方式、发布订阅方式和数据库访问方式；Step 602: Send an access request of the service requester based on the interaction modes of various self-defined standardized interfaces and the shared service interface corresponding to the upper-layer application; the interaction modes of the various self-defined standardized interfaces include message interaction mode, Publish and subscribe methods and database access methods;

步骤603、基于负载均衡算法，以及确定的服务器集群中不同服务器集群的集群性能权值，选择用于处理所述服务请求方的所述访问请求的目标服务器集群，并由所述目标服务器集群返回对应的响应数据；Step 603: Based on the load balancing algorithm and the determined cluster performance weights of different server clusters in the server cluster, select a target server cluster for processing the access request of the service requester, and return it from the target server cluster corresponding response data;

步骤604、基于所述数据处理模块获得的所述响应数据，向所述服务请求方提供可视化界面展示所述响应数据。Step 604: Based on the response data obtained by the data processing module, provide the service requester with a visual interface to display the response data.

具体地，本发明提供的数据共享方法，可面向不同场景，提供差异化的数据共享。针对不同厂商提供的多种应用，主要可通过三类差异化共享方式为上层应用提供数据支撑。上述三类差异化共享方式主要包括：实时KPI计算类，实时细节查询类，分析类非实时统计。其中，实时KPI计算类主要是通过实时信令KPI，数据质量预警和准时统计三方面对上层应用提供数据支撑；实时细节查询类主要是通过号码查询，用户记录查询，投诉故障查询三方面对上层应用提供数据支撑；分析类非实时统计主要是通过指标获取呈现，细节数据获取，使用数据共享系统计算/存储资源的应用对上层应用提供数据支撑。Specifically, the data sharing method provided by the present invention can provide differentiated data sharing for different scenarios. For various applications provided by different manufacturers, data support can be provided for upper-layer applications mainly through three types of differentiated sharing methods. The above three types of differentiated sharing methods mainly include: real-time KPI calculation, real-time detail query, and analysis non-real-time statistics. Among them, the real-time KPI calculation class mainly provides data support for upper-layer applications through real-time signaling KPI, data quality early warning and punctual statistics; the real-time detail query class mainly provides data support for upper-layer applications through number query, user record query, and complaint fault query. Applications provide data support; analytical non-real-time statistics are mainly presented through the acquisition of indicators, detailed data acquisition, and applications that use computing/storage resources of the data sharing system to provide data support for upper-level applications.

且通过实时KPI计算提供数据支撑的业务请求或上层应用需求，主要具有如下特点：对数据实时性要求较高，共享数据量较小，交互过程通常为单向，数据获取频率为实时。Moreover, business requests or upper-layer application requirements that provide data support through real-time KPI calculations mainly have the following characteristics: high requirements for real-time data, small amount of shared data, usually one-way interaction process, and real-time data acquisition frequency.

通过实时细节查询类提供数据支撑的业务请求或上层应用需求，主要具有如下特点：对数据实时性要求较低，共享数据量较小，交互过程通常为双向，数据获取频率为按需。The business requests or upper-layer application requirements that provide data support through the real-time detailed query class mainly have the following characteristics: low requirements for real-time data, small amount of shared data, usually two-way interaction process, and data acquisition frequency on demand.

通过分析类非实时统计提供数据支撑的业务请求或上层应用需求，主要具有如下特点：对数据实时性要求较低，共享数据量大小均可，交互过程通常为双向，数据获取频率为定时。Business requests or upper-layer application requirements that provide data support through analytical non-real-time statistics mainly have the following characteristics: low requirements for real-time data, the amount of shared data can be large or small, the interaction process is usually two-way, and the frequency of data acquisition is regular.

而上述三类差异化共享方式，则是通过数据共享系统中对应的共享服务接口模块根据上层应用的需求确定的，具体的上层应用的需求可以是满足上述时KPI计算类，实时细节查询类，分析类非实时统计提供数据支撑的业务请求或上层应用需求的特点中的任意一类；确定对应的共享服务接口。The above three types of differentiated sharing methods are determined through the corresponding sharing service interface module in the data sharing system according to the requirements of upper-layer applications. The specific requirements of upper-layer applications can be to meet the above-mentioned KPI calculation category, real-time detail query category, Analytical non-real-time statistics provide any type of data-supported business requests or characteristics of upper-layer application requirements; determine the corresponding shared service interface.

为了保证数据共享服务及接口的通用性、易用性、一致性与稳定性等要求。通过标准化接口和接口协议，保证任何需要获取数据的外部应用，只需要遵循数据共享接口规范要求，均能通过数据共享系统方便获取数据。In order to ensure the universality, ease of use, consistency and stability of data sharing services and interfaces. Through the standardized interface and interface protocol, it is guaranteed that any external application that needs to obtain data can easily obtain data through the data sharing system only by following the requirements of the data sharing interface specification.

其中，标准化接口和接口协议包括：消息队列、Web Service、FTP、数据库和Socket等。而针对标准化接口的交互方式，本发明提出了三种形式：消息交互方式、发布订阅方式和数据库方式。Among them, standardized interfaces and interface protocols include: message queue, Web Service, FTP, database and Socket, etc. For the interaction mode of the standardized interface, the present invention proposes three forms: a message interaction mode, a publish-subscribe mode and a database mode.

消息交互方式是服务请求方与服务提供方进行数据交互的方式。消息交互方式按照数据交互的方式分为“在线请求与同步响应方式”、“在线请求与异步响应方式”两种，分别对应不同的数据量和响应性能指标。通常用于一次性的获取共享数据。The message interaction mode is the way of data interaction between the service requester and the service provider. The message interaction mode is divided into two types according to the data interaction mode: "online request and synchronous response mode" and "online request and asynchronous response mode", which correspond to different data volumes and response performance indicators. Usually used for one-time access to shared data.

发布订阅方式主要是服务请求方可以周期性的获取订阅数据。The publish-subscribe method mainly enables the service requester to obtain subscription data periodically.

数据库方式只要是针对不同类型的数据库，可能采用不同的数据库连接协议，而本发明中通过通用的连接协议，避免了直接暴露数据库的物理模型，提高数据共享系统的耦合性。As long as the database mode is aimed at different types of databases, different database connection protocols may be used, but in the present invention, the general connection protocol avoids directly exposing the physical model of the database and improves the coupling of the data sharing system.

上述说明，主要是通过上述三种标准化接口的交互方式，对外部系统提供数据共享，在此之前，需要确定具体采用哪个目标服务器集群对该访问请求进行处理，在选择目标服务器集群时需要考虑服务器集群的集群性能权值，以及所示访问请求对应的业务类型，以及各服务器集群的负载量，基于负载均衡算法，确定最终的目标服务器集群。并在获得最终的目标服务器集群的响应数据后，通过可视化的界面展示给所述服务请求方，以便于服务请求方更直观的了解共享数据的分布情况。The above description is mainly to provide data sharing to external systems through the interaction of the above three standardized interfaces. Before that, it is necessary to determine which target server cluster to use to process the access request, and the server needs to be considered when selecting the target server cluster. The cluster performance weight of the cluster, the service type corresponding to the indicated access request, and the load of each server cluster are used to determine the final target server cluster based on the load balancing algorithm. After obtaining the final response data of the target server cluster, it is displayed to the service requester through a visual interface, so that the service requester can more intuitively understand the distribution of the shared data.

本发明提供的数据共享方法，通过不同的标准化接口，对外界系统访问该数据共享系统的方式进行统一的管理，采用服务器集群，对不同业务类型的不同服务请求方的访问请求选择性能匹配的服务器集群，提高本数据共享系统的数据共享服务的扩展性能。在保障数据安全和可靠性的前提下，更灵活的实现对数据的共享。The data sharing method provided by the present invention, through different standardized interfaces, uniformly manages the way that external systems access the data sharing system, adopts server clusters, and selects servers with matching performance for the access requests of different service requesters of different business types. Cluster, to improve the expansion performance of the data sharing service of the data sharing system. On the premise of ensuring data security and reliability, data sharing can be realized more flexibly.

可选地，所述共享接口模块用于根据上层应用的需求，确定每个所述上层应用对应的共享服务接口，包括以下至少一项：Optionally, the shared interface module is configured to determine the shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer application, including at least one of the following:

基于第三方应用对应的接入权限，控制所述第三方应用是否接入所述数据共享系统，并提供通道加密机制；Based on the access authority corresponding to the third-party application, control whether the third-party application accesses the data sharing system, and provide a channel encryption mechanism;

或，基于多租户的访问权限以及访问量需求，确定所述多租户中每个租户的可用接口；Or, based on the multi-tenant access rights and access requirements, determine the available interfaces of each tenant in the multi-tenant;

或，基于第三方应用的访问频度，确定是否对所述第三方应用的访问进行限制；Or, based on the access frequency of the third-party application, determine whether to restrict the access of the third-party application;

或，基于访问队列中访问请求的优先级次序以及所述访问请求的连接状态，调整不同优先级的所述访问请求的处理优先级。Or, based on the priority order of the access requests in the access queue and the connection status of the access requests, the processing priorities of the access requests with different priorities are adjusted.

可选地，所述数据仓库管理模块用于基于多种自定义标准化接口的交互方式以及所述上层应用对应的所述共享服务接口，发送服务请求方的访问请求，包括：Optionally, the data warehouse management module is configured to send the access request of the service requester based on the interaction modes of multiple custom standardized interfaces and the shared service interface corresponding to the upper-layer application, including:

确定所述访问请求用于数据共享或数据查询；determine that the access request is for data sharing or data query;

若所述访问请求用于数据共享，则基于消息交互方式、发布订阅方式和数据库方式中任意一种或多种方式的组合，发送所述访问请求至所述数据处理模块；If the access request is for data sharing, send the access request to the data processing module based on any one or a combination of message interaction, publish-subscribe, and database;

若所述访问请求用于数据查询，则基于KPI数据查询、数据模型结构查询和DQI数据查询中任意一种，发送所述访问请求至所述数据处理模块。If the access request is for data query, send the access request to the data processing module based on any one of KPI data query, data model structure query and DQI data query.

可选地，所述基于消息交互方式、发布订阅方式和数据库方式中任意一种或多种方式的组合，发送所述访问请求至所述数据处理模块，包括：Optionally, sending the access request to the data processing module based on any one or a combination of message interaction, publish-subscribe, and database methods includes:

若确定所述多种自定义标准化接口的交互方式为消息交互方式，则基于所述服务请求方的所述访问请求的数据量、响应时间需求以及对应响应性能指标，确定所述消息交互方式中对应的响应方式为同步响应或异步响应；If it is determined that the interaction modes of the multiple custom standardized interfaces are the message interaction modes, then based on the data volume of the access request of the service requester, the response time requirement and the corresponding response performance index, determine the message interaction mode in the message interaction mode. The corresponding response mode is synchronous response or asynchronous response;

若确定所述多种自定义标准化接口的交互方式为发布订阅方式，则基于服务请求方的注册请求，获取所述注册请求对应的响应数据；If it is determined that the interaction mode of the multiple custom standardized interfaces is the publish-subscribe mode, then based on the registration request of the service requester, the response data corresponding to the registration request is obtained;

若确定所述多种自定义标准化接口的交互方式为数据库方式，则基于ODBC或JDBC访问方式，确定外部系统对数据库的访问接口。If it is determined that the interaction mode of the various self-defined standardized interfaces is the database mode, then the access interface of the external system to the database is determined based on the ODBC or JDBC access mode.

可选地，所述若确定所述多种自定义标准化接口的交互方式为消息交互方式，则基于所述服务请求方的所述访问请求的数据量、响应时间需求以及对应响应性能指标，确定所述消息交互方式中对应的响应方式为同步响应或异步响应之后，包括：Optionally, if it is determined that the interaction mode of the various self-defined standardized interfaces is a message interaction mode, then based on the data volume of the access request of the service requester, the response time requirement and the corresponding response performance index, determine The corresponding response mode in the message interaction mode is after synchronous response or asynchronous response, including:

若所述消息交互方式中对应的响应方式为同步响应，则所述服务请求方进入阻塞状态等待服务提供方的响应；If the corresponding response mode in the message interaction mode is a synchronous response, the service requester enters a blocking state and waits for a response from the service provider;

若所述消息交互方式中对应的响应方式为异步响应，则所述服务请求方通过轮询或者事件通知确定所述响应数据是否准备完毕。If the corresponding response mode in the message interaction mode is an asynchronous response, the service requester determines whether the response data is ready through polling or event notification.

可选地，所述若确定所述多种自定义标准化接口的交互方式为发布订阅方式，则基于服务请求方的注册请求，获取所述注册请求对应的响应数据，包括：Optionally, if it is determined that the interaction mode of the various self-defined standardized interfaces is a publish-subscribe mode, then based on the registration request of the service requester, obtaining the response data corresponding to the registration request includes:

所述服务提供方确定所述响应数据的准备状态；The service provider determines the readiness status of the response data;

若服务提供方确定所述响应数据的准备状态为成功，则按照所述注册请求中包括的反馈数据的方式以及反馈周期，周期性地反馈响应数据给所述服务请求方。If the service provider determines that the preparation status of the response data is successful, it periodically feeds back the response data to the service requester according to the feedback data included in the registration request and the feedback cycle.

可选地，所述若所述访问请求用于数据查询，则基于KPI数据查询、数据模型结构查询和DQI数据查询中任意一种，发送所述访问请求至所述数据处理模块，包括：Optionally, if the access request is used for data query, sending the access request to the data processing module based on any one of KPI data query, data model structure query and DQI data query, including:

基于KPI数据查询中的维度属性和查询指标，以及Web Service/SOAP技术，确定与所述KPI数据查询对应数据查询结果；Based on the dimension attribute and query index in the KPI data query, and the Web Service/SOAP technology, determine the data query result corresponding to the KPI data query;

或，基于数据模型结构查询中的数据结构名称，维度指标，以及Web Service/SOAP技术，确定与所述数据模型结构查询对应的数据查询结果；Or, based on the data structure name, dimension index, and Web Service/SOAP technology in the data model structure query, determine the data query result corresponding to the data model structure query;

或，基于DQI数据查询中的维度属性和查询指标，以及Web Service/SOAP技术，确定与所述DQI数据查询对应的数据查询结果。Or, a data query result corresponding to the DQI data query is determined based on the dimension attributes and query indicators in the DQI data query, and the Web Service/SOAP technology.

可选地，所述数据处理模块用于基于负载均衡算法，以及确定的服务器集群中不同服务器集群的集群性能权值，选择用于处理所述服务请求方的所述访问请求的目标服务器集群，并由所述目标服务器集群返回对应的响应数据，包括：Optionally, the data processing module is configured to select a target server cluster for processing the access request of the service requester based on a load balancing algorithm and determined cluster performance weights of different server clusters in the server cluster, And the corresponding response data is returned by the target server cluster, including:

基于所述服务请求方的访问请求对应的业务类型、数据量或响应性能需求，在所有所述服务集群中，确定一个或多个匹配的待定服务器集群；Based on the service type, data volume or response performance requirements corresponding to the access request of the service requester, among all the service clusters, determine one or more matching pending server clusters;

基于负载均衡算法，以及每个所述待定服务器集群的集群性能权值，选择目标服务集群处理所述服务请求方的访问请求；Selecting a target service cluster to process the access request of the service requester based on the load balancing algorithm and the cluster performance weight of each of the pending server clusters;

发送所述目标服务器集群返回的对应的响应数据至所述服务请求方。Send corresponding response data returned by the target server cluster to the service requester.

可选地，所述系统还包括配置管理模块，用于对可视化管理模块中包括的所有可显示单元进行客制化配置；其中所述所有可显示单元包括服务地图、订阅管理、服务管理、审核管理和共享日志管理。Optionally, the system further includes a configuration management module, configured to customize all displayable units included in the visual management module; wherein all displayable units include service maps, subscription management, service management, auditing Management and shared log management.

本发明提供的数据共享方法，通过不同的标准化接口，对外界系统访问该数据共享系统的方式进行统一的管理，采用服务器集群，对不同业务类型的不同服务请求方的访问请求选择性能匹配的服务器集群，提高本数据共享系统的数据共享服务的扩展性能。在保障数据安全和可靠性的前提下，更灵活的实现对数据的共享。The data sharing method provided by the present invention conducts unified management on the ways in which external systems access the data sharing system through different standardized interfaces, adopts server clusters, and selects servers with matching performance for the access requests of different service requesters of different business types The cluster improves the expansion performance of the data sharing service of the data sharing system. On the premise of ensuring data security and reliability, more flexible realization of data sharing.

最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. A data sharing system, characterized in that the data sharing system comprises: the system comprises a sharing interface module, a data warehouse management module, a data processing module and a visualization management module;

the shared interface module is used for determining a shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer applications;

the data warehouse management module is used for sending an access request of a service requester based on interaction modes of various user-defined standardized interfaces and the shared service interface corresponding to the upper application; the interaction modes of the multiple user-defined standardized interfaces comprise a message interaction mode, a publish-subscribe mode and a database access mode;

the data processing module is used for selecting a target server cluster for processing the access request of the service requester based on a load balancing algorithm and the determined cluster performance weights of different server clusters in the server clusters, and returning corresponding response data by the target server cluster;

and the visualization management module provides a visualization interface for the service requester to display the response data based on the response data obtained by the data processing module.

2. The data sharing system according to claim 1, wherein the shared interface module is configured to determine the shared service interface corresponding to each upper layer application according to requirements of the upper layer application, and includes at least one of:

controlling whether the third-party application is accessed to the data sharing system or not based on the access authority corresponding to the third-party application, and providing a channel encryption mechanism;

or determining an available interface of each tenant in the multi-tenants based on the access authority and the access quantity requirement of the multi-tenants;

or determining whether to limit the access of the third-party application based on the access frequency of the third-party application;

or, based on the priority order of the access requests in the access queue and the connection state of the access requests, adjusting the processing priority of the access requests with different priorities.

3. The data sharing system according to claim 1, wherein the data warehouse management module is configured to send an access request of a service requester based on interaction modes of multiple customized standardized interfaces and the shared service interface corresponding to the upper application, and includes:

determining that the access request is for data sharing or data querying;

if the access request is used for data sharing, the access request is sent to the data processing module based on any one or combination of a plurality of modes of a message interaction mode, a publish-subscribe mode and a database mode;

and if the access request is used for data query, the access request is sent to the data processing module based on any one of key performance indicator KPI data query, data model structure query and design quality indicator DQI data query.

4. The data sharing system according to claim 3, wherein the sending the access request to the data processing module based on any one or a combination of more than one of a message interaction manner, a publish-subscribe manner, and a database manner includes:

if the interactive modes of the multiple user-defined standardized interfaces are determined to be message interactive modes, determining that the corresponding response mode in the message interactive modes is synchronous response or asynchronous response based on the data volume of the access request of the service requester, the response time requirement and the corresponding response performance index;

if the interactive mode of the multiple user-defined standardized interfaces is determined to be a publish-subscribe mode, acquiring response data corresponding to a registration request based on the registration request of a service requester;

and if the interactive mode of the plurality of user-defined standardized interfaces is determined to be a database mode, determining an access interface of an external system to the database based on an open database connection ODBC or Java database connection JDBC access mode.

5. The data sharing system of claim 4, wherein if it is determined that the interaction manners of the plurality of customized standardized interfaces are message interaction manners, after determining that the corresponding response manner in the message interaction manners is a synchronous response or an asynchronous response based on the data amount of the access request of the service requester, the response time requirement, and the corresponding response performance index, the method comprises:

if the corresponding response mode in the message interaction mode is synchronous response, the service requester enters a blocking state to wait for the response of the service provider;

and if the corresponding response mode in the message interaction mode is asynchronous response, the service requester determines whether the response data is prepared or not through polling or event notification.

6. The data sharing system according to claim 4, wherein if it is determined that the interaction manner of the plurality of customized standardized interfaces is a publish-subscribe manner, acquiring response data corresponding to a registration request based on the registration request of a service requester comprises:

the service provider determining a readiness state of the response data;

and if the service provider determines that the preparation state of the response data is successful, periodically feeding back the response data to the service requester according to a feedback data mode and a feedback period included in the registration request.

7. The data sharing system according to claim 3, wherein if the access request is for data query, the sending the access request to the data processing module based on any one of KPI data query, data model structure query and DQI data query includes:

determining a data query result corresponding to KPI data query based on dimension attributes and query indexes in KPI data query and a Web Service/Simple Object Access Protocol (SOAP) technology;

or, determining a data query result corresponding to the data model structure query based on the data structure name, the dimension index and the Web Service/SOAP technology in the data model structure query;

or, determining a data query result corresponding to the DQI data query based on the dimension attribute and the query index in the DQI data query and the Web Service/SOAP technology.

8. The data sharing system according to claim 1, wherein the data processing module is configured to select a target server cluster for processing the access request of the service requester based on a load balancing algorithm and the determined cluster performance weights of different server clusters in the server clusters, and return corresponding response data from the target server cluster, and includes:

determining one or more matched pending server clusters in all the service clusters based on the service type, data volume or response performance requirement corresponding to the access request of the service requester;

based on a load balancing algorithm and the cluster performance weight of each undetermined server cluster, selecting a target service cluster to process the access request of the service requester;

and sending the corresponding response data returned by the target server cluster to the service requester.

9. The data sharing system according to claim 1, wherein the system further comprises a configuration management module for custom configuring all displayable units included in the visualization management module; wherein the all displayable units include service maps, subscription management, service management, audit management, and shared log management.

10. A method for sharing data, comprising:

determining a shared service interface corresponding to each upper-layer application according to the requirements of the upper-layer applications;

sending an access request of a service requester based on an interactive mode of a plurality of user-defined standardized interfaces and the shared service interface corresponding to the upper application; the interaction modes of the multiple user-defined standardized interfaces comprise a message interaction mode, a publish-subscribe mode and a database access mode;

based on a load balancing algorithm and the determined cluster performance weights of different server clusters in the server clusters, selecting a target server cluster for processing the access request of the service requester, and returning corresponding response data by the target server cluster;

and providing a visual interface for the service requester to display the response data based on the response data obtained by the data processing module.

Images