CN111800443B - Data processing system and method, device and electronic equipment - Google Patents
Data processing system and method, device and electronic equipmentDownload PDFInfo
- Publication number
- CN111800443B CN111800443BCN201910277959.2ACN201910277959ACN111800443BCN 111800443 BCN111800443 BCN 111800443BCN 201910277959 ACN201910277959 ACN 201910277959ACN 111800443 BCN111800443 BCN 111800443B
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- edge cloud
- control device
- cloud node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545processingMethods0.000titleclaimsabstractdescription320
- 238000000034methodMethods0.000titleclaimsdescription23
- 238000003672processing methodMethods0.000claimsabstractdescription33
- 238000007689inspectionMethods0.000claimsdescription9
- 230000002159abnormal effectEffects0.000claimsdescription7
- 230000008569processEffects0.000claimsdescription5
- 230000005856abnormalityEffects0.000claims2
- 230000001360synchronised effectEffects0.000claims2
- 238000004140cleaningMethods0.000claims1
- 230000004044responseEffects0.000abstractdescription15
- 238000007726management methodMethods0.000description128
- 238000004891communicationMethods0.000description10
- 238000013500data storageMethods0.000description9
- 238000010586diagramMethods0.000description8
- 238000005516engineering processMethods0.000description7
- 230000003993interactionEffects0.000description5
- 230000008901benefitEffects0.000description4
- 230000002452interceptive effectEffects0.000description4
- 230000005236sound signalEffects0.000description4
- 230000009471actionEffects0.000description3
- 238000013480data collectionMethods0.000description3
- 230000002085persistent effectEffects0.000description3
- 230000005540biological transmissionEffects0.000description2
- 238000007405data analysisMethods0.000description2
- 238000012423maintenanceMethods0.000description2
- 230000003287optical effectEffects0.000description2
- 238000004364calculation methodMethods0.000description1
- 230000008859changeEffects0.000description1
- 230000007547defectEffects0.000description1
- 238000011161developmentMethods0.000description1
- 230000006870functionEffects0.000description1
- 239000004973liquid crystal related substanceSubstances0.000description1
- 238000012986modificationMethods0.000description1
- 230000004048modificationEffects0.000description1
- 238000012544monitoring processMethods0.000description1
- 230000003068static effectEffects0.000description1
Images
Classifications
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及云计算技术领域，尤其涉及一种数据处理系统和方法、装置以及电子设备。The present invention relates to the technical field of cloud computing, and in particular, to a data processing system and method, an apparatus and an electronic device.
背景技术Background technique
边缘节点由于更加靠近用户或数据源，能够就近提供最近端服务。从而能够产生更快的网络服务响应，以及满足用户在实时业务以及安全与隐私保护等方面的边缘侧需求。Because edge nodes are closer to users or data sources, they can provide the most recent services nearby. In this way, faster network service responses can be generated, and users' edge-side requirements in real-time services and security and privacy protection can be met.
近年来，由于网络的发展也出现了基于近用户侧的边缘节点，利用网络将用户附近的边缘节点变为该用户也可用的边缘节点，从而出现了类似云网络的边缘节点网络。能够更好地发挥用户侧的近用户计算的高响应速度，又能够满足用户需求的弹性变化。In recent years, due to the development of the network, edge nodes based on the near-user side have also appeared, using the network to change the edge nodes near the user into the edge nodes that are also available to the user, and thus the edge node network similar to the cloud network has emerged. It can better utilize the high response speed of near-user computing on the user side, and can also meet the elastic changes of user needs.
发明人在实现本发明的过程中，发现现有技术至少存在如下问题：在现有技术中，由近用户侧的多个边缘节点构成的边缘云与传统的由集中式的数据中心构成的中心云这两种计算资源，在针对用户的计算需求提供服务时，只能根据各自的优势针对性地进行处理，无法同时满足用户在响应速度和在计算存储能力上的需求。In the process of implementing the present invention, the inventor found that the prior art has at least the following problems: In the prior art, the edge cloud composed of multiple edge nodes near the user side and the traditional center composed of a centralized data center Cloud, the two computing resources, can only be processed according to their respective advantages when providing services for users' computing needs, and cannot meet users' needs in response speed and computing storage capacity at the same time.
发明内容SUMMARY OF THE INVENTION
本发明实施例提供一种数据处理系统和方法、装置以及电子设备，以解决现有技术中边缘云或中心云在提供服务时无法同时满足用户在响应速度和在计算存储能力上的需求的缺陷。Embodiments of the present invention provide a data processing system and method, an apparatus, and an electronic device, so as to solve the defect in the prior art that edge cloud or central cloud cannot simultaneously meet user requirements in response speed and computing storage capacity when providing services .
为达到上述目的，本发明实施例提供了一种数据处理系统，包括：边缘云节点和通过网络通道与所述边缘云节点连接的中心管控设备，To achieve the above object, an embodiment of the present invention provides a data processing system, including: an edge cloud node and a central management and control device connected to the edge cloud node through a network channel,
所述边缘云节点，用于获取第一数据并进行第一处理，生成第二数据，并将所述第二数据通过所述网络通道发送至所述中心管控设备；the edge cloud node, configured to acquire the first data and perform the first processing, generate the second data, and send the second data to the central management and control device through the network channel;
所述中心管控设备，用于对接收到的所述第二数据进行第二处理，生成第三数据，所述第三数据为所述第一数据的目标处理结果数据。The central management and control device is configured to perform second processing on the received second data to generate third data, where the third data is target processing result data of the first data.
本发明实施例还提供了一种数据处理方法，应用于包括边缘云节点和通过网络通道与所述边缘云节点相连的中心管控设备的网络中，所述方法包括：An embodiment of the present invention further provides a data processing method, which is applied in a network including an edge cloud node and a central management and control device connected to the edge cloud node through a network channel, the method comprising:
通过网络通道向边缘云节点发送第六数据以供所述边缘云节点进行第六处理生成所述第六数据的目标处理结果数据，所述第六数据是中心管控设备通过对获取的第五数据进行第五处理而生成的；或，Send sixth data to the edge cloud node through the network channel for the edge cloud node to perform sixth processing to generate the target processing result data of the sixth data, and the sixth data is the fifth data obtained by the central management and control device by generated by the fifth processing; or,
通过网络通道接收边缘云节点发送的第三数据，对所述第三数据进行第三处理，生成所述第三数据的目标处理结果数据。The third data sent by the edge cloud node is received through the network channel, and third processing is performed on the third data to generate target processing result data of the third data.
本发明实施例还提供了一种数据处理方法，应用于包括边缘云节点和通过网络通道与所述边缘云节点相连的中心管控设备的网络中，所述方法包括：An embodiment of the present invention further provides a data processing method, which is applied in a network including an edge cloud node and a central management and control device connected to the edge cloud node through a network channel, the method comprising:
对第一数据进行第一处理并生成第二数据；performing first processing on the first data and generating second data;
将所述第二数据通过网络通道发送给所述中心管控设备，以供所述中心管控设备根据所述第二数据的目标处理信息对所述第二数据进行第二处理并生成目标数据。The second data is sent to the central management and control device through a network channel, so that the central management and control device can perform second processing on the second data and generate target data according to the target processing information of the second data.
本发明实施例还提供了一种数据处理装置，应用于包括边缘云节点和通过网络通道与所述边缘云节点相连的中心管控设备的网络中，所述装置包括：An embodiment of the present invention further provides a data processing apparatus, which is applied in a network including an edge cloud node and a central management and control device connected to the edge cloud node through a network channel, the apparatus comprising:
数据发送模块，用于通过网络通道向边缘云节点发送第六数据以供所述边缘云节点进行第六处理生成所述第六数据的目标处理结果数据，所述第六数据是所述中心管控设备通过对获取的第五数据进行第五处理而生成的；以及A data sending module, configured to send sixth data to the edge cloud node through a network channel for the edge cloud node to perform sixth processing to generate target processing result data of the sixth data, and the sixth data is the central management and control generated by the device by performing fifth processing on the acquired fifth data; and
数据接收模块，用于通过网络通道接收边缘云节点发送的第三数据；a data receiving module for receiving the third data sent by the edge cloud node through the network channel;
第二处理模块，用于对所述第三数据进行第三处理，生成所述第二数据的目标处理结果数据。The second processing module is configured to perform third processing on the third data to generate target processing result data of the second data.
本发明实施例还提供了一种数据处理装置，应用于包括边缘云节点和通过网络通道与所述边缘云节点相连的中心管控设备的网络中，所述装置包括：An embodiment of the present invention further provides a data processing apparatus, which is applied in a network including an edge cloud node and a central management and control device connected to the edge cloud node through a network channel, the apparatus comprising:
第一处理模块，用于对第一数据进行第一处理并生成第二数据；a first processing module, configured to perform first processing on the first data and generate second data;
数据上报模块，用于将所述第二数据通过网络通道发送给所述中心管控设备，以供所述中心管控设备根据所述第二数据的目标处理信息对所述第二数据进行第二处理并生成目标数据。A data reporting module, configured to send the second data to the central management and control device through a network channel, so that the central management and control device can perform second processing on the second data according to the target processing information of the second data and generate target data.
本发明实施例还提供了一种数据处理系统，应用于包括边缘云节点和通过网络通道与所述边缘云节点相连的中心管控设备的网络中，所述边缘云节点包括边缘云管控设备，所述边缘云管控设备执行前述方法。An embodiment of the present invention further provides a data processing system, which is applied to a network including an edge cloud node and a central management and control device connected to the edge cloud node through a network channel, where the edge cloud node includes an edge cloud management and control device, so The edge cloud management and control device executes the foregoing method.
本发明实施例还提供一种数据处理系统，包括：边缘云节点和通过网络通道与所述边缘云节点连接的中心管控设备，An embodiment of the present invention further provides a data processing system, including: an edge cloud node and a central management and control device connected to the edge cloud node through a network channel,
所述边缘云节点，用于获取数据并处理，将所述处理的数据通过所述网络通道发送至所述中心管控设备；The edge cloud node is used for acquiring and processing data, and sending the processed data to the central management and control device through the network channel;
所述中心管控设备，用于获取并处理来自所述边缘云节点的数据，生成并存储最终数据。The central management and control device is used for acquiring and processing data from the edge cloud nodes, and generating and storing final data.
本发明实施例还提供了一种电子设备，包括：The embodiment of the present invention also provides an electronic device, including:
存储器，用于存储程序；memory for storing programs;
处理器，用于运行所述存储器中存储的所述程序，使得所述电子设备：a processor for running the program stored in the memory, so that the electronic device:
控制边缘云节点获取第一数据并进行第一处理，生成第二数据；Controlling the edge cloud node to obtain the first data and perform the first processing to generate the second data;
控制所述边缘云节点将所述第二数据通过网络通道发送至中心管控设备；controlling the edge cloud node to send the second data to the central management and control device through a network channel;
控制所述中心管控设备对所述第二数据进行第二处理，生成第三数据，所述第三数据为所述第一数据的目标处理结果数据。The central management and control device is controlled to perform second processing on the second data to generate third data, where the third data is target processing result data of the first data.
本发明实施例提供的数据处理系统和方法、装置以及电子设备，针对边缘云节点的数据，通过在边缘云节点和中心管控设备中进行两级处理，即，一方面在边缘云节点进行第一处理，以满足用户对响应速度的需求，另一方面在中心管控设备进行第二处理，以满足用户对数据的计算存储能力的需求，从而提高了用户体验。The data processing system and method, device, and electronic device provided by the embodiments of the present invention perform two-level processing in the edge cloud node and the central management and control device for the data of the edge cloud node. processing to meet the user's demand for response speed, on the other hand, the second processing is performed on the central management and control device to meet the user's demand for data computing and storage capacity, thereby improving user experience.
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.
附图说明Description of drawings
通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本申请的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustrating preferred embodiments only and are not to be considered limiting of the application. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:
图1a为本申请示例性实施例提供的一种网络系统的结构示意图；FIG. 1a is a schematic structural diagram of a network system provided by an exemplary embodiment of the present application;
图1b为本申请示例性实施例提供的另一种网络系统的结构示意图；FIG. 1b is a schematic structural diagram of another network system provided by an exemplary embodiment of the present application;
图2为本发明提供的数据处理系统实施例的系统框图；2 is a system block diagram of an embodiment of a data processing system provided by the present invention;
图3为本发明提供的数据处理方法一个实施例的流程图；3 is a flowchart of an embodiment of a data processing method provided by the present invention;
图4为本发明提供的数据处理方法另一个实施例的流程图；4 is a flowchart of another embodiment of a data processing method provided by the present invention;
图5为本发明提供的数据处理方法又一个实施例的流程图；5 is a flowchart of another embodiment of a data processing method provided by the present invention;
图6为本发明提供的数据处理方法再一个实施例的流程图；FIG. 6 is a flowchart of another embodiment of the data processing method provided by the present invention;
图7为本发明提供的电子设备实施例的结构示意图。FIG. 7 is a schematic structural diagram of an embodiment of an electronic device provided by the present invention.
具体实施方式Detailed ways
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.
与中心云或者传统的云计算平台相对应，边缘云是指相对靠近终端的云计算平台，或者说，与中心云或者传统的云计算平台相区别，中心云或者传统的云计算平台可以包括资源规模化且位置集中的数据中心，而边缘云节点覆盖的网络范围更广泛，也因此具备距离终端更近的特性，单个边缘云节点的资源规模较小，但是边缘云节点的数量多，多个边缘云节点构成了边缘云的组成部分。边缘云网络是基于中心云或者传统的云计算系统与终端之间的基础设施构建的网络。特别是边缘云网络能够与中心云或传统的云计算平台等中心网络、终端结合形成“云边端三体协同”的网络架构，在该网络架构中，可以将网络转发、存储、计算、智能化数据分析等任务放在各边缘云节点中处理，由于各边缘云节点更靠近终端，因此可以降低响应时延，减轻中心云或传统的云计算平台的压力，降低带宽成本。Corresponding to the central cloud or traditional cloud computing platform, the edge cloud refers to a cloud computing platform that is relatively close to the terminal. In other words, different from the central cloud or traditional cloud computing platform, the central cloud or traditional cloud computing platform can Large-scale and centrally located data centers, while edge cloud nodes cover a wider range of networks, and therefore have the characteristics of being closer to the terminal. The resource scale of a single edge cloud node is small, but the number of edge cloud nodes is large. Edge cloud nodes constitute an integral part of edge cloud. The edge cloud network is a network constructed based on the infrastructure between the central cloud or the traditional cloud computing system and the terminal. In particular, the edge cloud network can be combined with the central cloud or traditional cloud computing platform and other central networks and terminals to form a "cloud, edge and terminal three-body collaboration" network architecture. In this network architecture, network forwarding, storage, computing, intelligent Tasks such as data analysis are processed in each edge cloud node. Since each edge cloud node is closer to the terminal, it can reduce the response delay, reduce the pressure on the central cloud or traditional cloud computing platform, and reduce bandwidth costs.
上述实施例是对本发明实施例的技术原理的说明，下面通过多个实施例来进一步对本发明实施例具体技术方案进行详细描述。The foregoing embodiments are descriptions of the technical principles of the embodiments of the present invention, and the specific technical solutions of the embodiments of the present invention are further described in detail below through multiple embodiments.
实施例一Example 1
图1a为本申请示例性实施例提供的一种网络系统的结构示意图。图1b为本申请示例性实施例提供的另一种网络系统的结构示意图。如图1a和图1b所示，本实施例提供的数据处理系统包括：至少一个边缘云节点102和中心管控设备101。至少一个边缘云节点102分别与中心管控设备101通过网络通道连接。FIG. 1a is a schematic structural diagram of a network system provided by an exemplary embodiment of the present application. FIG. 1b is a schematic structural diagram of another network system provided by an exemplary embodiment of the present application. As shown in FIG. 1 a and FIG. 1 b , the data processing system provided in this embodiment includes: at least one
本实施例的数据处理系统是基于云计算技术和边缘计算的能力，构筑在边缘基础设施之上的云计算平台，是一种边缘位置的具备计算、网络、存储、安全等能力的云平台。The data processing system of this embodiment is a cloud computing platform built on edge infrastructure based on cloud computing technology and edge computing capabilities, and is a cloud platform with computing, network, storage, security and other capabilities at the edge.
与中心云或者传统的云计算平台相对应，本实施例的数据处理系统可以视为一种混合的边缘云网络系统。边缘云是个相对概念，边缘云是指相对靠近终端的云计算平台，或者说，与中心云或者传统的云计算平台相区别，中心云或者传统的云计算平台可以包括资源规模化且位置集中的数据中心，而边缘云节点覆盖的网络范围更广泛，也因此具备距离终端更近的特性，单个边缘云节点的资源规模较小，但是边缘云节点的数量多，多个边缘云节点构成了本实施例中边缘云的组成部分。本实施例的终端是指云计算服务的需求端，例如可以是互联网中的终端或者用户端，或者物联网中的终端或用户端。边缘云网络是基于中心云或者传统的云计算系统与终端之间的基础设施构建的网络。其中，数据处理系统包括至少一个边缘云节点102，每个边缘云节点102包括一系列的边缘基础设施，这些边缘基础设施包括但不限于：分布式数据中心(DC)、无线机房或集群，运营商的通信网络、核心网设备、基站、边缘网关、家庭网关、计算设备、存储设备等边缘设备及对应的网络环境等等。在此说明，不同边缘云节点102的位置、能力以及包含的基础设施可以相同，也可以不相同。Corresponding to a central cloud or a traditional cloud computing platform, the data processing system of this embodiment may be regarded as a hybrid edge cloud network system. Edge cloud is a relative concept. Edge cloud refers to a cloud computing platform that is relatively close to the terminal. In other words, it is different from the central cloud or traditional cloud computing platform. Data centers, while edge cloud nodes cover a wider range of networks, and therefore have the characteristics of being closer to the terminal. The resource scale of a single edge cloud node is small, but the number of edge cloud nodes is large, and multiple edge cloud nodes constitute the Components of the edge cloud in the embodiment. The terminal in this embodiment refers to the demand side of the cloud computing service, for example, it may be a terminal or a user terminal in the Internet, or a terminal or a user terminal in the Internet of Things. The edge cloud network is a network constructed based on the infrastructure between the central cloud or the traditional cloud computing system and the terminal. The data processing system includes at least one
其中，本实施例的数据处理系统与中心云或传统的云计算平台等中心网络、终端结合可形成“云边端三体协同”的网络架构，在该网络架构中，可以将网络转发、存储、计算、智能化数据分析等任务放在数据处理系统中的各边缘云节点102中处理，由于各边缘云节点102更靠近终端，因此可以降低响应时延，减轻中心云或传统的云计算平台的压力，降低带宽成本。Among them, the data processing system in this embodiment can be combined with a central network such as a central cloud or a traditional cloud computing platform, and terminals to form a network architecture of "cloud-side-terminal three-body collaboration". In this network architecture, the network can be forwarded, stored Tasks such as computing, intelligent data analysis, etc. are processed in each
如何合理地调度多个边缘云节点资源，以及如何管控好多个边缘云节点以正确和稳定的逻辑进行云计算服务，是一个重要的挑战。在本实施例的网络系统100中，部署有中心管控设备101，中心管控设备101以边缘云节点102为管控对象，在资源调度，镜像管理，实例管控，运维，网络，安全等各方面对数据处理系统中的至少一个边缘云节点102进行统一管控，从而将云计算服务放到各边缘云节点102中处理。在部署实施上，中心管控设备101可以部署在一个或多个云计算数据中心中，或者，可以部署在一个或多个传统数据中心中，中心管控设备101也可以和其管控的至少一个边缘云节点共同构成边缘云网络，本实施例对此不做限定。How to reasonably schedule the resources of multiple edge cloud nodes and how to manage and control multiple edge cloud nodes to perform cloud computing services with correct and stable logic is an important challenge. In the network system 100 of this embodiment, a central management and
对一个边缘云节点102来说，可以对外提供各种资源，例如CPU、GPU等计算资源，内存、硬盘等存储资源，带宽等网络资源等。另外，边缘云节点102还可以根据镜像创建相应实例，通过实例对外提供各种云计算服务。其中，镜像是在边缘云节点中创建实例所需的基础文件，例如可以是为用户提供云计算服务所需的操作系统、应用、或操作配置等镜像文件，其可以是符合边缘云节点计算部署要求，根据特定的一系列文件按照一定的格式制作成的文件。另外，镜像的形态是多样的，可以是虚拟机(Virtual Machine，VM)镜像文件、容器(Docker)镜像文件、各类型的应用打包文件等，镜像形态可以与云计算服务需要使用的虚拟化技术有关，本实施例对此不做限定。与镜像对应，实例的实现形态可以是虚拟机、容器或应用程序等。For an
值得说明的是，在数据处理系统中，中心管控设备101可以直接对至少一个边缘云节点102进行管控和调度，但并不限于此。例如，可以在一个或多个边缘云节点102中额外地设置一个或多个边缘云管控设备。例如图1b中所示的边缘云管控设备103。可选地，边缘云管控设备103可集中部署在一台设备上，也可以分散部署在多台设备上。中心管控设备101可以部署在一个或多个云计算数据中心或传统数据中心中，也可以和至少一个边缘云节点一起部署在边缘云网络中。边缘云管控设备103可辅助、配合中心管控设备101对至少一个边缘云节点102进行管控和调度。在边缘云管控设备103的协助下，中心管控设备101可以更加方便、高效地对至少一个边缘云节点102进行管控和调度，进而达到充分利用边缘资源的目的。It should be noted that, in the data processing system, the central management and
在本发明实施例中，边缘云节点102为近用户侧的边缘云中的边缘节点，用于对其获取到的第一数据进行第一处理，例如进行数据去重、除错等处理操作，以生成可以向用户反馈的第二数据，从而满足用户对响应速度的需求。然后，边缘云节点102将该第二数据通过网络通道发送至中心云管控设备101进行进一步处理，以满足用户对数据就计算能力的需求。中心云管控设备101可以为传统的由集中式数据中心构成的为用户提供云计算服务的云服务平台。中心云管控设备101在接收到边缘云节点102上传的第二数据后，对该第二数据进行第二处理，例如进行深入的计算处理等操作，以生成第一数据的目标处理结果数据。也就是说，在边缘云节点102获取了第一数据后，对其进行处理的最终目的是为了获取第一数据的目标处理结果数据，在本发明实施例的方案中，将对第一数据的处理分为两部分，即，由边缘云节点102本身执行的第一处理以及由中心云管控设备101执行的第二处理，经过这两级处理后，可同时满足用户对响应速度和对计算能力的需求。In the embodiment of the present invention, the
本发明实施例提供的边缘云的数据处理系统，针对边缘云节点的数据，通过在边缘云节点和中心管控设备中进行两级处理，即，一方面在边缘云节点进行第一处理，以满足用户对响应速度的需求，另一方面在中心管控设备进行第二处理，以满足用户对数据的计算存储能力的需求，从而提高了用户体验。The data processing system of the edge cloud provided by the embodiment of the present invention performs two-level processing in the edge cloud node and the central management and control device for the data of the edge cloud node. The user's demand for response speed, on the other hand, the second processing is performed on the central control device to meet the user's demand for data computing and storage capacity, thereby improving the user experience.
实施例二Embodiment 2
图2为本发明提供的数据处理系统实施例的系统框图。如图2所示，本实施例提供了一种边缘云的数据处理系统，该系统可以包括多个边缘云节点，图2中仅示出一个边缘云节点作为示例。在该系统中，边缘云节点102可以包括：数据采集模块11、第一处理模块12和数据上报模块13。FIG. 2 is a system block diagram of an embodiment of a data processing system provided by the present invention. As shown in FIG. 2 , this embodiment provides an edge cloud data processing system, and the system may include multiple edge cloud nodes, and only one edge cloud node is shown as an example in FIG. 2 . In this system, the
在本发明实施例的边缘云节点102中，数据采集模块11获取各种数据，即，第一数据，该第一数据可以包括但不限于：物理机数据、虚拟机数据、业务模块数据、实例带宽流量、实例当前运行情况、实例IO负载、物理机带宽流量、物理机当前运行情况、物理机IO负载、边缘管控模块运行情况、虚拟化组件运行情况等。在数据采集模块11获取了第一数据后，第一处理模块12对该第一数据进行第一处理生成第二数据，该第一处理可以包括但不限于：对第一数据的去重处理、除错处理、归档处理、合并处理等。然后，由数据上报模块13将第一处理模块12生成的第二数据通过高安全性、高可用性的网络通道发送至中心管控设备101进行后续的处理。In the
进一步地，边缘云节点102还可以包括：数据缓存模块14。当边缘云节点102与中心管控设备101之间的网络通道发生异常而无法正常传输数据时，数据上报模块13可以将第一处理模块12生成的第二数据发送至数据缓存模块14进行缓存处理，缓存的类型可以包括但不限于：磁盘缓存、内存缓存、数据库缓存等；在网络通道恢复正常时，数据上报模块13则再将数据缓存模块14缓存的第二数据通过网络通道重新发送至中心管控设备101。Further, the
此外，根据本申请实施例，边缘云节点102可以进一步包括边缘云管控设备，例如，如图1中所示的边缘云管控设备103，对应的，其可以对上述边缘云节点102的各模块进行管控，并负责与中心管控设备101进行数据交互。例如，边缘云管控设备可以根据第一处理模块12对第一数据的处理结果指示数据上报模块13将第二数据上报给中心管控设备101，或者当边缘云管控设备监控到边缘云节点102与中心管控设备101之间的网络通道发生异常而无法正常传输数据的情况时，可以指示数据缓存模块14对第一处理模块12生成的第二数据进行缓存，并且在监控到网络通道恢复正常时，再指示数据上报模块13将第二数据重新发送至中心管控设备101。In addition, according to the embodiment of the present application, the
此外，中心管控设备101可以包括：数据接收模块26，用于通过网络通道接收边缘云节点发送的数据。In addition, the central management and
另外，中心管控设备101可以包括：第二处理模块21。当边缘云节点1中的数据上报模块13将第二数据发送至中心管控设备101的数据接收模块26时，中心管控设备101中的第二处理模块21则对该第二数据进行第二处理，例如进行深入的计算处理等操作，或者，也可以进行去重处理、除错处理、归档处理、合并处理等操作，以生成第一数据的目标处理结果数据。In addition, the central management and
进一步地，中心管控设备101还可以包括数据发送模块25，用于通过网络通道向边缘云节点发送第六数据以供所述边缘云节点进行第六处理生成第六数据的目标处理结果数据，第六数据是中心管控设备101通过对获取的第五数据进行第五处理而生成的。Further, the central management and
在该情况下，如上所述，中心管控设备101的数据发送模块25也可以将第六数据发送给边缘云节点的边缘云管控设备，从而边缘云管控设备在接收到该数据时，指示第一处理模块对该数据进行处理，并且根据第一处理模块的处理结果，指示数据上报模块将处理结果，即，第六数据的目标处理结果数据发送给中心管控设备101。In this case, as described above, the
进一步地，中心管控设备101还可以包括：数据存储模块22。在第二数据处理模块21生成第三数据后，可以将第三数据存储到数据存储模块22进行集中式存储，以实现数据持久化存储。Further, the central management and
根据本申请的实施例，中心管控设备101也可以在第二处理模块21生成第三数据后，将第三数据存储到其他数据存储设备，例如，传统的云存储设备或者专用的数据存储设备中，以实现数据持久化存储。According to the embodiment of the present application, the central management and
此外，根据本申请实施例，边缘云节点可以采集各种设备信息，以进行第一处理，并且中心管控设备在接收到由边缘云节点采集到的设备信息之后，根据该设备信息查找对应的用户信息，以进行第二处理，从而生成目标数据。In addition, according to the embodiment of the present application, the edge cloud node can collect various device information to perform the first processing, and after receiving the device information collected by the edge cloud node, the central management and control device searches for the corresponding user according to the device information information for second processing to generate target data.
例如，在边缘云节点处理IO数据的情况下，边缘节点可以采集产生的设备所使用的IO数据，但是由于与该IO数据关联的用户信息并未存储在该边缘云节点上，因此，边缘云节点需要将该IO数据与该设备的标识信息(例如，设备ID等)一起发送给中心管控设备，从而中心管控设备可以利用设备信息来查找与该IO数据相对应的用户信息，从而基于该IO数据与根据设备信息查找到的用户信息来生成最终的IO指标数据并发送给用户。另外，中心管控设备也可以检查用户的IO指标数据，并且根据IO指标数据来通知边缘云节点进行相应的操作。For example, when an edge cloud node processes IO data, the edge node can collect the IO data used by the generated devices, but since the user information associated with the IO data is not stored on the edge cloud node, the edge cloud The node needs to send the IO data together with the identification information of the device (for example, device ID, etc.) to the central management and control device, so that the central management and control device can use the device information to find the user information corresponding to the IO data. The data and the user information found according to the device information generate the final IO indicator data and send it to the user. In addition, the central management and control device can also check the user's IO indicator data, and notify the edge cloud nodes to perform corresponding operations according to the IO indicator data.
另外，要说明的是，边缘云节点因为是靠近用户侧的节点，因此可以采集实际使用的设备的各种信息，例如，流量信息、设备标识等。而中心管控设备由于设置在大型集中服务器上，因此能够存储有更多的其他信息，例如，用户标识等。In addition, it should be noted that because the edge cloud node is a node close to the user side, it can collect various information of the device actually used, such as traffic information, device identification, and so on. On the other hand, the central management and control device can store more other information, such as user IDs, because it is set on a large centralized server.
更进一步地，中心管控设备101还可以包括：数据巡检模块23。该数据巡检模块23可以获取第二数据的目标处理信息，该目标处理信息用于描述针对第二数据的目标处理操作任务，即，通过获取第二数据的目标处理信息，可以分析出针对该数据将要进行哪些类型的处理，最终将得到哪些类型的数据。该目标处理信息可以从边缘云节点102获取得到，或者也可以根据第二数据的某个或某些属性而在中心管控设备101中查找得到。然后，数据巡检模块23判断该目标处理信息与目标处理结果数据是否匹配，若匹配，则表明获取到的第二数据是准确的，对该第二数据执行的第二处理操作也是准确的，从而生成的目标处理结果数据也是准确的；若不匹配，则需要通过网络通道重新从边缘云节点102获取第二数据，即，重新主动向该边缘云节点102同步最新的数据，以纠正数据内容。Further, the central management and
此外，根据本申请实施例，在第二数据或其目标处理信息中可以进一步包含有标识信息，用于标识该第二数据与其目标处理信息的对应关系。从而数据巡检模块23可以在判断目标处理信息与目标处理结果数据是否匹配之前，先判断该目标处理信息是否属于用于生成该目标处理结果数据所基于的第二数据，即检查目标处理信息与该第二数据的对应关系。根据本申请的实施例，数据巡检模块23也可以在获取目标处理信息之前，先获取第二数据中包含的标识信息，并根据该标识信息来进一步获取该第二数据的目标处理信息。In addition, according to the embodiment of the present application, the second data or its target processing information may further include identification information for identifying the correspondence between the second data and its target processing information. Therefore, before judging whether the target processing information matches the target processing result data, the
再进一步地，中心管控设备101还可以包括数据接口模块24。当接收到各种业务系统对数据的操作指令时，数据接口模块24可以根据接收到的操作指令，对存储于数据存储模块22的第三数据进行去重、合并、增加、查询、修改或删除等操作。本发明实施例提供的边缘云的数据处理系统，针对边缘云节点的数据，通过在边缘云节点和中心管控设备中进行两级处理，即，一方面在边缘云节点进行第一处理，以满足用户对响应速度的需求，另一方面在中心管控设备进行第二处理，以满足用户对数据的计算存储能力的需求，从而提高了用户体验；当网络通道无法正常传输数据时，在边缘云节点缓存已进行第一处理的数据，在恢复正常传输后能够将缓存的数据发往中心管控设备以进行进一步处理；中心管控设备通过对已存储的数据进行检查，来发现存在问题的数据，以重新获取相关数据并进行相关操作，从而提高了用户体验。Still further, the central management and
实施例三Embodiment 3
图3为本发明提供的数据处理方法一个实施例的流程图，该方法的执行主体可以为上述数据处理系统中的中心管控设备，也可以为具有数据处理能力的各种服务器，也可以为集成在这些设备上的装置或芯片。如图3所示，本实施例提供的数据处理方法包括以下步骤：3 is a flowchart of an embodiment of the data processing method provided by the present invention. The execution body of the method may be the central management and control device in the above-mentioned data processing system, or may be various servers with data processing capabilities, or may be integrated devices or chips on these devices. As shown in Figure 3, the data processing method provided by this embodiment includes the following steps:
S301，通过网络通道接收边缘云节点发送的第三数据。S301. Receive third data sent by an edge cloud node through a network channel.
在本发明实施例中，边缘云节点为近用户侧的边缘云中的边缘节点，中心管控设备可以从边缘云节点接收其对第一数据进行第一处理而生成的第三数据，以便于对其进行进一步的处理。In this embodiment of the present invention, the edge cloud node is an edge node in the edge cloud near the user side, and the central management and control device may receive third data generated by performing the first processing on the first data from the edge cloud node, so as to facilitate the processing of the first data. It undergoes further processing.
S302，对第三数据进行第三处理，生成第二数据。S302: Perform third processing on the third data to generate second data.
在本申请实施例中，第三数据可以为边缘云节点所获取的第一数据的目标处理结果数据。也就是说，在边缘云节点采集到某第一数据后，对其进行处理的最终目的是为了获取第二数据(即，第一数据的目标处理结果数据)，在本发明实施例的方案中，将对第一数据的处理分为两部分，即，由边缘云节点本身执行的第一处理以及由中心管控设备执行的第二处理，经过这两级处理后，可同时满足用户对响应速度和对计算能力的需求。In this embodiment of the present application, the third data may be target processing result data of the first data acquired by the edge cloud node. That is to say, after the edge cloud node collects a certain first data, the final purpose of processing it is to obtain the second data (that is, the target processing result data of the first data). In the solution of the embodiment of the present invention , the processing of the first data is divided into two parts, that is, the first processing performed by the edge cloud node itself and the second processing performed by the central control device. and the need for computing power.
要说明的是，在边缘云节点包括边缘云管控设备的情况下，本申请的实施例中描述的由边缘云节点执行的与中心管控设备的交互操作，也可以由边缘云管控设备来管理执行。本发明实施例提供的数据处理方法，针对边缘云节点的数据，通过在中心管控设备中对经由边缘云节点进行了初步处理的数据进行进一步的处理或者也可以由中心管控设备先进行初步处理并将处理结果发送给边缘云节点来进行进一步处理来获得目标处理结果数据，实现了用户侧的高响应速度，另一方面也满足用户对数据的计算存储能力的需求，从而提高了用户体验。It should be noted that, in the case where the edge cloud node includes an edge cloud management and control device, the interactive operations performed by the edge cloud node and the central management and control device described in the embodiments of this application may also be managed and executed by the edge cloud management and control device. . In the data processing method provided by the embodiment of the present invention, for the data of the edge cloud node, the data that has been preliminarily processed by the edge cloud node is further processed in the central management and control device, or the central management and control device can also perform preliminary processing and then The processing result is sent to the edge cloud node for further processing to obtain the target processing result data, which achieves high response speed on the user side, and on the other hand, meets the user's demand for data computing and storage capacity, thereby improving the user experience.
实施例四Embodiment 4
与图3的实施例相对的，中心管控设备与边缘云节点的数据交互不限于仅从边缘云节点获取数据，例如，中心管控设备也可以通过网络通道向边缘云节点发送六数据。In contrast to the embodiment in FIG. 3 , the data interaction between the central management and control device and the edge cloud nodes is not limited to obtaining data only from the edge cloud nodes. For example, the central management and control device can also send six data to the edge cloud nodes through a network channel.
此外，中心管控设备在获取到边缘云节点进行了初步处理的数据之后，也可以基于自身所拥有的数据或者从外部进一步获得的数据来对从边缘云节点获取的数据进行处理，从而生成目标处理结果数据。In addition, after the central management and control device obtains the data that has been preliminarily processed by the edge cloud node, it can also process the data obtained from the edge cloud node based on its own data or data obtained from the outside to generate target processing. result data.
要说明的是，在边缘云节点包括边缘云管控设备的情况下，本申请的实施例中描述的由边缘云节点执行的与中心管控设备的交互操作，也可以由边缘云管控设备来管理执行。在本申请实施例中，该第六数据是供边缘云节点进行第六处理生成所述第六数据的目标处理结果数据，并且该第六数据是中心管控设备通过对获取的第五数据进行第六处理而生成的。因此，可以由中心管控设备先进行初步处理并将处理结果发送给边缘云节点来进行进一步处理来获得该第六数据的目标处理结果数据，并且也可以通过在对边缘云节点初步处理的数据进行进一步处理时，引入外部数据源的数据来实现更好的数据处理结果。因此不仅实现了用户侧的高响应速度，另一方面也满足用户对数据的计算存储能力的需求，从而提高了用户体验。It should be noted that, in the case where the edge cloud node includes an edge cloud management and control device, the interactive operations performed by the edge cloud node and the central management and control device described in the embodiments of this application may also be managed and executed by the edge cloud management and control device. . In this embodiment of the present application, the sixth data is target processing result data for the edge cloud node to perform sixth processing to generate the sixth data, and the sixth data is the fifth data obtained by the central management and control device by performing the sixth processing on the sixth data. generated by six processing. Therefore, the central management and control device can first perform preliminary processing and send the processing results to the edge cloud nodes for further processing to obtain the target processing result data of the sixth data, and can also obtain the target processing result data of the sixth data by performing preliminary processing on the data initially processed by the edge cloud nodes. For further processing, data from external data sources is introduced to achieve better data processing results. Therefore, not only a high response speed on the user side is achieved, but on the other hand, the user's demand for data computing and storage capacity is met, thereby improving the user experience.
实施例五Embodiment 5
图4为本发明提供的数据处理方法另一个实施例的流程图。该方法的执行主体可以为上述数据处理系统中的边缘云节点，也可以为具有数据处理能力的各种服务器，也可以为集成在这些设备上的装置或芯片。如图4所示，本实施例提供的数据处理方法包括以下步骤：FIG. 4 is a flowchart of another embodiment of the data processing method provided by the present invention. The execution body of the method may be an edge cloud node in the above data processing system, or may be various servers with data processing capabilities, or may be devices or chips integrated on these devices. As shown in FIG. 4 , the data processing method provided by this embodiment includes the following steps:
S401，对第一数据进行第一处理，并生成第二数据。S401, perform first processing on the first data, and generate second data.
根据本申请实施例，边缘云节点可以获取各种数据作为第一数据，例如，物理机数据、虚拟机数据、业务模块数据、实例带宽流量、实例当前运行情况、实例IO负载、物理机带宽流量、物理机当前运行情况、物理机IO负载、边缘管控模块运行情况、虚拟化组件运行情况等。According to the embodiment of the present application, the edge cloud node can obtain various data as the first data, for example, physical machine data, virtual machine data, service module data, instance bandwidth traffic, current instance running status, instance IO load, physical machine bandwidth traffic , The current operation of the physical machine, the IO load of the physical machine, the operation of the edge management and control module, the operation of the virtualization components, etc.
边缘云节点获取了第一数据后，可以对该第一数据进行第一处理生成第二数据，该第一处理可以包括但不限于：对第一数据的去重处理、除错处理、归档处理、合并处理等。After the edge cloud node obtains the first data, it can perform first processing on the first data to generate second data, and the first processing can include but is not limited to: deduplication processing, debugging processing, and archiving processing on the first data , merge processing, etc.
S402，将第二数据通过网络通道发送给中心管控设备。S402: Send the second data to the central management and control device through a network channel.
根据本申请实施例，边缘云节点在生成了第二数据之后，可以将生成的第二数据通过例如高安全性、高可用性的网络通道发送至中心管控设备以进行后续的处理，从而能够经过中心管控设备的处理后生成第三数据。According to the embodiment of the present application, after generating the second data, the edge cloud node can send the generated second data to the central management and control device through, for example, a high-security and high-availability network channel for subsequent processing, so that it can pass through the central The third data is generated after processing by the control device.
因此，在边缘云节点采集到某第一数据后，对其进行处理的最终目的是为了获取第二数据(即，第一数据的目标处理结果数据)，在本发明实施例的方案中，将对第一数据的处理分为两部分，即，由边缘云节点本身执行的第一处理以及由中心管控设备执行的第二处理，经过这两级处理后，可同时满足用户对响应速度和对计算能力的需求。Therefore, after the edge cloud node collects a certain first data, the final purpose of processing it is to obtain the second data (that is, the target processing result data of the first data). In the solution of the embodiment of the present invention, the The processing of the first data is divided into two parts, that is, the first processing performed by the edge cloud node itself and the second processing performed by the central management and control equipment. computing power needs.
要说明的是，在边缘云节点包括边缘云管控设备的情况下，本申请的实施例中描述的由边缘云节点执行的与中心管控设备的交互操作，也可以由边缘云管控设备来管理执行。It should be noted that, in the case where the edge cloud node includes an edge cloud management and control device, the interactive operations performed by the edge cloud node and the central management and control device described in the embodiments of this application may also be managed and executed by the edge cloud management and control device. .
本发明实施例提供的数据处理方法，边缘云节点可以先获取第一数据并进行初步的处理，以最终在中心管控设备处完成进一步的处理。因此，通过在边缘云节点和中心管控设备中进行两级处理，即，一方面在边缘云节点进行基础处理，以满足用户对响应速度的需求，另一方面在中心管控设备中进行进一步处理，以满足用户对数据的计算存储能力的需求，从而提高了用户体验。In the data processing method provided by the embodiment of the present invention, the edge cloud node may first obtain the first data and perform preliminary processing, so as to finally complete further processing at the central management and control device. Therefore, by performing two-level processing in the edge cloud node and the central control device, on the one hand, basic processing is performed on the edge cloud node to meet the user's demand for response speed, and on the other hand, further processing is performed in the central control device. In order to meet the user's demand for data computing and storage capacity, the user experience is improved.
实施例六Embodiment 6
图5为本发明提供的数据处理方法又一个实施例的流程图。如图5所示，在上述图4所示实施例的基础上，本实施例提供的数据处理方法还可以包括以下步骤：FIG. 5 is a flowchart of another embodiment of the data processing method provided by the present invention. As shown in FIG. 5 , on the basis of the embodiment shown in FIG. 4 above, the data processing method provided by this embodiment may further include the following steps:
S501，对第一数据进行第一处理，生成第二数据。S501. Perform first processing on the first data to generate second data.
在本发明实施例中，步骤S501与上述图4所示实施例中的步骤S401相同，详见图4所示实施例的具体描述，在此不再赘述。In this embodiment of the present invention, step S501 is the same as step S401 in the above-mentioned embodiment shown in FIG. 4 . For details, please refer to the specific description of the embodiment shown in FIG. 4 , which will not be repeated here.
S502，当网络通道异常时，对第二数据进行缓存处理。S502, when the network channel is abnormal, perform cache processing on the second data.
S503，在网络通道恢复正常时，将缓存的第二数据通过网络通道重新发送至中心管控设备。S503, when the network channel returns to normal, resend the cached second data to the central management and control device through the network channel.
在本发明实施例中，当边缘云节点与中心管控设备之间的网络通道发生异常而无法正常传输数据时，可以将步骤S501生成的第二数据进行缓存处理，缓存的类型可以包括但不限于：磁盘缓存、内存缓存、数据库缓存等；在网络通道恢复正常时，则再将缓存的第二数据通过网络通道重新发送至中心管控设备。In this embodiment of the present invention, when the network channel between the edge cloud node and the central management and control device is abnormal and data cannot be transmitted normally, the second data generated in step S501 may be cached, and the type of the cache may include but not limited to : Disk cache, memory cache, database cache, etc. When the network channel returns to normal, the second cached data will be resent to the central management and control device through the network channel.
S504，将第二数据通过网络通道发送至中心管控设备。S504, the second data is sent to the central management and control device through the network channel.
在本发明实施例中，步骤S504与上述图4所示实施例中的步骤S402相同，详见图4所示实施例的具体描述，在此不再赘述。In this embodiment of the present invention, step S504 is the same as step S402 in the above-mentioned embodiment shown in FIG. 4 . For details, please refer to the specific description of the embodiment shown in FIG. 4 , which will not be repeated here.
要说明的是，在边缘云节点包括边缘云管控设备的情况下，本申请的实施例中描述的由边缘云节点执行的与中心管控设备的交互操作，也可以由边缘云管控设备来管理执行。It should be noted that, in the case where the edge cloud node includes an edge cloud management and control device, the interactive operations performed by the edge cloud node and the central management and control device described in the embodiments of this application may also be managed and executed by the edge cloud management and control device. .
本发明实施例提供的数据处理方法，当网络通道无法正常传输数据时，在边缘云节点缓存已进行基础处理的数据，在恢复正常传输后能够将缓存的数据发往中心管控设备以进行进一步处理，从而避免了由于网络故障导致的数据丢失等问题。According to the data processing method provided by the embodiment of the present invention, when the network channel cannot transmit data normally, the data that has been basically processed is cached at the edge cloud node, and after the normal transmission is restored, the cached data can be sent to the central management and control device for further processing. , so as to avoid problems such as data loss due to network failures.
实施例七Embodiment 7
图6为本发明提供的数据处理方法再一个实施例的流程图。如图6所示，在上述图3所示实施例的基础上，本实施例提供的数据处理方法还可以包括以下步骤：FIG. 6 is a flowchart of another embodiment of the data processing method provided by the present invention. As shown in FIG. 6 , on the basis of the above-mentioned embodiment shown in FIG. 3 , the data processing method provided in this embodiment may further include the following steps:
S601，通过网络通道从边缘云节点接收第二数据。S601. Receive second data from an edge cloud node through a network channel.
在本发明实施例中，边缘云节点为近用户侧的边缘云中的边缘节点，中心管控设备可以从边缘云节点接收其对第一数据进行第一处理而生成的第二数据，以便于对其进行进一步的处理。In this embodiment of the present invention, the edge cloud node is an edge node in the edge cloud near the user side, and the central management and control device may receive the second data generated by first processing the first data from the edge cloud node, so as to facilitate the processing of the first data. It undergoes further processing.
S602，对第二数据进行第二处理，生成第三数据。S602: Perform second processing on the second data to generate third data.
在本发明实施例中，步骤S601至S602与上述图3所示实施例中的步骤S401和S402相同，详见图3所示实施例的具体描述，在此不再赘述。In this embodiment of the present invention, steps S601 to S602 are the same as steps S401 and S402 in the above-mentioned embodiment shown in FIG. 3 . For details, refer to the specific description of the embodiment shown in FIG. 3 , which will not be repeated here.
S603，获取第二数据的目标处理信息。S603, acquiring target processing information of the second data.
S604，判断目标处理信息与第三数据是否匹配，若不匹配，则通过网络通道重新从边缘云节点获取第二数据，以进行第二处理。S604: Determine whether the target processing information matches the third data, and if not, obtain the second data from the edge cloud node again through the network channel to perform the second processing.
在本发明实施例中，目标处理信息用于描述针对第二数据的目标处理操作任务。通过获取第二数据的目标处理信息，可以分析出针对该数据将要进行哪些类型的处理，最终将得到哪些类型的数据。该目标处理信息可以从边缘云节点获取得到，也可以根据第二数据的某个或某些属性而在中心管控设备中查找得到。然后，判断该目标处理信息与第三数据是否匹配，若匹配，则表明获取到的第二数据是准确的，对该第二数据执行的第二处理操作也是准确的，从而生成的第三数据也是准确的；若不匹配，则需要通过网络通道重新从边缘云节点获取第二数据，即，重新主动向该边缘云节点同步最新的数据，以纠正数据内容。In this embodiment of the present invention, the target processing information is used to describe a target processing operation task for the second data. By acquiring the target processing information of the second data, it is possible to analyze what types of processing will be performed on the data, and what types of data will be finally obtained. The target processing information can be obtained from the edge cloud node, or can be searched in the central management and control device according to some or some attributes of the second data. Then, it is judged whether the target processing information matches the third data. If they match, it indicates that the acquired second data is accurate, and the second processing operation performed on the second data is also accurate, so that the generated third data It is also accurate; if it does not match, it is necessary to obtain the second data from the edge cloud node again through the network channel, that is, to actively synchronize the latest data to the edge cloud node again to correct the data content.
根据本申请实施例，在第二数据或其目标处理信息中可以进一步包含有标识信息，用于标识该第二数据与其目标处理信息的对应关系。例如，在步骤S603之后可以进一步包括：According to the embodiment of the present application, identification information may be further included in the second data or its target processing information, which is used to identify the corresponding relationship between the second data and its target processing information. For example, after step S603, it may further include:
S6031，判断目标处理信息是否属于第二数据。S6031, determine whether the target processing information belongs to the second data.
通过检查目标处理信息与该第二数据的对应关系，可以在判断目标处理信息是否与第三数据匹配时确保使用的是正确的目标处理信息。By checking the correspondence between the target processing information and the second data, it can be ensured that the correct target processing information is used when judging whether the target processing information matches the third data.
此外，根据本申请的实施例，如果S601步骤中获取的第二数据中已包含有标识信息，则也可以根据该标识信息来获取该第二数据的目标处理信息。In addition, according to the embodiment of the present application, if the second data obtained in step S601 already contains identification information, the target processing information of the second data may also be obtained according to the identification information.
S605，存储第三数据。S605, store the third data.
在本发明实施例中，在生成第三数据后，可以对第三数据进行集中式存储，以实现数据持久化存储。In the embodiment of the present invention, after the third data is generated, the third data may be stored in a centralized manner to realize persistent data storage.
S606，在接收到的操作指令后，根据操作指令，对存储的第三数据进行去重、合并、增加、查询、修改或删除操作。S606, after receiving the operation instruction, perform deduplication, merging, adding, querying, modifying or deleting operations on the stored third data according to the operation instruction.
在本发明实施例中，当接收到各种业务系统对数据的操作指令时，中心管控设备可以根据接收到的操作指令，对存储的第三数据进行去重、合并、增加、查询、修改或删除等操作。In the embodiment of the present invention, when receiving operation instructions on data from various business systems, the central management and control device may de-duplicate, merge, add, query, modify, or modify the stored third data according to the received operation instructions. delete, etc.
本发明实施例提供的边缘云的数据处理方法，中心管控设备可通过对已存储的数据进行巡检，来发现存在问题的数据，以重新获取相关数据并进行相关操作，并且还可以根据标识信息来进一步检查第二数据与目标处理信息的对应性，从而提高了用户体验。In the data processing method of the edge cloud provided by the embodiment of the present invention, the central management and control device can inspect the stored data to find problematic data, so as to re-obtain relevant data and perform relevant operations, and can also perform related operations according to the identification information. to further check the correspondence between the second data and the target processing information, thereby improving the user experience.
如前所述，边缘云节点中可以包含边缘管控设备、计算设备、存储设备、网络设备等各种角色，而所有涉及到逻辑交互和管理控制的可以由边缘云管控设备执行，比如数据交互、资源调度、监控运维等，其他几种设备主要是执行自己角色相关的服务，比如存储设备负责存储资源的创建、释放、服务等动作的执行。因此，在边缘云节点包括边缘管控设备的情况下，本申请的上述实施例中描述的由边缘云节点执行的与中心管控设备的交互操作，都可以由边缘管控设备来管理执行。As mentioned above, edge cloud nodes can contain various roles such as edge management and control devices, computing devices, storage devices, and network devices, and all those involving logical interaction and management control can be performed by edge cloud management and control devices, such as data interaction, Resource scheduling, monitoring operation and maintenance, etc., other devices mainly perform their own role-related services. For example, storage devices are responsible for the creation, release, and service execution of storage resources. Therefore, in the case where the edge cloud node includes an edge management and control device, the interaction operations performed by the edge cloud node with the central management and control device described in the above embodiments of this application can all be managed and executed by the edge management and control device.
实施例八Embodiment 8
以上描述了边缘云的数据处理系统的内部功能和结构，该系统可实现为一种电子设备。图7为本发明提供的电子设备实施例的结构示意图。如图7所示，该电子设备包括存储器51和处理器52。The internal functions and structures of the data processing system of the edge cloud are described above, and the system can be implemented as an electronic device. FIG. 7 is a schematic structural diagram of an embodiment of an electronic device provided by the present invention. As shown in FIG. 7 , the electronic device includes a
存储器51，用于存储程序。除上述程序之外，存储器51还可被配置为存储其它各种数据以支持在电子设备上的操作。这些数据的示例包括用于在电子设备上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。The
存储器51可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
处理器52，与存储器51耦合，执行存储器51所存储的程序，以执行下述操作：The
控制边缘云节点对第一数据进行第一处理，生成第二数据；controlling the edge cloud node to perform the first processing on the first data to generate the second data;
控制边缘云节点将第二数据通过网络通道发送至中心管控设备；Control the edge cloud node to send the second data to the central management and control device through the network channel;
控制中心管控设备对第二数据进行第二处理，生成第三数据，该第三数据为第一数据的目标处理结果数据。The control center management device performs second processing on the second data to generate third data, where the third data is target processing result data of the first data.
根据本申请实施例，处理器52可以包括设置在边缘云节点上的各种模块，例如，如图2中所示的数据采集模块11、第一数据处理模块12和数据上报模块13，以便于控制边缘云节点执行各种操作。此外，处理器52还可以包括设置在中心管控设备上的各种模块，例如，如图2中所示的第二数据处理模块21、数据存储模块22和数据巡检模块23，以便于控制中心管控设备执行各种操作。According to the embodiment of the present application, the
进一步，如图7所示，电子设备还可以包括：通信组件53、电源组件54、音频组件55、显示器56等其它组件。图7中仅示意性给出部分组件，并不意味着电子设备只包括图7所示组件。Further, as shown in FIG. 7 , the electronic device may further include: a
通信组件53被配置为便于电子设备和其他设备之间有线或无线方式的通信。电子设备可以接入基于通信标准的无线网络，如WiFi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件53经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件53还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。The
电源组件54，为电子设备的各种组件提供电力。电源组件54可以包括电源管理系统，一个或多个电源，及其他与为电子设备生成、管理和分配电力相关联的组件。The
音频组件55被配置为输出和/或输入音频信号。例如，音频组件55包括一个麦克风(MIC)，当电子设备处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器51或经由通信组件53发送。在一些实施例中，音频组件55还包括一个扬声器，用于输出音频信号。
显示器56包括屏幕，其屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by program instructions related to hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps including the above method embodiments are executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention. scope.
Claims (37)
Translated fromChinesePriority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910277959.2ACN111800443B (en) | 2019-04-08 | 2019-04-08 | Data processing system and method, device and electronic equipment |
| PCT/CN2020/083478WO2020207371A1 (en) | 2019-04-08 | 2020-04-07 | Data processing system and method, apparatus, and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910277959.2ACN111800443B (en) | 2019-04-08 | 2019-04-08 | Data processing system and method, device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111800443A CN111800443A (en) | 2020-10-20 |
| CN111800443Btrue CN111800443B (en) | 2022-04-29 |
Family
ID=72751908
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910277959.2AActiveCN111800443B (en) | 2019-04-08 | 2019-04-08 | Data processing system and method, device and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111800443B (en) |
| WO (1) | WO2020207371A1 (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112769891B (en) | 2020-12-11 | 2023-07-14 | 国汽(北京)智能网联汽车研究院有限公司 | An edge cloud system, server and cloud control platform |
| CN113487033B (en)* | 2021-07-30 | 2023-05-23 | 上海壁仞智能科技有限公司 | Reasoning method and device using graphic processor as execution core |
| CN116775624A (en)* | 2022-03-10 | 2023-09-19 | 国能智深控制技术有限公司 | A data management method and system for a coal preparation plant |
| CN117240462A (en)* | 2022-06-07 | 2023-12-15 | 华为云计算技术有限公司 | Cloud data processing system, method and related equipment |
| CN116743791B (en)* | 2022-09-30 | 2024-06-18 | 腾讯云计算（北京）有限责任公司 | Cloud edge synchronization method, device and equipment for subway cloud platform and storage medium |
| CN115358914B (en)* | 2022-10-20 | 2023-01-20 | 深圳市壹倍科技有限公司 | Data processing method and device for visual detection, computer equipment and medium |
| CN115361388B (en)* | 2022-10-20 | 2023-04-11 | 阿里巴巴（中国）有限公司 | Resource scheduling method and device in edge cloud computing system |
| CN115840726B (en)* | 2022-11-22 | 2025-09-23 | 湖北华中电力科技开发有限责任公司 | A method and device for processing node serial data acquisition |
| CN116132488B (en)* | 2022-12-08 | 2025-05-23 | 南方电网数字平台科技(广东)有限公司 | Physical terminal control method and system based on edge technology |
| CN116389584B (en)* | 2023-05-17 | 2023-08-15 | 南方电网调峰调频发电有限公司储能科研院 | Cloud edge data transmission method for electrochemical energy storage station |
| CN116976858B (en)* | 2023-08-09 | 2024-05-28 | 武汉西高电器有限公司 | Intelligent operation and maintenance management method and system for electrical cabinet and electronic equipment |
| CN117009960B (en)* | 2023-08-30 | 2024-02-02 | 广东轻工职业技术学院 | Data security cleaning method based on artificial intelligence |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105610944A (en)* | 2015-12-29 | 2016-05-25 | 北京物联远信息技术有限公司 | IOT-oriented fog computing architecture |
| CN107306279A (en)* | 2016-04-18 | 2017-10-31 | 阿里巴巴集团控股有限公司 | Data processing method, content delivery network node and residential gateway |
| CN108427992A (en)* | 2018-03-16 | 2018-08-21 | 济南飞象信息科技有限公司 | A kind of machine learning training system and method based on edge cloud computing |
| CN108833166A (en)* | 2018-06-19 | 2018-11-16 | 中国联合网络通信集团有限公司 | Edge cloud message forwarding method and system, network message forwarding method and system |
| CN109002357A (en)* | 2018-06-07 | 2018-12-14 | 阿里巴巴集团控股有限公司 | Resource allocation method, device and internet of things system |
| CN109151072A (en)* | 2018-10-26 | 2019-01-04 | 上海方融科技有限责任公司 | A kind of edge calculations system based on mist node |
| CN109379420A (en)* | 2018-10-10 | 2019-02-22 | 上海方融科技有限责任公司 | A kind of comprehensive energy service platform system based on distributed structure/architecture |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101977242A (en)* | 2010-11-16 | 2011-02-16 | 西安电子科技大学 | Layered distributed cloud computing architecture and service delivery method |
| US9791485B2 (en)* | 2014-03-10 | 2017-10-17 | Silver Spring Networks, Inc. | Determining electric grid topology via a zero crossing technique |
| KR102198995B1 (en)* | 2017-04-21 | 2021-01-06 | 에스케이텔레콤 주식회사 | Distributed cloud computing system, apparatus and control method thereof using the system |
| CN109408234A (en)* | 2018-10-19 | 2019-03-01 | 国云科技股份有限公司 | An augmented reality technology data optimization system and method based on edge computing |
- 2019
- 2019-04-08CNCN201910277959.2Apatent/CN111800443B/enactiveActive
- 2020
- 2020-04-07WOPCT/CN2020/083478patent/WO2020207371A1/ennot_activeCeased
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105610944A (en)* | 2015-12-29 | 2016-05-25 | 北京物联远信息技术有限公司 | IOT-oriented fog computing architecture |
| CN107306279A (en)* | 2016-04-18 | 2017-10-31 | 阿里巴巴集团控股有限公司 | Data processing method, content delivery network node and residential gateway |
| CN108427992A (en)* | 2018-03-16 | 2018-08-21 | 济南飞象信息科技有限公司 | A kind of machine learning training system and method based on edge cloud computing |
| CN109002357A (en)* | 2018-06-07 | 2018-12-14 | 阿里巴巴集团控股有限公司 | Resource allocation method, device and internet of things system |
| CN108833166A (en)* | 2018-06-19 | 2018-11-16 | 中国联合网络通信集团有限公司 | Edge cloud message forwarding method and system, network message forwarding method and system |
| CN109379420A (en)* | 2018-10-10 | 2019-02-22 | 上海方融科技有限责任公司 | A kind of comprehensive energy service platform system based on distributed structure/architecture |
| CN109151072A (en)* | 2018-10-26 | 2019-01-04 | 上海方融科技有限责任公司 | A kind of edge calculations system based on mist node |
Non-Patent Citations (1)
| Title |
|---|
| "边缘计算在物联网领域的应用及展望";楚俊生等;《信息通信技术》;20181231;第1-9页* |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111800443A (en) | 2020-10-20 |
| WO2020207371A1 (en) | 2020-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111800443B (en) | Data processing system and method, device and electronic equipment | |
| CN111090687B (en) | Data processing method, device and system and computer readable storage medium | |
| CN111352806B (en) | Log data monitoring method and device | |
| KR101871383B1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN107391276B (en) | Distributed monitor method, interception control device and system | |
| CN115004650B (en) | Node configuration method, device, distributed system and computer-readable medium | |
| WO2019041930A1 (en) | Inspection method, device and system, computer apparatus, and storage medium | |
| WO2022127504A1 (en) | Network element management method and apparatus, and storage medium | |
| CN114296675A (en) | Screen projection state determination method, device, equipment and computer readable storage medium | |
| CN110289991A (en) | A faulty gateway detection method, device, storage medium and terminal | |
| CN115242787B (en) | Message processing system and method | |
| CN109788251B (en) | Video processing method, device and storage medium | |
| WO2022068564A1 (en) | Device abnormality monitoring method and device | |
| CN113157475A (en) | Log processing method and device, storage medium and electronic equipment | |
| CN111104575B (en) | Data grabbing method and device and electronic equipment | |
| CN115914375A (en) | Disaster recovery processing method and device for distributed message platform | |
| CN114546671B (en) | Data processing method, device and electronic equipment | |
| CN113542424A (en) | Data processing method, apparatus, apparatus and computer program product | |
| WO2021134252A1 (en) | Inter-device communication method and apparatus, and storage medium | |
| CN115277652B (en) | Streaming media processing method, device and electronic equipment based on inference service | |
| CN116032619B (en) | State inspection method and device of intelligent safety networking equipment | |
| US12401578B2 (en) | System and method for using intelligent staggered telemetry for node capabilities management across nodes in a peripheral device workspace | |
| CN108805741A (en) | Fusion method, device and system of power quality data | |
| CN103684829B (en) | Network service system and management method thereof | |
| CN108881107B (en) | Distributed resource allocation method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |