CN113992941A - Cloud edge collaborative video analysis system and method based on server-free function computing - Google Patents

Abstract

The invention discloses a cloud-edge collaborative video analysis system based on serverless function computing, which comprises an edge state machine module, a cloud-edge collaborator module and a cloud application module; the edge state machine module is used for supplementing the edge capability for the Amazon step function, and is a system which can independently run at an edge end or run in cooperation with the Amazon step function at the cloud end; the cloud edge cooperator module is used for converting and cooperating an edge state machine and an Amazon step function; the cloud application module is used for defining a model assembly line of a video stream analysis task; and realizing video stream analysis tasks for performing different analyses according to whether the video content contains characters or not.

Description

Cloud edge collaborative video analysis system and method based on server-free function computing

Technical Field

The invention belongs to the technical field of computers, and particularly relates to a cloud-edge collaborative video analysis system and method based on serverless function computing.

Technical Field

The architecture effectively reduces network delay and improves application response speed by placing part of tasks on edge equipment which is close to a user terminal in physical distance, and meanwhile, the rest tasks are placed on a cloud end with elastic resource expansion capacity for computing. Since the online video processing task facing the user needs to use various types of deep learning models for reasoning, strict requirements are placed on low response time. In recent years, amazon platform develops a serverless service named amazon Lambda, which allows a user to execute a function while a third party governs the allocation and expansion of computing resources, and thus amazon Lambda has a characteristic of fast response speed and reducing the burden of a developer on managing computing resources. However, the Lambda function is stateless, event-driven, and expensive, so that the Lambda function and the edge algorithm are combined to achieve a better balance between latency and cost.

In order to achieve better balance, the task needs to be divided into a cloud end and an edge end, and then the cloud end cooperatively executes the task. The division mode of the tasks under the cloud edge collaborative scene can cause great influence on the user experience. The existing cloud edge collaborative model generally performs task division according to a functional composition flow of an application by a developer, and does not consider performing task division according to a model. Because the demands and response times of different models for the memory are greatly different, coarse-grained task scheduling may cause cloud-edge collaboration of tasks to fail to reach an optimal balance point, and slight differences of schemes for task division may cause severe jitter of response times.

In order to divide a task, many tasks divide the task into a plurality of subtasks, and the subtasks are combined into a directed acyclic graph in a pipeline manner and then divided. However, most of the directed acyclic graphs of these working designs are organized in a sequential structure, and do not consider the selection, circulation and branch relationships among the subtasks. When a task with complex internal logic is encountered, a directed acyclic graph which can accurately describe the task cannot be obtained.

To address this problem, some prior efforts have implemented the use of a framework provided by a cloud provider in the cloud to construct a directed acyclic graph with complex graph logic, such as amazon developing a pipeline service named amazon step function that can organize amazon Lambda functions into one state machine instance to coordinate the execution of multiple functions.

However, current work does not address providing an automated framework at the edge that partitions the task into directed acyclic graphs with complex graph logic. This results in that task partitioning at the edge end needs to completely depend on manual program writing, which causes non-uniformity of function interfaces, which leads to difficulty in multiplexing subtask modules. However, a plurality of identical subtask modules often appear in a cloud-edge collaborative block diagram, and non-uniform function interfaces lead to code redundancy and block diagram bloat.

In order to ensure satisfactory online service performance and reduce resource consumption, when the execution environment of a subtask is judged to be at a cloud end or an edge end, an existing cloud edge collaboration framework is not flexible enough, some methods judge whether the function is executed at the cloud end or the edge end by taking the function as a unit, although resource scheduling can be carried out in a fine-grained manner, the integrity of a directed acyclic graph and the dependence between subtasks are ignored, and the concurrency condition is only solved, namely the sequential inherited logical relationship between functions. Other methods divide the directed acyclic graph in advance and then use an algorithm to select the execution environment of the subgraph, and the method is a coarse-grained division method although the completeness of the graph is maintained. Therefore, it is difficult to satisfy both important features of graph integrity and segmentation sensitivity in task segmentation.

Therefore, the prior art has the following drawbacks:

1. in order to realize fine-grained resource scheduling of cloud-edge cooperative tasks, when the tasks are divided, a plurality of tasks are divided into a plurality of subtasks by a plurality of jobs, and the subtasks are combined into a directed acyclic graph in a pipeline mode and then graph division is carried out. However, most of the directed acyclic graphs of these working designs are organized in a sequential structure, and do not consider the selection, circulation and branch relationships among the subtasks. When a task with complex internal logic is encountered, a directed acyclic graph which can accurately describe the task cannot be obtained. Amazon's proposed amazon step function has implemented the use of cloud provider-provided framework in the cloud to construct directed acyclic graphs with complex graph logic, however, these efforts have not studied providing an automated framework at the edge that partitions the task into directed acyclic graphs with complex graph logic.

2. In order to balance satisfactory online service performance and reduce resource consumption, when the execution environment of a subtask is judged to be at a cloud end or an edge end, an existing cloud edge collaboration framework is not flexible enough, although an amazon Lambda function can be expanded to the edge end by an existing method, such as amazon internet of things green grass, the existing method does not realize a method for organizing the function in a directed acyclic graph form at the edge end, the function is still a complete task executed at the edge end, and fine-grained division is not performed in the task. Other methods implement the function organization in amazon state language semantics, such as the open-source framework Flogo, but the framework implementation is not complete, only implements the most basic sequence and selection branches, etc., but does not implement redefinition set branching and interfacing with amazon step functions, so the Flogo is only a functional incomplete and can only be a stand-alone system running on the edge end alone.

3. The existing method usually judges whether the function is executed at a cloud end or an edge end by taking the function as a unit, although fine-grained resource scheduling can be performed, the integrity of the directed acyclic graph, dependence between subtasks and high concurrency are ignored, and only the sequential inheritance logical relationship between functions can be realized. Other methods divide the directed acyclic graph under a plurality of different division points in advance, and then use an algorithm to select the execution environment of the subgraph, and the static cloud edge cooperation method maintains the integrity of the graph, but has insufficient sensitivity in the division of the graph. Therefore, it is difficult to satisfy both important features of graph integrity and segmentation sensitivity in task segmentation.

Disclosure of Invention

In view of the defects of the prior art, the invention provides a cloud edge collaborative video analysis system and method based on server-free function computing, in particular to a high-concurrency cloud edge collaborative video stream processing framework with finite state machine semantics based on a model pipeline, which is composed of a cloud model pipeline based on an amazon step function state machine and an edge model pipeline based on an edge high-concurrency state machine edge state machine independently developed by the invention. The model pipeline refers to a task form with a finite-state machine which is formed by a plurality of deep learning models with different memory requirements or different functions. The method has the characteristic that a model assembly line with complex graph logic can be formed while fine-grained resource scheduling can be realized at a cloud end and an edge end.

Firstly, after obtaining the request data and task mode of a user, mapping the task into a directed acyclic graph which is composed of a plurality of states and has complex graph logic. The edge state machine deployed on the edge device respectively encapsulates a plurality of states in the directed acyclic graph into independent function templates, and then combines the states into a model pipeline state machine with complex graph logic according to mapping rules and logic between the states in the directed acyclic graph. In order to solve the defect that a cloud edge collaborative framework is not flexible enough, after a complete model pipeline state machine is deployed at an edge end, the model pipeline state machine with the same function as that of the edge end is arranged at a cloud end by using an amazon step function, the model pipeline state machines of the cloud end and the edge end introduce a routing template for automatically judging a starting point and an end point, and self-adaptive cloud edge combination is carried out according to a user-defined cloud edge collaborative segmentation point. Because the directed acyclic graph has the characteristics of a single starting point and multiple end points, the routing template can well divide any subgraph of the directed acyclic graph by controlling the starting point and the end point of the directed acyclic graph, and therefore flexible cloud edge cooperation is conducted. And the deployment and operation of the routing module hardly affect the performance of the system. Compared with the traditional resource scheduling taking a function as a unit, the method keeps the integrity of the graph, because the routing module can dynamically organize the cloud edge collaborative directed acyclic graph according to the task request of the user, the method is more flexible than a static cloud edge collaborative method, and the calculation cost and the memory consumption of combining all possible cloud edge collaborative directed acyclic graphs in advance are avoided.

In order to achieve the purpose, the technical scheme adopted by the invention is as follows:

the cloud edge collaborative video analysis system based on the server-free function computing comprises an edge state machine module, a cloud edge collaborator module and a cloud application module; wherein

The edge state machine module is used for providing edge capability supplement for the Amazon step function, and is a system which can independently run at an edge end or run in cooperation with the Amazon step function at the cloud end;

the cloud edge cooperator module is used for converting and cooperating an edge state machine and an Amazon step function;

the cloud application module is used for defining a model assembly line of a video stream analysis task; and realizing video stream analysis tasks for performing different analyses according to whether the video content contains characters or not.

It should be noted that the edge state machine module includes:

a status pool module: the system is used for representing a set of all available states, wherein the states refer to functions which are customized by a user and realize application logic;

redefined state language state machine module: the redefined state language state machine provides functions of transition relations among defined states similar to Amazon state language at the edge end;

an analyzer module: for parsing redefined state languages, managing state pools, and dynamically determining transition relationships between states;

an actuator module: the cloud edge coordination strategy is used for defining an execution state machine, and a user can define the calling proportion of cloud resources and edge resources;

a message module: information for passing between templated states, an underlying data structure for the information is defined, and the type and value of the data is stored using a collection structure.

It should be noted that the cloud edge coordinator module includes:

a preprocessing module: the system comprises a state machine, a pipeline task and a data processing module, wherein the state machine is used for receiving request data of a user and calling the state machine to execute the pipeline task;

a state module: the main body application logic is used for realizing a user-defined state, and a specific function is realized by realizing a handle function of the state;

function factory module: state resources required for generating an edge state machine module and an amazon step function;

an edge state machine module: the method is used for realizing an edge state machine, generating an edge end state machine by using state resources generated by an edge state machine module and a function factory module, has user-defined state transition logic of a pipeline task, and is a specific implementation of a directed acyclic graph; the user can decide whether to call the cloud resources in the edge state machine module, if the cloud resources are called, cloud-edge coordination is realized, and if not, the system only calls the edge computing resources;

lambda function module: the function body logic is used for defining the Lambda function, and is encapsulated by using a software development kit of the Lambda function and the state module;

amazon status language module: the system comprises an Amazon state language module, an edge state machine module and an Amazon state language module, wherein the Amazon state language module is used for generating an Amazon state language file required by an Amazon step function, automatically generating the Amazon state language file by analyzing the edge state machine defined by the edge state machine module, and checking the grammatical specification of the Amazon state language file;

the SAM module: a script YAML file for generating an automated deploy amazon step function. The module indicates the positions of an amazon state language file and a Lambda function which form the amazon step function, packs corresponding resources and automatically deploys the amazon step function at the cloud end;

an execution module: program logic for implementing a user to invoke the state machine and automatically send data; the execution module uses Golang protocol to manage the sending of user requests, each protocol corresponds to a user request, and the user request calls a state machine instance;

it should be noted that the cloud application module includes:

a video segmentation module: for dividing the video into video frames at a fixed frequency;

a person detection module: the system is used for detecting whether a video frame contains a person or not;

the face detection module: the face recognition method comprises the steps of recognizing faces in video frames;

the framework detection module: detecting a skeleton shape of a person in a video frame;

a target detection module: for identifying and classifying objects in the video frames;

a scene conversion module: a scene style type for converting the video frame;

a routing module: the method is used for realizing the seamless connection of the cloud edges in the cloud edge collaboration. The cloud edge cooperation is based on the segmentation of the directed acyclic graph of the model pipeline task, so that the first half is executed at the edge end, and the second half is executed at the cloud end; the routing module can dynamically limit the starting point and the end point of the edge state machine and the cloud end state machine, so that seamless connection of cloud edges at different partition points is realized.

It should be noted that the redefined state language state machine module has 5 state attributes, including

Task: the application logic for realizing user self-definition is a main body of state execution;

straight line: the method is used for realizing a sequential structure between states, generally, two states are connected with each other, and the output of a state executed first is the input of a state executed later;

redefined set: the invention is used for using a plurality of different inputs to concurrently execute the same state, the invention uses an array sequence to store a plurality of inputs and calls a plurality of nested states in a set state in sequence;

selecting a branch: for implementing branching logic, the present invention uses an aggregate data structure to store mappings of conditions and states. Selecting the branch state selects the corresponding state according to the input condition;

the end point used for representing the state machine instance can be reached only when the state machine instance is successfully ended;

and failure is used for indicating the illegal terminal of the state machine instance, and the state machine instance stops running due to abnormity or errors.

It should be noted that the function factory is divided into:

factory: state resources required for producing edge state machines;

auxiliary plants: an interface for calling amazon step function exposure; in order to reduce economic expenditure, a cloud amazon step function is only used as supplement of edge end state computer computing resources; wherein, the auxiliary factory produces only when the factory can not generate the edge terminal state resource; the user may also customize the production logic of the auxiliary plant.

The invention also provides a cloud-edge collaborative video analysis method using the cloud-edge collaborative video analysis system based on the serverless function computing, which comprises the following steps:

s1 the Amazon status language module generates the Amazon status language file and returns the directory address of the Amazon status language file to the SAM module;

s2, the SAM module receives a directory address where the Amazon state language file is located, generates a YAML file, automatically deploys an Amazon step function of a cloud end, and returns a calling interface of the Amazon step function to an auxiliary factory module of the function factory;

the S3 execution module reads the video frame and sends POST request to the route defined in the preprocessing module according to the fixed speed, the POST request includes the data to be processed and the initial state information and the end state information of the state machine;

after receiving the request, the preprocessing module of S4 starts a protocol to call a state machine defined by the edge state machine module, and forwards the request data and the initial state information and the end state information of the state machine to the edge state machine module for execution;

s5 the edge state machine module receives the request, the module has the state machine defined by the edge state machine template, the module starts to execute from the initial state of the state machine defined in the request, when executing a state, the current state will call the function factory module to obtain the function resource defined by the state; according to a cloud edge cooperation strategy formulated by a user, if the state needs to call edge end resources, the state calls a factory generation edge end function of the function factory module, and if the state needs to call cloud end resources, the state calls an auxiliary factory of the function factory module to call an amazon step function of the cloud end; after the function resources are obtained and the data are processed, the current state transmits the next state information connected with the output current state to the routing module for continuous execution or successfully quits the state machine, and when running errors occur, the state stops running and quits the state machine in advance;

the S6 function factory module, when called, looks at the type of factory called. If the type of the factory is the factory type, the function factory module generates computing resources of the edge end and returns the function executed at the edge end; if the type of the auxiliary factory is the auxiliary factory type, the function factory module returns a function for calling the Amazon step function;

the S7 route module receives the output of the last state and the current state information to be executed, then judges if the current state is the end state defined by the user, if yes, the operation of the state machine is terminated, and the program is exited; if not, the output of the last state is transmitted to the edge state machine module, and the current state is executed.

The invention has the beneficial effects that:

1. the invention discloses a cloud edge collaborative video analysis system based on serverless function computing, which aims to realize cloud edge collaboration based on an Amazon step function, namely the Amazon step function can be operated at the cloud edge and can partially or completely expand a state machine to be executed at the edge. The invention constructs an edge state machine of an edge state machine system at the edge end, which is compatible with the Amazon step function. Compared with the amazon internet of things green grass which cannot provide the state machine capability and the open-source framework Flogo which only realizes a small amount of state machine capability in the traditional method, the edge state machine can be more completely compatible with the state machine execution capability of the amazon step function. And the edge state machine defines the message type compatible with the amazon step function and the edge state machine definition language similar to the amazon step language, compared with the use of the self-defined message type and the state machine definition Flogo which have great difference with the amazon step function, the edge state machine avoids the type conversion when the message is transmitted between cloud edges, and because the state machine definition languages of the edge end and the cloud end are similar, the state machine generated by the edge state machine can be perfectly compatible with the cloud end and the edge end at the same time without additionally modifying the state machine in order to adapt to the amazon step function of the cloud end.

2. The invention discloses a novel cloud-edge collaborative video analysis system based on server-free function computing, and provides an edge state machine interpreter cloud-edge collaborator. The cloud edge cooperator may perform translation and cooperation between the edge state machine and the amazon step function. Compared with the traditional method that the cloud sides respectively realize one system, the cloud side collaborator can multiplex the same codes of the cloud sides, so that the method has the advantages of high cohesion and low coupling. The cloud edge collaborator can automatically generate the amazon state language files and YAML files required by the amazon step function deployment according to the state machine defined by the edge state machine template, and compared with the traditional file manual compiling file, the cloud edge collaborator has the advantages of automatically deploying resources, avoiding the manual compiling cost and reducing the time overhead.

3. The invention provides a routing module, which realizes flexible cloud edge cooperation. In the traditional method, when the cloud-edge cooperation is realized, the partition unit at the cloud end is a single Lambda function, and a pipeline task is formed by taking Lambda as a unit without depending on a graph organization form of an Amazon step function. Therefore, task organization of the traditional method only has simple sequential inheritance relationship generally, and cannot realize complex transfer relationship between states of a state machine, so that the universality is low. The invention realizes the random selection of the partition points between the cloud edges according to the routing module in the view of the directed acyclic graph, realizes the seamless connection between the cloud edges, does not destroy the dependency relationship between the states in the graph, and has high universality. And once the state machine generated by the amazon step function is deployed, the state machine cannot be dynamically changed, and the appearance of the routing module enables multiple cloud edge coordination schemes to be possible only by using the same amazon step function state machine, so that the time overhead and the economic overhead for producing multiple amazon step functions are greatly saved.

Drawings

FIG. 1 is a schematic flow diagram disclosed in the present invention;

FIG. 2 is a directed acyclic graph in the form of a model pipeline for a video stream analysis task;

fig. 3 is a schematic diagram of a state machine with a routing module for the video stream analysis task.

Detailed Description

The present invention will be further described with reference to the accompanying drawings, and it should be noted that the present embodiment is based on the technical solution, and the detailed implementation and the specific operation process are provided, but the protection scope of the present invention is not limited to the present embodiment.

Example 1

The embodiment discloses a cloud-edge collaborative video analysis method based on serverless functional computing, a complete flow chart of the method is shown in fig. 1, and the specific implementation steps are as follows:

1. the cloud application module forms a directed acyclic graph of the video stream analysis task.

2. The edge state machine module generates a state machine from the directed acyclic graph.

3. The amazon status language module generates an amazon status language file and returns a directory address where the amazon status language file is located to the SAM module.

4. The SAM module receives a directory address where the Amazon state language file is located, generates a YAML file, automatically deploys an Amazon step function of the cloud, and returns a calling interface of the Amazon step function to an auxiliary factory module of the function factory.

5. The execution module reads the video frames and sends a POST request to the route defined in the pre-processing module at a fixed rate, the POST request containing the data to be processed and the start state information and end state information of the state machine.

6. After receiving the request, the preprocessing module calls a state machine defined by the edge state machine module, and forwards the request data and the initial state information and the end state information of the state machine to the edge state machine module for execution.

7. After the edge state machine module receives the request, the module has the state machine defined by the edge state machine template, the module starts to execute from the initial state of the state machine defined in the request, when executing a state, the current state will call the function factory module to obtain the function resource defined by the state. According to a cloud edge cooperation strategy formulated by a user, if the state needs to call edge end resources, the state calls a factory generation edge end function of the function factory module, and if the state needs to call cloud end resources, the state calls an auxiliary factory of the function factory module to call an amazon step function of the cloud end. After the function resources are obtained and the data is processed, the current state transmits the next state information connected with the current state to the routing module for continuous execution, or successfully exits the state machine, and when a runtime error is encountered, the state terminates the operation and exits the state machine in advance.

8. The function factory module, when called, looks at the type of factory called. If the type of the factory is the factory type, the function factory module generates computing resources of the edge end and returns the function executed at the edge end. If it is an auxiliary plant type, the function plant module returns a function that calls the Amazon step function.

9. After receiving the output of the last state and the current state information to be executed, the routing module judges whether the current state is a user-defined end state, and if so, the routing module terminates the running of the state machine and exits the program.

If not, the output of the last state is transmitted to the edge state machine module, and the current state is executed.

The embodiment discloses a directed acyclic graph in a model pipeline form for a video stream analysis task, as shown in fig. 2, the specific implementation steps are as follows:

1. a video segmentation module: the system is used for dividing the video into video frames according to a fixed frequency and transferring the video frames to a human detection module.

2. A person detection module: the method is used for detecting whether the video frame contains the person or not, and if yes, the method is switched to a face detection module.

3. The face detection module: the method is used for identifying the human face in the video frame and turning to the skeleton detection module.

4. The framework detection module: the method is used for detecting the skeleton shape of the person in the video frame and then the operation is finished.

5. A target detection module: the system is used for identifying and classifying objects in the video frame and turning to a scene detection module.

6. A scene conversion module: and converting the scene style type of the video frame to end.

The embodiment discloses a schematic diagram of an edge state machine added with a video stream analysis task of a routing module, as shown in fig. 3:

1. the Start state initiates the operation of the state machine, forwards to the route, and forwards the received input to the route, the input including a selection branch value defining the Start of the state machine.

2. The routing module carries out routing orientation and analyzes input:

1) if the branch value is selected as the continuous person detection, the state of the continuous person detection is switched to.

2) And if the branch value is selected as the continuous target detection, switching to a continuous target detection state.

3) And if the branch value is selected as the continuous scene conversion, switching to a continuous scene conversion state.

4) And if the branch value is selected to continue the face recognition, switching to a state of continuing the face recognition.

5) If the branch value does not match the above condition, the state is switched to the default state.

3. And continuing the person detection state to set the selection branch value for person detection, and switching to the selection branch routing.

4. And continuing the target detection state, setting the selective branch value as target detection, and switching to the selective branch route.

5. And continuing the scene conversion state, setting the option branch value as the scene conversion, and switching to the option branch route.

6. And continuing the face recognition state to set a selection branch value for face recognition, and switching to a selection branch route.

7. Selecting a branch route for route orientation, and analyzing input:

1) and if the branch value is selected for human detection, switching to a human detection state.

2) And if the branch value is selected as the target detection, switching to a target detection state.

3) And if the selected branch value is the scene transition, switching to a scene transition state.

4) And if the selected branch value is the face recognition, turning to a face recognition state.

5) If the branch value does not match the above condition, the state is switched to the default state.

8. The person detection state is used for carrying out person identification, and if persons exist in the frame, the selection branch value is set as continuous person detection; if not, setting the selection branch value as the continuous target detection. And turning to a selected state route.

9. Selecting a state route for routing orientation, and analyzing input:

1) and if the branch value is selected to continue the face recognition, switching to a state of continuing the face recognition.

2) And if the branch value is selected as the continuous target detection, switching to a continuous target detection state.

10. And carrying out face recognition in the face recognition state, and turning to the skeleton detection state.

11. And carrying out skeleton identification in the skeleton detection state, and turning to an end state 1.

12. End state 1 printout, go to end state.

13. And carrying out target detection in the target detection state, and switching to the scene switching state.

14. And the scene conversion state carries out scene conversion and is converted to an end state.

15. And ending the state, printing the execution success information, and turning to an End state.

16. And printing execution failure information in a default state, and turning to an End state.

17. The End state stops the execution of the state machine and the process ends.

Various modifications may be made by those skilled in the art based on the above teachings and concepts, and all such modifications are intended to be included within the scope of the present invention as defined in the appended claims.

Claims (7)

Translated fromChinese

1.基于无服务器函数计算的云边协同视频分析系统，其特征在于，所述系统包括边缘状态机模块、云边协同器模块和云应用模块；其中1. a cloud-edge collaborative video analysis system based on serverless function computing, wherein the system comprises an edge state machine module, a cloud-edge synergist module and a cloud application module; wherein所述边缘状态机模块用于为亚马逊步骤函数提供边缘能力的补充，是一种可以在边缘端独立运行或者和云端的亚马逊步骤函数协同运行的系统；The edge state machine module is used to supplement the edge capability for the Amazon step function, and is a system that can run independently on the edge or cooperate with the Amazon step function in the cloud;所述云边协同器模块用于进行边缘状态和亚马逊步骤函数之间的转换和协同；The cloud-edge synergist module is used to perform conversion and synergy between edge states and Amazon step functions;所述云应用模块用于定义视频流分析任务的模型流水线；实现根据视频内容是否包含人物来进行不同分析的视频流分析任务。The cloud application module is used to define the model pipeline of the video stream analysis task; realize the video stream analysis task of performing different analysis according to whether the video content contains characters.2.根据权利要求1所述的基于无服务器函数计算的云边协同视频分析系统，其特征在于，所述边缘状态机模块包括：2. The cloud-edge collaborative video analysis system based on serverless function computing according to claim 1, wherein the edge state machine module comprises:状态池模块：用于表示所有可用状态的集合，其中，状态是指用户自定义的实现应用逻辑的函数；State pool module: used to represent a collection of all available states, where state refers to a user-defined function that implements application logic;重定义状态语言状态机模块：重定义状态语言状态机在边缘端提供了类似亚马逊状态语言的定义状态间的转移关系的功能；Redefining state language state machine module: Redefining state language state machine provides the function of defining transition relationships between states similar to Amazon state language at the edge;解析器模块：用于解析重定义状态语言，管理状态池，以及动态判断状态之间的转移关系；Parser module: used to parse the redefined state language, manage the state pool, and dynamically determine the transition relationship between states;执行器模块：用于定义执行状态机的云边协同策略，用户可以自行定义云端资源和边缘端资源的调用比例；Actuator module: used to define the cloud-edge collaboration strategy for executing the state machine. Users can define the calling ratio of cloud resources and edge resources;消息模块：用于模板化状态之间传递的信息，定义了信息的基本数据结构，使用集合结构存储数据的类型和数值。Message module: It is used to template the information passed between states, defines the basic data structure of the information, and uses the collection structure to store the type and value of the data.3.根据权利要求1所述的基于无服务器函数计算的云边协同视频分析系统，其特征在于，所述云边协同器模块包括：3. The cloud-edge collaborative video analysis system based on serverless function computing according to claim 1, wherein the cloud-edge synergist module comprises:预处理模块：用于接收用户的请求数据和调用状态机以执行流水线任务；Preprocessing module: used to receive the user's request data and call the state machine to perform pipeline tasks;状态模块：用于实现用户自定义的状态的主体应用逻辑，通过实现状态的句柄函数来实现具体的功能；State module: the main application logic used to implement user-defined states, and implement specific functions by implementing state handle functions;函数工厂模块：用于产生边缘状态机模块和亚马逊步骤函数所需的状态资源；Function factory module: used to generate state resources required by edge state machine modules and Amazon step functions;边缘状态机模块：用于实现边缘状态机，使用边缘状态机模块和函数工厂模块所生成的状态资源生成边缘端状态机，具有用户自定义的流水线任务的状态转移逻辑，是有向无环图的具体实现；用户可以在边缘状态机模块决定是否调用云端资源，如果调用云端资源则实现了云边协同，否则，则系统只会调用边缘端计算资源；Edge state machine module: used to implement edge state machine, use the state resources generated by edge state machine module and function factory module to generate edge state machine, with user-defined state transition logic of pipeline tasks, it is a directed acyclic graph The specific implementation; the user can decide whether to call the cloud resources in the edge state machine module, if the cloud resources are called, the cloud-side collaboration is realized, otherwise, the system will only call the edge computing resources;Lambda函数模块：用于定义Lambda函数，使用Lambda函数的软件开发工具包和状态模块生成的函数主体逻辑进行封装；Lambda function module: used to define a Lambda function, which is encapsulated by the software development kit of the Lambda function and the function body logic generated by the state module;亚马逊状态语言模块：用于生成亚马逊步骤函数所需要的亚马逊状态语言文件，亚马逊状态语言模块通过解析边缘状态机模块所定义的边缘状态机，自动生成亚马逊状态语言文件，并且进行亚马逊状态语言文件语法规范的检查；Amazon state language module: used to generate the Amazon state language file required by the Amazon step function. The Amazon state language module automatically generates the Amazon state language file by parsing the edge state machine defined by the edge state machine module, and performs the Amazon state language file grammar. normative inspection;SAM模块：用于生成自动部署亚马逊步骤函数的脚本YAML文件。该模块指明构成亚马逊步骤函数的亚马逊状态语言文件和Lambda函数的位置，将对应资源打包并且在云端自动部署亚马逊步骤函数；SAM module: A script YAML file used to generate automated deployment Amazon step functions. This module specifies the location of the Amazon state language files and Lambda functions that constitute the Amazon step function, packages the corresponding resources and automatically deploys the Amazon step function in the cloud;执行模块：用于实现用户调用状态机并且自动发送数据的程序逻辑；执行模块使用Golang协程来管理用户请求的发送，每个协程对应一次用户请求，一次用户请求调用一次状态机实例。Execution module: It is used to implement the program logic that the user calls the state machine and automatically sends data; the execution module uses Golang coroutines to manage the sending of user requests. Each coroutine corresponds to a user request, and each user request calls a state machine instance.4.根据权利要求1所述的基于无服务器函数计算的云边协同视频分析系统，其特征在于，所述云应用模块包括：4. The cloud-side collaborative video analysis system based on serverless function computing according to claim 1, wherein the cloud application module comprises:视频分割模块：用于按照固定的频率将视频分割成视频帧；Video segmentation module: used to divide the video into video frames according to a fixed frequency;人物检测模块：用于检测视频帧中是否包含人物；Person detection module: used to detect whether the video frame contains a person;人脸检测模块：用于识别视频帧中的人脸；Face detection module: used to identify faces in video frames;骨架检测模块：用于检测视频帧中的人物的骨架形状；Skeleton detection module: used to detect the skeleton shape of the person in the video frame;目标检测模块：用于对视频帧中的物体进行识别和分类；Object detection module: used to identify and classify objects in video frames;场景转换模块：用于转换视频帧的场景风格类型；Scene conversion module: used to convert the scene style type of video frames;路由模块：用于实现云边协同时云边的无缝衔接。由于云边协同是基于对模型流水线任务的有向无环图的分割，从而前半部分在边缘端执行，后半部分在云端执行；路由模块可以动态地限制边缘状态机和云端状态机的起点和终点，从而实现不同分割点下云边的无缝衔接。Routing module: used to realize the seamless connection between cloud and edge during cloud-edge collaboration. Since the cloud-edge collaboration is based on the segmentation of the directed acyclic graph of the model pipeline tasks, the first half is executed on the edge, and the second half is executed on the cloud; the routing module can dynamically limit the starting point and the starting point of the edge state machine and the cloud state machine. End point, so as to realize the seamless connection of cloud edges under different segmentation points.5.根据权利要求2所述的基于无服务器函数计算的云边协同视频分析系统，其特征在于，所述重定义状态语言状态机模块具有5种状态属性，包括5. The cloud-edge collaborative video analysis system based on serverless function computing according to claim 2, wherein the redefinition state language state machine module has 5 state attributes, including任务：用于实现用户自定义的应用逻辑，是状态执行的主体；Task: used to implement user-defined application logic, and is the main body of state execution;直线：用于实现状态之间的顺序结构，通常是两个状态之间相互连接，先执行的状态的输出是后执行的状态的输入；Straight line: used to realize the sequential structure between states, usually two states are connected to each other, the output of the state executed first is the input of the state executed later;重定义集合：用于使用多个不同输入来并发执行同一个状态，本发明使用数组顺序存储多个输入，并按序调用集合状态内嵌套的多个状态；Redefining the set: used to execute the same state concurrently by using multiple different inputs, the present invention uses an array to store multiple inputs in sequence, and sequentially calls multiple states nested in the set state;选择分支：用于实现分支逻辑，本发明使用集合数据结构存储条件和状态的映射关系。选择分支状态将根据输入条件选择对应的状态；Select branch: used to implement branch logic, the present invention uses a set data structure to store the mapping relationship between conditions and states. Selecting the branch state will select the corresponding state according to the input conditions;成功:用于表示状态机实例的终点，只有在状态机实例成功结束时才能到达；success: used to indicate the end point of the state machine instance, which can only be reached when the state machine instance ends successfully;失败:用于表示状态机实例非法终端，此时状态机实例由于异常或错误停止运行。Failure: It is used to indicate an illegal terminal of the state machine instance, and the state machine instance stops running due to an exception or error.6.根据权利要求3所述的基于无服务器函数计算的云边协同视频分析系统，其特征在于，所述函数工厂分为：6. The cloud-edge collaborative video analysis system based on serverless function computing according to claim 3, wherein the function factory is divided into:工厂：用于生产边缘状态机所需要的状态资源；Factory: used to produce state resources required by edge state machines;辅助工厂：用于调用亚马逊步骤函数暴露的接口；为了减少经济开销，云端亚马逊步骤函数仅作为边缘端状态机计算资源的补充；其中，辅助工厂只在工厂无法生成边缘端状态资源时生产；用户也可自定义辅助工厂的生产逻辑。Auxiliary factory: used to call the interface exposed by the Amazon step function; in order to reduce economic overhead, the cloud Amazon step function is only used as a supplement to the computing resources of the edge state machine; among them, the auxiliary factory only produces when the factory cannot generate edge state resources; users The production logic of auxiliary factories can also be customized.7.一种利用如权要求1-5任一所述的基于无服务器函数计算的云边协同视频分析系统的云边协同视频分析方法，其特征在于，所述方法包括以下步骤：7. A cloud-side collaborative video analysis method utilizing the cloud-side collaborative video analysis system based on serverless function computing as described in any one of claims 1-5, wherein the method comprises the following steps:S1亚马逊状态语言模块生成亚马逊状态语言文件，并且返回亚马逊状态语言文件所在的目录地址给SAM模块；The S1 Amazon state language module generates the Amazon state language file, and returns the directory address where the Amazon state language file is located to the SAM module;S2 SAM模块接收亚马逊状态语言文件所在的目录地址，生成YAML文件并且自动部署云端的亚马逊步骤函数，且返回亚马逊步骤函数的调用接口给函数工厂的辅助工厂模块；The S2 SAM module receives the directory address of the Amazon state language file, generates a YAML file and automatically deploys the Amazon step function in the cloud, and returns the call interface of the Amazon step function to the auxiliary factory module of the function factory;S3执行模块读取视频帧，并且按照固定速率发送POST请求给预处理模块中定义的路由，POST请求包含所要处理的数据和状态机的起始状态信息和终点状态信息；The S3 execution module reads the video frame, and sends a POST request to the route defined in the preprocessing module at a fixed rate. The POST request contains the data to be processed and the start state information and end state information of the state machine;S4预处理模块接收到请求后，启动一个协程调用边缘状态机模块定义的状态机，将请求数据和状态机的起始状态信息和终点状态信息转发给边缘状态机模块执行；After receiving the request, the S4 preprocessing module starts a coroutine to call the state machine defined by the edge state machine module, and forwards the request data and the start state information and end state information of the state machine to the edge state machine module for execution;S5边缘状态机模块接收到请求后，该模块内有由边缘状态机模板定义的状态机，该模块从请求中定义的状态机的起始状态开始执行，当执行一个状态时，当前状态将会调用函数工厂模块以获得该状态定义的函数资源；根据用户制定的云边协同策略，如果该状态需要调用边缘端资源，该状态则调用函数工厂模块的工厂生成边缘端函数，如果该状态需要调用云端资源，该状态则调用函数工厂模块的辅助工厂调用云端的亚马逊步骤函数；在获得函数资源并且处理完数据后，当前状态将输出和当前状态所相连的下一个状态信息传给路由模块继续执行，或者是成功退出状态机，当遇到运行时错误时，该状态终止运行，并且提前退出状态机；After the S5 edge state machine module receives the request, the module has a state machine defined by the edge state machine template. The module starts to execute from the initial state of the state machine defined in the request. When a state is executed, the current state will be Call the function factory module to obtain the function resources defined by the state; according to the cloud-edge collaboration strategy formulated by the user, if the state needs to call the edge-side resources, the state will call the factory of the function factory module to generate the edge-side function, if the state needs to call Cloud resource, this state calls the auxiliary factory of the function factory module to call the Amazon step function in the cloud; after obtaining the function resource and processing the data, the current state will output the next state information connected to the current state to the routing module to continue execution , or exit the state machine successfully, when a runtime error is encountered, the state terminates the operation and exits the state machine in advance;S6函数工厂模块在被调用时，查看被调用工厂的类型。如果是工厂类型，函数工厂模块生成边缘端的计算资源，返回在边缘端执行的函数；如果是辅助工厂类型，函数工厂模块返回调用亚马逊步骤函数的函数；When the S6 function factory module is called, check the type of the called factory. If it is a factory type, the function factory module generates computing resources on the edge and returns the function executed on the edge; if it is an auxiliary factory type, the function factory module returns the function that calls the Amazon step function;S7路由模块在接收上一个状态的输出和当前要执行的状态信息后，判断当前状态是否是用户定义的终点状态，如果是，则终止状态机的运行，退出程序；若否，则将上一个状态的输出传给边缘状态机模块，执行当前状态。After receiving the output of the previous state and the current state information to be executed, the S7 routing module judges whether the current state is the end state defined by the user, and if so, terminates the operation of the state machine and exits the program; The output of the state is passed to the edge state machine module, which executes the current state.

Images