CN112241648A - Image processing system and image device - Google Patents
Image processing system and image deviceDownload PDFInfo
- Publication number
- CN112241648A CN112241648ACN201910640789.XACN201910640789ACN112241648ACN 112241648 ACN112241648 ACN 112241648ACN 201910640789 ACN201910640789 ACN 201910640789ACN 112241648 ACN112241648 ACN 112241648A
- Authority
- CN
- China
- Prior art keywords
- image
- model
- image algorithm
- image processing
- service platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/192—Recognition using electronic means using simultaneous comparisons or correlations of the image signals with a plurality of references
- G06V30/194—References adjustable by an adaptive method, e.g. learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
Description
Technical Field
The present disclosure relates to the field of image devices, and particularly to an image processing system and an image device.
Background
Imaging devices (e.g., cameras or NVRs) are widely used in a variety of scenarios. In order to detect image data using the image algorithm model (e.g., perform object recognition on the image data, etc.), the image device needs to upload the image data to a server running the algorithm model. In this way, the server can detect the image data through the algorithm model (i.e., the server detects the image data uploaded by the image device on line). However, the image device cannot flexibly perform offline detection (i.e., local detection) on the image data when the image data is not uploaded.
Disclosure of Invention
The application provides an image processing system and an image device, which can flexibly perform off-line detection on image data by using the image device.
According to an aspect of the present application, there is provided an image processing system including:
the model training platform is used for generating a trained image algorithm model;
an image device;
a service platform for:
determining one or more image algorithm models matched with the image equipment from the trained image algorithm models;
deploying the matched one or more image algorithm models to the image device;
instructing the image device to associate an image processing engine in the image device with an image algorithm model in the matched one or more image algorithm models;
determining an image processing task associated with the image processing engine and sending the image processing task to the image device;
the image device is used for executing the image processing task in the image processing engine based on an image algorithm model associated with the image processing engine to obtain a task output result, wherein the task output result comprises output values of parameter items in the image algorithm model.
In some embodiments, the image device is further to: sending the task output result to the service platform; and the service platform is also used for receiving the task output result.
In some embodiments, the model training platform is further configured to generate model description information corresponding to the trained image algorithm models, where the model description information corresponding to each image algorithm model is used to describe definition of parameter values of parameter items in the image algorithm model; the service platform is further configured to: acquiring the model description information; and analyzing the task output result according to the model description information to determine an event corresponding to the task output result.
In some embodiments, the model training platform is further configured to: storing the trained image algorithm model to a cloud storage platform, and acquiring an access address of the trained image algorithm model in the cloud storage platform; and sending the trained image algorithm model and the access address to the service platform.
In some embodiments, the service platform is further configured to: obtaining a model transmission mode of the image equipment, wherein the model transmission mode is used for describing whether the image equipment supports a push mode and a cloud download mode, the push mode represents that an image algorithm model is pushed to the image equipment by the service platform, and the cloud download mode represents that the image algorithm model is downloaded from the cloud storage platform; and determining a transmission mode for deploying the matched one or more image algorithm models to the image equipment according to the model transmission mode, wherein the transmission mode is a push mode or a cloud download mode.
In some embodiments, the business platform deploys the one or more image algorithm models to the image device according to: sending a model preset request containing the identifier of the transmission mode to the image device, wherein when the identifier of the transmission mode represents a push mode, the model preset request further comprises: an access address of the one or more image algorithm models in the cloud storage platform;
the camera is further configured to: receiving the model presetting request; when the identification of the transmission mode represents a cloud downloading mode, downloading the one or more image algorithm models according to the access address; and when the identification of the transmission mode represents a push mode, receiving the one or more image algorithm models pushed by the service platform.
In some embodiments, the image device performs the downloading of the one or more image algorithm models according to the access address according to: downloading data packets corresponding to the one or more image algorithm models according to the access address; and carrying out digital signature verification operation on the data packet, and carrying out decryption operation on the data packet after the data packet passes the data signature verification operation, so as to obtain the one or more image algorithm models.
In some embodiments, the model training platform generates the trained image algorithm model according to: acquiring one or more image sample sets; and training the image algorithm model to be trained according to each image sample set to obtain each trained image algorithm model.
In some embodiments, the service platform is further configured to: sending a configuration acquisition request to the image equipment; the image device is further to: responding to the configuration acquisition request, and sending configuration information to the service platform, wherein the configuration information comprises: the firmware version identification, the algorithm version identification and the chip type identification of the image equipment; the service platform determines one or more image algorithm models matched with the image equipment according to the following modes: and selecting one or more image algorithm models matched with the configuration information from the trained image algorithm models, and using the one or more image algorithm models as one or more image algorithm models matched with the image equipment.
In some embodiments, the service platform executes the selected one or more image algorithm models that match the configuration information according to: for any image algorithm model, determining whether a firmware version supported by the image algorithm model is matched with the firmware version identification; determining whether the chip type supported by the image algorithm model is matched with the chip type identifier; determining whether the algorithm version of the image algorithm model is matched with the algorithm version identification; and when the image algorithm model is determined to be matched with the firmware version identification, the algorithm version identification and the chip type identification, selecting the image algorithm model as an image algorithm model matched with the configuration information.
In some embodiments, the business platform determines the image processing task associated with the image processing engine according to: acquiring task types supported by the image equipment, wherein the supported task types comprise at least one of the following: target detection on a single-channel video stream, target detection on a multi-channel video stream of a training, target detection on an appointed picture and target detection on a training grab picture result; an image processing task corresponding to the supported task type and associated with the image processing engine is generated.
In some embodiments, the service platform is further configured to: determining whether the image device supports execution of the trained image algorithm model; when the image device is determined to support the execution of the trained image algorithm model, executing the determination of one or more image algorithm models matched with the image device from the trained image algorithm model.
According to an aspect of the present application, there is provided an image apparatus including:
one or more image processing engines;
a processor to:
receiving a model presetting request containing the identification of the transmission mode, which is sent by a service platform, wherein when the identification of the transmission mode represents a push mode, the model presetting request further comprises: an access address of one or more image algorithm models in the cloud storage platform;
when the identification of the transmission mode represents a cloud downloading mode, downloading the one or more image algorithm models according to the access address;
when the identification of the transmission mode represents a push mode, receiving the one or more image algorithm models pushed by the service platform;
receiving an indication that a service platform associates one of the one or more image processing engines with one of the one or more image algorithm models, and associating the one image processing engine with the one image algorithm model according to the indication;
receiving an image processing task associated with the one image processing engine;
in the image processing engine, the image processing task is executed based on an image algorithm model associated with the image processing engine to obtain a task output result, wherein the task output result comprises output values of parameter items in the image algorithm model.
Drawings
FIG. 1 illustrates an application scenario according to some embodiments of the present application;
FIG. 2 illustrates an interaction diagram according to some embodiments of the present application;
FIG. 3 illustrates a schematic diagram of processing a data packet according to some embodiments of the present application;
fig. 4A, 4B, and 4C respectively illustrate schematic diagrams of an image device 150 according to some embodiments of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail below with reference to the accompanying drawings and examples.
FIG. 1 illustrates an application scenario according to some embodiments of the present application. As shown in fig. 1, theapplication scenario 100 may include auser terminal 110, amodel training platform 120, abusiness platform 130, acloud storage platform 140, and a plurality of image devices, such as, but not limited to, the image devices 150, 160, and 170 shown in fig. 1. Theservice platform 130 can communicate with theuser terminal 110, themodel training platform 120, and the image devices 150, 160, and 170 through a wired or wireless communication network. Themodel training platform 120 may also communicate with thecloud storage platform 140 over a wired or wireless communication network. The image device may be, for example, a network camera or a Network Video Recorder (NVR) or other device capable of executing image algorithms. Theservice platform 130 may include, for example, one or more servers.
Themodel training platform 120 may include one or more servers. In other words, both themodel training platform 120 and the business platform may be a single server or a cluster of servers. Themodel training platform 120 may generate a trained image algorithm model. Here, the image algorithm model may be, for example, various deep learning models such as a convolutional neural network, but is not limited thereto.
Themodel training platform 120 may acquire one or more sets of image samples. According to each image sample set, themodel training platform 120 may train the image algorithm model to be trained corresponding to each image sample set, so as to obtain each trained image algorithm model. An image sample set may include a plurality of pictures that are labeled. For example, one image sample set is a plurality of material pictures about a cup. Themodel training platform 120 may obtain an image algorithm model capable of performing target detection on the teacup according to the plurality of material pictures of the teacup.
Themodel training platform 120 may upload the trained image algorithm model to thecloud storage platform 140. Here, thecloud storage platform 140 refers to a device that stores data in the cloud. Themodel training platform 120 may obtain an access address of the image algorithm model in thecloud storage platform 140.
Theservice platform 130 may determine one or more image algorithm models matching the image device from the trained image algorithm models. In other words, for any image device in theapplication scenario 100, the service platform may determine one or more image algorithm models that match the image device. For example, theservice platform 130 may determine one or more image algorithm models that match the image device 150.
For simplicity of description, the following describes the deployment process of the image algorithm model by taking the image device 150 as an example. Thebusiness platform 130 can deploy an image algorithm model that matches the image device 150 to the image device 150.
In some embodiments, image device 150 may include one or more artificial intelligence chips, such as GPU chips and the like. Each artificial intelligence chip may also be referred to herein as an image processing engine. In order to uniformly manage the image processing engines in the image device 150, theservice platform 130 may further determine an association relationship between each image processing engine in the image device 150 and the image algorithm model. Taking an image processing engine in the image device 150 as an example, theservice platform 130 may instruct the image device 150 to associate the image processing engine with an image algorithm model. When an image processing engine is associated with an image algorithm model, the image processing engine is configured to run the image algorithm model associated therewith.
Theservice platform 130 may also determine an image processing task associated with an image processing engine and send the image processing task to the image device 150. Thus, the image device 150 can execute the image processing task associated with the image processing engine in the image processing engine based on the image algorithm model associated with the image processing engine to obtain a task output result. In other words, the image device 150 may invoke the image algorithm model associated with the image processing engine when the associated image processing task is performed in the image processing engine. And the task output result comprises output values of parameter items in the image algorithm model.
For example, when the image processing task is a target detection task, the image device 150 may generate a task output result representing a target detection result when a target is detected. Theservice platform 130 may also receive a task output result of the image device 150 for the image processing task.
In summary, an image algorithm model suitable for running in an image device may be trained by themodel training platform 120 according to embodiments of the present application. Themodel training platform 120 may perform incremental training on the image algorithm model as the number of training materials (e.g., image samples) increases, so as to continuously improve the detection efficiency and detection accuracy of the image algorithm model.
In addition, the image algorithm model newly trained by themodel training platform 120 may be deployed to the image device through theservice platform 130 according to the embodiment of the present application. Thus, the embodiment of the application can avoid the trouble that the image algorithm model can be updated only when the image device updates the firmware, and can flexibly deploy the newly trained image algorithm model in a plurality of image devices.
In addition, the image device of the embodiment of the application can avoid the trouble of uploading the image data to the service platform and performing image processing (such as target detection) by the service platform by using the image algorithm model, and can flexibly execute the newly trained image algorithm model locally. In this way, the embodiments of the present application can flexibly perform the image processing task by making full use of the computing power of the edge node (i.e., each image device in the application scene).
In addition, the embodiment of the present application may flexibly control the image processing task of the image processing engine in each image device through theservice platform 130. Thus, the embodiment of the application can realize the unified management of the image processing tasks in a plurality of image devices and realize the cooperative task processing among different image devices. For example, for multiple image devices in a monitored area, theservice platform 130 may control the multiple image devices to perform the same image processing task, such as vehicle detection. In this way, theservice platform 130 may track movement of the vehicle in the monitored area through vehicle detection operations of the plurality of imaging devices.
In some embodiments,cloud storage platform 140 is deployed in the internet. Different image devices in theapplication scenario 100 may be in different network environments. For example, some image devices may be in a network environment that supports access tocloud storage platform 140, and other image devices may be in a network environment that does not support access tocloud storage platform 140. The image device may set the model transmission manner according to the network environment. Theservice platform 130 may also obtain a model transmission mode of the image device 150. The model transmission mode is used to describe whether the image device 150 supports a push mode and a cloud download mode, wherein the push mode represents that the image algorithm model is pushed to the image device 150 by theservice platform 130. The cloud download mode represents downloading of the image algorithm model from a cloud storage platform. For example, theservice platform 130 may send a get request for the model transfer mode to the image device 150, and the image device 150 may send a response message containing the model transfer mode to theservice platform 130. Examples of the contents of the response message regarding the model transmission mode are as follows:
{
"ISSUPPORUPLOADMODELWITHURL": true,/, bootan, indicates whether the image device 150 supports the cloud download mode >
"ISSUPPORUPLOADMODELWITHPUShBInaryData": true/' indicates whether the image device 150 supports the push mode >
}
When the model transmission mode indicates that the image device 150 supports the cloud download mode, theservice platform 130 sends the access address of the image algorithm model in thecloud storage platform 140 to the image device 150, so that the image device 150 downloads the image algorithm model according to the access address.
When the model transmission mode indicates that the image device 150 does not support the cloud download mode and supports the push mode, theservice platform 130 pushes the image algorithm model to the image device 150. In summary, according to theservice platform 130 of the present application, by determining whether the image device supports the cloud download mode and the push mode, a manner of deploying the image algorithm model to the image device can be flexibly determined. In this way, the embodiment of the present application can improve adaptability to a network environment (i.e., a network communication environment of the image device) by flexibly determining the deployment manner of the model.
In some embodiments, to determine one or more image algorithm models that match image device 150. Theservice platform 130 may send a configuration acquisition request to the image device 150.
The image device 150 sends configuration information to the service platform in response to the configuration acquisition request. The configuration information may include, for example, at least one of: firmware version identification, algorithm version identification, and chip type identification of the image device 150. Wherein the firmware version identification is used to identify the version of the firmware in the image device 150. The chip type identification is used to identify the type of central processor in the image device 150 that runs the firmware. The algorithm version identification is used to identify the version of the image algorithm model supported by the image device 150. In some embodiments, the configuration information may also include an identification of the image processing engine in image device 150. Examples of the data of the configuration information transmitted by the image device 150 are as follows:
theservice platform 130 may receive configuration information from the image device 150. Theservice platform 130 may select one or more image algorithm models matching the configuration information from the trained image algorithm models as one or more image algorithm models matching the image device 150. For example, theservice platform 130 may determine whether an image algorithm model matches the firmware version identification, algorithm version identification, and chip type identification of the image device 150.
Specifically, theservice platform 130 determines whether a firmware version supported by one image algorithm model is identified by a firmware version of the image device 150, determines whether an algorithm version of the image algorithm model matches the algorithm version identification, and determines whether a chip type supported by the image algorithm model matches the chip type identification.
Upon determining that the firmware version identification, the algorithm version identification, and the chip type identification of the image device 150 all match, theservice platform 130 may determine that the image algorithm model matches the image device 150. Thus, theservice platform 130 can select an image algorithm model matching the image device 150 according to the configuration information of the image device 150.
In some embodiments, the number of models supported by image device 150 is consistent with the number of image processing engines, but is not so limited. For example, the image device 150 is configured with 4 image processing engines. The image device 150 supports 4 image algorithm models. Theservice platform 130 may select the same number of image algorithm models as the number of engines of the image device 150.
In some embodiments, the number of models supported by image device 150 may be greater than the number of image processing engines.
In some embodiments, theservice platform 130 may also obtain the number of image processing engines in the image device 150 that are not associated with the image algorithm model. In this way, thebusiness platform 130 can select a consistent number of image algorithm models with the image processing engines of the unassociated models in the image device 150.
In some embodiments, theservice platform 130 may ensure that the number of selected image algorithm models does not exceed the difference value according to the difference value between the upper limit of the number of image algorithm models supported by the image device 150 and the number of image algorithm models existing in the image device 150. In this way, theservice platform 130 can avoid the number of image algorithm models in the image device 150 from being excessive, thereby saving storage resources in the image device 150.
In some embodiments, theservice platform 130 may determine a transmission mode for deploying the matched one or more image algorithm models to the image device according to the model transmission mode. The transmission mode is a push mode or a cloud downloading mode.
On this basis, theservice platform 130 may send a model preset request including an identification of a transmission method to the image device 150. Wherein, when the identification of the transmission mode represents the cloud download mode, the model preset request further comprises: an access address of the one or more image algorithm models in thecloud storage platform 140.
The process of deploying the image algorithm model is described in more detail below in conjunction with FIG. 2. FIG. 2 illustrates a flow diagram of interaction of theservice platform 130 with the image device 150 according to some embodiments of the present application.
As shown in fig. 2, theservice platform 130 may execute step S201 to send a model preset request to the image device 150.
In some embodiments, after receiving the model preset request, the image device 150 may perform step S202. In step S202, it is determined whether running the image algorithm model is supported. In other words, the image device 150 determines whether it has the ability to acquire and run the image algorithm model. For example, image device 150 does not support the image algorithm model when the image processing engine is not configured. When configured with an image processing engine, image device 150 supports an image algorithm model. When the image algorithm model is not supported, the image device 150 returns a failure message to theservice platform 130. In this way, theservice platform 130 and the image device 150 can end the current operational flow, i.e., end the operations associated with deploying the image algorithm model.
Upon determining that the image algorithm model is supported in step S202, the image device 150 may perform step S203. In step S203, it is determined whether the image device 150 has reached the maximum number of supported models. In other words, step S203 may determine whether the number of models existing in the image device 150 reaches the upper limit of the number of models supported by the image device 150. Upon determining that the image device 150 has reached the maximum number of models supported, the image device 150 returns a failure message to the service platform. In this way, through step S203, the image device 150 may detect the number of locally stored image algorithm models before acquiring the image algorithm models, so as to ensure that the number of models in the image device 150 does not exceed the upper limit of the number of models. In this way, the image device 150 may avoid an excessive number of image algorithm models, thereby saving storage resources in the image device 150.
When it is determined in step S203 that the image device 150 does not reach the maximum number of supported models, the image device 150 may execute step S204 to determine whether the transmission mode is the push mode according to the identifier of the transmission mode in the model presetting request. In some embodiments, the model provisioning request is an HTTPS request. Step S204 may use the Content Type (Content-Type) in the HTTPS request as the identifier of the transmission method. When the content type is "multipart/form-data", step S204 may determine the transmission mode as the push mode. When the content type is not "multipart/form-data", step S204 may determine that the model transmission mode is a cloud download mode, i.e., a non-push mode. When the model transmission mode is the cloud download mode, the image device 150 may execute step S205, obtain an access address, a model identifier, a model name, and model description information from the model preset request, and download a data packet of the image algorithm model corresponding to the model identifier from thecloud storage platform 140 according to the access address. In some embodiments, step S205 may establish a communication connection with thecloud storage platform 140 according to the access address, and then send an acquisition request for the image algorithm model to thecloud storage platform 140. The image device 150 acquires the packet length of the image algorithm model from the response message after receiving the response message to the acquisition request. In this way, the image device 150 may download the data package of the image algorithm model from thecloud storage platform 140 according to the length of the data package.
When the model transmission mode is the push mode, the image device 150 may execute step S206 to receive the data packet of the image algorithm model sent by theservice platform 130. In some embodiments, step S206 may obtain the packet length of the image algorithm model from the model preset request. On this basis, step S206 may create a thread to read the received data packet of the image algorithm model.
In some embodiments, the model preset request sent by theservice platform 130 to the image device 150 may further include an association relationship of the image algorithm model with the image processing engine. In this way, theservice platform 130 may instruct the image device 150 to associate the image algorithm model with the image processing engine after acquiring the image algorithm model through the association relationship.
The image apparatus 150 may further perform step S207 after performing step S205 or step S206. In step S207, a digital signature verification operation is performed on the data packet of the image algorithm model, and a decryption operation and a unpacking operation are performed on the data packet after the data signature verification operation is passed, so as to obtain the image algorithm model. Wherein, the decryption operation is to decrypt the encrypted data packet according to a data decryption algorithm. Therefore, the transmission safety of the image algorithm model can be ensured by an encryption transmission mode. The unpacking operation is to unpack the data packet encapsulated according to the transmission protocol to obtain one or more image algorithm models. Fig. 3 shows the processing procedure of the data packet in step S207, for example. Based on the digital signature in thedata packet 301, step S207 may perform digital signature verification. When the data signature verification is successful, step S207 may obtain thedata packet 302. Based on the key signature in thedata packet 302, step S207 may perform a decryption operation on thedata packet 302 to obtain adata packet 303. By the unpacking operation of thedata packet 303, the file data 1, 2, 3, and 4 can be obtained in step S207. Here, the document data 1, 2, 3, and 4 are respectively one image algorithm model.
When the model presetting request includes the association relationship, the image device 150 may further perform step S208 of associating the image algorithm model with the image processing engine. Here, theservice platform 130 may perform scheduling management on image processing resources (i.e., image processing engines) in the image device 150 by instructing the image device 150 to associate an image algorithm model with the image processing engine. In this way, the embodiment of the application can perform unified scheduling management on the image processing engines in the multiple image devices through theservice platform 130, so that the multiple image devices can be used for performing collaborative task processing.
It should be noted that the model presetting request may not include the association relationship between the image algorithm model and the image processing engine. After the image algorithm model is acquired by the image device 150, theservice platform 130 may instruct the image device 150 to associate the image algorithm model with the image processing engine. For example, theservice platform 130 may send an association request to the image device 150. The association request includes an association of the image algorithm model with the image processing engine. In this way, image device 150 may associate the image algorithm model with the image processing engine according to the association relationship. Additionally, theservice platform 130 may also instruct the image device 150 to disassociate the image algorithm model from the image processing engine.
For example, fig. 4A shows image processing engines 151, 152, 153, and 154 in image device 150. In addition, the image device 150 stores image algorithm models 155, 156, 157, and 158. The image processing engine is not associated with the image algorithm model in fig. 4A. FIG. 4B shows a schematic diagram of the image device 150 after the image algorithm model is associated with the image processing engine. Fig. 4B shows the association relationship between the image algorithm model and the image processing engine by a connecting line. For example, the image algorithm model 155 is associated with the image processing engine 151. In summary, theservice platform 130 can control whether the image algorithm model in the image device 150 is associated with the image processing engine.
In addition, theservice platform 130 may perform task deployment operations on the image processing engine. That is, theservice platform 130 may determine that the image processing engines in the image devices 150, 160, and 170 in fig. 1 perform task deployment. Taking the image processing engine 151 in fig. 3 as an example, theservice platform 130 generates a task corresponding to the image processing engine 151, for example, an object detection task 159. Thebusiness platform 130 can send the object detection task 159 corresponding to the image processing engine 151 to the image device 150. Image device 150 may pass through an image algorithm model associated with image processing engine 151. For example, FIG. 4C shows an associated image processing engine 151 and image processing task 159 in image device 150.
In some embodiments, theservice platform 130 may obtain the types of tasks supported by the image device 150. Here, the range of the selectable task types may include, for example, target detection on a single-channel video stream, target detection on a multi-channel video stream for a round, target detection on a designated picture, and target detection on a round grab result, but is not limited thereto. The task type actually supported by the image device 150 may be at least one of a range of task types.
Thebusiness platform 130 can generate one image processing task, such as task a, associated with the task types supported by the image device 150 and with the image processing engine. Theservice platform 130 may send task a to the image device 150. Task a is for example: the image algorithm model 155 is called by the image processing engine 151 to perform the face detection task during the 13:00-14:00 period. Thebusiness platform 130 can also instruct the image device 150 to associate the image processing engine 151 with the image algorithm model 156 after the image device 150 performs task a. Additionally, thebusiness platform 130 can generate and send task B to the image device 150. Task B is for example: during the 14:00-15:00 period, the image algorithm model 156 is called by the image processing engine 151 to perform vehicle detection tasks.
In some embodiments, the image processing task sent by theservice platform 130 is a target detection task for a single-channel video stream. Examples of message content related to image processing tasks are as follows:
the message content as described above may include an identification of the image processing task, an identification of the image processing engine associated with the image processing task, a channel number of the image device 150, a detection frame rate, and an alarm interval. Here, the detection frame rate refers to an image frame rate for target detection. The alert interval represents a time interval for sending the task output result to theservice platform 130.
In some embodiments, theservice platform 130 may perform the image processing task transmitted by theservice platform 130 as a target detection task for the multi-channel video stream of the training. Examples of message content related to image processing tasks are as follows:
in some embodiments, theservice platform 130 may use the image processing task sent by theservice platform 130 as a target detection task for a given picture. Examples of message content related to image processing tasks are as follows:
in some embodiments, theservice platform 130 may perform the image processing task transmitted by theservice platform 130 as a target detection task for the multi-channel video stream of the training. Examples of message content related to image processing tasks are as follows:
in addition, theservice platform 130 may perform a deleting operation, a modifying operation, and an inquiring operation on the image processing task in the image device.
In some embodiments, the image device 150 may generate different task output results for different image processing tasks and send the task output results to thebusiness platform 130.
In some embodiments, the image device 150 sends message content containing the task output result to theservice platform 130 when performing object detection on the single-channel video stream to obtain the task output result. Examples of message content are as follows:
as described above, the task output results may include the contents of "AIOPData".
In some embodiments, the image device 150 sends message content containing the task output result to theservice platform 130 when the task output result is obtained by performing object detection on the specified picture. Examples of message content are as follows:
in some embodiments, the image device 150 sends message content containing the task output results to theservice platform 130 when performing object detection on the trained multi-channel video stream resulting in the task output results. Examples of message content are as follows:
in some embodiments, the image device 150, when performing the target detection on the cartographic result to obtain the task output result, sends message content containing the task output result to theservice platform 130. Examples of message content are as follows:
in some embodiments, themodel training platform 120 is further configured to generate model description information corresponding to the trained image algorithm models, wherein the model description information corresponding to each image algorithm model is used to describe the definitions of the parameter items in the image algorithm model. . In some embodiments, the output results of the image algorithm model may be, for example, the output values of various parameter terms. For example, a parameter item is defined to represent a target type. The output values of the parameter items are, for example, M1, M2, and M3, M1 representing a human face, M2 representing a vehicle, and M3 representing an animal.
Theservice platform 130 may obtain model description information corresponding to the image algorithm model from themodel training platform 120. Theservice platform 130 may parse the task output result according to the model description information to determine an event corresponding to the task output result. For example, based on the task output result corresponding to the target detection task, theservice platform 130 may resolve meanings of parameter values of parameter items in the task output result, so that an event that can be understood by a user may be determined.
In some embodiments, the data content of the task output results is exemplified as follows:
the contents of the model description information are exemplified as follows:
as described above, the model description information of the image algorithm model includes the parameter item "attrType" representing the category number. Wherein the numbers 1-6 are respectively defined as: behavior, gender, jacket color, whether glasses are worn, height, body shape, and expression. For example, when the parameter value of "attrType" is "1", the "attrType" indicates the sex of the subject. In addition, the parameter item "attrValue" indicates values of various numbers in the parameter item "attrttype". For example, when the parameter value of attrType "is" 1 ", the value of" attrValue "is 0 and 1, respectively representing a male and a female.
In the above example of the task output result, the output value of "attrType" is 1, and the output value of "attrValue" is 1. Based on the example of the model description information, theservice platform 130 may determine that the detected target is a male.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of the present application.
Claims (13)
1. An image processing system, comprising:
the model training platform is used for generating a trained image algorithm model;
an image device;
a service platform for:
determining one or more image algorithm models matched with the image equipment from the trained image algorithm models;
deploying the matched one or more image algorithm models to the image device;
instructing the image device to associate an image processing engine in the image device with one of the matched one or more image algorithm models;
determining an image processing task associated with the image processing engine and sending the image processing task to the image device;
the image device is used for executing the image processing task to obtain a task output result in the image processing engine based on an image algorithm model associated with the image processing engine, wherein the task output result comprises output values of parameter items in the image algorithm model.
2. The system of claim 1, wherein the image device is further to: sending the task output result to the service platform;
and the service platform is also used for receiving the task output result.
3. The system of claim 2, wherein the model training platform is further configured to generate model description information corresponding to the trained image algorithm models, wherein the model description information corresponding to each image algorithm model is used to describe the definition of parameter values of parameter items in the image algorithm model;
the service platform is further configured to: acquiring the model description information;
and analyzing the task output result according to the model description information to determine an event corresponding to the task output result.
4. The system of claim 1, wherein the model training platform is further configured to:
storing the trained image algorithm model to a cloud storage platform, and acquiring an access address of the trained image algorithm model in the cloud storage platform;
and sending the trained image algorithm model and the access address to the service platform.
5. The system of claim 2, wherein the service platform is further configured to:
obtaining a model transmission mode of the image equipment, wherein the model transmission mode is used for describing whether the image equipment supports a push mode and a cloud download mode, the push mode represents that an image algorithm model is pushed to the image equipment by the service platform, and the cloud download mode represents that the image algorithm model is downloaded from the cloud storage platform;
and determining a transmission mode for deploying the matched one or more image algorithm models to the image equipment according to the model transmission mode, wherein the transmission mode is a push mode or a cloud download mode.
6. The system of claim 5, wherein the service platform deploys the one or more image algorithm models to the image device according to:
sending a model preset request containing the identifier of the transmission mode to the image device, wherein when the identifier of the transmission mode represents a push mode, the model preset request further comprises: an access address of the one or more image algorithm models in the cloud storage platform;
the camera is further configured to: receiving the model presetting request;
when the identification of the transmission mode represents a cloud downloading mode, downloading the one or more image algorithm models according to the access address;
and when the identification of the transmission mode represents a push mode, receiving the one or more image algorithm models pushed by the service platform.
7. The system of claim 6, wherein the image device performs the downloading of the one or more image algorithm models based on the access address according to:
downloading data packets corresponding to the one or more image algorithm models according to the access address;
and carrying out digital signature verification operation on the data packet, and carrying out decryption operation on the data packet after the data packet passes the data signature verification operation, so as to obtain the one or more image algorithm models.
8. The system of claim 1, wherein the model training platform generates the trained image algorithm model according to:
acquiring one or more image sample sets;
and training the image algorithm model to be trained according to each image sample set to obtain each trained image algorithm model.
9. The system of claim 1,
the service platform is further configured to: sending a configuration acquisition request to the image equipment;
the image device is further to: responding to the configuration acquisition request, and sending configuration information to the service platform, wherein the configuration information comprises: the firmware version identification, the algorithm version identification and the chip type identification of the image equipment;
the service platform determines one or more image algorithm models matched with the image equipment according to the following modes: and selecting one or more image algorithm models matched with the configuration information from the trained image algorithm models, and using the one or more image algorithm models as one or more image algorithm models matched with the image equipment.
10. The system of claim 9, wherein the service platform executes the selected one or more image algorithm models that match the configuration information according to:
for any image algorithm model, determining whether a firmware version supported by the image algorithm model is matched with the firmware version identification;
determining whether the chip type supported by the image algorithm model is matched with the chip type identifier;
determining whether the algorithm version of the image algorithm model is matched with the algorithm version identification;
and when the matching with the firmware version identification, the algorithm version identification and the chip type identification is determined, selecting the image algorithm model as an image algorithm model matched with the configuration information.
11. The system of claim 1, wherein the business platform determines image processing tasks associated with the image processing engine according to:
acquiring task types supported by the image equipment, wherein the supported task types comprise at least one of the following: target detection on a single-channel video stream, target detection on a multi-channel video stream of a round training, target detection on an appointed picture and target detection on a round training grab picture result;
an image processing task corresponding to the supported task type and associated with the image processing engine is generated.
12. The system of claim 1, wherein the service platform is further configured to:
determining whether the image device supports execution of the trained image algorithm model;
when the image device is determined to support the execution of the trained image algorithm model, executing the determination of one or more image algorithm models matched with the image device from the trained image algorithm model.
13. An image device, comprising:
one or more image processing engines;
a processor to:
receiving a model presetting request which is sent by a service platform and contains the identification of the transmission mode, wherein when the identification of the transmission mode represents a push mode, the model presetting request further comprises: an access address of one or more image algorithm models in the cloud storage platform;
when the identification of the transmission mode represents a cloud downloading mode, downloading the one or more image algorithm models according to the access address;
when the identification of the transmission mode represents a pushing mode, receiving the one or more image algorithm models pushed by the service platform;
receiving an indication that a service platform associates one of the one or more image processing engines with one of the one or more image algorithm models, and associating the one image processing engine with the one image algorithm model according to the indication;
receiving an image processing task associated with the one image processing engine;
in the image processing engine, the image processing task is executed to obtain a task output result based on an image algorithm model associated with the image processing engine, wherein the task output result comprises output values of parameter items in the image algorithm model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910640789.XACN112241648A (en) | 2019-07-16 | 2019-07-16 | Image processing system and image device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910640789.XACN112241648A (en) | 2019-07-16 | 2019-07-16 | Image processing system and image device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112241648Atrue CN112241648A (en) | 2021-01-19 |
Family
ID=74166745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910640789.XAPendingCN112241648A (en) | 2019-07-16 | 2019-07-16 | Image processing system and image device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112241648A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113435597A (en)* | 2021-07-07 | 2021-09-24 | 杭州海康威视数字技术股份有限公司 | Model generation and scheduling method, system and device |
| CN113823060A (en)* | 2021-06-22 | 2021-12-21 | 复旦大学附属肿瘤医院 | Intelligent early warning method and device for personnel behaviors |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102891891A (en)* | 2012-09-26 | 2013-01-23 | 广州市动景计算机科技有限公司 | Method and system for cross-device file transmission |
| CN108667850A (en)* | 2018-05-21 | 2018-10-16 | 济南浪潮高新科技投资发展有限公司 | An artificial intelligence service system and method for realizing artificial intelligence service |
| CN108846419A (en)* | 2018-05-25 | 2018-11-20 | 平安科技（深圳）有限公司 | Single page high load image-recognizing method, device, computer equipment and storage medium |
| CN109298868A (en)* | 2018-08-22 | 2019-02-01 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Intelligent dynamic deployment and unloading method for mapping image data processing software |
| CN110009052A (en)* | 2019-04-11 | 2019-07-12 | 腾讯科技（深圳）有限公司 | A kind of method of image recognition, the method and device of image recognition model training |
- 2019
- 2019-07-16CNCN201910640789.XApatent/CN112241648A/enactivePending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102891891A (en)* | 2012-09-26 | 2013-01-23 | 广州市动景计算机科技有限公司 | Method and system for cross-device file transmission |
| CN108667850A (en)* | 2018-05-21 | 2018-10-16 | 济南浪潮高新科技投资发展有限公司 | An artificial intelligence service system and method for realizing artificial intelligence service |
| CN108846419A (en)* | 2018-05-25 | 2018-11-20 | 平安科技（深圳）有限公司 | Single page high load image-recognizing method, device, computer equipment and storage medium |
| CN109298868A (en)* | 2018-08-22 | 2019-02-01 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Intelligent dynamic deployment and unloading method for mapping image data processing software |
| CN110009052A (en)* | 2019-04-11 | 2019-07-12 | 腾讯科技（深圳）有限公司 | A kind of method of image recognition, the method and device of image recognition model training |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113823060A (en)* | 2021-06-22 | 2021-12-21 | 复旦大学附属肿瘤医院 | Intelligent early warning method and device for personnel behaviors |
| CN113435597A (en)* | 2021-07-07 | 2021-09-24 | 杭州海康威视数字技术股份有限公司 | Model generation and scheduling method, system and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230175864A1 (en) | Method, device and system for processing a flight task | |
| TWI534723B (en) | Method and apparatus for recognizing objects in media content | |
| CN113168326A (en) | Method and apparatus for management of network-based media processing functions in a wireless communication system | |
| US11005965B2 (en) | Contextual services in a network using a deep learning agent | |
| CN107741899B (en) | Method, device and system for processing terminal data | |
| CN112906803B (en) | Model integration method, device, server and computer readable storage medium | |
| CN112698848B (en) | Downloading method, device, terminal and storage medium of machine learning model | |
| CN115729683A (en) | Task processing method, device, system, computer equipment and storage medium | |
| CN112241648A (en) | Image processing system and image device | |
| CN113849179A (en) | An artificial intelligence model publishing method and device | |
| US9838496B2 (en) | Managing data acquisition | |
| US12395765B2 (en) | Systems and methods for processing data proximate to the point of collection | |
| CN108989422A (en) | A kind of message and document handling method and system | |
| CN108353028B (en) | Method and apparatus for optimizing complex event handling in software defined networks | |
| CN104537796B (en) | A kind of warning information processing system and processing method | |
| US10104511B2 (en) | Recommendations and notifications over limited connections | |
| CN110830759B (en) | Method, apparatus and system for smart application deployment | |
| CN109819026B (en) | Method and device for transmitting information | |
| Dautov et al. | Towards a global intelligent surveillance system | |
| CN115150474B (en) | Information processing method, apparatus, device and storage medium | |
| CN116980495A (en) | Data conversion method, device, computer equipment, medium and product | |
| JP2020107181A (en) | In-vehicle apparatus, data collection system, and control method of in-vehicle apparatus | |
| JP2005092265A (en) | Data distribution system, server system, receiving terminal, data distribution method used for same and its program | |
| CN108897552A (en) | function realizing method and system | |
| CN113438506A (en) | Video file restoration method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |