CN115273178A - Training of image processing model, image processing method and related device - Google Patents

Abstract

Translated fromChinese

本公开提供了一种图像处理模型的训练、图像处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品，涉及人工智能技术领域，具体为深度学习、图像处理、计算机视觉技术领域。该方法的一具体实施方式包括：获取针对至少两个不同种类的图像处理请求，确定图像处理请求对应的检测特征，构建用于提取检测特征的特征提取层，获取与图像处理请求对应的特征处理层，封装该特征提取层及各特征处理层后得到初始模型，并利用附加有结果标签的样本图像训练该初始模型，得到目标图像处理模型。以该实施方式训练得到图像处理模型，可在仍满足多种图像处理请求的同时，减少图像处理模型的规模，进而减少用于承载图像处理模型的存储空间。

The present disclosure provides training of an image processing model, an image processing method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product, and relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, image processing, and computer vision. A specific implementation of the method includes: acquiring at least two different types of image processing requests, determining detection features corresponding to the image processing requests, constructing a feature extraction layer for extracting the detection features, and acquiring feature processing corresponding to the image processing requests After encapsulating the feature extraction layer and each feature processing layer, an initial model is obtained, and the initial model is trained by using the sample images attached with the result label to obtain the target image processing model. The image processing model obtained by training in this implementation manner can reduce the scale of the image processing model while still satisfying various image processing requests, thereby reducing the storage space for carrying the image processing model.

Description

Translated fromChinese

图像处理模型的训练、图像处理方法及相关装置Training of image processing model, image processing method and related device

技术领域technical field

本公开涉及人工智能技术领域，具体为深度学习、图像处理、计算机视觉技术领域，尤其涉及一种图像处理模型的训练和图像处理方法，以及以对应的装置、电子设备、计算机可读存储介质及计算机程序产品。The present disclosure relates to the technical field of artificial intelligence, specifically to the technical fields of deep learning, image processing, and computer vision, and in particular to an image processing model training and image processing method, as well as corresponding devices, electronic equipment, computer-readable storage media, and Computer Program Products.

背景技术Background technique

在人脸识别闸机、门禁等应用场景下，需要基于人工智能、深度学习技术构建图像处理模型，以完成对人脸图像的处理任务，例如：人脸检测任务、人脸关键点检测任务、人脸活体识别任务、人脸比对任务等。In application scenarios such as face recognition gates and access control, it is necessary to build an image processing model based on artificial intelligence and deep learning technology to complete face image processing tasks, such as: face detection tasks, face key point detection tasks, Human face recognition tasks, face comparison tasks, etc.

现有技术中，需针对不同的处理任务、处理请求单独配置相应的、完整的处理模型，以完成对应的功能、实现对应的请求。In the prior art, corresponding and complete processing models need to be individually configured for different processing tasks and processing requests, so as to complete corresponding functions and realize corresponding requests.

发明内容Contents of the invention

本公开实施例提出了一种图像处理模型的训练、图像处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品。Embodiments of the present disclosure provide an image processing model training, image processing method, device, electronic equipment, computer-readable storage medium, and computer program product.

第一方面，本公开实施例提出了一种图像处理模型的训练方法，包括：获取针对至少两个不同种类的图像处理请求；确定该图像处理请求对应的检测特征；构建用于提取该检测特征的特征提取层；获取与该图像处理请求对应的特征处理层；封装该特征提取层及该特征处理层，得到初始模型；利用附加有结果标签的样本图像训练该初始模型，得到目标图像处理模型。In the first aspect, the embodiment of the present disclosure proposes a training method for an image processing model, including: acquiring at least two different types of image processing requests; determining the detection features corresponding to the image processing requests; constructing a method for extracting the detection features feature extraction layer; obtain the feature processing layer corresponding to the image processing request; encapsulate the feature extraction layer and the feature processing layer to obtain the initial model; use the sample image with the result label to train the initial model to obtain the target image processing model .

第二方面，本公开实施例提出了一种图像处理模型的训练装置，包括：处理请求获取模块，被配置成获取针对至少两个不同种类的图像处理请求；检测特征确定模块，被配置成确定该图像处理请求对应的检测特征；特征提取层构建模块，被配置成构建用于提取该检测特征的特征提取层；特征处理层获取模块，被配置成获取与该图像处理请求对应的特征处理层；初始模型封装模块，被配置成封装该特征提取层及该特征处理层，得到初始模型；目标模型生成模块，被配置成利用附加有结果标签的样本图像训练该初始模型，得到目标图像处理模型。In the second aspect, the embodiment of the present disclosure proposes an image processing model training device, including: a processing request acquisition module configured to acquire image processing requests for at least two different types; a detection feature determination module configured to determine The detection feature corresponding to the image processing request; the feature extraction layer construction module is configured to construct a feature extraction layer for extracting the detection feature; the feature processing layer acquisition module is configured to obtain the feature processing layer corresponding to the image processing request The initial model encapsulation module is configured to encapsulate the feature extraction layer and the feature processing layer to obtain an initial model; the target model generation module is configured to use sample images with result labels to train the initial model to obtain a target image processing model .

第三方面，本公开实施例提出了一种图像处理方法，包括：获取用户上传的原始图像及图像处理请求；基于该图像处理请求调用图像处理模型，该图像处理模型根据如第一方面中任一实现方式描述的图像处理模型的训练方法得到；利用该图像处理模型处理该原始图像，生成与图像处理请求对应的图像处理结果。In the third aspect, the embodiment of the present disclosure proposes an image processing method, including: acquiring the original image uploaded by the user and an image processing request; calling an image processing model based on the image processing request, and the image processing model is based on any The training method of the image processing model described in an implementation manner is obtained; the original image is processed by using the image processing model, and an image processing result corresponding to the image processing request is generated.

第四方面，本公开实施例提出了一种图像处理装置，包括：待处理图像及请求获取模块，被配置成获取用户上传的原始图像及图像处理请求；模型调用模块，被配置成基于该图像处理请求调用图像处理模型，其中，该图像处理模型根据如第二方面中任一实现方式描述的图像处理模型的训练装置得到；图像处理模块，被配置成利用该图像处理模型处理该原始图像，生成与图像处理请求对应的图像处理结果。In the fourth aspect, the embodiment of the present disclosure proposes an image processing device, including: an image to be processed and a request acquisition module configured to acquire an original image uploaded by a user and an image processing request; a model calling module configured to obtain an image processing request based on the image The processing request invokes an image processing model, wherein the image processing model is obtained according to the image processing model training device described in any implementation manner in the second aspect; the image processing module is configured to use the image processing model to process the original image, An image processing result corresponding to the image processing request is generated.

第五方面，本公开实施例提供了一种电子设备，该电子设备包括：至少一个处理器；以及与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，该指令被至少一个处理器执行，以使至少一个处理器执行时能够实现如第一方面中任一实现方式描述的图像处理模型的训练方法或如第三方面中任一实现方式描述的图像处理方法。In a fifth aspect, an embodiment of the present disclosure provides an electronic device, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor , the instructions are executed by at least one processor, so that at least one processor can implement the training method of the image processing model as described in any implementation manner in the first aspect or the image processing method described in any implementation manner in the third aspect when executed. Approach.

第六方面，本公开实施例提供了一种存储有计算机指令的非瞬时计算机可读存储介质，该计算机指令用于使计算机执行时能够实现如第一方面中任一实现方式描述的图像处理模型的训练方法或如第三方面中任一实现方式描述的图像处理方法。In a sixth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, the computer instructions are used to enable a computer to implement the image processing model described in any implementation manner in the first aspect when executed The training method or the image processing method described in any implementation manner in the third aspect.

第七方面，本公开实施例提供了一种包括计算机程序的计算机程序产品，该计算机程序在被处理器执行时能够实现如第一方面中任一实现方式描述的图像处理模型的训练方法或如第三方面中任一实现方式描述的图像处理方法。In the seventh aspect, the embodiments of the present disclosure provide a computer program product including a computer program. When the computer program is executed by a processor, the image processing model training method described in any implementation manner in the first aspect can be implemented or as described in The image processing method described in any implementation manner in the third aspect.

本公开实施例提供的图像处理模型的训练、图像处理方法，可通过构建用于提供各检测特征的提取层的方式，对不同请求下所需的特征进行提取，实现对应各处理请求的处理模型中无需再单独的构建特征提取层的目的，以使得最终得到的图像处理模型在满足多图像处理请求的同时，通过减少特征提取层数量的方式减少图像处理模型的规模，进而减少用于承载图像处理模型的存储空间。The image processing model training and image processing method provided by the embodiments of the present disclosure can extract the required features under different requests by constructing an extraction layer for providing each detection feature, and realize a processing model corresponding to each processing request There is no need to build a separate feature extraction layer, so that the final image processing model can meet the multi-image processing request, and reduce the size of the image processing model by reducing the number of feature extraction layers, thereby reducing the number of images used to carry images. Handles the storage space for the model.

应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

附图说明Description of drawings

通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本公开的其它特征、目的和优点将会变得更明显：Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

图1是本公开可以应用于其中的示例性系统架构；FIG. 1 is an exemplary system architecture to which the present disclosure may be applied;

图2为本公开实施例提供的一种图像处理模型的训练方法的流程图；FIG. 2 is a flowchart of a method for training an image processing model provided by an embodiment of the present disclosure;

图3为本公开实施例提供的一种图像处理模型的训练方法下确定特征提取层的流程图；FIG. 3 is a flowchart of determining a feature extraction layer under an image processing model training method provided by an embodiment of the present disclosure;

图4为本公开实施例提供的在一应用场景下的图像处理模型的训练方法、图像处理方法的流程图；FIG. 4 is a flowchart of an image processing model training method and an image processing method in an application scenario provided by an embodiment of the present disclosure;

图5为本公开实施例提供的一种图像处理模型的训练装置的结构框图；FIG. 5 is a structural block diagram of a training device for an image processing model provided by an embodiment of the present disclosure;

图6为本公开实施例提供的一种图像处理装置的结构框图；FIG. 6 is a structural block diagram of an image processing device provided by an embodiment of the present disclosure;

图7为本公开实施例提供的一种适用于执行图像处理模型的训练方法和/或图像处理方法的电子设备的结构示意图。FIG. 7 is a schematic structural diagram of an electronic device suitable for executing an image processing model training method and/or an image processing method provided by an embodiment of the present disclosure.

具体实施方式Detailed ways

以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。需要说明的是，在不冲突的情况下，本公开中的实施例及实施例中的特征可以相互组合。Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness. It should be noted that, in the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

此外，本公开涉及的技术方案中，所涉及的用户个人信息的获取(例如本公开后续涉及的包含人脸对象的图像)、存储、使用、加工、运输、提供和公开等处理，均符合相关法律法规的规定，且不违背公序良俗。In addition, in the technical solutions involved in this disclosure, the acquisition of user personal information involved (such as images containing human face objects involved in this disclosure), storage, use, processing, transportation, provision, and disclosure are all in compliance with relevant Laws and regulations, and do not violate public order and good customs.

图1示出了可以应用本申请的用于训练人脸识别模型以及识别人脸的方法、装置、电子设备及计算机可读存储介质的实施例的示例性系统架构100。FIG. 1 shows anexemplary system architecture 100 to which embodiments of the method, device, electronic device, and computer-readable storage medium for training a face recognition model and recognizing faces of the present application can be applied.

如图1所示，系统架构100可以包括终端设备101、102、103，网络104和服务器105。网络104用以在终端设备101、102、103和服务器105之间提供通信链路的介质。网络104可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。As shown in FIG. 1 , asystem architecture 100 may includeterminal devices 101 , 102 , 103 , anetwork 104 and aserver 105 . Thenetwork 104 is used as a medium for providing communication links between theterminal devices 101 , 102 , 103 and theserver 105 . Network 104 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.

用户可以使用终端设备101、102、103通过网络104与服务器105交互，以接收或发送消息等。终端设备101、102、103和服务器105上可以安装有各种用于实现两者之间进行信息通讯的应用，例如图像复合处理类应用、深度学习开发类应用、图像处理类应用等。Users can useterminal devices 101 , 102 , 103 to interact withserver 105 vianetwork 104 to receive or send messages and the like. Various applications for information communication between theterminal devices 101, 102, 103 and theserver 105 can be installed, such as image composite processing applications, deep learning development applications, image processing applications, and the like.

终端设备101、102、103和服务器105可以是硬件，也可以是软件。当终端设备101、102、103为硬件时，可以是具有显示屏的各种电子设备，包括但不限于智能手机、平板电脑、膝上型便携计算机和台式计算机等等；当终端设备101、102、103为软件时，可以安装在上述所列举的电子设备中，其可以实现成多个软件或软件模块，也可以实现成单个软件或软件模块，在此不做具体限定。当服务器105为硬件时，可以实现成多个服务器组成的分布式服务器集群，也可以实现成单个服务器；服务器为软件时，可以实现成多个软件或软件模块，也可以实现成单个软件或软件模块，在此不做具体限定。Theterminal devices 101, 102, 103 and theserver 105 may be hardware or software. When theterminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.; when theterminal devices 101, 102 When , 103 is software, it can be installed in the electronic devices listed above, and it can be implemented as multiple software or software modules, or can be implemented as a single software or software module, which is not specifically limited here. When theserver 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; when the server is software, it can be implemented as multiple software or software modules, or as a single software or software The module is not specifically limited here.

服务器105通过内置的各种应用可以提供各种服务，以可以为用户同时人脸检测、人脸活体识别、人脸比对服务的图像复合处理类应用为例，服务器105在运行该活体人脸识别类应用时可实现如下效果：获取用户上传的原始图像及图像处理请求，其中，图像处理请求可以为人脸检测、人脸活体识别；然后，服务器105基于该图像处理请求调用图像处理模型，该图像处理模型可同时实现人脸检测、人脸活体识别两项图像处理请求；最后，服务器105利用该图像处理模型处理该原始图像，生成与图像处理请求对应的图像处理结果。Theserver 105 can provide various services through various built-in applications. Taking the image composite processing application that can provide users with simultaneous face detection, face recognition, and face comparison services as an example, theserver 105 is running the live face The following effects can be achieved when identifying applications: obtain the original image uploaded by the user and the image processing request, wherein the image processing request can be face detection or human face recognition; then, theserver 105 invokes the image processing model based on the image processing request, the The image processing model can simultaneously implement two image processing requests of face detection and face living recognition; finally, theserver 105 uses the image processing model to process the original image, and generates an image processing result corresponding to the image processing request.

其中，图像处理模型可由服务器105上内置的图像处理模型的训练类应用按如下步骤训练得到：服务器105，获取传入的人脸检测、人脸活体识别、人脸比对服务图像处理请求；然后，服务器105，分别确定图像处理请求所对应的检测特征；然后，服务器105，构建用于提取检测特征的特征提取层；然后，服务器105，获取与图像处理请求所对应的特征处理层；然后服务器105，封装该特征提取层及特征处理层得到初始模型；最后，服务器105，利用附加有结果标签的样本图像训练该初始模型，得到目标图像处理模型。Wherein, the image processing model can be obtained by the training application of the built-in image processing model on theserver 105 according to the following steps: theserver 105 obtains the incoming face detection, face living body recognition, and face comparison service image processing requests; and then , theserver 105 determines the corresponding detection feature of the image processing request respectively; then, theserver 105 constructs a feature extraction layer for extracting the detection feature; then, theserver 105 obtains the feature processing layer corresponding to the image processing request; then theserver 105. Encapsulate the feature extraction layer and feature processing layer to obtain an initial model; finally, theserver 105 trains the initial model using sample images with result labels attached to obtain a target image processing model.

由于为训练得到图像处理模型需要占用较多的运算资源和较强的运算能力，因此本申请后续各实施例所提供的图像处理模型的训练方法一般由拥有较强运算能力、较多运算资源的服务器105来执行，相应地，图像处理模型的训练装置一般也设置于服务器105中。但同时也需要指出的是，在终端设备101、102、103也具有满足要求的运算能力和运算资源时，终端设备101、102、103也可以通过其上安装的图像处理模型的训练类应用完成上述本交由服务器105做的各项运算，进而输出与服务器105同样的结果。相应的，图像处理模型的训练装置也可以设置于终端设备101、102、103中。在此种情况下，示例性系统架构100也可以不包括服务器105和网络104。Since the training of the image processing model requires more computing resources and strong computing power, the image processing model training methods provided in the subsequent embodiments of the present application are generally provided by those with strong computing power and more computing resources. Theserver 105 executes, and correspondingly, the training device of the image processing model is generally also set in theserver 105. However, it should also be pointed out that when theterminal devices 101, 102, and 103 also have computing capabilities and computing resources that meet the requirements, theterminal devices 101, 102, and 103 can also complete the training through the image processing model training applications installed on them. The above calculations are performed by theserver 105 , and then output the same results as theserver 105 . Correspondingly, the training device for the image processing model can also be set in theterminal devices 101 , 102 , 103 . In this case,exemplary system architecture 100 may also excludeserver 105 andnetwork 104 .

当然，用于训练得到图像处理模型的服务器可以不同于调用训练好的图像处理模型来使用的服务器。特殊的，经由服务器105训练得到的图像处理模型也可以通过模型蒸馏的方式得到适合置入终端设备101、102、103的轻量级的图像处理模型，即可以根据实际需求的识别准确度灵活选择使用终端设备101、102、103中的轻量级的图像处理模型，还是选择使用服务器105中的较复杂的图像处理模型。Certainly, the server used for training the image processing model may be different from the server used for invoking the trained image processing model. In particular, the image processing model trained by theserver 105 can also obtain a lightweight image processing model suitable for embedding in theterminal devices 101, 102, and 103 through model distillation, that is, it can be flexibly selected according to the recognition accuracy of actual needs Whether to use the lightweight image processing model in theterminal devices 101 , 102 , 103 or choose to use the more complex image processing model in theserver 105 .

应该理解，图1中的终端设备、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备、网络和服务器。It should be understood that the numbers of terminal devices, networks and servers in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

请参考图2，图2为本公开实施例提供的一种图像处理模型的训练方法的流程图，其中流程200包括以下步骤：Please refer to FIG. 2. FIG. 2 is a flow chart of an image processing model training method provided by an embodiment of the present disclosure, wherein theprocess 200 includes the following steps:

步骤201，获取针对至少两个不同种类的图像处理请求。Step 201, acquiring image processing requests for at least two different types.

在本实施例中，该步骤为获取配置图像处理模型、期望进行图像处理模型一侧用户所上传的针对至少两个不同种类的图像处理请求，该图像处理请求与最终得到的图像处理模型所能实现的功能相对应。In this embodiment, this step is to obtain at least two different types of image processing requests uploaded by the user who configures the image processing model and expects to perform the image processing model, and the image processing request and the finally obtained image processing model can corresponding to the realized functions.

在一些实施例中，该图像处理请求包括以下中的至少两种：人脸图像检测请求、人脸活体检测请求、人脸关键点检测请求、人脸比对请求等。In some embodiments, the image processing request includes at least two of the following: face image detection request, face liveness detection request, face key point detection request, face comparison request, and the like.

步骤202，分别确定图像处理请求所对应的检测特征。Step 202, respectively determine the detection features corresponding to the image processing requests.

在本实施例中，在基于上述步骤201中确定各图像处理请求后，基于各图像处理请求确定对应的检测特征，例如人脸检测请求对应的检测特征可以为人脸面部特征、五官特征等，人脸关键点检测请求对应的五官特征等。In this embodiment, after each image processing request is determined based on theabove step 201, the corresponding detection features are determined based on each image processing request. For example, the detection features corresponding to the face detection request can be facial features, facial features, etc. The facial features corresponding to the face key point detection request.

步骤203，构建用于提取检测特征的特征提取层。Step 203, constructing a feature extraction layer for extracting detection features.

在本实施例中，基于上述步骤202中确定检测特征后，构建多个不同的检测特征的特征提取层，即该构建出的特征提取层可同时用于提取多个检测特征，其中，该特征提取层通常可参照残差网络(Residual Network，简称ResNet)、轻量化网络(MobileNet)等构架进行构建。In this embodiment, after the detection features are determined in theabove step 202, a plurality of feature extraction layers of different detection features are constructed, that is, the constructed feature extraction layer can be used to extract multiple detection features at the same time, wherein the feature The extraction layer can usually be constructed with reference to frameworks such as Residual Network (ResNet for short) and lightweight network (MobileNet).

在实践中，在确定各检测特征后，若各检测特征无法基于同一特征提取层同时进行提取，则可对各检测特征进行分组，以通过将可利用同一特征检测层进行特征提取的各检测特征归为同一组、构建同一特征提取层。In practice, after each detection feature is determined, if each detection feature cannot be extracted simultaneously based on the same feature extraction layer, each detection feature can be grouped, so that each detection feature that can use the same feature detection layer for feature extraction Classify into the same group and build the same feature extraction layer.

步骤204，获取与图像处理请求所对应的特征处理层。Step 204, acquiring the feature processing layer corresponding to the image processing request.

在本实施例中，获取对应各图像处理请求的特征处理层，特征处理层与图像处理请求相对应，用于处理对应的检测特征，以实现对应的图像处理请求目的。In this embodiment, the feature processing layer corresponding to each image processing request is acquired, and the feature processing layer corresponds to the image processing request, and is used to process the corresponding detection feature, so as to realize the purpose of the corresponding image processing request.

步骤205，封装特征提取层及特征处理层，得到初始模型。Step 205, encapsulating the feature extraction layer and feature processing layer to obtain an initial model.

在本实施例中，获取基于步骤203中构建得到的特征提取层，以及步骤204中的特征处理层后，进行封装，得到初始模型。In this embodiment, after obtaining the feature extraction layer constructed instep 203 and the feature processing layer instep 204, packaging is performed to obtain an initial model.

步骤206，利用附加有结果标签的样本图像训练该初始模型，得到目标图像处理模型。Instep 206, the initial model is trained by using the sample image with the result label to obtain the target image processing model.

在本实施例中，在基于上述步骤205中得到初始模型后，基于各图像处理请求获取对应的、附加有结果标签的样本图像，例如在对应图像处理请求为人脸检测请求时，该样本图像可以为带有“人脸图像”、“非人脸图像”一类结果标签的样本图像，利用附加有对应各图像处理请求的结果标签的样本图像对初始模型进行训练，以得到最终的目标图像处理模型，该目标图像处理模型具有对应各图像处理请求的图像处理能力。In this embodiment, after the initial model is obtained based on the above-mentionedstep 205, the corresponding sample image with the result label attached is obtained based on each image processing request. For example, when the corresponding image processing request is a face detection request, the sample image can be It is a sample image with a result label such as "face image" and "non-face image", and uses the sample image with the result label corresponding to each image processing request to train the initial model to obtain the final target image processing model, the target image processing model has an image processing capability corresponding to each image processing request.

本公开实施例提供的图像处理模型的训练方法，可通过构建用于提供各检测特征的提取层的方式，对不同请求下所需的特征进行提取，实现对应各处理请求的处理模型中无需再单独的构建特征提取层的目的，以使得最终得到的图像处理模型在满足多图像处理请求的同时，通过减少特征提取层数量的方式减少图像处理模型的规模，进而减少用于承载图像处理模型的存储空间。The image processing model training method provided by the embodiments of the present disclosure can extract the features required under different requests by constructing an extraction layer for providing each detection feature, so that the processing model corresponding to each processing request does not need to The purpose of building a separate feature extraction layer is to make the final image processing model meet the multi-image processing request, and reduce the size of the image processing model by reducing the number of feature extraction layers, thereby reducing the number of images used to carry the image processing model. storage.

在本实施例的一些可选的实现方式中，该方法还包括：将该特征提取层封装为可用于组成其他模型的第一特征提取单元。In some optional implementation manners of this embodiment, the method further includes: encapsulating the feature extraction layer as a first feature extraction unit that can be used to compose other models.

具体的，在构建完成特征提取层后，还可将特征提取层独立进行封装，得到可用于组成其他模型的第一特征提取单元，以便于后续直接通过调取该第一特征提取单元的方式得到对应的特征提取层，获取可用于提取与构建该特征提取层时所针对的图像处理请求对应的检测特征的特征提取层，即通过复用的方式避免针对相同的特征提取层重复构建，使得后续构建、训练图像处理模型时，直接插入该第一特征提取单元，节省构建资源的同时提升后续训练、生成图像处理模型时的效率。Specifically, after the feature extraction layer is built, the feature extraction layer can also be packaged independently to obtain the first feature extraction unit that can be used to form other models, so that the subsequent direct access to the first feature extraction unit can be obtained The corresponding feature extraction layer is used to obtain the feature extraction layer that can be used to extract the detection features corresponding to the image processing request for the construction of the feature extraction layer, that is, to avoid repeated construction for the same feature extraction layer through multiplexing, so that the subsequent When building and training the image processing model, the first feature extraction unit is directly inserted, saving construction resources and improving the efficiency of subsequent training and generating the image processing model.

实践中，还可单独将该第一特征提取单元构建成特征提取模型，以便于利用该特征提取模型直接实现不同种类的检测特征的提取。In practice, the first feature extraction unit can also be constructed separately as a feature extraction model, so as to use the feature extraction model to directly realize the extraction of different types of detection features.

在本实施例的一些可选的实现方式中，该方法还包括：为该第一特征提取单元添加该特征提取单元所支持的图像处理请求的标签信息；将带有该标签信息的第一特征提取单元存入预先配置的特征提取单元数据库。In some optional implementations of this embodiment, the method further includes: adding tag information of the image processing request supported by the feature extraction unit to the first feature extraction unit; adding the first feature with the tag information The extraction units are stored in a pre-configured database of feature extraction units.

具体的，进一步在将特征提取层封装为第一特征提取单元后，还可基于该特征提取层(第一特征提取单元)所支持的图像处理请求生成标签信息，并为该第一特征提取单元添加该标签信息后，将带有标签信息的第一特征提取单元存入预先配置的特征提取单元数据库，以便于后续基于该标签信息从特征提取单元数据库中，提取与本次训练图像处理模型时所对应的图像处理请求的第一特征提取单元，更好的实现第一特征提取单元的存储、复用。Specifically, after further encapsulating the feature extraction layer as the first feature extraction unit, label information can also be generated based on the image processing requests supported by the feature extraction layer (first feature extraction unit), and be used for the first feature extraction unit After adding the label information, the first feature extraction unit with the label information is stored in the pre-configured feature extraction unit database, so that the following information can be extracted from the feature extraction unit database based on the label information when training the image processing model. The first feature extraction unit corresponding to the image processing request better realizes storage and multiplexing of the first feature extraction unit.

请参考图3，图3为本公开实施例提供的一种图像处理模型的训练方法，其中在图2实施例的基础上，在存在有特征提取单元数据库的情况下，如何完成图像处理模型的训练，即在存在有特征提取单元数据库的情况下，对上述图2所示的流程200中的步骤203进行改进的一种实现方式，流程200中的其它步骤并不做调整，也将本实施例所提供的具体实现方式以替换步骤203的方式得到一个新的完整实施例。其中流程300包括以下步骤：Please refer to FIG. 3. FIG. 3 is a training method for an image processing model provided by an embodiment of the present disclosure. On the basis of the embodiment in FIG. Training, that is, in the presence of a feature extraction unit database, an implementation of improvingstep 203 in theprocess 200 shown in Figure 2 above, other steps in theprocess 200 are not adjusted, and this implementation A new complete embodiment is obtained by replacingstep 203 with the specific implementation manner provided in the example. Wherein theprocess 300 includes the following steps:

步骤301，基于检测特征从特征提取单元数据库中提取对应的第二特征提取单元。Step 301, extracting a corresponding second feature extraction unit from the feature extraction unit database based on the detected features.

在本实施例中，确定各检测特征后，基于检测特征从特征提取单元数据库中提取对应第一特征提取单元后，将该第一特征提取单元确定为第二特征提取单元，其中，该特征提取单元数据中的对应第一特征提取单元带有相应的标签信息，该标签信息中记录有对应的第一特征提取单元可实现的图像处理请求与本次的各检测特征相对应，例如检测特征为人脸面部特征、五官特征、活体特征，则该标签信息中所能实现的图像处理请求至少包括上述三个检测特征。In this embodiment, after each detection feature is determined, after the corresponding first feature extraction unit is extracted from the feature extraction unit database based on the detection feature, the first feature extraction unit is determined as the second feature extraction unit, wherein the feature extraction The corresponding first feature extraction unit in the unit data has corresponding label information, and the image processing request that the corresponding first feature extraction unit can realize corresponds to each detection feature this time, for example, the detection feature is human Facial features, facial features, and living body features, the image processing request that can be implemented in the tag information includes at least the above three detection features.

实践中，在存在多个标签信息中记载内容均能实现图像处理时，优选除所请求的图像处理请求之外，所包括的非本次训练图像处理模型所请求的图像处理请求最少的特征提取单元，以进一步减少最后构筑得到的图像处理模型的规模。In practice, when there are multiple tag information that can achieve image processing, it is preferable to extract features that include the least number of image processing requests that are not requested by the training image processing model in addition to the requested image processing requests. units to further reduce the size of the final image processing model constructed.

步骤302，将该第二特征提取单元作为特征提取层。Step 302, the second feature extraction unit is used as a feature extraction layer.

在本实施例中，将基于上述步骤301中提取到的第二特征提取单元作为用于封装得到初始模型的特征提取层。In this embodiment, the second feature extraction unit extracted in theabove step 301 is used as the feature extraction layer for encapsulation to obtain the initial model.

在本实施例的一些可选的实现方式中，该方法还包括：在该数据库中不存在支持所有种类的图像处理请求的第一特征提取单元的情况下，该第二特征提取单元为该第一特征提取单元中支持图像处理请求数量最多的特征提取单元。In some optional implementations of this embodiment, the method further includes: when there is no first feature extraction unit supporting all types of image processing requests in the database, the second feature extraction unit is the first feature extraction unit A feature extraction unit that supports the largest number of image processing requests among the feature extraction units.

具体的，在特征提取单元数据库中不存在支持本次所请求的所有种类的图像处理请求的第一特征提取单元时，获取该特征提取单元数据库中各第一特征提取单元对于本次的图像处理请求中，所能支持的图像处理请求的数量，将支持图像处理请求数量最多的第一特征提取单元确定为第二特征提取单元，以便于基于现有的第一特征提取单元最大程度的减少图像处理模型的规模。Specifically, when there is no first feature extraction unit that supports all types of image processing requests requested this time in the feature extraction unit database, the image processing requirements of each first feature extraction unit in the feature extraction unit database for this time are obtained. In the request, the number of image processing requests that can be supported, the first feature extraction unit that supports the largest number of image processing requests is determined as the second feature extraction unit, so as to minimize image processing based on the existing first feature extraction unit. Handles the scale of the model.

进一步的，还可基于该目标特征提取单元无法实现的图像处理请求生成反馈信息，以便于利用该反馈信息对无法实现的图像处理请求进行反馈，以便于图像处理模型的配置方针对无法实现的图像处理请求进行二次配置，该二次配置的方式通常包括，针对无法实现的图像处理请求单独的配置特征提取层或基于该无法实现的图像处理请求所对应的检测特征，对该提取处的第二特征提取单元进行优化。Further, feedback information can also be generated based on the image processing request that cannot be realized by the target feature extraction unit, so that the feedback information can be used to feed back the image processing request that cannot be realized, so that the configuration policy of the image processing model can be used for the image that cannot be realized. Secondary configuration is performed on the processing request, and the secondary configuration method usually includes, for the unrealizable image processing request, a separate configuration feature extraction layer or based on the detection feature corresponding to the unrealizable image processing request, the first Two feature extraction units are optimized.

上述各实施例从各个方面阐述了如何训练得到图像处理模型，为了尽可能的从实际使用场景突出训练出的图像处理模型所起到的效果，本公开还具体提供了一种使用训练好的图像处理模型来解决实际问题的方案，一种图像处理方法包括如下步骤：The above-mentioned embodiments illustrate how to train the image processing model from various aspects. In order to highlight the effect of the trained image processing model from the actual use scene as much as possible, the disclosure also specifically provides a method of using the trained image A solution for processing models to solve practical problems, an image processing method includes the following steps:

获取用户上传的原始图像及图像处理请求，其中该图像处理请求通常为针对至少两个不同种类的图像处理请求，例如人脸图像检测请求、人脸活体检测请求、人脸关键点检测请求、人脸比对请求等中的至少两种；Obtain the original image uploaded by the user and the image processing request, where the image processing request is usually for at least two different types of image processing requests, such as face image detection request, face liveness detection request, face key point detection request, human At least two of face comparison requests, etc.;

然后，基于用户的图像处理请求调用图像处理模型，该图像处理模型的特征提取层可用于提取不同种类的图像处理请求对应的检测特征，该图像处理模型可基于上述图2、图3实施例中任一实现方式训练得到，此处不再重复进行说明。Then, the image processing model is invoked based on the user's image processing request, and the feature extraction layer of the image processing model can be used to extract detection features corresponding to different types of image processing requests. The image processing model can be based on the above-mentioned Figure 2 and Figure 3 embodiments It can be obtained through training in any implementation manner, and will not be described again here.

最后，利用该图像处理模型处理该原始图像，生成与图像处理请求对应的图像处理结果。Finally, the original image is processed by the image processing model to generate an image processing result corresponding to the image processing request.

为加深理解，本公开还结合一个具体应用场景，给出了一种具体的包括图像处理模型的训练方法、图像处理方法的实现方案，可如图4中所示的流程400所示，具体如下：In order to deepen understanding, this disclosure also provides a specific implementation scheme including a training method of an image processing model and an implementation of an image processing method in combination with a specific application scenario, which can be shown in theprocess 400 shown in FIG. 4 , specifically as follows :

获取用户传入的原始图像，以及图像处理请求(人脸图像检测请求、人脸活体检测请求、人脸关键点检测请求)后，进行该图像处理模型的构建。After obtaining the original image passed in by the user and the image processing request (face image detection request, face liveness detection request, face key point detection request), the image processing model is constructed.

分别确定人脸图像检测请求、人脸活体检测请求、人脸关键点检测请求所对应的检测特征后，基于各检测特征从特征提取单元数据库中提取对应的第二特征提取单元A。After respectively determining the detection features corresponding to the face image detection request, face liveness detection request, and face key point detection request, the corresponding second feature extraction unit A is extracted from the feature extraction unit database based on each detection feature.

将该第二特征提取单元A作为特征提取层后，获取与人脸图像检测请求、人脸活体检测请求、人脸关键点检测请求各自所对应的特征处理层，然后封装第二特征提取层A及各特征处理层后得到初始模型。After the second feature extraction unit A is used as the feature extraction layer, obtain the feature processing layer corresponding to the face image detection request, the face liveness detection request, and the face key point detection request, and then encapsulate the second feature extraction layer A and each feature processing layer to obtain the initial model.

利用附加有结果标签的样本图像训练该初始模型，得到(目标)图像处理模型。This initial model is trained using sample images with resulting labels attached to obtain a (target) image processing model.

最后，利用该图像处理模型对用户传入的原始图像进行处理，得到人脸图像检测结果、人脸活体检测结果、人脸关键点检测结果。Finally, the image processing model is used to process the original image passed in by the user to obtain face image detection results, face liveness detection results, and face key point detection results.

进一步参考图5和图6，作为对上述各图所示方法的实现，本公开分别提供了一种图像处理模型的训练装置实施例和一种图像处理装置的实施例，图像处理模型的训练装置实施例与图2所示的图像处理模型的训练方法实施例相对应，图像处理装置实施例与图像处理方法实施例相对应。上述装置具体可以应用于各种电子设备中。Further referring to FIG. 5 and FIG. 6, as the realization of the methods shown in the above-mentioned figures, the present disclosure provides an embodiment of an image processing model training device and an embodiment of an image processing device, and the image processing model training device The embodiment corresponds to the embodiment of the image processing model training method shown in FIG. 2 , and the embodiment of the image processing device corresponds to the embodiment of the image processing method. The above device can be specifically applied to various electronic devices.

如图5所示，本实施例的图像处理模型的训练装置500可以包括：处理请求获取模块501、检测特征确定模块502、特征提取层构建模块503、特征处理层获取模块504、初始模型封装模块505和目标模型生成模块506。其中，处理请求获取模块501，被配置成获取针对至少两个不同种类的图像处理请求；检测特征确定模块502，被配置成确定该图像处理请求对应的检测特征；特征提取层构建模块503，被配置成构建用于提取该检测特征的特征提取层；特征处理层获取模块504，被配置成获取与该图像处理请求对应的特征处理层；初始模型封装模块505，被配置成封装该特征提取层及该特征处理层，得到初始模型；目标模型生成模块506，被配置成利用附加有结果标签的样本图像训练该初始模型，得到目标图像处理模型。As shown in Figure 5, the image processingmodel training device 500 of this embodiment may include: a processingrequest acquisition module 501, a detectionfeature determination module 502, a feature extractionlayer construction module 503, a feature processinglayer acquisition module 504, and an initialmodel packaging module 505 and targetmodel generation module 506 . Among them, the processingrequest acquisition module 501 is configured to acquire image processing requests for at least two different types; the detectionfeature determination module 502 is configured to determine the detection feature corresponding to the image processing request; the feature extractionlayer construction module 503 is configured It is configured to construct a feature extraction layer for extracting the detection feature; the feature processinglayer acquisition module 504 is configured to acquire the feature processing layer corresponding to the image processing request; the initialmodel encapsulation module 505 is configured to encapsulate the feature extraction layer and the feature processing layer to obtain an initial model; the targetmodel generation module 506 is configured to use the sample images with result labels to train the initial model to obtain a target image processing model.

在本实施例中，图像处理模型的训练装置500中：处理请求获取模块501、检测特征确定模块502、特征提取层构建模块503、特征处理层获取模块504、初始模型封装模块505和目标模型生成模块506的具体处理及其所带来的技术效果可分别参考图2对应实施例中的步骤201-204的相关说明，在此不再赘述。In this embodiment, in the image processing model training device 500: processingrequest acquisition module 501, detectionfeature determination module 502, feature extractionlayer construction module 503, feature processinglayer acquisition module 504, initialmodel encapsulation module 505 and target model generation For the specific processing ofmodule 506 and the technical effects brought about by it, reference may be made to the relevant descriptions of steps 201-204 in the embodiment corresponding to FIG. 2 , which will not be repeated here.

在本实施例的一些可选的实现方式中，该图像处理模型的训练装置500，还包括：特征提取层封装模块，被配置成将该特征提取层封装为可用于组成其他模型的第一特征提取单元。In some optional implementations of this embodiment, the image processingmodel training apparatus 500 further includes: a feature extraction layer encapsulation module configured to encapsulate the feature extraction layer as a first feature that can be used to form other models Extract unit.

在本实施例的一些可选的实现方式中，该图像处理模型的训练装置500，还包括：标签添加模块，被配置成为该第一特征提取单元添加该特征提取单元所支持的图像处理请求的标签信息；数据库存入模块，被配置成将带有该标签信息的第一特征提取单元存入预先配置的特征提取单元数据库。In some optional implementations of this embodiment, the image processingmodel training apparatus 500 further includes: a label adding module configured to add, for the first feature extraction unit, the image processing request supported by the feature extraction unit The tag information; database input module is configured to store the first feature extraction unit with the tag information into a pre-configured feature extraction unit database.

在本实施例的一些可选的实现方式中，该特征提取层构建模块503，包括：特征提取单元提取子模块，被配置成基于该检测特征从该特征提取单元数据库中提取对应的第二特征提取单元；特征提取层构建子模块，被配置成将该第二特征提取单元作为该特征提取层。In some optional implementations of this embodiment, the feature extractionlayer construction module 503 includes: a feature extraction unit extraction submodule configured to extract the corresponding second feature from the feature extraction unit database based on the detection feature An extraction unit; a feature extraction layer construction sub-module configured to use the second feature extraction unit as the feature extraction layer.

在本实施例的一些可选的实现方式中，该特征提取单元提取子模块还包括：在该数据库中不存在支持所有种类的图像处理请求的第一特征提取单元的情况下，该第二特征提取单元为该第一特征提取单元中支持图像处理请求数量最多的特征提取单元。In some optional implementations of this embodiment, the feature extraction unit extraction submodule further includes: when there is no first feature extraction unit supporting all types of image processing requests in the database, the second feature The extraction unit is the feature extraction unit that supports the largest number of image processing requests among the first feature extraction units.

在本实施例的一些可选的实现方式中，该图像处理模型的训练装置500中，该图像处理请求包括以下中的至少两种：人脸图像检测请求、人脸关键点检测请求、人脸活体检测请求。In some optional implementations of this embodiment, in the image processingmodel training device 500, the image processing request includes at least two of the following: face image detection request, face key point detection request, face Liveness detection request.

如图7所示，本实施例的图像处理装置600可以包括：待处理图像及请求获取模块601、模型调用模块602和图像处理模块603。其中，待处理图像及请求获取模块601，被配置成获取用户上传的原始图像及图像处理请求；模型调用模块602，被配置成基于该图像处理请求调用图像处理模型；其中，该图像处理模型根据图像处理模型的训练装置500训练得到；图像处理模块603，利用该图像处理模型处理该原始图像，生成与图像处理请求对应的图像处理结果。As shown in FIG. 7 , theimage processing apparatus 600 of this embodiment may include: an image to be processed andrequest acquisition module 601 , amodel calling module 602 and animage processing module 603 . Among them, the image to be processed andrequest acquisition module 601 is configured to acquire the original image uploaded by the user and the image processing request; themodel calling module 602 is configured to call the image processing model based on the image processing request; wherein the image processing model is based on The image processing model is trained by thetraining device 500; theimage processing module 603 uses the image processing model to process the original image, and generates an image processing result corresponding to the image processing request.

在本实施例中，图像处理装置600中：待处理图像及请求获取模块601、模型调用模块602和图像处理模块603的具体处理及其所带来的技术效果可分别对应方法实施例中的相关说明，在此不再赘述。In this embodiment, in the image processing device 600: the specific processing of the image to be processed and therequest acquisition module 601, themodel calling module 602 and theimage processing module 603 and the technical effects brought by them can respectively correspond to the related methods in the method embodiment description and will not be repeated here.

本实施例作为对应于上述方法实施例的装置实施例存在，本实施例提供的图像处理模型的训练装置以及图像处理装置，可通过构建用于提供各检测特征的提取层的方式，对不同请求下所需的特征进行提取，实现对应各处理请求的处理模型中无需再单独的构建特征提取层的目的，以使得最终得到的图像处理模型在满足多图像处理请求的同时，通过减少特征提取层数量的方式减少图像处理模型的规模，进而减少用于承载图像处理模型的存储空间。This embodiment exists as a device embodiment corresponding to the above-mentioned method embodiment. The image processing model training device and the image processing device provided in this embodiment can implement different requests by constructing an extraction layer for providing detection features. The required features are extracted, so that the processing model corresponding to each processing request does not need to build a separate feature extraction layer, so that the final image processing model can satisfy multiple image processing requests at the same time, by reducing the feature extraction layer Quantitatively reduces the size of the image processing model, thereby reducing the storage space used to carry the image processing model.

根据本公开的实施例，本公开还提供了一种电子设备，该电子设备包括：至少一个处理器；以及与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，该指令被至少一个处理器执行，以使至少一个处理器执行时能够实现上述任一实施例描述的图像处理模型的训练方法和/或图像处理方法。According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, the electronic device includes: at least one processor; and a memory connected in communication with the at least one processor; wherein, the memory stores information executable by the at least one processor. An instruction, the instruction is executed by at least one processor, so that the at least one processor can implement the image processing model training method and/or image processing method described in any of the above embodiments when executed.

根据本公开的实施例，本公开还提供了一种可读存储介质，该可读存储介质存储有计算机指令，该计算机指令用于使计算机执行时能够实现上述任一实施例描述的图像处理模型的训练方法和/或图像处理方法。According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, the readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to implement the image processing model described in any of the above-mentioned embodiments. training methods and/or image processing methods.

本公开实施例提供了一种计算机程序产品，该计算机程序在被处理器执行时能够实现上述任一实施例描述的图像处理模型的训练方法和/或图像处理方法。An embodiment of the present disclosure provides a computer program product. When the computer program is executed by a processor, the image processing model training method and/or the image processing method described in any of the foregoing embodiments can be implemented.

图7示出了可以用来实施本公开的实施例的示例电子设备700的示意性框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。FIG. 7 shows a schematic block diagram of an exampleelectronic device 700 that may be used to implement embodiments of the present disclosure. Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

如图7所示，设备700包括计算单元701，其可以根据存储在只读存储器(ROM)702中的计算机程序或者从存储单元708加载到随机访问存储器(RAM)703中的计算机程序，来执行各种适当的动作和处理。在RAM 703中，还可存储设备700操作所需的各种程序和数据。计算单元701、ROM 702以及RAM 703通过总线704彼此相连。输入/输出(I/O)接口705也连接至总线704。As shown in FIG. 7, thedevice 700 includes acomputing unit 701 that can execute according to a computer program stored in a read-only memory (ROM) 702 or loaded from astorage unit 708 into a random-access memory (RAM) 703. Various appropriate actions and treatments. In theRAM 703, various programs and data necessary for the operation of thedevice 700 can also be stored. Thecomputing unit 701 ,ROM 702 , andRAM 703 are connected to each other through abus 704 . An input/output (I/O)interface 705 is also connected to thebus 704 .

设备700中的多个部件连接至I/O接口705，包括：输入单元706，例如键盘、鼠标等；输出单元707，例如各种类型的显示器、扬声器等；存储单元708，例如磁盘、光盘等；以及通信单元709，例如网卡、调制解调器、无线通信收发机等。通信单元709允许设备700通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。Multiple components in thedevice 700 are connected to the I/O interface 705, including: aninput unit 706, such as a keyboard, a mouse, etc.; anoutput unit 707, such as various types of displays, speakers, etc.; astorage unit 708, such as a magnetic disk, an optical disk, etc. ; and acommunication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like. Thecommunication unit 709 allows thedevice 700 to exchange information/data with other devices over a computer network such as the Internet and/or various telecommunication networks.

计算单元701可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元701的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元701执行上文所描述的各个方法和处理，例如图像处理模型的训练方法和/或图像处理方法。例如，在一些实施例中，图像处理模型的训练方法和/或图像处理方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元708。在一些实施例中，计算机程序的部分或者全部可以经由ROM 702和/或通信单元709而被载入和/或安装到设备700上。当计算机程序加载到RAM 703并由计算单元701执行时，可以执行上文描述的图像处理模型的训练方法和/或图像处理方法的一个或多个步骤。备选地，在其他实施例中，计算单元701可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行图像处理模型的训练方法和/或图像处理方法。Thecomputing unit 701 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computingunits 701 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. Thecomputing unit 701 executes various methods and processes described above, such as a training method of an image processing model and/or an image processing method. For example, in some embodiments, the training method of the image processing model and/or the image processing method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as thestorage unit 708 . In some embodiments, part or all of the computer program may be loaded and/or installed on thedevice 700 via theROM 702 and/or thecommunication unit 709 . When the computer program is loaded into theRAM 703 and executed by thecomputing unit 701, one or more steps of the above-described image processing model training method and/or image processing method may be executed. Alternatively, in other embodiments, thecomputing unit 701 may be configured in any other appropriate way (for example, by means of firmware) to execute an image processing model training method and/or an image processing method.

本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.

用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special purpose computer, or other programmable data processing devices, so that the program codes, when executed by the processor or controller, make the functions/functions specified in the flow diagrams and/or block diagrams Action is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.

可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.

计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决传统物理主机与虚拟专用服务器(VPS，Virtual Private Server)服务中存在的管理难度大，业务扩展性弱的缺陷。A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the management difficulties existing in traditional physical host and virtual private server (VPS, Virtual Private Server) services Large and weak business expansion.

本公开实施例的技术方案，可通过构建用于提供各检测特征的提取层的方式，对不同请求下所需的特征进行提取，实现对应各处理请求的处理模型中无需再单独的构建特征提取层的目的，以使得最终得到的图像处理模型在满足多图像处理请求的同时，通过减少特征提取层数量的方式减少图像处理模型的规模。The technical solution of the embodiment of the present disclosure can extract the required features under different requests by constructing the extraction layer for providing each detection feature, so that the processing model corresponding to each processing request does not need to build a separate feature extraction The purpose of each layer is to reduce the size of the image processing model by reducing the number of feature extraction layers while the final image processing model meets the multi-image processing request.

应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.

上述具体实施方式，并不构成对本公开保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本公开的精神和原则之内所作的修改、等同替换和改进等，均应包含在本公开保护范围之内。The specific implementation manners described above do not limit the protection scope of the present disclosure. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims (17)

1. A method of training an image processing model, comprising:

acquiring image processing requests for at least two different categories;

determining a detection characteristic corresponding to the image processing request;

constructing a feature extraction layer for extracting the detection features;

acquiring a feature processing layer corresponding to the image processing request;

encapsulating the feature extraction layer and the feature processing layer to obtain an initial model;

and training the initial model by using the sample image with the result label to obtain a target image processing model.

2. The method of claim 1, further comprising:

the feature extraction layer is encapsulated as a first feature extraction unit that can be used to compose other models.

3. The method of claim 2, further comprising:

adding label information of the image processing request supported by the feature extraction unit to the first feature extraction unit;

and storing the first feature extraction unit with the label information into a pre-configured feature extraction unit database.

4. The method of claim 3, wherein said constructing a feature extraction layer for extracting said detection features comprises:

extracting a corresponding second feature extraction unit from the feature extraction unit database based on the detection feature;

and taking the second feature extraction unit as the feature extraction layer.

5. The method of claim 4, further comprising:

in the case where there is no first feature extraction unit that supports all kinds of image processing requests in the database, the second feature extraction unit is a feature extraction unit that supports the largest number of image processing requests among the first feature extraction units.

6. The method of claim 1, wherein the image processing request includes at least two of:

the system comprises a face image detection request, a face key point detection request and a face living body detection request.

7. An image processing method comprising:

acquiring an original image uploaded by a user and an image processing request;

calling an image processing model based on the image processing request; wherein the image processing model is obtained according to the training method of the image processing model of any one of claims 1-6;

and processing the original image by using the image processing model to generate an image processing result corresponding to the image processing request.

8. A training apparatus of an image processing model, comprising:

a processing request acquisition module configured to acquire image processing requests for at least two different kinds;

a detection feature determination module configured to determine a detection feature corresponding to the image processing request;

a feature extraction layer construction module configured to construct a feature extraction layer for extracting the detection features;

a feature processing layer acquisition module configured to acquire a feature processing layer corresponding to the image processing request;

an initial model encapsulation module configured to encapsulate the feature extraction layer and the feature processing layer to obtain an initial model;

and the target model generation module is configured to train the initial model by using the sample image attached with the result label to obtain a target image processing model.

9. The apparatus of claim 8, the apparatus further comprising:

a feature extraction layer encapsulation module configured to encapsulate the feature extraction layer as a first feature extraction unit that can be used to compose other models.

10. The apparatus of claim 9, the apparatus further comprising:

a tag addition module configured to add tag information of the image processing request supported by the feature extraction unit to the first feature extraction unit;

and the database storage module is configured to store the first feature extraction unit with the label information into a pre-configured feature extraction unit database.

11. The apparatus of claim 10, wherein the feature extraction layer building module comprises:

a feature extraction sub-module configured to extract a corresponding second feature extraction unit from the feature extraction unit database based on the detected feature;

a feature extraction layer construction sub-module configured to take the second feature extraction unit as the feature extraction layer.

12. The apparatus of claim 11, the feature extraction unit extraction sub-module further comprising:

in the case where there is no first feature extraction unit that supports all kinds of image processing requests in the database, the second feature extraction unit is a feature extraction unit that supports the largest number of image processing requests among the first feature extraction units.

13. The apparatus of claim 7, wherein the image processing request includes at least two of:

the system comprises a face image detection request, a face key point detection request and a face living body detection request.

14. An image processing apparatus comprising:

the image to be processed and request acquisition module is configured to acquire an original image uploaded by a user and an image processing request;

a model calling module configured to call an image processing model based on the image processing request; wherein the image processing model is obtained according to the training device of the image processing model of any one of claims 8-13;

and the image processing module is used for processing the original image by using the image processing model and generating an image processing result corresponding to the image processing request.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of training an image processing model of any of claims 1-6 and/or the method of image processing of claim 7.

16. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of training an image processing model of any one of claims 1-6 and/or the method of image processing of claim 7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method of training an image processing model according to any one of claims 1-6 and/or the method of image processing of claim 7.

Images