CN107679559B

Movatterモバイル変換

Info

Publication number: CN107679559B
Application number: CN201710850225.XA
Authority: CN
Inventors: 柯秀华; 曹威; 王俊
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2017-09-15
Filing date: 2017-09-15
Publication date: 2020-01-10
Anticipated expiration: 2037-09-15
Also published as: CN107679559A; WO2019052316A1

Abstract

Translated fromChinese

本申请实施例涉及一种图像处理方法、装置、计算机可读存储介质和移动终端。上述方法包括：当接收到图像聚类请求，获取所述图像聚类请求对应的目标人脸图像；获取所述目标人脸图像中第一人脸特征信息；根据所述第一人脸特征信息对所述目标人脸图像进行聚类；所述接收到图像聚类请求包括：当特征识别模型更新，接收第一图像聚类请求；当接收到服务器发送的第一聚类信息，接收第二图像聚类请求，所述第一聚类信息为服务器对移动终端上传的第二人脸图像集合的聚类信息；当检测到新增图像，接收第三聚类请求。

The embodiments of the present application relate to an image processing method, an apparatus, a computer-readable storage medium, and a mobile terminal. The above method includes: when an image clustering request is received, obtaining a target face image corresponding to the image clustering request; obtaining first face feature information in the target face image; according to the first face feature information Clustering the target face images; the receiving an image clustering request includes: when the feature recognition model is updated, receiving a first image clustering request; when receiving the first clustering information sent by the server, receiving a second An image clustering request, the first clustering information is the clustering information of the second face image set uploaded by the server to the mobile terminal; when a new image is detected, a third clustering request is received.

Description

Translated fromChinese

图像处理方法、装置、计算机可读存储介质和移动终端Image processing method, apparatus, computer-readable storage medium, and mobile terminal

技术领域technical field

本发明涉及计算机技术领域，特别是涉及一种图像处理方法、装置、计算机可读存储介质和移动终端。The present invention relates to the field of computer technology, and in particular, to an image processing method, an apparatus, a computer-readable storage medium and a mobile terminal.

背景技术Background technique

随着智能移动终端的飞速发展，智能移动终端的功能越来越齐全，智能移动中的性能越来越完善。用户在采用智能移动终端拍照后，智能移动终端可将用户拍摄的图像上传服务器，使服务器根据图像信息进行分类。如根据图像时间信息、或根据图像地点信息，或根据图像中包含的人脸信息对图像进行分类，将相关联的图像分组显示，使得用户可以分门别类的查看图像。With the rapid development of intelligent mobile terminals, the functions of intelligent mobile terminals are becoming more and more complete, and the performance in intelligent mobility is becoming more and more perfect. After the user takes a photo with the intelligent mobile terminal, the intelligent mobile terminal can upload the image taken by the user to the server, so that the server can classify the image according to the image information. For example, the images are classified according to the image time information, or according to the image location information, or according to the face information contained in the images, and the associated images are displayed in groups, so that the user can view the images by category.

发明内容SUMMARY OF THE INVENTION

本申请实施例提供一种图像处理方法、装置、计算机可读存储介质和移动终端，可以对图像按照人脸特征信息进行聚类。Embodiments of the present application provide an image processing method, apparatus, computer-readable storage medium, and mobile terminal, which can cluster images according to facial feature information.

一种图像处理方法，包括：An image processing method, comprising:

当接收到图像聚类请求，获取所述图像聚类请求对应的目标人脸图像；When receiving the image clustering request, obtain the target face image corresponding to the image clustering request;

获取所述目标人脸图像中第一人脸特征信息；Obtain the first face feature information in the target face image;

根据所述第一人脸特征信息对所述目标人脸图像进行聚类；Clustering the target face image according to the first face feature information;

所述接收到图像聚类请求包括：The received image clustering request includes:

当特征识别模型更新，接收第一图像聚类请求；When the feature recognition model is updated, receive a first image clustering request;

当接收到服务器发送的第一聚类信息，接收第二图像聚类请求，所述第一聚类信息为服务器对移动终端上传的第二人脸图像集合的聚类信息；When receiving the first clustering information sent by the server, receiving a second image clustering request, the first clustering information is the clustering information of the second face image set uploaded by the server to the mobile terminal;

当检测到新增图像，接收第三聚类请求。When a newly added image is detected, a third clustering request is received.

一种图像处理装置，包括：An image processing device, comprising:

第一获取模块，用于当接收到图像聚类请求，获取所述图像聚类请求对应的目标人脸图像；a first acquiring module, configured to acquire a target face image corresponding to the image clustering request when an image clustering request is received;

第二获取模块，用于获取所述目标人脸图像中第一人脸特征信息；a second acquisition module, configured to acquire the first face feature information in the target face image;

聚类模块，用于根据所述第一人脸特征信息对所述目标人脸图像进行聚类；a clustering module, configured to perform clustering on the target face image according to the first face feature information;

一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如上所述的方法的步骤。A computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the steps of the method as described above.

一种移动终端，包括存储器及处理器，所述存储器中储存有计算机可读指令，所述指令被所述处理器执行时，使得所述处理器如上所述的方法的步骤。A mobile terminal includes a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the instructions are executed by the processor, the instructions cause the processor to perform the steps of the method described above.

附图说明Description of drawings

为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

图1为一个实施例中图像处理方法的应用环境示意图；1 is a schematic diagram of an application environment of an image processing method in one embodiment;

图2为一个实施例中图1中移动终端110与第一服务器120、第二服务器130进行交互的时序图；FIG. 2 is a sequence diagram of the interaction between themobile terminal 110 and thefirst server 120 and thesecond server 130 in FIG. 1 in one embodiment;

图3为一个实施例中图像处理方法的流程图；3 is a flowchart of an image processing method in one embodiment;

图4为一个实施例中图像处理装置的结构框图；4 is a structural block diagram of an image processing apparatus in one embodiment;

图5为另一个实施例中图像处理装置的结构框图；5 is a structural block diagram of an image processing apparatus in another embodiment;

图6为与本申请实施例提供的移动终端相关的手机的部分结构的框图。FIG. 6 is a block diagram of a partial structure of a mobile phone related to a mobile terminal provided by an embodiment of the present application.

具体实施方式Detailed ways

为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

可以理解，本发明所使用的术语“第一”、“第二”等可在本文中用于描述各种元件，但这些元件不受这些术语限制。这些术语仅用于将第一个元件与另一个元件区分。举例来说，在不脱离本发明的范围的情况下，可以将第一获取模块称为第二获取模块，且类似地，可将第二获取模块称为第一获取模块。第一获取模块和第二获取模块两者都是获取模块，但其不是同一获取模块。It will be understood that the terms "first", "second", etc., as used herein, may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first acquisition module may be referred to as a second acquisition module, and similarly, a second acquisition module may be referred to as a first acquisition module, without departing from the scope of the present invention. Both the first acquisition module and the second acquisition module are acquisition modules, but they are not the same acquisition module.

图1为一个实施例中图像处理方法的应用环境示意图。如图1所示，该应用环境包括移动终端110、第一服务器120和第二服务器130。移动终端110内存和SD(Secure DigitalMemory Card，安全数码卡)卡中可存储图像。移动终端110可对图像进行人脸识别，提取存储的图像中所包含的人脸图像。移动终端110可根据特征识别模型提取人脸图像中人脸特征信息，在将提取出人脸特征信息进行相似度匹配，从而对人脸图像进行聚类。移动终端110还可将人脸图像上传第一服务器120，第一服务器120根据特征识别模型提取人脸图像中人脸特征信息，并将提取出人脸特征信息上传第二服务器130，第二服务器130可对第一服务器120上传的人脸特征信息进行聚类。第二服务器130可将对人脸特征信息聚类的结果返回移动终端110，以使移动终端110将第二服务器130对人脸图像的聚类结果与移动终端110对人脸图像的聚类结果进行比较。其中，移动终端110和第二服务器130的特征识别模型可相同或不同。FIG. 1 is a schematic diagram of an application environment of an image processing method in one embodiment. As shown in FIG. 1 , the application environment includes amobile terminal 110 , afirst server 120 and asecond server 130 . Images can be stored in the internal memory of themobile terminal 110 and an SD (Secure Digital Memory Card, Secure Digital Memory Card) card. Themobile terminal 110 may perform face recognition on the image, and extract the face image contained in the stored image. Themobile terminal 110 may extract the face feature information in the face image according to the feature recognition model, and perform similarity matching on the extracted face feature information, thereby clustering the face images. Themobile terminal 110 can also upload the face image to thefirst server 120, thefirst server 120 extracts the face feature information in the face image according to the feature recognition model, and uploads the extracted face feature information to thesecond server 130, and thesecond server 130 may perform clustering on the facial feature information uploaded by thefirst server 120 . Thesecond server 130 can return the result of clustering the face feature information to themobile terminal 110, so that themobile terminal 110 can compare the clustering result of the face image by thesecond server 130 with the clustering result of the face image by themobile terminal 110 Compare. The feature recognition models of themobile terminal 110 and thesecond server 130 may be the same or different.

在一个实施例中，第一服务器120和第二服务器130可为同一服务器，即移动终端110将识别出人脸图像上传服务器。服务器对接收到的人脸图像进行人脸特征识别获取人脸特征信息，服务器对人脸特征信息进行聚类，并将聚类结果发送给移动终端110。In one embodiment, thefirst server 120 and thesecond server 130 may be the same server, that is, themobile terminal 110 will recognize the face image uploading server. The server performs face feature recognition on the received face image to obtain face feature information, the server clusters the face feature information, and sends the clustering result to themobile terminal 110 .

图2为一个实施例中图1中移动终端110与第一服务器120、第二服务器130进行交互的时序图。如图2所示，移动终端110与第一服务器120、第二服务器130进行交互的过程，主要包括以下步骤：FIG. 2 is a sequence diagram of the interaction between themobile terminal 110 in FIG. 1 and thefirst server 120 and thesecond server 130 in one embodiment. As shown in FIG. 2 , the process of the interaction between themobile terminal 110 and thefirst server 120 and thesecond server 130 mainly includes the following steps:

(1)移动终端110检测到特征识别模型更新，对移动终端110中人脸图像集合进行聚类。(1) Themobile terminal 110 detects that the feature recognition model is updated, and clusters the face image set in themobile terminal 110 .

当移动终端110检测到移动终端110中特征识别模型更新时，可获取移动终端110中存储的人脸图像，采用更新后特征识别模型提取人脸图像中人脸特征信息，对人脸特征信息进行聚类。When themobile terminal 110 detects that the feature recognition model in themobile terminal 110 is updated, the face image stored in themobile terminal 110 can be acquired, the updated feature recognition model is used to extract the face feature information in the face image, and the facial feature information is analyzed. clustering.

(2)移动终端110检测到移动终端110新增图像，对新增图像进行聚类。(2) Themobile terminal 110 detects that themobile terminal 110 newly added images, and performs clustering on the newly added images.

当移动终端110检测到有新增图像时，且新增图像中包含人脸，则提取新增图像中人脸特征信息，将新增图像的人脸特征信息与已聚类人脸图像的人脸特征信息进行匹配，对新增图像进行聚类。When themobile terminal 110 detects that there is a new image, and the new image contains a human face, it extracts the face feature information in the new image, and compares the face feature information of the new image with the person in the clustered face image. The face feature information is matched, and the newly added images are clustered.

(3)移动终端110将人脸图像上传第一服务器120。(3) Themobile terminal 110 uploads the face image to thefirst server 120 .

移动终端110可将移动终端110存储的图像中包含的人脸图像上传到第一服务器120。其中，移动终端110可将内存存储图像中包含的人脸图像上传到第一服务器120，移动终端110也可将SD卡存储图像中包含的人脸图像上传到第一服务器120，移动终端110也可将内存存储图像和SD卡存储图像中包含的人脸图像上传到第一服务器120。Themobile terminal 110 may upload the face image contained in the image stored by themobile terminal 110 to thefirst server 120 . Themobile terminal 110 can upload the face image contained in the image stored in the memory to thefirst server 120, themobile terminal 110 can also upload the face image contained in the image stored in the SD card to thefirst server 120, and themobile terminal 110 also The face image contained in the internal memory storage image and the SD card storage image may be uploaded to thefirst server 120 .

(4)第一服务器120提取人脸图像中人脸特征信息。(4) Thefirst server 120 extracts face feature information in the face image.

第一服务器120接收到移动终端110上传的人脸图像后，可根据特征识别模型从人脸图像中提取人脸特征信息。其中，移动终端110中人脸特征识别模型与服务器中人脸特征识别模型可相同或不同。After receiving the face image uploaded by themobile terminal 110, thefirst server 120 can extract face feature information from the face image according to the feature recognition model. The facial feature recognition model in themobile terminal 110 and the facial feature recognition model in the server may be the same or different.

(5)第一服务器120将人脸特征信息发送给第二服务器130。(5) Thefirst server 120 sends the face feature information to thesecond server 130 .

第一服务器120将获取的人脸特征信息发送给第二服务器130，以使第二服务器130可根据人脸特征信息进行聚类。Thefirst server 120 sends the acquired face feature information to thesecond server 130, so that thesecond server 130 can perform clustering according to the face feature information.

(6)移动终端110向第二服务器130发送聚类请求。(6) Themobile terminal 110 sends a clustering request to thesecond server 130 .

移动终端110将人脸图像上传完毕后，可向第二服务器130发送聚类请求。After uploading the face image, themobile terminal 110 may send a clustering request to thesecond server 130 .

(7)第二服务器130对人脸特征信息进行聚类。(7) Thesecond server 130 performs clustering on the facial feature information.

若第二服务器130接收到第一服务器120发送的人脸特征信息，且第二服务器130接收到移动终端110发送的聚类请求，第二服务器130可对人脸特征信息进行聚类。其中，第二服务器130对人脸特征信息进行聚类包括：将人脸特征信息进行相似度匹配，若相似度超过指定值，则将人脸特征信息划分为一组。移动终端110对人脸特征信息聚类的算法与第二服务器130对人脸特征信息聚类的算法可相同或不同。If thesecond server 130 receives the face feature information sent by thefirst server 120, and thesecond server 130 receives the clustering request sent by themobile terminal 110, thesecond server 130 can cluster the face feature information. Thesecond server 130 clustering the facial feature information includes: matching the facial feature information for similarity, and dividing the facial feature information into one group if the similarity exceeds a specified value. The algorithm for clustering the face feature information by themobile terminal 110 and the algorithm for clustering the face feature information by thesecond server 130 may be the same or different.

(8)第二服务器130将聚类结果返还移动终端110。(8) Thesecond server 130 returns the clustering result to themobile terminal 110 .

第二服务器130在对人脸特征信息聚类完成后，可将聚类结果发送给移动终端110。After thesecond server 130 completes the clustering of the facial feature information, the clustering result may be sent to themobile terminal 110 .

(9)移动终端110根据第二服务器130发送的聚类结果更新移动终端110中图像的聚类信息，对未上传的人脸图像进行聚类。(9) Themobile terminal 110 updates the clustering information of the images in themobile terminal 110 according to the clustering result sent by thesecond server 130, and performs clustering on the unuploaded face images.

移动终端110接收到第二服务器130发送的聚类结果后，可根据上述聚类结果对上传第一服务器120的人脸图像进行聚类。移动终端110还可获取未上传的人脸图像的人脸特征信息，对未上传的人脸图像进行聚类。After receiving the clustering result sent by thesecond server 130, themobile terminal 110 can perform clustering on the face images uploaded to thefirst server 120 according to the foregoing clustering result. Themobile terminal 110 may also acquire face feature information of the unuploaded face images, and perform clustering on the unuploaded face images.

本申请实施例中，移动终端和服务器均可对人脸图像进行特征识别，获取人脸特征信息，在根据人脸特征信息对人脸图像进行人脸聚类。图3为一个实施例中图像处理方法的流程图。如图3所示，一种图像处理方法，应用于移动终端，包括：In this embodiment of the present application, both the mobile terminal and the server can perform feature recognition on a face image, obtain face feature information, and perform face clustering on the face image according to the face feature information. FIG. 3 is a flowchart of an image processing method in one embodiment. As shown in Figure 3, an image processing method, applied to a mobile terminal, includes:

步骤302，当接收到图像聚类请求，获取图像聚类请求对应的目标人脸图像。Step 302, when an image clustering request is received, obtain a target face image corresponding to the image clustering request.

移动终端在接收到图像聚类请求后，可获取图像聚类请求对应的目标人脸图像。其中，移动终端接收到图像聚类请求包括：After receiving the image clustering request, the mobile terminal can acquire the target face image corresponding to the image clustering request. Wherein, the image clustering request received by the mobile terminal includes:

(1)当特征识别模型更新，接收第一图像聚类请求。(1) When the feature recognition model is updated, a first image clustering request is received.

(2)当接收到服务器发送的第一聚类信息，接收第二图像聚类请求，第一聚类信息为服务器对移动终端上传的第二人脸图像集合的聚类信息。(2) When the first clustering information sent by the server is received, a second image clustering request is received, where the first clustering information is the clustering information of the second face image set uploaded by the server to the mobile terminal.

(3)当检测到新增图像，接收第三聚类请求。(3) When a newly added image is detected, a third clustering request is received.

当移动终端检测到特征识别模型更新时会触发第一图像聚类请求；移动终端在接收到服务器下发图像聚类信息时会触发第二图像聚类请求；移动终端在检测到有新增图像时会触发第三图像聚类请求。特征识别模型可用于从人脸图像中提取人脸特征信息，移动终端可根据提取出的人脸特征信息对人脸图像聚类，当特征识别模型更新时，服务器可向移动终端发送新版特征识别模型，移动终端将本地特征识别模型与服务器下发的特征识别模型比较，若本地特征识别模型版本号低于服务器下发的特征识别模型版本号，则移动终端安装新版特征识别模型，并根据服务器下发的新版特征识别模型进行特征识别模型更新。服务器接收到移动终端上传的人脸图像后，根据特征识别模型对人脸图像进行特征识别，获取人脸特征信息，再对人脸特征信息进行聚类，服务器发送给移动终端的图像聚类结果即为服务器下发的第一聚类信息。其中，移动终端的特征识别模型与服务器的特征识别模型可相同或不同。移动终端上传服务器的人脸图像为移动终端内存存储的人脸图像。图像聚类信息中包括图像ID(Identification，身份证)和图像分组标记，移动终端在接收到图像聚类信息后，根据图像ID查找图像，根据图像分组标记将图像划分到对应的分组。When the mobile terminal detects that the feature recognition model is updated, it will trigger the first image clustering request; when the mobile terminal receives the image clustering information sent by the server, it will trigger the second image clustering request; when the mobile terminal detects that there is a new image will trigger a third image clustering request. The feature recognition model can be used to extract face feature information from face images, and the mobile terminal can cluster the face images according to the extracted face feature information. When the feature recognition model is updated, the server can send a new version of the feature recognition model to the mobile terminal. model, the mobile terminal compares the local feature recognition model with the feature recognition model issued by the server, if the version number of the local feature recognition model is lower than the version number of the feature recognition model issued by the server, the mobile terminal installs a new version of the feature recognition model, and according to the server The new version of the feature recognition model is released to update the feature recognition model. After receiving the face image uploaded by the mobile terminal, the server performs feature recognition on the face image according to the feature recognition model, obtains face feature information, and then clusters the face feature information, and the server sends the image clustering result to the mobile terminal. That is, the first clustering information sent by the server. The feature recognition model of the mobile terminal and the feature recognition model of the server may be the same or different. The face image uploaded by the mobile terminal to the server is the face image stored in the memory of the mobile terminal. The image clustering information includes an image ID (Identification, identity card) and an image grouping mark. After receiving the image clustering information, the mobile terminal searches for an image according to the image ID, and divides the image into corresponding groups according to the image grouping mark.

在接收到图像聚类请求后，移动终端可获取图像聚类请求对应的目标人脸图像。上述目标人脸图像即为待聚类的人脸图像。After receiving the image clustering request, the mobile terminal may obtain the target face image corresponding to the image clustering request. The above target face image is the face image to be clustered.

当特征识别模型更新时，即从人脸图像中提取人脸特征信息的算法更新，移动终端获取当前移动终端中存储的所有图像，包括内存图像和SD卡存储图像。对上述内存图像和SD卡存储图像进行人脸扫描，识别上述内存图像和SD卡存储图像中包含的人脸图像。上述人脸扫描是指根据人脸识别算法从图像中识别人脸，获取包含的人脸图像。当移动终端接收到服务器下发第一聚类信息时，即移动终端接收到服务器下发的人脸图像的分组信息时，移动终端获取SD卡存储图像，对SD卡存储图像进行人脸扫描，识别SD卡存储图像中包含的人脸图像。其中，在服务器下发图像聚类信息之前，移动终端可将内存图像中包含的人脸图像上传服务器，服务器根据特征识别模型对获取的人脸图像进行特征识别，获取人脸图像中人脸特征信息，根据人脸特征信息对人脸图像进行聚类，将图像聚类信息发送给移动终端。当移动终端新增图像时，移动终端获取新增图像，对新增图像进行人脸扫描，识别新增图像中包含的人脸图像。When the feature recognition model is updated, that is, the algorithm for extracting face feature information from the face image is updated, the mobile terminal acquires all images currently stored in the mobile terminal, including internal memory images and SD card storage images. Perform face scanning on the above-mentioned internal memory image and SD card storage image, and identify the human face image contained in the above-mentioned internal memory image and SD card storage image. The above-mentioned face scanning refers to recognizing the face from the image according to the face recognition algorithm, and obtaining the included face image. When the mobile terminal receives the first clustering information sent by the server, that is, when the mobile terminal receives the grouping information of the face image sent by the server, the mobile terminal obtains the image stored in the SD card, and performs face scanning on the image stored in the SD card, Identify face images contained in SD card stored images. Before the server sends the image clustering information, the mobile terminal can upload the face image contained in the memory image to the server, and the server performs feature recognition on the obtained face image according to the feature recognition model, and obtains the face features in the face image. information, cluster the face images according to the face feature information, and send the image clustering information to the mobile terminal. When the mobile terminal adds an image, the mobile terminal acquires the newly added image, performs a face scan on the newly added image, and identifies the face image contained in the newly added image.

当特征识别模型更新时，目标人脸图像为移动终端中第一人脸图像集合，即移动终端内存和SD卡中存储的人脸图像。当接收到服务器发送的第一聚类请求时，目标人脸图像为第一人脸图像集合中除第二人脸图像集合外的人脸图像。当检测到新增图像时，对新增图像进行人脸识别，若新增图像中包含人脸，则将新增图像作为目标人脸图像。When the feature recognition model is updated, the target face image is the first set of face images in the mobile terminal, that is, the face images stored in the internal memory of the mobile terminal and the SD card. When receiving the first clustering request sent by the server, the target face image is the face image in the first face image set except the second face image set. When the newly added image is detected, face recognition is performed on the newly added image, and if the newly added image contains a human face, the newly added image is used as the target face image.

步骤304，获取目标人脸图像中第一人脸特征信息。Step 304: Obtain first face feature information in the target face image.

在获取目标人脸图像后，移动终端可根据特征识别模型对目标人脸图像特征识别，提取目标人脸图像中第一人脸特征信息。人脸特征信息是指用于标识唯一人脸的信息，包括人脸的轮廓特征、五官特征等。After acquiring the target face image, the mobile terminal can identify the features of the target face image according to the feature recognition model, and extract the first face feature information in the target face image. Facial feature information refers to information used to identify a unique face, including contour features and facial features of the face.

步骤306，根据第一人脸特征信息对目标人脸图像进行聚类。Step 306: Cluster the target face images according to the first face feature information.

在获取到目标人脸图像的第一人脸特征信息后，可根据第一人脸特征信息对目标人脸图像进行聚类。其中，当特征识别模型更新时，移动终端直接将第一人脸图像集合的第一人脸特征信息进行相似度匹配，根据匹配结果对第一人脸特征信息进行聚类。当接收到服务器发送的第一聚类信息后，移动终端可将目标人脸图像的第一人脸特征信息与第二人脸图像集合的人脸特征信息进行相似度匹配，根据匹配结果对第一人脸特征信息进行聚类。当检测到新增图像时，移动终端可将目标人脸图像的第一人脸特征信息与已聚类人脸图像的人脸特征信息进行相似度匹配，根据匹配结果对第一人脸特征信息进行聚类。After acquiring the first face feature information of the target face image, the target face image can be clustered according to the first face feature information. Wherein, when the feature recognition model is updated, the mobile terminal directly performs similarity matching on the first face feature information of the first face image set, and clusters the first face feature information according to the matching result. After receiving the first clustering information sent by the server, the mobile terminal can perform similarity matching between the first face feature information of the target face image and the face feature information of the second set of face images, and perform similarity matching on the first face feature information according to the matching result. Face feature information for clustering. When the newly added image is detected, the mobile terminal can perform similarity matching between the first face feature information of the target face image and the face feature information of the clustered face image, and match the first face feature information to the first face feature information according to the matching result. to cluster.

本申请实施例中图像处理方法，在接收到不同的图像聚类请求时，根据图像聚类请求获取对应的目标人脸图像，实现不同的图像聚类请求获取不同的目标人脸图像，并非获取移动终端中所有图像，节省了移动终端资源，降低了移动终端资源消耗。In the image processing method in the embodiment of the present application, when different image clustering requests are received, the corresponding target face images are obtained according to the image clustering requests, and different image clustering requests are implemented to obtain different target face images, rather than obtaining All images in the mobile terminal save mobile terminal resources and reduce mobile terminal resource consumption.

在一个实施例中，接收到第一图像聚类请求，获取目标人脸图像中第一人脸特征信息包括：根据更新后特征识别模型对目标人脸图像进行人脸识别，获取第一人脸特征信息；或，根据特征信息转换模型将目标人脸图像的已存储人脸特征信息转换为第一人脸特征信息。In one embodiment, receiving the first image clustering request and obtaining the first face feature information in the target face image includes: performing face recognition on the target face image according to the updated feature recognition model, and obtaining the first face feature information; or, converting the stored face feature information of the target face image into the first face feature information according to the feature information conversion model.

当移动终端中特征识别模型更新时，即对人脸图像提取人脸特征信息的算法更新，移动终端可获取存储图像中人脸图像的集合作为目标人脸图像，即移动终端中内存图像和SD卡图像中人脸图像的集合作为目标人脸图像。When the feature recognition model in the mobile terminal is updated, that is, the algorithm for extracting face feature information from the face image is updated, and the mobile terminal can obtain the set of face images in the stored image as the target face image, that is, the memory image in the mobile terminal and the SD The collection of face images in the card image serves as the target face image.

移动终端可用更新后特征识别模型对目标人脸图像进行人脸识别，获取目标人脸图像中第一人脸特征信息。更新后特征识别模型提取的人脸特征信息与原有特征识别模型提取的人脸特征信息不一致，移动终端在采用更新后特征识别模型重新提取目标人脸中第一人脸特征信息后，可根据重新提取的第一人脸特征信息对目标人脸图像进行聚类。The mobile terminal can perform face recognition on the target face image by using the updated feature recognition model, and obtain the first face feature information in the target face image. The face feature information extracted by the updated feature recognition model is inconsistent with the face feature information extracted by the original feature recognition model. After the mobile terminal uses the updated feature recognition model to re-extract the first face feature information in the target face, it can The re-extracted first face feature information is used to cluster the target face image.

在一个实施例中，当移动终端中特征识别模型更新后，移动终端可将目标人脸图像的已存储人脸特征信息转换为更新后特征识别模型对应的人脸特征信息，即将目标人脸图像的已存储人脸特征信息转换为第一人脸特征信息。移动终端可根据第一人脸特征信息对目标人脸图像重新聚类。In one embodiment, after the feature recognition model in the mobile terminal is updated, the mobile terminal can convert the stored face feature information of the target face image into the face feature information corresponding to the updated feature recognition model, that is, the target face image The stored face feature information of is converted into the first face feature information. The mobile terminal may re-cluster the target face images according to the first face feature information.

本申请实施例中图像处理方法，在移动终端中特征识别模型更新后，重新提取人脸图像的人脸特征信息。实现在特征识别模型更新后，及时对人脸图像进行聚类，提高了人脸图像聚类的及时性。In the image processing method in the embodiment of the present application, after the feature recognition model in the mobile terminal is updated, the face feature information of the face image is re-extracted. After the feature recognition model is updated, the face images are clustered in time, which improves the timeliness of face image clustering.

在一个实施例中，当接收到第二图像聚类请求，获取图像聚类请求对应的目标人脸图像包括：获取第一人脸图像集合中除第二人脸图像集合外人脸图像作为目标人脸图像；根据第一人脸特征信息对目标人脸图像进行聚类包括：将第一人脸特征信息与第二图像集合中第二人脸特征信息进行匹配，若相似度超过预设值，获取第二人脸特征信息对应的第一图像，将目标人脸图像划分到第一图像在第一聚类信息中对应的聚类分组。In one embodiment, when a second image clustering request is received, obtaining the target face image corresponding to the image clustering request includes: obtaining face images in the first face image set except the second face image set as the target person face image; clustering the target face image according to the first face feature information includes: matching the first face feature information with the second face feature information in the second image set, if the similarity exceeds a preset value, The first image corresponding to the second face feature information is acquired, and the target face image is divided into clustering groups corresponding to the first image in the first clustering information.

移动终端可将内存图像中人脸图像上传服务器，服务器可对接收到人脸图像进行图像聚类，并将聚类结果发送给移动终端。若移动终端检测到服务器下发图像聚类信息，且服务器下发图像聚类信息对应的图像集合与第一人脸图像集合不等同，则根据服务器下发图像聚类信息更新移动终端第二人脸图像集合的聚类信息，再获取第一人脸图像集合中除第二人脸图像集合外人脸图像，即SD卡中人脸图像和服务器下发的图像聚类信息中不包括的内存人脸图像。移动终端可获取目标人脸图像的人脸特征信息，将上述目标人脸图像的人脸特征信息与第二人脸图像集合的人脸特征信息进行相似度匹配，若相似度大于预设阈值，则将上述目标人脸图像划分到第二人脸图像集合的人脸特征信息对应的聚类分组。The mobile terminal can upload the face image in the memory image to the server, and the server can perform image clustering on the received face image, and send the clustering result to the mobile terminal. If the mobile terminal detects that the image clustering information sent by the server and the image set corresponding to the image clustering information sent by the server is not equal to the first face image set, the second person on the mobile terminal is updated according to the image clustering information sent by the server. The clustering information of the face image set, and then obtain the face images in the first face image set except the second face image set, that is, the face image in the SD card and the memory person not included in the image clustering information sent by the server. face image. The mobile terminal may obtain the face feature information of the target face image, and perform similarity matching between the face feature information of the target face image and the face feature information of the second face image set, and if the similarity is greater than a preset threshold, Then, the above-mentioned target face image is divided into clustering groups corresponding to the face feature information of the second face image set.

本申请实施例中图像处理方法，在接收到服务器下发的图像聚类信息后，以服务器下发的图像聚类信息为标准对未上传服务器的人脸图像进行聚类，提高了对人脸图像聚类的准确性。In the image processing method in the embodiment of the present application, after receiving the image clustering information sent by the server, the image clustering information sent by the server is used as the standard to cluster the face images that have not been uploaded to the server, which improves the accuracy of face images. Image clustering accuracy.

在一个实施例中，移动终端在接收到服务器下发图像聚类信息后，根据服务器下发的图像聚类信息更新本地图像聚类信息。若检测到对同一图像，服务器下发的图像聚类信息与本地图像聚类信息不一致时，检测在本地图像聚类信息中，检测上述图像是否带有用户操作标记，若是，移动终端保存上述图像的本地聚类信息，并将图像的本地聚类信息上传服务器，使服务器用图像的本地聚类信息覆盖原有图像聚类信息；若否，移动终端用云端下发的图像聚类信息覆盖本地图像聚类信息。In one embodiment, after receiving the image clustering information delivered by the server, the mobile terminal updates the local image clustering information according to the image clustering information delivered by the server. If it is detected that the image clustering information sent by the server is inconsistent with the local image clustering information for the same image, it is detected in the local image clustering information to detect whether the above-mentioned image has a user operation mark, and if so, the mobile terminal saves the above-mentioned image and upload the local clustering information of the image to the server, so that the server uses the local clustering information of the image to overwrite the original image clustering information; if not, the mobile terminal uses the image clustering information delivered by the cloud to overwrite the local clustering information Image clustering information.

本申请实施例中图像处理方法，根据服务器下发的图像聚类信息更新移动终端的图像聚类信息时，若检测到对同一图像的图像聚类结果不同，且在移动终端上聚类结果为用户操作，则保留用户操作，并将用户操作的聚类信息上传服务器。上述方法，不仅保证了移动终端和服务器的多端数据同步，避免了数据混乱，又保留了用户操作，提高了用户粘性。In the image processing method in this embodiment of the present application, when updating the image clustering information of the mobile terminal according to the image clustering information sent by the server, if it is detected that the image clustering results for the same image are different, and the clustering results on the mobile terminal are: User operation, the user operation is retained, and the clustering information of the user operation is uploaded to the server. The above method not only ensures the synchronization of multi-terminal data between the mobile terminal and the server, avoids data confusion, but also preserves user operations and improves user stickiness.

在一个实施例中，当接收到第三图像聚类请求，获取图像聚类请求对应的目标人脸图像包括：对新增图像进行人脸识别，若新增图像中包含人脸，将包含人脸的新增图像作为目标人脸图像。In one embodiment, when a third image clustering request is received, acquiring the target face image corresponding to the image clustering request includes: performing face recognition on the newly added image, and if the newly added image contains a human face, it will contain a human face. The newly added image of the face is used as the target face image.

移动终端检测到有新增图像时，对上述新增图像进行人脸扫描，识别新增图像是否为人脸图像，若是，获取新增图像中人脸特征信息；若否，对新增图像不做处理。在获取新增图像中人脸特征信息后，将新增图像的人脸特征信息已聚类图像的人脸特征信息进行相似度比对，若相似度超过预设值，则将新增图像划分到已聚类图像对应的图像聚类。When the mobile terminal detects that there is a newly added image, it performs a face scan on the above newly added image to identify whether the newly added image is a face image, and if so, obtains the face feature information in the newly added image; deal with. After acquiring the face feature information in the new image, compare the face feature information of the new image with the face feature information of the clustered images. If the similarity exceeds the preset value, the new image will be divided into two groups. to the image clustering corresponding to the clustered image.

本申请实施例中信息处理方法，在检测到移动终端新增图像时，将新增图像的人脸特征信息与已聚类图像的人脸特征信息进行比对，保证了对新增图像进行聚类的及时性。In the information processing method in the embodiment of the present application, when a new image is detected on the mobile terminal, the face feature information of the new image is compared with the face feature information of the clustered images, so as to ensure that the clustering of the newly added images is performed. Class timeliness.

在一个实施例中，当移动终端存储图像被删除时，移动终端检测上述图像是否有图像聚类信息，若有，则删除上述图像对应的图像聚类信息。例如，移动终端中第一分组Group1包括图像1.jpg，第二分组Group2也包括图像1.jpg，当图像1.jpg从移动终端删除时，会对应从第一分组Group1和第二分组Group中删除图像1.jpg，即在移动终端相册第一分组Group1和第二分组Group2中不再显示图像1.jpg。In one embodiment, when the image stored by the mobile terminal is deleted, the mobile terminal detects whether the image has image clustering information, and if so, deletes the image clustering information corresponding to the image. For example, the first group Group1 in the mobile terminal includes image 1.jpg, and the second group Group2 also includes image 1.jpg. When image 1.jpg is deleted from the mobile terminal, it will be correspondingly removed from the first group Group1 and the second group Group. The image 1.jpg is deleted, that is, the image 1.jpg is no longer displayed in the first group Group1 and the second group Group2 of the album of the mobile terminal.

在一个实施例中，在获取图像聚类请求对应的目标人脸图像之前，当满足以下条件中至少一个时，获取图像聚类请求对应的目标人脸图像：In one embodiment, before obtaining the target face image corresponding to the image clustering request, when at least one of the following conditions is met, obtain the target face image corresponding to the image clustering request:

(1)当前时刻与上一次获取目标人脸图像的时刻之间的时间差超过预设时长。(1) The time difference between the current moment and the moment when the target face image was acquired last time exceeds a preset duration.

(2)当前时刻为预设时刻。(2) The current time is the preset time.

(3)移动终端处于充电状态。(3) The mobile terminal is in a charging state.

移动终端在接收到图像聚类请求后，可检测当前条件是否满足预设条件，若检测到当前条件满足预设条件，则获取图像聚类请求对应的目标人脸图像；若检测到当前条件不满足预设条件，则在移动终端满足预设条件时，再根据人脸扫描的类型确定待扫描图像。After receiving the image clustering request, the mobile terminal can detect whether the current condition satisfies the preset condition, and if it is detected that the current condition satisfies the preset condition, obtain the target face image corresponding to the image clustering request; If the preset conditions are met, when the mobile terminal satisfies the preset conditions, the image to be scanned is determined according to the type of face scanning.

在一个实施例中，在接收到图像聚类请求后，移动终端检测当前条件是否满足预设条件，若不满足，则移动终端按照预设的时间间隔检测当前条件是否满足预设条件。In one embodiment, after receiving the image clustering request, the mobile terminal detects whether the current condition satisfies the preset condition, and if not, the mobile terminal detects whether the current condition satisfies the preset condition according to the preset time interval.

在一个实施例中，移动终端按照预设的时间间隔检测当前条件是否满足预设条件，若检测到移动终端满足预设条件，再检测移动终端是否接收到图像聚类请求，若移动终端接收到图像聚类请求，获取图像聚类请求对应的目标人脸图像。In one embodiment, the mobile terminal detects whether the current condition satisfies the preset condition according to the preset time interval, and if it is detected that the mobile terminal satisfies the preset condition, then detects whether the mobile terminal receives an image clustering request, and if the mobile terminal receives the image clustering request Image clustering request, obtain the target face image corresponding to the image clustering request.

上述预设条件包括：获取移动终端上一次获取目标人脸图像的时刻，检测当前时刻与上一次获取目标人脸图像的时刻之间的时间差超过预设时长。例如，预设时长为48小时，当前时刻为2017年8月11日10点18分，上一次获取目标人脸图像的时刻为2017年8月8日9点5分，时间差为73小时13分超过预设时长48小时，则移动终端满足预设条件，获取图像聚类请求对应的目标人脸图像。移动终端还可检测当前时刻是否为预设时刻，若当前时刻为预设时刻，则获取图像聚类请求对应的目标人脸图像。例如，预设时刻为2:00AM至5:00AM，移动终端检测到当前时刻为3:15AM，则获取图像聚类请求对应的目标人脸图像。当移动终端当前处于充电状态时，移动终端也可获取图像聚类请求对应的目标人脸图像。The above preset conditions include: acquiring the time when the mobile terminal obtained the target face image last time, and detecting that the time difference between the current time and the last time the target face image was obtained exceeds a preset time length. For example, the preset duration is 48 hours, the current time is 10:18 on August 11, 2017, the last time the target face image was acquired is 9:5 on August 8, 2017, and the time difference is 73 hours and 13 minutes If the preset duration exceeds 48 hours, the mobile terminal satisfies the preset conditions and obtains the target face image corresponding to the image clustering request. The mobile terminal can also detect whether the current time is the preset time, and if the current time is the preset time, obtain the target face image corresponding to the image clustering request. For example, the preset time is 2:00AM to 5:00AM, and the mobile terminal detects that the current time is 3:15AM, and acquires the target face image corresponding to the image clustering request. When the mobile terminal is currently in a charging state, the mobile terminal may also acquire the target face image corresponding to the image clustering request.

本申请实施例中图像处理方法，在移动终端获取图像聚类请求对应的目标人脸图像之前，判定移动终端是否满足预设条件，在满足预设条件时获取图像聚类请求对应的目标人脸图像。由于移动终端对图像聚类耗时较长，且占用移动终端CPU资源，消耗移动终端电量。在移动终端处于充电时启用移动终端图像聚类，可以避免移动终端电量消耗过快的情况。在移动终端处于预设时刻时启动移动终端图像聚类，可避免移动终端图像聚类占用大量CPU资源造成移动终端卡顿的情况。在距离上次图像聚类的时间差超过指定时长即进行图像聚类，保证了图像聚类的及时性。In the image processing method in the embodiment of the present application, before the mobile terminal obtains the target face image corresponding to the image clustering request, it is determined whether the mobile terminal satisfies the preset conditions, and when the preset conditions are met, the target face corresponding to the image clustering request is obtained. image. Because the mobile terminal takes a long time to cluster the images, it occupies CPU resources of the mobile terminal, and consumes the power of the mobile terminal. The mobile terminal image clustering is enabled when the mobile terminal is charging, which can avoid the situation that the power of the mobile terminal is consumed too quickly. Starting the mobile terminal image clustering when the mobile terminal is at a preset time can avoid the situation that the mobile terminal is stuck due to the mobile terminal image clustering occupying a lot of CPU resources. Image clustering is performed when the time difference from the last image clustering exceeds the specified time, which ensures the timeliness of image clustering.

在一个实施例中，上述图像处理方法包括：当检测到目标人脸图像中人脸信息变更，重新获取目标人脸图像中第一人脸特征信息，根据第一人脸特征信息对目标人脸图像进行聚类。In one embodiment, the above-mentioned image processing method includes: when it is detected that the face information in the target face image is changed, re-acquiring first face feature information in the target face image, and performing an analysis on the target face according to the first face feature information. Images are clustered.

移动终端可对人脸图像中识别出人脸标识人脸状态，移动终端接收到服务器下发的图像聚类信息后，可根据服务器下发的图像聚类信息调整移动终端中人脸图像中人脸状态。当移动终端检测到目标人脸图像中人脸状态改变，例如人脸状态由显示变为隐藏、或由隐藏变为显示后，移动终端可重新获取目标人脸图像中第一人脸特征信息，根据重新获取的第一人脸特征信息对目标人脸图像进行聚类。The mobile terminal can identify the face in the face image and identify the face state. After receiving the image clustering information sent by the server, the mobile terminal can adjust the person in the face image in the mobile terminal according to the image clustering information sent by the server. face state. When the mobile terminal detects that the face state in the target face image changes, for example, after the face state changes from display to hide, or from hide to display, the mobile terminal can re-acquire the first face feature information in the target face image, The target face images are clustered according to the re-acquired first face feature information.

本申请实施例中图像处理方法，在检测到人脸图像中人脸信息变更时，重新获取人脸图像中人脸特征信息，根据人脸图像中人脸特征信息对人脸图像重新聚类，能够实现对人脸图像聚类的及时性。In the image processing method in the embodiment of the present application, when it is detected that the face information in the face image is changed, the face feature information in the face image is re-acquired, and the face image is re-clustered according to the face feature information in the face image, It can realize the timeliness of face image clustering.

图4为一个实施例中图像处理装置的结构框图。如图4所示，一种图像处理装置，包括：FIG. 4 is a structural block diagram of an image processing apparatus in an embodiment. As shown in Figure 4, an image processing device, comprising:

第一获取模块402，用于当接收到图像聚类请求，获取图像聚类请求对应的目标人脸图像。The first obtainingmodule 402 is configured to obtain a target face image corresponding to the image clustering request when an image clustering request is received.

第二获取模块404，用于获取目标人脸图像中第一人脸特征信息。The second obtainingmodule 404 is configured to obtain first face feature information in the target face image.

聚类模块406，用于根据第一人脸特征信息对目标人脸图像进行聚类。Theclustering module 406 is configured to cluster the target face images according to the first face feature information.

接收到图像聚类请求包括：当特征识别模型更新，接收第一图像聚类请求；当接收到服务器发送的第一聚类信息，接收第二图像聚类请求，第一聚类信息为服务器对移动终端上传的第二人脸图像集合的聚类信息；当检测到新增图像，接收第三聚类请求。Receiving the image clustering request includes: when the feature recognition model is updated, receiving the first image clustering request; when receiving the first clustering information sent by the server, receiving the second image clustering request, the first clustering information is the server pair. Clustering information of the second face image set uploaded by the mobile terminal; when a newly added image is detected, a third clustering request is received.

在一个实施例中，当接收到第一图像聚类请求，获取目标人脸图像中第一人脸特征信息包括：根据更新后特征识别模型对目标人脸图像进行人脸识别，获取第一人脸特征信息；或，根据特征信息转换模型将目标人脸图像的已存储人脸特征信息转换为第一人脸特征信息。In one embodiment, when a first image clustering request is received, obtaining the first face feature information in the target face image includes: performing face recognition on the target face image according to the updated feature recognition model, and obtaining the first face face feature information; or, converting the stored face feature information of the target face image into the first face feature information according to the feature information conversion model.

图5为另一个实施例中图像处理装置的结构框图。如图5所示，一种图像处理装置，包括第一获取模块502、第二获取模块504、聚类模块506和处理模块508。其中，第一获取模块502、第二获取模块504、聚类模块506与对应的模块功能相同。FIG. 5 is a structural block diagram of an image processing apparatus in another embodiment. As shown in FIG. 5 , an image processing apparatus includes a first acquisition module 502 , a second acquisition module 504 , a clustering module 506 and a processing module 508 . The first acquisition module 502, the second acquisition module 504, and the clustering module 506 have the same functions as the corresponding modules.

第一获取模块502还用于获取移动终端对第二人脸集合的第二聚类信息。The first obtaining module 502 is further configured to obtain second clustering information of the second face set by the mobile terminal.

处理模块508，用于当第一聚类信息与第二聚类信息的对比结果不相同时，根据对比结果的类型对应处理第二聚类信息。The processing module 508 is configured to process the second cluster information correspondingly according to the type of the comparison result when the comparison result between the first cluster information and the second cluster information is not the same.

在一个实施例中，第一获取模块502还用于若当前时刻与上一次获取目标人脸图像的时刻之间的时间差超过预设时长，获取图像聚类请求对应的目标人脸图像；或，若当前时刻为预设时刻，获取图像聚类请求对应的目标人脸图像；或，移动终端处于充电状态，获取图像聚类请求对应的目标人脸图像。In one embodiment, the first acquisition module 502 is further configured to acquire the target face image corresponding to the image clustering request if the time difference between the current moment and the moment when the target face image was acquired last time exceeds a preset duration; or, If the current time is a preset time, obtain the target face image corresponding to the image clustering request; or, when the mobile terminal is in a charging state, obtain the target face image corresponding to the image clustering request.

在一个实施例中，第二获取模块504还用于当检测到目标人脸图像中人脸信息变更，重新获取目标人脸图像中第一人脸特征信息。聚类模块506用于根据第一人脸特征信息对目标人脸图像进行聚类。In one embodiment, the second obtaining module 504 is further configured to re-acquire the first face feature information in the target face image when it is detected that the face information in the target face image is changed. The clustering module 506 is configured to cluster the target face images according to the first face feature information.

本申请实施例还提供了一种计算机可读存储介质。一个或多个包含计算机可执行指令的非易失性计算机可读存储介质，当所述计算机可执行指令被一个或多个处理器执行时，使得所述处理器执行如上所述的图像处理方法。Embodiments of the present application also provide a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the image processing method as described above .

本申请实施例还提供了一种移动终端。如图6所示，为了便于说明，仅示出了与本申请实施例相关的部分，具体技术细节未揭示的，请参照本申请实施例方法部分。该移动终端可以为包括手机、平板电脑、PDA(Personal Digital Assistant，个人数字助理)、POS(Point of Sales，销售终端)、车载电脑、穿戴式设备等任意终端设备，以移动终端为手机为例：The embodiments of the present application also provide a mobile terminal. As shown in FIG. 6 , for the convenience of description, only the parts related to the embodiments of the present application are shown, and the specific technical details are not disclosed, please refer to the method part of the embodiments of the present application. The mobile terminal can be any terminal device including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant, personal digital assistant), a POS (Point of Sales, a sales terminal), a vehicle-mounted computer, a wearable device, etc. Take the mobile terminal as an example of a mobile phone :

图6为与本申请实施例提供的移动终端相关的手机的部分结构的框图。参考图6，手机包括：射频(Radio Frequency，RF)电路610、存储器620、输入单元630、显示单元640、传感器650、音频电路660、无线保真(wireless fidelity，WiFi)模块670、处理器680、以及电源690等部件。本领域技术人员可以理解，图6所示的手机结构并不构成对手机的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。FIG. 6 is a block diagram of a partial structure of a mobile phone related to a mobile terminal provided by an embodiment of the present application. Referring to FIG. 6 , the mobile phone includes: a radio frequency (RF)circuit 610 , amemory 620 , aninput unit 630 , adisplay unit 640 , asensor 650 , anaudio circuit 660 , a wireless fidelity (WiFi)module 670 , and aprocessor 680 , and thepower supply 690 and other components. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 6 does not constitute a limitation on the mobile phone, and may include more or less components than shown, or combine some components, or arrange different components.

其中，RF电路610可用于收发信息或通话过程中，信号的接收和发送，可将基站的下行信息接收后，给处理器680处理；也可以将上行的数据发送给基站。通常，RF电路包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器(Low Noise Amplifier，LNA)、双工器等。此外，RF电路610还可以通过无线通信与网络和其他设备通信。上述无线通信可以使用任一通信标准或协议，包括但不限于全球移动通讯系统(Global System ofMobile communication，GSM)、通用分组无线服务(General Packet Radio Service，GPRS)、码分多址(Code Division Multiple Access，CDMA)、宽带码分多址(Wideband CodeDivision Multiple Access，WCDMA)、长期演进(Long Term Evolution，LTE))、电子邮件、短消息服务(Short Messaging Service，SMS)等。TheRF circuit 610 can be used for receiving and sending signals during transmission and reception of information or during a call. After receiving the downlink information of the base station, it can be processed by theprocessor 680; it can also send uplink data to the base station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition,RF circuitry 610 may communicate with networks and other devices via wireless communications. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access) Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE)), email, Short Messaging Service (Short Messaging Service, SMS) and the like.

存储器620可用于存储软件程序以及模块，处理器680通过运行存储在存储器620的软件程序以及模块，从而执行手机的各种功能应用以及数据处理。存储器620可主要包括程序存储区和数据存储区，其中，程序存储区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能的应用程序、图像播放功能的应用程序等)等；数据存储区可存储根据手机的使用所创建的数据(比如音频数据、通讯录等)等。此外，存储器620可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。Thememory 620 can be used to store software programs and modules, and theprocessor 680 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in thememory 620 . Thememory 620 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function (such as an application program for a sound playback function, an application program for an image playback function, etc.), etc.; The data storage area may store data (such as audio data, address book, etc.) created according to the usage of the mobile phone, and the like. Additionally,memory 620 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

输入单元630可用于接收输入的数字或字符信息，以及产生与手机600的用户设置以及功能控制有关的键信号输入。具体地，输入单元630可包括触控面板631以及其他输入设备632。触控面板631，也可称为触摸屏，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板631上或在触控面板631附近的操作)，并根据预先设定的程式驱动相应的连接装置。在一个实施例中，触控面板631可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给处理器680，并能接收处理器680发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板631。除了触控面板631，输入单元630还可以包括其他输入设备632。具体地，其他输入设备632可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)等中的一种或多种。Theinput unit 630 may be used to receive input numerical or character information, and generate key signal input related to user settings and function control of themobile phone 600 . Specifically, theinput unit 630 may include atouch panel 631 andother input devices 632 . Thetouch panel 631 , also referred to as a touch screen, can collect the user's touch operations on or near it (such as the user using a finger, a stylus, etc., any suitable object or accessory on or near the touch panel 631 ) operation), and drive the corresponding connection device according to the preset program. In one embodiment, thetouch panel 631 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To theprocessor 680, and can receive the commands sent by theprocessor 680 and execute them. In addition, thetouch panel 631 can be realized by various types of resistive, capacitive, infrared, and surface acoustic waves. Besides thetouch panel 631 , theinput unit 630 may further includeother input devices 632 . Specifically,other input devices 632 may include, but are not limited to, one or more of physical keyboards, function keys (such as volume control keys, switch keys, etc.), and the like.

显示单元640可用于显示由用户输入的信息或提供给用户的信息以及手机的各种菜单。显示单元640可包括显示面板641。在一个实施例中，可以采用液晶显示器(LiquidCrystal Display，LCD)、有机发光二极管(Organic Light-Emitting Diode,OLED)等形式来配置显示面板641。在一个实施例中，触控面板631可覆盖显示面板641，当触控面板631检测到在其上或附近的触摸操作后，传送给处理器680以确定触摸事件的类型，随后处理器680根据触摸事件的类型在显示面板641上提供相应的视觉输出。虽然在图6中，触控面板631与显示面板641是作为两个独立的部件来实现手机的输入和输入功能，但是在某些实施例中，可以将触控面板631与显示面板641集成而实现手机的输入和输出功能。Thedisplay unit 640 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. Thedisplay unit 640 may include adisplay panel 641 . In one embodiment, thedisplay panel 641 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. In one embodiment, thetouch panel 631 may cover thedisplay panel 641. When thetouch panel 631 detects a touch operation on or near it, thetouch panel 631 transmits it to theprocessor 680 to determine the type of the touch event, and then theprocessor 680 determines the type of the touch event according to the The type of touch event provides a corresponding visual output ondisplay panel 641 . Although in FIG. 6, thetouch panel 631 and thedisplay panel 641 are used as two independent components to realize the input and input functions of the mobile phone, in some embodiments, thetouch panel 631 and thedisplay panel 641 can be integrated to form a Realize the input and output functions of the mobile phone.

手机600还可包括至少一种传感器650，比如光传感器、运动传感器以及其他传感器。具体地，光传感器可包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节显示面板641的亮度，接近传感器可在手机移动到耳边时，关闭显示面板641和/或背光。运动传感器可包括加速度传感器，通过加速度传感器可检测各个方向上加速度的大小，静止时可检测出重力的大小及方向，可用于识别手机姿态的应用(比如横竖屏切换)、振动识别相关功能(比如计步器、敲击)等；此外，手机还可配置陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器等。Cell phone 600 may also include at least onesensor 650, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of thedisplay panel 641 according to the brightness of the ambient light, and the proximity sensor may turn off thedisplay panel 641 and/or when the mobile phone is moved to the ear. or backlight. Motion sensors can include acceleration sensors, which can detect the magnitude of acceleration in all directions, and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of mobile phones (such as switching between horizontal and vertical screens), and vibration recognition related functions (such as Pedometer, tapping), etc.; in addition, the mobile phone can also be equipped with other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc.

音频电路660、扬声器661和传声器662可提供用户与手机之间的音频接口。音频电路660可将接收到的音频数据转换后的电信号，传输到扬声器661，由扬声器661转换为声音信号输出；另一方面，传声器662将收集的声音信号转换为电信号，由音频电路660接收后转换为音频数据，再将音频数据输出处理器680处理后，经RF电路610可以发送给另一手机，或者将音频数据输出至存储器620以便后续处理。Audio circuit 660,speaker 661 andmicrophone 662 may provide an audio interface between the user and the cell phone. Theaudio circuit 660 can convert the received audio data into an electrical signal, and transmit it to thespeaker 661, and thespeaker 661 converts it into a sound signal for output; on the other hand, themicrophone 662 converts the collected sound signal into an electrical signal, which is converted by theaudio circuit 660 After receiving, the audio data is converted into audio data, and then the audio data is output to theprocessor 680 for processing, and can be sent to another mobile phone via theRF circuit 610, or the audio data can be output to thememory 620 for subsequent processing.

WiFi属于短距离无线传输技术，手机通过WiFi模块670可以帮助用户收发电子邮件、浏览网页和访问流式媒体等，它为用户提供了无线的宽带互联网访问。虽然图6示出了WiFi模块670，但是可以理解的是，其并不属于手机600的必须构成，可以根据需要而省略。WiFi is a short-distance wireless transmission technology. The mobile phone can help users to send and receive emails, browse web pages, and access streaming media through theWiFi module 670. It provides users with wireless broadband Internet access. Although FIG. 6 shows theWiFi module 670, it can be understood that it is not a necessary component of themobile phone 600 and can be omitted as required.

处理器680是手机的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在存储器620内的软件程序和/或模块，以及调用存储在存储器620内的数据，执行手机的各种功能和处理数据，从而对手机进行整体监控。在一个实施例中，处理器680可包括一个或多个处理单元。在一个实施例中，处理器680可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等；调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器680中。Theprocessor 680 is the control center of the mobile phone, using various interfaces and lines to connect various parts of the entire mobile phone, by running or executing the software programs and/or modules stored in thememory 620, and calling the data stored in thememory 620. Various functions of the mobile phone and processing data, so as to monitor the mobile phone as a whole. In one embodiment, theprocessor 680 may include one or more processing units. In one embodiment, theprocessor 680 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface and application programs, etc.; the modem processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into theprocessor 680 .

手机600还包括给各个部件供电的电源690(比如电池)，优选的，电源可以通过电源管理系统与处理器680逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。Themobile phone 600 also includes a power supply 690 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to theprocessor 680 through a power management system, so as to manage charging, discharging, and power consumption management functions through the power management system.

在一个实施例中，手机600还可以包括摄像头、蓝牙模块等。In one embodiment, themobile phone 600 may further include a camera, a Bluetooth module, and the like.

在本申请实施例中，该移动终端所包括的处理器680执行存储在存储器上的计算机程序时实现如上所述的图像处理方法。In this embodiment of the present application, when theprocessor 680 included in the mobile terminal executes the computer program stored in the memory, the image processing method described above is implemented.

本申请所使用的对存储器、存储、数据库或其它介质的任何引用可包括非易失性和/或易失性存储器。合适的非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)，它用作外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDR SDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)。Any reference to a memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.