CN108495038B

Movatterモバイル変換

Info

Publication number: CN108495038B
Application number: CN201810290702.6A
Authority: CN
Inventors: 何新兰
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2018-03-30
Filing date: 2018-03-30
Publication date: 2021-09-07
Anticipated expiration: 2038-03-30
Also published as: CN108495038A

Abstract

Translated fromChinese

本申请实施例公开了一种图像处理方法、装置、存储介质及电子设备，其中，本申请实施例首先获取摄像头的帧率信息，并获取对应该帧率信息的目标个数；然后从摄像头采集的同一场景的场景图像中，提取数量为目标个数的场景图像；再从提取的场景图像中确定出待处理图像，并从待处理图像中确定出待处理脸部图像；再从除待处理图像之外的其它场景图像中，获取对应待处理脸部图像的目标脸部图像；最后将待处理脸部图像替换为目标脸部图像。采用本申请实施例提供的技术方案，无需拍摄者掌握复杂拍摄技术，也无需拍摄者进行复杂的拍摄操作，即可获得符合要求的人像，可以降低人像拍摄的难度。

The embodiments of the present application disclose an image processing method, an apparatus, a storage medium and an electronic device, wherein the embodiments of the present application first acquire frame rate information of a camera, and acquire the number of targets corresponding to the frame rate information; From the scene images of the same scene, extract the scene images whose number is the target number; then determine the to-be-processed images from the extracted scene images, and determine the to-be-processed face images from the to-be-processed images; In other scene images other than the image, a target face image corresponding to the face image to be processed is acquired; finally, the to-be-processed face image is replaced with the target face image. By using the technical solutions provided by the embodiments of the present application, a portrait that meets the requirements can be obtained without the photographer having to master complex shooting techniques and without the need for the photographer to perform complex shooting operations, which can reduce the difficulty of shooting portraits.

Description

Image processing method, image processing device, storage medium and electronic equipment

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image processing method and apparatus, a storage medium, and an electronic device.

Background

Electronic equipment such as a mobile phone generally provides a photographing function for a user, and with continuous progress of hardware such as a camera module and an image processing algorithm, the photographing function of the electronic equipment is more and more powerful, and the user also uses the electronic equipment more and more frequently to photograph.

In the case of portrait shooting, the photographer is generally required to open the eyes and/or make a specified expression (such as smiling, etc.), but since shooting is performed at an instant of image acquisition, the photographer cannot easily grasp the instant at which the photographer makes the specified expression and/or opens the eyes, and further, when multi-person group shooting is performed, each photographer is changed, and it is more difficult to grasp the instant at which each photographer makes the specified expression and/or opens the eyes, resulting in a greater difficulty in portrait shooting.

Disclosure of Invention

The embodiment of the application provides an image processing method and device, a storage medium and an electronic device, which can improve the blurring effect of an image.

In a first aspect, an embodiment of the present application provides an image processing method, including:

acquiring frame rate information of a camera and acquiring the number of targets corresponding to the frame rate information;

extracting the scene images with the number of the targets from the scene images of the same scene collected by the camera;

determining an image to be processed from the extracted scene image, and determining a face image to be processed from the image to be processed, wherein the face feature of the face image to be processed is not matched with a preset face feature;

and acquiring a target face image corresponding to the face image to be processed from other scene images except the image to be processed, wherein the target face image and the face image to be processed belong to the same person, and the face feature of the target face image is matched with the preset face feature.

And replacing the facial image to be processed with the target facial image.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including:

the first acquisition module is used for acquiring the frame rate information of the camera and acquiring the number of targets corresponding to the frame rate information;

the image selection module is used for extracting the scene images with the number of the targets from the scene images of the same scene collected by the camera;

the image determining module is used for determining an image to be processed from the extracted scene image and determining a face image to be processed from the image to be processed, wherein the face feature of the face image to be processed is not matched with a preset face feature;

the second acquisition module is used for acquiring a target face image corresponding to the face image to be processed from other scene images except the image to be processed, wherein the target face image and the face image to be processed belong to the same person, and the face feature of the target face image is matched with the preset face feature;

and the image replacing module is used for replacing the facial image to be processed with the target facial image.

In a third aspect, a storage medium is provided in an embodiment of the present application, and has a computer program stored thereon, where the computer program is enabled to execute an image processing method according to any embodiment of the present application when the computer program runs on a computer.

In a fourth aspect, an embodiment of the present application provides an electronic device, including a central processing unit and a memory, where the memory has a computer program, and the central processing unit is configured to execute the image processing method provided in any embodiment of the present application by calling the computer program;

the method comprises the steps of firstly obtaining frame rate information of a camera and obtaining the number of targets corresponding to the frame rate information; then extracting the scene images with the number of targets from the scene images of the same scene collected by the camera; determining an image to be processed from the extracted scene image, and determining a face image to be processed from the image to be processed, wherein the face feature of the face image to be processed is not matched with a preset face feature; then, acquiring a target face image corresponding to the face image to be processed from other scene images except the image to be processed, wherein the target face image and the face image to be processed belong to the same person, and the face feature of the target face image is matched with the preset face feature; and finally, replacing the face image to be processed with the target face image. By adopting the technical scheme provided by the embodiment of the application, the scene image which is obtained by shooting and comprises the portrait is taken as the image to be processed, the face image which does not meet the requirement can be automatically replaced by the face image which meets the requirement, a photographer does not need to master a complex shooting technology, the photographer does not need to carry out complex shooting operation, the portrait which meets the requirement can be obtained, and the difficulty in portrait shooting is reduced.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application.

Fig. 2 is a schematic diagram of an operation of acquiring a plurality of scene images in the embodiment of the present application.

Fig. 3 is a schematic diagram of an operation of acquiring an image to be processed from a plurality of scene images in the embodiment of the present application.

Fig. 4 is an operation diagram for determining a face image to be processed from an image to be processed in the embodiment of the present application.

Fig. 5 is a schematic diagram illustrating an operation of acquiring a target face image from another scene image in the embodiment of the present application.

Fig. 6 is a schematic diagram of an operation of replacing the face image to be processed with the target face image in the embodiment of the present application.

Fig. 7 is a schematic diagram illustrating an operation of acquiring a target face image from a face image library in an embodiment of the present application.

Fig. 8 is another operation diagram for replacing the face image to be processed with the target face image in the embodiment of the present application.

Fig. 9 is an operation diagram for acquiring a target face image from a target electronic device in the embodiment of the present application.

Fig. 10 is a further flowchart of the image processing method provided in the embodiment of the present application.

Fig. 11 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.

Fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Fig. 13 is another schematic structural diagram of an electronic device according to an embodiment of the present application.

Fig. 14 is a schematic diagram of a detailed structure of an image processing circuit in the embodiment of the present application.

Detailed Description

Referring to the drawings, wherein like reference numbers refer to like elements, the principles of the present application are illustrated as being implemented in a suitable computing environment. The following description is based on illustrated embodiments of the application and should not be taken as limiting the application with respect to other embodiments that are not detailed herein.

In the description that follows, specific embodiments of the present application will be described with reference to steps and symbols executed by one or more computers, unless otherwise indicated. Accordingly, these steps and operations will be referred to, several times, as being performed by a computer, the computer performing operations involving a processing unit of the computer in electronic signals representing data in a structured form. This operation transforms the data or maintains it at locations in the computer's memory system, which may be reconfigured or otherwise altered in a manner well known to those skilled in the art. The data maintains a data structure that is a physical location of the memory that has particular characteristics defined by the data format. However, while the principles of the application have been described in language specific to above, it is not intended to be limited to the specific form set forth herein, and it will be recognized by those of ordinary skill in the art that various of the steps and operations described below may be implemented in hardware.

The term module, as used herein, may be considered a software object executing on the computing system. The various components, modules, engines, and services described herein may be viewed as objects implemented on the computing system. The apparatus and method described herein may be implemented in software, but may also be implemented in hardware, and are within the scope of the present application.

The terms "first", "second", and "third", etc. in this application are used to distinguish between different objects and not to describe a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or modules is not limited to only those steps or modules listed, but rather, some embodiments may include additional steps or modules not listed or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

The embodiment of the present application provides an image processing method, and an execution subject of the image processing method may be the image processing apparatus provided in the embodiment of the present application, or an electronic device integrated with the image processing apparatus, where the image processing apparatus may be implemented in a hardware or software manner. The electronic device may be a smart phone, a tablet computer, a palm computer, a notebook computer, or a desktop computer.

Referring to fig. 1, fig. 1 is a schematic flow chart of an image processing method according to an embodiment of the present disclosure. The specific flow of the image processing method provided by the embodiment of the application can be as follows:

instep 101, frame rate information of a camera is obtained, and the number of targets corresponding to the frame rate information is obtained.

It should be noted that, in order to facilitate shooting by a user, the electronic device generally provides an image preview function, that is, after the shooting application is started, the electronic device acquires an image of a scene to be shot in real time through the camera, and displays the image acquired in real time as a preview image on a preview interface of the shooting application. The image acquired by the camera in real time is cached in a preset image caching queue by the electronic device, the image caching queue may be a fixed-length queue, for example, the image caching queue may cache 8 latest acquired images by the camera.

In the embodiment of the application, when an image shooting request is received, frame rate information of a camera is firstly acquired, wherein the frame rate information is used for describing the number of scene images collected by the camera in unit time, for example, when the acquired frame rate information is 30fps, the camera is explained to collect 30 scene images of a scene to be shot per second; for another example, when the obtained frame rate information is 15fps, it is described that the camera acquires 15 scene images of the scene to be shot per second. When the ambient light brightness is high (or the electronic device is in a bright environment), the camera can complete exposure in a short time (for example, 30ms), so that the camera can acquire a scene image at a high frame rate; when the ambient light level is low (or the electronic device is in a dim environment), the camera takes a long time (e.g., 40ms to 60ms or more) to complete the exposure, so that it can only capture the scene image at a low frame rate.

After the frame rate information of the camera is acquired, the number of targets corresponding to the frame rate information is determined according to the frame rate information of the camera, where the number of targets and the frame rate information may be in a direct relationship, for example, when the frame rate information of the camera is 30fps, the number of targets may be determined to be 8, and when the frame rate information of the camera is 15fps, the number of targets may be determined to be 6.

The image shooting request can be triggered in various ways, such as by a virtual key, by a physical key, by a voice command, and the like.

For example, after a user operates the electronic device to start a photographing application (e.g., a system application "camera" of the electronic device), the user moves the electronic device so that a camera of the electronic device is aligned with a scene to be photographed, and then may trigger an image photographing request by clicking a "photographing" key (which is a virtual key) provided in a "camera" preview interface.

For another example, after the user operates the electronic device to start the photographing application, the user moves the electronic device so that the camera of the electronic device is aligned with the scene to be photographed, and may speak the voice command "photograph" to trigger the image photographing request, or directly click a physical photographing key provided in the electronic device to trigger the image photographing request.

In other embodiments, the specific number of scene images to be acquired may also be determined according to the blinking period, the expression change period, and the like of the user.

Instep 102, extracting scene images with the target number from the scene images of the same scene collected by the camera;

and after the target number corresponding to the camera frame rate information is determined, extracting the scene images of the target number from the image buffer queue. For example, referring to fig. 2, a scene to be photographed includes a person c, a person d, and aperson e 3 different persons, the camera acquires a scene image of the scene to be photographed in real time, and buffers the scene image into an image buffer queue with a length of 8, when frame rate information of the camera is 15fps, it is determined that the target number corresponding to the frame rate information "15 fps" is "4", and then 4 scene images are extracted from the image buffer queue, where the target numbers are: scene image a, scene image B, scene image C, and scene image D.

Instep 103, determining an image to be processed from the extracted scene image, and determining a face image to be processed from the image to be processed, wherein the face feature of the face image to be processed does not match with the preset face feature;

in the embodiment of the application, after a plurality of scene images with the target number are acquired, one scene image is determined from the acquired scene images to be used as an image to be processed. When the image to be processed is determined, one scene image can be selected from the acquired multiple scene images as the image to be processed, or one scene image can be selected from the acquired multiple scene images as the image to be processed according to a certain rule.

Specifically, in an embodiment, in order to improve the processing efficiency of the image to be processed, determining the image to be processed from the acquired multiple scene images includes:

determining the number of the matched face images contained in each extracted scene image, wherein the face features of the matched face images are matched with the preset face features;

and determining the scene image with the largest number of matched face images as the image to be processed.

After a plurality of scene images corresponding to the same scene (i.e., a scene to be photographed) are acquired, face recognition is performed on each acquired scene image, and the number of matching face images included in each scene image is determined, where a matching face image refers to a face feature of the face image matching a preset face feature, and the face feature includes but is not limited to an eye state (such as open eyes and closed eyes) and an expression type (such as smile, laugh, depression, surprise, etc.).

The preset facial features can be set according to actual shooting requirements, for example, when multi-person group photo is carried out, if expressions which people all have surprise need to be shot, the preset facial features can be configured to be 'surprise'; for another example, when a plurality of persons are group-photographed, if an expression that a person opens his eyes and smiles is to be photographed, the preset facial feature may be configured to be "smile and open his eyes".

After the number of the matching face images included in each scene image set is determined, the scene image including the largest number of the matching face images can be used as the image to be processed.

For example, referring to fig. 2 and fig. 3 in combination, the scene to be photographed includes a third person, a fourth person and a fifth person, which are 3 different persons, and the preset facial feature is configured as "laugh". The camera collects scene images of the scene to be shot in real time and caches the scene images into an image cache queue with the length of 8.

When receiving a triggered image shooting request, extracting 4 recently stored scene images from an image cache queue, wherein the scene images are respectively as follows: scene image a, scene image B, scene image C, and scene image D; then, face recognition is performed on the extracted 4 scene images respectively, the number of matching face images contained in each of the 4 scene images is determined, it is determined that the scene image a includes one matching face image (i.e., the third face image), it is determined that the scene image B does not have a matching face image, it is determined that the scene image C includes two matching face images (i.e., the third face image and the third face image), and it is determined that the scene image D includes one matching face image (i.e., the fifth face image), obviously, the number of matching face images contained in the scene image C is the largest, and therefore, the scene image C is taken as an image to be processed.

In the embodiment of the application, after the image to be processed is determined from the acquired scene images, the face image to be processed is further determined from the image to be processed, wherein the face feature of the face image to be processed does not match with the preset face feature.

For example, referring to fig. 4, the determined image to be processed includes a face image of a third person, and a face image of a fifth person, as shown in fig. 4, the face feature of the third face image is "laughing", the face feature of the third face image is also "laughing", the face feature of the fifth face image is "frustrated", if the preset face feature is "laughing", the face features of the third and fourth face images match the preset face feature, and the face feature of the fifth face image does not match the preset face feature, so the fifth face image is determined as the face image to be processed.

Instep 104, a target face image corresponding to the face image to be processed is obtained from other scene images except the image to be processed, the target face image and the face image to be processed belong to the same person, and the face feature of the target face image is matched with the preset face feature.

In the embodiment of the application, a to-be-processed face image in the to-be-processed image is determined, and a target face image corresponding to the to-be-processed face image is further acquired from other scene images except the to-be-processed image, wherein the target face image and the to-be-processed face image belong to the same person, and the face feature of the target face image is matched with the preset face feature.

For example, referring to fig. 5 and 2, of the four scene images shown in fig. 2, the scene image C is determined as the image to be processed, wherein the facial feature of the facial image is "depressed", does not match the preset facial feature "laugh", and is further determined as the facial image to be processed. Then, in other scene images (here, the scene image a, the scene image B, and the scene image D) than the image to be processed (here, the scene image C), it is found whether there is a pentahedral image (i.e., a face image of a character pentagon) whose face feature is "smile", and it is apparent that, as shown in fig. 5, there is a pentahedral image whose face feature is "smile" in the scene image D, at this time, the pentahedral image in the scene image D is determined as a target face image, and the pentahedral image is extracted from the scene image D and used for subsequent processing.

Instep 105, the face image to be processed is replaced with the target face image.

In the embodiment of the application, after the target face image corresponding to the face image to be processed is obtained, the face image to be processed can be replaced by the target face image.

For example, referring to fig. 6 and fig. 2 in combination, after the four scene images shown in fig. 2 (i.e., scene image a, scene image B, scene image C, and scene image D) are acquired, the scene image C is determined as an image to be processed, and further, a pentagon face image (i.e., a face image of a character pentagon) in the scene image C is determined as a face image to be processed; then, determining the pentahedral image in the scene image D as a target face image; then, the pentagon face image in the scene image C is replaced by the pentagon face image in the scene image D, so as to obtain the scene image C after the replacement processing, as shown in fig. 6, in the replaced scene image C, the face features of the face images of the persons are all "laughs", and are matched with the preset face feature "laughs".

In the embodiment of the application, after the image replacement is completed, the replaced image to be processed can be used as a result image of the image shooting request.

Optionally, in an embodiment, after replacing the face image to be processed with the target face image, the method further includes:

and performing noise reduction processing on the image to be processed according to other scene images except the image to be processed.

The noise reduction processing can be performed on the image to be processed in a multi-frame noise reduction mode. For example, 4 scene images, namely a scene image a, a scene image B, a scene image C, and a scene image D, are obtained in total, where the scene image D is determined as an image to be processed, and then multi-frame denoising may be performed on the scene image D according to the scene image a, the scene image B, and the scene image C.

Specifically, when performing multi-frame denoising, the scene image a, the scene image D, the scene image B, and the scene image C may be aligned first, and the pixel value of each group of aligned pixels may be obtained. If the pixel values of the same group of alignment pixels are not different, the pixel value average value of the group of alignment pixels can be calculated, and the pixel value average value is used for replacing the pixel value of the corresponding pixel of the scene image D. If the pixel values of the alignment pixels in the same group have a large difference, the pixel values in the scene image D may not be adjusted.

For example, the pixel P1 in the scene image a, the pixel P2 in the scene image D, the pixel P3 in the scene image B, and the pixel P4 in the scene image C are a group of mutually aligned pixels, where the pixel value of P1 is 101, the pixel value of P2 is 102, the pixel value of P3 is 103, and the pixel value of P4 is 104, and then the average value of the pixel values of the group of mutually aligned pixels is 102.5, then the terminal may adjust the pixel value of the P2 pixel in the scene image D from 102 to 102.5, thereby performing noise reduction on the P2 pixel in the scene image D. If the pixel value of P1 is 80, the pixel value of P2 is 102, the pixel value of P3 is 83, and the pixel value of P4 is 90, then the pixel value of P2 may not be adjusted at this time, i.e., the pixel value of P2 remains 102, because their pixel values are more different.

Optionally, in an embodiment, after acquiring a target face image corresponding to the to-be-processed face image from another scene image except the to-be-processed image, the method further includes:

when the target face image is failed to be acquired from other scene images, judging whether a local preset face image library stores the target face image or not;

when the target face image is stored in the face image library, the target face image stored in the face image library is extracted.

An alternative approach to acquiring a target facial image is provided herein, taking into account the failure to acquire the target facial image from other scene images.

Specifically, a face image library may be created locally in the electronic device in advance, where the face image library is used to store the face image acquired by the electronic device. For example, after the electronic device captures an external scene each time and obtains a scene image corresponding to the external scene, the electronic device performs face recognition on the scene image, and stores a face image recognized from the scene image into a face image library.

Correspondingly, when the target face image is failed to be acquired from other scene images, whether the target face image corresponding to the face image to be processed is stored or not can be searched in a local preset face image library. The target face image and the face image to be processed belong to the same person, and the face features of the target face image are matched with the preset face features.

And if the target face image corresponding to the face image to be processed is found in the face image library, extracting the target face image corresponding to the face image to be processed from the face image library. For example, referring to fig. 7 and 4 in combination, for the to-be-processed face image (i.e., pentagon face image) determined from the to-be-processed image shown in fig. 4, a target face image corresponding to the to-be-processed face image is extracted from the face image library (the target face image is identified from a certain image previously taken in a history and stored in the face image library), as shown in fig. 7, the extracted target face image and the to-be-processed face image belong to the same person, i.e., belong to the same person pentagon, and the face feature of the target face image is "smile", which conforms to the configured preset face feature.

Specifically, the extracting the target face image stored in the face image library includes:

when a plurality of target face images are stored in a face image library, acquiring the storage time of each target face image;

and extracting a target face image which is stored in the face image library and has the closest time to the current time.

For example, 3 target face images corresponding to images to be processed are found in the face image library, which are respectively thetarget face image 1, thetarget face image 2 and thetarget face image 3, the storage time of thetarget face image 1 is t1, the storage time of thetarget face image 2 is t2, the storage time of thetarget face image 3 is t3, if the storage time t1 is 1 day from the current time, the storage time t2 is 2 days from the current time, and the storage time t3 is 7 days from the current time, obviously, the storage time of thetarget face image 1 is the closest to the current time, and at this time, thetarget image 1 is extracted from the face image library for subsequent processing.

Specifically, because the target facial image is extracted from the facial image library at this time, the target facial image may be acquired from different shooting scenes, and the shooting scenes are different, and the illumination thereof is also different, therefore, in order to improve the image effect after image processing, the target facial image is replaced by the to-be-processed facial image, including:

acquiring illumination information of a face image to be processed;

migrating the acquired illumination information to a target face image;

and replacing the face image to be processed with the migrated target face image.

When the illumination information is migrated, a proper illumination migration algorithm can be selected according to actual needs to migrate the illumination information of the image to be processed to the target face image. Optional illumination migration algorithms include, but are not limited to: an illumination migration algorithm based on an entropy diagram, an illumination migration algorithm based on a 3D model, an illumination migration algorithm based on a filter decomposition, an illumination migration algorithm based on an intrinsic decomposition and the like.

And after the migration of the illumination information is finished, replacing the determined face image to be processed with the migrated target face image, so that the face features of all the face images in the replaced image to be processed are matched with the preset face features, and the illumination information is consistent.

For example, referring to fig. 8, after the illumination information of a certain determined face image to be processed is transferred to the corresponding target face image, the target face image and the face image to be processed obtain the same illumination effect, and then the determined face image to be processed is replaced with the transferred target face image, so that the facial features of all face images in the replaced image to be processed are matched with the preset facial features and the illumination information is consistent.

It should be noted that, as to what kind of illumination migration algorithm is adopted, the embodiment of the present application is not specifically limited, and a person skilled in the art may select the illumination migration algorithm according to actual needs, and the illumination migration algorithm may be the illumination migration algorithm listed in the embodiment of the present application, or may be the illumination migration algorithm not listed in the embodiment of the present application.

Optionally, in an embodiment, after determining whether the target face image is stored in the local preset face image library, the method further includes:

when a target face image corresponding to the face image to be processed is not stored in the face image library, acquiring character information corresponding to the face image to be processed;

sending an image acquisition request to target electronic equipment corresponding to the acquired character information, wherein the image acquisition request is used for indicating the target electronic equipment to search and return a target face image corresponding to the face image to be processed;

and receiving a target face image returned by the target electronic equipment.

In consideration of the situation that the target face image is not obtained from the local face image library, that is, when the target face image corresponding to the face image to be processed is not stored in the local face image library, another scheme for obtaining the target face image is provided herein.

Specifically, when the search of the face image library is completed and the target face image corresponding to the face image to be processed is not found in the face image library, that is, when the target face image corresponding to the face image to be processed is not stored in the face image library, the character information corresponding to the face image to be processed is obtained, that is, the face image of "who" the face image to be processed is determined.

After the figure information corresponding to the face image to be processed is obtained, the target electronic device corresponding to the obtained figure information is determined according to the incidence relation between the locally pre-stored figure information and the electronic device (the incidence relation is used for describing a user to which the electronic device belongs). For example, referring to fig. 4, regarding the to-be-processed image shown in fig. 4, it is determined that the to-be-processed image is a pentagon face image, at this time, the person information to which the pentagon face image is obtained is "pentagon", and it is further determined that a mobile phone of the pentagon is the target electronic device.

After the target electronic equipment is determined, an image acquisition request is generated according to a pre-agreed message format, and the generated image acquisition request is sent to the determined target electronic equipment. The image acquisition request is used for indicating the target electronic equipment to search whether a target face image corresponding to the face image to be processed exists locally or not, and if the target face image is found, the found target face image is returned. Specifically, the target electronic device also creates a face image library locally in advance, when receiving an image acquisition request, searches in the local face image library according to an instruction of the image acquisition request, and returns a target face image which corresponds to the face image to be processed if the target face image is found.

Correspondingly, after the image acquisition request is sent to the target electronic equipment, the target face image returned by the target electronic equipment is received.

For example, referring to fig. 9, on one hand, after acquiring an image to be processed (a group image of three people, namely propane, butane and pentane), the electronic device determines that a penta facial image in the image to be processed is the facial image to be processed, searches a local facial image library, does not find a target facial image corresponding to the facial image to be processed, determines that a penta mobile phone is the target electronic device at this time, and sends an image acquisition request to the target electronic device; on the other hand, after receiving the image acquisition request from the electronic device, the target electronic device searches in a local face image library, finds the target face image corresponding to the face image to be processed, and returns the found target face image to the electronic device.

As can be seen from the above, in the embodiment of the present application, first, frame rate information of a camera is obtained, and the number of targets corresponding to the frame rate information is obtained; then extracting the scene images with the number of targets from the scene images of the same scene collected by the camera; determining an image to be processed from the extracted scene image, and determining a face image to be processed from the image to be processed, wherein the face feature of the face image to be processed is not matched with a preset face feature; then, acquiring a target face image corresponding to the face image to be processed from other scene images except the image to be processed, wherein the target face image and the face image to be processed belong to the same person, and the face feature of the target face image is matched with the preset face feature; and finally, replacing the face image to be processed with the target face image. By adopting the technical scheme provided by the embodiment of the application, the scene image which is obtained by shooting and comprises the portrait is taken as the image to be processed, the face image which does not meet the requirement can be automatically replaced by the face image which meets the requirement, a photographer does not need to master a complex shooting technology, the photographer does not need to carry out complex shooting operation, the portrait which meets the requirement can be obtained, and the difficulty in portrait shooting is reduced.

The image processing method of the present application will be further described below on the basis of the methods described in the above embodiments. Referring to fig. 10, the image processing method may include:

instep 201, frame rate information of the cameras is acquired, and the number of targets corresponding to the frame rate information is acquired.

In the embodiment of the application, when an image shooting request is received, frame rate information of a camera is firstly acquired, wherein the frame rate information is used for describing the number of scene images collected by the camera in unit time, for example, when the acquired frame rate information is 30fps, the camera is explained to collect 30 scene images of a scene to be shot per second; for another example, when the obtained frame rate information is 15fps, it is described that the camera acquires 15 scene images of the scene to be shot per second.

When the ambient light brightness is high (or the electronic device is in a bright environment), the camera can complete exposure in a short time (for example, 30ms), so that the camera can acquire a scene image at a high frame rate; when the ambient light level is low (or the electronic device is in a dim environment), the camera takes a long time (e.g., 40ms to 60ms or more) to complete the exposure, so that it can only capture the scene image at a low frame rate.

Then, the number of targets corresponding to the frame rate information is determined according to the frame rate information of the camera, wherein the number of targets and the frame rate information may be in a proportional relationship, for example, when the frame rate information of the camera is 30fps, the number of targets may be determined to be 8, and when the frame rate information of the camera is 15fps, the number of targets may be determined to be 6.

Instep 202, the target number of scene images are extracted from the scene images of the same scene captured by the camera.

After the target number corresponding to the camera frame rate information is determined, the scene images of the target number can be extracted from the image cache queue. For example, when the frame rate information of the camera is 30fps, if the number of targets corresponding to the frame rate information "30 fps" is determined to be "8", 8 scene images are extracted from the image buffer queue.

Instep 203, the number of matching face images included in each extracted scene image is determined, wherein the face features of the matching face images match with the preset face features.

Instep 204, the scene image including the largest number of matching face images is determined as the image to be processed.

Instep 205, a face image to be processed in the image to be processed is determined, and the face feature of the face image to be processed does not match the preset face feature.

Instep 206, a target face image corresponding to the to-be-processed face image is obtained from other scene images except the to-be-processed image, the target face image and the to-be-processed face image belong to the same person, and the face feature of the target face image is matched with the preset face feature.

Instep 207, the face image to be processed is replaced with the target face image.

In an embodiment, animage processing apparatus 400 is further provided, please refer to fig. 11, and fig. 11 is a schematic structural diagram of theimage processing apparatus 400 according to an embodiment of the present disclosure. Theimage processing apparatus 400 is applied to an electronic device, and theimage processing apparatus 400 includes a first obtainingmodule 401, animage selecting module 402, animage determining module 403, a second obtainingmodule 404, and animage replacing module 405, as follows:

a first obtainingmodule 401, configured to obtain frame rate information of a camera and obtain a number of targets corresponding to the frame rate information;

animage selecting module 402, configured to extract scene images with the number of targets from scene images of the same scene acquired by the camera;

animage determining module 403, configured to determine an image to be processed from the extracted scene image, and determine a face image to be processed from the image to be processed, where a face feature of the face image to be processed does not match a preset face feature;

a second obtainingmodule 404, configured to obtain a target face image corresponding to the to-be-processed face image from other scene images except the to-be-processed image, where the target face image and the to-be-processed face image belong to the same person, and a face feature of the target face image matches a preset face feature;

and animage replacing module 405, configured to replace the face image to be processed with the target face image.

In an embodiment, theimage determining module 403 is specifically configured to:

In an embodiment, theimage processing apparatus 400 further comprises a noise reduction processing module configured to:

and performing noise reduction processing on the replaced image to be processed according to other scene images except the image to be processed.

In an embodiment, the second obtainingmodule 404 is further configured to:

when the target face image is failed to be acquired from other scene images except the image to be processed, judging whether a local preset face image library stores the target face image or not;

In an embodiment, theimage replacement module 405 is specifically configured to:

acquiring illumination information of a face image to be processed;

migrating the acquired illumination information to a target face image;

In an embodiment, the second obtainingmodule 404 is further configured to:

when the target face image is not stored in the face image library, acquiring character information corresponding to the face image to be processed;

sending an image acquisition request to target electronic equipment corresponding to the character information, wherein the image acquisition request is used for indicating the target electronic equipment to search and return a target face image;

and receiving a target face image returned by the target electronic equipment.

In specific implementation, the modules may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and specific implementation of the units may refer to the foregoing embodiments, which are not described herein again.

As can be seen from the above, in the image processing apparatus of this embodiment, the first obtainingmodule 401 can obtain the frame rate information of the camera, and obtain the number of targets corresponding to the frame rate information; theimage selection module 402 extracts scene images with the number of targets from scene images of the same scene collected by the camera; determining, by theimage determining module 403, an image to be processed from the extracted scene image, and determining a face image to be processed from the image to be processed, where a face feature of the face image to be processed does not match a preset face feature; acquiring a target face image corresponding to the to-be-processed face image from other scene images except the to-be-processed image by a second acquiringmodule 404, wherein the target face image and the to-be-processed face image belong to the same person, and the face feature of the target face image is matched with the preset face feature; the face image to be processed is replaced with the target face image by theimage replacement module 405. By adopting the technical scheme provided by the embodiment of the application, the portrait obtained by shooting is taken as the image to be processed, the facial image which does not meet the requirement can be automatically replaced by the facial image which meets the requirement, a photographer does not need to master a complex shooting technology, the photographer does not need to carry out complex shooting operation, the portrait which meets the requirement can be obtained, and the difficulty in portrait shooting is reduced.

The embodiment of the application also provides the electronic equipment. Referring to fig. 12, anelectronic device 500 includes acentral processing unit 501 and amemory 502. Thecentral processing unit 501 is electrically connected to thememory 502.

Thecpu 500 is a control center of theelectronic device 500, connects various parts of the whole electronic device through various interfaces and lines, and executes various functions of theelectronic device 500 and processes data by running or loading a computer program stored in thememory 502 and calling data stored in thememory 502, thereby implementing accurate identification of the gender of the user.

Thememory 502 may be used to store software programs and modules, and thecentral processing unit 501 executes various functional applications and data processing by operating the computer programs and modules stored in thememory 502. Thememory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, thememory 502 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, thememory 502 may also include a memory controller to provide thecentral processor 501 access to thememory 502.

In the embodiment of the present application, thecentral processing unit 501 in theelectronic device 500 executes the image processing method in any of the above embodiments by running the computer program stored in thememory 502, such as: firstly, acquiring frame rate information of a camera, and acquiring the number of targets corresponding to the frame rate information; then extracting the scene images with the number of targets from the scene images of the same scene collected by the camera; determining an image to be processed from the extracted scene image, and determining a face image to be processed from the image to be processed; then, acquiring a target face image corresponding to the face image to be processed from other scene images except the image to be processed; and finally, replacing the face image to be processed with the target face image.

Referring to fig. 13, in some embodiments, theelectronic device 500 may further include: adisplay 503,radio frequency circuitry 504,audio circuitry 505,power supply 506,image processing circuitry 507, and agraphics processor 508. Thedisplay 503, therf circuit 504, theaudio circuit 505, and thepower source 506 are electrically connected to thecentral processing unit 501.

Thedisplay 503 may be used to display information entered by or provided to the user as well as various graphical user interfaces, which may be made up of graphics, text, icons, video, and any combination thereof. TheDisplay 503 may include a Display panel, and in some embodiments, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

Therf circuit 504 may be used for transceiving rf signals to establish wireless communication with a network device or other electronic devices through wireless communication, and for transceiving signals with the network device or other electronic devices.

Theaudio circuit 505 may be used to provide an audio interface between the user and the electronic device through a speaker, microphone.

Thepower supply 506 may be used to power various components of theelectronic device 500. In some embodiments, thepower source 506 may be logically connected to thecentral processor 501 through a power management system, so as to manage charging, discharging, and power consumption management functions through the power management system.

TheImage Processing circuit 507 may be implemented by hardware and/or software components, and may include various Processing units defining an ISP (Image Signal Processing) pipeline, referring to fig. 14, in an embodiment, theImage Processing circuit 507 includes anISP processor 5071 and acontrol logic 5072. The image data captured bycamera 5073 is first processed byISP processor 5071, andISP processor 5071 analyzes the image data to capture image statistics that may be used to determine and/or one or more control parameters ofcamera 5073. Thecamera 5073 may include a camera with one or more lenses 50731 and an image sensor 50732. Image sensor 50732 may include an array of color filters (e.g., Bayer filters), and image sensor 50732 may acquire light intensity and wavelength information captured with each imaging pixel of image sensor 50732 and provide a set of raw image data that may be processed byISP processor 5071. A sensor 5074 (e.g., a gyroscope) may provide parameters of the acquired image processing (e.g., anti-shake parameters) to theISP processor 5071 based on the type ofsensor 5074 interface. Thesensor 5074 interface may utilize a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above.

Further, image sensor 50732 may also send raw image data tosensor 5074,sensor 5074 may provide raw image data toISP processor 5071 based onsensor 5074 interface type, orsensor 5074 may store raw image data inimage memory 5075.

TheISP processor 5071 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and theISP processor 5071 may perform one or more image processing operations on the raw image data, collecting statistics about the image data. Wherein the image processing operations may be performed with the same or different bit depth precision.

ISP processor 5071 may also receive image data fromimage memory 5075. For example, thesensor 5074 interface sends raw image data to theimage memory 5075, and the raw image data in theimage memory 5075 is then provided to theISP processor 5071 for processing. Theimage Memory 5075 may be part of a Memory device, a storage device, or a separate dedicated Memory within an electronic device, and may include a DMA (Direct Memory Access) feature.

Upon receiving raw image data from image sensor 50732 interface or fromsensor 5074 interface or fromimage memory 5075,ISP processor 5071 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to imagememory 5075 for additional processing before being displayed.ISP processor 5071 receives processed data fromimage memory 5075 and performs image data processing on the processed data in the raw domain and in the RGB and YCbCr color spaces. The image data processed byISP processor 5071 may be output to display 503 for viewing by a user and/or further processed by a graphics engine orimage processor 507. Further, the output ofISP processor 5071 may also be sent to imagememory 5075 and display 503 may read image data fromimage memory 5075. In one embodiment,image memory 5075 may be configured to implement one or more frame buffers. Further, the output of theISP processor 5071 may be sent to an encoder/decoder 5076 for encoding/decoding image data. The encoded image data may be saved and decompressed before being displayed on thedisplay 503 device. The encoder/decoder 5076 may be implemented by a CPU or GPU or coprocessor.

The statistics determined byISP processor 5071 may be sent to controllogic 5072 unit. For example, the statistical data may include image sensor 50732 statistics such as auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, lens 50731 shading correction, and the like. Thecontrol logic 5072 may include a processor and/or microcontroller that executes one or more routines (e.g., firmware) that determine control parameters of thecamera 5073 andISP processor 5071 based on the received statistical data. For example, the control parameters of thecamera 5073 may includesensor 5074 control parameters (such as gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, lens 50731 control parameters (such as focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), and lens 50731 shading correction parameters, among others.

Theimage processor 508 performs conversion driving of display data that the electronic device needs to display, and supplies a line scanning signal to thedisplay 503 to control thedisplay 503 to display correctly.

An embodiment of the present application further provides a storage medium, where the storage medium stores a computer program, and when the computer program runs on a computer, the computer program causes the computer to execute the image processing method in any one of the embodiments, such as: firstly, acquiring frame rate information of a camera, and acquiring the number of targets corresponding to the frame rate information; then extracting the scene images with the number of targets from the scene images of the same scene collected by the camera; determining an image to be processed from the extracted scene image, and determining a face image to be processed from the image to be processed; then, acquiring a target face image corresponding to the face image to be processed from other scene images except the image to be processed; and finally, replacing the face image to be processed with the target face image.

In the embodiment of the present application, the storage medium may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

It should be noted that, for the image processing method of the embodiment of the present application, it can be understood by a person skilled in the art that all or part of the process of implementing the image processing method of the embodiment of the present application can be completed by controlling the relevant hardware through a computer program, the computer program can be stored in a computer readable storage medium, such as a memory of the electronic device, and executed by at least one central processing unit in the electronic device, and the process of executing the computer program can include, for example, the process of the embodiment of the image processing method. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, etc.

In the image processing apparatus according to the embodiment of the present application, each functional module may be integrated into one processing chip, each module may exist alone physically, or two or more modules may be integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium, such as a read-only memory, a magnetic or optical disk, or the like.

The foregoing detailed description has provided an image processing method, an image processing apparatus, a storage medium, and an electronic device according to embodiments of the present application, and specific examples are applied herein to explain the principles and implementations of the present application, and the descriptions of the foregoing embodiments are only used to help understand the method and the core ideas of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. An image processing method, comprising:

acquiring frame rate information of a camera, and acquiring the number of targets corresponding to the frame rate information and the expression change period;

acquiring a target face image corresponding to the face image to be processed from other scene images except the image to be processed, wherein the target face image and the face image to be processed belong to the same person, and the face feature of the target face image is matched with the preset face feature;

when the target face image is failed to be acquired from the other scene images, judging whether a local preset face image library stores the target face image or not;

when the target face image is not stored in the face image library, acquiring figure information corresponding to the face image to be processed;

sending an image acquisition request to target electronic equipment corresponding to the character information, wherein the target electronic equipment is determined according to the incidence relation between locally pre-stored character information and the electronic equipment, the image acquisition request is generated according to a pre-agreed message format, and the image acquisition request is used for indicating the target electronic equipment to search and return the target face image;

receiving the target face image returned by the target electronic equipment;

replacing the facial image to be processed with the target facial image;

aligning the scene images with the target number, acquiring the pixel value of each group of aligned pixels, calculating the pixel value mean value of the group of aligned pixels if the pixel values of the same group of aligned pixels are not different, and replacing the pixel value of the corresponding pixel of the image to be processed with the pixel value mean value; and if the pixel values of the alignment pixels in the same group have more differences, not adjusting the pixel values in the image to be processed.

2. The image processing method of claim 1, wherein determining the image to be processed from the extracted scene image comprises:

3. The image processing method according to claim 1 or 2, wherein, after determining whether the target face image is stored in a locally preset face image library when the target face image is failed to be acquired from the other scene image, further comprising:

when the target face image is stored in the face image library, extracting the target face image stored in the face image library.

4. The image processing method according to claim 3, wherein replacing the face image to be processed with the target face image comprises:

acquiring illumination information of the face image to be processed;

migrating the acquired illumination information to the target face image;

5. An image processing apparatus characterized by comprising:

the first acquisition module is used for acquiring frame rate information of a camera and acquiring the number of targets corresponding to the frame rate information and the expression change period;

the second acquisition module is further used for judging whether a target face image is stored in a local preset face image library or not when the target face image is failed to be acquired from other scene images except the image to be processed;

the second acquisition module is further used for acquiring the figure information corresponding to the face image to be processed when the target face image is not stored in the face image library; sending an image acquisition request to target electronic equipment corresponding to the character information, wherein the target electronic equipment is determined according to the incidence relation between the locally pre-stored character information and the electronic equipment, the image acquisition request is generated according to a pre-agreed message format, and the image acquisition request is used for indicating the target electronic equipment to search and return a target face image; receiving a target face image returned by the target electronic equipment;

the image replacing module is used for replacing the facial image to be processed with the target facial image;

6. The image processing apparatus of claim 5, wherein the image determination module is specifically configured to:

7. A storage medium having stored thereon a computer program, characterized in that, when the computer program runs on a computer, it causes the computer to execute the image processing method according to any one of claims 1 to 4.

8. An electronic device comprising a central processing unit and a memory, said memory storing a computer program, wherein said central processing unit is adapted to execute the image processing method according to any one of claims 1 to 4 by calling said computer program.