CN112967273A - Image processing method, electronic device, and storage medium - Google Patents
Image processing method, electronic device, and storage mediumDownload PDFInfo
- Publication number
- CN112967273A CN112967273ACN202110322423.5ACN202110322423ACN112967273ACN 112967273 ACN112967273 ACN 112967273ACN 202110322423 ACN202110322423 ACN 202110322423ACN 112967273 ACN112967273 ACN 112967273A
- Authority
- CN
- China
- Prior art keywords
- image
- strategy
- processing
- image quality
- quality index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
Translated fromChinese
Description
Technical Field
Embodiments of the present invention relate to the field of image processing, and in particular, to an image processing method, an electronic device, and a storage medium.
Background
At present, due to the inherent characteristic influence of an image sensor, the image shot by the image sensor generally has the problems of poor contrast ratio of a target and a background, fuzzy edge, low image signal-to-noise ratio, low contrast ratio and the like. These problems need to be solved by image processing techniques. The image processing technology is mainly divided into image enhancement, image restoration, image super-resolution reconstruction and other technologies. In particular, the image enhancement technology is most widely used, and mainly includes the following methods: contrast transformation, spatial filtering (smoothing, sharpening, etc.), frequency domain filtering, tone mapping, etc. The methods improve the quality of the image and also improve the subjective feeling of the user for watching the image.
However, although there are many methods for processing images for a single scene, there is no mature method for processing images for multiple scenes.
Disclosure of Invention
An object of embodiments of the present invention is to provide an image processing method, an electronic device, and a storage medium, which enable the image processing method to perform optimization processing on images captured in multiple scenes.
In order to solve the above technical problem, an embodiment of the present invention provides an image processing method, including: acquiring the scores of all image quality indexes of the image; if the scores of the image quality indexes of the image do not meet the preset score requirement, selecting a processing strategy from a predefined strategy space based on the scores of the image quality indexes of the image, and processing the image to obtain a final image; the strategy space stores a plurality of image processing strategies in advance.
An embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the image processing method as mentioned in the above embodiments.
Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which when executed by a processor implements the image processing method mentioned in the above embodiments.
According to the image processing method, the electronic device and the storage medium provided by the embodiment of the invention, as the strategy space is pre-stored with the processing strategies of a plurality of images, the electronic device can select the processing strategy suitable for the image from the predefined strategy space based on the scores of the image quality indexes of the image, and process the image, so that the images shot in different scenes can be optimized in a proper mode.
In addition, the acquiring of the scores of the image quality indexes of the images comprises the following steps: determining an overexposed area of an image; and calculating the scores of all image quality indexes of the image according to other areas except the over-exposure area in the image. In the embodiment, the scores of all the image quality indexes of the calculated image of the over-exposure area are removed, so that the calculated scores of all the image quality indexes of the image have reference value, and the processing efficiency of the image is further improved.
In addition, if the score of each image quality index of the image meets the preset score requirement; the image is not processed. In the embodiment, the images meeting the preset requirements are not processed, so that the waste of computing resources caused by processing high-quality images can be avoided.
In addition, based on the scores of all image quality indexes of the image, a processing strategy is selected from a predefined strategy space, and the image is processed to obtain a final image, wherein the processing strategy comprises the following steps: determining at least one policy combination based on the scores of the image quality indicators of the images; the strategy combination at least comprises at least one processing strategy in the strategy space; processing the images respectively by using each processing strategy in the strategy combination; calculating the sum of the scores of all image quality indexes of the processed image; and screening the processed image with the highest sum as a final image. In this embodiment, the images are processed by using a plurality of policy combinations, and the processed image with the optimal processing result is selected as the final image, so that the quality of the final image is higher.
In addition, determining at least one policy combination based on the scores of the image quality indicators of the images comprises: judging whether the score of the image quality index of the image is larger than the preset score of the image quality index or not aiming at each image quality index; if not, determining the image quality index as the image quality index to be optimized; respectively selecting at least one processing strategy from the processing strategies corresponding to each image quality index to be optimized in the strategy space, and combining to obtain a strategy combination; or selecting N candidate strategy combinations from M pre-stored candidate strategy combinations as to-be-executed strategy combinations according to the to-be-optimized image quality index; m is not less than N, and M and N are positive integers. In the embodiment, the strategy combination of the optimized image is obtained based on the score of the image quality index, so that the processing strategy for processing the image is more targeted, and the quality of the image is more effectively improved.
In addition, the processing of the image by each processing policy in the policy combination includes: determining the value of a preset parameter of a processing strategy corresponding to the image quality index to be optimized in the strategy combination according to the score of the image quality index to be optimized of the image; and executing each processing strategy in the strategy combination on the image in sequence. In the embodiment, the value of the preset parameter of the processing strategy corresponding to the image quality index to be optimized in the strategy combination is determined based on the score of the image quality index to be optimized of the image, instead of randomly selecting a numerical value to determine the processing strength of the processing strategy, and the optimal value is determined through multiple attempts, so that the optimization speed of the image can be improved.
In addition, the preset score requirements include: the sum of the scores of the image quality indexes of the image is greater than a threshold value; or, for each image quality index, the score of the image quality index is greater than the preset score corresponding to the image quality index.
In addition, the image quality indicator includes a signal-to-noise ratio and/or a dynamic range.
In addition, the image is an infrared image.
Drawings
One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.
Fig. 1 is a flowchart of an image processing method according to a first embodiment of the present invention;
fig. 2 is a flowchart of an image processing method according to a second embodiment of the present invention;
FIG. 3 is a flow diagram of one implementation of the image processing method of FIG. 2;
fig. 4 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that numerous technical details are set forth in order to provide a better understanding of the present application in various embodiments of the present invention. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments.
A first embodiment of the present invention relates to an image processing method including the steps of: calculating the scores of all image quality indexes of the images; and processing the image according to the scores of all the image quality indexes of the image and a predefined strategy space. In this embodiment, since the policy space stores a plurality of image processing policies in advance, the electronic device may select a processing policy suitable for the image from the predefined policy space based on the score of each image quality index of the image, and process the image, so that the image captured in different scenes can be optimized by selecting a suitable manner.
The following describes details of the image processing method according to the present embodiment. The following disclosure provides implementation details for the purpose of facilitating understanding, and is not necessary to practice the present solution.
The image processing method in the present embodiment is applied to an electronic device. The electronic device may be a camera, a terminal, a server, a cloud server, etc. with processing capability. The image may be an infrared image or other image. As shown in fig. 1, the image processing method specifically includesstep 101 and step 102.
Step 101: and acquiring the scores of all image quality indexes of the images.
Specifically, the image quality index may be contrast, sharpness, brightness, signal-to-noise ratio, dynamic range, noise, texture, sharpness, exposure range, black-and-white balance, or the like, and the image quality index used in this embodiment may be any combination of the above indices.
It should be noted that, as will be understood by those skilled in the art, in practical applications, the score of each image quality index may be determined based on the calculated value of each image quality index of the image. For example, if the value of the image quality index is proportional to the image quality, the value of each image quality index of the image is set as the score of the corresponding image quality index, and if the value of the image quality index is inversely proportional to the image quality, the reciprocal of the value of each image quality index of the image is set as the score of the corresponding image quality index. For another example, the value of the image quality index is normalized, and the score of the image quality index is determined according to the value after the normalization. The present embodiment does not limit the manner of calculating the score of the image quality index.
Step 102: and if the scores of the image quality indexes of the image do not meet the preset score requirement, selecting a processing strategy from a predefined strategy space based on the scores of the image quality indexes of the image, and processing the image to obtain a final image.
Specifically, a plurality of image processing strategies are stored in the strategy space in advance. The predefined policy space refers to the totality of processing policies of the images that can be taken that are set in advance.
It should be noted that, as will be understood by those skilled in the art, the processing strategy in the strategy space may be set according to the selected image quality index. For example, the image quality indicator for the evaluation image includes a signal-to-noise ratio, and the policy space may include various processing policies for improving the signal-to-noise ratio. The present embodiment does not limit the number of processing strategies in the strategy space.
In one example, the electronic device determines whether the score of each image quality index of the image meets a preset score requirement; if yes, the image is not processed; if not, go to step 102.
It should be noted that, as will be understood by those skilled in the art, in practical applications, the score requirement may be set according to the quality requirement of the developer for the final image, for example, the preset score requirement includes: the sum of the scores of the image quality indexes of the image is greater than a threshold value; or, for each image quality index, the score of the image quality index is greater than the preset score corresponding to the image quality index. The threshold may be set according to the number of image quality indicators, and the like.
It is worth mentioning that the images meeting the preset requirements are not processed, so that the waste of computing resources caused by processing high-quality images can be avoided.
In one example, the policy space includes multiple processing policies, and the electronic device sequentially selects the processing policies in the policy space and processes the image respectively until all the processing policies in the policy space are traversed. The electronic device selects an image with the highest total score of the scores of the image quality indexes from the processed images as a final image.
In one example, selecting a processing strategy from a predefined strategy space based on the scores of the image quality indexes of the image, and processing the image to obtain a final image, includes: determining at least one policy combination based on the scores of the image quality indicators of the images; the strategy combination at least comprises at least one processing strategy in the strategy space; processing the images respectively by using each processing strategy in the strategy combination; calculating the sum of the scores of all image quality indexes of the processed image; and screening the processed image with the highest sum as a final image. In particular, each processing strategy may correspond to one or more image processing algorithms, e.g., an image denoising algorithm, an image sharpening algorithm, etc.
It is worth mentioning that the images are respectively processed by adopting a plurality of strategy combinations, and the processed image with the optimal processing result is selected as the final image, so that the quality of the final image is higher.
The following exemplifies a method of determining at least one policy combination based on the scores of the respective image quality indicators of the images.
The method comprises the following steps: judging whether the score of the image quality index of the image is larger than the preset score of the image quality index or not aiming at each image quality index; and if not, determining the image quality index as the image quality index to be optimized. And respectively selecting at least one processing strategy from the processing strategies corresponding to the image quality indexes to be optimized in the strategy space, and combining to obtain a strategy combination. Specifically, for each image quality index, one or more processing strategies for improving the image quality index score are stored in the strategy space. And when the score of a certain image quality index of the current image is not more than the preset score of the image quality index, taking the image quality index as the image quality index to be optimized. When the strategy combination is constructed, at least one processing strategy is respectively selected from the processing strategies corresponding to the image quality indexes to be optimized.
Optionally, the execution order of the processing strategies corresponding to the various image quality indexes is limited in the strategy space. For example, if a sharpening processing strategy and a denoising processing strategy exist in the strategy combination, the denoising processing strategy is executed first, and then the sharpening processing strategy is executed. The user presets the execution sequence of each processing strategy according to experience, so that the number of strategy combinations can be effectively reduced, and resource waste caused by executing unnecessary strategy combinations is avoided.
For example, there are 3 image quality indexes to be optimized a1, a2, and A3, and the execution sequence of the processing strategies corresponding to each image quality index to be optimized is: a1 → a2 → A3, 2 treatment strategies corresponding to a1, 3 treatment strategies corresponding to a2 and 2 treatment strategies corresponding to A3, at least 2 × 3 × 2 — 12 strategy combinations can be constructed. The electronic device uses the 12 strategy combinations to perform optimization processing on the image in turn.
The method 2 comprises the following steps: judging whether the score of the image quality index of the image is larger than the preset score of the image quality index or not aiming at each image quality index; and if not, determining the image quality index as the image quality index to be optimized. Selecting N candidate strategy combinations from M pre-stored candidate strategy combinations as strategy combinations to be executed according to the image quality index to be optimized; m is not less than N, and M and N are positive integers.
For example, there are 3 image quality indexes for setting in the electronic apparatus for evaluating image quality, B1, B2, and B3, respectively. Correspondingly, 6 types of candidate strategy combinations are stored in the strategy space, the first type of candidate strategy combination is used for improving the score of B1, the second type of candidate strategy combination is used for improving the scores of B1 and B2, the third type of candidate strategy combination is used for improving the scores of B1, B2 and B3, the fourth type of candidate strategy combination is used for improving the score of B2, the fifth type of candidate strategy combination is used for improving the scores of B2 and B3, and the sixth type of candidate strategy combination is used for improving the score of B3. If the electronic equipment determines that only the score of B1 is not greater than the preset score of B1, the first-class candidate strategy combination is used as a strategy combination to be executed; if the score of only B2 is determined not to be larger than the preset score of B2, the candidate strategy combination of the fourth class is taken as the strategy combination to be executed; if the score of only B3 is determined not to be larger than the preset score of B3, the candidate strategy combination of the sixth class is taken as the strategy combination to be executed; if the score of B1 is not larger than the preset score of B1 and the score of B2 is not larger than the preset score of B2, the candidate strategy combination of the second type is used as the strategy combination to be executed; if the score of B1 is determined to be not greater than the preset score of B1, the score of B2 is determined to be not greater than the preset score of B2, and the score of B3 is determined to be not greater than the preset score of B3, the candidate strategy combination of the third class is taken as the strategy combination to be executed; and if the score of B2 is determined to be not more than the preset score of B2 and the score of B3 is determined to be not more than the preset score of B3, the fifth type of candidate strategy combination is taken as the strategy combination to be executed.
It is worth mentioning that the strategy combination of the optimized image is obtained based on the score of the image quality index, so that the processing strategy for processing the image is more targeted, and the quality of the image is more effectively improved.
It should be noted that, as will be understood by those skilled in the art, in practical applications, other manners may be used to limit the processing strategy for processing the image, and the present embodiment is not limited thereto.
In one example, the processing of the image by using each processing policy in the policy combination comprises: determining the value of a preset parameter of a processing strategy corresponding to the image quality index to be optimized in the strategy combination according to the score of the image quality index to be optimized of the image; and executing each processing strategy in the strategy combination on the image in sequence. Specifically, the electronic device may store the score of the image quality index and the constraint relationship of each parameter in the processing policy corresponding to the image quality index. After the processing strategy to be executed is determined, the score of the image quality index corresponding to the processing strategy is brought into the constraint relation of each parameter in the processing strategy so as to obtain the value of each parameter of the processing strategy to be executed.
It is worth mentioning that, based on the score of the image quality index to be optimized of the image, the value of the preset parameter of the processing strategy corresponding to the image quality index to be optimized in the strategy combination is determined, instead of randomly selecting a numerical value to determine the processing strength of the processing strategy, and the optimal value is determined through multiple attempts, so that the optimization speed of the image can be improved.
It should be noted that, in practical applications, values of preset parameters of the processing policy may also be set according to a preset rule, for example, starting from a preset value, and gradually increasing or gradually decreasing according to a preset difference value until a maximum value or a minimum value is traversed. The present embodiment does not limit the setting manner of the value of the parameter.
The above description is only for illustrative purposes and does not limit the technical aspects of the present invention.
In the image processing method provided in this embodiment, since the policy space stores the processing policies of the plurality of images in advance, the electronic device may select a processing policy suitable for the image from the predefined policy space based on the score of each image quality index of the image, and process the image, so that the image shot in different scenes can be optimized by selecting a suitable mode.
A second embodiment of the present invention relates to an image processing method. The present embodiment is further detailed in the first embodiment, and a method of acquiring the score of each image quality index of an image will be described as an example.
Specifically, as shown in fig. 2, the present embodiment includessteps 201 to 203, wherestep 203 is substantially the same as step 102 in the first embodiment, and is not repeated here, and differences are mainly introduced below.
Step 201: an overexposed area of the image is determined.
Specifically, the electronic device may determine the over-exposed regions of the image by calculating the brightness of the regions of the image.
Step 202: and calculating the scores of all image quality indexes of the image according to other areas except the over-exposure area in the image.
Specifically, in the process of calculating each image quality index of an image, the score calculation of each image quality index is performed excluding an overexposed area in the image.
It is worth mentioning that if a certain part of the image is in an overexposed state, most information of the area will be lost, and the calculation of the signal-to-noise ratio, the dynamic range and the like will lose great effect. And removing the scores of all the image quality indexes of the calculated image of the over-exposure area, so that the scores of all the image quality indexes of the calculated image have reference value, and further the processing efficiency of the image is improved.
Step 203: and if the scores of the image quality indexes of the image do not meet the preset score requirement, selecting a processing strategy from a predefined strategy space based on the scores of the image quality indexes of the image, and processing the image to obtain a final image.
The following takes an image as an infrared image, and image quality indexes including a signal-to-noise ratio and a dynamic range as an example, and the image processing method mentioned in this embodiment is exemplified.
At present, infrared imaging technologies are mature and mainly divided into active infrared night vision technologies and passive infrared night vision technologies. Among them, active infrared night vision technology is the most common. The active infrared night vision technology is a night vision technology that performs observation by actively irradiating and reflecting infrared light of an infrared source with a target, and is often used in a surveillance-type camera. The active infrared night vision technology has the advantages that the night vision imaging is carried out by emitting infrared rays by the active infrared night vision technology without the help of external environment light, the night vision range is wide, and the influence of the environment is small. However, due to the inherent characteristics of the infrared sensor, the infrared image generally has the problems of poor target-to-background contrast, edge blurring and the like, and the image signal-to-noise ratio and the contrast may be low, which need to be solved by an image processing technology. The image processing methods are also different for infrared images of different scenes. For example, in an indoor environment, a long-distance image is blurred and noisy due to problems such as ambient light, and a spatial filtering method is mainly used. In outdoor environment, strong sunlight may make the image brightness and contrast lower, and more methods of tone mapping may be used. Common filtering methods include Gaussian Filter (GF), Bilateral Filter (BF), guided filter (guided filter), median filter (median filter), and mean filter (mean filter). The slightly more complicated filtering methods include local mean filtering, intelligent fuzzy filtering (smart blur), surface fuzzy filtering (surface blur), mean shift filtering (mean shift filter), Bi-Exponential Edge-Preserving smoothing (seeps) filtering, etc. These filtering methods are often used to remove noise from the infrared map. Some filtering methods also have the capability of removing noise while maintaining edge characteristics. Tone mapping (tone mapping) methods are also commonly used in image post-processing, which primarily map image colors to adjust Dynamic Range (DR) and contrast. Common global tone mapping methods are histogram equalization, Gamma (Gamma) mapping, log correction, and piecewise gray scale transformation. The local tone mapping method includes a block median histogram, adaptive histogram equalization, and the like. In addition, a sharpening method is also commonly used in image post-processing, and the sharpening is to highlight high-frequency information, such as edges and contours, on an image so as to make the image clearer. The sharpening operation is the reverse operation of image smoothing, and commonly used methods include a differential method (such as a robert gradient operator method and a laplacian operator method), a high-pass filtering method (such as an Unsharp Masking (USM) method in PS), and a template matching method. In addition, the defogging algorithm is a hot spot studied in current image processing, and increases the visibility of an image by eliminating the fogging in the image. The more common and effective defogging algorithm is the dark channel defogging method of nakeming. Although there are many image processing methods for a single scene, there is no mature image processing method that can be uniformly applied for infrared images of multiple scenes. In addition, although there is a method for adjusting parameters in an image adaptive manner in the unilateral technique, the degree of adaptation is not high enough. Without clear and definite indexes, the image quality cannot be improved. Therefore, there is a need for an adaptive image processing method capable of guiding multiple scenes according to image indexes. Aiming at infrared images in various scenes, the image problem is difficult to uniformly solve by the existing image post-processing technology. In order to improve the quality and the objective feeling of an infrared image and enhance the self-adaption degree of processing a multi-scene infrared image, so that the infrared image can be automatically processed offline, the embodiment provides an image processing method suitable for multiple scenes indoors and outdoors. As shown in fig. 3, one implementation of the image processing method in this embodiment includessteps 301 to 311 described below.
Step 301: and acquiring an infrared image I, and carrying out overexposure prior processing on the infrared image to obtain an infrared image J.
Specifically, brightness is a relatively important concept in an image, and is expressed in various aspects of the image. In this embodiment, the brightness is used to determine the exposure of the image as prior information for improving the signal-to-noise ratio and the dynamic range. If a certain area of the infrared image is in an overexposed state, most information of the area is lost. If this region is not removed, the processing signal-to-noise ratio and dynamic range will be largely lost. Therefore, a flag (mask) may be created to ignore this region information. If one pixel (I, j) in the infrared image I belongs to an overexposed area, mask (I, j) is 0, and if the pixel (I, j) belongs to a normal exposure area, mask (I, j) is 1. Where i is the abscissa of the pixel and j is the ordinate of the pixel. The infrared image J is the image of the infrared image I with the mask. The electronic equipment distinguishes whether each pixel belongs to the over-exposure area through the mask.
Step 302: the score of the infrared image J is calculated.
Specifically, the image quality index includes a signal-to-noise ratio and a dynamic range. And calculating the values of the signal-to-noise ratio and the dynamic range of other areas except the over-exposure area in the infrared image J, and then scoring to obtain the image quality of the image. For example. And calculating the fraction of the infrared image J by using the formula a.
Formula a: score ═ μ SNR + (1- μ) × DR;
wherein score represents the fraction of infrared image J; mu represents the weight coefficient of the signal-to-noise ratio for weighing the proportion of the signal-to-noise ratio and the dynamic range to the total mass, and can be set by a user according to the requirement; SNR represents the signal-to-noise ratio; DR denotes dynamic range.
After the score of the infrared image J is calculated, a subsequent operation is performed to try different strategy combinations of SIGNAL-to-NOISE RATIO (SNR) improvement and Dynamic Range (DR) to find the strategy combination with the highest score. Namely:
formula b:
where D denotes a policy combination corresponding to a final image, D1 denotes a first processing policy, D2 denotes a second processing policy, D1 denotes a policy space for increasing an SNR value, D2 denotes a policy space for increasing a DR value, and D1 and D2 constitute a total policy space. D1E D1 represents that D1 is the processing strategy selected from D1, D2E D2 represents that D2 is the processing strategy selected from D2, max represents a maximum function, and score represents the fraction of the infrared image J processed by the processing strategies D1 and D2.
It should be noted that the formula b is only an example, and in practical applications, a plurality of processing strategies may also be selected from one strategy space, and the present embodiment is not limited.
The following describes the snr and its processing strategy, respectively. The signal-to-noise ratio of an image is the ratio of the mean of the gray values to the standard deviation of the background. I.e. the signal-to-noise ratio can be calculated by formula c.
Formula c: SNR is musig/σsig；
Where SNR represents the signal-to-noise ratio, μ, of an imagesigRepresenting mean of grey values, σsigThe background standard deviation is indicated.
The higher the signal-to-noise ratio, the more information the image represents and the less noise. In order to improve the signal-to-noise ratio, it is necessary to reduce the unwanted noise and enhance the information of the image, mainly the edge features and contour texture of the image. In order to reduce noise and maintain the edge characteristics of the image, a denoising processing strategy, such as an edge-preserving filtering processing strategy, such as bilateral filtering, may be set in the strategy space. In order to enhance the edge characteristics of the image and compensate for the high frequency loss caused by filtering, the strategy space may set a sharpening processing strategy, such as Unsharp Masking (USM) sharpening.
Optionally, the processing order of each processing strategy is limited in the strategy space, for example, a denoising processing strategy is executed first, and then a sharpening processing strategy is executed.
The dynamic range and its handling strategy are exemplified below. The dynamic range refers to the range of light intensity from the brightest area to the darkest area, independent of noise. High-Dynamic Range (HDR) images can provide more Dynamic Range and image details and provide more sufficient image contrast than ordinary images. To improve the dynamic range, a defogging processing strategy and a tone mapping processing strategy can be used.
The foggy image dark channel has a large area of grayness, and the local uniform fog can be removed by using a local darkest point by using a dark channel defogging method.
There are many tone mapping processing strategies, and Gamma (Gamma) correction and block median histogram are the ones that perform well in this system. The principle of Gamma correction is simple, as expressed by formula d:
formula d: j. the design is a squareout＝Jinγ；
Wherein, JoutRepresenting the intensity value of the normalized output image, JinRepresenting the luminance values of the normalized input image. Gamma ray>1, enhancing the contrast of a high gray level area of the image; gamma ray<1, the contrast of the low-gray area of the image is enhanced. The method of Gamma correction can be adapted locally and also to segmentation.
The block median histogram is changed from a histogram equalization method, the input image is firstly subjected to average blocking and is cut into a plurality of sub-images, then histogram statistics is respectively carried out on each sub-image, and the contrast of the image can be obviously enhanced.
Step 303: and judging whether the score of the infrared image J meets the score requirement. If yes, go to step 304, and if no, go to step 305.
Step 304: the infrared image J is taken as the final infraredimage K. Step 311 is then performed.
Step 305: the score of the infrared image J is taken as the score to be compared.
Step 306: and selecting a strategy combination from the strategy space, and processing the infrared image J according to the strategy combination.
For example, if the selected policy combination includes a certain defogging processing policy, a certain denoising processing policy, a certain sharpening processing policy, and a certain tone mapping policy, the electronic device applies the certain defogging processing policy in the policy space to the infrared image J to obtain the infrared image J _ 1; carrying out a certain denoising processing strategy in a strategy space on the infrared image J _1 to obtain an infrared image J _ 2; carrying out a certain sharpening processing strategy in a strategy space on the infrared image J _2 to obtain an infrared image J _ 3; and (3) carrying out a certain tone mapping strategy in the strategy space on the infrared image J _3 to obtain an infrared image J _4, namely an infrared image L.
Step 307: the score of the infrared image L is calculated.
Step 308: and judging whether the score of the infrared image L is higher than the score to be compared. If yes, go to step 309, and if not, go to step 310.
Step 309: and taking the infrared image L as a final infrared image K, and taking the score of the infrared image L as a score to be compared. Step 310 is then performed.
Step 310: and judging whether all the strategy combinations in the strategy space are traversed. If yes, go to step 311, and if not, go to step 306.
Step 311: and outputting the final infrared image K.
The above description is only for illustrative purposes and does not limit the technical aspects of the present invention.
In the image processing method provided in this embodiment, since the policy space stores the processing policies of the plurality of images in advance, the electronic device may select a processing policy suitable for the image from the predefined policy space based on the score of each image quality index of the image, and process the image, so that the image shot in different scenes can be optimized by selecting a suitable mode. In addition, the scores of all the image quality indexes of the image calculated in the over-exposure area are removed, so that the calculated scores of all the image quality indexes of the image have reference value, and the processing efficiency of the image is improved.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
A third embodiment of the present invention relates to an electronic apparatus, as shown in fig. 4, including: at least oneprocessor 401; and amemory 402 communicatively coupled to the at least oneprocessor 401; thememory 402 stores instructions executable by the at least oneprocessor 401, and the instructions are executed by the at least oneprocessor 401, so that the at least oneprocessor 401 can execute the image processing method according to the above embodiment.
The electronic device includes: one ormore processors 401 and amemory 402, oneprocessor 401 being exemplified in fig. 4. Theprocessor 401 and thememory 402 may be connected by a bus or other means, and fig. 4 illustrates the connection by a bus as an example. Thememory 402 is a non-volatile computer-readable storage medium for storing non-volatile software programs, non-volatile computer-executable programs, and modules, such as algorithms corresponding to the processing strategies in the strategy space in the embodiment of the present application, stored in thememory 402. Theprocessor 401 executes various functional applications of the apparatus and data processing, i.e., implements the above-described image processing method, by executing nonvolatile software programs, instructions, and modules stored in thememory 402.
Thememory 402 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store a list of options, etc. Further, thememory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments,memory 402 may optionally include memory located remotely fromprocessor 401, which may be connected to an external device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
One or more modules are stored in thememory 402 and, when executed by the one ormore processors 401, perform the image processing method of any of the method embodiments described above.
The product can execute the method provided by the embodiment of the application, has corresponding functional modules and beneficial effects of the execution method, and can refer to the method provided by the embodiment of the application without detailed technical details in the embodiment.
A fourth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method of the above embodiments may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps in the method of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific embodiments for practicing the invention, and that various changes in form and details may be made therein without departing from the spirit and scope of the invention in practice.
Claims (10)
1. An image processing method, comprising:
acquiring the scores of all image quality indexes of the image;
if the scores of the image quality indexes of the image do not meet the preset score requirement, selecting a processing strategy from a predefined strategy space based on the scores of the image quality indexes of the image, and processing the image to obtain a final image; the strategy space stores a plurality of image processing strategies in advance.
2. The image processing method according to claim 1, wherein the obtaining of the score of each image quality index of the image comprises:
determining an overexposed area of the image;
and calculating the scores of all image quality indexes of the image according to other areas except the over-exposure area in the image.
3. The image processing method according to claim 1 or 2, wherein the selecting a processing policy from a predefined policy space based on the score of each image quality index of the image, and processing the image to obtain a final image comprises:
determining at least one policy combination based on the scores of the image quality indicators of the images; at least one processing strategy in the strategy space is included in the strategy combination;
processing the images respectively by utilizing each processing strategy in the strategy combination;
calculating the sum of the scores of all image quality indexes of the processed image;
and screening the processed image with the highest sum as the final image.
4. The method according to claim 3, wherein determining at least one policy combination based on the scores of the image quality indicators of the images comprises:
judging whether the score of the image quality index of the image is larger than a preset score of the image quality index or not aiming at each image quality index; if not, determining the image quality index as an image quality index to be optimized;
respectively selecting at least one processing strategy from the processing strategies corresponding to each image quality index to be optimized in the strategy space, and combining to obtain the strategy combination; or selecting N candidate strategy combinations from M pre-stored candidate strategy combinations according to the image quality index to be optimized, wherein the N candidate strategy combinations are used as the strategy combinations to be executed; m is not less than N, and M and N are positive integers.
5. The image processing method according to claim 4, wherein the processing the image using each processing policy of the policy combination comprises:
determining the value of a preset parameter of a processing strategy corresponding to the image quality index to be optimized in the strategy combination according to the score of the image quality index to be optimized of the image;
and executing each processing strategy in the strategy combination on the image in sequence.
6. The image processing method according to claim 1 or 2, wherein the preset score requirement comprises: the sum of the scores of the image quality indexes of the image is greater than a threshold value; or, for each image quality index, the score of the image quality index is greater than the preset score corresponding to the image quality index.
7. The image processing method according to claim 1 or 2, wherein the image quality indicator comprises a signal-to-noise ratio and/or a dynamic range.
8. The image processing method according to claim 1, wherein the image is an infrared image.
9. An electronic device, comprising: at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the image processing method of any one of claims 1 to 8.
10. A computer-readable storage medium, storing a computer program, wherein the computer program, when executed by a processor, implements the image processing method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322423.5ACN112967273B (en) | 2021-03-25 | 2021-03-25 | Image processing method, electronic device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322423.5ACN112967273B (en) | 2021-03-25 | 2021-03-25 | Image processing method, electronic device, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112967273Atrue CN112967273A (en) | 2021-06-15 |
| CN112967273B CN112967273B (en) | 2021-11-16 |
Family
ID=76278478
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110322423.5AActiveCN112967273B (en) | 2021-03-25 | 2021-03-25 | Image processing method, electronic device, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112967273B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113938602A (en)* | 2021-09-08 | 2022-01-14 | 荣耀终端有限公司 | Image processing method, electronic device, chip and readable storage medium |
| CN114255185A (en)* | 2021-12-16 | 2022-03-29 | 武汉高德智感科技有限公司 | Image processing method, device, terminal and storage medium |
| CN116385308A (en)* | 2023-04-15 | 2023-07-04 | 杨利容 | Combined image processing optimization strategy selection system |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020006231A1 (en)* | 2000-07-11 | 2002-01-17 | Mediaflow, Llc | Adaptive edge detection and enhancement for image processing |
| US20140180758A1 (en)* | 2012-12-21 | 2014-06-26 | Ebay Inc. | System and method for image quality scoring |
| JP2015053543A (en)* | 2013-09-05 | 2015-03-19 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
| CN106528879A (en)* | 2016-12-14 | 2017-03-22 | 北京小米移动软件有限公司 | Picture processing method and device |
| CN108230253A (en)* | 2017-05-08 | 2018-06-29 | 北京市商汤科技开发有限公司 | Image recovery method, device, electronic equipment and computer storage media |
| CN108665428A (en)* | 2018-04-26 | 2018-10-16 | 青岛海信移动通信技术股份有限公司 | Image enchancing method, device, equipment and storage medium |
| CN109360197A (en)* | 2018-09-30 | 2019-02-19 | 北京达佳互联信息技术有限公司 | Processing method, device, electronic equipment and the storage medium of image |
| CN110298310A (en)* | 2019-06-28 | 2019-10-01 | 深圳市商汤科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN110581957A (en)* | 2019-08-26 | 2019-12-17 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic device |
| CN110796624A (en)* | 2019-10-31 | 2020-02-14 | 北京金山云网络技术有限公司 | Image generation method and device and electronic equipment |
| CN111696057A (en)* | 2020-05-25 | 2020-09-22 | 北京的卢深视科技有限公司 | Depth image denoising method and device |
| CN111917986A (en)* | 2020-08-20 | 2020-11-10 | 安谋科技（中国）有限公司 | Image processing method and its medium and electronic device |
| CN112070739A (en)* | 2020-09-03 | 2020-12-11 | Oppo广东移动通信有限公司 | Image processing method, device, electronic device and storage medium |
- 2021
- 2021-03-25CNCN202110322423.5Apatent/CN112967273B/enactiveActive
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020006231A1 (en)* | 2000-07-11 | 2002-01-17 | Mediaflow, Llc | Adaptive edge detection and enhancement for image processing |
| US20140180758A1 (en)* | 2012-12-21 | 2014-06-26 | Ebay Inc. | System and method for image quality scoring |
| JP2015053543A (en)* | 2013-09-05 | 2015-03-19 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
| CN106528879A (en)* | 2016-12-14 | 2017-03-22 | 北京小米移动软件有限公司 | Picture processing method and device |
| CN108230253A (en)* | 2017-05-08 | 2018-06-29 | 北京市商汤科技开发有限公司 | Image recovery method, device, electronic equipment and computer storage media |
| CN108665428A (en)* | 2018-04-26 | 2018-10-16 | 青岛海信移动通信技术股份有限公司 | Image enchancing method, device, equipment and storage medium |
| CN109360197A (en)* | 2018-09-30 | 2019-02-19 | 北京达佳互联信息技术有限公司 | Processing method, device, electronic equipment and the storage medium of image |
| CN110298310A (en)* | 2019-06-28 | 2019-10-01 | 深圳市商汤科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN110581957A (en)* | 2019-08-26 | 2019-12-17 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic device |
| CN110796624A (en)* | 2019-10-31 | 2020-02-14 | 北京金山云网络技术有限公司 | Image generation method and device and electronic equipment |
| CN111696057A (en)* | 2020-05-25 | 2020-09-22 | 北京的卢深视科技有限公司 | Depth image denoising method and device |
| CN111917986A (en)* | 2020-08-20 | 2020-11-10 | 安谋科技（中国）有限公司 | Image processing method and its medium and electronic device |
| CN112070739A (en)* | 2020-09-03 | 2020-12-11 | Oppo广东移动通信有限公司 | Image processing method, device, electronic device and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| MICHELE A.SAAD等: "Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》* |
| 吕晓新: "基于深度学习的图像质量评价算法研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》* |
| 陈可昕: "基于卷积神经网络的图像去雾效果评价", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》* |
| 高琦等: "CT图像的质量评估策略:基于预恢复图像先验信息", 《南方医科大学学报》* |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113938602A (en)* | 2021-09-08 | 2022-01-14 | 荣耀终端有限公司 | Image processing method, electronic device, chip and readable storage medium |
| CN113938602B (en)* | 2021-09-08 | 2022-08-02 | 荣耀终端有限公司 | Image processing method, electronic device, chip and readable storage medium |
| CN114255185A (en)* | 2021-12-16 | 2022-03-29 | 武汉高德智感科技有限公司 | Image processing method, device, terminal and storage medium |
| CN114255185B (en)* | 2021-12-16 | 2022-11-25 | 武汉高德智感科技有限公司 | Image processing method, device, terminal and storage medium |
| CN116385308A (en)* | 2023-04-15 | 2023-07-04 | 杨利容 | Combined image processing optimization strategy selection system |
| CN116385308B (en)* | 2023-04-15 | 2024-05-07 | 广州海至亚传媒科技有限公司 | Combined image processing optimization strategy selection system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112967273B (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bai et al. | Underwater image enhancement based on global and local equalization of histogram and dual-image multi-scale fusion | |
| Galdran | Image dehazing by artificial multiple-exposure image fusion | |
| CN112967273B (en) | Image processing method, electronic device, and storage medium | |
| CN110889812B (en) | A method of underwater image enhancement based on multi-scale fusion of image feature information | |
| CN110503617B (en) | Underwater image enhancement method based on high-frequency and low-frequency information fusion | |
| CN105046658B (en) | A kind of low-light (level) image processing method and device | |
| CN108765342A (en) | A kind of underwater image restoration method based on improvement dark | |
| WO2016206087A1 (en) | Low-illumination image processing method and device | |
| CN104766284B (en) | The restored method of underwater colored blur image | |
| CN110349114A (en) | Applied to the image enchancing method of AOI equipment, device and road video monitoring equipment | |
| JP2024545795A (en) | Image noise removal method, device and storage medium | |
| CN119052629A (en) | Optimization method and system for management model of video image shot by unmanned aerial vehicle | |
| CN107742280A (en) | Image sharpening method and device | |
| CN112313700A (en) | Image processing method and device | |
| Agrawal et al. | A novel contrast and saturation prior for image dehazing | |
| Li et al. | Soft binary segmentation-based backlit image enhancement | |
| Xi et al. | Enhancement of unmanned aerial vehicle image with shadow removal based on optimized retinex algorithm | |
| CN118333902B (en) | Method, system, equipment and medium for clearing underwater non-uniform illumination image | |
| CN113379631B (en) | Image defogging method and device | |
| CN119831855A (en) | Infrared image enhancement method based on CLAHE | |
| Srinivas et al. | Channel prior based Retinex model for underwater image enhancement | |
| CN118247181B (en) | Image restoration model training method, electronic device and image restoration method | |
| Yang et al. | Correction of overexposure utilizing haze removal model and image fusion technique | |
| CN107451971A (en) | The blind convolved image restoring method of low-light (level) of priori is combined based on dark and Gauss | |
| Nie et al. | Image defogging based on joint contrast enhancement and multi-scale fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | Application publication date:20210615 Assignee:Anhui Xingtai Financial Leasing Co.,Ltd. Assignor:Hefei lushenshi Technology Co.,Ltd. Contract record no.:X2022980006062 Denomination of invention:Image processing method, electronic equipment and storage medium Granted publication date:20211116 License type:Exclusive License Record date:20220523 | |
| EE01 | Entry into force of recordation of patent licensing contract | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | Denomination of invention:Image processing method, electronic equipment and storage medium Effective date of registration:20220525 Granted publication date:20211116 Pledgee:Anhui Xingtai Financial Leasing Co.,Ltd. Pledgor:Hefei lushenshi Technology Co.,Ltd. Registration number:Y2022980006214 | |
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right | Date of cancellation:20230719 Granted publication date:20211116 Pledgee:Anhui Xingtai Financial Leasing Co.,Ltd. Pledgor:Hefei lushenshi Technology Co.,Ltd. Registration number:Y2022980006214 | |
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| EC01 | Cancellation of recordation of patent licensing contract | Assignee:Anhui Xingtai Financial Leasing Co.,Ltd. Assignor:Hefei lushenshi Technology Co.,Ltd. Contract record no.:X2022980006062 Date of cancellation:20230720 | |
| EC01 | Cancellation of recordation of patent licensing contract | ||
| TR01 | Transfer of patent right | Effective date of registration:20230814 Address after:230091 room 611-217, R & D center building, China (Hefei) international intelligent voice Industrial Park, 3333 Xiyou Road, high tech Zone, Hefei, Anhui Province Patentee after:Hefei lushenshi Technology Co.,Ltd. Address before:100083 room 3032, North B, bungalow, building 2, A5 Xueyuan Road, Haidian District, Beijing Patentee before:BEIJING DILUSENSE TECHNOLOGY CO.,LTD. Patentee before:Hefei lushenshi Technology Co.,Ltd. | |
| TR01 | Transfer of patent right |