wherein, X₁To the left boundary where image a and image b overlap, X₂Is the right boundary where image a and image b overlap, X is a specific column position where image a and image b overlap, W_aAs a fusion weight of the image a at the X position, W_bAs a fusion weight of image b at the X position, X_aFor data on image a at X position, X_bFor data on image b at X position, X_mergeIs the fused image data at the X position, [0, X₂]Is the length of image a, [ X ]₁,X₃]Is the length of image b, [ X ]₁,X₂]Is the overlapping region of image a and image b, [0, X₃]The image length after the image a and the image b are fused.

Fig. 3 schematically shows a diagram of the change of the weighting weights for overlapping pixel points.

Fig. 4 exemplarily shows a process of generating a target image after an original image including four color channels is subjected to image preprocessing, a deep neural network, a clamping process, a stretching process and a stitching process.

Fig. 5 exemplarily shows an original image, an original image after image preprocessing, and a target image. As can be seen from fig. 5, the image processing method provided by the embodiment can effectively improve the signal-to-noise ratio and the definition of the short-exposure image acquired in the dark environment.

According to the method, the original image acquired in the dark environment is subjected to image preprocessing, the original image subjected to image preprocessing is input into the deep neural network to be subjected to forward calculation, an output result is obtained, the target image is generated according to the output result, the original image acquired in the dark environment can be processed into the target image with high signal-to-noise ratio and low noise level, and the definition of the original image is effectively improved.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

Example two

As shown in fig. 6, the present embodiment provides animage processing system 5 for executing the method steps in the first embodiment, which may be a software program system in an image capturing apparatus or a computing apparatus, or a software program system executed when a computer program is executed by a processor of an image capturing apparatus or a computing apparatus.

In a Specific Application, the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

As shown in fig. 6, theimage processing system 5 includes:

apreprocessing module 501, configured to perform image preprocessing on an original image; the original image is an image acquired in an environment with brightness lower than a preset illuminance;

a calculatingmodule 502, configured to input the original image after image preprocessing into a deep neural network for forward calculation to obtain an output result;

and animage processing module 503, configured to generate a target image according to the output result. In an embodiment, the image processing module is specifically configured to perform clamping, stretching, and stitching on the output result to generate a target image.

In one embodiment, theimage processing system 5 further includes:

and the image acquisition module is used for acquiring an original image under the environment that the brightness is lower than the preset illuminance.

In a specific application, the modules may be implemented by independent processors, or may be integrated together into one processor.

EXAMPLE III

As shown in fig. 7, the present embodiment provides aterminal device 6, which includes: aprocessor 60, amemory 61 and acomputer program 62, such as a depth information estimation program, stored in saidmemory 61 and executable on saidprocessor 60. Theprocessor 60, when executing thecomputer program 62, implements the steps in the various depth information estimation method embodiments described above, such as steps S101 to S103 shown in fig. 1. Alternatively, theprocessor 60, when executing thecomputer program 62, implements the functions of the modules in the above device embodiments, such as the functions of themodules 501 to 505 shown in fig. 6.

Illustratively, thecomputer program 62 may be partitioned into one or more modules that are stored in thememory 61 and executed by theprocessor 60 to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of thecomputer program 62 in theterminal device 6. For example, thecomputer program 62 may be divided into a preprocessing module, a computing module, and an image processing module, each of which functions specifically as follows:

the calculation module is used for inputting the original image after image preprocessing into a deep neural network for forward calculation to obtain an output result; and the image processing module is used for generating a target image according to the output result.

Theterminal device 6 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, aprocessor 60, amemory 61. Those skilled in the art will appreciate that fig. 7 is merely an example of aterminal device 6 and does not constitute a limitation ofterminal device 6 and may include more or fewer components than shown, or some components may be combined, or different components, for example, the terminal device may also include input output devices, network access devices, buses, etc.

TheProcessor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

Thememory 61 may be an internal storage unit of theterminal device 6, such as a hard disk or a memory of theterminal device 6. Thememory 61 may also be an external storage device of theterminal device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on theterminal device 6. Further, thememory 61 may also include both an internal storage unit and an external storage device of theterminal device 6. Thememory 61 is used for storing the computer program and other programs and data required by the terminal device. Thememory 61 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated module, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims

1. An image processing method, comprising:

and generating a target image according to the output result.

2. The image processing method of claim 1, wherein the deep neural network includes 1 feature extraction layer, a preset number of down-sampling layers, 1 intermediate processing layer, and the preset number +2 up-sampling layers;

inputting the original image after image preprocessing into a deep neural network for forward calculation to obtain an output result, wherein the output result comprises the following steps:

inputting the original image after image preprocessing into the 1 feature extraction layer for feature extraction;

sequentially performing downsampling on the original image after the features are extracted through a preset number of downsampling layers;

performing inverse residual calculation on the downsampled original image through the 1 intermediate processing layer;

and sequentially carrying out upsampling on the original image after the reverse residual error calculation through the preset number of +2 upsampling layers to obtain an output result.

3. The image processing method of claim 2, wherein the 1 feature extraction layer includes 1 first convolution layer of a first step size and a convolution kernel size of a first size and 1 second convolution layer of a second step size and a convolution kernel size of a second size;

each downsampling layer comprises 1 third convolution layer with a third step length and a convolution kernel of the third size and 1 reverse residual block with an expansion coefficient being a preset coefficient;

the 1 intermediate processing layer comprises a preset number of reverse residual blocks with expansion coefficients as preset coefficients;

the pre-set number of the up-sampling layers are all constructed based on a bilinear interpolation algorithm and a short connection form and comprise 1 fourth convolution layer with a fourth step length and a convolution kernel with a fourth size and 1 reverse residual block with an expansion coefficient as a pre-set coefficient;

4. The image processing method of claim 3, wherein the short connection forms of the up-sampling layers of the previous preset number are constructed in a manner that:

5. The image processing method according to claim 3 or 4, wherein the preset number is 4;

the number of output channels of the first convolution layer is 32, and the number of output channels of the second convolution layer is 16;

the number of output channels of the preset number of downsampling layers is 32, 64, 128 and 256 respectively;

the number of output channels of the reverse residual blocks with the preset number of expansion coefficients as the preset coefficients is 256;

the number of output channels of the up-sampling layers with the preset number is 128, 64, 32 and 16 respectively;

the number of output channels of the first deconvolution layer is 16, the number of output channels of the fifth convolution layer is 16, and the number of output channels of the sixth convolution layer is 12;

the number of output channels of the second deconvolution layer is 3.

6. The image processing method of claims 1 to 4, wherein the image preprocessing of the original image comprises:

carrying out color channel separation on an original image, and storing the original image as images of a preset number of color channels according to the number and the sequence of the color channels of the original image;

performing black level correction on the images of the preset number of color channels;

normalizing the images of the preset number of color channels after the black level correction;

amplifying the images of the preset number of color channels after normalization processing;

the amplified images of the preset number of color channels are subjected to clamping processing;

and clipping the clamped images of the preset number of color channels.

7. The image processing method according to claims 1 to 4, wherein the deep neural network is subjected to global optimization training for the long-exposure image and the image-preprocessed short-exposure image in advance until convergence.

8. The image processing method of claims 1 to 4, wherein generating a target image from the output result comprises:

9. An image processing system, comprising:

10. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 8 when executing the computer program.