CN113141475B - Imaging system and pixel merging method - Google Patents

Abstract

An imaging system comprises a pixel array formed by a plurality of pixel units, a control circuit, a reading circuit and a functional logic control unit, wherein a first pixel in three adjacent pixel units in three adjacent columns forms a pixel unit block, a fourth pixel in three adjacent pixel units in three adjacent columns forms a pixel unit block, the nearest three second or third pixels in an m-th row, a second or third pixel in an m+1th row and an m-1 th row which are diagonally adjacent to the middle one of the three second or third pixels in the m-th row, and a second or third pixel in an m+2th row, an m-2 th row which is diagonally adjacent to the two pixels in the m+1th row and the m-1 th row form a pixel unit block. In the imaging system and the pixel merging method, due to the merging mode of the pixel unit blocks, all pixels participate in merging without loss, and the pixel merging is ensured not to be overlapped, so that the definition of an image is ensured, and the signal to noise ratio of the image is improved.

Description

Imaging system and pixel merging method

Technical Field

The present invention relates to the field of image sensors, and in particular, to an imaging system and a pixel merging method thereof.

Background

The image acquisition device generally includes an image sensor and a lens. The lens focuses light onto an image sensor to form an image, which converts the light signal into an electrical signal. The electrical signals are output from the image acquisition device to other components of the electronic system. The image acquisition device and other components of the electronic system constitute an imaging system. Image sensors have found widespread use in various electronic systems, such as cell phones, digital cameras, medical devices, and computers.

A typical image sensor includes a plurality of photosensitive image cells ("pixels") arranged in a two-dimensional array. The image sensor is enabled to form color images by forming a Color Filter Array (CFA) over the pixels. Typically, each pixel is covered by a single color filter, which in turn is covered by a microlens to focus the light onto a photodiode. The repeating pixel modules form a pixel array, wherein the pixel modules are a 2 x 2 array of four pixels covered with a red, a blue and two green filters, forming a well known bayer pattern CFA. Techniques for fabricating image sensors, particularly Complementary Metal Oxide Semiconductor (CMOS) image sensors, continue to advance. For example, the demand for higher resolution and lower power consumption has further prompted miniaturization and integration of these image sensors. However, miniaturization is accompanied by a decrease in the light sensitivity and dynamic range of the pixels, and new methods are required to solve these problems. In many applications, the sensor resolution has exceeded the optical resolution, meaning that increasing the pixel density does not necessarily lead to an increase in image quality.

As the pixel size decreases, the total amount of light absorbed within the pixel decreases and some advanced features are challenged. Typically, the output resolution of the camera system is lower than the resolution of the image sensor, and one way to increase the amount of light collected at that time to represent a point in the image sensor is to sum the signals from adjacent or nearby pixels that share the same color filter. This is called pixel binning (pixel binning) to improve the sensitivity when capturing images at low illumination.

In high resolution image sensors, pixel binning is required when the ambient light is very weak. As shown in fig. 1, pixels corresponding to the same color adjacent to a certain region are generally merged. However, such merging may have partial pixels overlapping and merging, resulting in reduced image sharpness, and if non-overlapping is to be ensured, there may be partial pixels missing, resulting in reduced signal-to-noise ratio.

Disclosure of Invention

The invention aims to provide an imaging system with clear image and high signal-to-noise ratio and a pixel merging method thereof.

The present invention provides an imaging system comprising:

a pixel array formed by a plurality of pixel units, wherein each pixel unit comprises four pixels: a first pixel, a second pixel, a third pixel and a fourth pixel, wherein the pixels in each pixel unit are arranged in a form of 2 x 2, the first pixel and the fourth pixel are diagonally adjacent, the second pixel and the third pixel are diagonally adjacent, the first pixel forms a first color pixel, the second pixel and the third pixel form a second color pixel, and the fourth pixel forms a third color pixel; wherein the pixels further constitute a plurality of pixel unit blocks, the first pixels in the pixel units of adjacent three rows in adjacent three columns constitute one pixel unit block, the fourth pixels in the pixel units of adjacent three rows in adjacent three columns constitute one pixel unit block, the nearest three second color pixels in the m-th row, the m+1th row and the m-1th row are all diagonally adjacent to the middle one of the three second color pixels in the m-th row, the second color pixels in the m+2th row are all diagonally adjacent to the two pixels in the m+1th row, and the second color pixels in the m-2th row are all diagonally adjacent to the two pixels in the m-1th row constitute one pixel unit block, wherein m is a positive integer greater than or equal to 3;

a control circuit coupled to the pixel array to control pixel output pixel values in the pixel array;

a readout circuit coupled to the pixel array to read pixel values output by the pixel array; and

and the functional logic control unit is coupled to the readout circuit and used for respectively merging pixel values of the first pixel, the second pixel, the third pixel and the fourth pixel in the pixel array according to the pixel unit block and then outputting a merged image.

The invention also provides a pixel merging method, which comprises the following steps:

reading a pixel value output by a pixel array, wherein the pixel array comprises a plurality of pixel units, and each pixel unit comprises four pixels: the pixel unit comprises a first pixel, a second pixel, a third pixel and a fourth pixel, wherein the pixels in each pixel unit are arranged in a form of 2 x 2, the first pixel and the fourth pixel are arranged diagonally adjacent, the second pixel and the third pixel are arranged diagonally adjacent, the pixels also form a plurality of pixel unit blocks, the first pixel forms a first color pixel, the second pixel and the third pixel form a second color pixel, and the fourth pixel forms a third color pixel;

receiving a pixel merging instruction, merging the first pixels in the pixel units of three adjacent rows in adjacent three columns into one pixel unit block, merging the fourth pixels in the pixel units of three adjacent rows in adjacent three columns into one pixel unit block, merging the nearest three second color pixels in an mth row, the second color pixels in an m+1th row and an m-1th row, which are diagonally adjacent to the middle one of the three second color pixels in the m-th row, the second color pixels in an m+2th row, which are diagonally adjacent to the two pixels in the m+1th row, and the second color pixels in an m-2th row, which are diagonally adjacent to the two pixels in the m-1th row, into one pixel unit block, and merging output signals of all pixels in each pixel unit block, thereby obtaining an image of a lower pixel;

and merging the pixel values of the first pixel, the second pixel, the third pixel and the fourth pixel in the pixel array according to the pixel unit block, and then outputting a merged image.

In the imaging system and the pixel merging method provided by the invention, due to the merging mode of the pixel unit blocks, all pixels participate in merging without loss, and the pixel merging is ensured not to be overlapped, so that the definition of an image is ensured, and the signal to noise ratio of the image is improved.

Drawings

Fig. 1 is a schematic diagram of a pixel merging layout in the prior art.

Fig. 2 is a schematic circuit diagram of an imaging system according to an embodiment of the invention.

FIG. 3 is a schematic diagram of a pixel binning layout of an imaging system in one embodiment of the present invention.

Fig. 4A and 4B are schematic diagrams illustrating different pixel combinations of an imaging system according to an embodiment of the invention.

Fig. 5 is a flow chart of a method of merging pixels of an imaging system in an embodiment of the invention.

Detailed Description

In order to further describe the technical manner and efficacy of the present invention for achieving the intended purpose, the following detailed description of the embodiments, structures, features and efficacy of the invention refers to the accompanying drawings and examples.

In this embodiment, as shown in fig. 2, the imaging system of the present invention includes a control circuit 15, a status register 16, a readout circuit 17, a pixel array 18, and a functional logic unit 19, the pixel array 18 is coupled to the control circuit 15 and the readout circuit 17, and the readout circuit 17 is coupled to the status register 16 and the functional logic unit 19. Control circuit 15 is coupled to a status register 16 and a pixel array 18. The control circuit 15 may include a row decoder and a row driver having a desired timing circuit, and the readout circuit 17 may include a column decoder and a column driver having a desired timing circuit, and an amplifying circuit, an analog-to-digital conversion circuit (ADC), and the like. In one example, readout circuitry 17 may read out image data row by row along readout column lines, or may read out image data using other techniques, such as serial readout or full parallel readout of all pixels. The status register 16 may include a digitally programmed selection system, such as a configuration, to determine whether the read mode is through a rolling exposure or through a global exposure, as well as to determine the timing and signal levels applied in each mode.

In one example, pixel array 18 comprises a two-dimensional (2D) array of a plurality of pixels arranged in rows and columns. Each column of pixels in the pixel array 18 is selectively turned on by a column select line and each row of pixels is respectively output all at the same time by a row select line. Each pixel has a row address and a column address. The column address of the pixel corresponds to a row select line driven by the column decode and drive circuitry, and the row address of the pixel corresponds to a row select line driven by the row decode and drive circuitry. The control circuit 15 controls the column decoding and driving circuit and the row decoding and driving circuit to selectively read out the pixel output signals corresponding to the appropriate rows and columns in the pixel array. After each pixel acquires its image data or image charge, the image data is read out by the readout circuit 17 according to the readout mode set by the status register 16, and then transferred to the functional logic unit 19. The functional logic 19 may store only image data or may process the image data according to a later image effect (e.g., cropping, rotating, removing red-eye, adjusting brightness, adjusting contrast, or otherwise).

Referring to fig. 3, in the imaging system according to an embodiment of the present invention, the pixel array 18 includes a plurality of pixel units 11, and each pixel unit 11 includes four pixels: the first pixel 112, the second pixel 114, the third pixel 116 and the fourth pixel 118, the pixels in each pixel unit 11 are arranged in a form of 2×2, the first pixel 112 and the fourth pixel 118 are diagonally adjacent, the second pixel 114 and the third pixel 116 are diagonally adjacent, the first pixel 112 forms a first color pixel, the second pixel 114 and the third pixel 116 forms a second color pixel, and the fourth pixel 118 forms a third color pixel. The pixels also constitute a plurality of pixel unit blocks 13, the first pixels 112 in the pixel units 11 of the adjacent three rows in the adjacent three columns constitute one pixel unit block 13, the fourth pixels 118 in the pixel units 11 of the adjacent three rows in the adjacent three columns constitute one pixel unit block 13, the closest three second color pixels in the m-th row, the second color pixels in the m+1th row and the m-1 th row, which are diagonally adjacent to the middle one of the three second color pixels in the m-th row, the second color pixels in the m+2th row, which are diagonally adjacent to the two pixels in the m+1th row, and the second color pixels in the m-2 th row, which are diagonally adjacent to the two pixels in the m-1 th row, constitute one pixel unit block 13. Wherein m is a positive integer greater than or equal to 3.

In this embodiment, the imaging system further includes color filters, and color filters of the same color are disposed on the pixels corresponding to each pixel unit block 13. Specifically, the color filters include a blue filter, a red filter, and a green filter, the blue filter is disposed on the first pixel 112, the red filter is disposed on the fourth pixel 118, and the green filter is disposed on the second pixel 114 and the third pixel 116. As shown in fig. 3, the first pixel 112 covers the blue filter B to form a blue pixel, the fourth pixel 118 covers the red filter R to form a red pixel, the second pixel 114 covers the green filter Gr to form a green pixel, and the third pixel 116 covers the green filter Gb to form a green pixel, thereby forming a bayer pattern color filter array.

Specifically, in the embodiment of the present invention, as shown in fig. 4A, the pixel unit block 13 composed of the blue pixel, that is, the first pixel 112 and the red pixel, that is, the fourth pixel 118 is disposed in the form of 3*3 and is rectangular; as shown in fig. 4B, the pixel unit blocks 13 composed of the second pixels 114 and the third pixels 116, which are green pixels, are arranged in the form of 3*3 and are in a diamond-shaped distribution. In the embodiment of the present invention, the control circuit 15 controls the pixels in the pixel array 18 to perform exposure according to the configuration of the status register 16, the readout circuit 17 reads the pixel values output by the pixel array 18 after the exposure is finished, and the functional logic unit 19 receives the pixel values of the pixel array output by the readout circuit 17 and combines the pixel values of the first pixel 112, the second pixel 114, the third pixel 116 and the fourth pixel 118 in the pixel array according to the pixel unit block 13 to output a combined image. In one embodiment, the imaging system of the functional logic 19 merges the pixel values of the first pixel 112, the second pixel 114, the third pixel 116 and the fourth pixel 118 in the pixel array according to the pixel unit block 13 in the form of a weighted average. The weighting coefficients of the weighted averages may be stored in the status register 16, and the weighting coefficients may be the same or different.

In the imaging system, due to the combination mode of the pixel unit blocks 13, all pixels participate in combination and are not lost, and the combination of the pixels is ensured not to be overlapped, so that the definition of an image is ensured, and the signal to noise ratio of the image is improved.

In this embodiment, the imaging system may further include a controller 20 and a detecting element 21, wherein the controller 20 is connected to the functional logic 19, and the detecting element 21 is connected to the controller 20. The controller 20 is used for controlling the operation mode of the functional logic 19, and the detecting element 21 is used for detecting the intensity of the ambient light. The detecting element 21 outputs a first indication signal to the controller 20 when the ambient light is weak (for example, lower than a preset value), and the controller 20 provides a pixel merging instruction to the functional logic 19, so that the functional logic 19 enters a pixel merging operation mode to merge the output signals of all pixels in each pixel unit block 13, thereby obtaining an image of a lower pixel. Of course, when the ambient light is strong, the detecting element 21 outputs the second indication signal to the controller 20, and the controller 20 provides the normal operation instruction to the functional logic unit 19, so that the functional logic unit 19 enters the normal operation mode, and the output signal of each pixel is output independently, so as to obtain the image of the higher pixel. That is, by the detection element 21 and the controller 20, whether the functional logic 19 combines the pixels output from the pixel array 18 can be automatically controlled according to the detection result of the detection element 21. In another embodiment, the imaging system may further include a control switch 22, where the control switch 22 is connected to the functional logic control unit 19 and is used to manually set the operation mode of the functional logic control unit 19, so that when the ambient light is weak, the control switch 22 is manually turned on, so as to provide a pixel merging instruction to the functional logic unit 19 to merge the pixel signals output by the pixel array 18.

Referring to fig. 5, the present invention further provides a pixel merging method of the above imaging system, which includes the following steps:

in step S11, the pixel value output by the pixel array 18 is read, the pixel array 18 includes a plurality of pixel units 11, and each pixel unit 11 includes four pixels: the first pixel 112, the second pixel 114, the third pixel 116 and the fourth pixel 118, the pixels in each pixel unit 11 are arranged in a form of 2×2, and the first pixel 112 and the fourth pixel 118 are diagonally adjacent, and the second pixel 114 and the third pixel 116 are diagonally adjacent. The pixels also constitute a plurality of pixel unit blocks 13. Specifically, the first pixel 112 is blue B, the fourth pixel 118 is red R, the second pixel 114 is green Gr, and the third pixel 116 is green Gb.

In step S13, a pixel merging instruction is accepted to merge the first pixels 112 in the three adjacent pixel units 11 in the adjacent three columns into one pixel unit block 13, merge the fourth pixels 118 in the three adjacent pixel units 11 in the adjacent three columns into one pixel unit block 13, merge the nearest three second pixels 114 or third pixels 116 in the m-th row, the m+1-th row and the m-1-th row into one pixel unit block 13, and merge the second pixels 114 or third pixels 116 in the m-th row and the middle one pixel 116 in the m-th row into all pixels in the pixel unit block 13, wherein the second pixels 114 or the third pixels 116 in the m+2-th row and the two pixels in the m+1-th row and the two pixels in the m-1-th row are all diagonally adjacent, and merge the second pixels 114 or the third pixels 116 in the m-2-th row and the two pixels in the m+1-th row are all diagonally adjacent, and merge all the pixels in the pixel unit block 13 are all output. Wherein m is a positive integer greater than or equal to 3.

Specifically, in step S13, the color filters of the same color are further provided on all pixels of each of the pixel unit blocks 13. The color filters include a blue filter, a red filter and a green filter, the blue filter is disposed on the first pixel 112, the red filter is disposed on the fourth pixel 118, and the green filter is disposed on the second pixel 114 and the third pixel 116.

The pixel unit blocks 13 composed of the first pixels 112 and the fourth pixels 118 are arranged in the form of 3*3 and are in rectangular distribution. The pixel unit blocks 13 composed of the second pixels 114 and the third pixels 116 are arranged in the form of 3*3 and are distributed in a diamond shape.

The receiving pixel merging instruction may specifically be: receiving an instruction for controlling the switch; or detecting the intensity of the ambient light, and according to a control instruction sent by the intensity of the ambient light, when the ambient light is larger than a preset value, the control instruction is a pixel merging instruction.

Step S15, merging pixel values of the first pixel 112, the second pixel 114, the third pixel 116 and the fourth pixel 118 according to the pixel unit block 13 to output a merged image. The merging of the pixel values of the first, second, third and fourth pixels 112, 114, 116 and 118 in the pixel array according to the pixel unit block 13 is in the form of a weighted average. The weighting coefficients of the weighted averages may be stored in the status register 16, and the weighting coefficients may be the same or different.

In the pixel merging method, due to the merging mode of the pixel unit blocks 13, all pixels participate in merging and are not lost, and the pixel merging is ensured not to be overlapped, so that the definition of an image is ensured, and the signal to noise ratio of the image is improved.

The present invention is not limited to the above embodiments, but is capable of modification and variation in all aspects, including those of ordinary skill in the art, without departing from the spirit and scope of the present invention.

Claims (18)

1. An imaging system, comprising:

a pixel array (18) of a plurality of pixel cells (11), each of said pixel cells (11) comprising four pixels: a first pixel (112), a second pixel (114), a third pixel (116) and a fourth pixel (118), wherein the pixels in each pixel unit (11) are arranged in a form of 2 x 2, the first pixel (112) and the fourth pixel (118) are arranged diagonally adjacent, the second pixel (114) and the third pixel (116) are arranged diagonally adjacent, the first pixel (112) forms a first color pixel, the second pixel (114) and the third pixel (116) forms a second color pixel, and the fourth pixel (118) forms a third color pixel;

wherein the pixels further constitute a plurality of pixel unit blocks (13), and the first pixels (112) in the pixel units (11) of adjacent three rows in adjacent three columns constitute one pixel unit block (13); -said fourth pixels (118) in said pixel cells (11) of adjacent three rows of adjacent three columns constitute one said block of pixel cells (13); the closest three second color pixels in the m-th row, the m+1-th row, and the second color pixel in the m-1-th row that are diagonally adjacent to a middle one of the three second color pixels in the m-th row, the second color pixel in the m+2-th row that are diagonally adjacent to the two pixels in the m+1-th row, and the second color pixel in the m-2-th row that are diagonally adjacent to the two pixels in the m-1-th row constitute one pixel unit block (13), wherein m is a positive integer greater than or equal to 3;

-a control circuit (15) coupled to the pixel array (18) to control pixel output pixel values in the pixel array (18);

-a readout circuit (17) coupled to the pixel array (18) to read pixel values output by the pixel array (18); and

-a functional logic control unit (19) coupled to the readout circuit (17) for merging the pixel values of the first pixel (112), the second pixel (114), the third pixel (116) and the fourth pixel (118) in the pixel array (18) according to the pixel cell block (13), respectively, and outputting a merged image.

2. The imaging system according to claim 1, wherein the pixel cell blocks (13) of the first pixels (112) and the fourth pixels (118) are arranged in the form of 3*3 and exhibit a rectangular distribution.

3. The imaging system according to claim 1, wherein the pixel cell blocks (13) of the second pixels (114), the third pixels (116) are arranged in the form of 3*3 and are in a diamond-shaped distribution.

4. The imaging system according to claim 1, wherein the functional logic control unit (19) combines pixel values of the first pixel (112), the second pixel (114), the third pixel (116) and the fourth pixel (118) in the pixel array according to the pixel unit block (13) in the form of weighted averages.

5. The imaging system according to claim 1, further comprising a color filter, said color filters of the same color being provided on all pixels of each of said pixel unit blocks (13).

6. The imaging system of claim 5, wherein the color filters include a blue filter disposed on the first pixel (112), a red filter disposed on the fourth pixel (118), and a green filter disposed on the second pixel (114) and the third pixel (116).

7. The imaging system of claim 4, further comprising a status register (16) coupled to the control circuit (15) and the readout circuit (17) to configure a read mode of the readout circuit (17) and a weighting factor for the pixel value combination.

8. The imaging system according to claim 7, wherein the pixel array includes a normal operation mode in which the output signal of each pixel is separately output to obtain an image of a higher pixel, and a pixel combination mode in which the output signals of all pixels in each of the pixel unit blocks (13) are combined to obtain an image of a lower pixel.

9. The imaging system of claim 7, further comprising a controller coupled to the control circuit (15) and the readout circuit (17), the controller for controlling the pixel array to operate in a normal operating mode or a binning mode.

10. The imaging system of claim 9, further comprising a detection element for detecting intensity of ambient light, the detection element being connected to the controller, the controller being configured to control an operation mode of the pixel array according to a detection result of the detection element.

11. The imaging system of claim 9, further comprising a control switch for controlling the pixel array to operate in the normal operating mode or the binning mode, the control switch being connected to the controller.

12. A pixel merging method, comprising:

-reading pixel values output by a pixel array (18), said pixel array comprising a plurality of pixel cells (11), each of said pixel cells (11) comprising four pixels: a first pixel (112), a second pixel (114), a third pixel (116) and a fourth pixel (118), wherein the pixels in each pixel unit (11) are arranged in a form of 2 x 2, the first pixel (112) and the fourth pixel (118) are arranged in a diagonally adjacent manner, the second pixel (114) and the third pixel (116) are arranged in a diagonally adjacent manner, the pixels further form a plurality of pixel unit blocks (13), the first pixel (112) forms a first color pixel, the second pixel (114) and the third pixel (116) forms a second color pixel, and the fourth pixel (118) forms a third color pixel;

receiving a pixel merging instruction, merging the first pixels (112) in the pixel units (11) of three adjacent rows in the adjacent three columns into one pixel unit block (13), merging the fourth pixels (118) in the pixel units (11) of three adjacent rows in the adjacent three columns into one pixel unit block (13), merging the nearest three second color pixels in the m-th row, the m+1th row and the middle one of the third second color pixels in the m-1 th row into one pixel unit block (13), merging the second color pixels diagonally adjacent to the two pixels in the m+1th row and the two pixels diagonally adjacent to the m-1 th row in the m-2 th row into one pixel unit block (13),

and merging pixel values of the first pixel (112), the second pixel (114), the third pixel (116) and the fourth pixel (118) in the pixel array (18) according to the pixel unit block (13) and then outputting a merged image.

13. A pixel merging method according to claim 12, characterized in that the pixel cell blocks (13) consisting of the first pixels (112) and the fourth pixels (118) are arranged in the form of 3*3 and present a rectangular distribution; the pixel unit blocks (13) composed of the second pixels (114) and the third pixels (116) are arranged in a 3*3 mode and are distributed in a diamond shape.

14. The pixel merging method according to claim 12, wherein the merging of the pixel values of the first pixel (112), the second pixel (114), the third pixel (116) and the fourth pixel (118) in the pixel array according to the pixel unit block (13) is merging in the form of a weighted average.

15. A pixel combination method according to claim 12, further providing color filters of the same color on all pixels of each of said pixel cell blocks (13).

16. The pixel merging method according to claim 15, wherein the color filters include a blue filter, a red filter and a green filter, the blue filter being disposed on the first pixel (112), the red filter being disposed on the fourth pixel (118), the green filter being disposed on the second pixel (114) and the third pixel (116).

17. The method of merging pixels as claimed in claim 12, wherein the receiving a pixel merging instruction specifically includes: and receiving an instruction for controlling the switch.

18. The method of merging pixels as claimed in claim 12, wherein the receiving a pixel merging instruction specifically includes: detecting the intensity of ambient light, and sending out a control instruction according to the intensity of the ambient light, wherein the control instruction is the pixel merging instruction when the ambient light is larger than a preset value.