Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the specific details need not be employed to practice the present invention. In other instances, well-known materials or methods have not been described in detail in order to avoid obscuring the present invention.
Reference throughout this specification to "one embodiment," "an embodiment," "one example," or "an example" means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment," "in an embodiment," "one example" or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable combinations and/or subcombinations in one or more embodiments or examples. The particular features, structures, or characteristics may be included in integrated circuits, electronic circuits, combinational logic circuits, or other suitable components that provide the desired functionality. Additionally, it should be appreciated that the drawings provided herewith are for explanation purposes to persons skilled in the art and that the drawings are not necessarily drawn to scale.
An example method and apparatus for reading image data from an image sensor is described according to examples of the teachings of this disclosure. Generally, image data of an entire line is read out from the CMOS image sensor line by line. As will be discussed, according to the teachings of this disclosure, a multi-phase transfer channel coupled to a CMOS pixel cell enables image data to be output from a variable number of pixel cells during one or more readout cycles from a column of an image sensor. In other words, less than an entire row of pixels may be output during one clock cycle, where the pixels output for each column may be selected to be stored or ignored. In one example, the number of pixel cells read out from each column in one clock cycle may be independent of the number of pixel cells read out from other columns in accordance with the teachings of this disclosure.
For example, as will be discussed in more detail below, an example image sensing system includes an array of pixel cells arranged in a plurality of rows and a plurality of columns. A plurality of multi-phase transfer channels are coupled to the pixel cell array such that each of the plurality of multi-phase transfer channels is coupled to a respective one of the plurality of columns of the pixel cell array. Readout circuitry is coupled to the plurality of multiphase transfer channels to readout image data from the array of pixel cells. The column control circuits are coupled to individually generate a plurality of sets of multi-phase transfer signals that are coupled to transistors in corresponding multi-phase transfer channels. According to the teachings of this disclosure, image data from a variable number of pixel cells of a selected column is transferred along a multi-phase transfer channel transistor-to-transistor in response to the multi-phase signal and output to a readout circuit.
As will be shown, in one example, a multi-phase transfer signal may be generated separately for each column. During one clock cycle, one or more columns may be selected to output one pixel charge in response to one cycle of the multi-phase transfer signal. Columns in which the multi-phase transfer signal is not asserted will not output pixel charge.
To illustrate, FIG. 1 is a diagram showing a portion of an example image sensing system 100, according to the teachings of this disclosure. As shown in the depicted example, the image sensing system 100 includes a pixel array 104 that includes a plurality of pixels 106 arranged in a plurality of rows and a plurality of columns. In the depicted example, the column control circuitry 102 is coupled to separately generate a plurality of sets of multi-phase transfer signals 112A, 112B, …, 112N that are coupled to be received by the pixel array 104 in one example. In one example, the multi-phase transfer channels are also coupled to receive corresponding sets of multi-phase transfer signals 112A, 112B, …, 112N to output image data from pixel cells 106 of pixel array 104 to readout circuitry 108. As shown in the depicted example, the readout circuitry 108 includes a plurality of column readout circuits 108A, 108B, …, 108N coupled to readout image data output from a respective one of a plurality of columns of the pixel array 104. In the illustrated example, the output generated by readout circuitry 108 is image data 110, as shown.
As shown in the example depicted in fig. 1, the column control circuitry 102 is coupled to separately generate a plurality of sets of multiphase transfer signals 112A, 112B, …, 112N in response to the control signals 114. In one example, control signal 114 may be received from an external control circuit separate from column control circuit 102. In one example, each set of multi-phase transfer signals 112A, 112B, …, 112N is coupled to independently control the output of image data from each column of the pixel array 104. As such, image data from a variable number of pixel cells 106 may be output from each selected column of the pixel array 104 in accordance with the teachings of this disclosure. In other words, the variable number of pixel cells 106 that may be serially output from each column may be independent of different variable numbers of pixel cells 106 that may be serially output from other columns in accordance with the teachings of this disclosure.
It should be appreciated that image data can be serially output from a different variable number of pixel cells of each column of pixel array 104 independently, with the flexibility of being able to control the output of image data from each column separately or independently of the other columns. Thus, memory savings may be realized because if only a portion of a row or column of image data is needed at any particular time, then all rows or all columns of image data need not be output at once.
For example, a distorted image taken with a fisheye lens has a region in which no image data exists toward the corners of the pixel array 104. Thus, according to the teachings of this disclosure, image data of pixels toward corners of pixel array 104 may be output and ignored, and performing fisheye lens distortion correction using image sensing system 100 may require less memory.
Similarly, it should be appreciated that when performing, for example, JPEG compression of raw image data read out of the pixel array 104, the image data may be read out of the pixel array 104 indirectly in the form of, for example, 8 x 8 macroblocks or the like, without the limitation that an entire row or column of image data must be read out of the pixel array 104 at once. As such, JPEG compression may be accomplished with less memory according to the teachings of this disclosure.
FIG. 2 is a diagram showing another example of a portion of an image sensing system 200 according to the teachings of this disclosure. As shown in the depicted example, the image sensing system 200 includes a plurality of pixels 206 arranged in a plurality of rows and a plurality of columns. In the depicted example, the imaging system 200 also includes a plurality of multi-phase transfer channels 216A, 216B, …, 216N. Each of the plurality of multiphase transfer channels 216A, 216B, …, 216N is coupled to a respective column of pixels 206. Each of the plurality of multi-phase transfer channels 216A, 216B, …, 216N is also coupled to receive a respective set of multi-phase transfer signals 212A, 212B, …, 212N. Additionally, the example in fig. 2 illustrates that, in one example, there is a readout circuit including a plurality of buffer circuits 208A, 208B, …, 208N, each of which is coupled to a respective one of a plurality of multiphase transfer channels 216A, 216B, …, 216N. In one example, image data 210 is output from a plurality of buffer circuits 208A, 208B, …, 208N, as shown. In one example, the plurality of buffer circuits 208A, 208B, …, 208N includes amplifier circuits coupled to amplify signals responsive to image charge transferred from the plurality of buffer circuits 208A, 208B, …, 208N to generate the image data 210.
Figure 3 is a schematic diagram showing increased detail of an example pixel cell 306 coupled to a portion of a multi-phase transfer channel 316 according to the teachings of this disclosure. In one example, it is to be appreciated that pixel cell 306 is one of the examples of pixel cell 106 of fig. 1 or pixel cell 206 of fig. 2. Similarly, it should be appreciated that the portion of the multi-phase transfer channel 316 illustrated in fig. 3 is one example of a portion of any of the plurality of multi-phase transfer channels 216A, 216B, …, 216N of fig. 2, and that the multi-phase transfer signals Va312A, Vb312B, and Vc312C are examples included in the set of multi-phase transfer signals 112A, 112B, and 112C of fig. 1 or in the set of multi-phase transfer signals 212A, 212B, and 212C of fig. 2. It is to be understood, therefore, that similarly named and numbered elements referred to below are coupled and function as described above.
Referring back to the example depicted in fig. 3, pixel cell 306 includes a photosensitive element 320 coupled to a global shutter transistor 322 and a transfer transistor 324. The multi-phase transfer channel 316 includes a plurality of transistors coupled in series, including transistors 318A, 318B, and 318C, which are part of the plurality of transistors of the multi-phase transfer channel 316. In the depicted example, transistors 318A, 318B, and 318C are coupled to be controlled in response to multi-phase transfer signals Va312A, Vb312B, and Vc312C, respectively, as shown.
In operation, light 334 representing a portion of an image is directed onto pixel cell 306. In one example, light 334 may be focused from a lens optically coupled to receive light 334 directed from an object being imaged with an image sensor, according to the teachings of this disclosure. In the example, the global shutter transistor 322 is coupled to receive a global shutter signal GS326 that, when activated, causes the photosensitive element 320 to accumulate image charge in response to the light 334.
Assuming now that image data from an image sensor including the pixel cell 306 is to be output, the transfer transistor 324 is activated in response to the transfer signal Va312A, which causes the image charge accumulated in the photosensitive element 320 to be transferred through the transistor 324 to the transistor 318A of the multi-phase transfer channel 316, as shown in fig. 3. As mentioned above, in one example, transfer signal Va312A is one of a plurality of multi-phase transfer signals 312A, 312B, and 312C coupled to be received by a plurality of transistors 318A, 318B, and 318C, respectively, of multi-phase transfer channel 316, as shown.
As will be discussed in further detail, the plurality of multi-phase transfer signals 312A, 312B, and 312C are out of phase with one another. As such, the plurality of transistors 318A, 318B, and 318C of the multi-phase transfer channel 316 are controlled in response to the plurality of multi-phase transfer signals 312A, 312B, and 312C to transfer the accumulated image charge through the multi-phase transfer channel 316 transistor-to-transistor along the plurality of transistors 318A, 318B, and 318C coupled in series, according to the teachings of this disclosure. For example, the image charge stored in transistor 318A is transistor-to-transistor transferred to transistor 318B in response to the multi-phase transfer signals 312A and 312B. In one example, the image charge may be stored in a bit well under the gate of the respective transistor. The image charge stored in transistor 318C is transistor-to-transistor transferred to transistor 318A in response to the multi-phase transfer signals 312C and 312A. The image charge stored in transistor 318B is transistor-to-transistor transferred to the corresponding transistor of the next row of the pixel array in response to the multi-phase transfer signals 312B and 312C. Similarly, image charge stored in the corresponding transistor of the previous row of the pixel array is transferred to transistor 318C in response to the multi-phase transfer signals 312C and 312B.
In one example, accumulated image charge from each pixel cell 306 of a column of the image sensor may be transferred through the multi-phase transfer channel 316 for each complete cycle of the multi-phase transfer signal. Thus, in one example, a variable number of cycles of the multi-phase transfer signal may be applied to each of the plurality of transistors of the multi-phase transfer channel 316 to output the accumulated image charge from a variable number of pixel cells 306 of each column in accordance with the teachings of this disclosure. In one example, the variable number may be greater than or equal to zero. In one example, the variable number of pixel cells serially output from one column may be independent of the variable number of pixel cells serially output from another column in accordance with the teachings of this disclosure.
FIG. 4 is a timing diagram showing an example multi-phase transfer signal and a global shutter signal, according to the teachings of this disclosure. In one example, it is appreciated that the multi-phase transfer signals Va412A, Vb412B, and Vc412C are examples of the set of multi-phase transfer signals 112A, 112B, and 112C of fig. 1 or the set of multi-phase transfer signals 212A, 212B, and 212C of fig. 2 or the multi-phase transfer signals Va312A, Vb312B, and Vc312C of fig. 3, and the global shutter signal GS426 is an example of the global shutter signal GS326 of fig. 3. It is to be understood, therefore, that similarly named and numbered elements referred to below are coupled and function as described above.
Referring now back to the timing diagram example depicted in FIG. 4, the global shutter signal GS426 is activated between times t0 and t 1. At this point, the accumulation of image charge begins in the photosensitive element (e.g., photosensitive element 320 of FIG. 3). At time t2, the multi-phase transfer signal Va412A is set to 3V, which in the depicted example, results in the transfer of image charge that has accumulated in the photosensitive element 320 to the transistor 318A of the multi-phase transfer channel 316 through the transfer transistor 324. In the example, note that the multi-phase transfer signal Va412A is set to 3V for all pixel cells in the pixel cell array, so that the accumulated image charge of all pixel cells in the pixel array is transferred to the corresponding transistors in the corresponding multi-phase transfer channels.
As can be observed in the example of fig. 4, the multi-phase transfer signals Va412A, Vb412B, and Vc412C are a plurality of control signals that are out of phase with one another and that are coupled to control a plurality of transistors coupled together in a multi-phase transfer channel, such as the multi-phase transfer channel 316 of fig. 3. To illustrate, at time t3, the multi-phase transfer signal Vb412B is activated at a time before the multi-phase transfer signal Va412A is deactivated at time t 4. Thus, during the time between t3 and t4, the accumulated image charge previously transferred to transistor 318A is transferred transistor-to-transistor from transistor 318A to transistor 318B. Similarly, at time t5, the multi-phase transfer signal Vc412C is activated at a time before the multi-phase transfer signal Vb412B is deactivated at time t 6. Thus, during the time between t5 and t6, the accumulated image charge previously transferred to transistor 318B is transistor-to-transistor transferred from transistor 318B to the next transistor coupled to transistor 318B, which in the example is controlled in response to the multi-phase transfer signal Vc 412C. In particular, referring back to the example of fig. 3, it should be appreciated that there is another transistor serially coupled to the transistor 318C that is controlled in response to the multi-phase transfer signal Vb312B, and there is another transistor serially coupled to the transistor 318B that is controlled in response to the multi-phase transfer signal Vc 312C.
Referring back to the example shown in fig. 4, the pattern of multi-phase transfer signals Va412A, Vb412B, and Vc412C similarly continues at and after times t7, t8, t9, t10 as described above, which results in transistor-to-transistor transfer of accumulated image charge along a multi-phase transfer channel, such as multi-phase transfer channel 316 of fig. 3. In one example, the mode continues for a variable number of clock cycles until a variable number of accumulated image charges are output from the multiphase transfer channel to a corresponding readout circuit, according to the teachings of this disclosure.
FIG. 5A is a schematic diagram showing another example pixel cell 506 coupled to a portion of a multi-phase transfer channel 516, according to the teachings of this disclosure. It should be appreciated that the portions of pixel cell 506 and multi-phase transfer channel 516 illustrated in FIG. 5A share many similarities with the portions of pixel cell 306 and multi-phase transfer channel 316 illustrated in FIG. 3. It is to be understood, therefore, that similarly named and numbered elements referred to below are coupled and function as described above.
One difference between the pixel cell 506 of fig. 5A and the pixel cell 306 of fig. 3 is that the pixel cell 506 of fig. 5A includes a transfer transistor 524 coupled between the photosensitive element 520 and the transistor 518A of the multi-phase transfer channel 516, the transfer transistor 524 being controlled in response to a transfer signal Tx528 instead of the multi-phase transfer signal Va 512A. Thus, in the example depicted in fig. 5A, the transfer of accumulated image charge from the photosensitive element 520 to the transistor 518A of the multi-phase transfer channel 516 may be independent of the multi-phase transfer signal Va 512A.
Figure 5B is a schematic diagram showing yet another example pixel cell 506 coupled to a multi-phase transfer channel 516 according to the teachings of this disclosure. It should be appreciated that the portions of the pixel cell 506 and the multi-phase transfer channel 516 illustrated in FIG. 5B share many similarities with the portions of the pixel cell 506 and the multi-phase transfer channel 516 illustrated in FIG. 5A and with the portions of the pixel cell 306 and the multi-phase transfer channel 316 illustrated in FIG. 3. It is to be understood, therefore, that similarly named and numbered elements referred to below are coupled and function as described above.
One difference between the pixel cell 506 of fig. 5B and the pixel cell 506 of fig. 5A is that the pixel cell 506 of fig. 5B includes an additional output transistor 530 coupled between the transfer transistor 524 and the transistor 518A of the multi-phase transfer channel 516. In the example depicted in fig. 5B, the output transistor 530 is controlled in response to the output gate signal OG 532. Thus, in the example depicted in fig. 5B, the transfer of accumulated image charge from the pixel cell 506 to the transistor 518A of the multi-phase transfer channel 516 is further performed in response to activation of the output gate signal OG 532.
Figure 5C is a schematic diagram showing yet another example pixel cell 506 coupled to a portion of a multi-phase transfer channel 516, according to the teachings of this disclosure. It should be appreciated that the portions of the pixel cell 506 and the multi-phase transfer channel 516 illustrated in FIG. 5C share many similarities with the portions of the pixel cell 506 and the multi-phase transfer channel 516 illustrated in FIG. 5A. It is to be understood, therefore, that similarly named and numbered elements referred to below are coupled and function as described above.
One difference between the pixel cell 506 of fig. 5C and the pixel cell 506 of fig. 5A is that the pixel cell 506 of fig. 5C includes a plurality of photosensitive elements 520A and 520B and a plurality of transfer transistors 524A and 524B, as shown. Thus, in the illustrated example, it should be appreciated that each of the plurality of photosensitive elements 520A and 520B is included in a shared architecture, with both photosensitive elements 520A and 520B coupled to the transistor 518A of the multi-phase transfer channel 516 through transfer transistors 524A and 524B, respectively. In the depicted example, transfer transistors 524A and 524B are controlled in response to transfer signals Tx1528A and Tx2528B, respectively. In one example, multiple photosensitive elements 520A and 520B may be utilized in combination, for example, to achieve an architecture of pixel cell 506 with increased sensitivity and/or increased dynamic range in accordance with the teachings of this disclosure.
FIG. 6 is a flow diagram showing an example process 650 of reading image data from an image sensor according to the teachings of this disclosure. It should be appreciated that the process illustrated in fig. 6 may refer to the structure and/or function described above with respect to fig. 1-5C. It is to be understood, therefore, that similarly named and numbered elements referred to below are coupled and function as described above.
As shown in the depicted example, the process 650 begins with process block 652, at which time the shutter is activated. In one example, the shutter may be a global shutter. Process block 654 shows that image charge is then accumulated in the pixel cells of the image sensor. In one example, the pixel cells may be included in a pixel array arranged in a plurality of rows and a plurality of columns. Process block 656 shows then transferring the accumulated image charge to a multi-phase transfer channel.
Process block 658 shows generating a multi-phase transfer signal. Process block 660 shows that one or more columns may then be selected. In one example, the selected column may be selected independently of previously selected columns. In another example, the selected column may be the next column in a sequence of columns. Process block 662 shows that the accumulated image charge for a variable number of pixel cells of the selected column can then be output. In one example, accumulated image charge for a variable number of pixel cells is serially output from a selected column in response to a multi-phase transfer signal. In one example, the accumulated image charges are serially output by transistor-to-transistor transfer along a multi-phase transfer channel until a variable number of image charges are output to a readout circuit in response to a multi-phase transfer signal. Process block 664 shows reading out image data from the readout circuitry.
Decision block 665 shows that if there is more image data to read, processing loops back to process block 660 so that another one or more columns may be selected. As previously mentioned, the next selected column may be selected independently of the previously selected column. In another example, the selected column may be the next column in a sequence of columns. Additionally, it should be appreciated that the variable number of pixel cells output from the next selected column in processing block 662 may be the same as the previous variable number of pixel cells output from the previously selected column. In another example, it should be appreciated that the variable number of pixel cells output from the next selected column in processing block 662 may be different than the previous variable number of pixel cells output from the previously selected column.
Of course, it is to be understood that the order in which the process blocks are described above may not necessarily occur in the same order as described and that some of the process blocks may occur concurrently, in accordance with the teachings of this disclosure.
Methods of reading image data from an image sensor according to the teachings of this disclosure may be used in a variety of applications including image processing applications, such as image rotation. To illustrate, fig. 7A is a diagram showing an example of a plurality of pixel cells 700. The pixel cells 700 represent a portion of the pixel array in the image sensing system 100 of FIG. 1 and are arranged in columns C1-C4 and rows R1-R4. During the first full cycle of the multi-phase transfer signal, which may also be referred to as readout cycle T1, the pixels at column C1 and row R4(C1, R4) are readout. During the second readout cycle T2, the pixels at (C1, R3) and (C2, R4) are readout. During the third readout cycle T3, the pixels at (C1, R2), (C2, R3), and (C3, R4) are read out. Similarly, pixels in subsequent readout cycles are read out along a diagonal across the pixel array. Pixels may be output from the image sensing system such that the resulting image is angled relative to the image as seen in fig. 7B. According to the teachings of this disclosure, by using this method of reading image data from an image sensor, fewer image processing steps and/or less memory may be required to acquire a rotated resulting image.
As mentioned previously, the method of reading image data from an image sensor as described in the present invention is applicable to applications in which fisheye distortion is corrected. FIG. 8A is a diagram showing a simplified example pixel array 800 including 24 pixels arranged in columns C1-C5 and rows R1-R4. If a fish-eye distorted image is captured in pixel array 800, some pixels in pixel array 800 will not contain image information that can be read out of the pixel array and are discarded. Such pixels are represented by shading in fig. 8A. During the first readout cycle T1, pixels at column 1, row 4(C1, R5), and (C6, R5) are read out of the pixel array 800, and image data from these particular pixels may be output from the pixel array 800 and discarded because these particular pixels contain no image information due to fisheye lens distortion. Image data output from the pixel array 800 during subsequent second and third readout cycles T2 and T3, respectively, is output from the pixel array 800 and discarded because these particular pixels do not contain image information, as shown in fig. 8A. Image data from pixels (C1, R2), (C2, R3), (C3, R4), (C4, R4), (C5, R3), and (C6, R2) are not read from the pixel array 800 until the fourth (and subsequent) readout cycle T4 and are output in the resulting image, as shown in fig. 8B, which shows the results of readout cycles T4 and T5. Indeed, as shown in fig. 8B, the fisheye lens distortion seen in the example shown in fig. 8A is corrected in fig. 8B by reading out the image data according to the teachings of this disclosure.
The above description of illustrated examples of the invention, including what is described in the summary, is not intended to be exhaustive or to be limited to the precise forms disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various equivalent modifications can be made without departing from the broader spirit and scope of the invention.
These modifications can be made to the examples of the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification and the claims. Rather, the scope is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.