WO1993014595A1 - Method for and apparatus for increasing the dynamic range of optical sensors - Google Patents

Abstract

A method and apparatus for increasing the dynamic range of an optical sensor (10) by generating reset enable pulses (RSTE) at time intervals of shorter duration than that of the regular reset pulses, a control signal (CS) when a predetermined characteristic of the sensor output exceeds a reference value, and a conditional reset pulse (RST) when there is coincidence between a reset enable pulse (RSTE) and a control signal (CS). The integration of the sensor (10) is started at the time of generation of either a regular reset pulse (RST) and is terminated at the time of generation of a sample pulse. The output of the sensor (10) is multiplied by a scaling factor representing the ratio of the full integration time to the effective integration time between the starting and termination times.

Description

METHOD FOR AND APPARATUS FOR INCREASING THE DYNAMIC RANGE OF OPTICAL SENSORS

The present invention relates to a method and apparatus for increasing the dynamic range of optical sensors, e.g., for receiving viewable light or infrared thermal radiation. The invention is particularly useful for increasing the dynamic range of video cameras or other optical sensors including a matrix or array of photo-sensitive elements and is therefore described below particularly with respect to such applications.

A known problem that exists today in optical sensors is the limited dynamic range of the photo-sensitive elements. Thus, the photo-sensitive elements commonly have a dynamic range of two to three orders of magnitude, while the range of illumination levels that they may be exposed to may be as high as eight or more orders of magnitude. For example, if the optical sensor is to sense a person in a darkened room standing near a lit window, the details outside the window which are at a high illumination level cannot be perceived together with the details of the person's face which are at a much lower illumination level. Under these conditions, the optical sensors encounter such problems as "blooming" .

In recent years, many solutions have been proposed for increasing the dynamic range of optical sensors. Some of the proposed solutions are based on compressing the information or clipping it, but both of these solutions result in information being lost. Other solutions have been proposed for controlling the integration time in an adaptive manner, but such solutions generally require processing operations which take one or more time frames to perform, thereby resulting in a delay or lag, and/or require very expensive hardware to implement.

The present invention provides a novel method and apparatus for increasing the dynamic range of an optical sensor to overcome the above problems . According to one aspect of the present invention, there is provided a method of increasing the dynamic range of an optical sensor by controlling the integration time during which the optical sensor integrates the light received thereon and produces an electrical output corresponding thereto, comprising: generating a series of regular reset pulses separated by prefixed time intervals of equal duration; generating a series of sample pulses each at a predetermined time interval after the generation of a regular reset pulse, which predetermined time interval represents a full integration time; generating a series of reset enable pulses at time intervals, between pulses, of shorter duration than that of the regular reset pulses; generating a control signal whenever a predetermined characteristic of the electrical output of the optical sensor exceeds a reference value; generating a conditional reset pulse whenever there is coincidence between a reset enable pulse and a control signal; starting the integration of the optical sensor at the time of generation of either a regular reset pulse or a conditional reset pulse; terminating the integration of the optical sensor at the time of generation of a sample pulse; determining the scaling factor, representing the ratio of the full integration time to the effective integration time between said starting and termination times; and multiplying the electrical output of the optical sensor by said scaling factor.

According to further features in the described preferred embodiments, the effective integration time is determined by counting the number of conditional reset pulses between the generation of a regular reset pulse and a sample pulse.

According to yet further features in the described preferred embodiments, the reset enable pulses are generated at prefixed, but unequal time periods. Preferably, the time intervals to the next sample pulse decrease according to a downgoing geometric series . The invention is particularly useful with respect to optical sensors, such as video cameras, which include a rectangular matrix or other array of photo-sensitive elements. When so used, it is preferable that the integration times of a block of photo-sensitive elements be controlled simultaneously by the control signal, although it is conceivable that each photo-sensitive elements can be individually controlled.

As will be more particularly described below, such a method can increase the dynamic range of an optical sensor in a real time manner so as to substantially reduce the above-mentioned "blooming" problem.

The invention also provides apparatus operating in accordance with the above method. Fig. 1 illustrates one form of optical sensor whose dynamic range may be increased in accordance with the present invention;

Fig. 2 illustrates a conventional (prior art) circuitry for normally controlling the integration time of each photo-sensitive element in a matrix-type optical sensor;

Fig. 3 is a timing diagram illustrating the conventional (prior art) manner of controlling integration times according to Fig. 2; Fig. 4 illustrates one form of control circuit in accordance with the present invention which may be used for controlling the integration time of the optical sensor illustrated in Fig. 1;

Figs. 4a, 4b and 4c illustrate examples of preferred ways of deriving the control signal in the control circuit of Fig. 4;

Fig. 5 is a timing diagram illustrating one example for operating the control signals of Fig. 4;

Fig. 6 schematically illustrates an example of a circuit for producing an output corresponding to the increased dynamic range of the optical sensor; Fig. 7 illustrates one form of digital circuit which may be used for determining the effective integration time in the novel method; and

Fig. 8 illustrates one form of analog circuit that may be used for determining the effective integration time in the novel method.

With reference first to Fig. 1, there is illustrated an optical sensor, generally designated 10, including a rectangular matrix of photo-sensitive elements 11, 12, 13, etc., as may be found in a conventional video camera. Each of the photo-sensitive elements 11-13 outputs an electrical signal corresponding to the amount of light received thereby in a predetermined time period, called the integration time. Figs. 2 and 3 illustrate the manner of determining the integration time in a conventional system. Thus, as shown in Fig. 2, each photo-sensitive element, therein designated 21 , is controlled by a transistor 23 receiving a series of regular reset pulses RST via line 22, separated by predetermined time intervals of equal duration. A second transistor 24 is supplied with a series of sample pulses SMPL at a predetermined interval after the generation of a regular reset pulse RST. Thus, as shown in Fig. 3, the normal integration time in a conventional video camera is the time from the end of a reset pulse RST to be the beginning of the sample pulse SMPL. This is called the full integration time of the photo-sensitive elements.

The circuit illustrated in Fig. 2 may include a storage element 26. This storage element, however, can be omitted when the readout is directly to the video line via readout circuit 27, for example.

In a conventional video camera, the integration time is the same for all the photo-sensitive elements. As pointed out earlier, however, when the illumination is high at a certain part of the picture, the integration time would be too long as the sensor would enter into saturation. On the other hand, in other parts of the picture where the illumination is weak, the integration time would be too low, producing a low output voltage such that the background noise might mask the signal.

According to the present invention, the integration time of the photo-sensitive elements are not necessarily the full integration time as described above and as illustrated in Figs. 2 and 3, but rather may be shortenend, to produce a shorter effective integration time, according to the amount of light received by the photo¬ sensitive elements at that particular instance. In the example illustrated in Fig. 1, the photo-sensitive elements are arranged in blocks, as shown at 15, each block including a plurality of such elements. Each block further includes a control circuit, schematically shown at 17, for controlling the effective integration time for the respective block. While Fig. 1 illustrates each block as comprising a plurality of photo-sensitive elements, which is generally preferred, it is contemplated that a block may consist of a single photo-sensitive element so that the photo-sensitive elements are controlled individually rather than in blocks.

The effective integration time for each block of sensors is determined and is used for computing the outputs of the photo-sensitive elements as appearing in the output line 18, Fig. 1. The electrical output appearing on output line 18 is subsequently multiplied by a scaling factor, e.g., as illustrated in Fig. 6 to be described below, so that it will more closely correspond to the amount of light actually received by the photo-sensitive elements in the respective frame. In this manner, the dynamic range of the photo-sensitive elements is effectively increased.

One example of a control circuit 17 (Fig. 1) that may be used for increasing the dynamic range of the optical sensor is illustrated in Fig. 4. Such a circuit also includes, as in the conventional system, means for generating a series of regular reset pulses RST separated by prefixed time intervals of equal duration, and a series of sample pulses SMPL, each at a predetermined time interval after the generation of a regular reset pulse RST. As described above with reference to Figs. 2 and 3, a full integration time is equal to the time from the end of the reset pulse RST to the beginning of the sample pulse SMPL. However, the control circuit illustrated in Fig. 4 may produce an "effective" integration time which is shorter than the full integration time defined by the regular reset pulses RST and the sample pulses SMPL.

For this purpose, the circuit illustrated in Fig. 4 further include means for generating a control signal CS corresponding to a predetermined characteristic of the optical sensor. This control signal CS may be derived from one of the photo-sensitive elements within the respective block 15, e.g., the centermost element as shown in Fig. 4a. Another alternative is to have the control signal derived from the weighted average illumination level of photo¬ sensitive elements in the respective block as shown in Fig. 4b. A further alternative is to derive the control signal CS in the manner illustrated in Fig. 4c, namely from a weighted average, produced by circuit 20, in the illumination levels of the surrounding area, e.g., blocks 15a-15d, including the controlled block 15.

This control signal CS is applied via lead 32, together with a reference signal Vref applied via lead 33, to a comparator 34, which produces an output whenever the control signal CS exceeds the reference signal Vref.

The circuit illustrated in Fig. 4 further includes means for generating a series of reset enable pulses RSTE at time intervals, between pulses, of shorter duration than that of the regular reset pulses RST. Fig. 5 illustrates a preferred example of the timing of the reset enable pulses RSTE as compared to the regular reset pulses RST and the sample pulses SMPL. In the example illustrated in Fig. 5, the reset enable pulses RSTE are generated at prefixed, but unequal, time periods decreasing according to the following downgoing geometric series: T/X, T/X , T/X , T/Xⁿ wherein T is the full integration time (as determined by the ending by the regular reset pulse RST at the beginning of the sample pulse SMPL), and "X " is a preselected constant, for example, "X" may equal "2".

The reset enable pulses RSTE are applied via lead 35 to a transistor 39, such that when an output pulse is produced from the comparator 34 in coincidence with a reset enable pulse RSTE, the transistor 39 produces a conditional reset pulse CRST. The so produced conditional reset pulse CRST is applied via lead 36 to an OR-gate 38 together with the regular reset pulse RST applied via lead 37, so that gate 38 produces an output on line 38A upon the occurrence of either pulse.

The output from OR-gate 38 is applied to a transistor 40 for each photo-sensitive element in the block 15 to start the integration time of the respective photo¬ sensitive elements. The sample pulses SMPL are applied via lead 41 to another transistor 42 to terminate the integration time for the respective photo-sensitive elements. The voltages produced by the photo-sensitive elements during this integration time are read out via readout circuit 43 to the output line 18. It will thus be seen that for each photo-sensitive element in the respective block 15, a conditional reset pulse CRST will be generated whenever the characteristic (e.g., illumination level) represented by control signal CS exceeds a reference (represented by a signal Vref) during the time of the generation of a reset enable pulse RSTE. It will also be seen that the integration time will restart upon the generation of either a conditional reset pulse CRST or a regular reset pulse RST. Thus, for those photo¬ sensitive elements which receive relatively low levels of illumination, the full integration time will be provided by the regular reset pulses RST; however, for those photo¬ sensitive elements which receive relatively high levels of illumination, the integration time will be shortened by the conditional reset pulses CRST so that those elements do not go into saturation.

The electrical output of the photo-sensitive elements is then multiplied by a scaling factor representing the ratio of the full integration time to the effective — o —

integration time, to produce an electrical output which more closely corresponds to the amount of illumination actually received by the photo-sensitive elements.

The latter operation is schematically shown in Fig. 6, wherein it will be seen that the video output from the photo-sensitive elements in the matrix are applied via output line 18 to an analog-to-digital converter 50 and converted to digital values before being applied to a multiplier 52. The multiplier 52 also receives, via input lead 54, the effective integration time for the respective photo-sensitive elements, determines the scaling factor, i.e., the ratio of the full integration time to the effective integration time, and multiplies the value outputted by the photo-sensitive elements by this scaling factor. For example, if the effective integration time was one-fourth the full integration time, the scaling factor would be "4", and the value outputted by the respective photo-sensitive elements would therefore be multiplied by "4". Thus, the output on line 56 more closely corresponds to the amount of illumination actually received by the respective photo-sensitive elements.

The effective integration time may be determined by counting the number of conditional reset pulses CRST. Fig. 7 illustrates a digital circuit that may be used for this purpose, including a counter having flip-flops 71 , 72 constituting a two-bit counter 75, and two further flip-flops 73, 74 constituting a two-bit register 76, for storing the number of conditional reset pulses CRST.

Fig. 8 illustrates an analog circuit that may be used for this purpose, including a capacitor 81 which is charged each time a regular reset pulse RST is generated, and is discharged each time a conditional reset pulse CRST is generated, such that the charge in capactitor 81 will be proportional to the number of conditional resets. This value is stored in the capacitor 82.

While the invention has been described with respect to several preferred embodiments, it will be appreciated that many variations may be made. For example, - y —

the effective integration time can be determined by a heuristic algorithm or other means, within or outside the chip, instead of by counting the conditional reset pulses. While the illustrated examples are based on MOS technology, the method and apparatus can also be used in other semiconductor technologies, e.g., CCD and CID. Many other variations, modifications and applications of the invention will be apparent.

Claims

WHAT IS CLAIMED IS:

1. A method of increasing the dynamic range of an optical sensor by controlling the integration time during which the optical sensor integrates the light received thereon and produces an electrical output corresponding thereto, comprising: generating a series of regular reset pulses separated by prefixed time intervals of equal duration; generating a series of sample pulses each at a predetermined time interval after the generation of a regular reset pulse, which predetermined time interval represents a full integration time; generating a series of reset enable pulses at time intervals, between pulses, of shorter duration than that of the regular reset pulses; generating a control signal whenever a predetermined characteristic of the electrical output of the optical sensor exceeds a reference value; generating a conditional reset pulse whenever there is coincidence between a reset enable pulse and a control signal; starting the integration of the optical sensor at the time of generation of either a regular reset pulse or a conditional reset pulse; terminating the integration of the optical sensor at the time of generation of a sample pulse; determining the scaling factor, representing the ratio of the full integration time to the effective integration time between said starting and termination times; and multiplying the electrical output of the optical sensor by said scaling factor.

2. The method according to Claim 1 , wherein the effective integration time is determined by counting the number of conditional reset pulses between the generation of a regular reset pulse and a sample pulse.

3. The method according to Claim 2, wherein said reset enable pulses are generated at prefixed but unequal time periods .

4. The method according to Claim 3, wherein said reset enable pulses are generated at time intervals, to the next sample pulse, decreasing according to a downgoing geometric series.

5. The method according to Claim 4, wherein said "3 y~\ downgoing geometric series is T/X, T/X , T/X , T/X , wherein T is the full integration time, and X is a preselected constant.

6. The method according to any one of Claims 1-5, wherein said optical sensor includes an array of photo¬ sensitive elements.

7. The method according to Claim 6, wherein the integration times of blocks of said photo-sensitive elements are controlled simultaneously by said control signal.

8. The method according to Claim 7, wherein said control signal is derived from a photo-sensitive element in the respective block.

9. The method according to Claim 7, wherein said control signal is derived from the weighted average illumination level in the respective block.

10. The method according to Claim 7, wherein said control signal is derived from a weighted average illumination level in the surrounding area including the respective controlled block.

11. Apparatus for increasing the dynamic range of an optical sensor by controlling the integration time during which the optical sensor integrates the light received thereon and produces an electrical output corresponding thereto, comprising: means for generating a series of regular reset pulses separated by prefixed time intervals of equal duration; means for generating a series of sample pulses each at a predetermined time interval after the generation of a regular reset pulse, which predetermined time interval represents a full integration time; means for generating a series of reset enable pulses at time intervals, between pulses, of shorter duration than that of the regular reset pulses; means for generating a control signal whenever a predetermined characteristic of the electrical output of the optical sensor exceeds a reference value; means for generating a conditional reset pulse whenever there is coincidence between a reset enable pulse and a control signal; means for starting the integration of the optical sensor at the time of generation of either a regular reset pulse or a conditional reset pulse; means for terminating the integration of the optical sensor at the time of generation of a sample pulse; means for determining the scaling factor, representing the ratio of the full integration time to the effective integration time between said starting and termination times; and means for multiplying the electrical output of the optical sensor by said scaling factor.