US20110187878A1 - Synchronization of projected illumination with rolling shutter of image sensor - Google Patents

Abstract

Imaging apparatus includes an illumination assembly, including a plurality of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene. An imaging assembly includes an image sensor and objective optics configured to form an optical image of the scene on the image sensor, which includes an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form an electronic image of the scene. A controller is coupled to actuate the radiation sources sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter.

Description

CROSS-REFERENCE TO RELATED APPLICATION
• This application claims the benefit of U.S. Provisional Patent Application 61/300,465, filed Feb. 2, 2010, which is incorporated herein by reference.
FIELD OF THE INVENTION
• The present invention relates generally to systems and methods for electronic imaging, and specifically to methods of illumination for enhancing the quality of captured images.
BACKGROUND OF THE INVENTION
• Most low-cost CMOS image sensors use a rolling shutter, in which successive rows of sensor elements are triggered sequentially to capture light. This method of image acquisition thus records each individual frame not as a single snapshot at a point in time, but rather as a sequence of image stripes scanning across the frame. The result of the rolling shutter is that not all parts of the optical image are recorded at exactly the same time (although the frame is stored as a single electronic image).
• The use of a rolling shutter introduces a temporal shear in the image frame, which can create artifacts in imaging of moving objects. Bradley et al. address this problem in “Synchronization and Rolling Shutter Compensation for Consumer Video Camera Arrays,”IEEE International Workshop on Projector-Camera Systems—PROCAMS2009 (Miami Beach, Fla., 2009), which is incorporated herein by reference. The authors propose to solve the problem using synchronized stroboscopic illumination.
SUMMARY
• Embodiments of the present invention that are described hereinbelow provide apparatus and methods for illuminating an object that can be advantageous when the object is imaged using a sensor with a rolling shutter.
• There is therefore provided, in accordance with an embodiment of the present invention, imaging apparatus, including an illumination assembly, including a plurality of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene. An imaging assembly includes an image sensor and objective optics configured to form an optical image of the scene on the image sensor, which includes an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form an electronic image of the scene. A controller is coupled to actuate the radiation sources sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter.
• In disclosed embodiments, each group includes one or more rows of the sensor elements, and the regions define stripes that extend across the scene in a direction parallel to the rows. Typically, each stripe illuminates a respective region that contains the areas of the scene from which the sensor elements in a respective set of multiple rows capture the radiation, and the controller is configured to actuate the radiation sources so that the projected radiation sweeps across the scene in a direction perpendicular to the rows.
• In a disclosed embodiment, the rolling shutter defines a frame time for capturing the entire electronic image, and the controller is configured to actuate each of the radiation sources for a respective actuation period that is less than half the frame time. The controller may actuate each of the radiation sources so that the illumination assembly illuminates each area of the scene only during a respective exposure period of a corresponding group of the sensor elements that captures the radiation from the area.
• In some embodiments, the projection optics include a patterning element, which is configured so that the radiation is projected onto the scene in a predefined pattern, which is detectable in the electronic image formed by the imaging assembly. Typically, the controller is configured to analyze the pattern in the electronic image so as to generate a depth map of the scene. In one embodiment, the radiation sources include a matrix of light-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction perpendicular to the substrate. In another embodiment, the radiation sources include a row of edge-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction parallel to the substrate, and the illumination assembly includes a reflector disposed on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the patterning element.
• There is also provided, in accordance with an embodiment of the present invention, a method for imaging, including arranging a plurality of radiation sources to project radiation onto different, respective regions of the scene. An image sensor, which includes an array of sensor elements arranged in multiple groups, is configured to receive an optical image of the scene, in which the groups of the sensor elements receive the radiation from different, respective areas of the scene. The groups of the sensor elements are triggered with a rolling shutter to capture the radiation from the scene in successive, respective exposure periods so as to form an electronic image of the scene. The radiation sources are actuated sequentially in a pulsed mode so as to illuminate the different, respective areas of the scene in synchronization with the rolling shutter.
• In one embodiment, configuring the image sensor includes arranging multiple image sensors, having respective rolling shutters, together with multiple, respective pluralities of the radiation sources to form respective electronic images of different, respective, overlapping parts of a scene, and actuating the radiation sources includes synchronizing the respective pluralities of the radiation sources over the multiple image sensors so as to control an overlap of the respective areas of the scene illuminated by the radiation sources at any given time. The method may include analyzing the pattern over the electronic images formed by the multiple image sensors in order to generate a depth map of the scene.
• There is additionally provided, in accordance with an embodiment of the present invention, imaging apparatus, including multiple imaging units. The imaging units include respective pluralities of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene, and respective imaging assemblies. The imaging assemblies include respective image sensors and objective optics configured to form respective optical images of different, respective, overlapping parts of the scene on the respective image sensors. Each image sensor includes an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form respective electronic images of the scene. The radiation sources are actuated sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter, while synchronizing the respective pluralities of the radiation sources over the multiple image sensors so as to control an overlap of the respective areas of the scene illuminated by the radiation sources at any given time.
• Typically, the overlap is controlled so that the respective areas of the scene illuminated by the radiation sources at any given time are non-overlapping.
• The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which:
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a schematic side view of an imaging system, in accordance with an embodiment of the present invention;
• FIG. 2 is a schematic representation of a portion of an image frame captured using stripe illumination, in accordance with an embodiment of the present invention;
• FIG. 3 is a timing diagram showing synchronization of stripe illumination with rolling shutter operation, in accordance with an embodiment of the present invention;
• FIG. 4A is a schematic side view of a projection module, in accordance with an embodiment of the present invention;
• FIG. 4B is a schematic top view of an optoelectronic subassembly used in the projection module ofFIG. 4A;
• FIGS. 5A and 5B are schematic side and top views, respectively, of an optoelectronic subassembly, in accordance with another embodiment of the present invention;
• FIG. 5C is a schematic pictorial view of a prism used in the subassembly ofFIGS. 5A and 5B;
• FIG. 6 is a schematic side view of an illumination assembly, in accordance with an alternative embodiment of the present invention;
• FIG. 7 is a schematic representation of a portion of an image frame illuminated by the illumination assembly ofFIG. 6;
• FIG. 8 is a schematic side view of an imaging system, in accordance with another embodiment of the present invention; and
• FIG. 9 is a schematic pictorial view of an imaging system, in accordance with yet another embodiment of the present invention.
DETAILED DESCRIPTION OF EMBODIMENTS OVERVIEW
• Various types of imaging systems include optical projectors for illuminating the scene of interest. For example, a projector may be used to cast a pattern of coded or structured light onto an object for purposes of three-dimensional (3D) depth mapping. In this regard, U.S. Patent Application Publication 2008/0240502, whose disclosure is incorporated herein by reference, describes an illumination assembly in which a light source, such as a laser diode or LED, transilluminates a transparency with optical radiation so as to project a pattern onto the object. (The terms “optical,” “light” and “illumination” as used herein refer generally to any of visible, infrared, and ultraviolet radiation.) An image sensor captures an image of the pattern that is projected onto the object, and a processor processes the image so as to reconstruct a three-dimensional (3D) map of the object.
• Systems based on projection of patterned light may suffer from low signal/background ratio due to limitations on the power of the projector, particularly in conditions of strong ambient light. Embodiments of the present invention address this problem by projecting radiation onto the scene of interest in a synchronized spatial sweep, which is timed to take advantage of the rolling shutter of the image sensor in order to improve the signal/background ratio of the system.
• In embodiments of the present invention, the rolling shutter is operated so as to cause different groups (typically successive rows) of sensor elements in the image sensor to capture radiation in different, successive exposure periods, which are much shorter than the total frame period (typically less than half, and possibly less than 10%). Each such group collects radiation from a different, respective area of the scene, which is focused onto the image sensor by objective optics. The illumination assembly is controlled so as to sweep the projected radiation over those areas of the scene in synchronization with the rolling shutter, so that each area of the scene is illuminated during the specific time that the corresponding group of sensor elements is active. As a result, the output power of the illumination assembly is concentrated, in each area of the scene, in the specific exposure periods during which the corresponding sensor elements are able to collect radiation from that area. Limitation of the exposure periods by the rolling shutter reduces the total amount of ambient radiation that is collected, without wasting any of the projected radiation. Therefore, the signal/background ratio of the system is enhanced substantially even without increasing the average power of the illumination.
• In the embodiments that are disclosed hereinbelow, the illumination assembly comprises an array of radiation sources, with projection optics that project radiation from the radiation sources onto different, respective regions of the scene. The spatial sweep of the projected radiation is accomplished by pulsing the radiation sources sequentially. The respective region of the scene that is illuminated by each radiation source overlaps the areas in the scene that are sensed by one or more of the groups of the sensor elements. Each radiation source is thus pulsed on only during the time that the corresponding groups of sensor elements are active. This sequential pulsed operation of the array of radiation sources provides full flexibility in choosing the optimal timing for the spatial sweep of radiation, as well as high reliability in that no moving parts or active optical elements (other than the radiation sources themselves) are required to implement the sweep.
• Although the embodiments that are described below relate specifically to projection of patterned light in a 3D sensing system, the principles of the present invention may similarly be applied to enhance the performance of other projection-based imaging systems. The rolling shutter in these embodiments is assumed to activate the sensor elements in the image sensor row by row, as in conventional CMOS image sensors that are known in the art; but the principles of the present invention may similarly be applied in conjunction with image sensors that use other sorts of sequential activation of groups of sensor elements, such as block-by-block activation.
System Description
• FIG. 1 is a schematic side view of animaging system20, in accordance with an embodiment of the present invention. A set of X-Y-Z axes is used in this figure and throughout the description that follows to aid in understanding the orientation of the figures, wherein the X-Y plane is the frontal plane ofsystem20, and the Z-axis extends perpendicularly from this plane toward the scene. The choice of axes, however, is arbitrary and is made solely for the sake of convenience in describing embodiments of the invention.
• Anillumination assembly22 projects a patternedradiation field24 onto an object26 (in this case a hand of a user of the system) in a scene. Animaging assembly28 captures an image of the scene within a field ofview30. Acontroller31 or other electronic processor processes the image in order to generate a 3D depth map ofobject26. Further details of this sort of mapping process are described, for example, in the above-mentioned US 2008/0240502 and in PCT International Publication WO 2007/105205, whose disclosure is also incorporated herein by reference. The 3D map of the user's hand (and/or other parts of the user's body) may be used in a gesture-based computer interface, but this sort of functionality is beyond the scope of the present patent application.
• Imaging assembly28 comprisesobjective optics36, which form an optical image of thescene containing object26 on animage sensor38, such as a CMOS integrated circuit image sensor. The image sensor comprises an array ofsensor elements40, arranged in multiple rows. The sensor elements generate respective signals in response to the radiation focused onto them byoptics36, wherein the pixel value of each pixel in the electronic images output byimage sensor38 corresponds to the signal from arespective sensor element40. The sensor elements are activated and deactivated, row by row, by a rolling shutter, whose timing is set bycontroller31. This sort of rolling shutter operation is a standard feature of many CMOS image sensors.
• Illumination assembly22 comprises aprojection module32, which generates a beam of patterned light, andprojection optics34, which project the beam ontofield24.Module32 typically comprises multiple radiation sources, along with optics for pattern generation.Controller31 actuates the radiation sources sequentially, in a pulsed mode, in synchronization with the rolling shutter ofimage sensor38. The design ofmodule32 and the synchronization of its operation with the rolling shutter are described in detail hereinbelow.
• FIG. 2 is a schematic representation of a portion of animage frame42 captured bysystem20, in accordance with an embodiment of the present invention.Frame42 comprises a matrix ofpixels44, each corresponding to the signal generated by a correspondingsensor element40 inimage sensor38. Thus, each row ofpixels44 corresponds to the area in the scene from which radiation is captured by the corresponding row of sensor elements.
• Illumination assembly22 generatesmultiple stripes46,48,50,52, . . . of illumination. Each such stripe is generated by a respective radiation source or group of radiation sources. (Example arrangements of radiation sources that can be used to generate this sort of multi-stripe illumination are shown in the figures that follow.) The region defined by each stripe covers the area of a number of the rows ofpixels44. In other words, each stripe illuminates a certain area of the scene from which the image sensors in the corresponding rows capture radiation. Althoughstripes46,48,50,52 are shown inFIG. 2, for the sake of simplicity, as being precisely adjacent to one another and non-overlapping, in practical systems there is generally a certain amount of overlap between the stripes in order to ensure that all areas of the scene are illuminated.
• FIG. 3 is a timing diagram illustrating synchronization of the sort of stripe illumination shown inFIG. 2 with the operation of a rolling shutter inimage sensor38, in accordance with an embodiment of the present invention.Traces56 correspond to the operation of the rolling shutter on successive rows of sensor elements, wherein the elements are active (i.e., convert received photons to electrons in the output signal from the image sensor) when the corresponding trace is high. The period during which a given row is active is referred to herein as the exposure period of that row. The exposure periods of successive rows are staggered, so that each row is activated shortly after the preceding row. The rows are arranged ingroups58,60, . . . , each group corresponding to the region covered by one ofstripes46,48, . . . .
• Traces62,64, . . . correspond to actuation of the respective radiation sources that generatestripes46,48, . . . . In other words, whentrace62 is high, the radiation source that generatesstripe46 is actuated, and so on. For eachgroup58,60, . . . , of the rows, the actuation period of the corresponding radiation source is set so as to fall entirely within the exposure periods of all the rows in the group. Thus, the illumination assembly illuminates each area of the scene only during the exposure periods of the sensor elements that capture the radiation from the area, and none of the illumination is wasted.
• Trace64 goes high just astrace62 goes low, and so forth over all the radiation sources inillumination assembly22. Thus, the stripe output of the illumination assembly sweeps across the scene in a sweep direction perpendicular to the rows of pixels44 (and sensor elements40), completing one such sweep in each image frame, in synchronization with the sweep of the rolling shutter ofimage sensor38. The duty cycle of each radiation source is roughly 1:N, wherein N is the number of stripes (each illuminated by a respective radiation source or group of radiation sources). In the timing scheme ofFIG. 3, the actuation period of each illumination stripe is approximately 1/(N*FR), while the exposure period of each row ofsensor elements40 is approximately 2/(N*FR), wherein FR is the frame rate, such as 30 frames/sec. These timing relations typically make optimal use of the available illumination power and provide the greatest possible enhancement of signal/background ratio.
• Alternatively, other timing relations may be used between the frame rate, actuation periods and exposure times. These alternative timing arrangements may be advantageous in situations in which the geometrical relationships between illumination stripes and sensor rows are not maintained as precisely as inFIG. 2, and particularly when successive stripes partially overlap.
Illumination Module with Edge Emitters
• FIG. 4A is a schematic side view ofillumination module32, whileFIG. 4B is a schematic top view of an optoelectronic subassembly used inillumination module32, in accordance with an embodiment of the present invention.Module32 comprises a row of edge-emittingoptoelectronic elements70, such as laser diodes, which are formed on asubstrate72, such as a silicon wafer. (Only one of the elements can be seen in the side view ofFIG. 4A.)Elements70 emit radiation in a direction parallel to the substrate. Areflector74 on the substrate turns the radiation emitted byelements70 away from the substrate, which is oriented in the X-Y plane, toward the Z-axis. The reflector may be integrally formed insubstrate72, as shown inFIG. 4A, or it may alternatively comprise a separate element, which is positioned on the substrate and aligned withoptoelectronic elements70.Reflector74 may simply comprise a flat reflecting surface, or it may alternatively comprise one or more curved surfaces or multiple flat surfaces in order to spread or focus the radiation, as illustrated inFIG. 4B, as well asFIG. 5C.
• A collectinglens76 collimates and directs the radiation fromoptoelectronic elements70 through one ormore patterning elements78. The patterning elements cause the radiation fromelements70 to be projected onto the scene in a predefined pattern, which is detectable in the electronic image formed by imagingassembly28. This pattern in the image is processed in order to compute the depth map of the scene.Patterning elements78 may comprise a patterned transparency, which may comprise a micro-lens array (MLA), as described, for example, in the above-mentioned US 2008/0240502 or WO 2007/105205, and/or one or more diffractive optical elements (DOEs), as described in U.S. Patent Application Publication 2009/0185274, whose disclosure is also incorporated herein by reference. Additionally or alternatively, whenelements70 emit coherent radiation, patterningelements78 may comprise a diffuser, which casts a laser speckle pattern on the scene.
• Each ofoptoelectronic elements70 emits radiation that forms arespective stripe80,82,84, . . . , as shown inFIG. 4B. (Although the figure shows six such elements and respective stripes, a larger or smaller number of elements and stripes may be used, depending on application requirements.)Reflector74 may be slightly curved, as shown in the figure, so that the stripes spread over a wider area and overlap the adjacent stripes at their edges. As explained above, controller31 (FIG. 1) activateselements70 to emit radiation sequentially, in synchronization with the rolling shutter ofimage sensor38, during each image frame captured by imagingassembly28. Thus, each region of the scene is illuminated during the exposure periods of the corresponding rows ofsensor elements40.
• In embodiments in whichpatterning elements78 comprise a MLA or other transparency, eachstripe80,82,84, . . . , passes through a different, respective region of the transparency, and thus creates a respective part of the overall illumination pattern corresponding to the pattern embedded in the transparency.Projection optics34 projects this pattern onto the object.
• On the other hand, in embodiments in whichpatterning elements78 comprise a DOE, eitherlens76 or one of elements78 (or the geometry of optoelectronic elements70) is typically configured to create an appropriate “carrier” angle for the beam emitted by each of the optoelectronic elements. In such embodiments, the beams emitted by the different optoelectronic elements use different parts oflens76, which may therefore be designed so that the collimated beams exit at respective angles corresponding to the desired vertical fan-out. Alternatively, the illumination module may comprise some other type of optics, such as a blazed grating with as many different zones as there are optoelectronic elements.
• Further details of the fabrication ofillumination module32, as well as other, similar sorts of modules, are described in the above-mentioned U.S. Provisional Patent Application 61/300,465.
• FIGS. 5A and 5B are schematic side and top views, respectively, of anoptoelectronic subassembly90, whileFIG. 5C is a schematic pictorial view of aprism92 used insubassembly90, in accordance with another embodiment of the present invention.Subassembly90 may be used in place of the corresponding components inmodule32.
• Optoelectronic subassembly90 comprises a row of edge-emittingoptoelectronic elements70, such as laser diodes, which may be fabricated on a suitable substrate as in the preceding embodiment. Insubassembly90, however, the radiation emitted byelements70 is reflected internally from an interior surface94 (typically with a suitable reflective coating) ofprism92. The radiation fromelements70 entersprism92 via acurved entry surface96. As a result, respective beams generated byelements70 spread apart and overlap partially with the adjacent beams.Controller31 actuateselements70 to emit radiation sequentially during each image frame in synchronization with the rolling shutter ofimage sensor38
Illumination Module with Surface Emitters
• FIG. 6 is a schematic side view of anillumination assembly100, in accordance with an alternative embodiment of the present invention.Assembly100 may be used insystem20 in place ofillumination assembly22.Assembly100 comprises radiation sources in the form of a two-dimensional matrix ofoptoelectronic elements110, which are arranged on asubstrate102 and emit radiation in a direction perpendicular to the substrate. AlthoughFIG. 6 shows only asingle row114 of elements arrayed along the X-axis,assembly100 actually comprises multiple, parallel rows of this sort, forming a grid in the X-Y plane.FIG. 6 illustrates an 8×8 grid, but larger or smaller matrices, not necessarily square or rectilinear, may alternatively be used.
• In contrast to the preceding embodiments,elements110 comprise surface-emitting devices, such as light-emitting diodes (LEDs) or vertical-cavity surface-emitting laser (VCSEL) diodes, which emit radiation directly into the Z-direction. An array of microlenses (or other suitable micro-optics, such as total internal reflection-based micro-structures)112 is aligned withelements110, so that a respective microlens collects the radiation from each element and directs it into anoptical module104. The optical module comprises, inter alia, asuitable patterning element106, as described above, and aprojection lens108, which projects the resulting pattern onto the scene.
• FIG. 7 is a schematic representation of a portion of an image frame illuminated byassembly100, in accordance with an embodiment of the present invention. Eachmicrolens112 spreads the radiation from the correspondingoptoelectronic element110 over a region of the scene that corresponds to a group ofpixels44. (Typically there is some overlap between neighboring regions, as in the preceding embodiments.)Elements110 are arranged inmultiple rows114,116, . . . . In typical operation,controller31 actuates all the optoelectronic elements in each row in turn in synchronization with the rolling shutter ofimage sensor38, in accordance with the scheme shown inFIG. 3, for example. Thus, as described above, the area of eachpixel44 is illuminated during the exposure period of the correspondingsensor element40.
• Although the above embodiments are described, for the sake of clarity, in the context ofsystem20 and certain specific geometrical configurations of illumination and sensing, the principles of the present invention may similarly be applied in systems and configurations of other sorts.
Synchronization Over Multiple Sensors
• FIG. 8 is a schematic side view of animaging system120, in accordance with another embodiment of the present invention. In this system, asynchronization controller121 synchronizes the operation ofmultiple sensing units122,124,126,128. Each of these sensing units typically comprises an illumination assembly and an imaging assembly, which operate in concert as insystem20. Eachsensing unit122,124,126,128 projects a respective patternedbeam132,134,136,138 onto ascene130 and forms a respective image of the part of the scene that is illuminated by the respective pattern.
• In order to coverscene130 completely, the projected patterned beams typically overlap inoverlap regions140. In conventional operation, the overlap of the patterns could lead to inability of sensingunits122,124,126,128 to detect their own patterns reliably inregions140 and thus to loss of 3D information in these regions. One way to overcome this problem could be to operate the sensing units at different wavelengths, so that each unit senses only its own pattern. This solution, however, can be cumbersome and require costly optoelectronics and optical filters.
• Therefore, insystem120,controller121 controls the timing of the illumination assemblies and the rolling shutters of the imaging assemblies in sensingunits122,124,126,128 so as to control the overlap between the regions that are illuminated at any given time. Typically, the sensing units are controlled so that they illuminate and capture radiation from respectivenon-overlapping stripes142,144,146,148. Within each sensing unit, the illumination stripe and the sensing area that is triggered to receive radiation by the rolling shutter are internally synchronized as described above. Furthermore, the timing of all the sensing units is coordinated to avoid interference. Thus, for example, all of the sensing units simultaneously activate theirrespective stripes142, followed bystripes144, and so on, so that no more than a single sensing unit is active within each overlapregion140 at any given time. Each sensing unit provides 3D mapping data with respect to its own part ofscene130, and a processing unit (such ascontroller121 or another computer) stitches the data together into a combined depth map.
• The scheme illustrated inFIG. 8 is just one example of a possible synchronization pattern, and alternative geometrical and timing patterns may also be implemented to achieve similar objectives. For example, the synchronized sensing units may be arranged in a two-dimensional array in order to cover a wider area ofscene130. Depending on the geometrical arrangement and the timing of the sensing units, systems of multiple synchronized sensing units may be used to capture depth information over greater areas of substantially any desired size and profile, or, alternatively or additionally, with greater speed.
• Alternatively, sensingunits122,124,126,128 may operate together without a centralized controller to regulate synchronization. For example, each sensing unit may adjust its own timing so as to maximize its depth readings. Thus, the entire system will converge to an optimal synchronization. Additionally or alternatively, the sensing units may communicate with one another using a token ring type protocol, without centralized control.
• FIG. 9 is a schematic pictorial view of animaging system150, in accordance with yet another embodiment of the present invention. This embodiment is similar in its principles of operation to the embodiment ofFIG. 8: Multiple sensingunits152,154, . . . , project respective patternedbeams156,158, . . . , onto a scene, while controlling the timing of their respective illumination assemblies and rolling shutters so as to illuminate and capture radiation from respective sequences ofstripes160.Beams156 and158 overlap in anoverlap region162. Although for the sake of simplicity, only two sensing units are shown inFIG. 9, any suitable number of sensing units may be arranged in this matter.
• Insystem150, however, sensingunits152 and154 and theirbeams156 and158 are offset from one another in a direction perpendicular to the scan direction of the illumination and rolling shutter (horizontal offset with vertical scan in the view shown inFIG. 9), as opposed to the parallel offset shown inFIG. 8. Therefore, most or all ofstripes160 may overlap with certain stripes of the neighboring sensing unit. The scans of sensingunits152,154, . . . , are therefore synchronized so that each stripe is illuminated in different time periods from its overlapping neighbors. As shown inFIG. 9, there is no need for precise overlap betweenstripes160 of the different sensing units, nor do the stripes need to be exactly parallel. Generally speaking, the sensing units may be arranged in any desired arrangement, as long as the synchronization schedule can make overlapping stripes disjoint in time.
• It will thus be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.

Claims (27)

1. Imaging apparatus, comprising:

an illumination assembly, comprising a plurality of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene;

an imaging assembly, comprising an image sensor and objective optics configured to form an optical image of the scene on the image sensor, which comprises an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form an electronic image of the scene; and

a controller, which is coupled to actuate the radiation sources sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter.

2. The apparatus according toclaim 1, wherein each group comprises one or more rows of the sensor elements, and wherein the regions define stripes that extend across the scene in a direction parallel to the rows.

3. The apparatus according toclaim 2, wherein each stripe illuminates a respective region that contains the areas of the scene from which the sensor elements in a respective set of multiple rows capture the radiation.

4. The apparatus according toclaim 2, wherein the controller is configured to actuate the radiation sources so that the projected radiation sweeps across the scene in a direction perpendicular to the rows.

5. The apparatus according toclaim 1, wherein the rolling shutter defines a frame time for capturing the entire electronic image, and wherein the controller is configured to actuate each of the radiation sources for a respective actuation period that is less than half the frame time.

6. The apparatus according toclaim 5, wherein the controller is configured to actuate each of the radiation sources so that the illumination assembly illuminates each area of the scene only during a respective exposure period of a corresponding group of the sensor elements that captures the radiation from the area.

7. The apparatus according toclaim 1, wherein the projection optics comprise a patterning element, which is configured so that the radiation is projected onto the scene in a predefined pattern, which is detectable in the electronic image formed by the imaging assembly.

8. The apparatus according toclaim 7, wherein the controller is configured to analyze the pattern in the electronic image so as to generate a depth map of the scene.

9. The apparatus according toclaim 7, wherein the radiation sources comprise a matrix of light-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction perpendicular to the substrate.

10. The apparatus according toclaim 7, wherein the radiation sources comprise a row of edge-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction parallel to the substrate.

11. The apparatus according toclaim 10, wherein the illumination assembly comprises a reflector disposed on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the patterning element.

12. A method for imaging, comprising:

arranging a plurality of radiation sources to project radiation onto different, respective regions of the scene;

configuring an image sensor, which comprises an array of sensor elements arranged in multiple groups, to receive an optical image of the scene, in which the groups of the sensor elements receive the radiation from different, respective areas of the scene;

triggering the groups of the sensor elements with a rolling shutter to capture the radiation from the scene in successive, respective exposure periods so as to form an electronic image of the scene; and

actuating the radiation sources sequentially in a pulsed mode so as to illuminate the different, respective areas of the scene in synchronization with the rolling shutter.

13. The method according toclaim 12, wherein each group comprises one or more rows of the sensor elements, and wherein the regions define stripes that extend across the scene in a direction parallel to the rows.

14. The method according toclaim 13, wherein each stripe illuminates a respective region that contains the areas of the scene from which the sensor elements in a respective set of multiple rows capture the radiation.

15. The method according toclaim 13, wherein actuating the radiation sources causes the projected radiation to sweep across the scene in a direction perpendicular to the rows.

16. The method according toclaim 12, wherein the rolling shutter defines a frame time for capturing the entire electronic image, and wherein actuating the radiation sources comprises actuating each of the radiation sources for a respective actuation period that is less than half the frame time.

17. The method according toclaim 16, wherein actuating each of the radiation sources comprises illuminating each area of the scene only during a respective exposure period of a corresponding group of the sensor elements that captures the radiation from the area.

18. The method according toclaim 12, wherein arranging the plurality of the radiation source comprises projecting the radiation onto the scene in a predefined pattern, which is detectable in the electronic image.

19. The method according toclaim 18, and comprising analyzing the pattern in the electronic image so as to generate a depth map of the scene.

20. The method according toclaim 18, wherein the radiation sources comprise a matrix of light-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction perpendicular to the substrate.

21. The method according toclaim 18, wherein the radiation sources comprise a row of edge-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction parallel to the substrate.

22. The method according toclaim 21, wherein arranging the plurality of the radiation sources comprises providing a reflector on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the scene.

23. The method according toclaim 12, wherein configuring the image sensor comprises arranging multiple image sensors, having respective rolling shutters, together with multiple, respective pluralities of the radiation sources to form respective electronic images of different, respective, overlapping parts of a scene, and

wherein actuating the radiation sources comprises synchronizing the respective pluralities of the radiation sources over the multiple image sensors so as to control an overlap of the respective areas of the scene illuminated by the radiation sources at any given time.

24. The method according toclaim 23, and comprising analyzing the pattern over the electronic images formed by the multiple image sensors in order to generate a depth map of the scene.

25. Imaging apparatus, comprising:

multiple imaging units, comprising:

respective pluralities of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene; and

respective imaging assemblies, comprising respective image sensors and objective optics configured to form respective optical images of different, respective, overlapping parts of the scene on the respective image sensors, each image sensor comprising an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form respective electronic images of the scene,

wherein the radiation sources are actuated sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter, while synchronizing the respective pluralities of the radiation sources over the multiple image sensors so as to control an overlap of the respective areas of the scene illuminated by the radiation sources at any given time.

26. The apparatus according toclaim 25, and comprising a controller, which is configured to analyze the pattern over the electronic images formed by the image sensors in the multiple imaging units so as to generate a depth map of the scene.

27. The apparatus according toclaim 25, wherein the overlap is controlled so that the respective areas of the scene illuminated by the radiation sources at any given time are non-overlapping.

Images