WO2021008209A1 - Depth measurement apparatus and distance measurement method - Google Patents

Abstract

Disclosed are a depth measurement apparatus and a distance measurement method. The depth measurement apparatus (10, 40) comprises: a light emission module (11, 41), which includes a light source (111) and a patterned optical element (112), wherein the light source (111) is used for emitting a light beam, the amplitude of which is modulated, on a time sequence, and the patterned optical element (112) is used for receiving the light beam and then emitting a structured light beam to a target object (20); a TOF image sensor (121), which includes at least one pixel (51), wherein the pixel (51) is used for receiving the structured light beam reflected by the target object (20) and forming an electrical signal; and a control and processing circuit (13, 43), which receives the electrical signal, calculates intensity information of the reflected structured light beam to form a structured light pattern, and calculates a depth image of the target object (20) by using the structured light pattern. The TOF image sensor (121) is combined with the structured light source, amplitude modulation is used at an emission end, and multi-tap pixel collection is used at a receiving end, thereby achieving a depth measurement method, which is hard to achieve by means of a traditional solution, capable of resisting ambient light interference.

Description

Translated fromChinese

深度测量装置及距离测量方法Depth measuring device and distance measuring method技术领域Technical field

本发明涉及光学测量技术领域，尤其涉及一种深度测量装置及距离测量方法。The present invention relates to the field of optical measurement technology, in particular to a depth measurement device and a distance measurement method.

背景技术Background technique

ToF的全称是Time-of-Flight，即飞行时间，ToF测距技术是一种通过测量光脉冲在发射/接收装置和目标物体间的往返飞行时间来实现精确测距的技术。在ToF技术中直接对光飞行时间进行测量的技术被称为dToF(direct-TOF)；对发射光信号进行周期性调制，通过对反射光信号相对于发射光信号的相位延迟进行测量，再由相位延迟对飞行时间进行计算的测量技术被成为iToF(Indirect-TOF)技术。按照调制解调类型方式的不同可以分为连续波(Continuous Wave，CW)调制解调方法和脉冲调制(Pulse Modulated，PM)调制解调方法。The full name of ToF is Time-of-Flight, that is, time of flight. ToF ranging technology is a technology that achieves precise ranging by measuring the round-trip flight time of light pulses between the transmitting/receiving device and the target object. In the ToF technology, the technology of directly measuring the optical time of flight is called dToF (direct-TOF); the emitted light signal is periodically modulated, and the phase delay of the reflected light signal relative to the emitted light signal is measured. The measurement technique that calculates the time of flight phase delay is called iToF (Indirect-TOF) technique. According to the different types of modulation and demodulation, it can be divided into continuous wave (CW) modulation and demodulation method and pulse modulation (Pulse Modulated, PM) modulation and demodulation method.

结构光测距技术则向空间物体发射结构光光束，其次采集被物体调制及反射后的结构光光束所形成的结构光图案，最后利用三角法进行深度计算以获取物体的深度数据。常用的结构光图案有不规则斑点图案、条纹图案、相移图案等。The structured light ranging technology emits a structured light beam to a space object, and then collects the structured light pattern formed by the structured light beam modulated and reflected by the object, and finally uses the triangulation method to perform depth calculation to obtain the depth data of the object. Commonly used structured light patterns include irregular spot patterns, stripe patterns, phase shift patterns and so on.

相比而言，TOF技术无需进行复杂的图像处理计算(如结构光图像匹配计算)，在中远距时能保持相对较高的测量精度；而结构光技术则在近距测量时具有非常高的精度，但随着距离增加精度会至少呈现线性增长。另外，结构光技术由于是通过采集反映反射光强度的图像来进行计算深度的，因此难免会受到环境光对光强度的影响，相对而言TOF则在环境光抗干扰方面要优于结构光技术。In contrast, TOF technology does not require complex image processing calculations (such as structured light image matching calculations), and can maintain relatively high measurement accuracy at medium and long distances; while structured light technology has very high measurement accuracy at close distances. Accuracy, but the accuracy will increase at least linearly as the distance increases. In addition, because structured light technology calculates depth by collecting images reflecting the intensity of reflected light, it will inevitably be affected by ambient light on light intensity. Relatively speaking, TOF is superior to structured light technology in terms of anti-interference of ambient light. .

发明内容Summary of the invention

本发明为了解决现有的问题，提供一种深度测量装置及距离测量方法。In order to solve the existing problems, the present invention provides a depth measuring device and a distance measuring method.

为了解决上述问题中的至少一个，本发明采用的技术方案如下所述：In order to solve at least one of the above problems, the technical solutions adopted by the present invention are as follows:

深度测量装置，包括：光发射模组，包含光源及图案化光学元件，所述光源用于发射时序上振幅被调制的光束，所述图案化光学元件用于接收所述光束后向目标物体发射结构光光束；TOF图像传感器，包含至少一个像素，所述像素用于接收由所述目标物体反射的结构光光束并形成电信号；控制和处理电路，接收所述电信号并计算出所述反射的结构光光束的强度信息以形成结构光图案，以及，利用所述结构光图案计算所述目标物体的深度图像。The depth measuring device includes: a light emitting module, including a light source and a patterned optical element, the light source is used to emit a light beam whose amplitude is modulated in time sequence, and the patterned optical element is used to receive the light beam and emit to a target object Structured light beam; TOF image sensor, including at least one pixel, the pixel is used to receive the structured light beam reflected by the target object and form an electrical signal; a control and processing circuit that receives the electrical signal and calculates the reflection The intensity information of the structured light beam is used to form a structured light pattern, and the structured light pattern is used to calculate a depth image of the target object.

在本发明的一种实施例中，所述结构光光束包含不规则斑点图案光束、条纹图案光束、二维编码图案化光束；所述不规则斑点图案光束中每个斑点对应的光束的振幅在时序上以连续波、方波或脉冲的方式被调制。In an embodiment of the present invention, the structured light beam includes an irregular spot pattern beam, a stripe pattern beam, and a two-dimensional coded patterned beam; the amplitude of the beam corresponding to each spot in the irregular spot pattern beam is The timing is modulated by continuous wave, square wave or pulse.

在本发明的又一种实施例中，所述像素包括至少3个抽头，所述抽头用于在单个帧周期内分别采集由包含所述目标物体反射的结构光光束所产生的电信号。 所述光发射模组发射时序上振幅被正弦波调制的光束，所述TOF图像传感器的各个像素至少包括4个抽头，分别用于在单个帧周期内分别采集4次光信号并转换成所述电信号C1、C2、C3和C4，4次采集的时间以及间隔相同；采用加权平均的方式计算所述结构光光束的强度B，具体如下：

或，采用加入环境光消除因子的方式计算所述结构光光束的强度B，具体如下：

所述光发射模组发射时序上振幅被脉冲调制的光束，所述TOF图像传感器的各个像素至少包括3个抽头，分别用于在单个帧周期内分别采集3次光信号并转换成电信号C1、C2和C3，第一抽头的触发时间与光脉冲的发射时间同步，且触发时长与脉冲的宽度相同，第二抽头、第三抽头在所述第一抽头结束时相继被触发，触发时长与所述第一抽头相同；采用加权平均的方式计算所述结构光光束的强度B，具体如下：

或，采用加入环境光消除因子的方式计算所述结构光光束的强度B，具体如下：B＝C₁+C₂-2C₃。In another embodiment of the present invention, the pixel includes at least 3 taps, and the taps are used to separately collect the electrical signals generated by the structured light beam reflected by the target object within a single frame period. The light emitting module emits a light beam whose amplitude is modulated by a sine wave in time sequence. Each pixel of the TOF image sensor includes at least 4 taps, which are used to collect and convert the light signal 4 times in a single frame period. The electrical signals C1, C2, C3 and C4 are collected at the same time and interval for 4 times; the weighted average method is used to calculate the intensity B of the structured light beam, as follows:

Or, calculate the intensity B of the structured light beam by adding an ambient light elimination factor, as follows:

The light emitting module emits a light beam whose amplitude is pulse-modulated in time sequence, and each pixel of the TOF image sensor includes at least 3 taps, which are respectively used to collect and convert the optical signal three times in a single frame period into an electrical signal C1 , C2 and C3, the trigger time of the first tap is synchronized with the emission time of the light pulse, and the trigger duration is the same as the pulse width. The second tap and the third tap are triggered successively at the end of the first tap, and the trigger duration is equal to The first taps are the same; the weighted average method is used to calculate the intensity B of the structured light beam, specifically as follows:

Or, the intensity B of the structured light beam is calculated by adding an ambient light elimination factor, specifically as follows: B=C₁ +C₂ -2C₃ .

本发明还提供一种距离测量方法，包括：利用光源发射时序上振幅被调制的光束，所述光束被图案化光学元件接收后形成结构光光束向目标物体发射；利用包含至少一个像素的TOF图像传感器的所述像素接收由所述目标物体反射的结构光光束并形成电信号；接收所述电信号并计算出所述反射的结构光光束的强度信息以形成结构光图案，以及，利用所述结构光图案计算所述目标物体的深度图像。The present invention also provides a distance measurement method, including: using a light source to emit a light beam whose amplitude is modulated in time sequence, and the light beam is received by a patterned optical element to form a structured light beam and emitted to a target object; using a TOF image containing at least one pixel The pixels of the sensor receive the structured light beam reflected by the target object and form an electrical signal; receive the electrical signal and calculate the intensity information of the reflected structured light beam to form a structured light pattern, and use the The structured light pattern calculates the depth image of the target object.

在本发明的一种实施例中，所述结构光光束包含不规则斑点图案光束、条纹图案光束、二维编码图案化光束；所述不规则斑点图案光束中每个斑点对应的光束的振幅在时序上以连续波、方波或脉冲的方式被调制。In an embodiment of the present invention, the structured light beam includes an irregular spot pattern beam, a stripe pattern beam, and a two-dimensional coded patterned beam; the amplitude of the beam corresponding to each spot in the irregular spot pattern beam is The timing is modulated by continuous wave, square wave or pulse.

在本发明的另一种实施例中，所述像素包括至少3个抽头，所述抽头用于在单个帧周期内分别采集由包含所述目标物体反射的结构光光束所产生的电信号。所述结构光光束是时序上振幅被正弦波调制的光束，各个所述像素至少包括4个抽头，分别用于在单个帧周期内分别采集4次光信号并转换成所述电信号C1、C2、C3和C4，4次采集的时间以及间隔相同；采用加权平均的方式计算所述结构光光束的强度B，具体如下：

或，采用加入环境光消除因子的方式计算所述结构光光束的强度B，具体如下：

所述结构光光束是时序上振幅被脉冲调制的光束，各个所述像素至少包括3个抽头，分别用于在单个帧周期内分别采集3次光信号并转换成电信号C1、C2和C3，第一抽头的触发时间与光脉冲的发射时间同步，且触发时长与脉冲的宽度相同，第二抽头、第三抽头在所述第一抽头结束时相继被触发，触发时长与所述第一抽头相同；采用加权平均的方式计算所述结构光光束的强度B，具体如下：

或，采用加入环境光消除因子的方式计算所述结构光光束的强度B，具体如下：B＝C₁+C₂-2C₃。In another embodiment of the present invention, the pixel includes at least 3 taps, and the taps are used to separately collect electrical signals generated by the structured light beam reflected by the target object within a single frame period. The structured light beam is a light beam whose amplitude is modulated by a sine wave in time series, and each pixel includes at least 4 taps, which are respectively used to collect 4 optical signals and convert them into the electrical signals C1 and C2 in a single frame period. , C3 and C4, the time and interval of the 4 acquisitions are the same; the weighted average method is used to calculate the intensity B of the structured light beam, as follows:

Or, calculate the intensity B of the structured light beam by adding an ambient light elimination factor, as follows:

The structured light beam is a light beam whose amplitude is pulse-modulated in time series. Each pixel includes at least 3 taps, which are respectively used to collect and convert optical signals three times in a single frame period into electrical signals C1, C2, and C3, The trigger time of the first tap is synchronized with the emission time of the light pulse, and the trigger duration is the same as the pulse width. The second tap and the third tap are triggered one after another at the end of the first tap, and the trigger duration is the same as that of the first tap. Same; the weighted average method is used to calculate the intensity B of the structured light beam, as follows:

Or, the intensity B of the structured light beam is calculated by adding an ambient light elimination factor, specifically as follows: B=C₁ +C₂ -2C₃ .

本发明的有益效果为：提供一种深度测量装置及距离测量方法，将TOF图像传感器与结构光光源相结合，发射端采用了振幅调制，在接收端采用了多抽头像素采集，从而可以使得本发明的装置和方法拥有比传统方案更多的功能，实现传统方案难以实现的抗环境光干扰的深度测量方法。The beneficial effects of the present invention are: a depth measurement device and a distance measurement method are provided, the TOF image sensor is combined with a structured light source, the transmitting end adopts amplitude modulation, and the receiving end adopts multi-tap pixel collection, which can make the The invented device and method have more functions than the traditional solution, and realize a depth measurement method against ambient light interference that is difficult to achieve with the traditional solution.

附图说明Description of the drawings

图1是本发明实施例中深度测量装置的结构示意图。Fig. 1 is a schematic structural diagram of a depth measuring device in an embodiment of the present invention.

图2是本发明实施例中一种深度测量装置原理示意图。Fig. 2 is a schematic diagram of the principle of a depth measuring device in an embodiment of the present invention.

图3是本发明实施例中又一种深度测量装置原理示意图。Fig. 3 is a schematic diagram of the principle of another depth measuring device in an embodiment of the present invention.

图4是本发明实施例中一种融合TOF与结构光方案的深度测量装置的结构示意图。Fig. 4 is a schematic structural diagram of a depth measurement device integrating TOF and structured light solutions in an embodiment of the present invention.

图5是本发明实施例中又一种融合TOF与结构光方案的深度测量装置的结构示意图。FIG. 5 is a schematic structural diagram of another depth measurement device that integrates TOF and structured light solutions in an embodiment of the present invention.

图6是本发明实施例中结构光与TOF时序控制示意图。Fig. 6 is a schematic diagram of structured light and TOF timing control in an embodiment of the present invention.

图7是本发明实施例中控制与处理电路架构示意图。FIG. 7 is a schematic diagram of the control and processing circuit structure in an embodiment of the present invention.

图8是本发明实施例中一种距离测量方法示意图。Fig. 8 is a schematic diagram of a distance measurement method in an embodiment of the present invention.

图9是本发明实施例中另一种距离测量方法示意图。Fig. 9 is a schematic diagram of another distance measurement method in an embodiment of the present invention.

图10是本发明实施例中一种对所述电信号进行处理方法示意图。Fig. 10 is a schematic diagram of a method for processing the electrical signal in an embodiment of the present invention.

图11是本发明实施例中另一种对所述电信号进行处理方法示意图。Fig. 11 is a schematic diagram of another method for processing the electrical signal in an embodiment of the present invention.

其中，10-深度测量装置，11-光发射模组，12-采集模组，13-控制与处理电路，14-泛光光源模组，111-光源，112-图案化光学元件，121-TOF图像传感器，122-透镜单元，20-目标物体，30-发射光束，40-反射光束，201-斑点结构光图案化光束，202-光斑，40-深度测量装置，41-光发射模组，42-采集模组，43-控制与处理电路，44-泛光光源模组，51-像素，52-光斑，70-芯片，71-相位计算模块，72-标定模块，73-第一后处理模块，74-1-分路器，74-2-复用器，75-振幅计算模块，76-前处理模块，77-匹配模块，78-第二后处理模块，79-融合模块。Among them, 10-depth measuring device, 11-light emission module, 12-acquisition module, 13-control and processing circuit, 14- flood light source module, 111-light source, 112-patterned optical element, 121-TOF Image sensor, 122-lens unit, 20-target object, 30-emission beam, 40-reflected beam, 201-spot structured light patterned beam, 202-spot, 40-depth measuring device, 41-light emitting module, 42 -Acquisition module, 43- control and processing circuit, 44- flood light source module, 51- pixel, 52- spot, 70- chip, 71- phase calculation module, 72- calibration module, 73- first post-processing module , 74-1-splitter, 74-2-multiplexer, 75-amplitude calculation module, 76-pre-processing module, 77-matching module, 78-second post-processing module, 79-fusion module.

具体实施方式Detailed ways

为了使本发明实施例所要解决的技术问题、技术方案及有益效果更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the technical problems, technical solutions, and beneficial effects to be solved by the embodiments of the present invention clearer and clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.

需要说明的是，当元件被称为“固定于”或“设置于”另一个元件，它可以直接在另一个元件上或者间接在该另一个元件上。当一个元件被称为是“连接于”另一个元件，它可以是直接连接到另一个元件或间接连接至该另一个元件上。另外，连接即可以是用于固定作用也可以是用于电路连通作用。It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the connection can be used for fixing or circuit connection.

需要理解的是，术语“长度”、“宽度”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明实施例和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。It should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top" , "Bottom", "Inner", "Outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, rather than indicating or implying. The device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多该特征。在本发明实施例的描述中，“多个”的含义是两个或两个以上，除非另有明确具体的限定。In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present invention, "a plurality of" means two or more than two, unless otherwise specifically defined.

图1是根据本发明一个实施例的深度测量装置。深度测量装置10包括光发射模组11、采集模组12以及控制与处理电路13，其中光发射模组11用于向目标物体20发射发射光束30，发射光束30是在时序上振幅被调制的结构光光束，该结构光光束至目标空间中以照明空间中的目标物体20，至少部分发射光束30经目标物体20反射后形成反射光束40，反射光束40的至少部分被采集模组12采集，控制与处理电路13分别与光发射模组11以及采集模组12连接，以控制光束的发射与采集，同时接收来自采集模组12所采集到的电信号，并对该电信号进行计算以获取目标物体的深度信息。Fig. 1 is a depth measuring device according to an embodiment of the present invention. Thedepth measuring device 10 includes alight emitting module 11, acollection module 12, and a control andprocessing circuit 13. Thelight emitting module 11 is used to emit alight beam 30 to atarget object 20, and thelight beam 30 is amplitude modulated in time series. A structured light beam that reaches the target space to illuminate thetarget object 20 in the space. At least part of the emittedlight beam 30 is reflected by thetarget object 20 to form a reflectedlight beam 40, and at least part of the reflectedlight beam 40 is collected by thecollection module 12, The control andprocessing circuit 13 is respectively connected with thelight emitting module 11 and thecollection module 12 to control the emission and collection of the light beam, and at the same time receive the electric signal collected from thecollection module 12, and calculate the electric signal to obtain Depth information of the target object.

光发射模组11包括光源111、图案化光学元件112以及光源驱动器(图中未示出)等。光源111可以是发光二极管(LED)、边发射激光器(EEL)、垂直腔面发射激光器(VCSEL)等光源，也可以是多个光源组成的光源阵列，光源所发射的光束可以是可见光、红外光、紫外光等。光源111在光源驱动器(其可以进一步被控制和处理电路13控制)的控制下以一定的时序振幅被调制后向外发射光束，比如在一个实施例中，光源111在控制下以一定的频率发射脉冲调制光束、方波调制光束、正弦波调制光束等光束。可以理解的是，可以利用控制和处理电路13中的一部分或者独立于控制和处理电路13存在的子电路来控制光源111发射相关的光束，比如脉冲信号发生器。Thelight emitting module 11 includes alight source 111, a patternedoptical element 112, a light source driver (not shown in the figure), and the like. Thelight source 111 can be a light source such as a light emitting diode (LED), an edge emitting laser (EEL), a vertical cavity surface emitting laser (VCSEL), or a light source array composed of multiple light sources. The light beam emitted by the light source can be visible light or infrared light. , UV light, etc. Thelight source 111 is controlled by the light source driver (which can be further controlled by the control and processing circuit 13) to emit light beams outward after being modulated with a certain timing amplitude. For example, in one embodiment, thelight source 111 emits light at a certain frequency under control. Pulse modulated beam, square wave modulated beam, sine wave modulated beam and other beams. It is understandable that a part of the control andprocessing circuit 13 or a sub-circuit independent of the control andprocessing circuit 13 can be used to control thelight source 111 to emit related light beams, such as a pulse signal generator.

图案化光学元件112接收来自光源111的光束，并向外发射结构光光束，比 如不规则排列的斑点图案化光束、条纹图案光束、二维编码图案化光束等。在一些实施例中，图案化光学元件112还用于将接收到的光束进行扩束，以扩大视场角。可以理解的是，通过图案化光学元件112调制后的光束其振幅依旧是以一定的时序被调制，即入射的正弦波调制光束，出射的依旧是正弦波调制光束。在本发明的一种实施例中，不规则斑点图案光束中每个斑点对应的光束的振幅在时序上以连续波、方波或脉冲的方式被调制。The patternedoptical element 112 receives the light beam from thelight source 111 and emits structured light beams, such as irregularly arranged spot patterned beams, striped patterned beams, and two-dimensional coded patterned beams. In some embodiments, the patternedoptical element 112 is also used to expand the received light beam to expand the angle of view. It can be understood that the amplitude of the light beam modulated by the patternedoptical element 112 is still modulated in a certain time sequence, that is, the incident sine wave modulated light beam still emits the sine wave modulated light beam. In an embodiment of the present invention, the amplitude of the beam corresponding to each spot in the irregular spot pattern beam is modulated in a continuous wave, square wave or pulse manner in time sequence.

采集模组12包括TOF图像传感器121、透镜单元122，还可以包含滤光片(图中未示出)，透镜单元122接收并将由目标物体反射回的至少部分结构光光束并成像在至少部分TOF图像传感器121上，滤光片需选择与光源波长相匹配的窄带滤光片，用于抑制其余波段的背景光噪声。TOF图像传感器121可以是电荷耦合元件(CCD)、互补金属氧化物半导体(CMOS)、雪崩二极管(AD)、单光子雪崩二极管(SPAD)等组成的图像传感器，阵列大小代表着该深度相机的分辨率，比如320x240等。一般地，与图像传感器121连接的还包括由信号放大器、时数转换器(TDC)、模数转换器(ADC)等器件中的一种或多种组成的读出电路(图中未示出)。Theacquisition module 12 includes aTOF image sensor 121, alens unit 122, and may also include a filter (not shown in the figure). Thelens unit 122 receives and reflects at least part of the structured light beam from the target object and images it on at least part of the TOF. On theimage sensor 121, the filter needs to select a narrow-band filter that matches the wavelength of the light source to suppress background light noise in the remaining wavelength bands. TheTOF image sensor 121 can be an image sensor composed of charge coupled devices (CCD), complementary metal oxide semiconductors (CMOS), avalanche diodes (AD), single photon avalanche diodes (SPAD), etc. The size of the array represents the resolution of the depth camera Rate, such as 320x240, etc. Generally, connected to theimage sensor 121 also includes a readout circuit composed of one or more of a signal amplifier, a time-to-digital converter (TDC), an analog-to-digital converter (ADC) and other devices (not shown in the figure). ).

一般地，TOF图像传感器121包括至少一个像素，与传统的仅用于拍照的图像传感器相比，这里每个像素则包含两个以上的抽头(tap，用于在相应电极的控制下存储并读取或者排出由入射光子产生的电荷信号)，比如包括3个抽头，在单个帧周期(或单次曝光时间内)内以一定的次序依次切换抽头以采集相应的光子，以用于接收光信号并转换成电信号。Generally, theTOF image sensor 121 includes at least one pixel. Compared with a traditional image sensor that is only used for taking pictures, each pixel here includes more than two taps (tap, used to store and read under the control of the corresponding electrode). Take or discharge the charge signal generated by incident photons), for example, including 3 taps, switch the taps in a certain order within a single frame period (or single exposure time) to collect the corresponding photons for receiving light signals And converted into electrical signals.

控制与处理电路13可以是独立的专用电路，比如包含CPU、存储器、总线等组成的专用SOC芯片、FPGA芯片、ASIC芯片等等，也可以包含通用处理电路，比如当该深度相机被集成到如手机、电视、电脑等智能终端中去，终端中的处理电路可以作为该控制与处理电路13的至少一部分。The control andprocessing circuit 13 can be an independent dedicated circuit, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc. composed of CPU, memory, bus, etc., or a general processing circuit, such as when the depth camera is integrated into In smart terminals such as mobile phones, TVs, and computers, the processing circuit in the terminal can be used as at least a part of the control andprocessing circuit 13.

控制与处理电路13用于提供光源111(或泛光光源模组14中的光源等)发射激光时所需的调制信号(发射信号)，光源在调制信号的控制下向目标物体发射光束。比如在一个实施例中，调制信号为连续波信号如正弦波信号，光源在该正弦波信号的调制下振幅在时序上以正弦波形式发射；在一个实施例中，调制信号为方波信号或脉冲信号，光源在该调制信号的调制下振幅被时序上调制以产生方波信号或者脉冲信号向外发射。The control andprocessing circuit 13 is used to provide a modulation signal (transmission signal) required by the light source 111 (or the light source in the flood light source module 14) to emit laser light, and the light source emits a light beam to a target object under the control of the modulation signal. For example, in one embodiment, the modulation signal is a continuous wave signal such as a sine wave signal, and the amplitude of the light source is modulated by the sine wave signal in the form of a sine wave in time sequence. In one embodiment, the modulation signal is a square wave signal or Pulse signal. The amplitude of the light source is modulated in time sequence under the modulation of the modulation signal to generate a square wave signal or a pulse signal for external emission.

此外，控制与处理电路13还提供TOF图像传感器121各像素中各抽头的解调信号(采集信号)，抽头在解调信号的控制下采集由包含目标物体反射回的反射的光束所产生的电信号。该电信号与反射光束的强度相关，控制与处理电路13随后对该电信号进行处理并计算出反映反射光束强度的强度信息以形成结构光图案，最后基于该结构光图案进行像匹配计算、三角法计算等计算以获得目标物 体的深度图像。In addition, the control andprocessing circuit 13 also provides the demodulated signal (collection signal) of each tap in each pixel of theTOF image sensor 121. The taps collect the electricity generated by the reflected light beam containing the target object under the control of the demodulated signal. signal. The electrical signal is related to the intensity of the reflected light beam. The control andprocessing circuit 13 then processes the electrical signal and calculates the intensity information reflecting the intensity of the reflected light beam to form a structured light pattern. Finally, based on the structured light pattern, it performs image matching calculations and triangulation. Method calculation and other calculations to obtain the depth image of the target object.

在一些实施例中，深度测量装置10还包括泛光光源模组14，用于向外发射时序上振幅被调制的泛光光束。控制与处理电路13可以同步对泛光光源模组14与TOF图像传感器121进行调制与解调，以实现传统TOF测距方法以获取目标的深度图像，即通过对泛光光束由发射到被接收所需要的时间来进行深度值计算的方式。In some embodiments, thedepth measuring device 10 further includes a floodlight source module 14 for emitting flood light beams whose amplitude is modulated in time series. The control andprocessing circuit 13 can synchronously modulate and demodulate the floodlight source module 14 and theTOF image sensor 121, so as to realize the traditional TOF ranging method to obtain the depth image of the target, that is, through the flood light beam from being emitted to being received It takes the time to calculate the depth value.

在一些实施例中，深度测量装置10还可以包括驱动电路、电源、彩色相机、红外相机、IMU等器件，在图中并未示出，与这些器件的组合可以实现更加丰富的功能，比如3D纹理建模、红外人脸识别、SLAM等功能。深度测量装置10可以被嵌入到手机、平板电脑、计算机等电子产品中。In some embodiments, thedepth measurement device 10 may also include a drive circuit, a power supply, a color camera, an infrared camera, an IMU, and other devices, which are not shown in the figure. The combination with these devices can achieve richer functions, such as 3D. Texture modeling, infrared face recognition, SLAM and other functions. Thedepth measuring device 10 can be embedded in electronic products such as mobile phones, tablet computers, and computers.

图2是根据本发明第一实施例的深度测量装置原理示意图。在控制与处理电路13的控制下，光发射模组11向目标物体发射振幅被正弦调制的结构光光束30，在本实施例中结构光光束30为不规则斑点结构光，即由多个光斑202以不规则排列形成的斑点结构光图案化光束201，各个光斑202的振幅在时序上被正弦调制。采集模组12中的TOF图像传感器的各个像素至少包括4个抽头，分别用于在单个帧周期内分别采集4次光信号并转换成电信号C1、C2、C3和C4，4次采集的时间以及间隔相同。Fig. 2 is a schematic diagram of the principle of the depth measuring device according to the first embodiment of the present invention. Under the control of the control andprocessing circuit 13, thelight emitting module 11 emits a structuredlight beam 30 whose amplitude is sinusoidally modulated to the target object. In this embodiment, the structuredlight beam 30 is an irregular spot structured light, which is composed of multiple light spots. The light patternedlight beam 201 with a spot structure formed in anirregular arrangement 202, and the amplitude of eachlight spot 202 is sinusoidally modulated in time sequence. Each pixel of the TOF image sensor in theacquisition module 12 includes at least 4 taps, which are respectively used to collect 4 optical signals and convert them into electrical signals C1, C2, C3 and C4 in a single frame period. The time of 4 acquisitions And the same interval.

控制与处理电路13接收电信号C1、C2、C3和C4计算出结构光光束的强度信息。在一个实施例中的，强度信息根据下式计算：The control and processing circuit 13 receives the electrical signals C1, C2, C3, and C4 to calculate the intensity information of the structured light beam. In one embodiment, the intensity information is calculated according to the following formula:

可以理解的是，这一计算方式类似于传统未被振幅调制的结构光光束，即在单个帧周期内通过单抽头连续积分的方式获取光信号的强度信息，只不过这里进行了平均。在获取所有像素的强度信息之后，就可以形成结构光图案了，最后再利用结构光图案进行匹配计算以获取视差以及根据视差计算出深度图像。It is understandable that this calculation method is similar to the traditional structured light beam that is not amplitude modulated, that is, the intensity information of the optical signal is obtained by means of single-tap continuous integration within a single frame period, except that it is averaged here. After obtaining the intensity information of all pixels, the structured light pattern can be formed, and finally the structured light pattern is used for matching calculation to obtain the parallax and calculate the depth image according to the parallax.

对于存在环境光信号时，这一光束强度计算方式与传统方式一样，均难以进行消除，从而导致最终的结构光图案信噪比较低。因此，在一个实施例，强度信息将根据下式计算：When there is an ambient light signal, this beam intensity calculation method is the same as the traditional method, which is difficult to eliminate, resulting in a lower signal-to-noise ratio of the final structured light pattern. Therefore, in one embodiment, the intensity information will be calculated according to the following formula:

上述信息实质上计算的是振幅调制信号的振幅值，这一振幅值计算由于进行了抽头之间的相减以及平方根，在非常小时间范围内可以看成环境光不变，因此相减正好可以十分有效地去除由环境光引起的噪声，同时该振幅值也非常完好地反映出了目标物体表面纹理所对应的反射光强度信息。可以理解的是，也可以通过对振幅进行平方等处理以得到光束的强度信息。The above information essentially calculates the amplitude value of the amplitude modulation signal. The calculation of this amplitude value is due to the subtraction between taps and the square root, which can be regarded as the ambient light unchanged in a very small time range, so the subtraction is just fine It effectively removes the noise caused by ambient light, and the amplitude value also perfectly reflects the reflected light intensity information corresponding to the surface texture of the target object. It is understandable that the intensity information of the light beam can also be obtained by squaring the amplitude.

上述基于4抽头的TOF图像传感器以及正弦波调制光发射信号的结构光图案获取方案同样也适用于其他抽头TOF图像传感器以及其他类型调制光发射信号的深度测量装置。The above-mentioned structured light pattern acquisition scheme based on the 4-tap TOF image sensor and the sine wave modulated light emission signal is also applicable to other tapped TOF image sensors and other types of depth measurement devices with modulated light emission signals.

如图3所示，是根据本发明第二实施例的深度测量装置原理示意图。在控制与处理电路13的控制下，光发射模组11向目标物体发射振幅被方波或脉冲调制的发射光束30为结构光光束，在本实施例中结构光光束为不规则斑点结构光，即由多个光斑202以不规则排列形成的斑点结构光图案化光束201，各个光斑202的振幅在时序上被方波或脉冲调制。采集模组12中的TOF图像传感器的各个像素至少包括3个抽头，分别用于在单个帧周期内分别采集3次光信号并转换成电信号C1、C2和C3，第一抽头的触发时间与光脉冲的发射时间同步，且触发时长与脉冲的宽度相同，第二、三抽头在第一抽头结束时相继被触发，触发时长与第一抽头相同。As shown in Fig. 3, it is a schematic diagram of the principle of the depth measuring device according to the second embodiment of the present invention. Under the control of the control andprocessing circuit 13, thelight emitting module 11 emits a structuredlight beam 30 whose amplitude is modulated by a square wave or pulse to the target object. In this embodiment, the structured light beam is irregular spot structured light. That is, the spot structured light patternedlight beam 201 formed by a plurality oflight spots 202 in an irregular arrangement, and the amplitude of eachlight spot 202 is modulated by a square wave or pulse in time sequence. Each pixel of the TOF image sensor in theacquisition module 12 includes at least 3 taps, which are used to collect and convert the optical signal three times in a single frame period into electrical signals C1, C2, and C3. The trigger time of the first tap and The emission time of the light pulse is synchronized, and the trigger duration is the same as the pulse width. The second and third taps are triggered one after another at the end of the first tap, and the trigger duration is the same as the first tap.

由此，反射回的结构光光束的将被第一、第二、第三抽头采集到，控制与处理电路13接收电信号C1、C2和C3计算出结构光光束的强度信息。在一个实施例中的，强度信息根据下式计算：Thus, the reflected structured light beam will be collected by the first, second, and third taps, and the control and processing circuit 13 receives the electrical signals C1, C2, and C3 to calculate the intensity information of the structured light beam. In one embodiment, the intensity information is calculated according to the following formula:

同样地，这种利用加权平均的方式虽然可以计算出反映结构光光束强度的数值，但却与传统方法一样无法消除背景光所引起的噪声问题。因此在一个实施例中，强度信息将根据下式计算：Similarly, although this method of using weighted average can calculate the value reflecting the intensity of the structured light beam, it cannot eliminate the noise problem caused by the background light like the traditional method. Therefore, in one embodiment, the intensity information will be calculated according to the following formula:

B＝C₁+C₂-2C₃     (4)B=C₁ +C₂ -2C₃ (4)

可以理解的是，也可以对上述取平均等计算以获取强度信息。与式(3)相比，由于第三抽头采集的是环境光信号，因此通过相减就可以有效消除环境光的影响从而提升信噪比。It is understandable that the above average can also be calculated to obtain the intensity information. Compared with formula (3), since the third tap collects the ambient light signal, the influence of ambient light can be effectively eliminated by subtraction, thereby improving the signal-to-noise ratio.

以上图2，3所示实施例中，分别举例了发射端以正弦波或者脉冲调制的方式，接收端采用4抽头或者3抽头的采集方式。在进行结构光图案强度信息计算时，均可以采用加权平均或者加入环境光消除因子的方式进行计算。可以理解的是，类似的发射端调制方式、接收端采集方式以及强度信息计算方式的组合均包含在本发明的范围之内。In the embodiments shown in Figures 2 and 3 above, the transmitter uses sine wave or pulse modulation, and the receiver uses 4-tap or 3-tap acquisition. When calculating the intensity information of the structured light pattern, a weighted average or an ambient light elimination factor can be used for calculation. It can be understood that the combination of similar modulation methods at the transmitting end, collection methods at the receiving end, and intensity information calculation methods are all included in the scope of the present invention.

与传统的结构光深度测量方法相比，本发明发射端采用了振幅调制，在接收端采用了多抽头像素采集，从而可以使得该方法拥有比传统方案更多的功能，比如可以实现传统方案难以实现的抗环境光干扰的深度测量方法。Compared with the traditional structured light depth measurement method, the present invention adopts amplitude modulation at the transmitting end and multi-tap pixel acquisition at the receiving end, so that the method can have more functions than the traditional scheme, for example, it can realize the difficulty of the traditional scheme. Realized depth measurement method against ambient light interference.

基于上述实施例的深度测量装置，还可以实现结构光与TOF结合的融合深度测量方案。图4是根据本发明一个实施例的融合TOF与结构光方案的深度测 量装置的结构示意图。深度测量装置40包括光发射模组41、采集模组42、泛光光源模组44以及控制与处理电路43。与前述各实施例相似，其中由光发射模组41、采集模组42以及控制与处理电路43可以组成结构光深度测量子单元，其获取深度图像的原理与图1～3所示实施例相同。此外，深度测量装置40中由泛光光源模组44、采集模组42以及控制与处理电路43组成了TOF深度测量子单元，类似地，泛光光源模组44中的光源同样被控制与处理电路43以一定的时序进行振幅调制，比如在一个实施例中，为了方便设计控制与处理电路43，泛光光源模组44中的光源与光发射模组41中的光源以相同的方式被调制，比如振幅调制强度、调制波型等参数均相同。Based on the depth measurement device of the foregoing embodiment, a fusion depth measurement solution combining structured light and TOF can also be realized. Fig. 4 is a schematic structural diagram of a depth measuring device integrating TOF and structured light solutions according to an embodiment of the present invention. Thedepth measuring device 40 includes a light emitting module 41, a collectingmodule 42, a floodlight source module 44 and a control andprocessing circuit 43. Similar to the foregoing embodiments, the light emitting module 41, theacquisition module 42 and the control andprocessing circuit 43 can form a structured light depth measurement subunit, and the principle of acquiring depth images is the same as that of the embodiments shown in FIGS. 1 to 3 . In addition, thedepth measurement device 40 consists of the floodlight source module 44, theacquisition module 42 and the control andprocessing circuit 43 to form a TOF depth measurement sub-unit. Similarly, the light source in the floodlight source module 44 is also controlled and processed. Thecircuit 43 performs amplitude modulation at a certain timing. For example, in one embodiment, in order to facilitate the design of the control andprocessing circuit 43, the light source in the floodlight source module 44 and the light source in the light emitting module 41 are modulated in the same manner , Such as amplitude modulation intensity, modulation waveform and other parameters are the same.

控制与处理电路43也可以包含多个子电路分别对应于结构光深度测量子单元以及TOF深度测量子单元。TOF深度测量子单元与结构光深度测量子单元不同的是控制与处理电路43中所执行的深度计算原理，对于结构光深度测量子单元而言，如前述各实施例所述，控制与处理电路43对由采集模组42所形成的电信号进行处理并计算出反映反射光束强度的强度信息以形成结构光图案，再基于结构光图案利用结构光三角法进行深度值计算；而对于TOF深度测量子单元而言，控制与处理电路43则对采集模组42所形成的电信号进行处理以计算出相位差，并基于该相位差计算反射光束由发射端发射到接收端接收所用的飞行时间，进一步基于该飞行时间计算出目标的深度值。控制与处理电路43可以是高度集成的电路元器件，比如FPGA、SOC、ASIC等芯片，其内部包含可分别进行结构光计算以及TOF计算的电路模块，本发明同样提供了一个具体的实施例，将在图7所示的实施例中进行详述。The control andprocessing circuit 43 may also include a plurality of sub-circuits respectively corresponding to the structured light depth measurement sub-unit and the TOF depth measurement sub-unit. The difference between the TOF depth measurement subunit and the structured light depth measurement subunit is the depth calculation principle executed in the control andprocessing circuit 43. For the structured light depth measurement subunit, as described in the foregoing embodiments, the control andprocessing circuit 43 Process the electrical signal formed by thecollection module 42 and calculate the intensity information reflecting the intensity of the reflected beam to form a structured light pattern, and then use the structured light triangulation method to calculate the depth value based on the structured light pattern; and for TOF depth measurement For the sub-unit, the control andprocessing circuit 43 processes the electrical signal formed by theacquisition module 42 to calculate the phase difference, and calculates the flight time of the reflected beam from the transmitting end to the receiving end based on the phase difference. The depth value of the target is further calculated based on the flight time. The control andprocessing circuit 43 can be highly integrated circuit components, such as FPGA, SOC, ASIC and other chips, which contain circuit modules that can perform structured light calculation and TOF calculation respectively. The present invention also provides a specific embodiment. This will be described in detail in the embodiment shown in FIG. 7.

回到图4，在本实施例中，结构光深度测量子单元以及TOF深度测量子单元分别用于测量范围在ΔZ₁以及ΔZ₂内的目标，并且ΔZ₁≠ΔZ₂。在一个实施例中，ΔZ₁为近距，ΔZ₂为远距，即由结构光深度测量子单元对近距的目标物体进行测量，由TOF深度测量子单元对远距的目标物体进行测量。由于结构光算法精度在近距离时较高且随距离增加会急速降低甚至无法测量，而TOF算法则在近距离测量时深度分辨率较低、而对远距离拥有较高的精度。可以同时启动两个子单元进行测量，比如前后帧切换的形式，在分别获取结构光深度图像以及TOF深度图像之后，可以将两幅深度图像进行融合以获取大范围、高精度的深度图像，融合方式可以采用基于置信度的加权平均等方式。比如在一个实施例中，根据相对精度设计置信度，对于结构光深度图像而言，深度值越大，置信度越低，而对于TOF深度图像而言，深度值越大，置信度越高。本实施例的方案所带来的有益效果是，可以使得深度测量装置40实现更大的测量范围，同时在大范围内均保持较高的测量精度。Returning to FIG. 4, in this embodiment, the structured light depth measurement subunit and the TOF depth measurement subunit are respectively used to measure targets within ΔZ₁ and ΔZ₂ , and ΔZ₁ ≠ΔZ₂ . In one embodiment, ΔZ₁ is a short distance, and ΔZ₂ is a long distance, that is, the structured light depth measurement subunit measures a short distance target object, and the TOF depth measurement subunit measures a long distance target object. Because the accuracy of the structured light algorithm is high at short distances and will rapidly decrease or even impossible to measure with the increase of distance, the TOF algorithm has lower depth resolution when measuring at short distances, and has higher accuracy for long distances. Two subunits can be started for measurement at the same time, such as the form of switching between front and rear frames. After obtaining the structured light depth image and the TOF depth image respectively, the two depth images can be fused to obtain a large-scale, high-precision depth image. A weighted average based on confidence level can be used. For example, in one embodiment, the confidence is designed according to the relative accuracy. For structured light depth images, the greater the depth value, the lower the confidence, and for TOF depth images, the greater the depth value, the higher the confidence. The beneficial effect brought by the solution of this embodiment is that thedepth measuring device 40 can achieve a larger measurement range while maintaining high measurement accuracy in a large range.

图5是根据本发明又一实施例的融合TOF与结构光方案的深度测量装置的 结构示意图。该深度测量装置同样包含结构光深度测量子单元与TOF深度测量子单元，可以理解的是，TOF深度测量子单元即可以是由泛光光源模组与采集模组组成，也可以与结构光深度测量子单元相同由光发射模组以及采集模组组成，区别在于控制与处理电路对电信号的处理方式不同，对于TOF深度测量子单元而言，其计算的是相位信息。在本实施例中，默认TOF深度测量子单元由泛光光源模组与采集模组组成，其可以获取采集模组各个像素的TOF深度值。Fig. 5 is a schematic structural diagram of a depth measurement device integrating TOF and structured light solutions according to another embodiment of the present invention. The depth measurement device also includes a structured light depth measurement subunit and a TOF depth measurement subunit. It can be understood that the TOF depth measurement subunit can be composed of a flood light source module and a collection module, or can be combined with a structured light depth measurement subunit. The measurement subunit is the same composed of a light emitting module and an acquisition module. The difference is that the control and processing circuit processes the electrical signal in a different way. For the TOF depth measurement subunit, it calculates phase information. In this embodiment, the default TOF depth measurement subunit is composed of a flood light source module and an acquisition module, which can obtain the TOF depth value of each pixel of the acquisition module.

由于结构光深度计算需要进行匹配计算，匹配计算是整个深度计算环节中最消耗资源也是对精度影响最大的环节。匹配计算一般包括初始深度值估计、迭代优化、亚像素高精度深度值计算等几个步骤，其中初始深度值估计直接影响到后面步骤的效率与计算精度。本实施例中，将由TOF深度测量子单元所获取的TOF深度值作为结构光深度计算的初始深度值，如图5所示，采集模组中的TOF图像传感器包含多个像素51，光发射模组所发射的斑点光束经目标物体反射后成像在TOF图像传感器的一些像素上形成光斑52，可以理解的是，这里假定光斑52大致占2x2＝4个像素，实际上光斑52可以是其他大小，在此不做限定。首先通过TOF深度测量子单元获取各个像素51的TOF深度值(对于由光发射模组及采集模组组成的TOF深度测量子单元而言，只能获取光斑52对应像素的TOF深度值D1～D4)，其次通过结构光深度测量子单元采集包含各个斑点图案的结构光图案，最后进行结构光深度值计算时将TOF深度值作为匹配计算的初始深度值d。Since structured light depth calculation requires matching calculation, matching calculation is the most resource-consuming link in the entire depth calculation link and the link that has the greatest impact on accuracy. Matching calculation generally includes several steps such as initial depth value estimation, iterative optimization, and sub-pixel high-precision depth value calculation. The initial depth value estimation directly affects the efficiency and calculation accuracy of the subsequent steps. In this embodiment, the TOF depth value acquired by the TOF depth measurement subunit is used as the initial depth value for structured light depth calculation. As shown in FIG. 5, the TOF image sensor in the acquisition module includes a plurality ofpixels 51, and the light emission mode The spot beams emitted by the group of light beams are reflected by the target object and then imaged to form aspot 52 on some pixels of the TOF image sensor. It is understandable that it is assumed that thespot 52 roughly occupies 2x2=4 pixels. In fact, thespot 52 can be of other sizes. There is no limitation here. First obtain the TOF depth value of eachpixel 51 through the TOF depth measurement subunit (for the TOF depth measurement subunit composed of the light emitting module and the acquisition module, only the TOF depth value D1～D4 of the pixel corresponding to thelight spot 52 can be obtained ), secondly, the structured light pattern containing each spot pattern is collected by the structured light depth measurement subunit, and the TOF depth value is used as the initial depth value d of the matching calculation when the structured light depth value is calculated finally.

由于TOF深度值本身就具备较高的精度，以此作为结构光深度匹配计算的初始值，再进行迭代计算实现亚像素的深度值估计，可以实现比单独结构光或者TOF计算更高精度的深度计算。Since the TOF depth value itself has high accuracy, it is used as the initial value of the structured light depth matching calculation, and then iterative calculation is carried out to realize the sub-pixel depth value estimation, which can achieve a higher precision depth than the structured light alone or TOF calculation. Calculation.

图4、5所示实施例中均涉及到结构光深度测量子单元与TOF深度测量子单元的融合，在实际操作过程中，两个子单元将以一定的时序进行工作，比如图6是根据本发明一个实施例的结构光与TOF时序控制示意图，在本实施例中，结构光与TOF以一定的频率切换工作，即在周期T1、T3、T5等中深度测量装置以结构光模式进行工作，而在周期T2、T4、T6等中以TOF模式进行工作。可以理解的是，根据实际需要也可以通过其他时序进行控制，比如对于图5所示实施而言，可以TOF一帧、结构光两帧的方式进行控制。The embodiments shown in Figures 4 and 5 all involve the fusion of the structured light depth measurement subunit and the TOF depth measurement subunit. In actual operation, the two subunits will work in a certain time sequence. For example, Figure 6 is based on this A schematic diagram of the timing control of structured light and TOF according to an embodiment of the invention. In this embodiment, the structured light and TOF are switched to work at a certain frequency, that is, the depth measuring device works in structured light mode during periods T1, T3, T5, etc., In cycles T2, T4, T6, etc., it works in TOF mode. It is understandable that other timings can be used for control according to actual needs. For example, for the implementation shown in FIG. 5, the control can be performed in the manner of one frame of TOF and two frames of structured light.

图7是根据本发明一个实施例的控制与处理电路架构示意图。为了尽可能提升计算效率以及降低功耗，将一些计算、控制功能设计成专用芯片，比如FPGA、ASIC或者SOC芯片的形式是非常有效的方式。本实施例提供一种可实现上述各实施例中控制或计算功能的专用控制与处理电路芯片架构。芯片70用于接收来自TOF图像传感器的电信号，通过内部各个模块的计算后输出深度图像，在计算过程中也会与其他器件进行数据通信，比如存储器等。Fig. 7 is a schematic diagram of a control and processing circuit architecture according to an embodiment of the present invention. In order to improve computing efficiency and reduce power consumption as much as possible, it is very effective to design some computing and control functions into dedicated chips, such as FPGA, ASIC or SOC chips. This embodiment provides a dedicated control and processing circuit chip architecture that can realize the control or calculation functions in the foregoing embodiments. Thechip 70 is used to receive electrical signals from the TOF image sensor, and output a depth image after calculation by various internal modules. During the calculation process, it will also communicate with other devices, such as memory.

芯片70可以实现多重模式深度计算，TOF深度计算模式、结构光深度计算模式以及融合深度计算模式。Thechip 70 can implement multi-mode depth calculation, TOF depth calculation mode, structured light depth calculation mode, and fusion depth calculation mode.

(1)TOF深度计算模式。(1) TOF depth calculation mode.

当在TOF深度计算模式下进行TOF深度计算时，芯片70接收来自TOF图像传感器的电信号，比如图2或图3所示实施例中的像素各个抽头所采集到的电信号值。电信号将首先进入分路器74-1，分路器74-1在主控制器(图中未示出)的控制下根据当前的模式选择性让电信号进入下一计算模块，在TOF深度计算模式下电信号将进入相位计算模块71。相位计算模块71执行对电信号的相位计算以获取相位值，该相位值与深度值之间存在线性关系，因此在一些实施例中，该相位计算模块也可以直接计算出深度值。随后相位值被送入标定模块进行标定，由于TOF测量常常受到噪声干扰，使得测量值与实际值之间存在一定的误差，因此在实际使用前将采用一个标定步骤，比如在一定的测量区间内每隔一段距离设置标定板，并且标定板的实际深度值已知，随后逐次对不同距离上的标定板进行实际测量得到各个距离对应的测量值，测量值与实际值之间的关系就可以作为预标定参数被存储到存储器中，标定模块在标定时将从存储器中调用预标定参数对当前测量值进行标定。这里的预标定参数即可以是实际值与测量值的对照表(index)，此时标定模块72标定过程实际上是查表过程；也可以是通过一定的数学手段对误差进行建模，并通过预先的多次测量以计算得到模型中的未知参数，标定模块72的标定过程实际上就是基于模型、测量值计算出实际值的过程。经标定后的相位值/深度值将被送入第一后处理模块73，第一后处理模块73对相位值/深度值进行例如图像增强、插值计算等对其进行优化，比如孔洞填充、边缘优化等。被第一后处理模块73处理后的相位值/深度值经由复用器74-2输出。When TOF depth calculation is performed in the TOF depth calculation mode, thechip 70 receives electrical signals from the TOF image sensor, such as electrical signal values collected by each tap of the pixel in the embodiment shown in FIG. 2 or FIG. 3. The electrical signal will first enter the splitter 74-1. Under the control of the main controller (not shown in the figure), the splitter 74-1 selectively allows the electrical signal to enter the next calculation module according to the current mode. In the calculation mode, the electrical signal will enter thephase calculation module 71. Thephase calculation module 71 performs phase calculation of the electrical signal to obtain the phase value, and there is a linear relationship between the phase value and the depth value. Therefore, in some embodiments, the phase calculation module can also directly calculate the depth value. Then the phase value is sent to the calibration module for calibration. Because TOF measurement is often interfered by noise, there is a certain error between the measured value and the actual value. Therefore, a calibration step will be adopted before actual use, such as within a certain measurement interval. The calibration board is set at intervals, and the actual depth value of the calibration board is known. Then, the calibration board at different distances is actually measured one by one to obtain the measured value corresponding to each distance. The relationship between the measured value and the actual value can be taken as The pre-calibration parameters are stored in the memory, and the calibration module will call the pre-calibration parameters from the memory to calibrate the current measured value during calibration. The pre-calibration parameter here can be a comparison table (index) between the actual value and the measured value. At this time, the calibration process of thecalibration module 72 is actually a table look-up process; it can also be a mathematical method to model the error and pass Multiple measurements are taken in advance to calculate the unknown parameters in the model. The calibration process of thecalibration module 72 is actually a process of calculating actual values based on the model and the measured values. The calibrated phase value/depth value will be sent to thefirst post-processing module 73. Thefirst post-processing module 73 optimizes the phase value/depth value such as image enhancement, interpolation calculation, etc., such as hole filling, edge Optimization etc. The phase value/depth value processed by thefirst post-processing module 73 is output via the multiplexer 74-2.

(2)结构光深度计算模式。(2) Structured light depth calculation mode.

当在结构光深度计算模式下进行结构光深度计算时，芯片70接收来自TOF图像传感器的电信号，比如图2或图3所示实施例中的像素各个抽头所采集到的电信号值。电信号将首先进入分路器74-1，分路器74-1在主控制器(图中未示出)的控制下根据当前的模式选择性让电信号进入下一计算模块，在结构光深度计算模式下电信号将进入振幅计算模块75。振幅计算模块75执行对电信号的相位计算以获取反映光束强度的强度信息，比如振幅值。随后振幅值被送入前处理模块76进行去噪、对比度增强等处理，也可以对振幅值执行图像畸变校正等前处理任务。被前处理后的振幅值随后进入匹配模块77进行匹配计算，在进行匹配计算时匹配模块77将从存储器中调用被预先存储的参考图像，在一个实施例中，匹配模块77采用零均值归一化最小平方距离函数对振幅值对应的图像与参考图像进行像素偏离值的匹配估计。根据结构光三角法，像素偏离值与目标的深度值之间存在一定的关系，因此在一些实施例中，匹配模块77也可以直接进行深度 值的计算，当然深度值的计算也可以放在后续其他模块执行。匹配模块77输出的偏离值/深度值随后被送入第二后处理模块78进行例如图像增强、插值计算等对其进行优化，比如孔洞填充、边缘优化等。被第二后处理模块78处理后的偏离值/深度值经由复用器74-2输出。When performing structured light depth calculation in the structured light depth calculation mode, thechip 70 receives electrical signals from the TOF image sensor, such as electrical signal values collected by each tap of the pixel in the embodiment shown in FIG. 2 or FIG. 3. The electrical signal will first enter the splitter 74-1. Under the control of the main controller (not shown in the figure), the splitter 74-1 selectively allows the electrical signal to enter the next calculation module according to the current mode. In the depth calculation mode, the electrical signal will enter theamplitude calculation module 75. Theamplitude calculation module 75 performs phase calculation of the electrical signal to obtain intensity information reflecting the intensity of the beam, such as an amplitude value. Subsequently, the amplitude value is sent to thepre-processing module 76 for processing such as denoising, contrast enhancement, etc., and pre-processing tasks such as image distortion correction can also be performed on the amplitude value. The pre-processed amplitude value then enters the matching module 77 for matching calculation. During the matching calculation, the matching module 77 will recall the pre-stored reference image from the memory. In one embodiment, the matching module 77 uses zero mean normalization The minimized least square distance function estimates the pixel deviation between the image corresponding to the amplitude value and the reference image. According to the structured light triangulation method, there is a certain relationship between the pixel deviation value and the target depth value. Therefore, in some embodiments, the matching module 77 can also directly calculate the depth value. Of course, the depth value calculation can also be placed in the subsequent Other modules execute. The deviation value/depth value output by the matching module 77 is then sent to thesecond post-processing module 78 for image enhancement, interpolation calculation, etc. to optimize it, such as hole filling, edge optimization, etc. The deviation value/depth value processed by thesecond post-processing module 78 is output via the multiplexer 74-2.

(3)融合深度计算模式。(3) Fusion depth calculation mode.

当在融合深度计算模式下进行深度计算时，上述两种模式下各个模块均进行工作，并由第一后处理模块73输出TOF深度图像，以及由第二后处理模块78输出结构光深度图像，TOF深度图像与结构光深度图像随后被送入融合模块79进行融合以输出最终的融合深度图像，融合深度图像经由复用器74-2进行输出。具体融合的方式可参考图4所示实施例。When performing depth calculation in the fusion depth calculation mode, each module in the above two modes works, and thefirst post-processing module 73 outputs the TOF depth image, and thesecond post-processing module 78 outputs the structured light depth image, The TOF depth image and the structured light depth image are then sent to thefusion module 79 for fusion to output the final fusion depth image, and the fusion depth image is output through the multiplexer 74-2. For the specific fusion manner, refer to the embodiment shown in FIG. 4.

在一些实施例中，由第一后处理模块73输出的TOF深度图像后，直接进入匹配模块77，以作为匹配计算的初始值，具体的融合方案参考图5所示实施例。In some embodiments, after the TOF depth image output by thefirst post-processing module 73, it directly enters the matching module 77 as the initial value of the matching calculation. For the specific fusion scheme, refer to the embodiment shown in FIG. 5.

在一些实施例中，相位计算模块71与振幅计算模块75可以是同一个计算模块，该模块将同步基于电信号计算出相位以及振幅，相位与振幅将被直接送入下一模块计算，或者被送入分路器74-1，分路器74-1根据当前的模式将相位和/或振幅送入相应的模块进行计算。In some embodiments, thephase calculation module 71 and theamplitude calculation module 75 may be the same calculation module, which will synchronously calculate the phase and amplitude based on the electrical signal. The phase and amplitude will be directly sent to the next module for calculation, or be It is sent to the splitter 74-1, and the splitter 74-1 sends the phase and/or amplitude to the corresponding module for calculation according to the current mode.

在一些实施例中，上述三种模式可以同步进行，复用器74-2可以实时同步或连续输出不同模式所输出的深度图像。In some embodiments, the above three modes can be performed synchronously, and the multiplexer 74-2 can synchronously or continuously output the depth images output by different modes in real time.

在一些实施例中，芯片70还可以包括如总线、主控制器、输入/输出接口等模块，比如若芯片70为集成SOC芯片时，将包括这些模块，分路器74-1通过输入接口接收数据，复用器74-2的数据通过输出接口输出，输入/输出接口可以是USB、MIPI、DVP等形式的接口；总线负责SOC内部各个模块以及外部模块之间的通信、传输；主控制器则负责内部各个模块之间的资源调配与功能触发等工作。In some embodiments, thechip 70 may also include modules such as a bus, a main controller, an input/output interface, etc. For example, if thechip 70 is an integrated SOC chip, these modules will be included, and the splitter 74-1 receives the input through the input interface. Data, the data of the multiplexer 74-2 is output through the output interface, the input/output interface can be USB, MIPI, DVP and other forms of interface; the bus is responsible for the communication and transmission between the internal modules of the SOC and the external modules; the main controller It is responsible for resource allocation and function triggering among various internal modules.

基于上述各实施例中的深度测量装置，本申请还提供下述深度测量方法。Based on the depth measurement devices in the foregoing embodiments, the present application also provides the following depth measurement methods.

如图8所示，本申请中一种距离测量方法如下：As shown in Figure 8, a distance measurement method in this application is as follows:

利用光源发射时序上振幅被调制的光束，所述光束被图案化光学元件接收后形成结构光光束向目标物体发射；The light source emits a light beam whose amplitude is modulated in time sequence, and the light beam is received by the patterned optical element to form a structured light beam and emitted to the target object;

利用包含至少一个像素的TOF图像传感器的所述像素接收由所述目标物体反射的结构光光束并形成电信号；Using the pixel of the TOF image sensor containing at least one pixel to receive the structured light beam reflected by the target object and form an electrical signal;

接收所述电信号并计算出所述反射的结构光光束的强度信息以形成结构光图案，以及，利用所述结构光图案计算所述目标物体的深度图像。The electric signal is received and the intensity information of the reflected structured light beam is calculated to form a structured light pattern, and the depth image of the target object is calculated by using the structured light pattern.

如图9所示，本申请中一种距离测量方法如下：As shown in Figure 9, a distance measurement method in this application is as follows:

利用光发射模组的光源发射时序上振幅被调制的光束，所述光束被图案化光学元件接收后形成结构光光束向目标物体发射；The light source of the light emitting module is used to emit a light beam whose amplitude is modulated in time sequence, and the light beam is received by the patterned optical element to form a structured light beam and emit it to the target object;

利用泛光光源模组向所述目标物体发射时序上振幅被调制的泛光光束；Using a flood light source module to emit a flood light beam whose amplitude is modulated in time sequence to the target object;

利用包含至少一个像素的TOF图像传感器的所述像素接收由所述目标物体反射的结构光光束并形成第一电信号；以及，接收所述目标物体反射的泛光光束并形成第二电信号；Using the pixel of the TOF image sensor containing at least one pixel to receive the structured light beam reflected by the target object and form a first electrical signal; and, receive the floodlight beam reflected by the target object and form a second electrical signal;

对所述光发射模组和所述泛光光源模组的时序进行控制，按所述时序分别接收所述目标物体反射的所述结构光光束形成的所述第一电信号，以及，所述目标物体反射的所述泛光光束形成的所述第二电信号；分别对所述第一电信号和第二电信号进行处理获得所述目标物体的深度图像。Control the timing of the light emitting module and the flood light source module, and respectively receive the first electrical signal formed by the structured light beam reflected by the target object according to the timing, and The second electrical signal formed by the floodlight beam reflected by the target object; the first electrical signal and the second electrical signal are respectively processed to obtain a depth image of the target object.

如图10所示，一种实施例中对所述电信号进行处理包括：As shown in FIG. 10, in an embodiment, processing the electrical signal includes:

根据所述结构光光束形成的所述第一电信号计算出所述结构光光束的强度信息以形成结构光图案，利用所述结构光图案计算所述目标物体的第一深度图像；Calculating the intensity information of the structured light beam according to the first electrical signal formed by the structured light beam to form a structured light pattern, and calculating a first depth image of the target object by using the structured light pattern;

根据所述泛光光束形成的所述第二电信号计算出相位差，并基于所述相位差计算所述泛光光束由发射到被接收所需要的飞行时间，基于所述飞行时间计算所述目标物体的深度值，利用所述深度值获取所述目标物体的第二深度图像；The phase difference is calculated according to the second electrical signal formed by the flood light beam, and the flight time required for the flood light beam from emission to reception is calculated based on the phase difference, and the flight time is calculated based on the flight time The depth value of the target object, using the depth value to obtain a second depth image of the target object;

将所述第一深度图像和所述第二深度图像融合获得所述目标物体的深度图像。Fusing the first depth image and the second depth image to obtain a depth image of the target object.

如图11所示，另一种实施例中对所述电信号进行处理包括：As shown in FIG. 11, processing the electrical signal in another embodiment includes:

根据所述泛光光束形成的所述第二电信号计算出相位差，并基于所述相位差计算所述泛光光束由发射到被接收所需要的飞行时间，基于该所述飞行时间计算各个所述像素的TOF深度值；The phase difference is calculated according to the second electrical signal formed by the flood light beam, and the flight time required for the flood light beam from emission to reception is calculated based on the phase difference, and each flight time is calculated based on the flight time TOF depth value of the pixel;

根据所述结构光光束形成的所述第一电信号计算出所述结构光光束的强度信息以形成结构光图案；Calculating the intensity information of the structured light beam according to the first electrical signal formed by the structured light beam to form a structured light pattern;

将所述TOF深度值作为所述结构光图案匹配计算的初始深度值，利用所述结构光图案计算所述目标物体的深度图像。The TOF depth value is used as the initial depth value calculated by the structured light pattern matching calculation, and the depth image of the target object is calculated by using the structured light pattern.

本发明实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专 利实践，计算机可读介质不包括电载波信号和电信信号。The present invention implements all or part of the processes in the above-mentioned embodiment methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the steps of the foregoing method embodiments can be realized. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted in accordance with the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

本发明达到的有益效果为，提供了一种基于振幅时序调制的结构光发射端与TOF图像传感器组成的深度测量装置，并提出了基于该装置的多种深度图像获取的方案，以实现目前传统方案难以实现的高精度、大范围深度测量。同时还提供了一种深度计算芯片架构以从芯片级实现该方案，最终实现低功耗、高效率计算的目的。The beneficial effect achieved by the present invention is to provide a depth measurement device composed of a structured light emitting end based on amplitude timing modulation and a TOF image sensor, and propose a variety of depth image acquisition schemes based on the device to realize the current traditional High-precision, wide-range depth measurement that the solution is difficult to achieve At the same time, it also provides a deep computing chip architecture to realize the solution from the chip level, and finally realize the purpose of low power consumption and high efficiency computing.

以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的技术人员来说，在不脱离本发明构思的前提下，还可以做出若干等同替代或明显变型，而且性能或用途相同，都应当视为属于本发明的保护范围。The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be considered that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art to which the present invention belongs, without departing from the concept of the present invention, several equivalent substitutions or obvious modifications can be made, and the same performance or use should be regarded as belonging to the protection scope of the present invention.

Claims (10)

Translated fromChinese

深度测量装置，其特征在于，包括：The depth measuring device is characterized in that it comprises:光发射模组，包含光源及图案化光学元件，所述光源用于发射时序上振幅被调制的光束，所述图案化光学元件用于接收所述光束后向目标物体发射结构光光束；The light emitting module includes a light source and a patterned optical element, the light source is used to emit a light beam whose amplitude is modulated in time sequence, and the patterned optical element is used to receive the light beam and emit a structured light beam to a target object;TOF图像传感器，包含至少一个像素，所述像素用于接收由所述目标物体反射的结构光光束并形成电信号；The TOF image sensor includes at least one pixel, and the pixel is used to receive the structured light beam reflected by the target object and form an electrical signal;控制和处理电路，接收所述电信号并计算出所述反射的结构光光束的强度信息以形成结构光图案，以及，利用所述结构光图案计算所述目标物体的深度图像。The control and processing circuit receives the electrical signal and calculates the intensity information of the reflected structured light beam to form a structured light pattern, and uses the structured light pattern to calculate a depth image of the target object.如权利要求1所述的深度测量装置，其特征在于，所述结构光光束包含不规则斑点图案光束、条纹图案光束、二维编码图案化光束；The depth measuring device according to claim 1, wherein the structured light beam comprises an irregular spot pattern beam, a stripe pattern beam, and a two-dimensional coded patterned beam;所述不规则斑点图案光束中每个斑点对应的光束的振幅在时序上以连续波、方波或脉冲的方式被调制。The amplitude of the beam corresponding to each spot in the irregular spot pattern beam is modulated in a continuous wave, square wave or pulse manner in time sequence.如权利要求1所述的深度测量装置，其特征在于，所述像素包括至少3个抽头，所述抽头用于在单个帧周期内分别采集由包含所述目标物体反射的结构光光束所产生的电信号。The depth measuring device according to claim 1, wherein the pixel includes at least 3 taps, and the taps are used to separately collect within a single frame period generated by the structured light beam reflected by the target object. electric signal.如权利要求1所述的深度测量装置，其特征在于，所述光发射模组发射时序上振幅被正弦波调制的光束，所述TOF图像传感器的各个像素至少包括4个抽头，分别用于在单个帧周期内分别采集4次光信号并转换成所述电信号C1、C2、C3和C4，4次采集的时间以及间隔相同；The depth measuring device according to claim 1, wherein the light emitting module emits a light beam whose amplitude is modulated by a sine wave in time sequence, and each pixel of the TOF image sensor includes at least 4 taps, which are respectively used in The optical signals are collected four times in a single frame period and converted into the electrical signals C1, C2, C3, and C4, and the time and interval of the four collections are the same;采用加权平均的方式计算所述结构光光束的强度B，具体如下：The weighted average method is used to calculate the intensity B of the structured light beam, as follows:

或，采用加入环境光消除因子的方式计算所述结构光光束的强度B，具体如下：Or, calculate the intensity B of the structured light beam by adding an ambient light elimination factor, as follows:

如权利要求1所述的深度测量装置，其特征在于，所述光发射模组发射时序上振幅被脉冲调制的光束，所述TOF图像传感器的各个像素至少包括3个抽头，分别用于在单个帧周期内分别采集3次光信号并转换成电信号C1、C2和 C3，第一抽头的触发时间与光脉冲的发射时间同步，且触发时长与脉冲的宽度相同，第二抽头、第三抽头在所述第一抽头结束时相继被触发，触发时长与所述第一抽头相同；The depth measuring device according to claim 1, wherein the light emitting module emits a light beam whose amplitude is pulse-modulated in time sequence, and each pixel of the TOF image sensor includes at least 3 taps, which are respectively used for single The optical signal is collected three times in the frame period and converted into electrical signals C1, C2 and C3. The trigger time of the first tap is synchronized with the emission time of the optical pulse, and the trigger duration is the same as the pulse width. The second tap and the third tap Are triggered successively at the end of the first tap, and the trigger duration is the same as that of the first tap;采用加权平均的方式计算所述结构光光束的强度B，具体如下：The weighted average method is used to calculate the intensity B of the structured light beam, as follows:

或，采用加入环境光消除因子的方式计算所述结构光光束的强度B，具体如下：Or, calculate the intensity B of the structured light beam by adding an ambient light elimination factor, as follows:B＝C_l+C₂-2C₃。B=C₁ +C₂ -2C₃ .一种距离测量方法，其特征在于，包括：A distance measurement method, characterized in that it comprises:利用光源发射时序上振幅被调制的光束，所述光束被图案化光学元件接收后形成结构光光束向目标物体发射；The light source emits a light beam whose amplitude is modulated in time sequence, and the light beam is received by the patterned optical element to form a structured light beam and emitted to the target object;利用包含至少一个像素的TOF图像传感器的所述像素接收由所述目标物体反射的结构光光束并形成电信号；Using the pixel of the TOF image sensor containing at least one pixel to receive the structured light beam reflected by the target object and form an electrical signal;接收所述电信号并计算出所述反射的结构光光束的强度信息以形成结构光图案，以及，利用所述结构光图案计算所述目标物体的深度图像。The electric signal is received and the intensity information of the reflected structured light beam is calculated to form a structured light pattern, and the depth image of the target object is calculated by using the structured light pattern.如权利要求6所述的距离测量方法，其特征在于，所述结构光光束包含不规则斑点图案光束、条纹图案光束、二维编码图案化光束；7. The distance measurement method according to claim 6, wherein the structured light beam comprises an irregular spot pattern beam, a stripe pattern beam, and a two-dimensional coded patterned beam;所述不规则斑点图案光束中每个斑点对应的光束的振幅在时序上以连续波、方波或脉冲的方式被调制。The amplitude of the beam corresponding to each spot in the irregular spot pattern beam is modulated in a continuous wave, square wave or pulse manner in time sequence.如权利要求6所述的距离测量方法，其特征在于，所述像素包括至少3个抽头，所述抽头用于在单个帧周期内分别采集由包含所述目标物体反射的结构光光束所产生的电信号。7. The distance measuring method according to claim 6, wherein the pixel includes at least 3 taps, and the taps are used to separately collect within a single frame period generated by the structured light beam reflected by the target object. electric signal.如权利要求6所述的距离测量方法，其特征在于，所述结构光光束是时序上振幅被正弦波调制的光束，各个所述像素至少包括4个抽头，分别用于在单个帧周期内分别采集4次光信号并转换成所述电信号C1、C2、C3和C4，4次采集的时间以及间隔相同；7. The distance measurement method according to claim 6, wherein the structured light beam is a beam whose amplitude is modulated by a sine wave in time series, and each pixel includes at least 4 taps, which are used to separate each in a single frame period. Collect 4 optical signals and convert them into the electrical signals C1, C2, C3, and C4, and the time and interval of the 4 collections are the same;采用加权平均的方式计算所述结构光光束的强度B，具体如下：The weighted average method is used to calculate the intensity B of the structured light beam, as follows:

或，采用加入环境光消除因子的方式计算所述结构光光束的强度B，具体如下：Or, calculate the intensity B of the structured light beam by adding an ambient light elimination factor, as follows:

如权利要求6所述的距离测量方法，其特征在于，所述结构光光束是时序上振幅被脉冲调制的光束，各个所述像素至少包括3个抽头，分别用于在单个帧周期内分别采集3次光信号并转换成电信号C1、C2和C3，第一抽头的触发时间与光脉冲的发射时间同步，且触发时长与脉冲的宽度相同，第二抽头、第三抽头在所述第一抽头结束时相继被触发，触发时长与所述第一抽头相同；The distance measurement method according to claim 6, wherein the structured light beam is a light beam whose amplitude is pulse-modulated in time series, and each of the pixels includes at least 3 taps, which are respectively used for collecting in a single frame period. The optical signal is three times and converted into electrical signals C1, C2 and C3. The trigger time of the first tap is synchronized with the emission time of the optical pulse, and the trigger duration is the same as the pulse width. The second tap and the third tap are in the first The taps are triggered one after another at the end, and the trigger duration is the same as the first tap;采用加权平均的方式计算所述结构光光束的强度B，具体如下：The weighted average method is used to calculate the intensity B of the structured light beam, as follows:

或，采用加入环境光消除因子的方式计算所述结构光光束的强度B，具体如下：Or, calculate the intensity B of the structured light beam by adding an ambient light elimination factor, as follows:B＝C₁+C₂-2C₃。B=C₁ +C₂ -2C₃ .

Images