CN101301200A - Method for Measuring Three-Dimensional Facial Topography of Patients with Facial Defects - Google Patents

Abstract

Translated fromChinese

测量颜面缺损患者面部三维形貌的方法，属于光学测量、机械工程和医学工程技术领域。本发明基于结构光投影三维视觉测量技术，采用用于患者颜面缺损修复的双单目白光三维测量系统，控制左幻灯投影仪和右幻灯投影仪分别将格雷码和相移编码结构光条纹图案投射至颜面缺损患者左右侧面部，左摄像机和右摄像机分别拍摄发生变形的结构光条纹图像后，通过图像解码、三角原理和双单目测量数据合并，实现对完整患者面部三维形貌的测量。本发明可在非接触条件下实现对患者面部自然状态软组织的精确数据采集，为临床颜面缺损修复治疗提供实用可靠的患者面部三维形貌原始数据。

The invention discloses a method for measuring the three-dimensional shape of the face of a patient with facial defect, belonging to the technical fields of optical measurement, mechanical engineering and medical engineering. Based on the structured light projection three-dimensional vision measurement technology, the present invention adopts a dual-monocular white light three-dimensional measurement system for repairing facial defects of patients, and controls the left slide projector and the right slide projector to respectively project Gray code and phase-shift code structured light stripe patterns To the left and right sides of patients with facial defects, the left camera and the right camera capture the deformed structured light stripe images respectively, and then realize the measurement of the three-dimensional shape of the complete patient's face through image decoding, triangulation principle and combination of binocular measurement data. The present invention can realize accurate data acquisition of the soft tissue of the patient's face in a natural state under non-contact conditions, and provide practical and reliable original data of the patient's facial three-dimensional appearance for clinical facial defect repair treatment.

Description

Translated fromChinese

测量颜面缺损患者面部三维形貌的方法Method for Measuring Three-Dimensional Facial Topography of Patients with Facial Defects

所属技术领域：Technical field:

本发明涉及一种测量颜面缺损患者面部三维形貌的方法。特别是一种基于结构光投影三维视觉测量技术，采用用于患者颜面缺损修复的双单目白光三维测量系统，对颜面缺损患者的面部形貌实现完整数据采集的三维测量方法。属于光学测量、机械工程和医学工程技术领域。The invention relates to a method for measuring the three-dimensional shape of the face of a patient with facial defect. In particular, a three-dimensional vision measurement technology based on structured light projection, using a dual-monocular white light three-dimensional measurement system for facial defect repair of patients, to achieve a three-dimensional measurement method for complete data collection of facial appearance of patients with facial defects. It belongs to the technical fields of optical measurement, mechanical engineering and medical engineering.

背景技术：Background technique:

各种原因造成的颜面缺损，不仅会给患者带来生理功能的障碍，同时会使患者产生严重的心理障碍，难以进行正常的工作和社交，甚至使许多患者丧失生活的信心。由于颌面器官的特殊解剖形态、组织结构及其他影响因素，颜面大面积缺损通常采用赝复体进行修复治疗。该技术的关键在于患者颜面缺损区信息的精确获取，以满足赝复体形状，定位，与组织面贴合，边缘处理及形态学等方面的诸多精度要求。Facial defects caused by various reasons will not only hinder the physiological function of the patients, but also cause serious psychological obstacles to the patients, making it difficult to carry out normal work and social interaction, and even make many patients lose their confidence in life. Due to the special anatomical shape, tissue structure and other influencing factors of the maxillofacial organs, large facial defects are usually repaired with prosthesis. The key to this technology lies in the accurate acquisition of information on the patient's facial defect area, in order to meet many precision requirements in terms of prosthesis shape, positioning, fit with tissue surface, edge processing and morphology.

传统的患者颜面缺损区数据采集技术一直停留在印模制取的阶段。它采用各类印模材料取得患者的缺损区的缺损状况，翻制模型后，再由技师凭经验进行手工的蜡形雕刻，充填后完成。这种印模制取方法存在很多缺点：比如说印模材料会对组织产生一定的刺激性，并可能引起软组织移位，造成模型偏差；同时存在制作工艺复杂、制作周期长、成型赝复体美学效果差，定位和贴合不准等问题。The traditional data acquisition technology of the patient's facial defect area has been stuck in the stage of impression making. It uses various impression materials to obtain the defect status of the patient's defect area. After the model is reproduced, the technician performs manual wax carving based on experience, and the filling is completed. This method of making an impression has many disadvantages: for example, the impression material will irritate the tissue to a certain extent, and may cause soft tissue displacement, resulting in model deviation; Poor aesthetic effect, inaccurate positioning and fit, etc.

随着科技的发展，部分国内外学者和临床医生开始采用电脑断层扫描(CT)、核磁共振成像(MRI)、激光扫描等技术实现患者缺损颜面的三维测量和图像重建，可替代传统的印模制取方法而获得相对较精确的缺损区信息。但这些测量方法仍然具有一定的局限性。CT和MRI测量方法空间分辨率低、测量精度差，获取数据时间长，对软组织的重建有一定的欠缺，而且扫描的放射线对人体有潜在的损害。激光测量的速度同样较慢，无法确保整个测量过程中患者面部形貌的绝对静止，加上激光光源对人眼有潜在的伤害，都限制了这些方法在颜面缺损修复中的进一步应用。With the development of science and technology, some domestic and foreign scholars and clinicians have begun to use computerized tomography (CT), magnetic resonance imaging (MRI), laser scanning and other technologies to achieve three-dimensional measurement and image reconstruction of patients' defective faces, which can replace traditional impressions The relatively accurate information of the defect area can be obtained by using the preparation method. However, these measurement methods still have certain limitations. CT and MRI measurement methods have low spatial resolution, poor measurement accuracy, and long data acquisition time, which has certain deficiencies in soft tissue reconstruction, and the scanned radiation has potential damage to the human body. The speed of laser measurement is also slow, and it is impossible to ensure the absolute stillness of the patient's facial shape during the entire measurement process. In addition, the laser light source has potential damage to the human eye, which limits the further application of these methods in the repair of facial defects.

白光三维测量是近年来迅速发展的一种测量方法，它采用类似照相的原理，在几秒之内，即可获取上百万个点的测量数据。而且白光测量安全，对人体没有任何伤害，是一种有前途的人体数据测量及三维重建的好方法。然而，由于患者颜面测量的特殊性，限制了现有白光三维测量技术在这领域的应用。比如现有的白光三维测量技术通常实现单一视角的测量，要获取完整数据需要多视角测量，并进行软件拼合。已有技术中，J.

在论文Three dimensional measurementof human face with structured-light illumination(Measurement Science Review，Volume 6，Section 2，No.1，2006)中提到在使用三维颜面单目测量系统(ShapeCam)来测量多个视角面部数据后，不得不再使用ShapeMatcher软件进行拼合以获得完整面部数据。由于整个测量过程中患者姿势和面部形态不可能保持绝对静止不动，所以多视角测量后拼合会引入较大的拼合误差，从而导致患者面部测量数据的不精确。White light three-dimensional measurement is a measurement method that has developed rapidly in recent years. It uses a principle similar to photography, and can obtain measurement data of millions of points within a few seconds. Moreover, white light measurement is safe and has no harm to the human body. It is a promising method for human body data measurement and three-dimensional reconstruction. However, due to the particularity of the patient's face measurement, the application of the existing white light three-dimensional measurement technology in this field is limited. For example, the existing white light 3D measurement technology usually realizes the measurement of a single viewing angle. To obtain complete data, multi-viewing measurement and software stitching are required. In the prior art, J.

In the paper Three dimensional measurement of human face with structured-light illumination (Measurement Science Review, Volume 6,Section 2, No.1, 2006), it is mentioned that using a three-dimensional facial monocular measurement system (ShapeCam) to measure multiple perspective facial data Finally, they had to use ShapeMatcher software to stitch together to obtain complete facial data. Since the patient's posture and facial shape cannot remain absolutely still during the entire measurement process, large stitching errors will be introduced after multi-view measurement, resulting in inaccurate patient facial measurement data.

发明内容：Invention content:

为克服已有技术中颜面缺损患者面部测量存在的测量精度低、测量速度慢和对人体有潜在损害等不足，本发明根据面部测量的特点，提出了一种全新的测量颜面缺损患者面部三维形貌的方法。采用用于患者颜面缺损修复的双单目白光三维测量系统，在非接触条件下实现对患者面部自然状态软组织的精确数据采集，为临床颜面缺损修复治疗提供实用可靠的患者面部三维形貌原始数据。In order to overcome the shortcomings of low measurement accuracy, slow measurement speed and potential damage to the human body in the face measurement of patients with facial defects in the prior art, the present invention proposes a brand-new method for measuring the three-dimensional shape of the face of patients with facial defects according to the characteristics of facial measurement. Appearance method. The dual-monocular white light three-dimensional measurement system for facial defect repair is used to realize accurate data collection of the natural state soft tissue of the patient's face under non-contact conditions, and to provide practical and reliable original data of the patient's facial three-dimensional shape for clinical facial defect repair. .

本发明是通过下述技术方案实现的。本发明提出的测量颜面缺损患者面部三维形貌的方法所使用的测量设备为经过双单目白光三维测量系统标定方法标定后的用于患者颜面缺损修复的双单目白光三维测量系统。该测量系统由左单目测量头、右单目测量头、测量头支架、测量头平移机构、控制柜、计算机组成。其中左单目测量头包括左幻灯投影仪和左摄像机。右单目测量头包括右幻灯投影仪和右摄像机。左幻灯投影仪和右幻灯投影仪均由白光源、隔热玻璃、物理编码光栅、光栅平移机构、投影镜头组成。白光源经过隔热玻璃隔热，背投至物理编码光栅，将物理编码光栅的结构光条纹图案通过投影镜头投射至患者面部。其中，每幅结构光条纹图案的切换通过移动光栅平移机构实现。本发明采用的结构光条纹图案由7幅格雷码图案和4幅相移图案组合进行编码。格雷码和相移图案组合编码的方式编码健壮性好，分辨率高，既可测量突变和不连续表面，又可获得表面的精细纹理，适用于颜面缺损患者面部三维形貌的测量。The present invention is achieved through the following technical solutions. The measurement equipment used in the method for measuring the three-dimensional facial appearance of patients with facial defects proposed by the present invention is a dual-monocular white-light three-dimensional measurement system calibrated by the calibration method of the dual-monocular white-light three-dimensional measurement system for patients with facial defect repair. The measurement system consists of a left monocular measuring head, a right monocular measuring head, a measuring head bracket, a measuring head translation mechanism, a control cabinet and a computer. Wherein the left monocular measuring head includes a left slide projector and a left camera. The right monocular measuring head includes a right slide projector and a right camera. Both the left slide projector and the right slide projector are made up of white light source, heat insulating glass, physical coding grating, grating translation mechanism, and projection lens. The white light source is insulated by heat-insulating glass, and projected to the physical coding grating at the back, and the structured light stripe pattern of the physical coding grating is projected to the patient's face through the projection lens. Among them, the switching of each structured light stripe pattern is realized by moving the grating translation mechanism. The structured light fringe pattern adopted in the present invention is coded by a combination of 7 Gray code patterns and 4 phase shift patterns. The combined coding method of Gray code and phase-shift pattern has good coding robustness and high resolution. It can not only measure abrupt and discontinuous surfaces, but also obtain the fine texture of the surface. It is suitable for the measurement of the three-dimensional facial shape of patients with facial defects.

格雷码图案第一幅为黑白条纹各一半，从第二幅开始，对条纹逐渐细分：前一幅黑条纹区域按照半黑半白进行等分，前一幅白条纹区域按照半白半黑进行等分。按照黑白条纹分别表示逻辑值0和1，7幅格雷码图案可得到7位的编码序列，可将空间分成2⁷个区域，按一个方向分别对应一个整数索引值m(m＝0，1，2，...，2ⁿ-1)。格雷码图案编码获得的空间分辨率较低。The first gray code pattern has half black and half stripes, and from the second one, the stripes are gradually subdivided: the black stripe area in the previous image is divided into half black and half white, and the white stripe area in the previous image is divided into half white and half black. Make equal parts. According to thelogic values 0 and 1 respectively represented by black and white stripes, 7 Gray code patterns can obtain a 7-bit coding sequence, which can divide the space into²⁷ areas, and correspond to an integer index value m in one direction (m=0, 1, 2,...,²ⁿ -1). Gray code pattern encoding achieves lower spatial resolution.

相移图案按四步相移的编码规则，其周期和格雷码图案划分的空间区域宽度相一致，并以第一幅相移图案为基准，第二幅开始分别比前一幅偏移半个条纹宽度。投射相移图案获得的光强分布L_q(q＝0，1，2，3)由下式表示：The phase-shift pattern follows the four-step phase-shift encoding rule, and its cycle is consistent with the width of the space area divided by the Gray code pattern. Based on the first phase-shift pattern, the second one is offset by half a step from the previous one. Stripe width. The light intensity distribution L_q (q=0, 1, 2, 3) obtained by projecting the phase shift pattern is expressed by the following formula:

${\mathrm{<span><span>I</span>I</span>}}_{\mathrm{<span><span>q</span>q</span>}}<span><span>=</span>=</span>{\mathrm{<span><span>I</span>I</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>+</span>+</span>{\mathrm{<span><span>I</span>I</span>}}^{<span><span>\&prime;</span>\&prime;</span><span><span>\&prime;</span>\&prime;</span>}<span><span>\&CenterDot;</span>\&Center\; Dot;</span>\mathrm{<span><span>cos</span>cos</span>}<span><span>(</span>(</span>{\mathrm{<span><span>\&phi;</span>\&phi;</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>+</span>+</span>\mathrm{<span><span>q</span>q</span>}<span><span>\&CenterDot;</span>\&Center\; Dot;</span>\frac{\mathrm{<span><span>\&pi;</span>\&pi;</span>}}{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>$

其中，I′为背景光强，I″为条纹光强对比度，φ′为相对相位。其中相对相位φ′可由下式计算得到：Among them, I' is the background light intensity, I" is the fringe light intensity contrast, φ' is the relative phase. The relative phase φ' can be calculated by the following formula:

${\mathrm{<span><span>\&phi;</span>\&phi;</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>=</span>=</span>\mathrm{<span><span>arctg</span>arctg</span>}\frac{{\mathrm{<span><span>I</span>I</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>I</span>I</span>}}_{\mathrm{<span><span>3</span>3</span>}}}{{\mathrm{<span><span>I</span>I</span>}}_{\mathrm{<span><span>0</span>0</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>I</span>I</span>}}_{\mathrm{<span><span>2</span>2</span>}}}$

相移图案编码在每个格雷码划分的空间区域内具有极高的空间分辨率。因为主相位周期和格雷码空间区域完全重合，所以这两种编码的组合可通过以下式子得到：The phase-shift pattern encoding has extremely high spatial resolution in the spatial region divided by each Gray code. Because the main phase period and the gray code space region coincide completely, the combination of these two codes can be obtained by the following formula:

φ＝2π·m+φ′φ＝2π·m+φ′

其中φ为组合编码的绝对相位。由此实现了空间光线投射方向的高分辨率编码，建立了这些方向和绝对相位之间对应关系。where φ is the absolute phase of the combined code. This enables high-resolution encoding of spatial ray projection directions and establishes a correspondence between these directions and absolute phases.

本发明提出的测量颜面缺损患者面部三维形貌的方法具体包括如下步骤：The method for measuring the three-dimensional facial appearance of patients with facial defects proposed by the present invention specifically includes the following steps:

1)调整左单目测量头和右单目测量头的位置以及调整左摄像机和右摄像机拍摄曝光量参数。开启左幻灯投影仪和右幻灯投影仪，投射结构光条纹图案至患者面部，打开左摄像机和右摄像机进行实时场景拍摄。控制测量头平移机构，将左单目测量头和右单目测量头上下和前后平移至合适测量位置，使患者面部同时位于左摄像机和右摄像机拍摄场景中心。根据拍摄图像的结构光条纹质量，调整左摄像机和右摄像机拍摄曝光量参数。1) Adjust the position of the left monocular measuring head and the right monocular measuring head and adjust the shooting exposure parameters of the left camera and the right camera. Turn on the left and right slide projectors to project structured light stripe patterns onto the patient's face, and turn on the left and right cameras for real-time scene shooting. Control the translation mechanism of the measuring head, and move the left monocular measuring head and the right monocular measuring head up and down and back and forth to a suitable measurement position, so that the patient's face is simultaneously in the center of the shooting scene of the left camera and the right camera. Adjust the exposure parameters of the left camera and the right camera according to the structured light streak quality of the captured image.

2)左单目测量头测量。关闭右幻灯投影仪，移动左幻灯投影仪的光栅平移机构，左幻灯投影仪投射一幅结构光条纹图案至患者左侧面部。左摄像机采集投射至患者左侧面部的结构光条纹图案。如果已完成所有结构光条纹图案的投射，将左幻灯投影仪的光栅平移机构回归原位；如果没有完成所有结构光条纹图案的投射，控制左幻灯投影仪的光栅平移机构移动，再投射一幅结构光条纹图案，左摄像机再次进行采集，直至所有结构光条纹图案投射完毕。2) Measure with the left monocular measuring head. Turn off the right slide projector, move the grating translation mechanism of the left slide projector, and the left slide projector projects a structured light stripe pattern to the patient's left face. The left camera captures the structured light fringe pattern projected onto the patient's left face. If the projection of all structured light fringe patterns has been completed, return the grating translation mechanism of the left slide projector to its original position; if the projection of all structured light fringe patterns has not been completed, control the movement of the grating translation mechanism of the left slide projector and project another For the structured light fringe pattern, the left camera collects again until all the structured light fringe patterns are projected.

3)右单目测量头测量。关闭左幻灯投影仪，开启右幻灯投影仪，移动右幻灯投影仪的光栅平移机构，右幻灯投影仪投射一幅结构光条纹图案至患者右侧面部。右摄像机采集投射到患者右侧面部的结构光条纹图案。如果已完成所有结构光条纹图案的投射，将右幻灯投影仪的光栅平移机构回归原位；如果没有完成所有结构光条纹图案的投射，控制右幻灯投影仪的光栅平移机构移动，再投射一幅结构光条纹图案，右摄像机再次进行采集，直至所有结构光条纹图案投射完毕。3) Measure with the right monocular measuring head. Turn off the left slide projector, turn on the right slide projector, move the grating translation mechanism of the right slide projector, and the right slide projector projects a structured light stripe pattern to the patient's right face. The right camera captures the structured light fringe pattern projected onto the patient's right face. If the projection of all structured light stripe patterns has been completed, return the grating translation mechanism of the right slide projector to its original position; if the projection of all structured light stripe patterns has not been completed, control the movement of the grating translation mechanism of the right slide projector and project another The structured light fringe pattern, the right camera collects again until all the structured light fringe patterns are projected.

4)结构光条纹图像处理、三维坐标点计算及结果显示。4) Structured light fringe image processing, three-dimensional coordinate point calculation and result display.

由于患者面部三维形貌的调制作用，投射的结构光条纹图案发生变形。这些变形的结构光条纹图案包含有面部三维形貌信息。可通过分别分析左摄像机和右摄像机拍摄获取的变形结构光条纹图像，实现患者面部形貌的三维重建。Due to the modulation effect of the patient's facial three-dimensional topography, the projected structured light fringe pattern is deformed. These deformed structured light fringe patterns contain facial three-dimensional topography information. The three-dimensional reconstruction of the patient's facial appearance can be realized by separately analyzing the deformed structured light fringe images captured by the left camera and the right camera.

对左摄像机拍摄的结构光条纹图像按照格雷码和相移编码规则进行解码，得到左摄像机每个图像像素点的绝对相位。根据每个图像像素点的绝对相位值可确定该图像像素点对应的左幻灯投影仪空间光线投射方向。The structured light fringe image captured by the left camera is decoded according to the Gray code and phase shift coding rules, and the absolute phase of each image pixel of the left camera is obtained. According to the absolute phase value of each image pixel point, the projection direction of the left slide projector space light corresponding to the image pixel point can be determined.

图像像素点位置和左摄像机参数，可确定左摄像机的成像直线；左幻灯投影仪空间光线投射方向和左幻灯投影仪参数可确定左幻灯投影仪的光线投射平面；根据左摄像机、左幻灯投影仪分别与左单目测量头测量坐标系的旋转和平移位姿转换矩阵，可将左摄像机的成像直线和左幻灯投影仪的光线投射平面转换至左单目测量头测量坐标系，求得该直线和平面的交点，即为患者左侧面部三维点在左单目测量头测量坐标系下空间位置。The image pixel position and the left camera parameters can determine the imaging straight line of the left camera; the left slide projector space light projection direction and the left slide projector parameters can determine the left slide projector light projection plane; according to the left camera, left slide projector Respectively with the rotation and translation pose transformation matrix of the left monocular measuring head measurement coordinate system, the imaging straight line of the left camera and the ray projection plane of the left slide projector can be transformed into the left monocular measuring head measurement coordinate system, and the straight line can be obtained The intersection point of and the plane is the spatial position of the three-dimensional point on the patient's left face in the coordinate system of the left monocular measuring head.

对右摄像头拍摄的结构光条纹图像的分析处理方式与上述对左摄像机拍摄的结构光条纹图像的处理完全一致，最终获得患者右侧面部三维点在右单目测量头测量坐标系下空间位置。The analysis and processing method of the structured light fringe image captured by the right camera is exactly the same as the above-mentioned processing of the structured light fringe image captured by the left camera, and finally the spatial position of the three-dimensional point on the patient's right face in the measurement coordinate system of the right monocular measuring head is obtained.

根据左单目测量头和右单目测量头测量坐标系间旋转和平移位姿转换矩阵关系，将患者右侧面部三维坐标点转换至左单目测量头测量坐标系，或将患者左侧面部三维坐标点转换至右单目测量头测量坐标系，从而获得完整的患者面部三维形貌数据，并在计算机上以点云形式进行显示，便于人机交互操作。According to the rotation and translation transformation matrix relationship between the left monocular measuring head and the right monocular measuring head measurement coordinate system, the three-dimensional coordinate points on the patient’s right face are transformed into the left monocular measuring head measurement coordinate system, or the patient’s left face The three-dimensional coordinate points are converted to the coordinate system of the right monocular measuring head, so as to obtain the complete three-dimensional shape data of the patient's face, and display it in the form of point cloud on the computer, which is convenient for human-computer interaction.

本发明的有益效果：Beneficial effects of the present invention:

本发明采用用于患者颜面缺损修复的双单目白光三维测量系统，能在非接触条件下实现自然状态患者面部软组织完整数据的快速采集，其测量精度在0.03mm以内，测量时间少于8s，可获得近200万数量三维点云。该测量方法可取代传统颜面缺损修复中赝复体制作的印模制取工艺过程，大大缩短治疗周期，减少了技师工作强度。由于这些数据直接来自患者自然状态，制成的赝复体与患者颜面缺损部位可做到十分精确的吻合，极大地减少由于人为原因造成的误差，提高赝复体制作精度，降低了手术的风险性。相对于CT、MRI、激光等技术，采用白光作为背投光源，对人体不产生任何潜在伤害。The present invention adopts a dual-monocular white light three-dimensional measurement system for repairing facial defects of patients, which can realize rapid collection of complete data of facial soft tissues of patients in a natural state under non-contact conditions. The measurement accuracy is within 0.03mm, and the measurement time is less than 8s. Nearly 2 million 3D point clouds can be obtained. This measurement method can replace the impression making process of prosthesis production in traditional facial defect repair, greatly shortening the treatment cycle and reducing the work intensity of technicians. Since these data come directly from the patient's natural state, the prosthesis made can be very accurately matched with the patient's facial defect, which greatly reduces the error caused by human factors, improves the accuracy of prosthesis production, and reduces the risk of surgery sex. Compared with CT, MRI, laser and other technologies, white light is used as the back-projection light source, which does not cause any potential harm to the human body.

附图说明：Description of drawings:

图1本发明测量方法流程框图Fig. 1 flow chart of measuring method of the present invention

图2本发明使用的双单目白光三维测量系统示意图Fig. 2 schematic diagram of the dual-monocular white light three-dimensional measurement system used in the present invention

图3本发明幻灯投影仪结构示意图Fig. 3 structural schematic diagram of slide projector of the present invention

图中1左单目测量头、2右单目测量头、3测量头支架、4测量头平移机构、5控制柜、6计算机、7左幻灯投影仪、8左摄像机、9右幻灯投影仪、10右摄像机、11白光源、12隔热玻璃、13物理编码光栅、14光栅平移机构、15投影镜头、16患者颜面In the figure, 1 left monocular measuring head, 2 right monocular measuring head, 3 measuring head bracket, 4 measuring head translation mechanism, 5 control cabinet, 6 computer, 7 left slide projector, 8 left camera, 9 right slide projector, 10 right camera, 11 white light source, 12 heat insulating glass, 13 physical coding grating, 14 grating translation mechanism, 15 projection lens, 16 patient face

具体实施方式：Detailed ways:

下面结合附图对本发明的具体实施做进一步描述。The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.

如图2和图3所示，本发明提出的测量颜面缺损患者面部三维形貌的方法所使用的测量设备为用于患者颜面缺损修复的双单目白光三维测量系统。该测量系统由左单目测量头1、右单目测量头2、测量头支架3、测量头平移机构4、控制柜5、计算机6组成。其中左单目测量头1包括左幻灯投影仪7和左摄像机8。右单目测量头2包括右幻灯投影仪9和右摄像机10。左幻灯投影仪7和右幻灯投影仪9均由白光源11、隔热玻璃12、物理编码光栅13、光栅平移机构14、投影镜头15组成。白光源11经过隔热玻璃12隔热，背投至物理编码光栅13上，将物理编码光栅13的结构光条纹图案通过投影镜头15投射至患者面部16。其中，每幅结构光条纹图案的切换通过移动光栅平移机构14实现。As shown in Fig. 2 and Fig. 3, the measuring equipment used in the method for measuring the three-dimensional facial appearance of patients with facial defects proposed by the present invention is a binocular white light three-dimensional measurement system for repairing facial defects of patients. The measurement system consists of a leftmonocular measuring head 1, a rightmonocular measuring head 2, a measuring head support 3, a measuring head translation mechanism 4, a control cabinet 5 and a computer 6. Wherein the leftmonocular measuring head 1 includes a left slide projector 7 and a left camera 8 . The rightmonocular measuring head 2 includes a right slide projector 9 and aright camera 10 . The left slide projector 7 and the right slide projector 9 are all made up ofwhite light source 11 ,heat insulating glass 12 , physical encoding grating 13 , gratingtranslation mechanism 14 , andprojection lens 15 . Thewhite light source 11 is insulated by the heat-insulatingglass 12 , and is rear-projected onto the physical encoding grating 13 , and the structured light stripe pattern of the physical encoding grating 13 is projected onto the patient'sface 16 through theprojection lens 15 . Wherein, the switching of each structured light fringe pattern is realized by moving thegrating translation mechanism 14 .

该测量系统使用双单目白光三维测量系统标定方法标定，标定的参数包括：左摄像机8和右摄像机10的焦距和主点，左幻灯投影仪7和右幻灯投影仪9的焦距和主相位，左摄像机8和左幻灯投影仪7分别相对于左单目测量头1测量坐标系的旋转和平移位姿转换矩阵，右摄像机10和右幻灯投影仪9分别相对于右单目测量头2测量坐标系的旋转和平移位姿转换矩阵，左单目测量头1和右单目测量头2测量坐标系间旋转和平移位姿转换矩阵。The measurement system is calibrated using a dual-monocular white light three-dimensional measurement system calibration method, and the parameters of the calibration include: the focal length and principal point of the left camera 8 and theright camera 10, the focal length and the principal phase of the left slide projector 7 and the right slide projector 9, The left camera 8 and the left slide projector 7 respectively measure the rotation and translation pose transformation matrix of the coordinate system relative to the leftmonocular measuring head 1, and theright camera 10 and the right slide projector 9 measure the coordinates respectively relative to the rightmonocular measuring head 2 The rotation and translation pose transformation matrix of the system, the leftmonocular measuring head 1 and the rightmonocular measuring head 2 measure the rotation and translation pose transformation matrix between the coordinate systems.

如图1所示，本发明提出的测量颜面缺损患者面部三维形貌的方法具体包括如下步骤：As shown in Figure 1, the method for measuring the three-dimensional facial appearance of patients with facial defects proposed by the present invention specifically includes the following steps:

1)调整左单目测量头1和右单目测量头2位置以及调整左摄像机8和右摄像机10拍摄曝光量参数。开启左幻灯投影仪7和右幻灯投影仪9，投射结构光条纹图案至患者面部，打开左摄像机8和右摄像机10进行实时场景拍摄。控制测量头平移机构4，将左单目测量头1和右单目测量头2上下和前后平移至合适测量位置，使患者面部同时位于左摄像机8和右摄像机10拍摄场景中心。根据拍摄图像中结构光条纹图像质量，调整左摄像机8和右摄像机10拍摄曝光量参数。1) Adjust the positions of the leftmonocular measuring head 1 and the rightmonocular measuring head 2 and adjust the shooting exposure parameters of the left camera 8 and theright camera 10 . The left slide projector 7 and the right slide projector 9 are turned on to project the structured light stripe pattern to the patient's face, and the left camera 8 and theright camera 10 are turned on for real-time scene shooting. Control the measuring head translation mechanism 4 to translate the leftmonocular measuring head 1 and the rightmonocular measuring head 2 up and down and back and forth to a suitable measurement position, so that the patient's face is located at the center of the shooting scene of the left camera 8 and theright camera 10 at the same time. According to the image quality of the structured light fringe in the captured image, adjust the parameters of the shooting exposure of the left camera 8 and theright camera 10 .

2)左单目测量头1测量。关闭右幻灯投影仪9，移动左幻灯投影仪7的光栅平移机构14，左幻灯投影仪7投射一幅结构光条纹图案至患者左侧面部。左摄像机8采集投射至患者左侧面部的结构光条纹图案。如果已完成所有结构光条纹图案的投射，将左幻灯投影仪7的光栅平移机构14回归原位；如果没有完成所有结构光条纹图案的投射，控制左幻灯投影仪7的光栅平移机构14移动，再投射一幅结构光条纹图案，左摄像机8再次进行采集，直至所有结构光条纹图案投射完毕。2) Measure with the leftmonocular measuring head 1. Turn off the right slide projector 9, move thegrating translation mechanism 14 of the left slide projector 7, and the left slide projector 7 projects a structured light stripe pattern to the patient's left face. The left camera 8 collects the structured light fringe pattern projected onto the patient's left face. If the projection of all structured light stripe patterns has been completed, thegrating translation mechanism 14 of the left slide projector 7 is returned to its original position; if the projection of all structured light stripe patterns has not been completed, thegrating translation mechanism 14 of the left slide projector 7 is controlled to move, Project another structured light fringe pattern, and the left camera 8 collects again until all the structured light fringe patterns are projected.

3)右单目测量头2测量。关闭左幻灯投影仪7，开启右幻灯投影仪9，移动右幻灯投影仪9的光栅平移机构14，右幻灯投影仪9投射一幅结构光条纹图案至患者右侧面部。右摄像机10采集投射到患者右侧面部的结构光条纹图案。如果已完成所有结构光条纹图案的投射，将右幻灯投影仪9的光栅平移机构14回归原位；如果没有完成所有结构光条纹图案的投射，控制右幻灯投影仪9的光栅平移机构14移动，再投射一幅结构光条纹图案，右摄像机10再次进行采集，直至所有结构光条纹图案投射完毕。3) Measure with the rightmonocular measuring head 2. Turn off the left slide projector 7, turn on the right slide projector 9, move thegrating translation mechanism 14 of the right slide projector 9, and the right slide projector 9 projects a structured light stripe pattern to the patient's right face. Theright camera 10 collects the structured light fringe pattern projected onto the patient's right face. If the projection of all structured light stripe patterns has been completed, thegrating translation mechanism 14 of the right slide projector 9 is returned to its original position; if the projection of all structured light stripe patterns has not been completed, thegrating translation mechanism 14 of the right slide projector 9 is controlled to move, Another structured light fringe pattern is projected, and theright camera 10 collects again until all the structured light fringe patterns are projected.

4)结构光条纹图像处理、三维坐标点计算及结果显示。4) Structured light fringe image processing, three-dimensional coordinate point calculation and result display.

对左摄像机8拍摄的结构光条纹图像按照格雷码和相移编码规则进行解码，得到左摄像机8每个图像像素点的绝对相位。根据每个图像像素点的绝对相位值可确定该图像像素点对应的左幻灯投影仪7空间光线投射方向。The structured light fringe image captured by the left camera 8 is decoded according to Gray code and phase shift coding rules to obtain the absolute phase of each image pixel of the left camera 8 . According to the absolute phase value of each image pixel point, the spatial light projection direction of the left slide projector 7 corresponding to the image pixel point can be determined.

图像像素点位置和左摄像机8参数，可确定左摄像机8的成像直线；左幻灯投影仪7空间光线投射方向和左幻灯投影仪7参数可确定左幻灯投影仪7的光线投射平面；根据左摄像机8、左幻灯投影仪7分别与左单目测量头1测量坐标系之间旋转和平移位姿转换矩阵，可将左摄像机8的成像直线和左幻灯投影仪7的光线投射平面转换至左单目测量头1测量坐标系，求得该直线和平面的交点，即为患者左侧面部三维点在左单目测量头1测量坐标系下空间位置。The image pixel position and the left camera 8 parameters can determine the imaging straight line of the left camera 8; the left slide projector 7 space light projection direction and the left slide projector 7 parameters can determine the light projection plane of the left slide projector 7; according to the left camera 8. The rotation and translation pose transformation matrix between the left slide projector 7 and the leftmonocular measuring head 1 measurement coordinate system can convert the imaging straight line of the left camera 8 and the light projection plane of the left slide projector 7 to the left monocular Theeye measuring head 1 measures the coordinate system, and the intersection point of the straight line and the plane is obtained, which is the spatial position of the three-dimensional point on the patient's left face in the leftmonocular measuring head 1 measuring coordinate system.

对右摄像头10拍摄的结构光条纹图像的分析处理方式与上述对左摄像机8拍摄的结构光条纹图像的处理完全一致，最终获得患者右侧面部三维点在右单目测量头2测量坐标系下空间位置。The analysis and processing method of the structured light fringe image captured by theright camera 10 is completely consistent with the above-mentioned processing of the structured light fringe image captured by the left camera 8, and finally the three-dimensional point on the right side of the patient's face is obtained under the measurement coordinate system of the rightmonocular measuring head 2 Spatial location.

根据左单目测量头1和右单目测量头2测量坐标系间旋转和平移位姿转换矩阵关系，将患者右侧面部三维坐标点转换至左单目测量头1测量坐标系下，或将患者左侧面部三维坐标点转换至右单目测量头2测量坐标系下，从而获得完整的患者面部三维形貌数据，并在计算机上以点云形式进行显示，便于人机交互操作。According to the rotation and translation transformation matrix relationship between the leftmonocular measuring head 1 and the rightmonocular measuring head 2 measurement coordinate system, the three-dimensional coordinate points on the patient's right face are converted to the leftmonocular measuring head 1 measuring coordinate system, or The three-dimensional coordinate points on the left side of the patient's face are converted to the coordinate system of the rightmonocular measuring head 2, so as to obtain the complete three-dimensional topography data of the patient's face, and display it in the form of point cloud on the computer, which is convenient for human-computer interaction.

Claims (4)

Translated fromChinese

1.一种测量颜面缺损患者面部三维形貌的方法，其特征在于该测量方法使用了用于患者颜面缺损修复的双单目白光三维测量系统；该测量系统的参数经过双单目白光三维测量系统标定方法的标定；测量患者面部三维形貌的方法包括如下步骤：1. A method for measuring the facial three-dimensional appearance of patients with facial defects, characterized in that the measurement method uses a double-monocular white light three-dimensional measurement system for patient facial defect repair; the parameters of the measurement system are measured by double-monocular white light three-dimensional The calibration of the system calibration method; the method for measuring the three-dimensional shape of the patient's face comprises the following steps:1)调整左单目测量头(1)和右单目测量头(2)位置以及调整左摄像机(8)和右摄像机(10)拍摄曝光量参数：开启左幻灯投影仪(7)和右幻灯投影仪(9)，投射结构光条纹图案至患者面部；打开左摄像机(8)和右摄像机(10)进行实时场景拍摄；控制测量头平移机构(4)，将左单目测量头(1)和右单目测量头(2)上下和前后平移，使患者面部同时位于左摄像机(8)和右摄像机(10)拍摄场景中心；根据拍摄图像中结构光条纹图像质量，调整左摄像机(8)和右摄像机(10)拍摄曝光量参数；1) Adjust the position of the left monocular measuring head (1) and the right monocular measuring head (2) and adjust the shooting exposure parameters of the left camera (8) and the right camera (10): turn on the left slide projector (7) and the right slide The projector (9) projects the structured light stripe pattern onto the patient's face; the left camera (8) and the right camera (10) are turned on for real-time scene shooting; the measuring head translation mechanism (4) is controlled, and the left monocular measuring head (1) and the right monocular measuring head (2) to move up and down and back and forth, so that the patient's face is simultaneously located in the center of the shooting scene of the left camera (8) and the right camera (10); according to the image quality of the structured light stripes in the captured image, adjust the left camera (8) Shoot exposure parameters with the right camera (10);2)左单目测量头(1)测量：关闭右幻灯投影仪(9)，移动左幻灯投影仪(7)的光栅平移机构(14)，左幻灯投影仪(7)投射一幅结构光条纹图案至患者左侧面部；左摄像机(8)采集投射至患者左侧面部的结构光条纹图案；如果已完成所有结构光条纹图案的投射，将左幻灯投影仪(7)的光栅平移机构(14)回归原位；如果没有完成所有结构光条纹图案的投射，控制左幻灯投影仪(7)的光栅平移机构(14)移动，再投射一幅结构光条纹图案，左摄像机(8)再次进行采集，直至所有结构光条纹图案投射完毕；2) Left monocular measuring head (1) measurement: turn off the right slide projector (9), move the grating translation mechanism (14) of the left slide projector (7), and the left slide projector (7) projects a structured light stripe pattern to the left face of the patient; the left camera (8) collects the structured light fringe pattern projected on the patient's left face; if the projection of all the structured light fringe patterns has been completed, the grating translation mechanism (14) of the left slide projector (7) ) returns to the original position; if the projection of all structured light stripe patterns is not completed, control the grating translation mechanism (14) of the left slide projector (7) to move, and then project a structured light stripe pattern, and the left camera (8) collects again , until all the structured light stripe patterns are projected;3)右单目测量头(2)测量：关闭左幻灯投影仪(7)，开启右幻灯投影仪(9)，移动右幻灯投影仪(9)的光栅平移机构(14)，右幻灯投影仪(9)投射一幅结构光条纹图案至患者右侧面部；右摄像机(10)采集投射到患者右侧面部的结构光条纹图案；如果已完成所有结构光条纹图案的投射，将右幻灯投影仪(9)的光栅平移机构(14)回归原位；如果没有完成所有结构光条纹图案的投射，控制右幻灯投影仪(9)的光栅平移机构(14)移动，再投射一幅结构光条纹图案，右摄像机(10)再次进行采集，直至所有结构光条纹图案投射完毕；3) Right monocular measuring head (2) measurement: turn off the left slide projector (7), turn on the right slide projector (9), move the grating translation mechanism (14) of the right slide projector (9), and the right slide projector (9) Project a structured light stripe pattern to the patient's right side face; the right camera (10) collects the structured light stripe pattern projected onto the patient's right side face; if the projection of all the structured light stripe patterns has been completed, the right slide projector The grating translation mechanism (14) of (9) returns to its original position; if the projection of all structured light stripe patterns has not been completed, control the movement of the grating translation mechanism (14) of the right slide projector (9), and then project a structured light stripe pattern , the right camera (10) collects again until all the structured light stripe patterns are projected;4)结构光条纹图像处理、三维坐标点计算及结果显示：4) Structured light fringe image processing, three-dimensional coordinate point calculation and result display:对左摄像机(8)拍摄的结构光条纹图像按照格雷码和相移编码规则进行解码，得到左摄像机(8)每个图像像素点的绝对相位；根据每个图像像素点的绝对相位值可确定该图像像素点对应的左幻灯投影仪(7)空间光线投射方向；The structured light fringe image captured by the left camera (8) is decoded according to Gray code and phase shift coding rules to obtain the absolute phase of each image pixel of the left camera (8); it can be determined according to the absolute phase value of each image pixel The left slide projector (7) space light projection direction corresponding to this image pixel point;图像像素点位置和左摄像机(8)参数可确定左摄像机(8)的成像直线；左幻灯投影仪(7)空间光线投射方向和左幻灯投影仪(7)参数可确定左幻灯投影仪(7)的光线投射平面；根据左摄像机(8)、左幻灯投影仪(7)分别与左单目测量头(1)测量坐标系的旋转和平移位姿转换矩阵，可将左摄像机(8)的成像直线和左幻灯投影仪(7)的光线投射平面转换至左单目测量头(1)测量坐标系，求得该直线和平面的交点，即为患者左侧面部三维点在左单目测量头(1)测量坐标系下空间位置；Image pixel point position and left camera (8) parameter can determine the imaging straight line of left camera (8); ) ray projection plane; according to the rotation and translation pose transformation matrix of the left camera (8), left slide projector (7) and left monocular measuring head (1) measurement coordinate system respectively, the left camera (8) can be The imaging straight line and the ray projection plane of the left slide projector (7) are converted to the left monocular measuring head (1) measurement coordinate system, and the intersection point of the straight line and the plane is obtained, which is the three-dimensional point on the left side of the patient's face measured with the left monocular. The head (1) measures the spatial position in the coordinate system;对右摄像头(10)拍摄的结构光条纹图像的分析处理方式与上述对左摄像机(8)拍摄的结构光条纹图像的处理完全一致，最终获得患者右侧面部三维点在右单目测量头(2)测量坐标系下空间位置；The analysis and processing method of the structured light fringe image taken by the right camera (10) is exactly the same as the above-mentioned processing of the structured light fringe image taken by the left camera (8), and finally the three-dimensional point on the right side of the patient's face is obtained at the right monocular measuring head ( 2) Measure the spatial position in the coordinate system;根据左单目测量头(1)和右单目测量头(2)测量坐标系间旋转和平移位姿转换矩阵关系，将患者右侧面部三维坐标点转换至左单目测量头(1)测量坐标系下，或将患者左侧面部三维坐标点转换至右单目测量头(2)测量坐标系下，从而获得完整患者面部三维形貌数据，并在计算机上以点云形式进行显示。According to the transformation matrix relationship between the left monocular measuring head (1) and the right monocular measuring head (2) to measure the rotation and translation pose transformation matrix, the three-dimensional coordinate points on the right side of the patient's face are converted to the measurement of the left monocular measuring head (1) coordinate system, or convert the three-dimensional coordinate points of the patient’s left face to the coordinate system of the right monocular measuring head (2), so as to obtain the complete three-dimensional facial data of the patient, and display it in the form of point cloud on the computer.2.根据权利要求1所述的测量颜面缺损患者面部三维形貌的方法，其特征是所述的用于患者颜面缺损修复的双单目白光三维测量系统包括左单目测量头(1)、右单目测量头(2)、测量头支架(3)、测量头平移机构(4)、控制柜(5)和计算机(6)；其中左单目测量头(1)包括左幻灯投影仪(7)和左摄像机(8)，右单目测量头(2)包括右幻灯投影仪(9)和右摄像机(10)；左幻灯投影仪(7)和右幻灯投影仪(9)均由白光源(11)、隔热玻璃(12)、物理编码光栅(13)、光栅平移机构(14)、投影镜头(15)组成；白光源(11)经过隔热玻璃(12)隔热，背投至物理编码光栅(13)上，将物理编码光栅(13)的结构光条纹图案通过投影镜头(15)投射至患者面部(16)。2. The method for measuring the facial three-dimensional topography of patients with facial defects according to claim 1, characterized in that the described dual-monocular white light three-dimensional measuring system for repairing facial defects of patients comprises a left monocular measuring head (1), Right monocular measuring head (2), measuring head bracket (3), measuring head translation mechanism (4), control cabinet (5) and computer (6); wherein left monocular measuring head (1) includes left slide projector ( 7) and left camera (8), right monocular measuring head (2) comprises right slide projector (9) and right camera (10); Left slide projector (7) and right slide projector (9) are all made of white Light source (11), heat-insulating glass (12), physical coding grating (13), grating translation mechanism (14), projection lens (15); white light source (11) is heat-insulated by heat-insulating glass (12), rear projection onto the physical coding grating (13), and project the structured light fringe pattern of the physical coding grating (13) onto the patient's face (16) through a projection lens (15).3.根据权利要求1所述的测量颜面缺损患者面部三维形貌的方法，其特征是所述的结构光条纹图案由7幅格雷码图案和4幅相移图案组成，每幅结构光条纹图案的切换是通过控制光栅平移机构(14)的移动来实现。3. The method for measuring the facial three-dimensional topography of patients with facial defects according to claim 1, wherein the structured light fringe pattern is composed of 7 Gray code patterns and 4 phase shift patterns, each structured light fringe pattern The switching is realized by controlling the movement of the grating translation mechanism (14).4.根据权利要求1所述的测量颜面缺损患者面部三维形貌的方法，其特征是所述的双单目白光三维测量系统标定方法标定的测量系统参数包括：左摄像机(8)和右摄像机(10)的焦距和主点，左幻灯投影仪(7)和右幻灯投影仪(9)的焦距和主相位，左摄像机(8)和左幻灯投影仪(7)分别相对于左单目测量头(1)测量坐标系的旋转和平移位姿转换矩阵，右摄像机(10)和右幻灯投影仪(9)分别相对于右单目测量头(2)测量坐标系的旋转和平移位姿转换矩阵，左单目测量头(1)和右单目测量头(2)测量坐标系间旋转和平移位姿转换矩阵。4. The method for measuring the facial three-dimensional topography of patients with facial defects according to claim 1, characterized in that the measurement system parameters demarcated by the two-monocular white light three-dimensional measurement system calibration method include: left camera (8) and right camera The focal length and principal point of (10), the focal length and principal phase of the left slide projector (7) and the right slide projector (9), the left camera (8) and the left slide projector (7) are respectively measured relative to the left monocular The rotation and translation pose transformation matrix of the head (1) measurement coordinate system, the rotation and translation pose transformations of the right camera (10) and the right slide projector (9) relative to the right monocular measurement head (2) measurement coordinate system Matrix, the rotation and translation pose transformation matrix between the measurement coordinate systems of the left monocular measuring head (1) and the right monocular measuring head (2).

Images