CN111420301A - Robotized localization and tracking system of body surface lesions - Google Patents

Abstract

The invention relates to a robot body surface focus area positioning and tracking system, which can position and track skin focuses by using artificial markers or skin surface features. Tracking the focus area by detecting the pose of the artificial marker; and searching for characteristic points in a characteristic matching mode, marking the focus area by using the characteristic points and realizing the tracking of the focus area. According to the invention, the lesion area can be automatically positioned by utilizing the characteristics on the surface layer of the skin according to the diagnosis result of a doctor, the high-precision positioning of the lesion area is realized by utilizing the high-precision three-dimensional reconstruction technology of the surface of the skin, and the treatment is automatically completed by combining the robot technology. The positioning tracking treatment of detection is realized through the pose of the identification mark, and the treatment accuracy is improved.

Description

Translated fromChinese

机器人化体表病灶区域定位跟踪系统Robotized localization and tracking system of body surface lesions

技术领域technical field

本发明属于医疗器械领域，具体说是一种机器人化体表病灶区域定位跟踪系统。The invention belongs to the field of medical instruments, in particular to a robotized body surface lesion area positioning and tracking system.

背景技术Background technique

随着基础医学与科技的发展，许多先进的科学技术都已经转化为先进的仪器设备用于皮肤病的治疗，比如利用308nm准分子激光治疗白癜风，311窄谱UVB治疗皮肤病、激光祛痘等等。但目前的设备大多存在一下几个问题：With the development of basic medicine and technology, many advanced science and technology have been transformed into advanced instruments and equipment for the treatment of skin diseases, such as the use of 308nm excimer laser to treat vitiligo, 311 narrow-band UVB to treat skin diseases, laser acne treatment, etc. Wait. However, most of the current equipment has the following problems:

1、定位不准确：目前的治疗设备大多是通过人工定位的方式将仪器的输出端放置在病变处，这样就会导致治疗范围不准确，治疗精准度不足。1. Inaccurate positioning: Most of the current treatment equipment places the output end of the instrument at the lesion by manual positioning, which will lead to inaccurate treatment range and insufficient treatment accuracy.

2、需要人工切换病变区域：当有多处病变需要治疗时，大多需要人工变换仪器位置，效率较低。2. Need to manually switch the lesion area: When there are multiple lesions that need to be treated, most of them need to manually change the position of the instrument, which is inefficient.

3、病人移动导致病变区域无法正常接受治疗：在治疗过程中，病人难免不会发生运动，从而导致病变区域的移动，治疗效果下降。3. The diseased area cannot be treated normally due to the movement of the patient: During the treatment process, the patient will inevitably not move, which will lead to the movement of the diseased area and reduce the treatment effect.

发明内容SUMMARY OF THE INVENTION

本发明目的是提供一种机器人化体表病灶区域定位跟踪系统。The purpose of the present invention is to provide a robotized body surface lesion area localization and tracking system.

本发明为实现上述目的所采用的技术方案是：机器人化体表病灶区域定位跟踪系统，该系统用于利用人工标识物或皮肤表面特征定位跟踪皮肤病灶。The technical solution adopted by the present invention to achieve the above objects is: a robotized body surface lesion area localization and tracking system, which is used to locate and track skin lesions by using artificial markers or skin surface features.

机器人化体表病灶区域定位跟踪系统，包含：Robotized body surface lesion area localization and tracking system, including:

图像采集装置，用于采集包含标识物的图像并识别标识，获取标识物位姿；识别假设病灶区域，获取病灶操作点位置；The image acquisition device is used for collecting the image containing the marker, identifying the marker, and obtaining the pose of the marker; identifying the hypothetical lesion area, and obtaining the position of the lesion operation point;

定位模块，用于根据标识物所在平面的位置和姿态得到机器人末端位姿；The positioning module is used to obtain the robot end pose according to the position and pose of the plane where the marker is located;

人机交互系统，用于人工筛选含有标识物标记的皮肤表面病灶区域；Human-computer interaction system for manual screening of skin surface lesion areas marked with markers;

机器人控制系统，用于控制机器人移动至操作点，使机器人以得到末端位姿对操作点进行操作。The robot control system is used to control the robot to move to the operation point, so that the robot can operate the operation point by obtaining the end pose.

所述采集包含标识物的图像信息并识别标识，包含以下步骤：The acquisition of the image information including the marker and the identification of the marker includes the following steps:

首先将采集到的RGB图像信息进行图像空间转换，转换到HSV颜色空间，并选取H通道作为下一步的处理数据；First, convert the collected RGB image information to the image space, convert it to the HSV color space, and select the H channel as the next processing data;

然后采用自适应阈值的方法对H通道图像进行阈值处理，转换为二值图像；提取二值图像轮廓，并利用人工标识物的几何特性筛选出人工标识物；Then adopt the method of adaptive threshold to threshold the H channel image and convert it into a binary image; extract the outline of the binary image, and use the geometric characteristics of the artificial markers to screen out the artificial markers;

再利用几何投影关系获取标识物所在平面的位置、姿态，并反馈给机器人控制系统。Then use the geometric projection relationship to obtain the position and attitude of the plane where the marker is located, and feed it back to the robot control system.

所述识别假设病灶区域，获取病灶操作点位置，包含以下步骤：The identifying the hypothetical lesion area and acquiring the position of the lesion operating point includes the following steps:

首先采集人工标识物标记皮肤表面的多光谱图像，根据人体表层皮肤组织在多光谱成像下的物理特性设定阈值，并对多光谱图像进行阈值分割，提取分割轮廓得到假设病灶区域；Firstly, collect the multispectral images of the skin surface marked by artificial markers, set thresholds according to the physical properties of human skin tissue under multispectral imaging, perform threshold segmentation on the multispectral images, and extract the segmentation contours to obtain the hypothetical lesion area;

根据假设病灶区域的轮廓、颜色、病灶分布对皮肤表面进行分析，得出的皮肤表面参数包括病灶区域占比、病灶等级中的至少一种；然后再对假设病灶区域进行表面三维重建，选取假设病灶区域内表面法线方向与人工标识物法线夹锐角最小的点作为操作点。The skin surface is analyzed according to the contour, color, and distribution of the hypothetical lesion area, and the obtained skin surface parameters include at least one of the proportion of the lesion area and the lesion grade; The point with the smallest acute angle between the normal direction of the surface of the lesion area and the normal of the artificial marker was used as the operation point.

机器人化体表病灶区域定位跟踪系统，包含：Robotized body surface lesion area localization and tracking system, including:

图像采集装置，用于对指定的病灶检测区域进行皮肤表面多光谱成像，获取假设病灶区域；The image acquisition device is used to perform multispectral imaging of the skin surface on the designated lesion detection area to obtain the hypothesized lesion area;

人机交互系统，用于人工设定病灶检测区域和人工筛选假设病灶区域；The human-computer interaction system is used to manually set the lesion detection area and manually screen the hypothetical lesion area;

定位模块，用于根据病灶周围的皮肤表面特征，定位跟踪病灶治疗点；The positioning module is used to locate and track the treatment point of the lesion according to the skin surface features around the lesion;

机器人控制系统，用于控制机器人移动至操作点，使机器人根据病灶位姿进行操作。The robot control system is used to control the robot to move to the operating point, so that the robot can operate according to the position and posture of the lesion.

所述对指定的病灶检测区域进行皮肤表面多光谱成像，获取假设病灶区域，包含以下步骤：The performing multispectral imaging of the skin surface on the designated lesion detection area to obtain a hypothetical lesion area includes the following steps:

首先采集病灶检测区域内皮肤表面的多光谱图像，根据人体表层皮肤组织在多光谱成像下的物理特性设定阈值，并对多光谱图像进行阈值分割，提取分割轮廓得到假设病灶区域；Firstly, the multispectral image of the skin surface in the lesion detection area is collected, and the threshold is set according to the physical characteristics of the human skin tissue under multispectral imaging, and the multispectral image is thresholded, and the segmentation contour is extracted to obtain the hypothetical lesion area;

根据病灶区域的轮廓、颜色、病灶分布对皮肤表面进行分析，得出的皮肤表面参数包括病灶区域占比、病灶等级中的至少一种；然后再对假设病灶区域进行表面三维重建，选取假设病灶区域内表面法线方向与人工标识物法线夹锐角最小的点作为操作点。The skin surface is analyzed according to the contour, color, and distribution of the lesion area, and the obtained skin surface parameters include at least one of the proportion of the lesion area and the lesion grade. Then, the surface of the hypothetical lesion area is reconstructed in three dimensions, and the hypothetical lesion area is selected. The point with the smallest acute angle between the normal direction of the surface in the area and the normal of the artificial marker is used as the operation point.

所述筛选假设病灶区域，并设定需治疗区域具体如下：显示分割得到的假设病灶区域位置，通过人机交互系统筛选出的假设病灶区域作为待治疗病灶区域，添加至等待队列中。The selection of the hypothetical lesion area and the setting of the area to be treated are as follows: the position of the hypothetical lesion area obtained by segmentation is displayed, and the hypothetical lesion area screened by the human-computer interaction system is used as the lesion area to be treated and added to the waiting queue.

所述利用皮肤表面特征，定位病灶位姿，具体为循环等待队列，完成对队列中全部区域的操作：循环至当前待治疗病灶区域，并在病灶周边提取特征点，利用特征匹配方法匹配到当前皮肤表面的对应病灶区域；然后再对待治疗病灶区域进行表面三维重建，并在待治疗病灶区域内选定操作点，选定规则是操作点的法线方向与所有特征点通过拟合得到的平面法线方向所构成的锐角最小。The described use of skin surface features to locate the position of the lesion, specifically a circular waiting queue, completes the operation on all areas in the queue: loops to the current lesion area to be treated, and extracts feature points around the lesion, and uses the feature matching method to match to the current area. The corresponding lesion area on the skin surface; then the surface 3D reconstruction of the lesion area to be treated is performed, and the operation point is selected in the lesion area to be treated. The selection rule is the normal direction of the operation point and the plane obtained by fitting all feature points. The acute angle formed by the normal direction is the smallest.

所述控制机器人移动至操作点进行操作，具体为：根据当前病灶操作点位姿得到机械臂末端位姿，并控制机器人末端运动至该位姿；机器人以该位姿对操作点进行操作；所述机械臂末端位姿为当机器人末端运动到该位姿之后，末端搭载的图像采集装置的镜头光轴线与病灶操作点法线重合，且治疗点位于图像中心。The controlling the robot to move to the operating point for operation, specifically: obtaining the pose of the end of the robotic arm according to the pose of the current operating point of the lesion, and controlling the end of the robot to move to the pose; the robot operates the operating point with the pose; The posture of the end of the robot arm is that after the end of the robot moves to this posture, the optical axis of the lens of the image acquisition device mounted on the end coincides with the normal of the operation point of the lesion, and the treatment point is located in the center of the image.

本发明具有以下有益效果及优点：The present invention has the following beneficial effects and advantages:

1.本发明能够根据医生的诊断结果，利用在皮肤表层特征自动定位病变区域，利用皮肤表面高精度三维重建技术，实现对病灶区域高精度定位，并自动完成治疗。1. The present invention can automatically locate the lesion area according to the diagnosis results of the doctor, using the features on the skin surface, and use the high-precision three-dimensional reconstruction technology of the skin surface to realize the high-precision positioning of the lesion area and automatically complete the treatment.

2.本发明能够根据医生的诊断结果以及医生设定的检测区域标识，通过识别标识的位姿实现对检测的定位跟踪治疗，提升治疗的精准度。2. According to the diagnosis result of the doctor and the detection area identification set by the doctor, the present invention can realize the positioning and tracking treatment of the detection by identifying the pose of the identification mark, and improve the accuracy of the treatment.

3.本发明能够利用高精度定位技术，实现对同一检测区域治疗前后的皮肤表层情况给出分析，并辅助医生对治疗效果进行评估。3. The present invention can utilize high-precision positioning technology to analyze the skin surface condition before and after treatment in the same detection area, and assist doctors in evaluating the treatment effect.

附图说明Description of drawings

图1为本发明的一种机器人化体表病灶区域定位跟踪系统工作原理图；Fig. 1 is a working principle diagram of a robotized body surface lesion area localization and tracking system of the present invention;

图2为激光治疗器与图像数据采集设备位置关系示意图；Fig. 2 is a schematic diagram of the positional relationship between the laser therapy device and the image data acquisition equipment;

图3为人工标识物辅助病灶定位跟踪流程；Fig. 3 is the artificial marker-assisted lesion location tracking process;

图4为人工标识物辅助病灶定位跟踪示意图；Fig. 4 is a schematic diagram of manual marker-assisted lesion localization and tracking;

图5为无人工标识物病灶定位跟踪流程；Fig. 5 is a process of locating and tracking lesions without artificial markers;

图6为无人工标识物病灶定位跟踪示意图。FIG. 6 is a schematic diagram of the location and tracking of lesions without artificial markers.

具体实施方式Detailed ways

下面结合实施例对本发明做进一步的详细说明。The present invention will be further described in detail below in conjunction with the embodiments.

本实施例提供一种机器人化体表病灶区域定位跟踪系统，如图1所示。其中，硬件包括：相机、光源、机械臂及其搭载的激光治疗设备；软件包括图像数据的采集、处理、分析，以及图形化的人机交互系统。This embodiment provides a robotized body surface lesion area positioning and tracking system, as shown in FIG. 1 . Among them, the hardware includes: camera, light source, robotic arm and the laser treatment equipment it carries; the software includes image data acquisition, processing, analysis, and a graphical human-computer interaction system.

该系统的主要功能是定位皮肤表面病灶，并实现机器人自动化辅助治疗。在治疗过程中，操作人员可以通过人机交互系统实时监测治疗过程，并可对治疗过程进行操作，如停止、调参、恢复等。同时，人机交互系统负责统筹全部数据采集、处理分析、机器人操作反馈及控制等，并将全部需要可视化的信息，以形象完整的形式展现给操作人员，以便操作人员对场景情况有全面的了解。除此之外，人机交互系统还提供数据存储功能，可以将治疗参数、治疗效果、病灶数据等数据分组存储，用于对患者治疗情况的全程记录。操作人员可以提取患者的历史治疗情况，并与当前情况进行对比，系统会根据对比结果给出治疗效果的数值参考。因此，人机交互系统，所有需人工干预操作，如识别区域设定、筛选病灶区域，机器人运行控制等。The main function of the system is to locate the skin surface lesions and realize robot-assisted treatment. During the treatment process, the operator can monitor the treatment process in real time through the human-computer interaction system, and can operate the treatment process, such as stopping, adjusting parameters, and recovering. At the same time, the human-computer interaction system is responsible for coordinating all data collection, processing and analysis, robot operation feedback and control, etc., and presents all the information that needs to be visualized to the operator in a complete form, so that the operator can have a comprehensive understanding of the scene. . In addition, the human-computer interaction system also provides a data storage function, which can store data such as treatment parameters, treatment effects, and lesion data in groups for recording the entire treatment of patients. The operator can extract the patient's historical treatment situation and compare it with the current situation, and the system will provide a numerical reference for the treatment effect based on the comparison result. Therefore, in the human-computer interaction system, all operations that require manual intervention, such as identifying the area setting, screening the lesion area, and robot operation control, etc.

图像采集装置用于采集皮肤表面的图像信息，其中包括多光谱成像、表面三维重建等。皮肤表面多光谱成像是利用人体表层皮肤组织在多光谱成像下的反射、散射、吸收等物理特性，获取皮肤表层不同深度的亮度信息，该信息以图像的形式存储、处理、分析；皮肤表面三维重建则是用于对皮肤表面病灶区域进行局部三维重建，以获取治疗点表面的位置、姿态。The image acquisition device is used to acquire image information of the skin surface, including multispectral imaging, surface three-dimensional reconstruction, and the like. Skin surface multispectral imaging is to use the physical properties of reflection, scattering and absorption of human surface skin tissue under multispectral imaging to obtain brightness information at different depths of the skin surface, which is stored, processed and analyzed in the form of images; three-dimensional skin surface Reconstruction is used to perform local three-dimensional reconstruction of the lesion area on the skin surface to obtain the position and posture of the surface of the treatment point.

参见图2，图2给出本系统激光治疗器与图像数据采集设备之间的一种位置关系示意图。图像采集装置与激光治疗器通过固化装置固连在一起，两者的轴线相交于A点，且A点位置满足人机安全的距离要求。在治疗的过程中，先将A点与治疗点重合，然后通过伸缩机构将激光治疗器伸出至A点，开始激光治疗过程；同时，在激光治疗器运动的过程中，还要实时检测激光治疗器所受到的力以确保人机安全。Referring to Fig. 2, Fig. 2 shows a schematic diagram of a positional relationship between the laser therapy device of the system and the image data acquisition device. The image acquisition device and the laser therapy device are fixedly connected together by the curing device, and the axes of the two intersect at point A, and the position of point A meets the requirements of human-machine safety distance. In the process of treatment, the point A and the treatment point are first overlapped, and then the laser treatment device is extended to point A through the telescopic mechanism to start the laser treatment process; at the same time, during the movement of the laser treatment device, it is necessary to detect the laser light in real time. The force on the treatment device to ensure the safety of human and machine.

本系统主要给出两种病灶定位跟踪模式，具体如下：This system mainly provides two types of lesion localization and tracking modes, as follows:

模式一：定位跟踪人工标识物，参见图3。图3展示的是该模式下的病灶定位跟踪流程，具体如下：Mode 1: Position and track artificial markers, see Figure 3. Figure 3 shows the process of lesion localization and tracking in this mode, as follows:

1.1人工标识物设定：由医生诊断并选定治疗区域之后，在病灶表面配置人工标识物，本实例提供一种可行方案如图4所示。1.1 Manual marker setting: After the doctor diagnoses and selects the treatment area, an artificial marker is arranged on the surface of the lesion. This example provides a feasible solution as shown in Figure 4.

1.2识别标识物，获取标识物位姿：设定完标识物后，将患者引领至图像采集设备下，采集包含标识物的图像信息并识别标识，识别过程如下：首先将采集到的RGB图像信息进行图像空间转换，转换到HSV颜色空间，并选取H通道作为下一步的处理数据；然后采用自适应阈值的方法对H通道图像进行阈值处理，转换为二值图像；提取二值图像轮廓，并利用人工标识物的几何特性筛选出人工标识物。标识物识别到之后，再利用几何投影关系获取标识物所在平面的位置、姿态，并反馈给机器人控制系统。1.2 Identify the marker and obtain the pose of the marker: After setting the marker, lead the patient to the image acquisition device, collect the image information containing the marker, and identify the marker. The recognition process is as follows: First, the collected RGB image information is collected. Perform image space conversion, convert to HSV color space, and select H channel as the next processing data; then use adaptive threshold method to threshold the H channel image and convert it into a binary image; extract the outline of the binary image, and The artificial markers are screened out by their geometric properties. After the marker is recognized, the position and posture of the plane where the marker is located are obtained by using the geometric projection relationship, and fed back to the robot control system.

1.3机器人控制系统控制机器人到达标识物位置。机器人控制系统利用坐标变换规则及系统标定，计算末端位姿，并控制末端达到指定位姿。所述指定位姿是指当机器人末端运动到该位姿之后，末端搭载的图像采集系统能正对于人工标识所在平面，且标识中心位于图像中心。1.3 The robot control system controls the robot to reach the position of the marker. The robot control system uses the coordinate transformation rules and system calibration to calculate the end pose and control the end to reach the specified pose. The designated pose refers to that after the robot end moves to the pose, the image acquisition system mounted on the end can face the plane where the artificial mark is located, and the mark center is located at the center of the image.

1.4病灶区域识别，获取病灶精准位置。首先利用图像采集模块采集人工标识物内部皮肤表面的多光谱信息，根据人体表层皮肤组织在多光谱成像下的反射、散射、吸收等物理特性，设定阈值并对多光谱图像进行阈值分割，提取分割轮廓得到假设病灶区域，同时会根据病灶区域的轮廓、颜色、病灶分布对皮肤表面情况进行分析，如病灶区域占比、病灶等级等；然后再利用局部表面三维重建技术，对病灶区域进行表面三维重建，选取病灶区域内表面法线方向与人工标识物法线夹锐角最小的点作为治疗点。1.4 Identify the lesion area and obtain the precise location of the lesion. Firstly, the multi-spectral information of the skin surface inside the artificial marker is collected by the image acquisition module, and the threshold is set according to the reflection, scattering, absorption and other physical properties of the skin tissue on the human surface under multi-spectral imaging, and the multi-spectral image is thresholded and extracted. The hypothetical lesion area is obtained by segmenting the contour, and the skin surface condition is analyzed according to the contour, color and lesion distribution of the lesion area, such as the proportion of the lesion area, the lesion grade, etc.; For 3D reconstruction, select the point with the smallest acute angle between the normal direction of the inner surface of the lesion area and the normal of the artificial marker as the treatment point.

1.5操作人员确认，完成治疗过程。在确定病灶位置之后，需要由操作人员在人机交互系统中确认无误之后才能开始治疗，且在治疗过程中，操作人员可以通过人机交互系统实时的控制系统，以确保治疗过程的精准、安全。1.5 The operator confirms and completes the treatment process. After determining the location of the lesion, the operator needs to confirm that it is correct in the human-computer interaction system before starting the treatment. During the treatment process, the operator can control the system in real time through the human-computer interaction system to ensure the accuracy and safety of the treatment process. .

模式二：操作人员由人机交互系统选定区域，病灶区域由其周围图像特征进行定位。如图5所示，具体流程如下：Mode 2: The operator selects the area by the human-computer interaction system, and the lesion area is located by its surrounding image features. As shown in Figure 5, the specific process is as follows:

2.1多光谱成像，病灶区域分割。操作人员由人机交互系统设定病灶检测区域，然后对指定的病灶检测区域进行皮肤表面多光谱成像，获取假设病灶区域，具体可参见1.4。2.1 Multispectral imaging, lesion area segmentation. The operator sets the lesion detection area by the human-computer interaction system, and then performs multispectral imaging of the skin surface on the designated lesion detection area to obtain the hypothetical lesion area. For details, please refer to 1.4.

2.2操作人员筛选假设病灶区域，并设定需治疗区域。人机交互系统中会显示出分割得到的假设病灶区域位置，分析得到的病灶相关参数，操作人员根据图像、分析数据等筛选假设病灶区域，并在治疗等待队列中添加待治疗区域。所述治疗等待队列是由操作人员设定的可由当前模式治疗的病灶区域图像队列，图像中只有病灶区域及其指定邻域内的皮肤表面图像。2.2 The operator screens the hypothetical lesion area and sets the area to be treated. The human-computer interaction system will display the location of the hypothetical lesion area obtained by segmentation, and analyze the relevant parameters of the lesion. The operator will filter the hypothetical lesion area according to the image and analysis data, and add the area to be treated in the treatment waiting queue. The treatment waiting queue is a queue of images of the lesion area that can be treated by the current mode set by the operator, and only the images of the skin surface in the lesion area and its designated neighborhood are in the images.

2.3利用皮肤表面特征，定位病灶位姿。循环治疗等待队列，完成对全部区域的治疗。具体治疗过程如下：循环至当前带治疗病灶区域，并在病灶周边提取特征点，如SIFT、SURF等，利用特征匹配方法匹配到当前皮肤表面的对应病灶区域；然后再利用局部表面三维重建技术，对病灶区域进行表面三维重建，并在病灶区域内选定治疗点，选定规则是治疗点的法线方向与所有特征点通过拟合得到的平面法线方向夹锐角最小。2.3 Use the skin surface features to locate the lesion pose. Circulate the treatment waiting queue to complete the treatment of all areas. The specific treatment process is as follows: cycle to the current lesion area with treatment, and extract feature points around the lesion, such as SIFT, SURF, etc., and use the feature matching method to match the corresponding lesion area on the current skin surface; then use the local surface 3D reconstruction technology, The surface of the lesion area is reconstructed three-dimensionally, and the treatment point is selected in the lesion area. The selection rule is that the normal direction of the treatment point and the plane normal direction obtained by all feature points through fitting have the smallest acute angle.

2.4机器人移动至治疗点，并保持人机安全距离。机器人控制系统根据当前病灶治疗点位姿，计算机械臂末端位姿，并控制末端运动至该位姿。所述机械臂末端位姿是指当机器人末端运动到该位姿之后，末端搭载的图像采集系统轴线与病灶治疗点法线重合，且治疗点位于图像中心，如图6。2.4 The robot moves to the treatment point and maintains a safe distance between man and machine. The robot control system calculates the position of the end of the robotic arm according to the current position of the lesion treatment point, and controls the end to move to this position. The posture of the end of the robot arm means that when the end of the robot moves to this posture, the axis of the image acquisition system mounted on the end coincides with the normal of the lesion treatment point, and the treatment point is located in the center of the image, as shown in Figure 6.

2.5操作人员确认，完成治疗过程。可参见1.5。2.5 The operator confirms and completes the treatment process. See also 1.5.

对于模式一，其是通过检测人工标识物的位姿实现对病灶区域的跟踪；对于模式二，其是通过特征匹配的方式寻找特征点，利用特征点标记病灶区域并实现对病灶区域的跟踪。For mode 1, it is to track the lesion area by detecting the pose of the artificial marker; for mode 2, it searches for feature points by feature matching, and uses the feature points to mark the lesion area and realize the tracking of the lesion area.

Claims (10)

1. A robotized body surface lesion area positioning and tracking system is characterized in that the system is used for tracking skin lesions by using artificial markers.

2. The robotic body surface lesion area locating and tracking system of claim 1, comprising:

the image acquisition device is used for acquiring an image containing the marker, identifying the marker and acquiring the pose of the marker; identifying a supposed focus area, and acquiring the position of a focus operating point;

the positioning module is used for obtaining the terminal pose of the robot according to the position and the posture of the plane where the marker is located;

the human-computer interaction system is used for manually screening the lesion area on the surface of the skin marked by the marker;

and the robot control system is used for controlling the robot to move to the operation point so that the robot operates the operation point by obtaining the terminal pose.

3. The robotic body surface lesion area locating and tracking system of claim 2, wherein the acquiring of image information including a marker and the identification of the marker comprises the steps of:

firstly, performing image space conversion on the acquired RGB image information, converting the RGB image information into HSV color space, and selecting an H channel as processing data of the next step;

then, performing threshold processing on the H channel image by adopting a self-adaptive threshold method, and converting the H channel image into a binary image; extracting a binary image contour, and screening the artificial marker by utilizing the geometric characteristic of the artificial marker;

and then, acquiring the position and the posture of the plane of the marker by using the geometric projection relation, and feeding back to the robot control system.

4. The system for positioning and tracking the lesion area on the robotic body surface according to claim 2, wherein the step of identifying the assumed lesion area and obtaining the position of the lesion operating point comprises the steps of:

firstly, collecting a multispectral image of the skin surface marked by an artificial marker, setting a threshold value according to the physical characteristics of skin tissues on the surface layer of a human body under multispectral imaging, carrying out threshold value segmentation on the multispectral image, and extracting a segmentation contour to obtain an assumed focus area;

analyzing the skin surface according to the outline, color and focus distribution of the assumed focus area, wherein the obtained skin surface parameters comprise at least one of focus area ratio and focus grade; and then, performing surface three-dimensional reconstruction on the assumed focus area, and selecting a point with the smallest acute angle between the normal direction of the inner surface of the assumed focus area and the normal of the artificial marker as an operation point.

5. A robotized body surface lesion area positioning and tracking system is characterized by being used for positioning and tracking skin lesions by using skin surface features.

6. The robotic body surface lesion area locating and tracking system of claim 5, comprising:

the image acquisition device is used for performing multispectral imaging on the skin surface of the appointed focus detection area to acquire an assumed focus area;

the human-computer interaction system is used for manually setting a focus detection area and manually screening a hypothesis focus area;

the positioning module is used for positioning and tracking a focus treatment point according to the skin surface characteristics around the focus;

and the robot control system is used for controlling the robot to move to an operation point so that the robot operates according to the position and the posture of the focus.

7. The system according to claim 6, wherein the step of performing multispectral imaging of the skin surface on the specified lesion detection area to obtain the assumed lesion area comprises the steps of:

firstly, acquiring a multispectral image of the skin surface in a focus detection area, setting a threshold value according to the physical characteristics of human body surface skin tissues under multispectral imaging, performing threshold value segmentation on the multispectral image, and extracting a segmentation contour to obtain an assumed focus area;

analyzing the skin surface according to the contour, color and focus distribution of the focus area to obtain skin surface parameters including at least one of focus area ratio and focus grade; and then, performing surface three-dimensional reconstruction on the assumed focus area, and selecting a point with the smallest acute angle between the normal direction of the inner surface of the assumed focus area and the normal of the artificial marker as an operation point.

8. The system of claim 6, wherein the system for locating and tracking the lesion area on the body surface comprises: and displaying the position of the assumed lesion region obtained by segmentation, and adding the assumed lesion region screened out by a human-computer interaction system into a waiting queue as a lesion region to be treated.

9. The system according to claim 6, wherein the robotic body surface lesion area positioning and tracking system is configured to position the pose of the lesion by using the skin surface features, specifically to cyclically wait for the queue, to complete the operations on all areas in the queue: circulating to the current focus area to be treated, extracting characteristic points at the periphery of the focus, and matching to the corresponding focus area on the current skin surface by using a characteristic matching method; and then, carrying out surface three-dimensional reconstruction on the focus area to be treated, and selecting an operation point in the focus area to be treated, wherein the selection rule is that an acute angle formed by the normal direction of the operation point and the normal direction of a plane obtained by fitting all the characteristic points is the minimum.

10. The system for positioning and tracking the lesion area on the robot body surface according to claim 6, wherein the control robot moves to an operation point for operation, specifically: obtaining the pose of the tail end of the mechanical arm according to the pose of the current focus operation point, and controlling the tail end of the robot to move to the pose; the robot operates the operation point at the pose; the terminal pose of the mechanical arm is that after the terminal of the robot moves to the pose, the optical axis of a lens of an image acquisition device carried by the terminal coincides with the normal of a focus operation point, and a treatment point is positioned in the center of an image.

Images