CN115330736A - Method and device for matching medical images - Google Patents
Method and device for matching medical imagesDownload PDFInfo
- Publication number
- CN115330736A CN115330736ACN202210997485.0ACN202210997485ACN115330736ACN 115330736 ACN115330736 ACN 115330736ACN 202210997485 ACN202210997485 ACN 202210997485ACN 115330736 ACN115330736 ACN 115330736A
- Authority
- CN
- China
- Prior art keywords
- matching
- feature data
- data
- image
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30016—Brain
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及图像处理技术领域，更具体地，涉及一种医学图像的匹配方法及装置。The present invention relates to the technical field of image processing, and more specifically, to a matching method and device for medical images.
背景技术Background technique
在脑部医疗导航系统中，为了实现精准的导航，需要建立患者医疗影像中头部位置和患者实际头部位置的变换关系。利用患者医疗影像中的面部点云和实时采集的患者面部点云进行配准，可以得到导航系统需要的变换关系。但在治疗过程中，患者的头部可能会移动，这对导航系统的处理能力和反应速度提出了更高的要求。In the brain medical navigation system, in order to achieve accurate navigation, it is necessary to establish the transformation relationship between the patient's head position in the medical image and the patient's actual head position. The transformation relationship required by the navigation system can be obtained by using the facial point cloud in the patient's medical image and the patient's facial point cloud collected in real time for registration. However, during the treatment process, the patient's head may move, which puts forward higher requirements on the processing power and response speed of the navigation system.
相关技术中，利用ICP(Iterative Closest Point，最近点迭代)算法计算患者医疗影像中的面部点云和实时采集的患者面部点云之间的关系。ICP算法，是基于数据配准法，利用最近点搜索法，从而解决基于自由形态曲面的一种算法。具体地，ICP算法需要遍历两组点云中所有的点查找最适合的匹配点。因此，所以ICP算法所需的配准时间与两组点云中点的数量呈现正相关趋势。点云数量越多，需要的配准时间越长。而由上述可知，由于患者的头部不可能完全保持静止，因此，动态图像的获取时实时的。动态图像中的点云的数量变化较大，不仅会导致配准的相应速度降低，鲁棒性差，还会导致配准的时间变化有较大波动。In related technologies, an ICP (Iterative Closest Point, Iterative Closest Point) algorithm is used to calculate the relationship between the face point cloud in the patient's medical image and the patient's face point cloud collected in real time. The ICP algorithm is based on the data registration method and uses the closest point search method to solve an algorithm based on free-form surfaces. Specifically, the ICP algorithm needs to traverse all the points in the two sets of point clouds to find the most suitable matching point. Therefore, the registration time required by the ICP algorithm is positively correlated with the number of points in the two sets of point clouds. The larger the number of point clouds, the longer the registration time required. However, it can be known from the above that since the head of the patient cannot be kept still completely, the acquisition of dynamic images is real-time. The number of point clouds in the dynamic image changes greatly, which will not only reduce the corresponding speed of registration and poor robustness, but also cause large fluctuations in the time change of registration.
发明内容Contents of the invention
有鉴于此，本发明提供了一种医学图像的匹配方法及装置，能够降低配准的时间，提高配准的效率和鲁棒性。In view of this, the present invention provides a medical image matching method and device, which can reduce registration time and improve registration efficiency and robustness.
第一方面，本发明提供一种医学图像的匹配方法，该方法包括：In a first aspect, the present invention provides a method for matching medical images, the method comprising:
提取第一影像中的源特征数据，并获取第一匹配关系和第二影像中的目标特征数据；extracting source feature data in the first image, and obtaining the first matching relationship and target feature data in the second image;
利用所述源特征数据和所述目标特征数据进行对比识别；comparing and identifying the source characteristic data and the target characteristic data;
若识别成功，则对所述源特征数据和所述第一匹配关系进行匹配，获得所述第一影像中生物组织的表示和所述第一匹配关系的对应关系。If the identification is successful, the source feature data is matched with the first matching relationship to obtain a corresponding relationship between the representation of the biological tissue in the first image and the first matching relationship.
在一种可能的实现方式中，所述方法还包括：In a possible implementation, the method further includes:
利用所述源特征数据和所述目标特征数据进行对比识别；comparing and identifying the source characteristic data and the target characteristic data;
若识别失败，则重新采集并提取所述第一影像中的所述源特征数据。If the identification fails, the source feature data in the first image is re-acquired and extracted.
在一种可能的实现方式中，所述源特征数据和所述目标特征数据是点云数据、颜色数据或特征值数据中的任意一种。In a possible implementation manner, the source feature data and the target feature data are any one of point cloud data, color data, or feature value data.
在一种可能的实现方式中，所述利用所述源特征数据和所述目标特征数据进行对比识别包括：In a possible implementation manner, the comparing and identifying by using the source feature data and the target feature data includes:
提取初始源特征数据，并删除非感兴趣区域的所述初始源特征数据，将保留的所述初始源特征数据确定为所述源特征数据；extracting the initial source feature data, and deleting the initial source feature data of the non-interest region, and determining the retained initial source feature data as the source feature data;
获取初始目标特征数据，并删除非感兴趣区域的所述初始目标特征数据，将保留的所述初始目标特征数据确定为所述目标特征数据。Acquiring initial target feature data, deleting the initial target feature data of the non-interest region, and determining the retained initial target feature data as the target feature data.
在一种可能的实现方式中，所述初始源特征数据通过提取所述第一影像获取。In a possible implementation manner, the initial source feature data is obtained by extracting the first image.
在一种可能的实现方式中，所述初始目标特征数据通过提取所述第二影像获取或第三图像获取，其中，所述第三图像指的是预设的被测者的生物组织的图像。In a possible implementation manner, the initial target characteristic data is obtained by extracting the second image or the third image, wherein the third image refers to a preset image of the biological tissue of the subject .
在一种可能的实现方式中，所述获取第一匹配关系包括：In a possible implementation manner, the acquiring the first matching relationship includes:
提取所述初始源特征数据，并获取所述初始目标特征数据；extracting the initial source feature data, and obtaining the initial target feature data;
对所述初始源特征数据和所述初始目标特征数据进行匹配，得到所述第一匹配关系。Matching the initial source feature data and the initial target feature data to obtain the first matching relationship.
在一种可能的实现方式中，分别提取所述源特征数据和所述目标特征数据；In a possible implementation manner, the source characteristic data and the target characteristic data are respectively extracted;
若所述源特征数据包含于所述目标特征数据中，则确定对所述第一影像的识别成功。If the source feature data is included in the target feature data, it is determined that the recognition of the first image is successful.
在一种可能的实现方式中，对数据的提取和/或获取是自动执行的，或，用户执行的。In a possible implementation manner, the extraction and/or acquisition of data is performed automatically, or performed by a user.
在一种可能的实现方式中，对所述源特征数据和所述第一匹配关系进行匹配利用最近点迭代算法。In a possible implementation manner, a closest point iterative algorithm is used to match the source feature data and the first matching relationship.
在一种可能的实现方式中，对所述初始源特征数据和所述初始目标特征数据进行匹配利用Ransac算法。In a possible implementation manner, a Ransac algorithm is used to match the initial source feature data and the initial target feature data.
与现有技术相比，本发明的医学图像的匹配方法，至少实现了如下的有益效果：Compared with the prior art, the medical image matching method of the present invention at least achieves the following beneficial effects:
该方法通过提取第一影像中的源特征数据和第二影像中的目标特征数据，并对源特征数据和目标特征数据进行识别。当识别成功时，才对源特征数据和第一匹配关系进行匹配，这样能够剔除大部分无关特征数据，如背景中的无关物体或者误入相机拍摄区的医护人员的面部。当确认识别成功时，说明当前获取的第一影像能够比较清楚地显示感兴趣区域的特征。此时，再确定源特征数据及第一匹配关系的匹配关系，能够提高匹配的运行效率，降低配准时间，同时能够提高匹配的鲁棒性。The method extracts the source feature data in the first image and the target feature data in the second image, and identifies the source feature data and the target feature data. When the recognition is successful, the source feature data is matched with the first matching relationship, so that most irrelevant feature data can be eliminated, such as irrelevant objects in the background or the faces of medical staff who strayed into the camera shooting area. When it is confirmed that the recognition is successful, it means that the currently acquired first image can clearly display the features of the region of interest. At this time, determining the matching relationship between the source feature data and the first matching relationship can improve matching operation efficiency, reduce registration time, and improve matching robustness.
第二方面，本发明提供一种医学图像的匹配装置，包括：In a second aspect, the present invention provides a medical image matching device, including:
提取模块，提取第一影像中的源特征数据，并获取第一匹配关系和第二影像中的目标特征数据；The extraction module extracts the source feature data in the first image, and obtains the first matching relationship and the target feature data in the second image;
识别模块，利用所述源特征数据和所述目标特征数据进行对比识别；An identification module, using the source feature data and the target feature data for comparison and identification;
匹配模块，若识别成功，则对所述源特征数据和所述第一匹配关系进行匹配，获得所述第一影像中生物组织的表示和所述第一匹配关系的对应关系。The matching module, if the recognition is successful, matches the source feature data with the first matching relationship, and obtains a corresponding relationship between the representation of the biological tissue in the first image and the first matching relationship.
在一种可能的实现方式中，所述识别模块，利用所述源特征数据和所述目标特征数据进行对比识别；In a possible implementation manner, the identification module uses the source feature data and the target feature data to perform comparative identification;
若识别失败，则重新采集并提取所述第一影像中的所述源特征数据。If the identification fails, the source feature data in the first image is re-acquired and extracted.
在一种可能的实现方式中，所述源特征数据和所述目标特征数据是点云数据、颜色数据或特征值数据中的任意一种。In a possible implementation manner, the source feature data and the target feature data are any one of point cloud data, color data, or feature value data.
在一种可能的实现方式中，所述提取模块，提取初始源特征数据，并删除非感兴趣区域的所述初始源特征数据，将保留的所述初始源特征数据确定为所述源特征数据；In a possible implementation manner, the extraction module extracts initial source feature data, deletes the initial source feature data of non-interest regions, and determines the retained initial source feature data as the source feature data ;
所述提取模块，获取初始目标特征数据，并删除非感兴趣区域的所述初始目标特征数据，将保留的所述初始目标特征数据确定为所述目标特征数据。The extracting module acquires initial target feature data, deletes the initial target feature data in non-interest regions, and determines the retained initial target feature data as the target feature data.
在一种可能的实现方式中，所述初始源特征数据通过提取所述第一影像获取。In a possible implementation manner, the initial source feature data is obtained by extracting the first image.
在一种可能的实现方式中，所述初始目标特征数据通过提取所述第二影像获取或第三图像获取，其中，所述第三图像指的是预设的被测者的生物组织的图像。In a possible implementation manner, the initial target characteristic data is obtained by extracting the second image or the third image, wherein the third image refers to a preset image of the biological tissue of the subject .
在一种可能的实现方式中，所述提取模块，提取所述初始源特征数据，并获取所述初始目标特征数据；In a possible implementation manner, the extraction module extracts the initial source feature data, and acquires the initial target feature data;
所述匹配模块，对所述初始源特征数据和所述初始目标特征数据进行匹配，得到所述第一匹配关系。The matching module matches the initial source feature data and the initial target feature data to obtain the first matching relationship.
在一种可能的实现方式中，所述提取模块，分别提取所述源特征数据和所述目标特征数据；In a possible implementation manner, the extraction module extracts the source characteristic data and the target characteristic data respectively;
若所述源特征数据包含于所述目标特征数据中，则确定对所述第一影像的识别成功。If the source feature data is included in the target feature data, it is determined that the recognition of the first image is successful.
在一种可能的实现方式中，所述提取模块，对数据的提取和/或获取是自动执行的，或，用户执行的。In a possible implementation manner, the extraction and/or acquisition of data by the extraction module is performed automatically, or performed by a user.
在一种可能的实现方式中，所述匹配模块，对所述源特征数据和所述第一匹配关系进行匹配利用最近点迭代算法。In a possible implementation manner, the matching module uses a closest point iterative algorithm to match the source feature data and the first matching relationship.
在一种可能的实现方式中，所述匹配模块，对所述初始源特征数据和所述初始目标特征数据进行匹配利用Ransac算法。In a possible implementation manner, the matching module uses a Ransac algorithm to match the initial source feature data and the initial target feature data.
本发明中第二方面及其各种实现方式的具体描述，可以参考第一方面中的详细描述；并且，第二方面及其各种实现方式的有益效果，可以参考第一方面中的有益效果分析，此处不再赘述。For the specific description of the second aspect and its various implementations in the present invention, you can refer to the detailed description in the first aspect; and, for the beneficial effects of the second aspect and its various implementations, you can refer to the beneficial effects of the first aspect analysis and will not be repeated here.
当然，实施本发明的任一产品必不特定需要同时达到以上所述的所有技术效果。Of course, any product implementing the present invention does not necessarily need to achieve all the above-mentioned technical effects at the same time.
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings.
附图说明Description of drawings
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.
图1为本发明实施例所提供的一种医学图像的匹配方法的流程图；FIG. 1 is a flowchart of a matching method for medical images provided by an embodiment of the present invention;
图2为本发明实施例所提供的另一种医学图像的匹配方法的流程图；FIG. 2 is a flowchart of another medical image matching method provided by an embodiment of the present invention;
图3为本发明实施例所提供的一种医学图像的匹配装置的组成示意图。Fig. 3 is a schematic composition diagram of a medical image matching device provided by an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本公开实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。另外，“基于”或“根据”的使用意味着开放和包容性，因为“基于”或“根据”一个或多个所述条件或值的过程、步骤、计算或其他动作在实践中可以基于额外条件或超出所述的值。Hereinafter, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, a feature defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more. In addition, the use of "based on" or "according to" is meant to be open and inclusive, as a process, step, calculation or other action "based on" or "according to" one or more stated conditions or values may in practice be based on additional condition or exceed the stated value.
相关技术中，利用ICP(Iterative Closest Point，最近点迭代)算法计算患者医疗影像中的面部点云和实时采集的患者面部点云之间的关系。ICP算法，是基于数据配准法，利用最近点搜索法，从而解决基于自由形态曲面的一种算法。具体地，ICP算法需要遍历两组点云中所有的点查找最适合的匹配点。因此，所以ICP算法所需的配准时间与两组点云中点的数量呈现正相关趋势。点云数量越多，需要的配准时间越长。而由上述可知，由于患者的头部不可能完全保持静止，因此，动态图像的获取时实时的。动态图像中的点云的数量变化较大，不仅会导致配准的相应速度降低，鲁棒性差，还会导致配准的时间变化有较大波动。In related technologies, an ICP (Iterative Closest Point, Iterative Closest Point) algorithm is used to calculate the relationship between the face point cloud in the patient's medical image and the patient's face point cloud collected in real time. The ICP algorithm is based on the data registration method and uses the closest point search method to solve an algorithm based on free-form surfaces. Specifically, the ICP algorithm needs to traverse all the points in the two sets of point clouds to find the most suitable matching point. Therefore, the registration time required by the ICP algorithm is positively correlated with the number of points in the two sets of point clouds. The larger the number of point clouds, the longer the registration time required. However, it can be known from the above that since the head of the patient cannot be kept still completely, the acquisition of dynamic images is real-time. The number of point clouds in the dynamic image changes greatly, which will not only reduce the corresponding speed of registration and poor robustness, but also cause large fluctuations in the time change of registration.
为解决相关技术中存在的上述问题，本发明实施例提供一种医学图像的匹配方法，该方法通过提取第一影像中的源特征数据和第二影像中的目标特征数据，并对源特征数据和目标特征数据进行识别。当识别成功时，才对源特征数据和第一匹配关系进行匹配，这样能够剔除大部分无关特征数据，如背景中的无关物体或者误入相机拍摄区的医护人员的面部。当确认识别成功时，说明当前获取的第一影像能够比较清楚地显示感兴趣区域的特征。此时，再确定源特征数据及第一匹配关系的匹配关系，能够提高匹配的运行效率，降低配准时间，同时能够提高匹配的鲁棒性。In order to solve the above-mentioned problems in the related art, an embodiment of the present invention provides a medical image matching method, the method extracts the source feature data in the first image and the target feature data in the second image, and compares the source feature data and target feature data for identification. When the recognition is successful, the source feature data is matched with the first matching relationship, so that most irrelevant feature data can be eliminated, such as irrelevant objects in the background or the faces of medical staff who strayed into the camera shooting area. When it is confirmed that the recognition is successful, it means that the currently acquired first image can clearly display the features of the region of interest. At this time, determining the matching relationship between the source feature data and the first matching relationship can improve matching operation efficiency, reduce registration time, and improve matching robustness.
本发明实施例提供一种医学图像的匹配方法。参照图1所示，图1为本发明实施例所提供的一种医学图像的匹配方法的流程图，可以包括以下S01-S03。An embodiment of the present invention provides a matching method for medical images. Referring to FIG. 1, FIG. 1 is a flowchart of a medical image matching method provided by an embodiment of the present invention, which may include the following S01-S03.
S01、提取第一影像中的源特征数据，并获取第一匹配关系和第二影像中的目标特征数据。S01. Extract source feature data in the first image, and acquire the first matching relationship and target feature data in the second image.
可以理解的是，源特征数据是第一影像中生物组织的表示，目标特征数据是第二影像中生物组织的表示。匹配是指的将第一影像中生物组织的表示第一匹配关系进行匹配，第二影像作为第一影像的识别依据。It can be understood that the source feature data is the representation of the biological tissue in the first image, and the target feature data is the representation of the biological tissue in the second image. Matching refers to matching the first matching relationship represented by the biological tissue in the first image, and the second image is used as the identification basis of the first image.
在本发明所提供的一种可选实施例中，源特征数据和目标特征数据是点云数据、颜色数据或特征值数据中的任意一种。In an optional embodiment provided by the present invention, the source feature data and the target feature data are any one of point cloud data, color data or feature value data.
可以理解的是，源特征数据和目标特征数据分别用于表示第一影像和第二影像。源特征数据和目标特征数据可以是点云数据、颜色数据或特征值数据中的任意一种。示例性的，当特征数据为颜色数据时，特征数据包括影像中每个像素点的红色(Red)、绿色(Green)、蓝色(Blue)和深度(Depth)四个数据。其中，红色、绿色和蓝色数据可以通过彩色相机拍摄获得，深度数据可以通过深度相机拍摄获得。It can be understood that the source feature data and the target feature data are respectively used to represent the first image and the second image. The source feature data and target feature data can be any one of point cloud data, color data or feature value data. Exemplarily, when the feature data is color data, the feature data includes four data of red (Red), green (Green), blue (Blue) and depth (Depth) of each pixel in the image. Among them, the red, green and blue data can be obtained by shooting with a color camera, and the depth data can be obtained by shooting with a depth camera.
进一步地，不同类型的特征数据之间可以进行相互转化，示例性的，颜色数据中的深度数据和点云数据的变换方法为：Further, different types of feature data can be converted to each other. Exemplarily, the transformation method of depth data and point cloud data in color data is:
首先，获取深度相机的相机参数：First, get the camera parameters of the depth camera:
相机参数为深度相机的固有属性，一般在出厂前由生产厂家标定并存储于深度相机内部，可以直接获取。相机参数可以表示为：Camera parameters are inherent properties of the depth camera, which are generally calibrated by the manufacturer before leaving the factory and stored inside the depth camera, which can be obtained directly. The camera parameters can be expressed as:
其中，fx＝f/dx，fy＝f/dy；Wherein, fx =f/dx , fy =f/dy ;
f，用于表示深度相机的等效焦距；f, used to represent the equivalent focal length of the depth camera;
dx，用于表示像素点在x方向上的物理长度(物理长度，具体指一个像素在感光板上的长度)；dx , used to indicate the physical length of the pixel point in the x direction (physical length, specifically refers to the length of a pixel on the photosensitive plate);
dy，用于表示像素点在y方向上的物理长度；dy , used to represent the physical length of the pixel in the y direction;
cx，用于表示像素坐标系的原点和点云坐标系的原点在x方向上的平移距离；cx , used to represent the translation distance between the origin of the pixel coordinate system and the origin of the point cloud coordinate system in the x direction;
cy，用于表示像素坐标系的原点和点云坐标系的原点在y方向上的平移距离。cy , used to indicate the translation distance between the origin of the pixel coordinate system and the origin of the point cloud coordinate system in the y direction.
然后，获取像素坐标系中的坐标(u，v，zc)；Then, obtain the coordinates (u, v, zc ) in the pixel coordinate system;
其中，u，用于表示像素坐标系中的沿x方向的距离；Among them, u is used to represent the distance along the x direction in the pixel coordinate system;
v，用于表示像素坐标系中的沿y方向的距离；v, used to represent the distance along the y direction in the pixel coordinate system;
zc，用于表示像素坐标系中的沿z方向的距离，即深度数据；zc , used to represent the distance along the z direction in the pixel coordinate system, that is, the depth data;
最后，确定像素坐标系和点云坐标系的变换关系：Finally, determine the transformation relationship between the pixel coordinate system and the point cloud coordinate system:
其中，
进一步地，第二影像是预设的，第二影像可以电脑断层扫描(CT,Computerizedtomography)、核磁共振成像(NMRI,Nuclear magnetic resonance imaging)等技术手段获取。因此，目标特征数据是固定的。而第一影像是在患者治疗或诊断的过程中获取的，由于患者不能够完全保持静止，因此，源特征数据是不断变化。Further, the second image is preset, and the second image can be obtained by technical means such as computerized tomography (CT, Computerizedtomography), nuclear magnetic resonance imaging (NMRI, Nuclear magnetic resonance imaging). Therefore, the target feature data is fixed. While the first image is acquired during the treatment or diagnosis of the patient, since the patient cannot remain completely still, the source feature data is constantly changing.
S02、利用源特征数据和目标特征数据进行对比识别；S02. Using the source feature data and the target feature data to compare and identify;
具体地，在本发明所提供的一种可选实施例中，参照图2所示，图2为本发明实施例所提供的另一种医学图像的匹配方法的流程图。S02可以通过S21-S23实现。Specifically, in an optional embodiment provided by the present invention, refer to FIG. 2 , which is a flowchart of another medical image matching method provided by an embodiment of the present invention. S02 can be realized through S21-S23.
S21、提取初始源特征数据，并删除非感兴趣区域的初始源特征数据，将保留的初始源特征数据确定为源特征数据。S21. Extract the initial source feature data, delete the initial source feature data of the non-interest region, and determine the retained initial source feature data as the source feature data.
可以理解的是，初始源特征数据通过提取第一影像获取。It can be understood that the initial source feature data is obtained by extracting the first image.
具体地，将第一影像导入python开源库face_recognition中，提取第一影像的初始源特征数据。利用face_recognition定位第一影像中的感兴趣区域的68个特征点，并遍历68个特征点在图像坐标系中的坐标值。将横坐标和纵坐标的最大值和最小值分别标记为x_max、x_min、y_max和y_min。将(x_max，y_max)、(x_max，y_min)、(x_min，y_max)和(x_mix，y_min)组成的矩形确定为为第一影像中的感兴趣区域。Specifically, the first image is imported into the python open source library face_recognition, and the initial source feature data of the first image is extracted. Use face_recognition to locate the 68 feature points of the region of interest in the first image, and traverse the coordinate values of the 68 feature points in the image coordinate system. Label the maximum and minimum values of the abscissa and ordinate as x_max, x_min, y_max and y_min, respectively. The rectangle formed by (x_max, y_max), (x_max, y_min), (x_min, y_max) and (x_mix, y_min) is determined as the ROI in the first image.
进一步地，利用深度卷积神经网络(Dynamic Convolution Neural Network)处理感兴趣区域，将感兴趣区域的初始源特征数据确定为源特征数据。Further, a deep convolutional neural network (Dynamic Convolution Neural Network) is used to process the region of interest, and the initial source feature data of the region of interest is determined as the source feature data.
S22、获取初始目标特征数据，并删除非感兴趣区域的初始目标特征数据，将保留的初始目标特征数据确定为目标特征数据。S22. Acquire initial target feature data, delete initial target feature data in non-interest regions, and determine the retained initial target feature data as target feature data.
可以理解的是，初始目标特征数据通过提取第二影像获取或第三图像获取，其中，第三图像指的是预设的被测者的生物组织的图像。It can be understood that the initial target characteristic data is obtained by extracting the second image or the third image, wherein the third image refers to a preset image of the biological tissue of the subject.
需要说明的是，初始目标特征数据可以通过第二影像或第三图像获取，第二影像和第三图像均是预设的。但是，第二影像指的是通过电脑断层扫描(CT,Computerizedtomography)、核磁共振成像(NMRI,Nuclear magnetic resonance imaging)等技术手段获取的医学图像。第三图像指的是通过相机获取的RGB图像。It should be noted that the initial target feature data can be acquired through the second image or the third image, both of which are preset. However, the second image refers to a medical image acquired by technical means such as computerized tomography (CT, Computerizedtomography) and nuclear magnetic resonance imaging (NMRI, Nuclear magnetic resonance imaging). The third image refers to an RGB image acquired by a camera.
通过提取第二影像或第三图像获得初始目标特征数据，参照上述S21所言，得到目标特征数据。The initial target feature data is obtained by extracting the second image or the third image, referring to the above S21, to obtain the target feature data.
S23、分别提取源特征数据和目标特征数据；S23. Extracting source feature data and target feature data respectively;
若源特征数据包含于目标特征数据中，则确定对第一影像的识别成功。If the source feature data is included in the target feature data, it is determined that the recognition of the first image is successful.
具体地，根据支持向量机(Support Vector Machine,SVM)算法确定源特征数据和目标特征数据的匹配率。若匹配率大于或等于预设阈值，则确定源特征数据包含于目标特征数据中，并确定对第一影像的识别成功，继续执行S03。若匹配率小于预设阈值，则确定源特征数据不包含于目标特征数据中，并确定对第一影像的识别失败，需要返回S01重新提取源特征数据，直到对第一影像的识别成功。Specifically, the matching rate of the source feature data and the target feature data is determined according to a support vector machine (Support Vector Machine, SVM) algorithm. If the matching rate is greater than or equal to the preset threshold, it is determined that the source feature data is included in the target feature data, and it is determined that the recognition of the first image is successful, and the execution of S03 is continued. If the matching rate is less than the preset threshold, it is determined that the source feature data is not included in the target feature data, and it is determined that the recognition of the first image fails, and it is necessary to return to S01 to re-extract the source feature data until the recognition of the first image is successful.
需要说明的是，预设阈值可以根据实际需要进行设定。具体地，当预设阙值较高时，表明源特征数据和目标特征数据需要具有较高的匹配率才能认定为对第一影像的识别成功。此时，第一影像的无关特征数据较少，精准度高，但是，识别成功的概率低，对患者的配合度有较高要求。当预设阈值较低时，表明源特征数据和目标特征数据具有较低的匹配率即可认定为对第一影像的识别成功。此时，第一影像中的无关点云较多，精准度低，但是，识别成功的概率高，有利于进入下一步骤。由于第一影像是在患者治疗或诊断过程中获取的，并且在随时变化。因此，需要平衡精准度和效率的双重要求。It should be noted that the preset threshold can be set according to actual needs. Specifically, when the preset threshold is high, it indicates that the source feature data and the target feature data need to have a high matching rate before it can be deemed that the recognition of the first image is successful. At this time, the irrelevant feature data of the first image is less, and the accuracy is high, but the probability of successful recognition is low, and there is a high requirement for the patient's cooperation. When the preset threshold is low, it indicates that the source feature data and the target feature data have a low matching rate, which means that the identification of the first image is successful. At this time, there are many irrelevant point clouds in the first image, and the accuracy is low, but the probability of successful recognition is high, which is conducive to entering the next step. Because the first image is acquired during the treatment or diagnosis of the patient, and it changes at any time. Therefore, it is necessary to balance the dual requirements of accuracy and efficiency.
进一步地，获取第一匹配关系包括：Further, obtaining the first matching relationship includes:
通过S21提取初始源特征数据，并通过S22获取初始目标特征数据。The initial source feature data is extracted through S21, and the initial target feature data is acquired through S22.
对初始源特征数据和初始目标特征数据进行匹配，得到第一匹配关系。The initial source feature data and the initial target feature data are matched to obtain a first matching relationship.
可以理解的是，第一匹配关系利用Ransac算法获得，第一匹配关系是一种粗匹配的关系。It can be understood that the first matching relationship is obtained by using the Ransac algorithm, and the first matching relationship is a rough matching relationship.
S03、对源特征数据和第一匹配关系进行匹配，获得第一影像中生物组织的表示和第一匹配关系的对应关系。S03. Match the source feature data with the first matching relationship to obtain a corresponding relationship between the representation of the biological tissue in the first image and the first matching relationship.
可以理解的是，对源特征数据和第一匹配关系进行匹配利用最近点迭代算法(ICP，Iterative Closest Point)，是一种精匹配的关系。It can be understood that matching the source feature data with the first matching relationship uses an iterative closest point algorithm (ICP, Iterative Closest Point), which is a precise matching relationship.
若识别失败，则重新采集并提取第一影像中的源特征数据。If the identification fails, the source feature data in the first image is re-acquired and extracted.
需要说明的是，对数据的提取和/或获取是自动执行的，或，用户执行的。It should be noted that the extraction and/or acquisition of data is performed automatically, or performed by a user.
综上，本发明提供的医学图像的匹配方法至少实现了如下的有益效果：In summary, the medical image matching method provided by the present invention at least achieves the following beneficial effects:
该方法通过提取第一影像中的源特征数据和第二影像中的目标特征数据，并对源特征数据和目标特征数据进行识别。当识别成功时，才对源特征数据和第一匹配关系进行匹配，这样能够剔除大部分无关特征数据，如背景中的无关物体或者误入相机拍摄区的医护人员的面部。当确认识别成功时，说明当前获取的第一影像能够比较清楚地显示感兴趣区域的特征。此时，再确定源特征数据及第一匹配关系的匹配关系，能够提高匹配的运行效率，降低配准时间，同时能够提高匹配的鲁棒性。The method extracts the source feature data in the first image and the target feature data in the second image, and identifies the source feature data and the target feature data. When the recognition is successful, the source feature data is matched with the first matching relationship, so that most irrelevant feature data can be eliminated, such as irrelevant objects in the background or the faces of medical staff who strayed into the camera shooting area. When it is confirmed that the recognition is successful, it means that the currently acquired first image can clearly display the features of the region of interest. At this time, determining the matching relationship between the source feature data and the first matching relationship can improve matching operation efficiency, reduce registration time, and improve matching robustness.
基于同一发明构思，本发明实施例还提供一种医学图像的匹配装置。参照图3所示，图3为本发明实施例所提供的一种医学图像的匹配装置的组成示意图。医学图像的匹配装置包括：Based on the same inventive concept, an embodiment of the present invention also provides a medical image matching device. Referring to FIG. 3 , FIG. 3 is a schematic composition diagram of a medical image matching device provided by an embodiment of the present invention. Matching devices for medical images include:
提取模块31，提取第一影像中的源特征数据，并获取第一匹配关系和第二影像中的目标特征数据；The
识别模块32，利用源特征数据和目标特征数据进行对比识别；
匹配模块33，若识别成功，则对源特征数据和第一匹配关系进行匹配，获得第一影像中生物组织的表示和第一匹配关系的对应关系。The
需要说明的是，源特征数据和目标特征数据是点云数据、颜色数据或特征值数据中的任意一种。It should be noted that the source feature data and the target feature data are any one of point cloud data, color data or feature value data.
需要说明的是，提取模块31，对数据的提取和/或获取是自动执行的，或，用户执行的。It should be noted that, the extracting
在本发明提供的一种实施例中，识别模块32，利用源特征数据和目标特征数据进行对比识别；In an embodiment provided by the present invention, the
若识别失败，则重新采集并提取第一影像中的源特征数据。If the identification fails, the source feature data in the first image is re-acquired and extracted.
在本发明提供的一种实施例中，提取模块31，提取初始源特征数据，并删除非感兴趣区域的初始源特征数据，将保留的初始源特征数据确定为源特征数据；In an embodiment provided by the present invention, the
提取模块31，获取初始目标特征数据，并删除非感兴趣区域的初始目标特征数据，将保留的初始目标特征数据确定为目标特征数据。The
在本发明提供的一种实施例中，初始源特征数据通过提取第一影像获取。In an embodiment provided by the present invention, the initial source feature data is obtained by extracting the first image.
在本发明提供的一种实施例中，初始目标特征数据通过提取第二影像获取或第三图像获取，其中，第三图像指的是预设的被测者的生物组织的图像。In an embodiment provided by the present invention, the initial target feature data is obtained by extracting a second image or a third image, wherein the third image refers to a preset image of the biological tissue of the subject.
在本发明提供的一种实施例中，提取模块31，提取初始源特征数据，并获取初始目标特征数据；In an embodiment provided by the present invention, the
匹配模块33，对初始源特征数据和初始目标特征数据进行匹配，得到第一匹配关系。The
在本发明提供的一种实施例中，提取模块31，分别提取源特征数据和目标特征数据；In an embodiment provided by the present invention, the
若源特征数据包含于目标特征数据中，则确定对第一影像的识别成功。If the source feature data is included in the target feature data, it is determined that the recognition of the first image is successful.
在本发明提供的一种实施例中，匹配模块33，对源特征数据和第一匹配关系进行匹配利用最近点迭代算法。In an embodiment provided by the present invention, the
在本发明提供的一种实施例中，匹配模块33，对初始源特征数据和初始目标特征数据进行匹配利用Ransac算法。In an embodiment provided by the present invention, the
可以理解的是，为了实现上述功能，装置包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的算法步骤，本发明能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。It can be understood that, in order to realize the above functions, the device includes corresponding hardware structures and/or software modules for performing various functions. Those skilled in the art should easily realize that, in combination with the algorithm steps of the examples described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何在本发明揭露的技术范围内的变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto, and any changes or replacements within the technical scope disclosed in the present invention shall be covered within the protection scope of the present invention . Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.
Claims (22)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210997485.0ACN115330736A (en) | 2022-08-19 | 2022-08-19 | Method and device for matching medical images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210997485.0ACN115330736A (en) | 2022-08-19 | 2022-08-19 | Method and device for matching medical images |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115330736Atrue CN115330736A (en) | 2022-11-11 |
Family
ID=83926373
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210997485.0APendingCN115330736A (en) | 2022-08-19 | 2022-08-19 | Method and device for matching medical images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115330736A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090245692A1 (en)* | 2006-03-23 | 2009-10-01 | Tokyo Institute Of Technology | Image registration method |
| CN103512579A (en)* | 2013-10-22 | 2014-01-15 | 武汉科技大学 | Map building method based on thermal infrared camera and laser range finder |
| CN106780459A (en)* | 2016-12-12 | 2017-05-31 | 华中科技大学 | A kind of three dimensional point cloud autoegistration method |
| CN111950520A (en)* | 2020-08-27 | 2020-11-17 | 重庆紫光华山智安科技有限公司 | Image recognition method and device, electronic equipment and storage medium |
| CN111968160A (en)* | 2020-07-15 | 2020-11-20 | 上海联影智能医疗科技有限公司 | Image matching method and storage medium |
| CN112001955A (en)* | 2020-08-24 | 2020-11-27 | 深圳市建设综合勘察设计院有限公司 | Point cloud registration method and system based on two-dimensional projection plane matching constraint |
- 2022
- 2022-08-19CNCN202210997485.0Apatent/CN115330736A/enactivePending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090245692A1 (en)* | 2006-03-23 | 2009-10-01 | Tokyo Institute Of Technology | Image registration method |
| CN103512579A (en)* | 2013-10-22 | 2014-01-15 | 武汉科技大学 | Map building method based on thermal infrared camera and laser range finder |
| CN106780459A (en)* | 2016-12-12 | 2017-05-31 | 华中科技大学 | A kind of three dimensional point cloud autoegistration method |
| CN111968160A (en)* | 2020-07-15 | 2020-11-20 | 上海联影智能医疗科技有限公司 | Image matching method and storage medium |
| CN112001955A (en)* | 2020-08-24 | 2020-11-27 | 深圳市建设综合勘察设计院有限公司 | Point cloud registration method and system based on two-dimensional projection plane matching constraint |
| CN111950520A (en)* | 2020-08-27 | 2020-11-17 | 重庆紫光华山智安科技有限公司 | Image recognition method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113826143B (en) | Feature point detection | |
| Deng et al. | Retinal fundus image registration via vascular structure graph matching | |
| Crihalmeanu et al. | Enhancement and registration schemes for matching conjunctival vasculature | |
| CN108985210A (en) | A kind of Eye-controlling focus method and system based on human eye geometrical characteristic | |
| CN107506770A (en) | Diabetic retinopathy eye-ground photography standard picture generation method | |
| CN108427918B (en) | Face privacy protection method based on image processing technology | |
| WO2020038312A1 (en) | Multi-channel tongue body edge detection device and method, and storage medium | |
| CN109166177A (en) | Air navigation aid in a kind of art of craniomaxillofacial surgery | |
| KR102561109B1 (en) | Apparatus for 3D image registration and method of fully automatic and markerless 3D image registration using the same | |
| CN116152073B (en) | Improved multi-scale fundus image stitching method based on Loftr algorithm | |
| JP2003070742A (en) | Eye gaze detection device and eye gaze detection method | |
| CN111986137B (en) | Biological organ lesion detection method, apparatus, device, and readable storage medium | |
| WO2023103609A1 (en) | Eye tracking method and apparatus for anterior segment octa, device, and storage medium | |
| CN120088302A (en) | A cross-modal fundus image registration method, device, equipment and medium | |
| CN110929570A (en) | Iris rapid positioning device and positioning method thereof | |
| CN112800966B (en) | Sight tracking method and electronic equipment | |
| CN114187339A (en) | Neural navigation assistance method and device, electronic equipment and storage medium | |
| CN115330736A (en) | Method and device for matching medical images | |
| CN110232684B (en) | A method for automatic segmentation of 3D medical images based on spectral analysis | |
| CN116486398A (en) | Focal image extraction method and device, electronic equipment and storage medium | |
| CN114782537B (en) | Human carotid artery positioning method and device based on 3D vision | |
| WO2023169108A1 (en) | Target region positioning method, electronic device, and medium | |
| Kim et al. | Multi-modal fundus image registration with deep feature matching and image scaling | |
| CN115330735A (en) | Medical image matching method and device | |
| Wang et al. | Follicular Unit Registration Based on Binocular Stereo Vision for Hair Transplantation Surgery Robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | Effective date of registration:20240429 Address after:Room 504, floor 5, building 2, hospital 9, Yiyi Road, Life Science Park, Changping District, Beijing 102206 Applicant after:Beijing Yinhe Fangyuan Technology Co.,Ltd. Country or region after:China Address before:Room 504, floor 5, building 2, hospital 9, Yiyi Road, Life Science Park, Changping District, Beijing 102206 Applicant before:Beijing Yinhe Fangyuan Technology Co.,Ltd. Country or region before:China Applicant before:Beijing yone Galaxy Technology Co.,Ltd. | |
| TA01 | Transfer of patent application right |