技术领域technical field
本发明涉及医疗手术技术领域，特别是涉及一种光学定位下的超声图像与穿刺活检针的术中信息融合方法。The invention relates to the technical field of medical operations, in particular to an intraoperative information fusion method of an ultrasonic image under optical positioning and a puncture biopsy needle.
背景技术Background technique
对于超声引导下的经皮介入手术治疗，经常会存在手术视野不足、术中信息较为单一的问题，手术难度较大、操作精确性难以保障，因此需要医生具备完备的解剖学知识和精细的手术器械操作水平。在进行手术操作时，医生通常只能通过观察术中医学影像进行手术操作，而无法获得手术器械相对于观察的医学影像的详细位姿信息，只能凭借术中影像和临床经验做出判断，一方面安全性无法保障，另一方面降低了医生手术操作的快速性。这对于医生的操作带来很大的不便。For ultrasound-guided percutaneous interventional surgery, there are often problems of insufficient surgical field of view and relatively single intraoperative information. The operation is difficult and the accuracy of the operation is difficult to guarantee. Therefore, doctors need to have complete anatomical knowledge and fine surgery Equipment operation level. During surgical operations, doctors usually can only perform surgical operations by observing intraoperative medical images, but cannot obtain detailed pose information of surgical instruments relative to the observed medical images. They can only make judgments based on intraoperative images and clinical experience. On the one hand, the safety cannot be guaranteed, and on the other hand, the speed of the doctor's operation is reduced. This brings great inconvenience to the doctor's operation.
图像引导手术系统有助于降低经皮介入等手术操作的难度和危险性，然而其精度对于手术的准确实施有着至关重要的作用。对于超声系统而言，超声标定可准确地提供超声探头和超声图像之间的相对关系，这对于图像引导手术而言，可确保手术实施的精度。而现有超声导航系统在未标定的情况下(如专利CN102860841A)可能会导致超声探头和超声图像之间的相对关系不够精确。The image-guided surgery system helps to reduce the difficulty and risk of surgical operations such as percutaneous intervention, but its accuracy plays a vital role in the accurate implementation of surgery. For ultrasound systems, ultrasound calibration can accurately provide the relative relationship between ultrasound probes and ultrasound images, which can ensure the accuracy of surgery for image-guided surgery. However, if the existing ultrasonic navigation system is not calibrated (such as patent CN102860841A), the relative relationship between the ultrasonic probe and the ultrasonic image may not be accurate enough.
现有的自由臂超声标定方法(如专利CN103584885A)采用手动对齐针尖与成像平面，在这种情况下，超声成像平面有厚度存在，超声探头在扫描器械针尖点时，会受到伪像的影响。因此自由臂超声标定方法在实现针尖与成像平面精确对齐、以及针尖点的定位方面，会带来较大的误差。The existing free-arm ultrasonic calibration method (such as patent CN103584885A) uses manual alignment of the needle tip and the imaging plane. In this case, the ultrasonic imaging plane has thickness, and the ultrasonic probe will be affected by artifacts when scanning the needle point of the instrument. Therefore, the free-arm ultrasonic calibration method will bring large errors in the precise alignment of the needle tip with the imaging plane and the positioning of the needle point.
对于现有导航手术而言，通常术前利用MRI、CT等成像设备获得操作空间的三维环境(如专利CN104739512A)，然而却不具备实时性的成像功能，因此需要保证在术前成像之后，术中病人等的相对位置关系不能发生变化。For the existing navigation surgery, MRI, CT and other imaging equipment are usually used to obtain the three-dimensional environment of the operating space (such as patent CN104739512A), but they do not have real-time imaging functions. Therefore, it is necessary to ensure that after the preoperative imaging, the surgical The relative positional relationship of patients etc. cannot be changed.
发明内容Contents of the invention
本发明的目的是针对现有技术中存在的技术缺陷，而提供一种光学定位下的超声图像与穿刺活检针的术中信息融合方法。利用三维虚拟环境下的超声图像引导系统，能实现手术中跟踪穿刺活检针并将器械的位姿信息与超声影像数据实时融合，准确地反映穿刺活检针和超声设备所扫描超声图像之间的三维空间下的相对位姿关系，使得医生对于手术器械相对于病灶结构的位置一目了然，以提高手术效率、降低手术难度，有助于医生实现更加安全的经皮介入等手术操作，进一步提升外科手术的精确性、安全性，减少潜在的并发症和患者的不适。The object of the present invention is to provide an intraoperative information fusion method of an ultrasonic image under optical positioning and a biopsy needle in view of the technical defects in the prior art. Utilizing the ultrasound image guidance system in the 3D virtual environment, it is possible to track the biopsy needle during the operation and fuse the position and orientation information of the instrument with the ultrasound image data in real time, accurately reflecting the three-dimensional relationship between the biopsy needle and the ultrasound image scanned by the ultrasound equipment. The relative position and posture relationship in the space enables the doctor to know the position of the surgical instrument relative to the lesion structure at a glance, so as to improve the operation efficiency and reduce the difficulty of the operation, which helps the doctor to achieve safer percutaneous intervention and other surgical operations, and further improves the surgical accuracy. Accuracy, safety, reducing potential complications and patient discomfort.
为实现本发明的目的所采用的技术方案是：The technical scheme adopted for realizing the purpose of the present invention is:
一种光学定位下的超声图像与穿刺活检针的术中信息融合方法，包括以下步骤：A method for intraoperative information fusion of ultrasonic images under optical positioning and biopsy needles, comprising the following steps:
利用超声图像采集系统采集术中实时超声图像；Use the ultrasound image acquisition system to acquire intraoperative real-time ultrasound images;
利用光学跟踪系统获取超声图像采集系统的超声探头以及手术用穿刺活检针的实时空间位姿信息；Use the optical tracking system to obtain the real-time spatial position and orientation information of the ultrasonic probe of the ultrasonic image acquisition system and the biopsy needle used for surgery;
图像处理工作站将所述实时超声图像以及实时位姿信息进行融合处理，在三维虚拟环境下显示实时融合结果，以反映导航穿刺活检针和超声图像平面之间的相对位置关系。The image processing workstation performs fusion processing on the real-time ultrasound image and real-time pose information, and displays the real-time fusion result in a three-dimensional virtual environment to reflect the relative positional relationship between the navigation biopsy needle and the plane of the ultrasound image.
其中，所述超声探头上固定有可被所述的光学跟踪系统所识别的反光标记定位装置A；Wherein, a reflective marker positioning device A that can be identified by the optical tracking system is fixed on the ultrasonic probe;
所述手术用穿刺活检针上固定有可被所述的光学跟踪系统所识别的反光标记定位装置B；A reflective marker positioning device B that can be identified by the optical tracking system is fixed on the surgical puncture biopsy needle;
所述图像处理工作站与光学跟踪系统相连接，所述图像处理工作站实时接收来自于所述光学跟踪系统采集到的反光标记定位装置A、反光标记定位装置B的空间位姿信息，并通过术中信息融合系统进行三维虚拟环境下的可视化模型匹配与空间位姿信息的显示；The image processing workstation is connected with the optical tracking system, and the image processing workstation receives the spatial pose information of the reflective marker positioning device A and the reflective marker positioning device B collected by the optical tracking system in real time, and passes the intraoperative The information fusion system performs visual model matching and display of spatial pose information in a 3D virtual environment;
所述的图像处理工作站与超声图像采集系统相连接，所述图像处理工作站通过图像采集卡接收来自于超声图像采集系统的实时术中影像信息，并通过所述术中信息融合系统进行三维虚拟环境下的可视化实时显示。The image processing workstation is connected to the ultrasonic image acquisition system, the image processing workstation receives the real-time intraoperative image information from the ultrasonic image acquisition system through the image acquisition card, and performs a three-dimensional virtual environment through the intraoperative information fusion system. The visualization below is displayed in real time.
所述光学定位下的超声图像与穿刺活检针的术中信息融合方法，还包括有超声标定装置，所述超声标定装置是一个四个侧面上布置有若干等距间隙小孔的水槽状模型，顶面上的一角固定有可被所述光学跟踪系统识别的反光标记定位装置C；所述图像处理工作站安装有匹配的超声探头离线标定系统，通过不同坐标系之间的层次传递求得超声图像坐标系与超声探头坐标系之间的转换矩阵，以实现对超声探头的标定。The intraoperative information fusion method of the ultrasonic image under optical positioning and the biopsy needle further includes an ultrasonic calibration device, and the ultrasonic calibration device is a tank-shaped model with several equidistant small holes arranged on four sides, One corner of the top surface is fixed with a reflective marker positioning device C that can be recognized by the optical tracking system; the image processing workstation is equipped with a matching ultrasonic probe offline calibration system, and the ultrasonic image is obtained through hierarchical transfer between different coordinate systems The transformation matrix between the coordinate system and the ultrasound probe coordinate system is used to calibrate the ultrasound probe.
其中，利用所述超声标定装置及超声探头离线标定系统进行定标的方法步骤如下：Wherein, the method steps of using the ultrasonic calibration device and the ultrasonic probe offline calibration system for calibration are as follows:
步骤1：检查超声标定装置，确保超声标定装置上的钢丝绳保持笔直状态，避免因为形变而产生的对于钢丝绳交叉点的真实三维位置的影响；Step 1: Check the ultrasonic calibration device to ensure that the steel wire rope on the ultrasonic calibration device remains straight to avoid the influence of deformation on the real three-dimensional position of the intersection point of the steel wire rope;
步骤2：安装光学跟踪系统、超声探头，并保持光学跟踪系统、超声标定装置之间的相对位置关系固定不动，把超声标定装置放置于温水中；Step 2: Install the optical tracking system and ultrasonic probe, and keep the relative positional relationship between the optical tracking system and the ultrasonic calibration device fixed, and place the ultrasonic calibration device in warm water;
步骤3：启动光学跟踪系统、超声图像采集系统和超声探头；Step 3: Start the optical tracking system, ultrasound image acquisition system and ultrasound probe;
步骤4：将超声探头采集平面放置于水面以下，对水下超声标定装置的目标点进行超声图像采集；采集过程中，来回移动超声探头，直到目标点的超声图像可清晰地用肉眼观察出来；Step 4: Place the ultrasonic probe acquisition plane below the water surface, and perform ultrasonic image acquisition on the target point of the underwater ultrasonic calibration device; during the acquisition process, move the ultrasonic probe back and forth until the ultrasonic image of the target point can be clearly observed with the naked eye;
步骤5：将每次采集到的超声图像上的二维点位置信息和定位得到的超声标定装置的三维点位置信息进行匹配；Step 5: Match the two-dimensional point position information on the ultrasonic image collected each time with the three-dimensional point position information of the ultrasonic calibration device obtained by positioning;
步骤6：利用最小二乘法寻找二维和三维点之间的对应关系矩阵，求解超声图像和超声探头之间的矩阵关系，利用超声探头离线标定系统进行超声标定。Step 6: Find the corresponding relationship matrix between two-dimensional and three-dimensional points by using the least square method, solve the matrix relationship between the ultrasonic image and the ultrasonic probe, and perform ultrasonic calibration by using the ultrasonic probe offline calibration system.
其中，利用所述超声图像目标点的二维位置信息与超声标定装置的三维位置信息之间存在的如下空间转换关系矩阵，作为标定方程，进行超声标定：Wherein, the following spatial transformation relationship matrix existing between the two-dimensional position information of the ultrasonic image target point and the three-dimensional position information of the ultrasonic calibration device is used as a calibration equation to perform ultrasonic calibration:
其中，PC为超声标定装置特征点的三维位置坐标[XC,YC,ZC,1]，PI为超声图像采集系统采集的超声图像的二维像素坐标；为超声探头坐标系到光学跟踪系统坐标系的转换矩阵；为超声图像坐标系到超声探头坐标系的转换矩阵；Wherein, PC is the three-dimensional position coordinates [XC , YC , ZC , 1] of the feature points of the ultrasonic calibration device, and PI is the two-dimensional pixel coordinates of the ultrasonic image collected by the ultrasonic image acquisition system; is the transformation matrix from the coordinate system of the ultrasonic probe to the coordinate system of the optical tracking system; is the transformation matrix from the ultrasound image coordinate system to the ultrasound probe coordinate system;
每次标定得到一个超声标定装置标定特征点的三维位置坐标和超声图像二维像素坐标，当标定超声图像数目N满足N≥3时，进行标定参数的求解。Each calibration obtains the three-dimensional position coordinates of the calibration feature points of the ultrasonic calibration device and the two-dimensional pixel coordinates of the ultrasound image. When the number N of the calibration ultrasound images satisfies N≥3, the calibration parameters are solved.
本发明中，通过超声图像所实时反映的医学影像信息和定位信息，医生可以对病人体表内部病变组织相对于操作器械的位姿关系进行判断，直观感受到导航穿刺活检针针尖点和超声图像上的特征点之间的相对位置关系。In the present invention, through the real-time medical image information and positioning information reflected by the ultrasonic image, the doctor can judge the position and posture relationship of the diseased tissue inside the patient's body surface relative to the operating device, and intuitively feel the needle point of the navigation biopsy needle and the ultrasonic image. The relative positional relationship between the feature points on .
将穿刺活检针针尖点逐渐向超声探头所扫描的超声图像下的目标组织区域靠近，当采集的实时超声图像上开始出现高亮度白斑时，说明穿刺活检针针尖已经到达真实的超声图像平面。此时超声图像上反映的目标特征点和穿刺活检针针尖点在三维虚拟环境系统下是重合的。Gradually approach the needlepoint of the biopsy needle to the target tissue area under the ultrasound image scanned by the ultrasound probe. When high-brightness white spots begin to appear on the collected real-time ultrasound image, it means that the needlepoint of the needle biopsy has reached the real plane of the ultrasound image. At this time, the target feature points reflected on the ultrasound image and the needle point of the biopsy needle coincide under the three-dimensional virtual environment system.
当三维融合环境下的穿刺活检针针尖点与目标特征点重合时，说明已经准确定位目标点，医生可以观察虚拟环境下医学影像与活检穿刺针模型的相对位置关系，对病灶组织进行精细化的手术操作。When the needle point of the biopsy needle in the 3D fusion environment coincides with the target feature point, it means that the target point has been accurately located, and the doctor can observe the relative positional relationship between the medical image and the biopsy needle model in the virtual environment, and fine-tune the lesion tissue Surgical operation.
本发明提供的基于光学跟踪的超声图像与导航穿刺活检针的术中信息融合方法，与现有技术相比较，具有如下优点：Compared with the prior art, the intraoperative information fusion method of ultrasonic images based on optical tracking and navigation biopsy needles provided by the present invention has the following advantages:
1.三维虚拟环境下的术中信息融合，可充分发挥计算机辅助手术导航系统的优势，结合穿刺活检针位姿信息和术中影像信息进行手术引导，将相对位置关系进行三维虚拟环境下的可视化，为医生的操作上带来更大的便利性，提升了手术的快速性；1. The intraoperative information fusion in the 3D virtual environment can give full play to the advantages of the computer-aided surgical navigation system, combine the position and posture information of the biopsy needle and the intraoperative image information for surgical guidance, and visualize the relative position relationship in the 3D virtual environment , bringing greater convenience to the doctor's operation and improving the speed of the operation;
2.利用超声图像获取体表内部信息和操作点位置的相对位置关系，有助于医生对穿刺活检针针尖点与病灶目标的相对关系的判断，有助于提升手术操作的安全性；2. Using ultrasound images to obtain the relative positional relationship between the internal information of the body surface and the position of the operation point will help doctors judge the relative relationship between the needle point of the biopsy needle and the target of the lesion, and help improve the safety of surgical operations;
3.相比于MRI、CT术前成像技术，超声图像能够实时进行术中的成像而无需术前成像，能够将手术环境的变化实时反映在超声图像中；3. Compared with MRI and CT preoperative imaging technologies, ultrasound images can perform intraoperative imaging in real time without preoperative imaging, and can reflect changes in the surgical environment in real time in ultrasound images;
4.相比于利用器械针尖进行标定的方法，通过立体标定模型，减少了超声成像平面厚度与伪影对于标定结果的影响；同时三维的标定模型可以增强标定计算过程中的约束条件，提升标定结果的可靠性；4. Compared with the method of using the needle tip of the instrument for calibration, the three-dimensional calibration model reduces the influence of the thickness of the ultrasound imaging plane and artifacts on the calibration results; at the same time, the three-dimensional calibration model can enhance the constraints in the calibration calculation process and improve the calibration. the reliability of the results;
5.标定方程中的未知量只有在进行超声标定时可减少标定计算的复杂程度；5. Calibration equation The only unknowns in When performing ultrasonic calibration, the complexity of calibration calculation can be reduced;
6.相比于电磁定位等其他定位方式，光学跟踪设备在手术中不存在电磁场形变带来的定位误差等问题；定位时的标记不用通过导线连接，可以减少对于医生的操作空间的约束。6. Compared with other positioning methods such as electromagnetic positioning, optical tracking equipment does not have positioning errors caused by electromagnetic field deformation during surgery; the marks during positioning do not need to be connected by wires, which can reduce the constraints on the doctor's operating space.
总之，本发明通过利用在三维虚拟环境下利用超声的图像引导系统，准确地反映导航穿刺活检针和超声设备所扫描超声图像之间的三维空间下的相对位置关系，结合穿刺活检针位姿信息和术中影像信息进行手术引导，有助于对穿刺活检针与病灶的相对位置关系进行三维虚拟环境下的可视化，提升手术操作的便利性和安全性。In a word, the present invention accurately reflects the relative positional relationship in three-dimensional space between the navigation biopsy needle and the ultrasonic image scanned by the ultrasound equipment by using the image guidance system using ultrasound in the three-dimensional virtual environment, and combines the position and posture information of the biopsy needle Surgical guidance with intraoperative image information helps to visualize the relative positional relationship between the biopsy needle and the lesion in a three-dimensional virtual environment, improving the convenience and safety of surgical operations.
附图说明Description of drawings
图1是本发明的术中信息融合系统的硬件组成的示意图；Fig. 1 is a schematic diagram of the hardware composition of the intraoperative information fusion system of the present invention;
图2是本发明的超声标定装置的示意图；Fig. 2 is the schematic diagram of ultrasonic calibration device of the present invention;
图3是本发明的超声标定过程的示意图；Fig. 3 is the schematic diagram of the ultrasonic calibration process of the present invention;
图4是本发明的超声图像与穿刺活检针术中信息融合操作的过程图；Fig. 4 is a process diagram of the information fusion operation of the ultrasonic image and the biopsy needle of the present invention;
图5是本发明的手术环境下操作的示意图；Fig. 5 is a schematic diagram of operation under the surgical environment of the present invention;
图6是本发明的三维虚拟环境下超声图像与穿刺活检针的术中信息融合效果演示图。Fig. 6 is a demonstration diagram of the intraoperative information fusion effect of the ultrasonic image and the biopsy needle in the three-dimensional virtual environment of the present invention.
具体实施方式Detailed ways
以下结合附图和具体实施例对本发明作进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
本发明光学定位下的超声图像与穿刺活检针的术中信息融合方法，采用如图1所示的平台来实现，在实现前，需要提前搭建术中导航硬件设施平台，组成部分包括光学跟踪系统1、超声图像采集系统2、手术用穿刺活检针3、图像处理工作站4。其中，本发明要融合的信息有两大类，一是超声图像系统采集的术中实时超声图像，二是光学跟踪系统获得的穿刺活检针的实时位姿信息；最终在三维虚拟环境下显示融合结果。可以依据超声探头和手术用穿刺活检针的相对位置估计视点，得到一个虚拟视点与图像平面的适当的视距。The intraoperative information fusion method of ultrasonic images under optical positioning and biopsy needles of the present invention is implemented using the platform shown in Figure 1. Before the implementation, an intraoperative navigation hardware platform needs to be built in advance, and the components include an optical tracking system. 1. Ultrasonic image acquisition system 2. Surgical puncture biopsy needle 3. Image processing workstation 4. Among them, there are two types of information to be fused in the present invention, one is the intraoperative real-time ultrasound image collected by the ultrasound image system, and the other is the real-time position and orientation information of the biopsy needle obtained by the optical tracking system; finally, the fusion is displayed in a three-dimensional virtual environment result. The viewpoint can be estimated according to the relative positions of the ultrasound probe and the surgical biopsy needle, and an appropriate visual distance between a virtual viewpoint and the image plane can be obtained.
其中，所述的图像处理工作站4上安装有超声图像与导航穿刺活检针的术中信息融合软件系统，能利用超声标定装置和光学跟踪系统进行超声探头标定，信息融合软件系统能通过实时采集术中超声图像，实时采集光学跟踪系统位姿信息，对超声探头、手术用穿刺活检针进行实时位姿跟踪，在三维虚拟环境下基于采集到的定位信息确定穿刺活检针模型的空间位姿，将定义的可视化模型与采集到的位姿信息进行匹配，在三维虚拟环境下将光学跟踪的超声图像与导航穿刺活检针进行术中信息融合；并可依据超声探头和手术用穿刺活检针的相对位置估计虚拟视点，得到一个虚拟视点与图像平面的适当的视距，以方便术中观看操作。Wherein, the image processing workstation 4 is equipped with an intraoperative information fusion software system of ultrasonic images and navigation biopsy needles, which can use the ultrasonic calibration device and optical tracking system to calibrate the ultrasonic probe, and the information fusion software system can collect surgical data in real time. Ultrasound images, real-time acquisition of pose information of the optical tracking system, real-time pose tracking of ultrasound probes and surgical biopsy needles, and determination of the spatial pose of the biopsy needle model based on the collected positioning information in a three-dimensional virtual environment. The defined visualization model is matched with the collected pose information, and the optically tracked ultrasound image and the navigation biopsy needle are fused in the 3D virtual environment; and the relative position of the ultrasound probe and the surgical biopsy needle can be used Estimate the virtual viewpoint and obtain an appropriate viewing distance between the virtual viewpoint and the image plane to facilitate intraoperative viewing.
在搭建好本发明的方法所采用的硬件平台后，按以下操作步骤进行操作来实现。After setting up the hardware platform adopted by the method of the present invention, it is realized by operating according to the following steps.
步骤1.1：按照相应的操作规范启动光学跟踪系统、超声图像采集系统和超声探头；Step 1.1: Start the optical tracking system, ultrasound image acquisition system, and ultrasound probe according to the corresponding operating specifications;
步骤1.2：参照图1，将反光标记定位装置A6固定于超声探头上，将反光标记定位装置B7固定于穿刺活检针上，光学跟踪系统1可以对这2个反光标记定位装置进行实时的跟踪与定位。Step 1.2: Referring to Figure 1, fix the reflective marker positioning device A6 on the ultrasonic probe, and fix the reflective marker positioning device B7 on the biopsy needle. The optical tracking system 1 can track and track the two reflective marker positioning devices in real time. position.
步骤1.3：参照图1，将图像处理工作站4与光学跟踪系统1相连接，确保图像处理工作站4能够实时接收来自于光学跟踪系统采集到的反光标记定位装置的位姿信息。Step 1.3: Referring to FIG. 1 , connect the image processing workstation 4 with the optical tracking system 1 to ensure that the image processing workstation 4 can receive the pose information of the reflective marker positioning device collected by the optical tracking system in real time.
步骤1.4：参照图1，将图像处理工作站4与超声图像采集系统2相连接，图像处理工作站4通过图像采集卡实时接收来自于超声图像采集系统的实时术中影像信息。Step 1.4: Referring to FIG. 1, the image processing workstation 4 is connected to the ultrasound image acquisition system 2, and the image processing workstation 4 receives real-time intraoperative image information from the ultrasound image acquisition system in real time through the image acquisition card.
步骤2：参照图2和图3，利用超声标定装置8和光学定位进行超声探头标定，求得超声图像坐标系与超声探头坐标系之间的转换矩阵。超声标定装置8是一个四个侧面上布置有若干等距间隙小孔9的水槽状模型，其每个小孔相对于自身的相对位置关系是已知的，其顶面上的一角固定有可以被所述的光学跟踪系统1识别的反光标记定位装置C11，内部有十字状交叉布置的上下排列的多组笔直状态的钢丝绳10，每组自上而下水平间隔垂直排列设置。Step 2: Referring to Fig. 2 and Fig. 3, use the ultrasonic calibration device 8 and optical positioning to calibrate the ultrasonic probe, and obtain the transformation matrix between the ultrasonic image coordinate system and the ultrasonic probe coordinate system. The ultrasonic calibration device 8 is a tank-shaped model with several equidistant gap holes 9 arranged on four sides, the relative positional relationship of each hole relative to itself is known, and a corner on the top surface is fixed with a The reflective marker positioning device C11 recognized by the optical tracking system 1 has multiple groups of straight steel wire ropes 10 arranged up and down in a cross-like arrangement inside, and each group is arranged vertically at intervals from top to bottom.
步骤2.1：检查超声标定装置，确保超声标定装置上的钢丝绳10保持笔直状态，避免因为形变而产生的对于钢丝绳的真实三维位置的影响；Step 2.1: Check the ultrasonic calibration device to ensure that the steel wire rope 10 on the ultrasonic calibration device remains straight to avoid the influence of deformation on the real three-dimensional position of the steel wire rope;
步骤2.2：安装固定超声标定装置，将其放置于水槽12中，将反光标记定位装置C11固定于超声标定装置8顶面上的一角，并保持光学跟踪系统支架、光学跟踪系统、超声标定装置之间的相对位置关系固定不动。为了得到人眼可辨的超声采集图像，把超声标定装置放置于温水中；Step 2.2: Install and fix the ultrasonic calibration device, place it in the water tank 12, fix the reflective marker positioning device C11 on a corner of the top surface of the ultrasonic calibration device 8, and keep the optical tracking system bracket, the optical tracking system, and the ultrasonic calibration device together. The relative positional relationship between them is fixed. In order to obtain ultrasonic acquisition images that can be distinguished by human eyes, place the ultrasonic calibration device in warm water;
步骤2.3：将超声探头采集面放置于水面以下，对水下超声标定装置的目标点进行超声图像的采集。在采集的过程当中，可以来回移动超声探头，直到目标点的超声图像可以清晰地用肉眼观察出来；Step 2.3: Place the acquisition surface of the ultrasonic probe below the water surface, and collect ultrasonic images of the target points of the underwater ultrasonic calibration device. During the acquisition process, the ultrasonic probe can be moved back and forth until the ultrasonic image of the target point can be clearly observed with the naked eye;
步骤2.4：通过每次采集到的超声图像上的二维点位置信息和理论上超声标定装置的三维位置信息进行一一对应；Step 2.4: Carry out one-to-one correspondence between the two-dimensional point position information on the ultrasonic image collected each time and the theoretical three-dimensional position information of the ultrasonic calibration device;
步骤2.5：利用最小二乘法寻找二维和三维点之间的对应关系矩阵，求解超声图像和超声探头之间的矩阵关系，进行超声标定。Step 2.5: use the least square method to find the corresponding relationship matrix between two-dimensional and three-dimensional points, solve the matrix relationship between the ultrasonic image and the ultrasonic probe, and perform ultrasonic calibration.
其中，所检测的超声图像目标点的二维位置信息与真实环境下超声标定装置的三维位置信息之间存在固定的空间转换关系矩阵：Among them, there is a fixed spatial conversion relationship matrix between the two-dimensional position information of the detected ultrasonic image target point and the three-dimensional position information of the ultrasonic calibration device in the real environment:
其中，PI——超声系统采集的超声图像的二维像素坐标；Among them, PI ——the two-dimensional pixel coordinates of the ultrasound image collected by the ultrasound system;
PM——真实环境下超声标定装置的三维位置坐标；PM ——The three-dimensional position coordinates of the ultrasonic calibration device in the real environment;
——光学跟踪设备坐标系到超声标定装置的转换矩阵； ——The transformation matrix from the coordinate system of the optical tracking equipment to the ultrasonic calibration device;
——超声探头坐标系到光学跟踪设备坐标系的转换矩阵； ——transformation matrix from the coordinate system of the ultrasonic probe to the coordinate system of the optical tracking device;
——超声图像坐标系到超声探头坐标系的转换矩阵； ——transformation matrix from the ultrasound image coordinate system to the ultrasound probe coordinate system;
利用自制的超声标定装置可以得到超声标定装置在光学跟踪坐标系下的三维位置坐标PC，因而无需求解真实环境下超声标定装置的三维位置坐标。此时空间转换关系可以进行简化为：The three-dimensional position coordinates PC of the ultrasonic calibration device in the optical tracking coordinate system can be obtained by using the self-made ultrasonic calibration device, so there is no need to solve the three-dimensional position coordinates of the ultrasonic calibration device in the real environment. At this time, the spatial conversion relationship can be simplified as:
因此，本发明中，超声标定可以在超声探头离线标定软件上进行操作。利用上述的所检测的超声图像目标点的二维位置信息与真实环境下超声标定装置的三维位置信息之间存在固定的空间转换关系矩阵，作为标定方向，即可快速实现超声标定；其中，PC为超声标定装置特征点的三维位置坐标[XC,YC,ZC,1]，当标定超声图像数目N满足N≥3时，可以进行标定参数的求解。Therefore, in the present invention, the ultrasonic calibration can be performed on the offline calibration software of the ultrasonic probe. Utilizing the above-mentioned two-dimensional position information of the detected ultrasonic image target point and the three-dimensional position information of the ultrasonic calibration device in the real environment, there is a fixed spatial conversion relationship matrix, as the calibration direction, and the ultrasonic calibration can be quickly realized; among them, PC is the three-dimensional position coordinates [XC , YC , ZC ,1] of the feature points of the ultrasonic calibration device. When the number N of calibrated ultrasonic images satisfies N≥3, the calibration parameters can be solved.
具体实验操作时，可以通过增加超声图像数目N的方法，提高超声标定的精确度，减小标定误差。In the specific experimental operation, the accuracy of ultrasonic calibration can be improved and the calibration error can be reduced by increasing the number N of ultrasonic images.
步骤3：参照图4，在图像处理工作站4中打开术中信息融合三维虚拟环境，观察实时接收的来自于光学跟踪系统1和超声图像采集系统2的传输数据的三维融合图像13。光学跟踪系统1实时采集反光标记定位装置A和反光标记定位装置B的位置信息，超声图像采集系统2实时采集超声探头5的超声图像。Step 3: Referring to FIG. 4 , open the 3D virtual environment for intraoperative information fusion in the image processing workstation 4 , and observe the 3D fused image 13 received in real time from the transmission data from the optical tracking system 1 and the ultrasonic image acquisition system 2 . The optical tracking system 1 collects the position information of the reflective marker positioning device A and the reflective marker positioning device B in real time, and the ultrasonic image acquisition system 2 collects the ultrasonic image of the ultrasonic probe 5 in real time.
步骤4：参照图4和图5，进行手术操作前，医生通过超声探头接触病人体表14，扫描人体体内的超声图像信息，确定进行穿刺操作的目标特征点区域。Step 4: Referring to Fig. 4 and Fig. 5, before performing the surgical operation, the doctor touches the patient's body surface 14 through the ultrasonic probe, scans the ultrasonic image information in the human body, and determines the target feature point area for the puncture operation.
步骤5：参照图4和图5，在目标特征点附近的体表处进行穿刺活检针的操作，通过人体表面穿刺到目标软组织内部。Step 5: Referring to Fig. 4 and Fig. 5 , perform a puncture biopsy needle operation on the body surface near the target feature point, and puncture into the target soft tissue through the body surface.
步骤6：参照图6，通过超声图像所实时反映的医学影像信息和定位信息，医生可以对病人体表内部病变组织相对于操作器械的位姿关系进行判断，直观感受到导航穿刺活检针针尖点和真实的超声图像平面区域的相对位置关系。Step 6: Referring to Figure 6, through the real-time medical image information and positioning information reflected by the ultrasound image, the doctor can judge the position and posture relationship of the diseased tissue inside the patient's body surface relative to the operating device, and intuitively feel the needle point of the navigation biopsy needle The relative positional relationship with the real ultrasound image plane area.
步骤7：参照图6，将穿刺活检针针尖点逐渐向超声探头所扫描的超声图像下的目标组织区域靠近。当采集的实时超声图像上开始出现高亮度白斑时，说明穿刺活检针针尖已经到达真实的超声图像平面区域。此时，超声图像上反映的目标特征点和穿刺活检针针尖点在三维虚拟环境系统下是重合的。Step 7: Referring to FIG. 6 , the needlepoint of the biopsy needle is gradually approached to the target tissue area under the ultrasound image scanned by the ultrasound probe. When high-brightness white spots begin to appear on the acquired real-time ultrasound image, it means that the needle tip of the biopsy needle has reached the real plane area of the ultrasound image. At this time, the target feature points reflected on the ultrasound image and the needle point of the biopsy needle coincide under the three-dimensional virtual environment system.
步骤8：当术中信息融合环境下的穿刺活检针针尖点与目标特征点已经重合时，说明穿刺活检针针尖点已经准确定位真实环境下的目标特征点，医生可以观察虚拟环境下医学影像与活检穿刺针模型的相对位姿关系，对病灶组织进行精细化的手术操作。Step 8: When the needle point of the biopsy needle in the intraoperative information fusion environment coincides with the target feature point, it means that the needle point of the biopsy needle has accurately positioned the target feature point in the real environment, and the doctor can observe the medical image and the target feature point in the virtual environment. The relative position and posture relationship of the biopsy needle model can be used to perform refined surgical operations on the lesion tissue.
本发明通过在三维虚拟环境下利用超声的图像引导系统，能准确地反映导航穿刺活检针和超声设备所扫描超声图像之间的三维空间下的相对位置关系，结合穿刺活检针位姿信息和术中影像信息进行手术引导，有助于对穿刺活检针与病灶的相对位置关系进行三维虚拟环境下的可视化，提升手术操作的便利性和安全性。The present invention can accurately reflect the relative positional relationship in three-dimensional space between the navigation puncture biopsy needle and the ultrasonic image scanned by the ultrasonic equipment by using the image guidance system of ultrasound in the three-dimensional virtual environment, and combines the pose information of the biopsy needle and the operation Surgical guidance based on image information in the center helps to visualize the relative positional relationship between the biopsy needle and the lesion in a three-dimensional virtual environment, improving the convenience and safety of surgical operations.
以上所述仅是本发明的优选实施方式，应当指出的是，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。The above is only a preferred embodiment of the present invention, it should be pointed out that, for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, these improvements and Retouching should also be regarded as the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910532652.2ACN110279467A (en) | 2019-06-19 | 2019-06-19 | Ultrasound image under optical alignment and information fusion method in the art of puncture biopsy needle |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910532652.2ACN110279467A (en) | 2019-06-19 | 2019-06-19 | Ultrasound image under optical alignment and information fusion method in the art of puncture biopsy needle |
| Publication Number | Publication Date |
|---|---|
| CN110279467Atrue CN110279467A (en) | 2019-09-27 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910532652.2APendingCN110279467A (en) | 2019-06-19 | 2019-06-19 | Ultrasound image under optical alignment and information fusion method in the art of puncture biopsy needle |
| Country | Link |
|---|---|
| CN (1) | CN110279467A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111026269A (en)* | 2019-12-04 | 2020-04-17 | 上海褚信医学科技有限公司 | Haptic feedback method, device and equipment of biological tissue structure based on force feedback |
| CN111658065A (en)* | 2020-05-12 | 2020-09-15 | 北京航空航天大学 | Digital guide system for mandible cutting operation |
| CN112288742A (en)* | 2019-12-31 | 2021-01-29 | 无锡祥生医疗科技股份有限公司 | Navigation method and device for ultrasonic probe, storage medium and electronic equipment |
| CN113048878A (en)* | 2019-12-27 | 2021-06-29 | 苏州因确匹电子科技有限公司 | Optical positioning system and method and multi-view three-dimensional reconstruction system and method |
| CN113421356A (en)* | 2021-07-01 | 2021-09-21 | 北京华信傲天网络技术有限公司 | System and method for inspecting equipment in complex environment |
| CN113547515A (en)* | 2021-07-16 | 2021-10-26 | 华中科技大学 | A coordinate calibration method based on ultrasonic servo surgical robot |
| CN113648060A (en)* | 2021-05-14 | 2021-11-16 | 上海交通大学 | Ultrasonic-guided soft tissue deformation tracking method, device, storage medium and system |
| CN113940737A (en)* | 2021-11-12 | 2022-01-18 | 祥云县人民医院 | Visual aspiration biopsy external member |
| CN113974830A (en)* | 2021-11-02 | 2022-01-28 | 中国人民解放军总医院第一医学中心 | A surgical navigation system for ultrasound-guided thermal ablation of thyroid tumors |
| CN114246690A (en)* | 2021-01-26 | 2022-03-29 | 马元 | Operation simulation method and system of ultrasonic guide bronchoscope |
| CN114271911A (en)* | 2021-12-17 | 2022-04-05 | 领汇医疗科技(武汉)有限公司 | A puncture target, targeted puncture positioning device and method |
| WO2022218389A1 (en)* | 2021-04-17 | 2022-10-20 | 诺创智能医疗科技(杭州)有限公司 | Ultrasonic and x-ray combined execution operation method, device and system, and computer-readable storage medium |
| CN115553880A (en)* | 2022-03-29 | 2023-01-03 | 中国人民解放军总医院第一医学中心 | A positioning method and system for a puncture needle point |
| CN115607277A (en)* | 2022-08-24 | 2023-01-17 | 杭州维纳安可医疗科技有限责任公司 | Needle insertion guidance method, device, equipment and storage medium |
| WO2023006072A1 (en)* | 2021-07-30 | 2023-02-02 | 北京迈迪斯医疗技术有限公司 | Puncture system |
| CN115998334A (en)* | 2021-10-22 | 2023-04-25 | 深圳迈瑞生物医疗电子股份有限公司 | Method for displaying ablation effect and ultrasonic imaging system |
| CN117137628A (en)* | 2023-09-26 | 2023-12-01 | 磅客策(上海)智能医疗科技有限公司 | Biopsy needle and biopsy method based on magnetic navigation |
| CN119055358A (en)* | 2024-11-07 | 2024-12-03 | 合肥工业大学 | A surgical operation force feedback guidance method based on virtual marker tracking and instrument posture |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050080333A1 (en)* | 2003-09-30 | 2005-04-14 | Piron Cameron Anthony | Hybrid imaging method to monitor medical device delivery and patient support for use in the method |
| CN102860841A (en)* | 2012-09-25 | 2013-01-09 | 陈颀潇 | Aided navigation system and method of puncture operation under ultrasonic image |
| CN104398306A (en)* | 2014-12-15 | 2015-03-11 | 柏云云 | Ultrasound intervention puncture needle guidance monitoring system and method |
| CN106063726A (en)* | 2016-05-24 | 2016-11-02 | 中国科学院苏州生物医学工程技术研究所 | Puncture navigation system and air navigation aid thereof in real time |
| CN107028626A (en)* | 2017-05-26 | 2017-08-11 | 北京龙慧珩医疗科技发展有限公司 | A kind of prostate elastic registrating method and device |
| CN109124770A (en)* | 2018-08-01 | 2019-01-04 | 复旦大学附属中山医院 | A kind of prostate biopsy robot |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050080333A1 (en)* | 2003-09-30 | 2005-04-14 | Piron Cameron Anthony | Hybrid imaging method to monitor medical device delivery and patient support for use in the method |
| CN102860841A (en)* | 2012-09-25 | 2013-01-09 | 陈颀潇 | Aided navigation system and method of puncture operation under ultrasonic image |
| CN104398306A (en)* | 2014-12-15 | 2015-03-11 | 柏云云 | Ultrasound intervention puncture needle guidance monitoring system and method |
| CN106063726A (en)* | 2016-05-24 | 2016-11-02 | 中国科学院苏州生物医学工程技术研究所 | Puncture navigation system and air navigation aid thereof in real time |
| CN107028626A (en)* | 2017-05-26 | 2017-08-11 | 北京龙慧珩医疗科技发展有限公司 | A kind of prostate elastic registrating method and device |
| CN109124770A (en)* | 2018-08-01 | 2019-01-04 | 复旦大学附属中山医院 | A kind of prostate biopsy robot |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111026269B (en)* | 2019-12-04 | 2023-10-10 | 上海褚信医学科技有限公司 | Haptic feedback method, device and equipment for biological tissue structure based on force feedback |
| CN111026269A (en)* | 2019-12-04 | 2020-04-17 | 上海褚信医学科技有限公司 | Haptic feedback method, device and equipment of biological tissue structure based on force feedback |
| CN113048878A (en)* | 2019-12-27 | 2021-06-29 | 苏州因确匹电子科技有限公司 | Optical positioning system and method and multi-view three-dimensional reconstruction system and method |
| CN113048878B (en)* | 2019-12-27 | 2023-08-29 | 苏州因确匹电子科技有限公司 | Optical positioning system and method and multi-view three-dimensional reconstruction system and method |
| CN112288742A (en)* | 2019-12-31 | 2021-01-29 | 无锡祥生医疗科技股份有限公司 | Navigation method and device for ultrasonic probe, storage medium and electronic equipment |
| CN111658065A (en)* | 2020-05-12 | 2020-09-15 | 北京航空航天大学 | Digital guide system for mandible cutting operation |
| CN114246690A (en)* | 2021-01-26 | 2022-03-29 | 马元 | Operation simulation method and system of ultrasonic guide bronchoscope |
| CN114246690B (en)* | 2021-01-26 | 2023-07-21 | 马元 | Operation simulation method and system for ultrasonic guided bronchoscope |
| WO2022218389A1 (en)* | 2021-04-17 | 2022-10-20 | 诺创智能医疗科技(杭州)有限公司 | Ultrasonic and x-ray combined execution operation method, device and system, and computer-readable storage medium |
| CN113648060B (en)* | 2021-05-14 | 2024-02-27 | 上海交通大学 | Ultrasonic guided soft tissue deformation tracking method, device, storage medium and system |
| CN113648060A (en)* | 2021-05-14 | 2021-11-16 | 上海交通大学 | Ultrasonic-guided soft tissue deformation tracking method, device, storage medium and system |
| CN113421356A (en)* | 2021-07-01 | 2021-09-21 | 北京华信傲天网络技术有限公司 | System and method for inspecting equipment in complex environment |
| CN113547515B (en)* | 2021-07-16 | 2022-07-12 | 华中科技大学 | A coordinate calibration method based on ultrasonic servo surgical robot |
| CN113547515A (en)* | 2021-07-16 | 2021-10-26 | 华中科技大学 | A coordinate calibration method based on ultrasonic servo surgical robot |
| WO2023006072A1 (en)* | 2021-07-30 | 2023-02-02 | 北京迈迪斯医疗技术有限公司 | Puncture system |
| CN115998334B (en)* | 2021-10-22 | 2025-09-02 | 深圳迈瑞生物医疗电子股份有限公司 | Ablation effect display method and ultrasound imaging system |
| CN115998334A (en)* | 2021-10-22 | 2023-04-25 | 深圳迈瑞生物医疗电子股份有限公司 | Method for displaying ablation effect and ultrasonic imaging system |
| CN113974830A (en)* | 2021-11-02 | 2022-01-28 | 中国人民解放军总医院第一医学中心 | A surgical navigation system for ultrasound-guided thermal ablation of thyroid tumors |
| CN113974830B (en)* | 2021-11-02 | 2024-08-27 | 中国人民解放军总医院第一医学中心 | A surgical navigation system for ultrasound-guided thermal ablation of thyroid tumors |
| CN113940737A (en)* | 2021-11-12 | 2022-01-18 | 祥云县人民医院 | Visual aspiration biopsy external member |
| CN114271911A (en)* | 2021-12-17 | 2022-04-05 | 领汇医疗科技(武汉)有限公司 | A puncture target, targeted puncture positioning device and method |
| CN115553880B (en)* | 2022-03-29 | 2024-08-16 | 中国人民解放军总医院第一医学中心 | Positioning method and system for puncture needle tip |
| CN115553880A (en)* | 2022-03-29 | 2023-01-03 | 中国人民解放军总医院第一医学中心 | A positioning method and system for a puncture needle point |
| CN115607277A (en)* | 2022-08-24 | 2023-01-17 | 杭州维纳安可医疗科技有限责任公司 | Needle insertion guidance method, device, equipment and storage medium |
| CN117137628A (en)* | 2023-09-26 | 2023-12-01 | 磅客策(上海)智能医疗科技有限公司 | Biopsy needle and biopsy method based on magnetic navigation |
| CN119055358A (en)* | 2024-11-07 | 2024-12-03 | 合肥工业大学 | A surgical operation force feedback guidance method based on virtual marker tracking and instrument posture |
| CN119055358B (en)* | 2024-11-07 | 2025-02-14 | 合肥工业大学 | Surgical operation force feedback guiding method based on virtual mark tracking and instrument pose |
| Publication | Publication Date | Title |
|---|---|---|
| CN110279467A (en) | Ultrasound image under optical alignment and information fusion method in the art of puncture biopsy needle | |
| CN102266250B (en) | Ultrasonic operation navigation system and ultrasonic operation navigation method | |
| CN103356284B (en) | Operation piloting method and system | |
| CN101474075B (en) | Minimally Invasive Surgery Navigation System | |
| US20070225553A1 (en) | Systems and Methods for Intraoperative Targeting | |
| CN100591282C (en) | System for guiding a medical device inside a patient | |
| US20170065248A1 (en) | Device and Method for Image-Guided Surgery | |
| CN100536792C (en) | A navigation system and navigation method supporting multiple modes | |
| US20080123910A1 (en) | Method and system for providing accuracy evaluation of image guided surgery | |
| US20050085718A1 (en) | Systems and methods for intraoperative targetting | |
| US20050085717A1 (en) | Systems and methods for intraoperative targetting | |
| US20090221908A1 (en) | System and Method for Alignment of Instrumentation in Image-Guided Intervention | |
| CN101862205A (en) | An Intraoperative Tissue Tracking Method Combined with Preoperative Imaging | |
| WO2013035393A1 (en) | Ultrasound diagnostic device and ultrasound image display method | |
| JP2014510609A (en) | Ultrasound-guided positioning of a heart replacement valve with 3D visualization | |
| EP3973896B1 (en) | Puncture needle positioning system and computer program | |
| US10682126B2 (en) | Phantom to determine positional and angular navigation system error | |
| CN109907801B (en) | Locatable ultrasonic guided puncture method | |
| CN112022304A (en) | Visual puncture system under frameless navigation | |
| Kim et al. | Improving the safety of telerobotic drilling of the skull base via photoacoustic sensing of the carotid arteries | |
| US20230017291A1 (en) | Systems and methods for acquiring ultrasonic data | |
| CN208017582U (en) | Area of computer aided Minimally Invasive Surgery device | |
| Shahidi et al. | Volumetric image guidance via a stereotactic endoscope | |
| Giraldez et al. | Multimodal augmented reality system for surgical microscopy | |
| CN120605046A (en) | Ultrasonic positioning system for brain tumor operation |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20190927 | |
| RJ01 | Rejection of invention patent application after publication |