CN114983315A

Movatterモバイル変換

Info

Publication number: CN114983315A
Application number: CN202210639272.0A
Authority: CN
Inventors: 翁显杰
Original assignee: Shanghai Shutu Precision Instrument Co ltd
Current assignee: Shanghai Shutu Precision Instrument Co ltd
Priority date: 2022-06-07
Filing date: 2022-06-07
Publication date: 2022-09-02

Abstract

Translated fromChinese

本发明公开了一种基于手术显微镜和显微内窥镜融合系统的融合方法，所述融合系统包括显微内窥镜、手术显微镜模组及目视模组，显微内窥镜的图像数据传送至目视模组；手术显微镜模组输出的光传至目视模组；目视模组用于观察手术显微镜模组下的场景图像和显微内窥镜下的组织图像，融合方法的步骤：S1，将显微内窥镜与显微内窥镜图像获取单元电性连接；S2，将显微内窥镜图像获取单元生成的图像成像至目镜的焦平面上；S3：将显微内窥镜置于手术显微镜模组的视场范围内；S4：观察者在目镜中同时看到显微内窥镜下的组织图像及手术显微镜模组下的场景图像，实现对组织的实时在线活检及治疗。

The invention discloses a fusion method based on a surgical microscope and a microendoscope fusion system. The fusion system includes a microendoscope, a surgical microscope module and a visual module, and image data of the microendoscope It is transmitted to the visual module; the light output by the surgical microscope module is transmitted to the visual module; the visual module is used to observe the scene image under the surgical microscope module and the tissue image under the microendoscope. Steps: S1, electrically connecting the microendoscope with the microendoscope image acquisition unit; S2, imaging the image generated by the microendoscope image acquisition unit on the focal plane of the eyepiece; S3: placing the microscopic image on the focal plane of the eyepiece The endoscope is placed in the field of view of the operating microscope module; S4: The observer sees the tissue image under the microscope endoscope and the scene image under the operating microscope module in the eyepiece at the same time, realizing real-time online organization of the tissue Biopsy and treatment.

Description

Translated fromChinese

一种基于手术显微镜和内窥镜融合系统的融合方法A fusion method based on surgical microscope and endoscope fusion system

技术领域technical field

本发明属于医疗器械领域，尤其涉及一种基于手术显微镜和内窥镜融合系统的融合方法。The invention belongs to the field of medical devices, and in particular relates to a fusion method based on a surgical microscope and an endoscope fusion system.

背景技术Background technique

目前，对于人体腹腔各器官的不明病变或可疑肿瘤病变性质的诊治过程，一般都是先对病患做活体取样手术，而后经病理分析后在进行治疗手术。该诊断治疗过程周期很长，前期取样分析检测是一个损伤、耗时、昂贵的过程，而且伴随着出血、感染等风险，另外，病患在取样手术后也需要一段创伤恢复期，有可能耽误最佳治疗时机，影响后期手术治疗效果。根据医疗的发展需要，在手术过程中能实现诊断和治疗功能一体的技术也成为重要研究方向之一。At present, for the diagnosis and treatment of unknown lesions or suspicious tumor lesions of various organs in the human abdominal cavity, the patient is generally subjected to biopsy operation first, and then the treatment operation is performed after pathological analysis. The diagnosis and treatment process takes a long time. Pre-sampling, analysis and testing is a traumatic, time-consuming and expensive process, and it is accompanied by risks such as bleeding and infection. In addition, the patient also needs a period of trauma recovery after the sampling operation, which may delay The optimal timing of treatment affects the effect of later surgical treatment. According to the development needs of medical treatment, the technology that can realize the integration of diagnosis and treatment functions in the operation process has also become one of the important research directions.

手术显微镜和显微内窥镜是对病患进行诊断和治疗过程中经常用到的仪器设备。其中，手术显微镜在手术过程中的目的是让医生精确手术，其视场比较大，但不具备高的分辨率，不足以支撑活检精度。而高分辨率显微内窥镜对活体组织的显微观察能够达到与组织学样品的体外显微成像相当的放大倍数和分辨率，无需取样活检即可实现体内器官的实时高分辨率组织病理学诊断，但其。可见，两者各有优点，无法彼此取代。在很多手术中，需要利用显微内窥镜和手术显微镜两者进行配合来取得满意的手术效果，然而，目前即使在同一台手术中应用两者，显微内窥镜与手术显微镜都是作为两个独立的设备来进行工作，存在一些缺点：1)各自都有自己的显示器、图像采集系统、光源等，每套设备都占有相对较大的空间，在同一台手术过程中很容易受手术室空间限制；2)显微内窥镜的图像一般都是直接由大显示器显示，由于绝大部分医生手术都是在目镜下完成的，在屏幕和目镜之间的不断切换会产生诸多不方便也影响手术效率。Operating microscopes and microendoscopes are instruments that are often used in the process of diagnosing and treating patients. Among them, the purpose of the operating microscope during the operation is to allow doctors to perform precise operations. Its field of view is relatively large, but it does not have high resolution, which is not enough to support the accuracy of biopsy. The microscopic observation of living tissue with high-resolution microendoscopes can achieve magnification and resolution comparable to in vitro microscopic imaging of histological samples, and real-time high-resolution histopathology of in vivo organs can be achieved without sampling and biopsy. Physical diagnosis, but its. It can be seen that both have their own advantages and cannot replace each other. In many operations, it is necessary to use both the microendoscope and the operating microscope to achieve satisfactory surgical results. However, at present, even if the two are used in the same operation, the microendoscope and the operating microscope are both used as There are some disadvantages for two independent devices to work: 1) Each has its own display, image acquisition system, light source, etc., each set of equipment occupies a relatively large space, and it is easy to be affected by the operation during the same operation. 2) The images of the microendoscope are generally displayed directly on the large display. Since most of the surgeon's operations are done under the eyepiece, the constant switching between the screen and the eyepiece will cause a lot of inconvenience. It also affects the efficiency of surgery.

因此，设计出一种能将显微内窥镜与手术显微镜更好地融合来实现诊断治疗一体的方法迫在眉睫。Therefore, it is urgent to design a method that can better integrate the microendoscope and the surgical microscope to realize the integration of diagnosis and treatment.

发明内容SUMMARY OF THE INVENTION

为此，本发明提供了一种基于手术显微镜和内窥镜融合系统的融合方法，将显微内窥镜图像获取单元设置于该融合系统的目视模组内，使得医生在手术过程中通过目镜既看到手术显微镜模组下的场景，又能清晰看到医生想看到的组织细节，可在目镜视野内调整显微内窥镜的位置，该方法不仅提高了手术效率，也有效提升了手术效果。To this end, the present invention provides a fusion method based on an operating microscope and an endoscope fusion system. The microscopic endoscope image acquisition unit is set in the visual module of the fusion system, so that the doctor can pass through the operation during the operation. The eyepiece can not only see the scene under the operating microscope module, but also clearly see the tissue details that the doctor wants to see. The position of the microendoscope can be adjusted within the field of view of the eyepiece. This method not only improves the efficiency of surgery, but also effectively improves the effect of surgery.

为实现上述目的，本发明主要采用以下技术方案：To achieve the above object, the present invention mainly adopts the following technical solutions:

一种基于手术显微镜和显微内窥镜融合系统的融合方法，所述融合系统包括显微内窥镜、手术显微镜模组及目视模组，所述显微内窥镜包括成像镜头、照明系统、相机及启动电路，所述显微内窥镜的图像数据传送至所述目视模组；所述手术显微镜模组包括物镜和照明装置，所述手术显微镜模组输出的光传至所述目视模组；所述目视模组包括第一成像透镜、显微内窥镜图像获取单元及目镜，所述显微内窥镜图像获取单元设置于所述目镜和所述第一成像透镜之间，所述目视模组用于观察所述手术显微镜模组下的场景图像和所述显微内窥镜下的组织图像，所述融合方法包括以下步骤：S1，将所述显微内窥镜与所述显微内窥镜图像获取单元电性连接：通过所述启动电路将所述显微内窥镜调至工作模式，使所述显微内窥镜获取的图像数据能传输至所述显微内窥镜图像获取单元；S2，将所述显微内窥镜图像获取单元生成的图像成像至所述目镜的焦平面上；S3：将所述显微内窥镜置于所述手术显微镜模组的视场范围内：在所述目镜中观察到所述显微内窥镜并将其移动至观察者想看到的位置；S4：观察者在所述目镜中同时看到所述显微内窥镜下的组织图像及所述手术显微镜模组下的场景图像，实现对组织的实时在线活检及治疗。A fusion method based on a surgical microscope and a microendoscope fusion system, the fusion system comprising a microendoscope, a surgical microscope module and a visual module, the microendoscope comprising an imaging lens, an illumination a system, a camera and a start-up circuit, the image data of the microendoscope is transmitted to the visual module; the surgical microscope module includes an objective lens and an illumination device, and the light output by the surgical microscope module is transmitted to the the visual module; the visual module includes a first imaging lens, a microendoscope image acquisition unit and an eyepiece, and the microendoscope image acquisition unit is arranged on the eyepiece and the first imaging unit Between the lenses, the visual module is used to observe the scene image under the operating microscope module and the tissue image under the microscopic endoscope. The fusion method includes the following steps: S1: The microendoscope is electrically connected to the image acquisition unit of the microendoscope: the microendoscope is adjusted to the working mode through the start-up circuit, so that the image data obtained by the microendoscope can be obtained. transmitting to the microendoscope image acquisition unit; S2, imaging the image generated by the microendoscope image acquisition unit on the focal plane of the eyepiece; S3: placing the microendoscope on the focal plane Within the field of view of the operating microscope module: observe the microendoscope in the eyepiece and move it to the position that the observer wants to see; S4: the observer is in the eyepiece at the same time See the tissue image under the microendoscope and the scene image under the operating microscope module, and realize the real-time online biopsy and treatment of the tissue.

优选地，所述显微内窥镜图像获取单元包括透明显示器，步骤S1中所述电性连接的步骤具体为：所述显微内窥镜与所述透明显示器通过无线或有线连接方式实现电性连接，所述透明显示器接收所述显微内窥镜获取的图像数据并生成图像；步骤S2中所述成像至所述目镜的焦平面上的步骤具体为：将所述透明显示器设置于所述目镜的焦平面上。Preferably, the image acquisition unit of the microscopic endoscope includes a transparent display, and the step of electrical connection in step S1 is specifically: the microscopic endoscope and the transparent display are connected wirelessly or wiredly to realize electrical connection. The transparent display receives the image data obtained by the microendoscope and generates an image; the step of imaging on the focal plane of the eyepiece in step S2 is specifically: disposing the transparent display on the focal plane of the eyepiece. on the focal plane of the eyepiece.

优选地，所述显微内窥镜图像获取单元包括微显示器、投影物镜及第一分光棱镜，步骤S1中所述电性连接的步骤具体为：所述显微内窥镜与所述微显示器通过无线或有线连接方式实现电性连接，所述微显示器接收所述显微内窥镜获取的图像数据并生成图像；步骤S2中所述成像至所述目镜的焦平面上的步骤具体为：所述微显示器的生成图像依次经过所述投影物镜及所述第一分光棱镜成像至所述目镜的焦平面上。Preferably, the microendoscope image acquisition unit includes a microdisplay, a projection objective lens and a first beam splitter prism, and the step of electrically connecting in step S1 is specifically: the microendoscope and the microdisplay The electrical connection is realized by wireless or wired connection, and the microdisplay receives the image data obtained by the microendoscope and generates an image; the step of imaging on the focal plane of the eyepiece in step S2 is specifically: The generated image of the microdisplay is sequentially imaged onto the focal plane of the eyepiece through the projection objective lens and the first dichroic prism.

优选地，还包括步骤S3.1，调整所述显微内窥镜图像获取单元生成的图像的亮度和倍率。Preferably, the method further includes step S3.1 of adjusting the brightness and magnification of the image generated by the microendoscope image acquisition unit.

优选地，所述目视系统还包括保罗棱镜组，所述保罗棱镜组用于调整所述目镜焦平面上图像的成像方向。Preferably, the visual system further comprises a Paul prism group, the Paul prism group is used to adjust the imaging direction of the image on the focal plane of the eyepiece.

优选地，所述融合系统还包括屏前探测模组，用于供助手观察者观看，所述屏前探测模组包括第二成像棱镜、第二成像透镜、第二相机及显示器，所述第二成像棱镜将所述手术显微镜模组输出的光分成两部分，一部分进入所述目视模组，一部分进入所述屏前探测模组，其中，进入所述屏前探测模组的光经所述第二成像透镜汇聚至所述第二相机的感光片上，再经所述显示器输出所述手术显微镜模组下的场景图像。Preferably, the fusion system further comprises a front-screen detection module for viewing by an assistant observer, the front-screen detection module comprises a second imaging prism, a second imaging lens, a second camera and a display, the first Two imaging prisms divide the light output by the operating microscope module into two parts, one part enters the visual module, and the other part enters the front-screen detection module, wherein the light entering the front-screen detection module passes through the The second imaging lens converges on the photosensitive sheet of the second camera, and then outputs the scene image under the operating microscope module through the display.

优选地，还包括步骤S4.1，将所述屏前探测模组中的所述显示器与所述显微内窥镜电性连接，使所述显示器上能同时显示出所述手术显微镜模组下的场景图像和所述显微内窥镜下的组织图像。Preferably, it also includes step S4.1, electrically connecting the display in the front-screen detection module with the microendoscope, so that the display can simultaneously display the surgical microscope module The scene image under the microscope and the tissue image under the microendoscope.

优选地，所述透明显示器的尺寸小于所述目镜的视场且为所述显微内窥镜中所述相机的尺寸的整数倍。Preferably, the size of the transparent display is smaller than the field of view of the eyepiece and is an integer multiple of the size of the camera in the microendoscope.

优选地，所述微显示器成像至所述目镜焦平面上的像的尺寸小于所述目镜的视场且为所述显微内窥镜中所述相机的尺寸的整数倍。Preferably, the size of the image formed by the microdisplay on the focal plane of the eyepiece is smaller than the field of view of the eyepiece and is an integer multiple of the size of the camera in the microendoscope.

优选地，所述融合系统还包括光路补偿装置，所述光路补偿装置用于补偿光路中的成像像差，其设置于所述手术显微镜模组与所述第一成像透镜之间的光路中。Preferably, the fusion system further includes an optical path compensation device, which is used for compensating imaging aberrations in the optical path, and is disposed in the optical path between the operating microscope module and the first imaging lens.

优选地，所述光路补偿装置为空间光调制器或非球面透镜。Preferably, the optical path compensation device is a spatial light modulator or an aspheric lens.

与现有技术相比，本发明的有益效果是：1)本发明的融合方法能使医生在手术过程中直接通过目镜同时观察到手术显微镜模组下的场景及组织的精密细节，能实时在线活检，能同时实现对组织的诊断和治疗，且减少了医生在屏幕与目镜之间的切换适应时间，显著提高了手术效率；2)该融合方法使医生能在手术显微镜模组的视野中移动显微内窥镜至手术过程中想看到的位置，使显微内窥镜的移动位置更加精确，能更好地适应医生在操作过程中的需求，提升了手术效果。Compared with the prior art, the beneficial effects of the present invention are: 1) The fusion method of the present invention enables the doctor to observe the scene and the precise details of the tissue under the operating microscope module directly through the eyepiece during the operation, and can be online in real time. Biopsy, which can diagnose and treat tissue at the same time, reduce the doctor's switching adaptation time between the screen and the eyepiece, and significantly improve the operation efficiency; 2) This fusion method enables doctors to move the display in the field of view of the operating microscope module. The microendoscope is moved to the desired position during the operation, which makes the moving position of the microendoscope more accurate, which can better meet the needs of the doctor during the operation, and improve the operation effect.

附图说明Description of drawings

图1为本发明实施例中手术显微镜和内窥镜融合系统示意图一；1 is a schematic diagram 1 of a fusion system of an operating microscope and an endoscope in an embodiment of the present invention;

图2为本发明实施例中手术显微镜和内窥镜融合系统示意图二；FIG. 2 is a second schematic diagram of a fusion system of an operating microscope and an endoscope in an embodiment of the present invention;

图3为本发明实施例中手术显微镜和内窥镜融合方法的工作流程图。FIG. 3 is a working flow chart of a fusion method of an operating microscope and an endoscope in an embodiment of the present invention.

具体实施方式Detailed ways

下面将结合示意图对本发明的具体实施方式进行更详细的描述。根据下列描述，本发明的优点和特征将更清楚。需说明的是，附图均采用非常简化的形式且均使用非精准的比例，仅用以方便、明晰地辅助说明本发明实施例的目的。The specific embodiments of the present invention will be described in more detail below with reference to the schematic diagrams. The advantages and features of the present invention will become more apparent from the following description. It should be noted that, the accompanying drawings are all in a very simplified form and in inaccurate scales, and are only used to facilitate and clearly assist the purpose of explaining the embodiments of the present invention.

本发明提供了一种手术显微镜和显微内窥镜的融合系统，如图1所示，包括显微内窥镜1、手术显微镜模组2及目视模组3，手术显微镜模组2包括物镜、连续变倍体及照明装置，目视模组3包括第一成像透镜31、显微内窥镜图像获取单元及目镜33，被观察组织的反射光依次经物镜、连续变倍体、第一成像透镜31传至目镜33，其中，显微内窥镜1可设于手术显微镜模组2的视野范围内，显微内窥镜1根据被观察组织的反射光所采集到的图像数据是通过有线或无线方式传至显微内窥镜图像获取单元，显微内窥镜图像获取单元将获取的图像成像至目镜33的焦平面上。手术显微模组2的分辨率在10微米左右，手术显微镜模组2下的场景用于让医生进行精确手术，其分辨率不能达到对组织的活检精度，而显微内窥镜1的分辨率可达到对组织的活检精度，也即能观察到组织的精密细节。在手术操作过程中，本发明所提供的融合系统可使医生经目镜33能同时观察到由手术显微镜模组2下所观察到的图像和由显微内窥镜1获取的组织的精密细节，医生可在手术显微镜模组2的视野下移动显微内窥镜1至医生想看到的位置，非常有利于精细手术，有效提高了手术效果和手术效率。The present invention provides a fusion system of an operating microscope and a microscopic endoscope, as shown in FIG. 1 , including amicroscopic endoscope 1 , anoperating microscope module 2 and avisual module 3 , and theoperating microscope module 2 includes The objective lens, the continuous variable magnification and the lighting device, thevisual module 3 includes afirst imaging lens 31, a microscopic endoscope image acquisition unit and aneyepiece 33, and the reflected light of the tissue to be observed passes through the objective lens, the continuous variable magnification, and the third in turn. Animaging lens 31 is transmitted to theeyepiece 33, wherein themicroendoscope 1 can be set in the field of view of theoperating microscope module 2, and the image data collected by themicroendoscope 1 according to the reflected light of the observed tissue is It is transmitted to the microendoscope image acquisition unit in a wired or wireless manner, and the microendoscope image acquisition unit images the acquired image on the focal plane of theeyepiece 33 . The resolution of thesurgical microscope module 2 is about 10 microns. The scene under thesurgical microscope module 2 is used to allow doctors to perform precise surgery. The accuracy of the biopsy of the tissue can be achieved, that is, the precise details of the tissue can be observed. During the surgical operation, the fusion system provided by the present invention enables the doctor to simultaneously observe the image observed under theoperating microscope module 2 and the precise details of the tissue acquired by themicroendoscope 1 through theeyepiece 33, The doctor can move themicroendoscope 1 to the position the doctor wants to see under the field of view of theoperating microscope module 2, which is very beneficial to fine surgery and effectively improves the surgical effect and efficiency.

在本实施方案中，显微内窥镜1是一个体积较小的一体式光学仪器，包含成像镜头、照明系统、相机(也称为显微内窥镜相机)及启动电路，其中，相机可根据需要选用CCD相机或CMOS相机，启动电路用于将显微内窥镜1从未工作状态调至工作状态，以实现通过无线连接或电缆等有线连接将显微内窥镜1获取的图像数据传至显微内窥镜图像获取单元。该显微内窥镜1的分辨率高，一般在微米或亚微米精度，可达到对组织实现活检的要求。该显微内窥镜1在本实施方案中被安装在一个支架(图中未示出)上并可相对支架移动，医生在手术的操作过程中需要该显微内窥镜1配合使用时，直接用手将安装在支架上的显微内窥镜1移动至所需位置即可。该显微内窥镜1获取的图像数据是通过电信号实现传播，并不通过成像光路传输光学信号，该图像数据传输方式不仅使传输结构简单，且还能保证显微内窥镜图像获取单元对图像的处理，如控制图像的放大倍率或图像的亮度，以更好地满足医生在手术过程中的需求。In the present embodiment, themicroendoscope 1 is a small integrated optical instrument, including an imaging lens, an illumination system, a camera (also called a microendoscope camera) and a start-up circuit, wherein the camera can A CCD camera or a CMOS camera is selected as required, and the start-up circuit is used to adjust themicroendoscope 1 from the non-working state to the working state, so as to realize the image data obtained by themicroendoscope 1 through a wired connection such as a wireless connection or a cable. to the microendoscope image acquisition unit. The resolution of themicroendoscope 1 is high, generally in micron or sub-micron precision, and can meet the requirements for biopsy of tissue. In this embodiment, themicroendoscope 1 is mounted on a bracket (not shown in the figure) and can move relative to the bracket. When the doctor needs themicroendoscope 1 to be used together during the operation Themicroendoscope 1 mounted on the bracket can be moved to the desired position directly by hand. The image data acquired by themicroscopic endoscope 1 is transmitted through electrical signals, and does not transmit optical signals through the imaging optical path. This image data transmission method not only makes the transmission structure simple, but also ensures the image acquisition unit of the microscopic endoscope. Image processing, such as controlling the magnification of the image or the brightness of the image, to better meet the needs of doctors during surgery.

目视模组3中的显微内窥镜图像获取单元可包括一个透明显示器321，该透明显示器321设置于目镜33的焦平面上。该透明显示器321是透明的，其尺寸小于目镜33的视场(标准22mm)且同时是显微内窥镜相机尺寸的整数倍，其亮度可根据需要进行调节，且能对生成图像进行不同倍率的放大，利于图像观察，能使图像显示地更加清晰，利于医生的观察进而提高手术效果。透明显示器321与显微内窥镜1之间电性连接，实现图像数据连接，如采用无线或有线方式连接，也即透明显示器321上会显示显微内窥镜1下组织的精密细节的图像，医生通过目镜33可以同时观察到手术显微镜模组2下的场景及组织的精密细节，可以在手术过程中实现实时在线活检，提高手术效果及手术效率。该透明显示器321的功能类似一块玻璃平板，由于制造误差，其在本发明的融合系统中可能会给成像光路带来像差，可以在系统中添加光路补偿装置进行校正。该补偿装置可设置在手术显微镜模组2与第一成像透镜31之间的平行光路中，其可优选为空间光调制器或非球面透镜。透明显示器321与显微内窥镜1的放置方式不同，目镜33中观察到显微内窥镜1的像的成像方向也不同：显微内窥镜1正置放置且透明显示器321倒置放置时，目镜33接收的显微内窥镜1的像为正像；或者，显微内窥镜1倒置放置且透明显示器321正置放置，这样也能保证目镜33观察到的为正像。同理，显微内窥镜1倒置放置且透明显示器321正置放置时，目镜33接收的显微内窥镜1的像为倒像；或者，显微内窥镜1正置放置且透明显示器321倒置放置，这样也能保证目镜33观察到的为倒像。The microendoscope image acquisition unit in thevisual module 3 may include atransparent display 321 , and thetransparent display 321 is arranged on the focal plane of theeyepiece 33 . Thetransparent display 321 is transparent, its size is smaller than the field of view of the eyepiece 33 (standard 22mm), and at the same time is an integer multiple of the size of the microendoscope camera, its brightness can be adjusted as required, and different magnifications can be applied to the generated image The magnification is beneficial to image observation, which can make the image display clearer, which is beneficial to the doctor's observation and thus improves the operation effect. Thetransparent display 321 is electrically connected with themicroendoscope 1 to realize the image data connection. For example, if the connection is wireless or wired, that is, thetransparent display 321 will display the image of the precise details of the tissue under themicroendoscope 1. , the doctor can simultaneously observe the scene under the operatingmicroscope module 2 and the precise details of the tissue through theeyepiece 33, which can realize real-time online biopsy during the operation, and improve the operation effect and operation efficiency. The function of thetransparent display 321 is similar to a glass plate. Due to manufacturing errors, it may bring aberrations to the imaging optical path in the fusion system of the present invention, and an optical path compensation device can be added to the system for correction. The compensation device can be arranged in the parallel light path between the operatingmicroscope module 2 and thefirst imaging lens 31 , and it can preferably be a spatial light modulator or an aspheric lens. The placement of thetransparent display 321 and themicroendoscope 1 is different, and the imaging direction of the image of themicroendoscope 1 observed in theeyepiece 33 is also different: when themicroendoscope 1 is placed upright and thetransparent display 321 is placed upside down , the image of themicroendoscope 1 received by theeyepiece 33 is a positive image; or, themicroendoscope 1 is placed upside down and thetransparent display 321 is placed upright, which can also ensure that what theeyepiece 33 observes is a positive image. Similarly, when themicroendoscope 1 is placed upside down and thetransparent display 321 is placed upright, the image of themicroendoscope 1 received by theeyepiece 33 is an inverted image; or, when themicroendoscope 1 is placed upright and thetransparent display 321 is placed upside down, which can also ensure that what theeyepiece 33 observes is an inverted image.

在另一实施方案中，如图2所示，目视模组3中的显微内窥镜图像获取单元包括微显示器322、投影物镜323及第一分光棱镜324，第一分光棱镜324设于目镜33与第一成像透镜31之间的成像光路中，微显示器322、投影物镜323及第一分光棱镜324之间形成的成像光路垂直于目镜33及第一分光棱镜324之间的成像光路。微显示器322上的图像通过投影物镜323及第一分光棱镜324成像至目镜33的焦平面上。微显示器322与显微内窥镜1之间采用无线或有线的电性连接方式实现图像数据传输，也即微显示器322上会显示显微内窥镜1下组织的精密细节的图像，医生通过目镜33也可以同时观察到手术显微镜模组2下的场景及组织的精密细节，可以在手术过程中实现实时在线活检，提高手术效果及手术效率。对于微显示器322，经投影物镜323投至目镜33焦平面上的像的尺寸小于目镜33视场(标准22mm)，微显示器322的尺寸为显微内窥镜相机尺寸的整数倍(如1倍，2倍等)。微显示器322的亮度可根据需要进行调整，且其可对生成图像的倍率也可根据需要进行不同倍率的放大，利于观察图像，优选在微显示器322上设置亮度调整旋钮和倍率调整旋钮。第一分光棱镜324的分光比可设置为9:1，也即第一成像透镜31输出光的透过率与投影物镜323输出光的透过率的比值为9:1，通过分光比的设置和微显示器322亮度的调节可使图像更加清晰，利于医生的手术观察，可提高手术效果。微显示器322与显微内窥镜1的放置方式不同，目镜33中观察到显微内窥镜1的像的成像方向也不同：微显示器322倒置放置且显微内窥镜1倒置放置时，目镜33接收的显微内窥镜1的像为正像；或者，微显示器322正置放置且显微内窥镜1正置放置，这样也能保证目镜33观察到的为正像。同理，微显示器322正置放置且显微内窥镜1倒置放置时，目镜33接收的显微内窥镜1的像为倒像；或者，微显示器322倒置放置且显微内窥镜1正置放置，这样也能保证目镜33观察到的为倒像。In another embodiment, as shown in FIG. 2 , the microendoscope image acquisition unit in thevisual module 3 includes a microdisplay 322 , a projectionobjective lens 323 and a firstbeam splitting prism 324 , and the firstbeam splitting prism 324 is located in the In the imaging optical path between theeyepiece 33 and thefirst imaging lens 31 , the imaging optical path formed between the microdisplay 322 , theprojection objective lens 323 and the firstdichroic prism 324 is perpendicular to the imaging optical path between theeyepiece 33 and the firstdichroic prism 324 . The image on the microdisplay 322 is imaged onto the focal plane of theeyepiece 33 through theprojection objective lens 323 and the firstdichroic prism 324 . Wireless or wired electrical connection is used between the microdisplay 322 and themicroendoscope 1 to realize image data transmission, that is, the microdisplay 322 will display the precise details of the tissue under themicroendoscope 1. Theeyepiece 33 can also observe the scene under the operatingmicroscope module 2 and the precise details of the tissue at the same time, which can realize real-time online biopsy during the operation, and improve the operation effect and efficiency. For the microdisplay 322, the size of the image projected onto the focal plane of theeyepiece 33 through theprojection objective 323 is smaller than the field of view of the eyepiece 33 (standard 22mm), and the size of the microdisplay 322 is an integer multiple of the size of the microendoscope camera (eg 1 times , 2 times, etc.). The brightness of the microdisplay 322 can be adjusted as required, and the magnification of the generated image can also be magnified at different magnifications as required, which is beneficial for observing the image. The splitting ratio of the firstbeam splitting prism 324 can be set to 9:1, that is, the ratio of the transmittance of the output light of thefirst imaging lens 31 to the transmittance of the output light of theprojection objective lens 323 is 9:1. And the adjustment of the brightness of the microdisplay 322 can make the image clearer, which is beneficial to the doctor's surgical observation and can improve the surgical effect. The placement of the microdisplay 322 and themicroendoscope 1 is different, and the imaging direction of the image of themicroendoscope 1 observed in theeyepiece 33 is also different: when the microdisplay 322 is placed upside down and themicroendoscope 1 is placed upside down, The image of theendoscope 1 received by theeyepiece 33 is an erect image; or, the microdisplay 322 is placed upright and theendoscope 1 is placed upright, which can also ensure that the image observed by theeyepiece 33 is an erect image. Similarly, when the microdisplay 322 is placed upright and themicroendoscope 1 is placed upside down, the image of themicroendoscope 1 received by theeyepiece 33 is an inverted image; or, when the microdisplay 322 is placed upside down and themicroendoscope 1 It is placed in an upright position, which can also ensure that the image observed by theeyepiece 33 is an inverted image.

本发明所提供的该融合系统还包括屏前探测模组，屏前探测模组包括显示器(图中未示出)、第二相机41、第二成像透镜42及第二分光棱镜43，第二分光棱镜43设于手术显微镜模组2与第一成像透镜31之间的平行光路中，第二分光棱镜43、第二成像透镜42及第二相机41组成的成像光路垂直于手术显微镜模组2与第一成像透镜31之间的平行光路，也即由手术显微镜模组2输出的平行光被第二分光棱镜43分成两部分，一部分进入目视模组3，一部分进入屏前探测模组。进入屏前探测模组的光经第二成像透镜43汇聚至第二相机41的感光片上，再经显示器输出，该显示器具有较大显示屏，以便供助手医生查看手术显微镜模组2下的场景。屏前探测模组中的显示器还能与显微内窥镜1实现电性连接，如采用无线或有线的连接方式，也即该显示器也能显示显微内窥镜1获取的图像，该显示器可将来自显微内窥镜1的观察图像(即组织精密细节的图像)及手术显微镜模组2下的场景图像同时显示出来，方便助手医生的查看。屏前探测模组中的显示器上可通过两个不同区域来同时显示这两个图像(显微内窥镜1的观察图像和手术显微镜模组2下的场景图像)，能使观察者通过触摸屏或按键选择性地使显微内窥镜1的观察图像或手术显微镜模组2下的场景图像在较大区域内显示。The fusion system provided by the present invention further includes a detection module in front of the screen, and the detection module in front of the screen includes a display (not shown in the figure), asecond camera 41, asecond imaging lens 42 and a secondbeam splitting prism 43, and the second Thedichroic prism 43 is arranged in the parallel light path between the operatingmicroscope module 2 and thefirst imaging lens 31 , and the imaging optical path composed of the seconddichroic prism 43 , thesecond imaging lens 42 and thesecond camera 41 is perpendicular to the operatingmicroscope module 2 . The parallel light path with thefirst imaging lens 31, that is, the parallel light output by the operatingmicroscope module 2 is divided into two parts by the secondbeam splitting prism 43, one part enters thevisual module 3, and the other part enters the front-screen detection module. The light entering the detection module in front of the screen is concentrated on the photosensitive sheet of thesecond camera 41 through thesecond imaging lens 43, and then output through the display, which has a larger display screen, so that the assistant doctor can view the scene under the operatingmicroscope module 2 . The display in the detection module in front of the screen can also be electrically connected to themicroendoscope 1. If a wireless or wired connection is used, that is, the display can also display the image acquired by themicroendoscope 1. The observation image from the microendoscope 1 (that is, the image of the precise details of the tissue) and the scene image under the operatingmicroscope module 2 can be displayed at the same time, which is convenient for the assistant doctor to view. The display in the front-screen detection module can display these two images simultaneously through two different areas (the observation image of themicroendoscope 1 and the scene image under the operating microscope module 2), which enables the observer to pass the touch screen Or press the button to selectively display the observation image of themicroendoscope 1 or the scene image under the operatingmicroscope module 2 in a larger area.

目视模组3还可包括保罗棱镜组34，保罗棱镜组34设于目镜33和第一成像透镜31之间，其用于折返光路，可以改变成像方向，也即通过它将第一成像透镜31所成的倒像再次上下颠倒变成正像，以使医生在目镜33中观察到手术显微镜模组2下的场景的正像。含有保罗棱镜组34的目视模组3使本发明该融合系统的目镜33成像与人体眼睛观察物体方向一样，有利于医生的观察和操作，提高手术效率和手术效果。Thevisual module 3 can also include aPaul prism group 34, which is arranged between theeyepiece 33 and thefirst imaging lens 31, which is used for refolding the optical path, and can change the imaging direction, that is, through the first imaging lens. The inverted image formed by 31 is turned upside down again to become an erect image, so that the doctor can observe the erect image of the scene under the operatingmicroscope module 2 in theeyepiece 33 . Thevision module 3 including thePaul prism group 34 enables theeyepiece 33 of the fusion system of the present invention to image in the same direction as the human eye observes the object, which is beneficial to the doctor's observation and operation, and improves the operation efficiency and effect.

本发明还提供的一种手术显微镜和显微内窥镜的融合方法，该方法配合一种手术显微镜和显微内窥镜融合系统使用，该系统包括显微内窥镜1、手术显微镜模组2及目视模组3，目视模组3包括目镜33、显微内窥镜图像获取单元及第一成像透镜31，如图3所示，包括以下步骤：The present invention also provides a fusion method of a surgical microscope and a microendoscope. The method is used in conjunction with a surgical microscope and a microendoscope fusion system. The system includes amicroendoscope 1 and a surgical microscope module. 2 and avisual module 3, thevisual module 3 includes aneyepiece 33, a microendoscope image acquisition unit and afirst imaging lens 31, as shown in Figure 3, including the following steps:

S1：将显微内窥镜1与显微内窥镜图像获取单元电性连接。S1: Electrically connect themicroendoscope 1 to the image acquisition unit of the microendoscope.

通过设置显微内窥镜1启动电路的操作开关将显微内窥镜1调至工作模式，显微内窥镜1获取的图像数据可传输至显微内窥镜图像获取单元。当显微内窥镜图像获取单元为透明显示器321时，显微内窥镜1与透明显示器321电性连接(无线或电缆等有线连接)；当显微内窥镜图像获取单元包括微显示器322时，显微内窥镜1与微显示器322电性连接。By setting the operation switch of the start-up circuit of themicroendoscope 1 to adjust themicroendoscope 1 to the working mode, the image data acquired by themicroendoscope 1 can be transmitted to the microendoscope image acquisition unit. When the microendoscope image acquisition unit is thetransparent display 321, themicroendoscope 1 is electrically connected to the transparent display 321 (wired connection such as wireless or cable); when the microendoscope image acquisition unit includes the microdisplay 322 , themicroendoscope 1 is electrically connected to the microdisplay 322 .

S2：将显微内窥镜图像获取单元所获取的图像成像至目镜33焦平面上。S2: Imaging the image acquired by the microendoscope image acquisition unit onto the focal plane of theeyepiece 33 .

当显微内窥镜图像获取单元包括透明显示器321时，透明显示器321置于目镜33的焦平面上，透明显示器321将显微内窥镜获取的图像显示至其显示屏上，由于透明显示器321具有一定厚度，可能会给成像光路带来像差，可以在手术显微镜模组2与第一成像透镜31之间的平行光路中设置补偿装置(图中未示出)进行校正，补偿装置优选空间光调制器或非球面透镜；当显微内窥镜图像获取单元包括微显示器322、投影物镜323及第一分光棱镜324时，微显示器322上的图像通过投影物镜323及第一分光棱镜324成像至目镜33的焦平面上。When the microendoscope image acquisition unit includes thetransparent display 321, thetransparent display 321 is placed on the focal plane of theeyepiece 33, and thetransparent display 321 displays the image acquired by the microendoscope on its display screen, because thetransparent display 321 It has a certain thickness, which may bring aberration to the imaging light path. A compensation device (not shown in the figure) can be set in the parallel light path between the operatingmicroscope module 2 and thefirst imaging lens 31 for correction. The compensation device preferably has a space for Light modulator or aspheric lens; when the microendoscope image acquisition unit includes the microdisplay 322, theprojection objective lens 323 and the firstbeam splitter prism 324, the image on the microdisplay 322 is formed by theprojection objective lens 323 and the firstbeam splitter prism 324 to the focal plane of theeyepiece 33.

S3：将显微内窥镜1置于手术显微镜模组2的视场范围内。S3: Place themicroendoscope 1 within the field of view of the operatingmicroscope module 2.

显微内窥镜1设置于一个支架上且能相对支架移动，在医生操作过程中，显微内窥镜1可在手术显微镜模组2视野范围内移动至医生想看到的位置。Themicroendoscope 1 is arranged on a bracket and can move relative to the bracket. During the operation of the doctor, themicroendoscope 1 can be moved to the position that the doctor wants to see within the field of view of the operatingmicroscope module 2 .

S4：医生在目镜33中能同时看到显微内窥镜1所获取到的组织精密细节的图像及手术显微镜模组2下所观察到的场景图像，能对组织实时活检，实现诊断和治疗为一体的手术操作，也即能实现对组织实时在线活检并治疗。S4: In theeyepiece 33, the doctor can simultaneously see the image of the precise details of the tissue obtained by themicroendoscope 1 and the scene image observed under the operatingmicroscope module 2, and can biopsy the tissue in real time to realize diagnosis and treatment. It is an integrated surgical operation, that is, it can realize real-time online biopsy and treatment of tissue.

作为优选步骤，在S3步骤之后，医生可先调整显微内窥镜图像获取单元的图像亮度和放大倍率，以便更清楚地观察图像。在本实施方案中，在透明显示器321或微显示器322上设置有亮度调整旋钮和倍率调整旋钮，医生在目镜中观察组织图像时，若需要更好地观察效果，可以通过调整旋钮来实现对亮度和放大倍率的调整。As a preferred step, after step S3, the doctor may first adjust the image brightness and magnification of the microendoscope image acquisition unit, so as to observe the image more clearly. In this embodiment, a brightness adjustment knob and a magnification adjustment knob are provided on thetransparent display 321 or the microdisplay 322. When the doctor observes the tissue image in the eyepiece, if a better observation effect is required, the adjustment knob can be used to adjust the brightness. and adjustment of magnification.

作为优选步骤，在S1步骤之前，将保罗棱镜组34设置于目镜33与第一成像透镜31之间的成像光路中。保罗棱镜组34用于折返光路，可以调整目镜33中的成像方向，使本发明该融合系统的目镜33成像与人体眼睛观察物体方向一样。As a preferred step, before step S1 , thePaul prism group 34 is arranged in the imaging optical path between theeyepiece 33 and thefirst imaging lens 31 . ThePaul prism group 34 is used to return the light path, and the imaging direction in theeyepiece 33 can be adjusted, so that the imaging direction of theeyepiece 33 of the fusion system of the present invention is the same as the direction of the human eye to observe the object.

作为优选步骤，在S1步骤之前，在该融合系统中添加屏前探测模组，供助手医生观看。屏前探测模组包括第二成像棱镜43、第二成像透镜42、第二相机41及显示器，第二成像棱镜43将手术显微镜模组2输出的平行光分成两部分，一部分进入目视模组3，一部分进入屏前探测模组。进入屏前探测模组的光经第二成像透镜42汇聚至第二相机41的感光片上，再经显示器输出，也即显示出手术显微镜模组2下的场景图像。As a preferred step, before step S1, a front-screen detection module is added to the fusion system for the assistant doctor to watch. The detection module in front of the screen includes asecond imaging prism 43, asecond imaging lens 42, asecond camera 41 and a display. Thesecond imaging prism 43 divides the parallel light output by the operatingmicroscope module 2 into two parts, and a part enters thevisual module 3. Part of it enters the detection module in front of the screen. The light entering the detection module in front of the screen is concentrated on the photosensitive sheet of thesecond camera 41 through thesecond imaging lens 42 , and then output through the display, that is, the scene image under the operatingmicroscope module 2 is displayed.

在包含有屏前探测模组的融合系统，屏前探测模组的显示器还能与显微内窥镜1电性连接，使该显示器能同时显示出显微内窥镜1的观察图像及手术显微镜模组2下的场景图像，在此融合系统下的融合方法，其屏前探测模组的显示器与显微内窥镜1的电性连接的步骤可根据需要设置实施顺序。In the fusion system including the detection module in front of the screen, the display of the detection module in front of the screen can also be electrically connected with themicroendoscope 1, so that the display can simultaneously display the observation image of themicroendoscope 1 and the operation The scene image under themicroscope module 2, the fusion method under this fusion system, the steps of the electrical connection between the display of the front-screen detection module and themicroendoscope 1 can be implemented in an order as required.

综上所述，本发明提供了一种手术显微镜和显微内窥镜的融合方法及与其相配合的融合系统，在目视系统3中设置显微内窥镜图像获取单元，该显微内窥镜获取单元与显微内窥镜1之间电性连接来传输图像数据，且显微内窥镜1设置于手术显微镜模组2的视野范围内，实现了手术显微镜与显微内窥镜1的融合，使医生可以在目镜33下同时观察到显微内窥镜1的观察图像及手术显微镜模组2下的场景图像，并能在目镜33观察下移动显微内窥镜1至想看到的位置，该融合系统可以在手术过程中对组织实时活检，实现诊断与治疗一体，提高了手术效率与手术效果。To sum up, the present invention provides a fusion method of an operating microscope and a microscopic endoscope and a fusion system matched with it. The endoscope acquisition unit and themicroendoscope 1 are electrically connected to transmit image data, and themicroendoscope 1 is arranged in the field of view of the operatingmicroscope module 2, realizing the operation microscope and the microendoscope The fusion of 1 enables the doctor to observe the observation image of themicroendoscope 1 and the scene image of the operatingmicroscope module 2 at the same time under theeyepiece 33, and can move themicroendoscope 1 to the desired position under the observation of theeyepiece 33. The fusion system can biopsy the tissue in real time during the operation, realize the integration of diagnosis and treatment, and improve the operation efficiency and effect.

以上所述仅是本发明的优选实施例而已，并非用以限定本发明，任何熟悉本专业的技术人员，在不脱离本发明的原理和宗旨的情况下可以对这些实施例进行多种变化、修改、替换和变型，本发明的范围由权利要求及其等同物限定。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any person skilled in the art can make various changes to these embodiments without departing from the principle and purpose of the present invention. Modifications, substitutions and variations, the scope of the invention is defined by the claims and their equivalents.

Claims

1. A fusion method based on a fusion system of an operation microscope and a micro endoscope is characterized in that the fusion system comprises the micro endoscope, an operation microscope module and a visual module, the micro endoscope comprises an imaging lens, an illuminating system, a camera and a starting circuit, and image data of the micro endoscope is transmitted to the visual module; the operation microscope module comprises an objective lens and an illuminating device, and light output by the operation microscope module is transmitted to the visual module; the visual module comprises a first imaging lens, a micro-endoscope image acquisition unit and an eyepiece, the micro-endoscope image acquisition unit is arranged between the eyepiece and the first imaging lens, the visual module is used for observing a scene image under the surgical microscope module and a tissue image under the micro-endoscope, and the fusion method comprises the following steps:

s1, electrically connecting the micro endoscope with the micro endoscope image acquisition unit: the start circuit is used for adjusting the micro endoscope to a working mode, so that image data acquired by the micro endoscope can be transmitted to the micro endoscope image acquisition unit;

s2, imaging the image generated by the microscopic endoscope image acquisition unit to the focal plane of the ocular lens;

s3: placing the micro-endoscope within a field of view of the surgical microscope module: observing the microendoscope in the eyepiece and moving it to a position that is desired by an observer;

s4: an observer can see the tissue image under the micro endoscope and the scene image under the operation microscope module in the eyepiece simultaneously, so that real-time online biopsy and treatment of the tissue are realized.

2. The fusion method based on the surgical microscope and the micro-endoscope fusion system according to claim 1, wherein the micro-endoscope image acquisition unit comprises a transparent display, and the step of electrically connecting in step S1 specifically comprises: the endoscope is electrically connected with the transparent display in a wireless or wired connection mode, and the transparent display receives image data acquired by the endoscope and generates an image; the step of imaging onto the focal plane of the eyepiece in step S2 specifically is: the transparent display is disposed on a focal plane of the eyepiece.

3. The fusion method of claim 1, wherein the endoscopic microscope image capturing unit comprises a micro-display, a projection objective and a first beam splitter prism, and the step of electrically connecting in step S1 specifically comprises: the micro endoscope is electrically connected with the micro display in a wireless or wired connection mode, and the micro display receives image data acquired by the micro endoscope and generates an image; the step of imaging onto the focal plane of the eyepiece in step S2 specifically is: and the generated image of the micro display sequentially passes through the projection objective and the first beam splitter prism to be imaged on a focal plane of the eyepiece.

4. The fusion method based on the surgical microscope and the micro endoscope fusion system according to the claim 1, characterized by further comprising the step S3.1 of adjusting the brightness and the magnification of the image generated by the micro endoscope image acquisition unit.

5. The fusion method based on a surgical microscope and a micro-endoscope fusion system according to claim 1, characterized in that the visual system further comprises a paul prism set for adjusting the imaging direction of the image on the focal plane of the eyepiece.

6. The fusion method based on the fusion system of the surgical microscope and the micro endoscope, according to claim 1, wherein the fusion system further comprises a front screen detection module for an assistant observer to view, the front screen detection module comprises a second imaging prism, a second imaging lens, a second camera and a display, the second imaging prism divides the light output by the surgical microscope module into two parts, one part enters the visual module and the other part enters the front screen detection module, wherein the light entering the front screen detection module converges on the photosensitive sheet of the second camera through the second imaging lens, and then the scene image under the surgical microscope module is output through the display.

7. The fusion method based on the fusion system of the surgical microscope and the micro endoscope, according to claim 6, further comprising a step S4.1 of electrically connecting the display in the front screen detection module with the micro endoscope, so that the scene image under the surgical microscope module and the tissue image under the micro endoscope can be displayed on the display at the same time.

8. A fusion method based on a surgical microscope and a microendoscope fusion system according to claim 2, wherein the size of the transparent display is smaller than the field of view of the eyepiece and is an integer multiple of the size of the camera in the microendoscope.

9. The fusion method of claim 3 in which the size of the image imaged by the microdisplay onto the focal plane of the eyepiece is smaller than the field of view of the eyepiece and is an integer multiple of the size of the camera in the microendoscope.

10. The fusion method based on the fusion system of the surgical microscope and the micro endoscope, as claimed in claim 2, further comprising an optical path compensation device for compensating imaging aberration in the optical path, which is disposed in the optical path between the surgical microscope module and the first imaging lens.

11. The fusion method based on the surgical microscope and the micro endoscope fusion system according to claim 10, wherein the optical path compensation device is a spatial light modulator or an aspheric lens.