WO2023078290A1 - Mark sharing method and apparatus for surgical robot, and system, device and medium - Google Patents

Abstract

The present application relates to a mark sharing method and apparatus for a surgical robot, and a system, a device and a medium. The mark sharing method comprises: acquiring the currently generated initial medical image; after a first mark instruction, which is input by a user, is acquired, sending first mark information to a second surgical robot end in real time, and fusing the first mark information into the initial medical image, wherein the first mark instruction comprises the first mark information; and after second mark information, which is sent by the second surgical robot end in real time, is received, fusing the second mark information into the initial medical image. Therefore, the real-time sharing of data marks of a master end and a slave end during surgery is realized, thereby effectively solving the problem of poor communication between a doctor at the master end and an assistant doctor at the slave end during a surgical operation process of a remote robot.

Description

Translated fromChinese

用于手术机器人的标记共享方法、装置、系统、设备及介质Marker sharing method, device, system, equipment and medium for surgical robot技术领域technical field

本申请涉及医疗技术领域，具体涉及一种用于手术机器人的标记共享方法、标记共享装置、手术机器人系统、可读存储介质与电子设备。The present application relates to the field of medical technology, in particular to a marker sharing method for a surgical robot, a marker sharing device, a surgical robot system, a readable storage medium and electronic equipment.

背景技术Background technique

随着科技的进步，5G技术的发展，AI人工智能的崛起。远程医疗手术在大环境下有了快速发展的契机。远程医疗是指通过计算机技术、遥感、遥测、遥控技术为依托，充分发挥大医院或专科医疗中心的医疗技术和医疗设备优势，对医疗条件较差的边远地区、海岛或舰船上的伤病员进行远距离诊断、治疗和咨询，旨在提高诊断与医疗水平，降低医疗开支，满足广大人民群众保健需求的一项全新的医疗服务。With the advancement of science and technology, the development of 5G technology, the rise of AI artificial intelligence. Telemedicine surgery has an opportunity for rapid development in the general environment. Telemedicine refers to relying on computer technology, remote sensing, telemetry, and remote control technology, giving full play to the advantages of medical technology and medical equipment in large hospitals or specialized medical centers, and treating injuries and diseases in remote areas, islands, or ships with poor medical conditions. It is a brand-new medical service that aims to improve the level of diagnosis and medical treatment, reduce medical expenses, and meet the health needs of the general public through remote diagnosis, treatment and consultation.

目前，远程医疗技术已经从最初的电视监护、电话远程诊断发展到利用高速网络进行数字、图像、语音的综合传输，并且实现了实时的语音和高清晰图像的交流，为现代医学的应用提供了更广阔的发展空间。国外在这一领域的发展已有40多年的历史，而我国只在最近几年才得到重视和发展。目前的远程医疗所采用的技术包含有：(1)画板标记技术，提供一种远程实时分会诊系统，系统包括图像浏览、图像标记、文字输入、音视频的实时同步；(2)医学图像立体视觉显示，计算机通过输入设备获取医学影像，再通过软件系统进行图像处理，最后通过立体视觉模块、3D模块或2D模块显示并进行视觉联动；(3)医疗影像远程实时会诊，提供一种病例实时分会诊系统，包括会诊创建管理模块、电子白板、音视频采集-回放模块、网络数据传输模块和中央处理模块等。At present, telemedicine technology has developed from the initial TV monitoring and telephone remote diagnosis to the comprehensive transmission of numbers, images, and voices using high-speed networks, and has realized real-time voice and high-definition image communication, which provides a solid foundation for the application of modern medicine. Broader development space. The development of foreign countries in this field has a history of more than 40 years, while our country has only received attention and development in recent years. The current technologies used in telemedicine include: (1) drawing board marking technology, which provides a remote real-time sub-consultation system, which includes image browsing, image marking, text input, and real-time synchronization of audio and video; (2) medical image stereo Visual display, the computer obtains medical images through input devices, then processes the images through the software system, and finally displays and performs visual linkage through the stereo vision module, 3D module or 2D module; (3) remote real-time consultation of medical images, providing a real-time case Sub-consultation system, including consultation creation management module, electronic whiteboard, audio and video collection-playback module, network data transmission module and central processing module, etc.

以上技术基本都是独立系统，大多是针对远程会诊场景，针对远程手术的画板共享系统等，重在教学和分享，不具备在实时远程医疗手术中进行多端设备数据融合同步，并实现指导医师手术的作用。The above technologies are basically independent systems, most of which are for remote consultation scenarios, for remote surgery drawing board sharing systems, etc., focusing on teaching and sharing, and do not have the ability to perform multi-terminal device data fusion and synchronization in real-time telemedicine operations, and to guide physicians in surgery role.

发明内容Contents of the invention

为了解决现有技术中的技术问题，本申请的目的在于提供一种用于手术机器人的标记共享方法、标记共享装置、手术机器人系统、可读存储介质与电子设备，可实现手术中主从两端的数据标记实时共享，有效地解决远程机器人手术过程中主端医生和从端辅助医生之间交流不畅的问题，从而有效提高该场景应用中的手术精准率，确保手术的安全顺利进行。In order to solve the technical problems in the prior art, the purpose of this application is to provide a marker sharing method for surgical robots, a marker sharing device, a surgical robot system, a readable storage medium and electronic equipment, which can realize master-slave dual Real-time sharing of data markers on the end, effectively solving the problem of poor communication between the master doctor and the slave assistant doctor during remote robot surgery, thereby effectively improving the surgical accuracy in this scenario application and ensuring the safe and smooth operation.

为实现上述目的，根据本申请的第一个方面，提供了一种用于手术机器人的标记共享方法，适用于第一手术机器人端，所述标记共享方法包括：In order to achieve the above purpose, according to the first aspect of the present application, a marker sharing method for surgical robots is provided, which is applicable to the first surgical robot end, and the marker sharing method includes:

获取当前生成的初始医学影像；Obtain the currently generated initial medical image;

当获取到用户输入的第一标记指令后，将所述第一标记信息实时发送到第二手术机器人端，并在所述初始医学影像中融合所述第一标记信息，其中，所述第一标记指令包括所述第一标记信息；After obtaining the first marking instruction input by the user, the first marking information is sent to the second surgical robot in real time, and the first marking information is fused in the initial medical image, wherein the first The marking instruction includes the first marking information;

当接收到所述第二手术机器人端实时发送的第二标记信息后，在所述初始医学影像中融合所述第二标记信息。After receiving the second marker information sent by the second surgical robot in real time, the second marker information is fused in the initial medical image.

可选地，当接收到所述第二手术机器人端实时发送的第二标记信息后，在融合所述第一标记信息的所述初始医学影像中融合所述第二标记信息。Optionally, after receiving the second marker information sent by the second surgical robot in real time, the second marker information is fused in the initial medical image fused with the first marker information.

可选地，将融合所述第一标记信息和/或所述第二标记信息的所述初始医学影像实时发送至所述第二手术机器人端。Optionally, the initial medical image fused with the first marker information and/or the second marker information is sent to the second surgical robot in real time.

可选地，在获取到用户输入的第一标记指令之前，还包括：启动标记模式，所述标记模式允许用户 输入所述第一标记指令。Optionally, before acquiring the first marking instruction input by the user, the method further includes: starting a marking mode, where the marking mode allows the user to input the first marking instruction.

可选地，在所述初始医学影像中融合所述第一标记信息，包括：Optionally, fusing the first marker information in the initial medical image includes:

根据所述第一标记信息中的位置信息，在所述初始医学影像上重建对应于所述第一标记信息的第一标记；Reconstructing a first marker corresponding to the first marker information on the initial medical image according to the position information in the first marker information;

在所述初始医学影像中融合所述第二标记信息，包括：Fusing the second marker information in the initial medical image includes:

根据所述第二标记信息中的位置信息，在所述初始医学影像上重建对应于所述第二标记信息的第二标记。Reconstructing a second marker corresponding to the second marker information on the initial medical image according to the position information in the second marker information.

可选地，在所述初始医学影像中融合所述第一标记信息和/或在所述初始医学影像中融合所述第二标记信息之后，所述标记共享方法还包括：Optionally, after fusing the first marker information in the initial medical image and/or fusing the second marker information in the initial medical image, the marker sharing method further includes:

获取当前生成的所述初始医学影像的图像变化信息，并根据所述初始医学影像的图像变化信息，更新所述第一标记信息和/或所述第二标记信息，以使所述第一标记信息和/或所述第二标记信息实时跟随所述初始医学影像的变化。Acquiring image change information of the currently generated initial medical image, and updating the first marker information and/or the second marker information according to the image change information of the initial medical image, so that the first marker information and/or the second marker information follows changes in the initial medical image in real time.

可选地，所述标记共享方法还包括：Optionally, the tag sharing method also includes:

获取所述第一手术机器人端和/或所述第二手术机器人端的标记状态，并根据所述第一手术机器人端和/或所述第二手术机器人端的标记状态，输出主从控制指令或启用标记指令；所述主从控制指令用于控制所述第一手术机器人端或所述第二手术机器人端的运动；所述启用标记指令用于启动所述标记模式。Obtain the flag state of the first surgical robot end and/or the second surgical robot end, and output master-slave control instructions or enable A marking command; the master-slave control command is used to control the movement of the first surgical robot end or the second surgical robot end; the enabling marking command is used to start the marking mode.

可选地，所述标记共享方法还包括：获取用户输入的退出标记指令，并根据所述退出标记指令退出所述标记模式。Optionally, the marking sharing method further includes: obtaining an exit marking instruction input by a user, and exiting the marking mode according to the exit marking instruction.

可选地，还包括：Optionally, also include:

根据所述启用标记指令进入操作锁定状态，以禁止所述第一手术机器人端或所述第二手术机器人端的运动；Entering into an operation lock state according to the activation flag instruction, so as to prohibit the movement of the first surgical robot end or the second surgical robot end;

根据所述退出标记指令进入操作解锁状态，以允许所述第一手术机器人端或所述第二手术机器人端的运动。An operation unlock state is commanded according to the exit flag to allow movement of the first surgical robot end or the second surgical robot end.

可选地，所述标记共享方法还包括：Optionally, the tag sharing method also includes:

获取所述第二手术机器人端的标记状态，并在确定所述第二手术机器人端当前为非标记状态时启动所述标记模式，并实时向所述第二手术机器人端发送本地当前的标记状态。Acquiring the marking state of the second surgical robot, and starting the marking mode when it is determined that the second surgical robot is currently in a non-marking state, and sending the local current marking state to the second surgical robot in real time.

可选地，所述第一标记信息和所述第二标记信息包括图形和/或文字信息。Optionally, the first marking information and the second marking information include graphics and/or text information.

可选地，所述图形包括自定义图形或预设图形。Optionally, the graphics include custom graphics or preset graphics.

可选地，所述标记共享方法还包括：Optionally, the tag sharing method also includes:

获取所述第一手术机器人端的第一脚踏板的操作状态，并根据所述第一脚踏板的操作状态启动所述标记模式；Acquiring the operating state of the first foot pedal at the first surgical robot end, and starting the marking mode according to the operating state of the first foot pedal;

获取所述第一手术机器人端的第二脚踏板的操作状态，并根据所述第二脚踏板的操作状态退出所述标记模式。Obtaining the operation state of the second foot pedal on the first surgical robot end, and exiting the marking mode according to the operation state of the second foot pedal.

可选地，所述标记共享方法还包括：Optionally, the tag sharing method also includes:

根据所述第一手术机器人端的交互界面上的标记按钮所输出的指令启动所述标记模式；Starting the marking mode according to an instruction output by a marking button on the interactive interface of the first surgical robot;

根据所述第一手术机器人端的所述交互界面上的退出标记按钮所输出的指令退出所述标记模式。Exit the marking mode according to an instruction output by the exit marking button on the interaction interface of the first surgical robot.

可选地，所述标记共享方法还包括：向所述第二手术机器人端实时无线传输音频数据和/或视频数据。Optionally, the marker sharing method further includes: wirelessly transmitting audio data and/or video data to the second surgical robot in real time.

可选地，所述获取当前生成的初始医学影像，包括：利用三维建模和人工智能算法对实时采集到的 影像数据进行处理，以生成三维的所述初始医学影像。Optionally, the acquiring the currently generated initial medical image includes: using 3D modeling and artificial intelligence algorithms to process the image data collected in real time, so as to generate the 3D initial medical image.

为实现上述目的，根据本申请的第二个方面，提供了一种用于手术机器人的标记共享装置，适用于第一手术机器人端，包括：In order to achieve the above purpose, according to the second aspect of the present application, a marker sharing device for surgical robots is provided, which is suitable for the first surgical robot end, including:

影像数据获取模块，用于获取当前生成的初始医学影像；An image data acquisition module, configured to acquire the currently generated initial medical image;

标记指令获取模块，用于获取用户输入的第一标记指令，所述第一标记指令包括第一标记信息，还用于获取第二手术机器人端实时发送的第二标记信息；A marking instruction acquisition module, configured to acquire a first marking instruction input by a user, the first marking instruction including first marking information, and also used to acquire second marking information sent in real time by the second surgical robot end;

数据融合模块，用于在所述初始医学影像中融合所述第一标记信息和/或所述第二标记信息。A data fusion module, configured to fuse the first marker information and/or the second marker information in the initial medical image.

可选地，所述标记共享装置还包括信息共享模块，用于将融合所述第一标记信息和/或所述第二标记信息的所述初始医学图像实时发送至所述第二手术机器人端。Optionally, the marker sharing device further includes an information sharing module, configured to send the initial medical image fused with the first marker information and/or the second marker information to the second surgical robot in real time .

可选地，所述标记共享装置还包括显示模块，用于显示融合所述第一标记信息和/或所述第二标记信息的所述初始医学图像。Optionally, the marker sharing device further includes a display module, configured to display the initial medical image fused with the first marker information and/or the second marker information.

为实现上述目的，根据本申请的第三个方面，提供了一种手术机器人系统，包括第一手术机器人端和第二手术机器人端，所述第一手术机器人端包括上述的用于手术机器人的标记共享装置。In order to achieve the above object, according to the third aspect of the present application, a surgical robot system is provided, including a first surgical robot end and a second surgical robot end, the first surgical robot end includes the above-mentioned surgical robot end Mark the shared device.

可选地，所述手术机器人系统还包括图像采集装置，用于获取预定对象的初始医学影像。Optionally, the surgical robot system further includes an image acquisition device, configured to acquire an initial medical image of a predetermined object.

可选地，所述第一手术机器人端还包括第一脚踏板和第二脚踏板，所述标记指令获取模块包括第一操作臂和第二操作臂；Optionally, the first surgical robot end further includes a first foot pedal and a second foot pedal, and the marking instruction acquisition module includes a first operating arm and a second operating arm;

所述第一脚踏板用于输出启用标记指令，以启动标记模式；The first foot pedal is used to output an enable flag command to start the flag mode;

所述第二脚踏板用于输出退出标记指令，以退出所述标记模式；The second foot pedal is used to output an exit marking command to exit the marking mode;

所述第一操作臂用于根据用户输入的操作指令选择对应于所述第一标记信息的第一标记；The first operating arm is used to select a first marker corresponding to the first marker information according to an operation instruction input by a user;

所述第二操作臂用于根据用户输入的操作指令，采用已选择的所述第一标记，在所述初始医学影像上创建所述第一标记。The second operating arm is configured to use the selected first marker to create the first marker on the initial medical image according to an operation instruction input by a user.

可选地，所述第一手术机器人端还包括交互界面，所述交互界面包括标记按钮和退出标记按钮；所述标记指令获取模块包括键盘和鼠标；Optionally, the first surgical robot terminal also includes an interactive interface, the interactive interface includes a marking button and an exit marking button; the marking instruction acquisition module includes a keyboard and a mouse;

所述交互界面显示所述初始医学影像；The interactive interface displays the initial medical image;

所述标记按钮用于输出启用标记指令，以启动标记模式；The mark button is used to output the enable mark instruction to start the mark mode;

所述退出标记按钮用于输出退出标记指令，以退出所述标记模式；The exit marking button is used to output an exit marking instruction to exit the marking mode;

所述键盘和所述鼠标用于根据用户输入的指令生成对应于所述第一标记信息的第一标记。The keyboard and the mouse are used to generate a first mark corresponding to the first mark information according to an instruction input by a user.

可选地，所述手术机器人系统还包括音频和/或视频采集模块，用于采集音频和/或视频数据。Optionally, the surgical robot system further includes an audio and/or video collection module for collecting audio and/or video data.

为实现上述目的，根据本申请的第四个方面，提供了一种可读存储介质，其上存储有程序，当所述程序被执行时，执行上述的用于手术机器人的标记共享方法。In order to achieve the above object, according to a fourth aspect of the present application, a readable storage medium is provided, on which a program is stored, and when the program is executed, the above-mentioned marker sharing method for a surgical robot is executed.

为实现上述目的，根据本申请的第五个方面，提供了一种电子设备，用于执行用于手术机器人的标记共享方法，包括处理器和所述的可读存储介质，所述处理器被配置用于执行所述可读存储介质上所存储的程序。In order to achieve the above object, according to a fifth aspect of the present application, an electronic device is provided for performing a marker sharing method for a surgical robot, including a processor and the readable storage medium, and the processor is controlled by configured to execute the program stored on the readable storage medium.

与现有技术相比，本申请的用于手术机器人的标记共享方法、标记共享装置、手术机器人系统、可读存储介质与电子设备具有如下优点：Compared with the prior art, the marker sharing method for surgical robots, marker sharing device, surgical robot system, readable storage medium and electronic equipment of the present application have the following advantages:

本申请利用数据融合技术和标记共享技术，实现手术过程中主端和从端的数据标记实时共享，有效地解决远程手术机器人手术过程中主端医生和从端辅助医生对手术信息，如确定病灶位置、手术方案等交流不明确的问题，从而有效提高远程医疗中的手术精准率，确保手术的安全顺利进行。如此，本申请通过数据标记实时共享，实现了手术过程中主端医生和从端辅助医生之间的有效沟通和交流，使主端医生能够更好的指导从端辅助医生进行手术操作，提高手术效率和手术成功率。This application uses data fusion technology and marker sharing technology to realize real-time sharing of data markers between the master end and slave end during the operation process, effectively solving the problem of operation information between the master end doctor and the slave end assistant doctor during the remote surgical robot operation, such as determining the location of the lesion , Surgical plans and other unclear communication issues, so as to effectively improve the surgical accuracy rate in telemedicine and ensure the safe and smooth operation. In this way, this application realizes effective communication and communication between the master doctor and the slave assistant doctor during the operation process through the real-time sharing of data tags, so that the master doctor can better guide the slave assistant doctor to perform the operation and improve the operation efficiency. efficiency and surgical success.

本申请通过手术机器人系统中已有设备，如脚踏板、操作手臂、键盘和鼠标等进行标记操作，方便了医生操作，有效减少了标记操作对手术的干扰。This application uses the existing equipment in the surgical robot system, such as foot pedals, operating arms, keyboards, and mice, to perform marking operations, which facilitates the operation of doctors and effectively reduces the interference of marking operations on surgery.

本申请利用三维建模和人工智能算法对实时采集到的影像数据进行处理，以生成三维的初始医学影像，在三维初始医学影像上进行标记和主从端之间的交流，以提高更精准的参考价值，使得远程手术场景应用精确率大幅提升。This application uses 3D modeling and artificial intelligence algorithms to process the image data collected in real time to generate a 3D initial medical image, and to mark and communicate between the master and slave on the 3D initial medical image to improve more accurate The reference value greatly improves the application accuracy of remote surgery scenarios.

附图说明Description of drawings

本申请的实施方法以及相关实施例的特征、性质和优势将通过结合下列附图进行描述，其中：The features, properties and advantages of the methods of implementing the application and related embodiments will be described with reference to the following drawings, in which:

图1是根据本申请一优选实施例的手术机器人系统的主端和从端异地的手术场景示意图；Fig. 1 is a schematic diagram of a surgical scene where the master end and the slave end of a surgical robot system are in different places according to a preferred embodiment of the present application;

图2是根据本申请一优选实施例的手术机器人系统的结构框图；Fig. 2 is a structural block diagram of a surgical robot system according to a preferred embodiment of the present application;

图3是根据本申请一优选实施例的医生主控制台的结构示意图；Fig. 3 is a schematic structural diagram of a doctor's main console according to a preferred embodiment of the present application;

图4是根据本申请一优选实施例的从端控制装置的结构示意图；FIG. 4 is a schematic structural diagram of a slave control device according to a preferred embodiment of the present application;

图5是根据本申请一优选实施例的主端标记创建的工作流程图；Fig. 5 is the working flow chart that the main end mark creates according to a preferred embodiment of the present application;

图6是根据本申请一优选实施例的主端标记创建的操作原理图；Fig. 6 is a schematic diagram of the operation principle of master-side tag creation according to a preferred embodiment of the present application;

图7是根据本申请一优选实施例的主端显示模块的主端交互界面显示主端标记、主端撤销按钮、图形选择窗口的示意图；Fig. 7 is a schematic diagram showing a main terminal mark, a main terminal cancel button, and a graphic selection window on the main terminal interactive interface of the main terminal display module according to a preferred embodiment of the present application;

图8是根据本申请一优选实施例的主端向从端共享主端标记的流程图；FIG. 8 is a flow chart of a master end sharing a master end mark to a slave end according to a preferred embodiment of the present application;

图9是根据本申请一优选实施例的从端标记创建的工作流程图；Fig. 9 is a workflow flow diagram of creation of slave-end tags according to a preferred embodiment of the present application;

图10是根据本申请一优选实施例的从端标记创建的操作原理图；Fig. 10 is a schematic diagram of the operation principle of creating a slave tag according to a preferred embodiment of the present application;

图11是根据本申请一优选实施例的从端显示模块的从端交互界面显示从端标记、标记按钮、标记撤销按钮、图形选择窗口的示意图；Fig. 11 is a schematic diagram showing a slave mark, a mark button, a mark cancel button, and a graphic selection window on the slave end interactive interface of the slave display module according to a preferred embodiment of the present application;

图12是根据本申请一优选实施例的从端向主端共享从端标记的流程图；Fig. 12 is a flow chart of sharing a slave mark from a slave to a master according to a preferred embodiment of the present application;

图13是根据本申请一优选实施例的手术机器人系统将数据进行融合的原理图；Fig. 13 is a schematic diagram of data fusion by the surgical robot system according to a preferred embodiment of the present application;

图14是根据本申请一优选实施例的手术机器人系统将数据进行融合并进行显示的状态示意图。Fig. 14 is a schematic diagram of a state of fusion and display of data by the surgical robot system according to a preferred embodiment of the present application.

图中：100-主端；In the figure: 100-master end;

101-医生主控制台；1011-主端影像数据获取模块；1012-主端标记指令获取模块；1013-主端数据融合模块；1014-主端信息共享模块；1015-主端显示模块；1016-主端音频和/或视频采集模块；1017-主端数据传输模块；101-doctor main console; 1011-master image data acquisition module; 1012-master marking command acquisition module; 1013-master data fusion module; 1014-master information sharing module; 1015-master display module; 1016- Main-end audio and/or video acquisition module; 1017-master-end data transmission module;

1041-左脚踏板；1042-右脚踏板；1043-左操作臂；1044-右操作臂；1045-左控制手柄；1046-右控制手柄；1041-left pedal; 1042-right pedal; 1043-left operating arm; 1044-right operating arm; 1045-left control handle; 1046-right control handle;

G1-主端交互界面；G2-主端的画图工具；G3-矩形的预设主端标记；G4-圆形的预设主端标记；G5-圆形的主端标记；G6-自定义的主端标记；G7-矩形的主端标记；G8-主端撤销按钮；G1-master interface; G2-master drawing tool; G3-rectangular preset master mark; G4-circular preset master mark; G5-circular master mark; G6-custom master End mark; G7-rectangular main end mark; G8-master end undo button;

200-从端；200-slave;

201-手术机器人；201 - surgical robot;

202-从端控制装置；2021-从端影像数据获取模块；2022-从端标记指令获取模块；2023-从端数据融合模块；2024-从端信息共享模块；2025-从端显示模块；2026-从端音频和/或视频采集模块；2027-从端数据传输模块；202-slave control device; 2021-slave image data acquisition module; 2022-slave label instruction acquisition module; 2023-slave data fusion module; 2024-slave information sharing module; 2025-slave display module; 2026- Slave audio and/or video acquisition module; 2027-slave data transmission module;

2071-键盘；2072-键盘；2071-keyboard; 2072-keyboard;

H1-从端交互界面；H2-从端的画图工具；H3-矩形的预设从端标记；H4-圆形的预设从端标记；H5-圆形的从端标记；H6-自定义的从端标记；H7-矩形的从端标记；H8-从端撤销按钮；H9-标记按钮；H1-slave interactive interface; H2-slave drawing tool; H3-rectangular preset slave mark; H4-round preset slave mark; H5-circular slave mark; H6-custom slave End mark; H7-Rectangular slave end mark; H8-Slave end undo button; H9-Mark button;

203-图像台车；204-无菌台；211-隔板。203-image trolley; 204-sterile table; 211-partition.

具体实施方式Detailed ways

以下通过特定的具体实例说明本申请的实施方式，本领域技术人员可由本说明书所揭露的内容轻易地了解本申请的其他优点与功效。本申请还可以通过另外不同的具体实施方式加以实施或应用，本说明书中的各项细节也可以基于不同观点与应用，在没有背离本申请的精神下进行各种修饰或改变。需要说明的是，本实施例中所提供的图示仅以示意方式说明本申请的基本构想，遂图式中仅显示与本申请中有关的组件而非按照实际实施时的组件数目、形状及尺寸绘制，其实际实施时各组件的型态、数量及比例可随意的进行改变，且其组件布局型态也可能更为复杂。Embodiments of the present application are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the diagrams provided in this embodiment are only schematically illustrating the basic idea of the application, and only the components related to the application are shown in the diagrams rather than the number, shape and shape of the components in actual implementation. Dimensional drawing, the type, quantity and proportion of each component can be changed freely during its actual implementation, and its component layout type may also be more complicated.

另外，以下说明内容的各个实施例分别具有一或多个技术特征，然此并不意味着使用本申请者必需同时实施任一实施例中的所有技术特征，或仅能分开实施不同实施例中的一部或全部技术特征。换句话说，在实施为可能的前提下，本领域技术人员可依据本申请的公开内容，并视设计规范或实作需求，选择性地实施任一实施例中部分或全部的技术特征，或者选择性地实施多个实施例中部分或全部的技术特征的组合，借此增加本申请实施时的弹性。In addition, each embodiment of the content described below has one or more technical features, but this does not mean that the applicant must implement all the technical features in any embodiment at the same time, or can only implement different embodiments separately. Some or all of the technical features. In other words, on the premise that the implementation is possible, those skilled in the art can selectively implement some or all of the technical features in any embodiment according to the disclosure of the application and depending on the design specifications or implementation requirements, or Selectively implement a combination of some or all of the technical features in multiple embodiments, thereby increasing the flexibility of implementing the present application.

如在本说明书中所使用的，单数形式“一”、“一个”以及“该”包括复数对象，复数形式“多个”包括两个以上的对象，除非内容另外明确指出外。如在本说明书中所使用的，术语“或”通常是以包括“和/或”的含义而进行使用的，除非内容另外明确指出外，以及术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接。可以是机械连接，也可以是电连接。可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通或两个元件的相互作用关系。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本申请中的具体含义。As used in this specification, the singular forms "a", "an" and "the" include plural objects, and the plural form "a plurality" includes two or more objects, unless the content clearly states otherwise. As used in this specification, the term "or" is generally used in the sense including "and/or", unless the content clearly indicates otherwise, and the terms "install", "connect" and "connect" should be To understand it in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection. It can be a mechanical connection or an electrical connection. It can be directly connected or indirectly connected through an intermediary, and it can be the internal communication of two elements or the interaction relationship between two elements. Those of ordinary skill in the art can understand the specific meanings of the above terms in this application according to specific situations.

为使本申请的目的、优点和特征更加清楚，以下结合附图对本申请作进一步详细说明。需说明的是，附图均采用非常简化的形式且均使用非精准的比例，仅用以方便、明晰地辅助说明本申请实施例的目的。在不冲突的情况下，下述的实施方式及实施方式中的特征可以相互补充或相互组合。In order to make the purpose, advantages and features of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. It should be noted that the drawings are all in very simplified form and use imprecise scales, and are only used to facilitate and clearly assist the purpose of illustrating the embodiments of the present application. In the case of no conflict, the following embodiments and the features in the embodiments may complement each other or be combined with each other.

图1示出了根据本申请一优选实施例的手术机器人系统的应用场景。如图1所示，本申请一优选实施例提供一种手术机器人系统，包括通信连接的第一手术机器人端和第二手术机器人端。所述第一手术机器人端和第二手术机器人端中的一个为主端100，另一个为从端200。为了简明起见，在以下描述中假设第一手术机器人端为主端100，第二手术机器人端为从端200，本领域技术人员应当能够修改以下描述，在细节上作适当修改后将所述描述用于第一手术机器人端为从端200，以及第二手术机器人端为主端100的情况。Fig. 1 shows an application scenario of a surgical robot system according to a preferred embodiment of the present application. As shown in FIG. 1 , a preferred embodiment of the present application provides a surgical robot system, including a first surgical robot end and a second surgical robot end connected in communication. One of the first surgical robot end and the second surgical robot end is themaster end 100 , and the other is theslave end 200 . For the sake of brevity, in the following description, it is assumed that the first surgical robot end is themain end 100, and the second surgical robot end is theslave end 200. Those skilled in the art should be able to modify the following description, and describe the description after making appropriate modifications in details. It is used when the first surgical robot end is theslave end 200 and the second surgical robot end is themain end 100 .

所述主端100和从端200可位于不同的房间、不同的医院或不同的城市，以实现远程医疗。所述主端100包括医生主控制台101，以供医生使用双手及脚来实现远程医疗。所述从端200包括手术机器人201，所述手术机器人201包括机械臂，所述机械臂的末端用于可拆卸地连接手术器械或图像采集装置，所述图像采集装置包括但不限于内窥镜，以控制操作手术器械或图像采集装置。在手术过程中，利用医生主控制台101与机械臂及手术器械所构成的主从控制关系，医生通过医生主控制台101的主操作单元远程操作手术机器人201，使机械臂和手术器械根据医生主控制台101的运动而运动，如根据医生手部的操作而运动，从而对病床上的患者进行微创伤手术治疗。Themaster terminal 100 and theslave terminal 200 may be located in different rooms, different hospitals or different cities, so as to realize telemedicine. Themain terminal 100 includes a doctor'smain console 101 for the doctor to use his hands and feet to realize telemedicine. Theslave end 200 includes asurgical robot 201, thesurgical robot 201 includes a mechanical arm, and the end of the mechanical arm is used to detachably connect to a surgical instrument or an image acquisition device, and the image acquisition device includes but is not limited to an endoscope , to control the operation of surgical instruments or image acquisition devices. During the operation, using the master-slave control relationship formed by the doctor'smain console 101 and the robotic arm and surgical instruments, the doctor remotely operates thesurgical robot 201 through the main operation unit of the doctor'smain console 101, so that the robotic arm and surgical instruments operate according to the doctor's The movement of themain console 101 moves, for example, according to the operation of the doctor's hands, so as to perform minimally invasive surgical treatment on the patient on the hospital bed.

如图1和图2所示，所述从端200还包括从端控制装置202，通常与图像台车203相集成。所述图像台车203还可放置内窥镜以及其他相关设备(如从端显示模块和一些线缆)。所述内窥镜用于获取人体组织器官、手术器械、血管以及体液等手术环境的图像，且所述内窥镜和手术器械分别通过患者身体上的创口进入病患位置。所述图像台车203设置在病床的旁边，并与手术机器人201分开设置。As shown in FIG. 1 and FIG. 2 , theslave end 200 also includes a slaveend control device 202 , which is usually integrated with animage trolley 203 . Theimage trolley 203 can also place an endoscope and other related equipment (such as a slave display module and some cables). The endoscope is used to obtain images of surgical environments such as human tissues and organs, surgical instruments, blood vessels, and body fluids, and the endoscope and surgical instruments enter the patient's position through wounds on the patient's body. Theimage trolley 203 is set beside the hospital bed and is set separately from thesurgical robot 201 .

所述从端控制装置202与手术机器人201有线或无线通信。所述医生主控制台101用于根据外界指令生成并输出主从控制指令，并发送至从端控制装置202。所述从端控制装置202用于根据所接收的主从控制指令控制手术机器人201的运动。更详细地，所述从端控制装置202用于根据医生主控制台101所发送的运动信息和预设的主从映射关系，输出主从控制指令，以控制手术机器人201执行主从控制指令来驱动手术器械或内窥镜的运动。例如，所述从端控制装置202根据获取的主操作单元的移动速度，控制手术机器人201以驱动手术器械或内窥镜移动，并根据获取的主操作单元的转动角度或转动速度，控制手术机器人201以驱动手术器械或内窥镜转动，还可根据获取的主操作单元的弯曲角度或弯曲方向，控制手术机器人201以驱动手术器械或内窥镜弯曲。Theslave control device 202 communicates with thesurgical robot 201 by wire or wirelessly. Thedoctor master console 101 is used to generate and output master-slave control instructions according to external instructions, and send them to the slave-end control device 202 . Theslave control device 202 is used to control the movement of thesurgical robot 201 according to the received master-slave control instructions. In more detail, theslave control device 202 is used to output master-slave control instructions according to the motion information sent by the doctor'smaster console 101 and the preset master-slave mapping relationship, so as to control thesurgical robot 201 to execute the master-slave control instructions to Drives the movement of surgical instruments or endoscopes. For example, theslave control device 202 controls thesurgical robot 201 to drive the surgical instrument or endoscope to move according to the acquired moving speed of the main operating unit, and controls the surgical robot according to the acquired rotational angle or rotational speed of the main operating unit. 201 to drive the surgical instrument or endoscope to rotate, and also control thesurgical robot 201 to drive the surgical instrument or endoscope to bend according to the obtained bending angle or bending direction of the main operation unit.

进一步地，所述从端200还可包括手术中的辅助设备，如无菌台、呼吸机、检测装置等。本实施例中，所述从端200还包括无菌台204，设置在病床的旁边，用于放置手术器械，以建立无菌屏障，防止无菌手术器械及辅料再污染，而且也可防止手术器械、辅料遗漏、遗失等。Further, theslave end 200 may also include auxiliary equipment in the operation, such as a sterile table, a ventilator, a detection device, and the like. In this embodiment, theslave end 200 also includes a sterile table 204, which is set next to the hospital bed for placing surgical instruments to establish a sterile barrier to prevent recontamination of sterile surgical instruments and auxiliary materials, and also to prevent surgical Omission or loss of equipment and auxiliary materials.

此外，本实施例的手术机器人系统旨在实现远程医疗，并能够在远程医疗手术过程中进行主端和从端之间的信息交流，所交流的信息可包括主端医生和从端辅助医生之间对于组织、血管和病灶边界等的定位，还可包括手术路径、手术方案以及对于医学影像中显示不明确的位置的交流等，以有效地解决远程手术过程中主端医生和从端辅助医生之间信息交流不畅的问题，从而有效提高该远程医疗场景中的手术精准率，确保手术的安全顺利进行，并提高手术效率。In addition, the surgical robot system of this embodiment aims to realize telemedicine, and can exchange information between the master end and the slave end during the telemedicine operation. The positioning of tissues, blood vessels, and lesion boundaries can also include surgical paths, surgical plans, and communication of unclear positions displayed in medical images, so as to effectively solve the problem between the master doctor and the slave assistant doctor during remote surgery. The problem of poor information exchange between them can effectively improve the accuracy rate of surgery in this telemedicine scene, ensure the safe and smooth operation of the operation, and improve the efficiency of the operation.

更具体地，本实施例提供一种用于手术机器人的标记共享方法，可适用于主端100或从端200，以主端100为例，包括如下步骤：More specifically, this embodiment provides a marker sharing method for a surgical robot, which is applicable to themaster terminal 100 or theslave terminal 200, taking themaster terminal 100 as an example, including the following steps:

步骤一：获取当前生成的初始医学影像；这里的初始医学影像可以是三维医学影像，当然在其他情况下也可以是二维医学影像；此外，初始医学影像数据包括但不限于内窥镜采集的图像，还可以是其他图像采集装置采集的影像数据，如影像设备采集的CT或MRI数据；另外，可由医生主控制台101根据初始医学影像数据，创建目标组织、器官或血管等体内预定对象的初始医学影像，并将创建的初始医学影像共享给从端200；Step 1: Obtain the currently generated initial medical image; the initial medical image here can be a 3D medical image, and of course it can also be a 2D medical image in other cases; in addition, the initial medical image data includes but is not limited to The image can also be image data collected by other image acquisition devices, such as CT or MRI data collected by imaging equipment; in addition, the doctor'smain console 101 can create images of predetermined objects in the body such as target tissues, organs or blood vessels based on the initial medical image data. an initial medical image, and share the created initial medical image with theslave end 200;

步骤二：当获取到用户输入的第一标记指令后，将所述第一标记信息实时发送到从端200，并在所述初始医学影像中融合所述第一标记信息，所述第一标记指令包括第一标记信息；所述第一标记信息主要包括对应于第一标记的坐标信息；Step 2: After obtaining the first marking instruction input by the user, sending the first marking information to theslave terminal 200 in real time, and fusing the first marking information in the initial medical image, the first marking information The instruction includes first marking information; the first marking information mainly includes coordinate information corresponding to the first marking;

步骤三：当接收到从端200实时发送的第二标记信息后，在所述初始医学影像中融合所述第二标记信息；所述第二标记信息主要包括对应于第二标记的坐标信息。Step 3: After receiving the second marker information sent in real time from the terminal 200, fuse the second marker information into the initial medical image; the second marker information mainly includes coordinate information corresponding to the second marker.

优选地，以上标记共享方法还包括：当接收到从端200实时发送的第二标记信息后，在融合所述第一标记信息的所述初始医学影像中融合所述第二标记信息；从而得到融合了所述第一标记信息和所述第二标记信息的所述初始医学影像。Preferably, the above marker sharing method further includes: after receiving the second marker information sent in real time from the terminal 200, fusing the second marker information in the initial medical image fused with the first marker information; thus obtaining The initial medical image fused with the first marker information and the second marker information.

优选地，所述标记共享方法还包括：将融合所述第一标记信息和/或所述第二标记信息的所述初始医学图像实时发送至从端200。此时，从端200可省去数据融合的步骤，直接可从主端100处获取融合第一标记和/或第二标记的初始医学影像。其中第一标记对应于第一标记信息，如可以是图形和/或文字信息；第二标记对应于第二标记信息，也可以是图形和/或文字信息。Preferably, the marker sharing method further includes: sending the initial medical image fused with the first marker information and/or the second marker information to theslave terminal 200 in real time. At this time, theslave terminal 200 can omit the step of data fusion, and can directly obtain the initial medical image fused with the first marker and/or the second marker from themaster terminal 100 . The first mark corresponds to the first mark information, such as graphic and/or text information; the second mark corresponds to the second mark information, which can also be graphic and/or text information.

当然，以上标记共享方法也可适用于从端200，此时，以上步骤大部分相同，不同的是，在步骤二中，从端200将所述第一标记信息实时发送到主端100，且在步骤三中，当从端200接收到主端100实时发送的第二标记信息后，在所述初始医学影像中融合所述第二标记信息。另外不同的是，在优选步骤中，从端200将融合所述第一标记信息和/或所述第二标记信息的所述初始医学图像实时发送至主端100，进一步在优选步骤中，从端200将融合所述第一标记信息和/或所述第二标记信息的所述初始医学 图像实时发送至主端100。那么，主端100也可省去数据融合的步骤，直接可从从端200处获取含有第一标记和/或第二标记的初始医学影像。Of course, the above tag sharing method can also be applied to theslave end 200. At this time, the above steps are mostly the same, the difference is that in step 2, theslave end 200 sends the first tag information to themaster end 100 in real time, and In step 3, after receiving the second marker information sent by themaster terminal 100 in real time, theslave terminal 200 fuses the second marker information in the initial medical image. Another difference is that, in the preferred step, theslave terminal 200 sends the initial medical image fused with the first marker information and/or the second marker information to themaster terminal 100 in real time, and further in the preferred step, from the The terminal 200 sends the initial medical image fused with the first marker information and/or the second marker information to themaster terminal 100 in real time. Then, themaster terminal 100 can also omit the step of data fusion, and can directly obtain the initial medical image containing the first marker and/or the second marker from theslave terminal 200 .

在本申请实施例中，以上标记共享方法既适用于主端100，也适用于从端200，从而主端100和从端200各自能够进行数据融合而得到含有标记的初始医学影像(定义为标记医学影像)，最终在各自的显示模块上显示融合第一标记和/或第二标记的初始医学影像，即可查看主端医生和从端辅助医生在初始医学影像上所作的标记。In the embodiment of the present application, the above marker sharing method is applicable to both themaster terminal 100 and theslave terminal 200, so that themaster terminal 100 and theslave terminal 200 can each perform data fusion to obtain the initial medical image containing the marker (defined as marker medical image), and finally display the initial medical image fused with the first marker and/or the second marker on the respective display modules, so that the markers made by the master doctor and the slave doctor on the initial medical image can be viewed.

对应的，本申请一实施例还提供一种用标记共享装置，可适用于主端100或从端200，并用于执行以上标记共享方法。Correspondingly, an embodiment of the present application also provides a label sharing device, which is applicable to themaster terminal 100 or theslave terminal 200, and is used to implement the above label sharing method.

以适用于主端100为例，所述标记共享装置包括影像数据获取模块、标记指令获取模块和数据融合模块；所述影像数据获取模块用于获取当前生成的初始医学影像；所述标记指令获取模块用于获取用户输入的第一标记指令，还用于获取从端200实时发送的第二标记信息；所述数据融合模块用于在所述初始医学影像中融合所述第一标记信息和/或所述第二标记信息。优选地，所述标记共享装置还包括信息共享模块，用于将融合所述第一标记信息和/或所述第二标记信息的所述初始医学图像实时发送至从端200。该情况下，所应理解，所述第一标记指令为主端医生在主端100所输入的标记指令，并将该第一标记指令定义为主端标记指令，而从端辅助医生在从端200所输入的标记指令定义为第二标记指令，所述第二标记指令包括第二标记信息，并将该第二标记指令定义为从端标记指令。Taking themain terminal 100 as an example, the label sharing device includes an image data acquisition module, a label instruction acquisition module and a data fusion module; the image data acquisition module is used to acquire the currently generated initial medical image; the label instruction acquisition module The module is used to obtain the first marking instruction input by the user, and is also used to obtain the second marking information sent in real time from the terminal 200; the data fusion module is used to fuse the first marking information and/or in the initial medical image or the second tag information. Preferably, the marker sharing device further includes an information sharing module, configured to send the initial medical image fused with the first marker information and/or the second marker information to theslave end 200 in real time. In this case, it should be understood that the first marking instruction is the marking instruction input by the doctor at themaster end 100, and the first marking instruction is defined as the marking instruction at the main end, while the auxiliary doctor at the slave end The input marking command at 200 is defined as a second marking command, the second marking command includes second marking information, and the second marking command is defined as a slave marking command.

若适用于从端200时，与适用于主端的标记共享装置所不同的是，所述标记指令获取模块用于获取主端100实时发送的第二标记指令，且优选方案中，所述信息共享模块用于将融合所述第一标记信息和/或所述第二标记信息的所述初始医学图像实时发送至主端100。这种情况下，所述第一标记指令即为从端辅助医生在从端200所输入的标记指令，并将此时的第一标记指令定义为从端标记指令，而此时的第二标记指令为主端医生在主动100所输入的标记指令，并将此时的第二标记指令定义为主端标记指令。When applicable to theslave end 200, the difference from the tag sharing device applicable to the master end is that the tag instruction acquisition module is used to obtain the second tag instruction sent by themaster end 100 in real time, and in a preferred solution, the information sharing The module is configured to send the initial medical image fused with the first marker information and/or the second marker information to themain terminal 100 in real time. In this case, the first marking instruction is the marking instruction input by the assistant doctor at theslave end 200, and the first marking instruction at this time is defined as the marking instruction from the slave end, and the second marking instruction at this time is The instruction is a marking instruction input by the client doctor at the active 100 , and the second marking instruction at this time is defined as the client marking instruction.

在本申请实施例中，所述主端100和从端200均包括标记共享装置。为了更清楚起见，所述主端100的标记共享装置定义为主端标记共享装置，所述从端200的标记共享装置定义为从端标记共享装置。因此，所述主端100和从端200均能够利用数据融合技术获取含有第一标记和/或第二标记的初始医学影像，从而方便在主端100和从端200均显示含有第一标记和/或第二标记的初始医学影像，实现主端医生和从端辅助医生之间的信息的及时且有效的沟通。In the embodiment of the present application, both themaster end 100 and theslave end 200 include a tag sharing device. For more clarity, the tag sharing device of themaster 100 is defined as a tag sharing device of the host, and the tag sharing device of theslave 200 is defined as a tag sharing device of the slave. Therefore, both themaster terminal 100 and theslave terminal 200 can use data fusion technology to obtain the initial medical image containing the first marker and/or the second marker, so that both themaster terminal 100 and theslave terminal 200 can display the initial medical image containing the first marker and/or the second marker. /or the initial medical image of the second mark, to realize timely and effective communication of information between the master doctor and the slave doctor.

如图2所示，所述主端标记共享装置包括主端影像数据获取模块1011、主端标记指令获取模块1012、主端数据融合模块1013和主端信息共享模块1014。所述主端影像数据获取模块1011用于获取当前生成的初始医学影像；所述主端标记指令获取模块1012用于获取用户输入的主端标记指令，所述主端标记指令包括主端标记信息，还用于获取从端200实时发送的从端标记信息；所述主端数据融合模块1013用于在所述初始医学影像中融合所述主端标记信息和/或所述从端标记信息，得到融合主端标记信息和/或从端标记信息的初始医学影像。进一步地，所述主端信息共享模块1014用于将融合所述主端标记信息和/或所述从端标记信息的所述初始医学图像实时发送至从端200。进一步地，所述主端标记共享装置还包括主端显示模块1015，用于显示融合所述主端标记信息和/或所述从端标记信息的所述初始医学图像。As shown in FIG. 2 , the main-end label sharing device includes a main-end imagedata acquisition module 1011 , a main-end labelinstruction acquisition module 1012 , a main-enddata fusion module 1013 and a main-endinformation sharing module 1014 . The main-end imagedata acquisition module 1011 is used to acquire the currently generated initial medical image; the main-end markinginstruction acquisition module 1012 is used to acquire the main-end marking instruction input by the user, and the main-end marking instruction includes main-end marking information , and is also used to obtain the slave-end tag information sent by the slave-end 200 in real time; the master-enddata fusion module 1013 is used to fuse the master-end tag information and/or the slave-end tag information in the initial medical image, An initial medical image fused with the marker information of the master end and/or the marker information of the slave end is obtained. Further, the master-endinformation sharing module 1014 is configured to send the initial medical image fused with the master-end tag information and/or the slave-end tag information to theslave end 200 in real time. Further, the master-end mark sharing device further includes a master-end display module 1015, configured to display the initial medical image fused with the master-end mark information and/or the slave-end mark information.

在本申请实施例中，所述医生主控制台101中集成有所述主端标记共享装置，主端医生可在医生主控制台101对所述初始医学影像进行标记，并在医生主控制台101的显示器上显示不含标记的初始医学影像和含有标记的初始医学影像。不含标记的初始医学影像和含有标记的初始医学影像可以在不同窗口显示，或者当初始医学影像被标记后，仅显示含有标记的初始医学影像。In the embodiment of the present application, the master-end mark sharing device is integrated in the doctor'smain console 101, and the master-end doctor can mark the initial medical image on the doctor'smain console 101, and An initial medical image without a marker and an initial medical image with a marker are displayed on the display at 101 . The initial medical image without a mark and the initial medical image with a mark can be displayed in different windows, or after the initial medical image is marked, only the initial medical image with a mark is displayed.

继续参阅图2，所述从端标记共享装置包括从端影像数据获取模块2021、从端标记指令获取模块2022、从端数据融合模块2023和从端信息共享模块2024。所述从端影像数据获取模块2021用于获取当前生成的初始医学影像；所述从端标记指令获取模块2022用于获取用户输入的从端标记指令，所述从端标记指令包括从端标记信息，还用于获取主端100发送的主端标记信息；所述从端数据融合模块2023用于在所述初始医学影像中融合所述主端标记信息和/或所述从端标记信息，得到融合所述主端标记信息和/或从端标记信息的初始医学影像。进一步地，所述从端信息共享模块2024用于将融合所述主端标记信息和/或所述从端标记信息的所述初始医学图像实时发送至主端100。进一步地，所述从端标记共享装置还包括从端显示模块2025，用于显示融合所述主端标记信息和/或所述从端标记信息的所述初始医学图像。Continuing to refer to FIG. 2 , the device for sharing a slave tag includes a slave imagedata acquisition module 2021 , a slave tagcommand acquisition module 2022 , a slave data fusion module 2023 and a slaveinformation sharing module 2024 . The slave imagedata acquisition module 2021 is used to acquire the currently generated initial medical image; the slave labelinstruction acquisition module 2022 is used to acquire a user input slave label instruction, and the slave label instruction includes slave label information , is also used to obtain the master-end tag information sent by the master-end 100; the slave-end data fusion module 2023 is used to fuse the master-end tag information and/or the slave-end tag information in the initial medical image to obtain The primary medical image is fused with the primary-side marker information and/or the slave-side marker information. Further, the slave-endinformation sharing module 2024 is configured to send the initial medical image fused with the master-end tag information and/or the slave-end tag information to themaster end 100 in real time. Further, the device for sharing markers of the slave end further includes aslave display module 2025, configured to display the initial medical image fused with the marker information of the master end and/or the marker information of the slave end.

在本申请实施例中，所述从端控制装置200如图像台车203中集成有所述从端标记共享装置，从端辅助医生可在从端控制装置200对所述初始医学影像进行标记，并在从端控制装置200的显示器上显示不含标记的初始医学影像和含有标记的初始医学影像。在从端标记共享装置中，不含标记的初始医学影像和含有标记的初始医学影像可以在不同窗口显示，或者当初始医学影像被标记后，仅显示含有标记的初始医学影像。In the embodiment of the present application, the slave-end control device 200, such as theimage trolley 203, is integrated with the slave-end marking sharing device, and the slave-end assistant doctor can mark the initial medical image on the slave-end control device 200, And the initial medical image without marker and the initial medical image with marker are displayed on the display of the slave-end control device 200 . In the device for sharing markers from the end, the initial medical images without markers and the initial medical images with markers can be displayed in different windows, or after the initial medical images are marked, only the initial medical images with markers are displayed.

所需理解，所述主端数据融合模块1013和从端数据融合模块2023分别采用数据融合算法来实现数据融合。数据融合算法主要通过标记的坐标来实现，且如何实现数据融合为本领域的公知技术，因此，本申请对其不作详细的描述。It needs to be understood that the master-enddata fusion module 1013 and the slave-end data fusion module 2023 respectively adopt data fusion algorithms to realize data fusion. The data fusion algorithm is mainly realized through the marked coordinates, and how to realize the data fusion is a well-known technology in the field, so this application will not describe it in detail.

在一示例中，所述标记共享装置根据所述主端标记信息中的位置信息，在所述初始医学影像上重建对应于所述主端标记信息的主端标记，并根据所述从端标记信息中的位置信息，在所述初始医学影像上重建对应于所述从端标记信息的从端标记。In an example, the marker sharing device reconstructs the master-end marker corresponding to the master-end marker information on the initial medical image according to the position information in the master-end marker information, and based on the slave-end marker The location information in the information, reconstruct the secondary marker corresponding to the secondary marker information on the initial medical image.

更具体地，所述标记共享装置根据所述主端标记信息中的坐标信息，得到所述初始医学影像中对应于所述主端标记信息所在位置的图像，并在所述主端标记信息所在位置的图像上生成主端标记，由此将主端标记与初始医学图像进行融合。所述标记共享装置还根据所述从端标记信息中的坐标信息，得到所述初始医学影像中对应于所述从端标记信息所在位置的图像，并在所述从端标记信息所在位置的图像上生成从端标记，从而将从端标记也与初始医学图像进行融合；最终主端数据融合模块1013获取融合了所述主端标记和所述从端标记的所述初始医学影像。More specifically, the marker sharing device obtains the image corresponding to the location of the master-end marker information in the initial medical image according to the coordinate information in the master-end marker information, and The main-end marker is generated on the image of the position, thereby fusing the main-end marker with the original medical image. The marker sharing device also obtains the image corresponding to the position of the slave marker information in the initial medical image according to the coordinate information in the slave marker information, and places the image at the position of the slave marker information generate the slave-end markers, so as to fuse the slave-end markers with the initial medical image; finally, the master-enddata fusion module 1013 obtains the initial medical image fused with the master-end markers and the slave-end markers.

在一具体实施例中，所述标记共享装置还能够获取启用或禁用标记指令，并根据所述禁用标记指令输出主从控制指令，所述主从控制指令用于控制主端100或从端200的运动，以及根据所述启用标记指令启动标记模式，以允许用户对初始医学影像进行标记。In a specific embodiment, the tag sharing device can also obtain an enable or disable tag instruction, and output a master-slave control instruction according to the disable tag instruction, and the master-slave control instruction is used to control themaster terminal 100 or theslave terminal 200 , and activate the marking mode according to the enabling marking instruction, so as to allow the user to mark the initial medical image.

如对主端100而言，主端标记共享装置根据获取的启用或禁用标记指令，使医生主控制台101能够被允许或禁止操作创建主端标记。且当允许创建主端标记时，用户能够通过医生主控制台101对所述初始医学影像进行标记，并保存所述主端标记的位置信息(包括坐标)。如对从端200而言，从端标记共享装置根据获取的启用或禁用标记指令，使从端控制装置200能够被允许或禁止操作创建从端标记。且当允许创建从端标记时，用户能够通过从端控制装置200对所述初始医学影像进行标记，并保存所述从端标记的位置信息(包括坐标)。For themain terminal 100, the main terminal mark sharing device enables or prohibits the doctormain console 101 from operating to create the main end mark according to the obtained enable or disable mark instruction. And when it is allowed to create the main-end mark, the user can mark the initial medical image through the doctor'smain console 101 and save the position information (including coordinates) of the main-end mark. For theslave 200, the slave tag sharing device enables or prohibits theslave control device 200 from creating a slave tag according to the acquired enabling or disabling tag instruction. And when the slave-end mark is allowed to be created, the user can mark the initial medical image through the slave-end control device 200 and save the position information (including coordinates) of the slave-end mark.

进一步地，所述主端信息共享模块1014除了向从端200发送标记信息和含有标记信息的初始医学影像外，还能够接收从端200所发送过来的数据，如从端标记信息，并将所接收的从端标记信息传输至主端标记指令获取模块1012。进一步地，所述主端标记指令获取模块1012能够解析从端标记信息，以获取从端标记信息中的位置信息。进一步地，所述主端标记指令获取模块1012能够打包主端标记信息，打包后的主端标记信息，通过主端信息共享模块1014发送至从端200。Further, the masterinformation sharing module 1014 can receive the data sent by theslave 200, such as the slave marker information, and send the tag information and the initial medical image containing the tag information to theslave 200, and send the The received slave tag information is transmitted to the master taginstruction acquiring module 1012 . Further, the master-end markinstruction acquisition module 1012 can parse the slave-end mark information to obtain the position information in the slave-end mark information. Further, the master-end markinginstruction acquiring module 1012 can pack the master-end marking information, and the packed master-end marking information is sent to the slave-end 200 through the master-endinformation sharing module 1014 .

进一步地，所述从端信息共享模块2024除了向主端100发送标记信息和含有标记信息的初始医学影像外，还能够接收主端100所发送过来的数据，如主端标记信息，并将所接收的主端标记信息传输至从动标记指令获取模块2022。进一步地，所述从端标记指令获取模块2022能够解析主端标记信息，以获取主端标记信息中的位置信息。进一步地，所述从动标记指令获取模块2022能够打包从端标记信息，打包后的从端标记信息通过从端信息共享模块2024发送至主端100。Further, the slaveinformation sharing module 2024 can receive the data sent by themaster 100, such as the master marker information, in addition to sending the marker information and the initial medical image containing the marker information to themaster 100, and The received master tag information is transmitted to the slave taginstruction acquiring module 2022 . Further, the slave taginstruction acquisition module 2022 can parse the master tag information to obtain the position information in the master tag information. Further, the slave taginstruction acquiring module 2022 can pack the slave tag information, and the packed slave tag information is sent to themaster 100 through the slaveinformation sharing module 2024 .

非限制性地，所述主端影像数据获取模块1011还能够根据初始医学影像数据，建立组织、器官或血管等预定对象的初始医学影像，该主端影像数据获取模块1011优选提供三维建模功能，以获取三维医学影像。此外，所述主端显示模块1015优选为3D显示屏，以立体显示三维医学影像，以提高更精准的参考价值，使得远程手术场景应用精确率大幅提升。为了提高手术的准确性，所述主端影像数据获取模块1011主要利用三维建模和人工智能算法对实时采集到的影像数据进行处理，以生成三维的所述初始医学影像。人工智能算法可实现全自动化建模，无需人工修复，具有高保真、无扭曲、模型体积小且方便展示的特点，有效降低三维模型的建模时间，并提高三维模型建模的精度。进一步地，所述从端显示模块2025可以是2D显示屏或3D显示屏。Without limitation, the main-end imagedata acquisition module 1011 can also establish initial medical images of predetermined objects such as tissues, organs or blood vessels according to the initial medical image data, and the main-end imagedata acquisition module 1011 preferably provides a three-dimensional modeling function , to obtain 3D medical images. In addition, the main-end display module 1015 is preferably a 3D display to display three-dimensional medical images stereoscopically, so as to improve a more accurate reference value, so that the application accuracy of remote surgery scenarios is greatly improved. In order to improve the accuracy of the operation, the imagedata acquisition module 1011 of the main terminal mainly uses 3D modeling and artificial intelligence algorithms to process the image data collected in real time to generate the 3D initial medical image. The artificial intelligence algorithm can realize fully automatic modeling without manual repair. It has the characteristics of high fidelity, no distortion, small size and convenient display, effectively reducing the modeling time of 3D models and improving the accuracy of 3D model modeling. Further, theslave display module 2025 may be a 2D display or a 3D display.

为了进一步改善主端100和从端200之间的信息交流的顺畅性，在一实施例中，所述标记共享方法还包括：从端200向主端100实时无线传输音频数据和/或视频数据。在一实施例中，所述标记共享方法还包括：所述主端100向从端200实时无线传输音频数据和/或视频数据。In order to further improve the smoothness of information exchange between themaster terminal 100 and theslave terminal 200, in an embodiment, the mark sharing method further includes: real-time wireless transmission of audio data and/or video data from theslave terminal 200 to themaster terminal 100 . In an embodiment, the mark sharing method further includes: themaster terminal 100 wirelessly transmits audio data and/or video data to theslave terminal 200 in real time.

进一步的，所述手术机器人系统还包括音频和/或视频采集模块，用于采集音频和/或视频数据。Further, the surgical robot system also includes an audio and/or video acquisition module, configured to acquire audio and/or video data.

如图2所示，所述医生主控制台101还配置有主端音频和/或视频采集模块1016，用于采集主端100的音频和/或视频数据，并向从端200实时传输所述主端的音频和/或视频数据，从而方便主端医生与从端辅助医生之间通过语音和/或视频进行在线沟通，以进一步提高沟通效率。所述主端100可通过主端数据传输模块1017在主端100和从端200之间传输音视频数据，并提供实时音视频低延时传输方案，保证数据传输过程中的低延时和高质量。As shown in Figure 2, the doctor'smain console 101 is also configured with a main-end audio and/orvideo acquisition module 1016, which is used to collect audio and/or video data of themain end 100, and transmit the described data to theslave end 200 in real time. The audio and/or video data of the master terminal facilitates online communication between the master doctor and the slave assistant doctor through voice and/or video to further improve communication efficiency. Themaster end 100 can transmit audio and video data between themaster end 100 and theslave end 200 through the master enddata transmission module 1017, and provide a real-time audio and video low-latency transmission scheme to ensure low delay and high latency during data transmission. quality.

类似的，所述从端控制装置202配置有从端音频和/或视频采集模块2026，用于采集从端200的音频和/或视频数据，并向主端100实时传输所述从端的音频和/或视频数据，从而方便主端医生与从端辅助医生之间通过语音和/或视频进行在线沟通，以进一步提高沟通效率。所述从端200进一步通过从端数据传输模块2027在主端100和从端200之间传输音视频数据，并提供实时音视频低延时传输方案，保证数据传输过程中的低延时和高质量。Similarly, the slave-side control device 202 is configured with a slave-side audio and/orvideo acquisition module 2026 for collecting audio and/or video data from the slave-side 200, and transmitting the slave-side audio and/or video data to themaster 100 in real time. /or video data, so as to facilitate online communication between the master doctor and the slave doctor through voice and/or video, so as to further improve communication efficiency. Theslave end 200 further transmits audio and video data between themaster end 100 and theslave end 200 through the slave enddata transmission module 2027, and provides a real-time audio and video low-latency transmission scheme to ensure low delay and high latency during data transmission. quality.

所述主端音频和/或视频采集模块1016，以及从端音频和/或视频采集模块2026包括但不限于摄像头、麦克风等音视频设备。所述主端音频和/或视频采集模块1016，以及从端音频和/或视频采集模块2026均可连接有线或无线网路，优选连接5G Wi-Fi，传输速度更快，传输质量更好。其中，所述从端音频和/或视频采集模块2026与手术机器人201进行数据通信，以接受图像采集装置所传输的视频数据。The master audio and/orvideo acquisition module 1016 and the slave audio and/orvideo acquisition module 2026 include but are not limited to audio and video equipment such as cameras and microphones. The master-end audio and/orvideo acquisition module 1016, and the slave-end audio and/orvideo acquisition module 2026 can be connected to a wired or wireless network, preferably connected to 5G Wi-Fi, with faster transmission speed and better transmission quality. Wherein, the slave-end audio and/orvideo acquisition module 2026 performs data communication with thesurgical robot 201 to receive the video data transmitted by the image acquisition device.

所述手术机器人系统还包括图像采集装置，所述初始医学影像数据可利用图像采集装置(如内窥镜)采集体内的图像而确定。The surgical robot system further includes an image acquisition device, and the initial medical image data can be determined by using an image acquisition device (such as an endoscope) to collect images in the body.

进一步地，考虑到图像采集装置采集的图像会实时发生变化，此时，为了提高标记效率，在所述初始医学影像中融合所述主端标记信息和/或在所述初始医学影像中融合所述从端标记信息之后，还包括：获取当前生成的所述初始医学影像的图像变化信息，并根据所述初始医学影像的图像变化信息，更新所述主端标记信息和/或所述从端标记信息，以使所述主端标记信息和/或所述从端标记信息，实时跟随图像采集装置采集的初始医学影像的图像的变化；从而使标记动态跟随图像的变化，使标记效率更高，操作也更方便。Further, considering that the image captured by the image acquisition device will change in real time, at this time, in order to improve the labeling efficiency, the primary end label information is fused in the initial medical image and/or all information is fused in the initial medical image. After the tag information of the slave end, it also includes: acquiring the image change information of the initial medical image currently generated, and updating the tag information of the master end and/or the slave end according to the image change information of the initial medical image mark information, so that the master-end mark information and/or the slave-end mark information follow the changes in the image of the initial medical image collected by the image acquisition device in real time; so that the mark dynamically follows the change of the image, making the mark more efficient , and the operation is more convenient.

进一步地，所述标记共享方法还包括：在获取到用户输入的第一标记指令之前，启动标记模式， 所述标记模式允许用户输入所述第一标记指令。可以理解的是，当所述第一标记指令为主端标记指令时，主端100在获取到用户输入的主端标记指令之前，还包括启动标记模式，启动标记模式后，才允许用户在主端100输入主端标记指令。而当所述第一标记指令为从端标记指令时，从端200在获取到用户输入的从端标记指令之前，还包括启动标记模式，启动标记模式后，才允许用户在从端200输入从端标记指令。Further, the marking sharing method further includes: before acquiring the first marking instruction input by the user, starting a marking mode, where the marking mode allows the user to input the first marking instruction. It can be understood that, when the first marking command is the marking command of the main end, before themain end 100 obtains the main end marking command input by the user, it also includes starting the marking mode, and only after the marking mode is started, the user is allowed to start the marking mode on the main end. The terminal 100 inputs the master label command. And when the first mark command is a slave mark command, before theslave terminal 200 obtains the slave mark command input by the user, it also includes a start mark mode, after the start mark mode, the user is allowed to input the slave end tag directive.

在一具体实施例中，所述标记共享方法还包括：获取主端100和/或从端200的标记状态，并根据主端100和/或从端200的标记状态，输出主从控制指令或启用标记指令。所述主从控制指令用于控制主端100或从端200的运动状态(包括操作状态)，所述启用标记指令用于启动所述标记模式。In a specific embodiment, the label sharing method further includes: acquiring the label status of themaster terminal 100 and/or theslave terminal 200, and outputting a master-slave control instruction or Enable markup directives. The master-slave control command is used to control the motion state (including the operation state) of themaster terminal 100 or theslave terminal 200, and the enable flag command is used to start the flag mode.

进一步地，所述标记共享方法还包括：获取用户输入的退出标记指令，并根据所述退出标记指令退出所述标记模式。Further, the marking sharing method further includes: obtaining an exit marking instruction input by a user, and exiting the marking mode according to the exit marking instruction.

更进一步地，所述标记共享方法还包括：根据所述启用标记指令，使主端100和/或从端200进入操作锁定状态，以禁止主端100和/或从端200的运动，以使所述主端100和/或从端200禁止控制手术机器人201运动。如当所述手术机器人系统进入标记模式时，医生主控制台101的主操作单元即处于操作锁定状态，主操作单元不能遥操作手术机器人201，例如从端200接收主操作单元的运动状态，但是不能够据此控制手术机器人运动，即主操作单元的操作手柄无法实现遥操作控制手术机器人运动及手术操作，和/或，当手术机器人系统进入标记模式时，手术机器人201自动锁定而无法运动。Furthermore, the tag sharing method further includes: according to the enable tag command, making themaster 100 and/or theslave 200 enter an operation lock state, so as to prohibit the movement of themaster 100 and/or theslave 200, so that Themaster end 100 and/or theslave end 200 are prohibited from controlling the movement of thesurgical robot 201 . For example, when the surgical robot system enters the marking mode, the main operating unit of the doctor’smain console 101 is in an operation locked state, and the main operating unit cannot teleoperate thesurgical robot 201, for example, theslave terminal 200 receives the motion state of the main operating unit, but The movement of the surgical robot cannot be controlled accordingly, that is, the operating handle of the main operating unit cannot realize teleoperation to control the movement and operation of the surgical robot, and/or, when the surgical robot system enters the marking mode, thesurgical robot 201 is automatically locked and cannot move.

更进一步地，所述标记共享方法还包括：根据所述退出标记指令，使主端100和/或从端200进入操作解锁状态，以允许主端100和/或从端200的运动，以使主端100和/或从端200允许控制所述手术机器人运动。如当所述手术机器人系统退出标记模式时，医生主控制台101的主操作单元即处于操作解锁状态，主操作单元能够遥操作手术机器人201，例如从端200接收主操作单元的运动状态，并据此控制手术机器人运动，即主操作单元的操作手柄能够实现遥操作控制手术机器人运动及手术操作，和/或，当手术机器人系统退出标记模式时，手术机器人201自动解锁而可以运动。Furthermore, the tag sharing method further includes: according to the exit tag command, making themaster terminal 100 and/or theslave terminal 200 enter the operation unlock state, so as to allow the movement of themaster terminal 100 and/or theslave terminal 200, so that Themaster end 100 and/or theslave end 200 allow to control the movement of the surgical robot. For example, when the surgical robot system exits the marking mode, the main operating unit of the doctor’smain console 101 is in the operation unlock state, and the main operating unit can remotely operate thesurgical robot 201, for example, theslave terminal 200 receives the motion state of the main operating unit, and According to this, the movement of the surgical robot is controlled, that is, the operating handle of the main operation unit can control the movement of the surgical robot and the surgical operation by remote operation, and/or, when the surgical robot system exits the marking mode, thesurgical robot 201 is automatically unlocked and can move.

所应理解，当手术机器人系统进入操作锁定状态时，手术机器人系统锁定主从映射关系，以方便从端200或主端100进行标记操作；而当手术机器人系统处于操作解锁状态时，所述手术机器人系统能够重建主从映射关系，以根据经确定的主从控制指令，允许主端100对从端200进行遥操作，使手术机器人201能够根据主从控制指令控制手术器械或内窥镜的运动。It should be understood that when the surgical robot system enters the operation lock state, the surgical robot system locks the master-slave mapping relationship to facilitate the marking operation of theslave end 200 or themaster end 100; and when the surgical robot system is in the operation unlock state, the operation The robot system can reconstruct the master-slave mapping relationship to allow themaster terminal 100 to perform teleoperation on theslave terminal 200 according to the determined master-slave control instructions, so that thesurgical robot 201 can control the movement of surgical instruments or endoscopes according to the master-slave control instructions .

参阅图3，在示例性实施例中，所述主端100包括左脚踏板1041和右脚踏板1042，这些脚踏板集成在医生主控制台101上。其中左脚踏板1041和右脚踏板1042中的一个构造为第一脚踏板，左脚踏板1041和右脚踏板1042中的另一个构造为第二脚踏板。所述第一脚踏板用于输出启用标记指令，主端100根据第一脚踏板的启用标记指令启动标记模式；所述第二脚踏板用于输出退出标记指令，主端100根据第二脚踏板的退出标记指令退出标记模式。使用时，主端医生可通过双脚控制对应的脚踏板即可进入标记模式或退出标记模式。例如左脚踏板1041当前的操作状态为被踩下而输出启用标记指令，则主端100确定进入标记模式；反之右脚踏板1042当前的操作状态为被踩下而输出退出标记指令，则主端100确定退出标记模式。Referring to FIG. 3 , in an exemplary embodiment, themain terminal 100 includes aleft foot pedal 1041 and aright foot pedal 1042 , and these foot pedals are integrated on the doctor'smain console 101 . One of theleft footrest 1041 and theright footrest 1042 is configured as a first footrest, and the other of theleft footrest 1041 and theright footrest 1042 is configured as a second footrest. The first pedal is used to output the enable flag instruction, and themaster terminal 100 starts the flag mode according to the enable flag instruction of the first pedal; The Exit Mark command of the second pedal exits Mark mode. When in use, the doctor at the main end can enter the marking mode or exit the marking mode by controlling the corresponding pedals with both feet. For example, the current operating state of theleft foot pedal 1041 is to be stepped on and an enable flag instruction is output, then themaster terminal 100 determines to enter the flag mode; otherwise, the current operating state of theright foot pedal 1042 is to be stepped on and output an exit flag instruction, then Themaster 100 determines to exit the marking mode.

继续参阅图3，在一具体实施例中，所述主端标记指令获取模块1012可包括左操作臂1043和右操作臂1044。这些操作臂也集成在医生主控制台101上。所述左操作臂1043和右操作臂1044构成主操作单元，可以接受外界指令以输出运动信息，而且所述左操作臂1043和右操作臂1044中的一个构造为第一操作臂，另一个构造为第二操作臂。所述第一操作臂用于根据用户输入的操作指令选择对应于主端标记信息的主端标记。所述第二操作臂用于根据用户输入的操作指令，采用已选择的主端标记，在所述初始医学影像上创建所述主端标记。Continuing to refer to FIG. 3 , in a specific embodiment, the master-side flaginstruction acquiring module 1012 may include aleft operating arm 1043 and aright operating arm 1044 . These manipulation arms are also integrated on the doctor'smain console 101 . Theleft operating arm 1043 and theright operating arm 1044 constitute the main operating unit, which can accept external instructions to output motion information, and one of theleft operating arm 1043 and theright operating arm 1044 is configured as a first operating arm, and the other is configured as a first operating arm. is the second operating arm. The first operating arm is used for selecting the main-end mark corresponding to the main-end mark information according to the operation instruction input by the user. The second operating arm is configured to use the selected main-end marker to create the main-end marker on the initial medical image according to an operation instruction input by a user.

如图3和图7所示，所述主端显示模块1015包括主端显示界面G1。用户通过操控所述第一操作臂，即可选择主端交互界面上的画图工具，所述画图工具提供各种对应于预设图形的绘图命令，从而选择画图工具中的图形来创建对应于主端标记指令的主端标记。此外，用户还可通过操控所述第二操作臂，即可根据第一操作臂所确定的预设图形在初始医学影像上绘制对应于主端标记指令的主端标记。例如左操作臂1043用于选择画图工具，右操作臂1044用于通过画图工具绘制图形。As shown in FIG. 3 and FIG. 7 , themaster display module 1015 includes a master display interface G1. By manipulating the first operating arm, the user can select the drawing tool on the interactive interface of the main terminal, and the drawing tool provides various drawing commands corresponding to the preset graphics, so as to select the graphics in the drawing tool to create a drawing corresponding to the main drawing tool. The master endian flag for the endian flag directive. In addition, by manipulating the second operating arm, the user can draw a main-end marking corresponding to the main-end marking instruction on the initial medical image according to the preset graphic determined by the first operating arm. For example, theleft operating arm 1043 is used to select a drawing tool, and theright operating arm 1044 is used to draw graphics with the drawing tool.

在一具体实施例中，主端医生通过操控左操作臂1043的运动来移动光标位置，以选择所需的图形以及需要标记的位置，并经由左操作臂1043的左控制手柄1045来确定选中已选的图形，且由右操作臂1044的右控制手柄1029的手指操作来进行捏合选择(如鼠标左键的功能)和绘图操作。In a specific embodiment, the doctor at the primary end controls the movement of theleft operating arm 1043 to move the position of the cursor to select the desired graphic and the position to be marked, and determines that the selected image is selected via the left control handle 1045 of theleft operating arm 1043. The selected graphic is operated by the fingers of the right control handle 1029 of theright operating arm 1044 to perform pinching selection (such as the function of the left mouse button) and drawing operations.

如图3和图7所示，在一具体实施例中，所述主端交互界面G1提供画图工具G2和主端撤销按钮G8。所述主端交互界面G1的画图工具G2提供各种预设图形，如矩形的预设主端标记G3，圆形的预设主端标记G4，当然预设图形不限于此举例的，还可以是其他各种形状，而且形状不限于规则图形。所述主端交互界面G1的画图工具G2还可提供画笔，方便用户自定义绘制图形。如此，用户可通过鼠标操作所述画图工具，并利用所述画图工具G2对主端交互界面G1上显示的初始医学影像进行图形的绘制，以得到所述主端标记。As shown in FIG. 3 and FIG. 7 , in a specific embodiment, the master-end interaction interface G1 provides a drawing tool G2 and a master-end undo button G8. The drawing tool G2 of the master-end interactive interface G1 provides various preset graphics, such as a rectangular preset master-end mark G3 and a circular preset master-end mark G4. Of course, the preset graphics are not limited to this example, and can also be are various other shapes, and the shapes are not limited to regular figures. The drawing tool G2 of the main-end interactive interface G1 can also provide a paintbrush to facilitate user-defined drawing of graphics. In this way, the user can use the mouse to operate the drawing tool, and use the drawing tool G2 to draw graphics on the initial medical image displayed on the main-end interaction interface G1, so as to obtain the main-end mark.

在图7示出的示例中，用户通过鼠标并根据画图工具G2提供的预设图形，可绘制圆形的主端标记G5，还可绘制矩形的主端标记G7，还可绘制自定义的主端标记G6。此外，所述主端撤销按钮G8用于提供撤销当前所绘制的对应于主端标记的图形的操作命令，用户只要通过鼠标点击该主端撤销按钮G8即可撤销当前图形。当然主端交互界面G1除显示标记、医学影像外，还用于显示含有标记的初始医学影像。In the example shown in Figure 7, the user can draw a circular main end mark G5, a rectangular main end mark G7, and a user-defined main end mark G5 through the mouse and according to the preset graphics provided by the drawing tool G2. End labeled G6. In addition, the master cancel button G8 is used to provide an operation command for canceling the currently drawn graphic corresponding to the master marker, and the user can cancel the current graphic by clicking the master cancel button G8 with the mouse. Of course, in addition to displaying markers and medical images, the main-end interactive interface G1 is also used to display initial medical images containing markers.

图5示出了根据本申请一优选实施例的主端标记创建的工作流程。如图5所示，主端标记创建的工作流程包括以下步骤：Fig. 5 shows the workflow of master-side tag creation according to a preferred embodiment of the present application. As shown in Figure 5, the workflow of master-side marker creation includes the following steps:

步骤A1：启动主端标记指令获取模块1012；由于在医生主控制台101的运行程序中嵌入有主端标记指令获取模块1012，因此，在运行医生主控制台101程序后，即自动启动主端标记功能，但后续需要启动标记模式才能进行标记操作。Step A1: Start the main terminal markingcommand acquisition module 1012; since the main terminal markingcommand acquisition module 1012 is embedded in the operating program of the doctor'smain console 101, after running the doctor'smain console 101 program, the main terminal is automatically started Marking function, but the marking mode needs to be activated in order to carry out the marking operation.

步骤A2：判断从端是否处于标记状态；在一实施例中，主端和从端的标记功能为互斥状态，即同一时刻，只有一个能进行标记操作；如果当前从端200处于标记状态，则主端100自动进入步骤A10以退出主端标记模式；如果当前从端200未处于标记状态，则主端100可进入步骤A3的主端标记模式。因此，所述主端标记共享装置优选还获取从端的标记状态，当从端的标记状态为当前处于非标记状态时，则启动主端标记模式，进而在所述初始医学影像上进行标记以获取主端标记指令，并优选在步骤S4中向从端200实时发送对应于所述主端的本地当前的标记状态。Step A2: Determine whether the slave end is in the marking state; in one embodiment, the marking functions of the master end and the slave end are mutually exclusive, that is, at the same time, only one can perform marking operations; if thecurrent slave end 200 is in the marking state, then Themaster 100 automatically enters step A10 to exit the master marking mode; if thecurrent slave 200 is not in the marking state, themaster 100 can enter the master marking mode in step A3. Therefore, the tag sharing device at the master end preferably also acquires the tag status of the slave end, and when the tag status of the slave end is currently in an unmarked state, it starts the tag mode of the master end, and then marks the initial medical image to obtain the tag status of the master end. terminal marking instruction, and preferably in step S4, send the local current marking status corresponding to the master terminal to theslave terminal 200 in real time.

步骤A3：主端进入主端标记模式；进入主端标记模式后，执行步骤A4：由主端100向从端200实时发送对应于主端的本地当前的标记状态，从而将主端100正在标记的状态发送给从端200，以便从端200根据主端的标记状态来判断是否能够进入标记模式。Step A3: The master end enters the master end mark mode; after entering the master end mark mode, execute step A4: themaster end 100 sends the local current mark state corresponding to the master end to theslave end 200 in real time, so that themaster end 100 is marking The state is sent to theslave end 200, so that theslave end 200 can judge whether it can enter the mark mode according to the mark state of the master end.

进入主端标记模式后，还执行步骤A5：判断是否选择预设标记图形；该步骤A5中，用户可选择系统预设的图形来绘制主端标记，也可选择自定义图形来绘制图形。After entering the main-end marking mode, step A5 is also performed: judging whether to select a preset marking graphic; in this step A5, the user can select a system-preset graphic to draw the main-end marking, or select a custom graphic to draw a graphic.

进一步地进入步骤A6：按照上一步骤A5中选择的绘图方式来绘制主端标记，以生成主端标记指令；用户可通过左右控制手柄在初始医学图像上绘制主端标记。Go further to step A6: draw the main-end mark according to the drawing method selected in the previous step A5 to generate a main-end mark instruction; the user can draw the main-end mark on the initial medical image through the left and right control handles.

完成主端标记的绘制后，通过步骤A7可进一步判断是否撤销已绘制的主端标记；如果是，则流转至步骤A8以删除最新主端标记，否则程序自动流转至步骤A9。After completing the drawing of the main-end mark, step A7 can be used to further determine whether to cancel the drawn main-end mark; if yes, then flow to step A8 to delete the latest main-end mark, otherwise the program automatically flows to step A9.

步骤A8：删除最新主端标记，优选每次只能删除最近一次的主端标记。Step A8: Delete the latest main-end mark, preferably only the latest main-end mark can be deleted each time.

步骤A9：判断是否退出主端标记；如果是，则退出标记模式A10，如果否，则可继续循环流程A5至A9。Step A9: Determine whether to exit the master-end marking; if yes, exit the marking mode A10, if not, continue to loop the process A5 to A9.

在优选实施例中，当主端100进入主端标记模式后，其主端交互界面G1自动在屏幕中显现预设的主端标记图形和主端撤销按钮，而当主端100退出主端标记模式后，主端交互界面G1自动在屏幕中隐藏预设的主端标记图形和主端撤销按钮。In a preferred embodiment, when themain terminal 100 enters the main terminal marking mode, its main terminal interactive interface G1 automatically displays the preset main terminal marking graphics and the main terminal cancel button on the screen, and when themain terminal 100 exits the main terminal marking mode , the master-end interactive interface G1 automatically hides the preset master-end marker graphics and master-end cancel button on the screen.

更详细地，在一具体操作方式中，如图6所示，主端医生触发特定的脚踏板B1以输出启用主端标记指令，从而进入主端标记模式A3，进入主端标记模式A3后，可选择两种标记方式，包括：In more detail, in a specific mode of operation, as shown in Figure 6, the doctor at the main end triggers a specific pedal B1 to output an instruction to enable the main end marking, thereby entering the main end marking mode A3, after entering the main end marking mode A3 , you can choose two marking methods, including:

第一种标记方式：主端医生确定绘制预设图形B2；然后，主端医生可通过左操作臂1043选中预设图形B3；之后，主端医生可通过右操作臂1044绘制预设图形B4；The first marking method: the doctor at the main end determines to draw the preset graphic B2; then, the doctor at the main end can select the preset graphic B3 through theleft operating arm 1043; after that, the doctor at the main end can draw the preset graphic B4 through theright operating arm 1044;

第二种标记方式：主端医生确定绘制自定义图形B5；然后，主端医生可通过右操作臂1044绘制自定义图形路径B6。The second way of marking: the doctor at the main end determines to draw the custom graphic B5; then, the doctor at the main end can draw the custom graphic path B6 through theright operating arm 1044 .

以上标记方式最终都可执行撤销操作B7；若确定撤销，则可通过左操作臂1043选择主端撤销按钮B8，撤销当前的主端标记图形，撤销后，又可循环选择预设图形并进行绘制，或者又可重新绘制自定义图形路径。如用户通过左操作臂1043移动光标到主端交互界面G1的图形选择窗口(即画图工具G2)，捏合左手手指进行选择预设图形，选中后可松开左手。接着，用户通过右操作臂1044移动光标到需要标记的位置，然后捏合右手手指并移动位置进行标记操作，标记完成后松开手指。The above marking methods can finally perform the undo operation B7; if the undo is determined, the main end undo button B8 can be selected through theleft operating arm 1043 to undo the current main end marking graphics. After undoing, the preset graphics can be cyclically selected and drawn , or you can redraw a custom graphics path. For example, the user uses theleft operating arm 1043 to move the cursor to the graphic selection window (ie, the drawing tool G2) of the main-end interactive interface G1, pinches the fingers of the left hand to select a preset graphic, and releases the left hand after selection. Next, the user moves the cursor to the position to be marked through theright operating arm 1044 , then pinches the fingers of the right hand and moves the position to perform the marking operation, and releases the fingers after the marking is completed.

在一具体实施例中，在不选择预设标记图形的状态下，主端100默认绘制自定义标记图形，使用右操作臂1044移动光标到需要标记的位置，捏合手指进行标记和移动，松开后完成一次标记操作。另外，左操作臂1043移动光标到主端撤销按钮G8的位置，捏合手指进行撤销操作即可。In a specific embodiment, in the state of not selecting a preset marker graphic, themain terminal 100 draws a custom marker graphic by default, uses theright operating arm 1044 to move the cursor to the position to be marked, pinches the fingers to mark and move, releases the After completing a marking operation. In addition, theleft operating arm 1043 moves the cursor to the position of the undo button G8 on the main end, and only needs to pinch the fingers to perform the undo operation.

以上提供的是主端和从端只能一方进行标记，但在其他实施例中，主端和从端也可同时进行标记并分别保存各自的标记信息。进一步地，当主端和从端仅可一方进行标记时，所述手术机器人系统可设置标记优先级别，如主端和从端都同时发出标记指令，此时，根据优先级别来确定执行标记的对象，如主端的标记优先级别高于从端，以允许主端先执行标记操作。What is provided above is that only one of the master end and the slave end can perform marking, but in other embodiments, the master end and the slave end can also perform marking at the same time and store their respective marking information. Further, when only one of the master end and the slave end can be marked, the surgical robot system can set the mark priority level, such as the master end and the slave end both issue mark instructions at the same time, at this time, determine the object to perform mark according to the priority level , if the marking priority of the master end is higher than that of the slave end, it allows the master end to perform marking operations first.

图8示出了根据发明一优选实施例的主端向从端共享主端标记的流程。如图8所示，在一实施例中，所述主端向从端共享主端标记的流程包括以下步骤：Fig. 8 shows the flow of the master end sharing the master end mark with the slave end according to a preferred embodiment of the invention. As shown in FIG. 8, in an embodiment, the process of sharing the master-end mark from the master end to the slave end includes the following steps:

步骤1：主端完成主端标记，以生成主端标记指令；具体地，主端医生在主端100通过操控右操作臂1044和左操作臂1043和手指捏合完成主端标记操作；Step 1: The main end completes the main end marking to generate the main end marking instruction; specifically, the main end doctor completes the main end marking operation by manipulating theright operating arm 1044 and theleft operating arm 1043 and pinching fingers at themain end 100;

步骤2：主端保存主端标记的位置信息，主要是坐标信息；主端标记指令获取模块1012会实时保存主端交互界面的屏幕中所有主端标记点的X和Y坐标信息，而且考虑到主端显示屏和从端显示屏的分辨率的差异，优选还保存主端屏幕的分辨率信息；Step 2: The main end saves the position information of the main end mark, mainly the coordinate information; the main end markinstruction acquisition module 1012 will save the X and Y coordinate information of all the main end mark points in the screen of the main end interactive interface in real time, and consider The difference between the resolution of the master display screen and the slave display screen, preferably also save the resolution information of the master screen;

步骤3：向从端发送主端标记的位置信息；主端标记完成后，主端信息共享模块1014会发送主端标记的位置信息至从端200；Step 3: Send the position information of the master end mark to the slave end; after the master end mark is completed, the master endinformation sharing module 1014 will send the position information of the master end mark to theslave end 200;

步骤4：从端接收主端标记的位置信息；从端200接受到主端标记的位置信息后进行数据解析，以获取主端标记的坐标；Step 4: The slave end receives the position information of the master mark; after receiving the position information of the master mark, theslave end 200 performs data analysis to obtain the coordinates of the master mark;

步骤5：从端根据解析数据重建主端标记；在绘制过程中，从端根据从端显示屏的分辨率进行等比例绘制主端标记；Step 5: The slave end reconstructs the master end mark according to the analysis data; during the drawing process, the slave end draws the master end mark proportionally according to the resolution of the slave end display screen;

步骤6：从端绘制完成主端标记后，最终将所述主端标记、所述从端标记和所述医学影像进行数据融合，获取融合主端标记和从端标记的初始医学影像并进行显示。Step 6: After the master marker is drawn from the slave end, data fusion is finally performed on the master marker, the slave marker and the medical image, and the initial medical image fused with the master marker and the slave marker is acquired and displayed .

结合图4和图11，在一具体实施例中，所述从端显示模块2025包括从端交互界面H1，能够显示初始医学影像和标记医学影像。进一步地，所述从端交互界面H1提供画图工具H2(即图形选择窗口)、 标记按钮H9和从端撤销按钮H8。所述从端交互界面H1的画图工具H2亦能够提供各种预设图形，如矩形的预设从端标记H3，圆形的预设从端标记H4，当然预设图形不限于此举例的，还可以是其他各种形状，而且形状不限于规则图形。所述从端交互界面H1的画图工具H2还可提供画笔，方便用户自定义绘制图形；使得用户可通过操作画图工具H2，并利用画图工具H2对从端交互界面H1上显示的初始医学影像进行对应于从端标记的图形的绘制，以得到所述从端标记。Referring to FIG. 4 and FIG. 11 , in a specific embodiment, theslave display module 2025 includes a slave interactive interface H1 capable of displaying initial medical images and marked medical images. Further, the interactive interface H1 of the slave terminal provides a drawing tool H2 (ie, a graph selection window), a mark button H9 and a cancel button H8 of the slave terminal. The drawing tool H2 of the slave-end interactive interface H1 can also provide various preset graphics, such as a rectangular preset slave-end mark H3, and a circular preset slave-end mark H4. Of course, the preset graphics are not limited to this example. Various other shapes are also possible, and the shape is not limited to regular figures. The drawing tool H2 of the slave-side interactive interface H1 can also provide a paintbrush to facilitate user-defined drawing of graphics; the user can operate the drawing tool H2 and utilize the drawing tool H2 to perform an initial medical image display on the slave-side interactive interface H1. Drawing of a graph corresponding to the slave end marker to obtain the slave end marker.

在图11示出的示例中，用户根据从端交互界面H1的画图工具H2提供的预设图形，可绘制圆形的从端标记H5，还可绘制矩形的从端标记H7，还可绘制自定义的从端标记H6。此外，所述从端撤销按钮H8用于提供撤销当前所绘制的对应于从端标记的图形的操作命令，用户只要鼠标左键点击该从端撤销按钮H8即可撤销当前图形。当然从端交互界面H1除显示标记、医学影像外，还用于显示含标记的初始医学影像。In the example shown in Figure 11, the user can draw a circular slave mark H5, a rectangular slave mark H7, and a slave mark H7 according to the preset graphics provided by the drawing tool H2 of the slave interactive interface H1. The defined slave end is labeled H6. In addition, the slave cancel button H8 is used to provide an operation command for canceling the currently drawn figure corresponding to the slave mark, and the user can cancel the current figure by clicking the slave cancel button H8 with the left mouse button. Of course, in addition to displaying markers and medical images, the slave terminal interaction interface H1 is also used to display initial medical images containing markers.

在一些实施例中，所述从端交互界面H1还提供退出标记按钮(未显示)，所述退出标记按钮用于输出退出标记指令，所述从端200根据所述退出标记按钮的退出标记指令退出标记模式。在一些实施例中，所述标记按钮H9用于输出启用标记指令，所述从端200根据标记按钮H9的启用标记指令启动所述标记模式。在另一实施例中，所述标记按钮H9和退出标记按钮集成为一个按钮，在进入从端标记模式后，标记按钮H9转变为退出标记按钮，而在退出标记模式后，退出标记按钮转变为标记按钮H9。In some embodiments, the slave terminal interaction interface H1 also provides an exit mark button (not shown), and the exit mark button is used to output an exit mark instruction, and theslave terminal 200 is configured according to the exit mark instruction of the exit mark button. Exit marker mode. In some embodiments, the marking button H9 is used for outputting an enabling marking instruction, and theslave terminal 200 starts the marking mode according to the marking enabling instruction of the marking button H9. In another embodiment, the mark button H9 and the exit mark button are integrated into one button. After entering the slave mark mode, the mark button H9 is transformed into an exit mark button, and after exiting the mark mode, the exit mark button is transformed into Label button H9.

如图4所示，在示例性实施例中，所述从端标记指令获取模块2022包括键盘2071和鼠标2072；键盘2071供用户按键输入和文字输入以方便生成文字形式的从端标记，从而实现文字标注；鼠标2072供用户选择和绘制图形，以方便生成图形形式的从端标记。在一具体实施例中，所述从端控制装置202与图像台车203集成，所述图像台车202可配置隔板211，用于放置键盘2071和鼠标2072。As shown in Figure 4, in an exemplary embodiment, the slave-side markinstruction acquisition module 2022 includes akeyboard 2071 and amouse 2072; thekeyboard 2071 is provided for user key input and text input to facilitate generation of the slave-side mark in text form, thereby realizing Text annotation; themouse 2072 is for the user to select and draw graphics, so as to facilitate the generation of slave-end marks in graphic form. In a specific embodiment, theslave control device 202 is integrated with theimage trolley 203 , and theimage trolley 202 can be configured with apartition 211 for placing akeyboard 2071 and amouse 2072 .

图9示出了根据本申请一优选实施例的从端标记创建的工作流程。如图9所示，在一非限制性实施例中共，所述从端标记创建的流程包括以下步骤：Fig. 9 shows the workflow of slave-side tag creation according to a preferred embodiment of the present application. As shown in Figure 9, in a non-limiting embodiment, the process of creating a slave tag includes the following steps:

步骤C1：启动从端标记指令获取模块；由于在从端的手术机器人运行程序中嵌入有从端标记指令获取模块2022，因此，在运行手术机器人程序后，即自动启动从端标记功能，后续需要进入从端标记模式才能进行从端标记操作。Step C1: Start the module for acquiring the slave marking instruction; since the slave markinginstruction acquiring module 2022 is embedded in the running program of the surgical robot at the slave end, after running the surgical robot program, the slave marking function will be automatically started, and the follow-up needs to enter Only in the slave marking mode can the slave marking operation be performed.

步骤C2：判断主端100是否处于标记状态；在一实施例中，主从端的标记功能为互斥状态，同一时刻，只有一个能进行标记操作；如果当前主端100处于标记状态，则从端200自动进入步骤C10以退出从端标记模式；如果当前主端100未处于标记状态，则从端200可进入步骤C3的从端标记模式。因此，所述从端控制装置202优选能够根据所述主端的非标记状态，确定在所述初始医学影像上进行标记以得到所述从端标记，并优选在步骤C4中向主端100发送对应于所述从端的本地当前的标记状态。Step C2: Determine whether themaster terminal 100 is in the marking state; in one embodiment, the marking functions of the master and slave terminals are mutually exclusive, and at the same time, only one can perform the marking operation; if thecurrent master terminal 100 is in the marking state, then the slave terminal The 200 automatically enters step C10 to exit the slave marking mode; if thecurrent master 100 is not in the marking state, theslave 200 can enter the slave marking mode in step C3. Therefore, the slave-end control device 202 is preferably able to determine to mark the initial medical image to obtain the slave-end mark according to the non-marking state of the master end, and preferably sends a corresponding message to themaster end 100 in step C4. The local current marking state on the slave.

步骤C3:从端进入从端标记模式；进入从端标记模式后，执行步骤C4：由从端200向主端100发送对应于从端的标记状态，从而将从端200正在标记的状态发送给主端100，以便主端100根据从端的标记状态来判断是否能够进入从端标记模式。如可通过鼠标2072点击从端交互界面的屏幕的标记按钮进入从端标记模式，优选进入从端标记模式后，从端交互界面在屏幕中自动显现预设的标记图形和从端撤销按钮。Step C3: enter the slave end mark mode from the end; after entering the slave end mark mode, perform step C4: send the mark state corresponding to the slave end by theslave end 200 to themaster end 100, thereby the state that theslave end 200 is marking is sent to themaster terminal 100, so that themaster terminal 100 can judge whether it can enter the slave terminal marking mode according to the marking status of the slave terminal. For example, themouse 2072 can be used to click the mark button on the screen of the slave-end interactive interface to enter the slave-end mark mode, preferably after entering the slave-end mark mode, the slave-end interactive interface automatically displays a preset mark graphic and a slave-end cancel button on the screen.

进入从端标记模式后，还执行步骤C5:判断是否选择预设标记图形；该步骤C5中，用户可选择系统预设的图形来绘制从端标记，也可选择自定义图形来绘制图形。After entering the slave marking mode, step C5 is also performed: judging whether to select the preset marking graphics; in this step C5, the user can select the graphics preset by the system to draw the slave markings, or select custom graphics to draw graphics.

进一步地进入步骤C6:按照上一步骤C5中选择的绘图方式来绘制从端标记；用户可通过鼠标的移动在医学图像上绘制从端标记。Go further to step C6: draw the slave end mark according to the drawing method selected in the previous step C5; the user can draw the slave end mark on the medical image by moving the mouse.

完成从端标记的绘制后，通过步骤C7可进一步判断是否撤销已绘制的从端标记；如果是，则流转至步骤C8以删除最新从端标记，否则程序自动流转至步骤C9。After completing the drawing of the slave mark, step C7 can be used to further determine whether to cancel the drawn slave mark; if yes, then flow to step C8 to delete the latest slave mark, otherwise the program automatically flows to step C9.

步骤C8：删除最新从端标记，优选每次只能删除最近一次的从端标记。Step C8: delete the latest slave mark, preferably only the latest slave mark can be deleted each time.

步骤C9：判断是否退出从端标记；如果是，则退出标记模式C10，如果否，则可继续循环流程C5至C9。Step C9: Judging whether to exit the slave marking; if yes, then exit the marking mode C10, if not, then continue the loop process C5 to C9.

在优选实施例中，当从端200进入从端标记模式后，其从端交互界面自动在屏幕中显现预设的从端标记图形和从端撤销按钮，而当从端200退出从端标记模式后，从端交互界面自动在屏幕中隐藏预设的从端标记图形和从端撤销按钮。In a preferred embodiment, after theslave terminal 200 enters the slave terminal marking mode, its slave terminal interactive interface automatically displays a preset slave terminal marking graphic and a slave cancel button on the screen, and when theslave terminal 200 exits the slave terminal marking mode After that, the slave terminal interactive interface automatically hides the preset slave mark graphics and slave cancel button on the screen.

更详细地，在一具体操作方式中，如图10所示，从端辅助医生通过鼠标点击标记按钮以输出从端标记指令，从而进入从端标记模式C3，进入从端标记模式C3后，与主端类似，亦可选择两种标记方式，包括：In more detail, in a specific operation mode, as shown in FIG. 10 , the slave-end assistant doctor clicks the mark button with the mouse to output the slave-end mark instruction, thereby entering the slave-end mark mode C3, and after entering the slave-end mark mode C3, and Similar to the main end, you can also choose two marking methods, including:

第一种标记方式：从端辅助医生确定绘制预设图形D2；然后，从端辅助医生通过鼠标左键选择预设图形D3；之后，从端辅助医生通过鼠标左键绘制预设图形D4；The first marking method: the assistant doctor at the slave end determines to draw the preset graphic D2; then, the assistant doctor at the slave end selects the preset graphic D3 with the left button of the mouse; after that, the assistant doctor at the slave end draws the preset graphic D4 with the left button of the mouse;

第二种标记方式：从端辅助医生确定绘制自定义图形D5；然后，从端助可通过鼠标左键绘制自定义图形路径D6。The second marking method: the assistant doctor from the terminal determines to draw the custom graphic D5; then, the assistant from the terminal can draw the custom graphic path D6 through the left button of the mouse.

以上标记方式最终都可执行撤销操作D7；若确定撤销，则可通过鼠标左键选择从端撤销按钮，撤销当前的从端标记图形，又可循环选择预设图形并进行绘制，或者又可重新绘制自定义图形路径。The above marking methods can finally perform the undo operation D7; if the undo is confirmed, you can use the left mouse button to select the slave end undo button to cancel the current slave end marking graphics, and you can cycle through the preset graphics and draw them, or you can re- Draw a custom graphics path.

因此，从端辅助医生通过鼠标点击从端交互界面的标记按钮，进入从端标记模式C3，在绘制预设标记图形时，首先鼠标移动光标到图形选择窗口，鼠标左键点击选择预设标记图形，选中后松开鼠标。接着，使用鼠标将光标移动到绘制区域，点击鼠标左键进行移动绘制操作，绘制完成后松开鼠标。Therefore, the slave-side assistant doctor clicks the mark button on the slave-side interactive interface with the mouse to enter the slave-end mark mode C3. When drawing the preset mark graph, first move the cursor to the graph selection window, and click the left mouse button to select the preset mark graph , select and release the mouse. Then, use the mouse to move the cursor to the drawing area, click the left button of the mouse to move and draw, and release the mouse after the drawing is completed.

在一具体实施例中，在不选择预设标记图形的状态下，从端200默认绘制自定义标记图形，使用鼠标将光标移动到绘制区域，点击鼠标左键进行移动绘制操作，绘制完成后松开鼠标。在撤销操作时，使用鼠标将光标移动到绘制区域撤销按钮位置，点击鼠标左键进行撤销操作。In a specific embodiment, in the state of not selecting a preset mark figure, theslave terminal 200 draws a custom mark figure by default, uses the mouse to move the cursor to the drawing area, clicks the left button of the mouse to perform a moving and drawing operation, and releases the Turn on the mouse. When undoing the operation, use the mouse to move the cursor to the position of the undo button in the drawing area, and click the left mouse button to undo the operation.

图12示出了根据本申请一优选实施例的从端向主端共享从端标记的流程。如图12所示，在一实施例中，所述从端向主端共享从端标记的流程包括以下步骤：Fig. 12 shows a process of sharing a slave tag from a slave to a master according to a preferred embodiment of the present application. As shown in Figure 12, in one embodiment, the process of sharing the slave mark from the slave to the master includes the following steps:

步骤11：从端完成从端标记，以生成从端标记指令；具体地，从端辅助医生在从端200通过鼠标和/或键盘完成从端标记操作；Step 11: the slave end completes the slave end marking to generate a slave end marking instruction; specifically, the slave end assists the doctor to complete the slave end marking operation through the mouse and/or keyboard at theslave end 200;

步骤12：从端保存从端标记的位置信息，主要是坐标信息；从端标记指令获取模块2022会实时保存从端交互界面的屏幕中所有从端标记点的X和Y坐标信息，而且考虑到主端显示屏和从端显示屏的分辨率的差异，优选还保存从端屏幕的分辨率信息；Step 12: Save the position information of the slave mark from the end, mainly coordinate information; the slave markinstruction acquisition module 2022 will save the X and Y coordinate information of all slave mark points in the screen of the slave end interactive interface in real time, and take into account The difference between the resolutions of the master display screen and the slave display screen, preferably saving the resolution information of the slave screen;

步骤13：向主端发送从端标记的位置信息；从端标记完成后，从端信息共享模块2024会发送从端标记的位置信息至主端100；Step 13: Send the position information of the slave mark to the master; after the slave mark is completed, the slaveinformation sharing module 2024 will send the position information of the slave mark to themaster 100;

步骤14：主端接收从端标记的位置信息；主端100接受到从端标记的位置信息后进行数据解析，以获取从端标记的坐标。Step 14: The master end receives the position information of the slave end mark; after receiving the position information of the slave end mark, themaster end 100 performs data analysis to obtain the coordinates of the slave end mark.

步骤15：主端根据解析数据重建从端标记，在绘制过程中，主端根据主端显示屏的分辨率进行等比例绘制从端标记；Step 15: The master rebuilds the slave markers based on the analysis data. During the drawing process, the master draws the slave markers proportionally according to the resolution of the master display screen;

步骤16：主端绘制完成从端标记后，最终将所述主端标记、所述从端标记和所述初始医学影像进行数据融合，获取融合所述主端标记和所述从端标记的所述初始医学影像并进行显示。Step 16: After the master end draws the slave end mark, finally perform data fusion of the master end mark, the slave end mark and the initial medical image, and obtain all The initial medical image and display it.

更具体地，如图13所示，所述主端100获取从端200所共享的从端标记信息Q1，并利用已保存的主端标记信息和建立的初始医疗影像Q2，基于数据融合技术Q3来获取含有标记的医学影像并进行对于主端的显示Q4，同理，所述从端200获取主端100所共享的主端标记信息Q5，并利用已保存的从端标记信息和建立的初始医疗影像Q2，基于数据融合技术Q3来获取含有标记的初始医学影像并进行 对于从端的显示Q6。More specifically, as shown in FIG. 13, themaster terminal 100 obtains the slave-end label information Q1 shared by the slave-end 200, and utilizes the saved master-end label information and the established initial medical image Q2, based on the data fusion technology Q3 To obtain medical images containing markers and display Q4 on the master end, similarly, theslave end 200 acquires the master end marker information Q5 shared by themaster end 100, and uses the saved slave end marker information and the established initial medical image The image Q2 is based on the data fusion technology Q3 to obtain the initial medical image containing the marker and display Q6 to the slave.

因此，所述主端100和从端200分别对各自获取的数据进行数据融合。主从端首先获取医疗影像数据，包括但不限于内窥视频数据、CT影像。然后获取从端标记信息和主端标记信息，在主端100和从端200分别获取这些标记的位置信息后，通过数据融合算法得到带有标记信息的医疗影像，最终，通过主从端各自的显示模块显示对应的含有标记的初始医疗影像。Therefore, themaster end 100 and theslave end 200 respectively perform data fusion on the data acquired by them. The master-slave first obtains medical image data, including but not limited to endoscopic video data and CT images. Then obtain the tag information of the slave end and the tag information of the master end. After themaster end 100 and theslave end 200 respectively obtain the position information of these tags, the medical image with the tag information is obtained through a data fusion algorithm. Finally, the master and slave ends respectively The display module displays the corresponding initial medical image containing the marker.

更详细地，如图14所示，在一具体实施例中，数据融合的原理为：In more detail, as shown in Figure 14, in a specific embodiment, the principle of data fusion is:

将流程L1中的自定义的主端标记L2，流程L3中的预设的从端标记L4，以及流程L5中的初始医学影像，经过流程L6中的数据融合后，得到主端融合后的标记医疗影像L7并通过主端显示屏显示，以及还可得到从端融合后的标记医疗影像L8并通过从端显示屏显示。从而主端医生能够在远程透过主端显示屏观看手术场景并与从端辅助医生进行交流，而且从端辅助医生能够在手术室透过从端显示屏及时获悉主端医生的手术指导并与主端医生进行交流，确保手术的顺利安全进行。After the self-defined master-end mark L2 in the process L1, the preset slave-end mark L4 in the process L3, and the initial medical image in the process L5, after the data fusion in the process L6, the master-end fused mark is obtained The medical image L7 is displayed through the display screen of the master end, and the marked medical image L8 fused by the slave end is also obtained and displayed through the display screen of the slave end. In this way, the doctor at the master end can watch the operation scene remotely through the display screen of the master end and communicate with the assistant doctor at the slave end, and the assistant doctor at the slave end can learn about the surgical guidance of the doctor at the master end in time through the display screen at the slave end in the operating room and communicate with the assistant doctor at the slave end. The host doctor communicates with each other to ensure the smooth and safe operation.

更进一步地，本申请一优选实施例还提供了一种可读存储介质，所述可读存储介质上存储有程序，当所述程序被执行时，执行如前标记共享装置所执行的标记共享方法。Furthermore, a preferred embodiment of the present application also provides a readable storage medium, on which a program is stored, and when the program is executed, the label sharing performed by the previous label sharing device is performed. method.

以及，本申请一优实施例还提供了一种电子设备，用于执行用于手术机器人的标记共享方法，所述电子设备包括处理器和如前所述的可读存储介质，所述处理器被配置用于执行所述可读存储介质上所存储的程序。And, a preferred embodiment of the present application also provides an electronic device for performing a marker sharing method for a surgical robot, the electronic device includes a processor and the above-mentioned readable storage medium, the processor configured to execute the programs stored on the readable storage medium.

还需理解，以上所说明的任一标记共享方法同样适用于本申请提供的标记共享装置。It should also be understood that any tag sharing method described above is also applicable to the tag sharing device provided in this application.

虽然本申请披露如上，但并不局限于此。本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。Although the present application discloses as above, it is not limited thereto. Those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims (16)

Translated fromChinese

一种用于手术机器人的标记共享方法，适用于第一手术机器人端，其特征在于，包括：A tag sharing method for surgical robots, applicable to the first surgical robot end, characterized in that it includes:获取当前生成的初始医学影像；Obtain the currently generated initial medical image;当获取到用户输入的第一标记指令后，将第一标记信息实时发送到第二手术机器人端，并在所述初始医学影像中融合所述第一标记信息，其中，所述第一标记指令包括所述第一标记信息；After obtaining the first marking instruction input by the user, the first marking information is sent to the second surgical robot in real time, and the first marking information is fused in the initial medical image, wherein the first marking instruction including the first tag information;当接收到所述第二手术机器人端实时发送的第二标记信息后，在所述初始医学影像中融合所述第二标记信息。After receiving the second marker information sent by the second surgical robot in real time, the second marker information is fused in the initial medical image.根据权利要求1所述的用于手术机器人的标记共享方法，其特征在于，当接收到所述第二手术机器人端实时发送的第二标记信息后，在融合所述第一标记信息的所述初始医学影像中融合所述第二标记信息。The marker sharing method for surgical robots according to claim 1, wherein after receiving the second marker information sent by the second surgical robot in real time, after fusing the first marker information The second marker information is fused into the initial medical image.根据权利要求1或2所述的用于手术机器人的标记共享方法，其特征在于，在获取到用户输入的第一标记指令之前，还包括：启动标记模式，所述标记模式允许用户输入所述第一标记指令。The marking sharing method for a surgical robot according to claim 1 or 2, further comprising: starting a marking mode, the marking mode allows the user to input the First mark command.根据权利要求1或2所述的用于手术机器人的标记共享方法，其特征在于，在所述初始医学影像中融合所述第一标记信息，包括：The marker sharing method for surgical robots according to claim 1 or 2, wherein fusing the first marker information in the initial medical image comprises:根据所述第一标记信息中的位置信息，在所述初始医学影像上重建对应于所述第一标记信息的第一标记；Reconstructing a first marker corresponding to the first marker information on the initial medical image according to the position information in the first marker information;在所述初始医学影像中融合所述第二标记信息，包括：Fusing the second marker information in the initial medical image includes:根据所述第二标记信息中的位置信息，在所述初始医学影像上重建对应于所述第二标记信息的第二标记。Reconstructing a second marker corresponding to the second marker information on the initial medical image according to the position information in the second marker information.根据权利要求1或2所述的用于手术机器人的标记共享方法，其特征在于，在所述初始医学影像中融合所述第一标记信息和/或在所述初始医学影像中融合所述第二标记信息之后，还包括：The marker sharing method for a surgical robot according to claim 1 or 2, wherein the first marker information is fused in the initial medical image and/or the first marker information is fused in the initial medical image After the second mark information, also include:获取当前生成的所述初始医学影像的图像变化信息，并根据所述初始医学影像的图像变化信息，更新所述第一标记信息和/或所述第二标记信息，以使所述第一标记信息和/或所述第二标记信息实时跟随所述初始医学影像的变化。Acquiring image change information of the currently generated initial medical image, and updating the first marker information and/or the second marker information according to the image change information of the initial medical image, so that the first marker information and/or the second marker information follows changes in the initial medical image in real time.根据权利要求3所述的用于手术机器人的标记共享方法，其特征在于，还包括：The label sharing method for surgical robots according to claim 3, further comprising:获取所述第一手术机器人端和/或所述第二手术机器人端的标记状态，并根据所述第一手术机器人端和/或所述第二手术机器人端的标记状态，输出主从控制指令或启用标记指令；所述主从控制指令用于控制所述第一手术机器人端或所述第二手术机器人端的运动；所述启用标记指令用于启动所述标记模式。Obtain the flag state of the first surgical robot end and/or the second surgical robot end, and output master-slave control instructions or enable A marking command; the master-slave control command is used to control the movement of the first surgical robot end or the second surgical robot end; the enabling marking command is used to start the marking mode.根据权利要求6所述的用于手术机器人的标记共享方法，其特征在于，还包括：获取用户输入的退出标记指令，并根据所述退出标记指令退出所述标记模式。The marking sharing method for a surgical robot according to claim 6, further comprising: obtaining an exit marking instruction input by a user, and exiting the marking mode according to the exit marking instruction.根据权利要求7所述的用于手术机器人的标记共享方法，其特征在于，还包括：The label sharing method for surgical robots according to claim 7, further comprising:根据所述启用标记指令进入操作锁定状态，以禁止所述第一手术机器人端和/或所述第二手术机器人端的运动；Entering into an operation lock state according to the activation flag instruction, so as to prohibit the movement of the first surgical robot end and/or the second surgical robot end;根据所述退出标记指令进入操作解锁状态，以允许所述第一手术机器人端和/或所述第二手术机器人端的运动。An operation unlock state is commanded according to the exit flag to allow movement of the first surgical robot end and/or the second surgical robot end.根据权利要求3所述的用于手术机器人的标记共享方法，其特征在于，还包括：The label sharing method for surgical robots according to claim 3, further comprising:获取所述第二手术机器人端的标记状态，并在确定所述第二手术机器人端当前为非标记状态时启动所述标记模式，并实时向所述第二手术机器人端发送本地当前的标记状态。Acquiring the marking state of the second surgical robot, and starting the marking mode when it is determined that the second surgical robot is currently in a non-marking state, and sending the local current marking state to the second surgical robot in real time.根据权利要求3所述的用于手术机器人的标记共享方法，其特征在于，还包括：The label sharing method for surgical robots according to claim 3, further comprising:获取所述第一手术机器人端的第一脚踏板的操作状态，并根据所述第一脚踏板的操作状态启动所述标记模式；Acquiring the operating state of the first foot pedal at the first surgical robot end, and starting the marking mode according to the operating state of the first foot pedal;获取所述第一手术机器人端的第二脚踏板的操作状态，并根据所述第二脚踏板的操作状态退出所述标记模式；或者，还包括：Obtaining the operating state of the second foot pedal on the first surgical robot end, and exiting the marking mode according to the operating state of the second foot pedal; or, further comprising:根据所述第一手术机器人端的交互界面上的标记按钮所输出的指令启动所述标记模式；Starting the marking mode according to an instruction output by a marking button on the interactive interface of the first surgical robot;根据所述第一手术机器人端的所述交互界面上的退出标记按钮所输出的指令退出所述标记模式。Exit the marking mode according to an instruction output by the exit marking button on the interaction interface of the first surgical robot.一种用于手术机器人的标记共享装置，适用于第一手术机器人端，其特征在于，包括：A marker sharing device for a surgical robot, suitable for the first surgical robot end, characterized in that it includes:影像数据获取模块，用于获取当前生成的初始医学影像；An image data acquisition module, configured to acquire the currently generated initial medical image;标记指令获取模块，用于获取用户输入的第一标记指令，所述第一标记指令包括第一标记信息，还用于获取第二手术机器人端实时发送的第二标记信息；A marking instruction acquisition module, configured to acquire a first marking instruction input by a user, the first marking instruction including first marking information, and also used to acquire second marking information sent in real time by the second surgical robot end;数据融合模块，用于在所述初始医学影像中融合所述第一标记信息和/或所述第二标记信息。A data fusion module, configured to fuse the first marker information and/or the second marker information in the initial medical image.一种手术机器人系统，包括第一手术机器人端和第二手术机器人端，其特征在于，所述第一手术机器人端包括如权利要求11所述的用于手术机器人的标记共享装置。A surgical robot system, comprising a first surgical robot end and a second surgical robot end, wherein the first surgical robot end comprises the marker sharing device for surgical robots according to claim 11 .根据权利要求12所述的手术机器人系统，其特征在于，所述第一手术机器人端还包括第一脚踏板和第二脚踏板，所述标记指令获取模块包括第一操作臂和第二操作臂；The surgical robot system according to claim 12, wherein the first surgical robot end further includes a first foot pedal and a second foot pedal, and the marking instruction acquisition module includes a first operating arm and a second operating arm. operating arm;所述第一脚踏板用于输出启用标记指令，以启动标记模式；The first foot pedal is used to output an enable flag command to start the flag mode;所述第二脚踏板用于输出退出标记指令，以退出所述标记模式；The second foot pedal is used to output an exit marking command to exit the marking mode;所述第一操作臂用于根据用户输入的操作指令选择对应于所述第一标记信息的第一标记；The first operating arm is used to select a first marker corresponding to the first marker information according to an operation instruction input by a user;所述第二操作臂用于根据用户输入的操作指令，采用已选择的所述第一标记，并在所述初始医学影像上创建所述第一标记。The second operating arm is configured to adopt the selected first marker according to an operation instruction input by a user, and create the first marker on the initial medical image.根据权利要求12所述的手术机器人系统，其特征在于，所述第一手术机器人端还包括交互界面，所述交互界面包括标记按钮和退出标记按钮；所述标记指令获取模块包括键盘和鼠标；The surgical robot system according to claim 12, wherein the first surgical robot terminal further includes an interactive interface, the interactive interface includes a marking button and an exit marking button; the marking instruction acquisition module includes a keyboard and a mouse;所述交互界面显示所述初始医学影像；The interactive interface displays the initial medical image;所述标记按钮用于输出启用标记指令，以启动标记模式；The mark button is used to output the enable mark instruction to start the mark mode;所述退出标记按钮用于输出退出标记指令，以退出所述标记模式；The exit marking button is used to output an exit marking instruction to exit the marking mode;所述键盘和所述鼠标用于根据用户输入的指令生成对应于所述第一标记信息的第一标记。The keyboard and the mouse are used to generate a first mark corresponding to the first mark information according to an instruction input by a user.一种可读存储介质，其上存储有程序，当所述程序被执行时，执行如权利要求1-10中任一项所述的用于手术机器人的标记共享方法。A readable storage medium, on which a program is stored, and when the program is executed, the marker sharing method for a surgical robot according to any one of claims 1-10 is executed.一种电子设备，用于执行用于手术机器人的标记共享方法，其特征在于，所述电子设备包括处理器和如权利要求15所述的可读存储介质，所述处理器被配置用于执行所述可读存储介质上所存储的程序。An electronic device for performing a marker sharing method for a surgical robot, wherein the electronic device includes a processor and a readable storage medium as claimed in claim 15, the processor is configured to execute The program stored on the readable storage medium.

Images