CN113648057B - Surgical navigation system and method for corresponding virtual space three-dimensional model to real space position - Google Patents
Surgical navigation system and method for corresponding virtual space three-dimensional model to real space positionDownload PDFInfo
- Publication number
- CN113648057B CN113648057BCN202110949777.2ACN202110949777ACN113648057BCN 113648057 BCN113648057 BCN 113648057BCN 202110949777 ACN202110949777 ACN 202110949777ACN 113648057 BCN113648057 BCN 113648057B
- Authority
- CN
- China
- Prior art keywords
- coordinate data
- mixed reality
- display device
- reality display
- surgical instrument
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034methodMethods0.000titleclaimsabstractdescription27
- 210000000056organAnatomy0.000claimsabstractdescription116
- 230000003287optical effectEffects0.000claimsabstractdescription53
- 230000009466transformationEffects0.000claimsabstractdescription15
- 238000012545processingMethods0.000claimsabstractdescription9
- 239000011521glassSubstances0.000description13
- 238000006243chemical reactionMethods0.000description10
- 238000004891communicationMethods0.000description8
- 238000001356surgical procedureMethods0.000description7
- 210000005036nerveAnatomy0.000description6
- 230000000007visual effectEffects0.000description5
- 238000002059diagnostic imagingMethods0.000description4
- 208000002193PainDiseases0.000description3
- 210000004204blood vesselAnatomy0.000description3
- 238000002595magnetic resonance imagingMethods0.000description3
- 230000036407painEffects0.000description3
- 208000027418Wounds and injuryDiseases0.000description2
- 239000008186active pharmaceutical agentSubstances0.000description2
- 239000003086colorantSubstances0.000description2
- 238000002591computed tomographyMethods0.000description2
- 230000006378damageEffects0.000description2
- 230000007547defectEffects0.000description2
- 238000010586diagramMethods0.000description2
- 239000003814drugSubstances0.000description2
- 238000003384imaging methodMethods0.000description2
- 208000014674injuryDiseases0.000description2
- 238000013507mappingMethods0.000description2
- 238000012986modificationMethods0.000description2
- 230000004048modificationEffects0.000description2
- 230000001131transforming effectEffects0.000description2
- 206010027677Fractures and dislocationsDiseases0.000description1
- 208000020307Spinal diseaseDiseases0.000description1
- 230000003190augmentative effectEffects0.000description1
- 238000013170computed tomography imagingMethods0.000description1
- 229940079593drugDrugs0.000description1
- 230000000694effectsEffects0.000description1
- 238000005516engineering processMethods0.000description1
- 210000003205muscleAnatomy0.000description1
- 210000001519tissueAnatomy0.000description1
Images
Classifications
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/102—Modelling of surgical devices, implants or prosthesis
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/107—Visualisation of planned trajectories or target regions
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/108—Computer aided selection or customisation of medical implants or cutting guides
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2055—Optical tracking systems
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
Landscapes
- Health & Medical Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Robotics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Technical Field
The invention relates to the technical field of mixed reality, in particular to a surgical navigation system and a method for corresponding a virtual space three-dimensional model to a real space position.
Background
In the treatment of spinal diseases such as thoracolumbar fractures and dislocation, the joint positioning is required by embedding pedicle screws in the spine. The key requirement of pedicle screw placement is that important tissues such as nerves, muscles and blood vessels are protected, and a doctor places pedicle screws into the spine accurately.
In existing mixed reality eyeglass positioning schemes, most rely on cameras of mixed reality eyeglasses themselves, and virtual targets are positioned by combining image recognition in an AR (Augmented Reality ) algorithm. However, the spatial positioning method by means of the camera and the AR algorithm has the defects of low precision and large positioning delay, is difficult to apply to pedicle screw placement operation navigation, and can increase medical hidden trouble.
Disclosure of Invention
The invention aims to overcome the defects of low positioning precision and large positioning delay in the mixed reality eyeglass positioning scheme in the prior art, and provides a surgical navigation system and a method for corresponding a virtual space three-dimensional model to a real space position.
The invention solves the technical problems by the following technical scheme:
a first aspect of the present invention provides a surgical navigation system comprising a communicatively coupled optical positioning device and a mixed reality display device;
the optical positioning device is used for respectively acquiring coordinate data of a target organ, a surgical instrument and the mixed reality display device in a real space and transmitting all acquired coordinate data to the mixed reality display device;
the mixed reality display device is used for carrying out transformation processing on the received coordinate data to respectively obtain coordinate data of the target organ, the surgical instrument and the mixed reality display device in a virtual space, and displaying a pre-stored target three-dimensional model and a three-dimensional model of the surgical instrument in the virtual space according to the coordinate data in the virtual space so that the position of the target three-dimensional model in the virtual space corresponds to the position of the target organ in a real space and the position of the three-dimensional model of the surgical instrument in the virtual space corresponds to the position of the surgical instrument in the real space;
the target three-dimensional model is used for displaying the three-dimensional structure of the target organ and the operation position on the target organ.
Optionally, a first positioning tool is arranged on the target organ, a second positioning tool is arranged on the surgical instrument, and a third positioning tool is arranged on the mixed reality display device;
the optical positioning device is specifically configured to collect first coordinate data of the first positioning tool, and use the first coordinate data as coordinate data of the target organ in a real space; collecting second coordinate data of the second positioning tool, and taking the second coordinate data as coordinate data of the surgical instrument in a real space; and collecting third coordinate data of the third positioning tool, and taking the third coordinate data as coordinate data of the mixed reality display device in a real space.
Optionally, the mixed reality display device is further configured to display a two-dimensional cross-sectional view in the virtual space according to the target three-dimensional model.
Optionally, the target organ is a spinal column, and the surgical instrument comprises a pedicle screw;
the mixed reality display device is also used for outputting prompt information according to the distance between the pedicle screw and the operation position in the virtual space.
Optionally, the mixed reality display device is further configured to collect gesture information, and adjust a three-dimensional model displayed in the virtual space according to the gesture information.
A second aspect of the present invention provides a method for a virtual space three-dimensional model corresponding to a real space position, comprising the steps of:
the optical positioning device respectively collects coordinate data of a target organ, a surgical instrument and a mixed reality display device in a real space;
transmitting all the acquired coordinate data to a mixed reality display device;
the mixed reality display device performs transformation processing on the received coordinate data to respectively obtain coordinate data of the target organ, the surgical instrument and the mixed reality display device in a virtual space;
displaying a pre-stored target three-dimensional model and a three-dimensional model of a surgical instrument in a virtual space according to the coordinate data in the virtual space, so that the position of the target three-dimensional model in the virtual space corresponds to the position of the target organ in a real space, and the position of the three-dimensional model of the surgical instrument in the virtual space corresponds to the position of the surgical instrument in the real space;
the target three-dimensional model is used for displaying the three-dimensional structure of the target organ and the operation position on the target organ.
Optionally, a first positioning tool is arranged on the target organ, a second positioning tool is arranged on the surgical instrument, and a third positioning tool is arranged on the mixed reality display device;
the step of the optical positioning device respectively collecting coordinate data of the target organ, the surgical instrument and the mixed reality display device in the real space specifically comprises the following steps:
the optical positioning device collects first coordinate data of the first positioning tool and takes the first coordinate data as coordinate data of the target organ in a real space; collecting second coordinate data of the second positioning tool, and taking the second coordinate data as coordinate data of the surgical instrument in a real space; and collecting third coordinate data of the third positioning tool, and taking the third coordinate data as coordinate data of the mixed reality display device in a real space.
Optionally, the method further comprises the steps of: the mixed reality display device displays a two-dimensional cross-sectional view in the virtual space according to the target three-dimensional model.
Optionally, the target organ is a spinal column, and the surgical instrument comprises a pedicle screw;
the method further comprises the steps of: the mixed reality display device outputs prompt information according to the distance between the pedicle screw and the operation position in the virtual space.
Optionally, the method further comprises the steps of: and the mixed reality display equipment acquires gesture information and adjusts the three-dimensional model displayed in the virtual space according to the gesture information.
The invention has the positive progress effects that: the surgical navigation system provided by the invention realizes surgical navigation based on the optical positioning equipment and the mixed reality display equipment, acquires coordinate data of the target organ, the surgical instrument and the mixed reality display equipment in the real space based on the optical positioning equipment, can greatly improve the accuracy of surgical navigation, can guide a doctor to perform operation at the visual blind spot position by means of the target three-dimensional model displayed in the virtual space by the mixed reality display equipment, can assist the doctor to find the surgical position more quickly and accurately, can assist the doctor to avoid damaging important parts such as nerve vessels and the like in the operation, reduces the burden of the doctor and the pain of patients, and has higher clinical application value.
Drawings
Fig. 1 is a block diagram of a surgical navigation system according to embodiment 1 of the present invention.
Fig. 2 is an application scenario diagram of a surgical navigation system provided in embodiment 1 of the present invention.
Fig. 3 is a flowchart of a method for mapping a virtual space three-dimensional model to a real space position according to embodiment 2 of the present invention.
Detailed Description
The invention is further illustrated by means of the following examples, which are not intended to limit the scope of the invention.
Example 1
As shown in fig. 1, the present embodiment provides asurgical navigation system 100 comprising a communicatively coupledoptical pointing device 20 and mixedreality display device 30. Theoptical positioning device 20 and the mixedreality display device 30 may be in communication connection through a wireless manner, such as Wi-Fi (wireless communication technology), or may be in communication connection through a wired manner, such as a data line.
Theoptical positioning device 20 is used for respectively acquiring coordinate data of a target organ, a surgical instrument and a mixed reality display device in a real space, and transmitting all acquired coordinate data to the mixedreality display device 30.
In a specific implementation, the target organ may be a site to be operated on. The surgical instrument may include an object to be placed in the target organ and an instrument required for placing the object in the target organ.
In an alternative embodiment, the target organ is provided with a first positioning tool, the surgical instrument is provided with a second positioning tool, and the mixed reality display device is provided with a third positioning tool. The optical positioning device is specifically configured to collect first coordinate data of the first positioning tool, and use the first coordinate data as coordinate data of the target organ in a real space; collecting second coordinate data of the second positioning tool, and taking the second coordinate data as coordinate data of the surgical instrument in a real space; and collecting third coordinate data of the third positioning tool, and taking the third coordinate data as coordinate data of the mixed reality display device in a real space.
In order to ensure that the coordinate data acquired by the optical positioning device does not deviate in the operation process, the first positioning tool is rigidly connected with the target organ, the second positioning tool is rigidly connected with the surgical instrument, and the third positioning tool is rigidly connected with the mixed reality display device.
In a specific implementation, at least three optical positioning target balls are mounted on each of the first positioning tool, the second positioning tool and the third positioning tool. In a specific example, four optical positioning target balls are mounted on each of the first positioning tool, the second positioning tool, and the third positioning tool.
It will be appreciated that during surgery, the position of the surgical instrument and the mixed reality display device in real space will vary with the operation of the physician, and therefore, the optical positioning device is to acquire the coordinate data of the target organ, the surgical instrument and the mixed reality display device in real space, i.e. the first coordinate data of the first positioning tool, the second coordinate data of the second positioning tool and the third coordinate data of the third positioning tool in real time.
The mixedreality display device 30 is configured to perform transformation processing on the received coordinate data to obtain coordinate data of the target organ, the surgical instrument, and the mixed reality display device in a virtual space, and display a pre-stored target three-dimensional model and a three-dimensional model of the surgical instrument in the virtual space according to the coordinate data in the virtual space, so that a position of the target three-dimensional model in the virtual space corresponds to a position of the target organ in the real space, and a position of the three-dimensional model of the surgical instrument in the virtual space corresponds to a position of the surgical instrument in the real space.
The target three-dimensional model is used for displaying the three-dimensional structure of the target organ and the operation position on the target organ.
In a specific implementation, the mixed reality display device may be a head-mounted or ear-mounted mixed reality glasses. During the operation, a doctor can wear the head-mounted mixed reality glasses and see the three-dimensional model displayed in the virtual space through the head-mounted mixed reality glasses.
The mixed reality display device stores a target three-dimensional model and a three-dimensional model of a surgical instrument in advance. Specifically, medical imaging of the target organ can be obtained by means of medical imaging means such as CT (Computed Tomography, electronic computed tomography) or MRI (Magnetic Resonance Imaging ). Typically, the medical image is in DICOM (Digital Imaging and Communications in Medicine, digital imaging and communication in medicine) format, and image processing is required to be performed on the medical image in DICOM format to obtain a three-dimensional model of the target organ that can be presented in mixed reality glasses. In the operation planning stage, the operation position is marked in the three-dimensional model of the target organ through lines with different colors, so that a target three-dimensional model is formed. The data of the target three-dimensional model and the data of the three-dimensional model of the surgical instrument are then input into the mixed reality display device.
In an optional implementation manner, the mixed reality display device is configured to perform first coordinate transformation on coordinate data of the target organ and the surgical instrument in a real space, to obtain coordinate data of the target organ and the surgical instrument in a local coordinate system, and perform second coordinate transformation on coordinate data of the target organ and the surgical instrument in the local coordinate system according to the coordinate data of the mixed reality display device in the real space and the coordinate data of the mixed reality display device in the virtual space, to obtain coordinate data of the target organ and the surgical instrument in the virtual space.
In this embodiment, the mixed reality display device converts coordinate data in a real space coordinate system into coordinate data in a local coordinate system through first coordinate conversion; and converting the coordinate data under the local coordinate system into the coordinate data under the virtual space coordinate system through the second coordinate conversion. The real space coordinate system may also be referred to as a coordinate system of the optical positioning device, and the local coordinate system may also be referred to as a coordinate system of the mixed reality display device.
In an optional implementation manner, the mixed reality display device is used for subtracting the target coordinate data from the coordinate data of the target organ and the surgical instrument in the real space to obtain the coordinate data of the target organ and the surgical instrument in the local coordinate system. The target coordinate data are coordinate data, in real space, of the mixed reality display device, acquired during initialization of the optical positioning device, namely an origin of a local coordinate system.
In this embodiment, since the unit of the real space coordinate system and the local coordinate system is 1mm, there is no scaling relationship between the two coordinate systems. The coordinate data of the target organ in the real space is NDIM, the coordinate data of the surgical instrument in the real space is NDIP, the coordinate data of the mixed reality display device acquired during the initialization of the optical positioning device in the real space is NDIH1, the coordinate data of the target organ in the local coordinate system can be obtained to be NDIM-NDIH1, and the coordinate data of the surgical instrument in the local coordinate system is NDIP-NDIH 1.
In an optional implementation manner, the mixed reality display device is used for multiplying coordinate data of the target organ and the surgical instrument in a local coordinate system by conversion coefficients respectively to obtain coordinate data of the target organ and the surgical instrument in a virtual space. The conversion coefficient is the coordinate data of the mixed reality display device in a virtual space divided by the coordinate data of the mixed reality display device in a real space. It should be noted that, the coordinate data of the mixed reality display device in the real space is real-time data, which is different from the coordinate data of the mixed reality display device in the real space collected during the initialization of the optical positioning device.
In this embodiment, the coordinate data of the mixed reality display device in the virtual space is UnityH1, and specifically, can be obtained through the API of Unity. Real-time coordinate data of the mixed reality display device in a real space is NDIH2, and a conversion coefficient m=unityh1/NDIH 2. Multiplying the coordinate data (NDIM-NDIH 1) of the target organ in the local coordinate system by M to obtain the coordinate data (NDIM-NDIH 1) of the target organ in the virtual space as UnityH 1/NDIH 2. Multiplying the coordinate data (NDIP-NDIH 1) of the surgical instrument in the local coordinate system by M, to obtain the coordinate data (NDIP-NDIH 1) of the surgical instrument in the virtual space as uni-NDIH 1/NDIH 2.
In one example of the pedicle screw placement operation, as shown in fig. 2, the target organ is aspinal column 42, thesurgical instrument 43 includes pedicle screws to be placed in the spinal column, theoptical positioning device 20 collects coordinate data of thespinal column 42, thesurgical instrument 43, and the head-mountedmixed reality glasses 44, and transmits the collected coordinate data to the head-mountedmixed reality glasses 44, and the head-mountedmixed reality glasses 44 present a three-dimensional model in a virtual space after transforming the coordinate data.
In the above example of pedicle screw placement, the surgical site on the target organ is the location of pedicle screw placement into the spine.
In this embodiment, the optical positioning device performs optical positioning on the target organ, the surgical instrument and the mixed reality display device in real time, the mixed reality display device presents a three-dimensional model in the virtual space according to real-time position information provided by the optical positioning device, a doctor coincides with the position of the actual target organ through the target three-dimensional model seen by the mixed reality display device, and coincides with the position of the actual surgical instrument, meanwhile, the surgical position planned before surgery can be displayed on the actual target organ, and the doctor can operate the actual surgical instrument according to the surgical position in the target three-dimensional model, thereby realizing surgical navigation.
The operation navigation system provided by the embodiment realizes operation navigation based on the optical positioning equipment and the mixed reality display equipment, can greatly improve the accuracy of operation navigation based on the optical positioning equipment, can guide a doctor to operate at the position of the visual blind spot by means of the target three-dimensional model displayed in the virtual space by the mixed reality display equipment, can assist the doctor to find the operation position faster and more accurately, avoid damaging important parts such as nerve blood vessels in the assisting operation, lighten the burden of the doctor and the pain of a patient, and has higher clinical application value.
In the specific implementation, the setting position of the first positioning tool on the target organ determines whether the position of the target organ in the real space can completely coincide with the position of the target three-dimensional model in the virtual space, and similarly, the setting position of the second positioning tool on the surgical instrument determines whether the position of the surgical instrument in the real space can completely coincide with the position of the three-dimensional model of the surgical instrument in the virtual space. Therefore, manual calibration is often required before surgery, specifically, manual fine adjustment is performed on the three-dimensional model displayed in the virtual space by the mixed reality display device, so that the position of the target organ in the real space completely coincides with the position of the target three-dimensional model in the virtual space, and the position of the surgical instrument in the real space completely coincides with the position of the three-dimensional model of the surgical instrument in the virtual space.
In an alternative embodiment, to further assist the doctor in determining the surgical position, the mixed reality display device is further configured to display a two-dimensional cross-sectional view in the virtual space according to the target three-dimensional model.
In the above example of pedicle screw placement, the two-dimensional cross-sectional view displayed by the mixed reality display device includes at least two cross-sections, wherein the picture of one cross-section is a plane formed by the nail tip and the nail post passing through the pedicle screw, and the picture formed by cutting the three-dimensional model of the target; the other profile is taken from a plane formed through the tip of the pedicle screw and perpendicular to the post of the pedicle screw, and from a three-dimensional model of the object.
In the above example of pedicle screw placement surgery, the mixed reality display device is further configured to output hint information based on a distance between the pedicle screw and the surgical site within the virtual space. Note that, in general, a prompt message is output according to a distance between the tip of the pedicle screw and the operation position, so as to prompt a doctor whether the tip of the pedicle screw deviates from the operation position. The prompt information may be a sound, for example, the mixed reality display device outputs a sound through a speaker to prompt a doctor of a real-time distance between the nail tip and the operation position. The prompt information may also be an image, for example, the mixed reality display device prompts the doctor by displaying the image in the virtual space.
In a specific implementation, the coordinate data of the tip of the pedicle screw in the virtual space may be determined from the length of the pedicle screw, the location of the pedicle screw on the surgical instrument, and the coordinate data of the surgical instrument in the virtual space. The distance between the two can be calculated according to the coordinate data of the nail tip of the pedicle screw and the operation position in the virtual space. Wherein, can export different prompt messages according to different distances. In a specific example, if the distance between the spike tip and the surgical site is greater than a preset distance, no prompt message is output; and if the distance between the nail tip and the operation position is smaller than or equal to the preset distance, outputting prompt information. In another specific example, the cue is a sound, and the volume of the cue sound increases as the distance between the tip of the nail and the surgical site increases.
In an optional implementation manner, the mixed reality display device is further configured to collect gesture information, and adjust a three-dimensional model or a two-dimensional cross-sectional view displayed in the virtual space according to the gesture information. In order to enable a doctor to more accurately determine an operation position and assist in avoiding injury to important parts such as nerve vessels in operation, in a specific implementation, the doctor can rotate, move, enlarge or reduce a three-dimensional model of a target, a three-dimensional model of a surgical instrument or a two-dimensional section view displayed in a virtual space through specific gestures, so that different visual angles can be freely checked.
Example 2
The present embodiment provides a method for mapping a virtual space three-dimensional model to a real space position, which can be implemented using the surgical navigation system described in embodiment 1. As shown in fig. 3, the method provided in this embodiment may include the following steps S301 to S304:
step S301, the optical positioning device acquires coordinate data of the target organ, the surgical instrument and the mixed reality display device in a real space respectively.
In a specific implementation, the target organ may be a site to be operated on. The surgical instrument may include an object to be placed in the target organ and an instrument required for placing the object in the target organ.
In an alternative embodiment, the target organ is provided with a first positioning tool, the surgical instrument is provided with a second positioning tool, and the mixed reality display device is provided with a third positioning tool. In this embodiment, step S301 specifically includes: the optical positioning device collects first coordinate data of the first positioning tool and takes the first coordinate data as coordinate data of the target organ in a real space; collecting second coordinate data of the second positioning tool, and taking the second coordinate data as coordinate data of the surgical instrument in a real space; and collecting third coordinate data of the third positioning tool, and taking the third coordinate data as coordinate data of the mixed reality display device in a real space.
In order to ensure that the coordinate data acquired by the optical positioning device does not deviate in the operation process, the first positioning tool is rigidly connected with the target organ, the second positioning tool is rigidly connected with the surgical instrument, and the third positioning tool is rigidly connected with the mixed reality display device.
In a specific implementation, at least three optical positioning target balls are mounted on each of the first positioning tool, the second positioning tool and the third positioning tool. In a specific example, four optical positioning target balls are mounted on each of the first positioning tool, the second positioning tool, and the third positioning tool.
It will be appreciated that during surgery, the position of the surgical instrument and the mixed reality display device in real space will vary with the operation of the physician, and therefore, the optical positioning device is to acquire the coordinate data of the target organ, the surgical instrument and the mixed reality display device in real space, i.e. the first coordinate data of the first positioning tool, the second coordinate data of the second positioning tool and the third coordinate data of the third positioning tool in real time.
And step S302, all the acquired coordinate data are sent to the mixed reality display device. The optical positioning device is in communication connection with the mixed reality display device, and the optical positioning device sends all acquired coordinate data to the mixed reality display device. The optical positioning device and the mixed reality display device can be in communication connection in a wireless mode, such as Wi-Fi, and can also be in communication connection in a wired mode, such as a data wire.
Step S303, the mixed reality display device performs transformation processing on the received coordinate data, so as to obtain coordinate data of the target organ, the surgical instrument and the mixed reality display device in a virtual space respectively.
And step S304, displaying a pre-stored target three-dimensional model and a three-dimensional model of the surgical instrument in the virtual space according to the coordinate data in the virtual space, so that the position of the target three-dimensional model in the virtual space corresponds to the position of the target organ in the real space, and the position of the three-dimensional model of the surgical instrument in the virtual space corresponds to the position of the surgical instrument in the real space.
The target three-dimensional model is used for displaying the three-dimensional structure of the target organ and the operation position on the target organ.
In a specific implementation, the mixed reality display device may be a head-mounted mixed reality glasses, and in the operation process, a doctor may wear the head-mounted mixed reality glasses and see the three-dimensional model displayed in the virtual space through the head-mounted mixed reality glasses.
The mixed reality display device stores a target three-dimensional model and a three-dimensional model of a surgical instrument in advance. Specifically, medical imaging of the target organ can be obtained by means of medical imaging means such as CT or MRI. In general, a medical image is in DICOM format, and it is necessary to perform image processing on the medical image in DICOM format to obtain a three-dimensional model of a target organ that can be presented in mixed reality glasses. In the operation planning stage, the operation position is marked in the three-dimensional model of the target organ through lines with different colors, so that a target three-dimensional model is formed. The data of the target three-dimensional model and the data of the three-dimensional model of the surgical instrument are then input into the mixed reality display device.
In an optional implementation manner, the mixed reality display device performs first coordinate transformation on coordinate data of the target organ and the surgical instrument in a real space to obtain coordinate data of the target organ and the surgical instrument in a local coordinate system, and performs second coordinate transformation on coordinate data of the target organ and the surgical instrument in the local coordinate system according to the coordinate data of the mixed reality display device in the real space and the coordinate data of the surgical instrument in the virtual space to obtain coordinate data of the target organ and the coordinate data of the surgical instrument in the virtual space.
In this embodiment, the mixed reality display device converts coordinate data in a real space coordinate system into coordinate data in a local coordinate system through first coordinate conversion; and converting the coordinate data under the local coordinate system into the coordinate data under the virtual space coordinate system through the second coordinate conversion. The real space coordinate system may also be referred to as a coordinate system of the optical positioning device, and the local coordinate system may also be referred to as a coordinate system of the mixed reality display device.
In an optional implementation manner, the mixed reality display device subtracts the target coordinate data from the coordinate data of the target organ and the surgical instrument in real space, so as to obtain the coordinate data of the target organ and the surgical instrument in a local coordinate system. The target coordinate data are coordinate data, in real space, of the mixed reality display device, acquired during initialization of the optical positioning device, namely an origin of a local coordinate system.
In this embodiment, since the unit of the real space coordinate system and the local coordinate system is 1mm, there is no scaling relationship between the two coordinate systems. The coordinate data of the target organ in the real space is NDIM, the coordinate data of the surgical instrument in the real space is NDIP, the coordinate data of the mixed reality display device acquired during the initialization of the optical positioning device in the real space is NDIH1, the coordinate data of the target organ in the local coordinate system can be obtained to be NDIM-NDIH1, and the coordinate data of the surgical instrument in the local coordinate system is NDIP-NDIH 1.
In an optional implementation manner, the mixed reality display device multiplies the coordinate data of the target organ and the surgical instrument in the local coordinate system by a conversion coefficient to obtain the coordinate data of the target organ and the surgical instrument in the virtual space. The conversion coefficient is the coordinate data of the mixed reality display device in a virtual space divided by the coordinate data of the mixed reality display device in a real space. It should be noted that, the coordinate data of the mixed reality display device in the real space is real-time data, which is different from the coordinate data of the mixed reality display device in the real space collected during the initialization of the optical positioning device.
In this embodiment, the coordinate data of the mixed reality display device in the virtual space is UnityH1, and specifically, can be obtained through the API of Unity. Real-time coordinate data of the mixed reality display device in a real space is NDIH2, and a conversion coefficient m=unityh1/NDIH 2. Multiplying the coordinate data (NDIM-NDIH 1) of the target organ in the local coordinate system by M to obtain the coordinate data (NDIM-NDIH 1) of the target organ in the virtual space as UnityH 1/NDIH 2. Multiplying the coordinate data (NDIP-NDIH 1) of the surgical instrument in the local coordinate system by M, to obtain the coordinate data (NDIP-NDIH 1) of the surgical instrument in the virtual space as uni-NDIH 1/NDIH 2.
In one example of pedicle screw placement, the target organ is the spinal column, the surgical instrument includes pedicle screws to be placed in the spinal column, the optical locating device collects coordinate data of the spinal column, the surgical instrument, and the head-mounted mixed reality glasses, and transmits the collected coordinate data to the head-mountedmixed reality glasses 44, which present a three-dimensional model in the virtual space after transforming the coordinate data.
In the above example of pedicle screw placement, the surgical site on the target organ is the location of pedicle screw placement into the spine.
In this embodiment, the optical positioning device performs optical positioning on the target organ, the surgical instrument and the mixed reality display device in real time, the mixed reality display device presents a three-dimensional model in the virtual space according to real-time position information provided by the optical positioning device, a doctor coincides with the position of the actual target organ through the target three-dimensional model seen by the mixed reality display device, and coincides with the position of the actual surgical instrument, meanwhile, the surgical position planned before surgery can be displayed on the actual target organ, and the doctor can operate the actual surgical instrument according to the surgical position in the target three-dimensional model, thereby realizing surgical navigation.
According to the method, surgical navigation is realized based on the optical positioning equipment and the mixed reality display equipment, the accuracy of surgical navigation can be greatly improved based on the optical positioning equipment, a doctor can be guided to operate at the position of the visual blind spot by means of the target three-dimensional model displayed in the virtual space by the mixed reality display equipment, the doctor can be assisted to find the surgical position more quickly and accurately, important parts such as nerve blood vessels are prevented from being damaged in the auxiliary operation, the burden of the doctor and pains of patients are relieved, and the method has higher clinical application value.
In the specific implementation, the setting position of the first positioning tool on the target organ determines whether the position of the target organ in the real space can completely coincide with the position of the target three-dimensional model in the virtual space, and similarly, the setting position of the second positioning tool on the surgical instrument determines whether the position of the surgical instrument in the real space can completely coincide with the position of the three-dimensional model of the surgical instrument in the virtual space. Therefore, manual calibration is often required before surgery, specifically, manual fine adjustment is performed on the three-dimensional model displayed in the virtual space by the mixed reality display device, so that the position of the target organ in the real space completely coincides with the position of the target three-dimensional model in the virtual space, and the position of the surgical instrument in the real space completely coincides with the position of the three-dimensional model of the surgical instrument in the virtual space.
In an alternative embodiment, to further assist the surgeon in determining the surgical site, the method further comprises the steps of: the mixed reality display device displays a two-dimensional cross-sectional view in the virtual space according to the target three-dimensional model.
In the above example of pedicle screw placement, the two-dimensional cross-sectional view displayed by the mixed reality display device includes at least two cross-sections, wherein the picture of one cross-section is a plane formed by the nail tip and the nail post passing through the pedicle screw, and the picture formed by cutting the three-dimensional model of the target; the other profile is taken from a plane formed through the tip of the pedicle screw and perpendicular to the post of the pedicle screw, and from a three-dimensional model of the object.
In the above example of pedicle screw placement, the method may further comprise the steps of: the mixed reality display device outputs prompt information according to the distance between the pedicle screw and the operation position in the virtual space. Note that, in general, a prompt message is output according to a distance between the tip of the pedicle screw and the operation position, so as to prompt a doctor whether the tip of the pedicle screw deviates from the operation position. The prompt information may be a sound, for example, the mixed reality display device outputs a sound through a speaker to prompt a doctor of a real-time distance between the nail tip and the operation position. The prompt information may also be an image, for example, the mixed reality display device prompts the doctor by displaying the image in the virtual space.
In a specific implementation, the coordinate data of the tip of the pedicle screw in the virtual space may be determined from the length of the pedicle screw, the location of the pedicle screw on the surgical instrument, and the coordinate data of the surgical instrument in the virtual space. The distance between the two can be calculated according to the coordinate data of the nail tip of the pedicle screw and the operation position in the virtual space. Wherein, can export different prompt messages according to different distances. In a specific example, if the distance between the spike tip and the surgical site is greater than a preset distance, no prompt message is output; and if the distance between the nail tip and the operation position is smaller than or equal to the preset distance, outputting prompt information. In another specific example, the cue is a sound, and the volume of the cue sound increases as the distance between the tip of the nail and the surgical site increases.
In an alternative embodiment, the method further comprises the steps of: and the mixed reality display equipment acquires gesture information and adjusts a three-dimensional model or a two-dimensional section view displayed in the virtual space according to the gesture information. In order to enable a doctor to more accurately determine an operation position and assist in avoiding injury to important parts such as nerve vessels in operation, in a specific implementation, the doctor can rotate, move, enlarge or reduce a three-dimensional model of a target, a three-dimensional model of a surgical instrument or a two-dimensional section view displayed in a virtual space through specific gestures, so that different visual angles can be freely checked.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the principles and spirit of the invention, but such changes and modifications fall within the scope of the invention.
Claims (10)
1. A surgical navigation system comprising a communicatively coupled optical positioning device and a mixed reality display device;
the optical positioning device is used for respectively acquiring coordinate data of a target organ, a surgical instrument and the mixed reality display device in a real space and transmitting all acquired coordinate data to the mixed reality display device; the position of the surgical instrument and the mixed reality display device in the real space changes along with the operation of a doctor;
the mixed reality display device is used for carrying out transformation processing on the received coordinate data to respectively obtain coordinate data of the target organ, the surgical instrument and the mixed reality display device in a virtual space, and displaying a pre-stored target three-dimensional model and a three-dimensional model of the surgical instrument in the virtual space according to the coordinate data in the virtual space so that the position of the target three-dimensional model in the virtual space corresponds to the position of the target organ in the real space, and the position of the three-dimensional model of the surgical instrument in the virtual space corresponds to the position of the surgical instrument in the real space;
the target three-dimensional model is used for displaying the three-dimensional structure of the target organ and the operation position on the target organ;
the mixed reality display device is used for respectively carrying out first coordinate transformation on coordinate data of the target organ and the surgical instrument in a real space to obtain coordinate data of the target organ and the surgical instrument in a local coordinate system, and respectively carrying out second coordinate transformation on coordinate data of the target organ and the surgical instrument in the local coordinate system according to the coordinate data of the mixed reality display device in the real space and the coordinate data of the mixed reality display device in a virtual space to obtain coordinate data of the target organ and the surgical instrument in the virtual space.
2. The surgical navigation system of claim 1, wherein the target organ has a first positioning tool thereon, the surgical instrument has a second positioning tool thereon, and the mixed reality display device has a third positioning tool thereon;
the optical positioning device is specifically configured to collect first coordinate data of the first positioning tool, and use the first coordinate data as coordinate data of the target organ in a real space; collecting second coordinate data of the second positioning tool, and taking the second coordinate data as coordinate data of the surgical instrument in a real space; and collecting third coordinate data of the third positioning tool, and taking the third coordinate data as coordinate data of the mixed reality display device in a real space.
3. The surgical navigation system of claim 1, wherein the mixed reality display device is further configured to present a two-dimensional cross-sectional view within the virtual space according to the target three-dimensional model.
4. The surgical navigation system of claim 1, wherein the target organ is a spinal column and the surgical instrument comprises a pedicle screw;
the mixed reality display device is also used for outputting prompt information according to the distance between the pedicle screw and the operation position in the virtual space.
5. The surgical navigation system of any of claims 1-4, wherein the mixed reality display device is further configured to collect gesture information and adjust a three-dimensional model presented in the virtual space based on the gesture information.
6. A method for a virtual space three-dimensional model corresponding to a real space location, comprising the steps of:
the positions of the surgical instrument and the mixed reality display device in the real space are changed along with the operation of a doctor;
transmitting all acquired coordinate data to a mixed reality display device, wherein the coordinate data are coordinate data of a target organ, a surgical instrument and the mixed reality display device which are respectively acquired according to an optical positioning device in a real space;
the mixed reality display device performs transformation processing on the received coordinate data to respectively obtain coordinate data of the target organ, the surgical instrument and the mixed reality display device in a virtual space;
the mixed reality display device is used for respectively carrying out first coordinate transformation on the coordinate data of the target organ and the surgical instrument in a real space to obtain the coordinate data of the target organ and the surgical instrument in a local coordinate system, and respectively carrying out second coordinate transformation on the coordinate data of the target organ and the surgical instrument in the local coordinate system according to the coordinate data of the mixed reality display device in the real space and the coordinate data of the mixed reality display device in a virtual space to obtain the coordinate data of the target organ and the surgical instrument in the virtual space;
displaying a pre-stored target three-dimensional model and a three-dimensional model of a surgical instrument in a virtual space according to the coordinate data in the virtual space, so that the position of the target three-dimensional model in the virtual space corresponds to the position of the target organ in a real space, and the position of the three-dimensional model of the surgical instrument in the virtual space corresponds to the position of the surgical instrument in the real space;
the target three-dimensional model is used for displaying the three-dimensional structure of the target organ and the operation position on the target organ.
7. The method of claim 6, wherein a first positioning tool is provided on the target organ, a second positioning tool is provided on the surgical instrument, and a third positioning tool is provided on the mixed reality display device;
the step of the optical positioning device respectively collecting coordinate data of the target organ, the surgical instrument and the mixed reality display device in the real space specifically comprises the following steps:
the optical positioning device collects first coordinate data of the first positioning tool and takes the first coordinate data as coordinate data of the target organ in a real space; collecting second coordinate data of the second positioning tool, and taking the second coordinate data as coordinate data of the surgical instrument in a real space; and collecting third coordinate data of the third positioning tool, and taking the third coordinate data as coordinate data of the mixed reality display device in a real space.
8. The method of claim 6, further comprising the step of: the mixed reality display device displays a two-dimensional cross-sectional view in the virtual space according to the target three-dimensional model.
9. The method of claim 6, wherein the target organ is a spinal column and the surgical instrument comprises a pedicle screw;
the method further comprises the steps of: the mixed reality display device outputs prompt information according to the distance between the pedicle screw and the operation position in the virtual space.
10. The method according to any one of claims 6-9, further comprising the step of: and the mixed reality display equipment acquires gesture information and adjusts the three-dimensional model displayed in the virtual space according to the gesture information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110949777.2ACN113648057B (en) | 2021-08-18 | 2021-08-18 | Surgical navigation system and method for corresponding virtual space three-dimensional model to real space position |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110949777.2ACN113648057B (en) | 2021-08-18 | 2021-08-18 | Surgical navigation system and method for corresponding virtual space three-dimensional model to real space position |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113648057A CN113648057A (en) | 2021-11-16 |
| CN113648057Btrue CN113648057B (en) | 2023-04-25 |
Family
ID=78481045
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110949777.2AActiveCN113648057B (en) | 2021-08-18 | 2021-08-18 | Surgical navigation system and method for corresponding virtual space three-dimensional model to real space position |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113648057B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114159157A (en)* | 2021-12-06 | 2022-03-11 | 北京诺亦腾科技有限公司 | Method, device and equipment for assisting moving of instrument and storage medium |
| CN114366330B (en)* | 2022-01-13 | 2024-09-24 | 北京歌锐科技有限公司 | Mixed reality system and marker registration method based on mixed reality |
| CN114533267A (en)* | 2022-02-18 | 2022-05-27 | 李静 | 2D image surgery positioning navigation system and method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202027702U (en)* | 2011-01-28 | 2011-11-09 | 李明 | Assisting positioning navigation and screw channel verification device for realizing vertebral column pedicle screw placement by aid of ultrasound |
| CN108784832A (en)* | 2017-04-26 | 2018-11-13 | 中国科学院沈阳自动化研究所 | A kind of minimally invasive spine surgical augmented reality air navigation aid |
| CN109512514A (en)* | 2018-12-07 | 2019-03-26 | 陈玩君 | A kind of mixed reality orthopaedics minimally invasive operation navigating system and application method |
| US11574446B2 (en)* | 2019-08-30 | 2023-02-07 | National Central University | Digital image reality aligning kit and method applied to mixed reality system for surgical navigation |
| CN113081268A (en)* | 2021-03-31 | 2021-07-09 | 上海电气集团股份有限公司 | AR and IoT based surgical guidance system |
- 2021
- 2021-08-18CNCN202110949777.2Apatent/CN113648057B/enactiveActive
Also Published As
| Publication number | Publication date |
|---|---|
| CN113648057A (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11957419B2 (en) | External fixator deformity correction systems and methods | |
| CN113648057B (en) | Surgical navigation system and method for corresponding virtual space three-dimensional model to real space position | |
| Ma et al. | Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study | |
| Lu et al. | Applications of mixed reality technology in orthopedics surgery: a pilot study | |
| US11730389B2 (en) | Method and system for supporting medical interventions | |
| Ma et al. | Visualization, registration and tracking techniques for augmented reality guided surgery: a review | |
| EP3903724A1 (en) | Calibration method and device for dental implantology navigation surgeries, and tracking method and device for dental implantology navigation surgeries | |
| JP2017056212A (en) | Surgical operation support system, surgical operation support device, surgical operation support method, surgical operation support program and information processor | |
| JP7242646B2 (en) | surgical instrument mounted display system | |
| KR101650620B1 (en) | Laser guidance devices and method for surgery, recording medium for performing the method | |
| CN106408652B (en) | Screw path positioning method and system for acetabulum anterior column forward screw | |
| KR20190058190A (en) | Spine surgical navigation system and method using augmented reality technology | |
| Meng et al. | Feasibility of the application of mixed reality in mandible reconstruction with fibula flap: a cadaveric specimen study | |
| CN113648056B (en) | Mixed reality display equipment and operation navigation system | |
| CN110638525A (en) | Surgical navigation method and system integrating augmented reality | |
| EP3948801A1 (en) | Orthopedic fixation control and visualization | |
| US20240362811A1 (en) | Patterned incision foil and method for determining a geometry of an anatomical surface | |
| KR20230013716A (en) | C-arm based surgical navigating system using the fracture treatment plan of image type | |
| CN117679173B (en) | Robot-assisted navigation spinal surgery system and surgical equipment | |
| KR20240146875A (en) | Surgical navigation procedure and method for total shoulder arthroplasty using XR technology | |
| TWI671763B (en) | An augmented reality instrument for accurately positioning pedical screw in minimally invasive spine surgery | |
| WO2023161858A1 (en) | Surgical imaging and display system, and related methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |