CN112386332B

Movatterモバイル変換

Info

Publication number: CN112386332B
Application number: CN202011211927.1A
Authority: CN
Inventors: 黄建红; 方烈
Original assignee: Zhejiang Saihong Zhongzhi Network Technology Co ltd
Current assignee: Zhejiang Saihong Zhongzhi Network Technology Co ltd
Priority date: 2020-11-03
Filing date: 2020-11-03
Publication date: 2022-04-19
Anticipated expiration: 2040-11-03
Also published as: CN112386332A

Abstract

The invention provides an AR display method of lung argon-helium surgical path data, which comprises the following steps: an initial three-dimensional model of the patient's chest is generated from the chest DICOM digital imaging and communications in medicine file of the patient's chest. Marking the mark of the position to be treated. In the initial three-dimensional model, the position mark to be treated is used as a scattering origin, the puncture length of a surgical cutter is used as a scattering radius, the diameter of the surgical cutter is used as the ray width, a plurality of scattering paths of a plurality of surgical cutters reaching the external surface are obtained, and one or more paths without junction points are obtained. Thereby obtaining puncture path data for one or more surgical tools. According to the invention, based on the alignment image, the three-dimensional model and the simulated operation path of the patient are superposed on the current human body image, so that the operation path which is high in safety and suitable for being implemented by an actual operation cutter is obtained. The wound area is reduced, and the safety of the operation is ensured. The invention also provides a system for generating the path data of the argon-helium cryosurgical operation for the lung.

Description

AR/MR display method and system for lung argon-helium scalpel operation path data

Technical Field

The invention relates to the field of adjuvant medicine. Is applied to minimally invasive surgery. The invention particularly relates to an AR/MR display method and system for lung argon-helium surgical path data.

Background

In minimally invasive treatment of pulmonary nodules or tumors, a physician performs surgery based on the location of the lesion as determined by the patient prior to surgery. The surgical path is determined by diagnosing the location of the lesion prior to surgery. During the operation, the lesion position is treated according to the operation path in a manner of matching with the endoscope imaging. Because the pulmonary nodule or tumor focus is adjacent to the main trachea, the artery and the heart area, and the focus imaging of the patient in the early stage can change along with the body position of the patient, the accuracy of the operation path has errors, and the operation accuracy is required to be high, so the operation risk is high, the large-area wound is easy to be caused, and the treatment effect is poor.

Disclosure of Invention

The invention aims to provide an AR/MR display method of lung argon-helium scalpel operation path data, which is used for assisting minimally invasive operation treatment of lung nodules or tumors by superposing a three-dimensional model and a simulated operation path of a patient on current human body imaging in an AR/MR display mode based on an alignment image. The wound surface area is reduced, and the wound surface is effectively avoided, so that the operation safety and reliability are high.

The invention aims to provide an AR/MR display system of pulmonary argon-helium-knife surgical path data, which assists in minimally invasive surgical treatment of pulmonary nodules or tumors. The wound surface area is reduced, and the wound surface is effectively avoided, so that the operation safety and reliability are high.

The invention provides an AR/MR display method of the pulmonary argon-helium-scalpel surgical path data, wherein the AR/MR display method of the pulmonary argon-helium-scalpel surgical path data is realized through AR/MR glasses capable of being worn on an operator.

The AR/MR display method of the lung argon-helium scalpel operation path data comprises the following steps:

and step S101, acquiring a current image of the current patient in the lying position through AR/MR glasses. The current image includes an alignment image. To bit mapImageIs a flat image on a setting display surface. Currently the patient can lie on a lying surface. The lying surface can be parallel to the display surface.

Step S102, the AR/MR glasses acquire the position information of the contraposition image from the current image.

In step S103, the AR/MR glasses acquire a set of puncture path data. The puncture path data includes: a reference position data, a start position data at the current patient's epidermis, an end position data at the lesion, and a path trajectory data. The reference position data corresponds to the start position data. The reference position data corresponds to the end position data.

And step S104, the AR/MR glasses update the datum position data according to the currently acquired alignment image. And loading the initial position data according to the corresponding relation between the reference position data and the initial position data. And loading the initial position data according to the corresponding relation between the reference position data and the end position data.

In step S105, the AR/MR glasses display the start position data, the end position data, and the path trajectory data.

In still another embodiment of the present invention, the puncture path data further includes: three-dimensional model data of the chest of the patient. The baseline position data, the starting position data, and the end position data are present within a three-dimensional model constructed from three-dimensional model data of the patient's chest. The three-dimensional model of the chest of the patient is constructed by the three-dimensional model data, and the three-dimensional model has a first construction surface which can be parallel to the back direction of the patient. The first building surface is parallel to the setting display surface.

In still another embodiment of the present invention, step S105 further includes: the AR/MR glasses display three-dimensional model data of the chest of the patient. The start position data and the end position data can be displayed superimposed within a three-dimensional model constructed from three-dimensional model data of the patient's chest.

In another embodiment of the present invention, step S104 further includes:

in step S201, the AR/MR glasses obtain the current tilt angle of the currently set display surface according to the display size and position of the first identification point image, the second identification point image, and the third identification point image, and the distance and width of the first connection line, the second connection line, and the third connection line.

Step S202, acquiring the inclination angle of the first building surface according to the current inclination angle.

And step S203, loading the three-dimensional model data of the chest of the patient according to the inclination angle of the first building surface.

And step S204, loading reference position data on a model generated by the three-dimensional model data of the chest of the patient according to the position of the currently acquired alignment image. And loading the initial position data and the final position data according to the reference position data.

In another embodiment of the present invention, step S103 further includes:

in step S301, an initial three-dimensional model of the patient 'S chest is generated from the chest DICOM digital imaging and communications file of the patient' S chest. Chest DICOM digital medical imaging and communications files are obtained by scanning a patient's chest with a radiological or imaging device. The chest radiation or imaging data includes a lung region of the patient. The outer face of the initial three-dimensional model corresponds to a chest body surface of the patient. The initial three-dimensional model comprises: double lung model, skeleton model and thoracic organ model. The thoracic organ model comprises a plurality of tracheas, artery tissues, vein tissue models and organ models positioned between two lungs in the thoracic cavity.

Step S302, marking a position mark to be treated in the initial three-dimensional model according to the three-dimensional position of the set marked focus.

Step S303, in the initial three-dimensional model, the position mark to be treated is used as a scattering origin, the puncture length of one surgical knife is used as a scattering radius, and the diameter of the surgical knife is used as a ray width to obtain a plurality of scattering paths of a plurality of surgical knives to the external surface.

Step S304, acquiring the intersection points of the multiple scattering paths, the bone model and the thoracic organ model on the single scattering path. And removing the scattering paths with the junction points from the plurality of scattering paths to obtain one or more paths without the junction points.

Step S305, acquiring puncture path data of one or more surgical tools according to one or more non-junction paths.

In yet another embodiment of the present invention, the chest DICOM digital imaging and communications in medicine file has a topogram image information in step S301. And acquiring reference position data according to the image information of the positioning sheet. After the locating piece is worn on the chest of the patient, the chest DICOM medical digital imaging and communication file is obtained by scanning the chest of the patient through a radiation or imaging device. The locating plate is located in the middle of the two lung parts.

In still another embodiment of the present invention, step S301 further includes:

step S401, reading the chest DICOM medical digital imaging and communication file, and acquiring DICOM TAG file header information and corresponding data set information. And analyzing Series TAG unit data in the DICOM TAG header information to acquire position information and image sequence information of a plurality of image information arranged in one axial direction. The data set information is parsed to obtain a series of images associated with the image sequence information.

Step S402, acquiring a transverse plane, a coronal plane and a sagittal plane according to the position information of the plurality of image information and the image sequence information corresponding series images. The coronal plane is perpendicular and intersects the sagittal plane. The transverse planes are perpendicular and intersect the coronal plane perpendicular to the sagittal plane.

In step S403, an initial three-dimensional model is generated from the transverse plane, coronal plane, and sagittal plane.

In still another embodiment of the present invention, step S402 includes:

in step S501, a series of images corresponding to the position information and the image sequence information of the plurality of pieces of image information are acquired.

Step S502, acquiring a thoracic organ series image from the series image according to the gray scale corresponding to the thoracic organ in the CT image.

And acquiring a skeleton series image from the series images according to the gray scale corresponding to the skeleton in the CT image.

And acquiring a series of images of the lung from the series of images according to the gray level corresponding to the lung in the CT image.

In step S503, one organ transverse plane, one organ coronal plane, and one organ sagittal plane are generated by superimposing the thoracic organ series images and the corresponding position information and image series information in one set axial direction. The coronal plane of the viscera is perpendicular to and intersects the sagittal plane of the viscera. The organ transverse section plane is vertical and is intersected with the organ coronal plane and is vertical to the organ sagittal plane.

And according to the lung series images and the corresponding position information and image sequence information thereof, generating a lung transverse plane, a lung coronal plane and a lung sagittal plane by setting axial superposition. The coronal plane of the lung is perpendicular and intersects the sagittal plane of the lung. The transverse plane of the lung is vertical and is intersected with the coronal plane of the lung and is vertical to the sagittal plane of the lung.

And generating a bone transverse plane, a bone coronal plane and a bone sagittal plane by setting axial superposition according to the bone series images and the corresponding position information and image sequence information thereof. The coronal plane of the bone is perpendicular and intersects the sagittal plane of the bone. The bone transverse plane is vertical and is intersected with the bone coronal plane and is vertical to the bone sagittal plane.

In step S504, an initial organ three-dimensional model is generated from the organ transverse plane, the organ coronal plane, and the organ sagittal plane. And superposing the locating plate image into the initial organ three-dimensional model according to the datum position data and generating a locating plate mark in the initial organ three-dimensional model.

And generating an initial lung three-dimensional model according to the lung transverse plane, the lung coronal plane and the lung sagittal plane. And generating a topogram marker in the initial three-dimensional lung model according to the datum position data.

And generating an initial skeleton three-dimensional model according to the skeleton transverse plane, the skeleton coronal plane and the skeleton sagittal plane. Topogram markers are generated in the initial three-dimensional model of the bone from the reference position data.

And S505, according to the positioning sheet mark, superposing the initial organ three-dimensional model, the lung three-dimensional model and the initial skeleton three-dimensional model to obtain an initial three-dimensional model.

In a second aspect of the present invention, an AR/MR display system for pulmonary argon-helium-scalpel surgical path data is provided, wherein the AR/MR display system for pulmonary argon-helium-scalpel surgical path data is implemented by an AR/MR glasses capable of being worn by an operator.

The AR/MR display system of the pulmonary argon-helium surgical path data comprises:

an AR/MR eyewear comprising: an acquisition unit, a processing unit and a display unit.

The acquisition unit is configured to acquire a current image of the current patient in the lying position through AR/MR glasses. The current image includes an alignment image. To bit mapImageIs a flat image on a setting display surface. Currently the patient can lie on a lying surface. Lying downThe lying surface can be parallel to the display surface.

The processing unit is configured to acquire position information of the alignment image from the current image.

A set of puncture path data is acquired. The puncture path data includes: a reference location data, a starting location data at the current patient's epidermis, and an ending location data at the lesion. The reference position data has a relative positional correspondence with the start position data and the end position data.

And loading reference position data according to the currently acquired alignment image. And loading the initial position data and the final position data according to the reference position data.

The display unit is configured to display the start position data and the end position data with the AR/MR glasses.

The characteristics, technical features, advantages and implementation of the AR/MR display method and system for pulmonary argon-helium scalpel surgical path data will be further described in a clear and understandable manner with reference to the accompanying drawings.

Drawings

FIG. 1 is a flow diagram illustrating a method for displaying the AR/MR pulmonary argon-helium-surgical path data in accordance with one embodiment of the present invention.

FIG. 2 is a schematic diagram illustrating a pulmonary argon-helium surgical puncture path in one embodiment of the present invention.

FIG. 3 is a schematic diagram illustrating a pulmonary argon helium surgical puncture path in another embodiment of the present invention. .

FIG. 4 is a schematic diagram for illustrating different positions of the bit map according to an embodiment of the present invention.

FIG. 5 is a schematic flow chart diagram illustrating a method for displaying the AR/MR pulmonary argon-helium-scalpel surgical path data in accordance with another embodiment of the present invention.

FIG. 6 is a schematic diagram illustrating the generation of a pulmonary argon-helium surgical pathway in one embodiment of the present invention.

FIG. 7 is a schematic diagram of the components of AR/MR glasses in an AR/MR display system for pulmonary argon-helium-gas surgical path data in accordance with one embodiment of the present invention.

Detailed Description

In order to more clearly understand the technical features, objects and effects of the present invention, embodiments of the present invention will now be described with reference to the accompanying drawings, in which the same reference numerals indicate the same or structurally similar but functionally identical elements.

"exemplary" means "serving as an example, instance, or illustration" herein, and any illustration, embodiment, or steps described as "exemplary" herein should not be construed as a preferred or advantageous alternative. For the sake of simplicity, the drawings only schematically show the parts relevant to the present exemplary embodiment, and they do not represent the actual structure and the true scale of the product.

As shown in FIG. 1, the AR/MR display method of the pulmonary argon-helium-scalpel surgical path data comprises the following steps:

step S101, acquiring a current image of a current patient in a lying position.

In this step, as shown in fig. 2, the image capturing device of the AR/MR glasses captures the current image of the patient 90 lying in the lying position. The current image includes an alignment image 10. The alignment image 10 is a planar image on onesetting display surface 20. The patient 90 can now lie on a lyingsurface 91. Lyingsurface 91 can be parallel to the display surface.

Step S102, position information is acquired.

In this step, the AR/MR glasses acquire the position information of the alignment image 10 from the current image, that is, acquire the three-dimensional coordinate information of the alignment image 10, and the three-dimensional coordinate information can be determined by referring to the set three-dimensional coordinate system information. The position is convenient for data processing, and any dimension surface of the three-dimensional coordinate system is set to be parallel to the lyingsurface 91.

In step S103, puncture path data is acquired.

In this step, as shown in fig. 2, the AR/MR glasses acquire a set of puncture path data. The puncture path data includes: areference position data 31, astart position data 32 located in the epidermis of thecurrent patient 90, anend position data 33 located at the lesion, and apath trajectory data 34. Thereference position data 31 corresponds to thestart position data 32. Thereference position data 31 corresponds to theend position data 33.

And step S104, updating the datum position data according to the currently acquired alignment image.

In this step, as shown in fig. 2, the AR/MR glasses update thereference position data 31 based on the currently acquired alignment image 10. Thestart position data 32 is loaded based on the correspondence relationship between thereference position data 31 and thestart position data 32. Thestart position data 32 is loaded based on the correspondence relationship between thereference position data 31 and theend position data 33.

In step S105, the AR/MR glasses display the path trajectory data.

In this step, as shown in fig. 2, the AR/MR glasses display startposition data 32,end position data 33, andpath trajectory data 34.

When the invention is implemented, a doctor wears AR/MR glasses with operation auxiliary software to perform an operation. The planned operation path information is imported before the operation. The doctor scans the locating plate on the body of the patient through the camera in the AR/MR glasses and carries out registration through the locating plate and the position of the locating plate in the three-dimensional model of the patient. After the registration is finished, based on the alignment image, a three-dimensional model and a simulated operation path of the patient are superposed on the current human body image, the position of the operation puncture opening is determined and superposed on the body of the patient through AR/MR glasses, and information such as puncture angle and depth is displayed in the glasses, so that a doctor is helped to quickly determine the position, angle and position of the operation puncture opening.

The invention provides the accuracy and the safety of the operation by imaging the operation path through the AR/MR glasses.

In another embodiment of the present invention, aligning image 10 includes: as shown in fig. 2, three firstidentification point images 41, one secondidentification point image 42, and one thirdidentification point image 43 are oppositely arranged. Between the firstidentification point image 41 and the secondidentification point image 42 is a first connectingline 44. Between the secondidentification point image 42 and the thirdidentification point image 43 is a second connectingline 45. Between the thirdidentification point image 43 and the firstidentification point image 41 is a third connectingline 46.

In still another embodiment of the present invention, as shown in fig. 3, the puncture path data further includes: three-dimensional model 51 data of the patient's chest. Thedatum position data 31, thestart position data 32 and theend position data 33 are present within a three-dimensional model built from the three-dimensional model 51 data of the patient's chest. The three-dimensional model of the patient'schest 51 data is constructed to have a first build plane that can be parallel to the patient's back. The first building surface is parallel to the settingdisplay surface 20.

In still another embodiment of the present invention, step S105 further includes: the AR/MR glasses display the three-dimensional model 51 data of the patient's chest. Thestart position data 32 and theend position data 33 can be displayed superimposed within the three-dimensional model constructed from the three-dimensional model 51 of the patient's chest.

In still another embodiment of the present invention, as shown in fig. 5, step S104 further includes:

step S201, a current tilt angle of the currently set display surface is obtained.

In this step, the AR/MR glasses obtain the current tilt angle of the currently setdisplay surface 20 according to the display size and position of the firstidentification point image 41, the secondidentification point image 42, and the thirdidentification point image 43, and the distance and width of thefirst connection line 44, thesecond connection line 45, and thethird connection line 46.

As shown in fig. 4, when the doctor is positioned at a different angle with respect to the patient, the registered image 10 is changed from the middle solid line a image to the broken line B image. The areas and the degree of the parallelograms of the firstidentification point image 41, the secondidentification point image 42, and the thirdidentification point image 43 in the a image are changed accordingly, and the distances, widths, and angles of the first connectinglines 44, the second connectinglines 45, and the third connectinglines 46 are also changed accordingly. So that the correspondence of the above-mentioned figure to the relative angle of the patient and the doctor can be obtained. Thereby obtaining the current tilt angle of the currentsetting display surface 20.

Step S202, the inclination angle of the first building surface is obtained.

In this step, the inclination angle of the first building surface is obtained according to the current inclination angle. The first building surface is a surface that can be parallel to the lyingsurface 91.

Step S203, three-dimensional model data of the chest of the patient is loaded.

In this step, data of the three-dimensional model 51 of the patient's chest is loaded or generated according to the inclination angle of the first building surface.

Step S204, loading the initial position data and the final position data.

In this step, thereference position data 31 is loaded on the model generated from the three-dimensional model 51 of the patient's chest based on the position of the currently acquired registration image 10. Thestart position data 32 and theend position data 33 are loaded based on thereference position data 31.

In another embodiment of the present invention, step S103 further includes:

As shown in fig. 6, the surgical knife is an argon-helium knife that can be used for pulmonary surgery. The argon-helium knife has a knife head and a knife handle connected with an argon-helium instrument. The argon-helium knife has various sizes, and the length and the diameter of the knife head can be selected from various specifications, wherein the knife head is the effective puncture length. The diameter of the tool tip results in

scatter paths

71, 72, 73, 74 and 75 for the width of the rays as the width of the scatter path.

As shown in fig. 6, the

scatter paths

71, 73 having their intersection points with the bone model are obtained from the

scatter paths

71, 72, 73, 74 and 75. The intersection point of thescatter paths 71 is 61. The convergence point of thescatter path 73 is 62. The intersection of thescatter path 75 and the thoracic organ model is 63. So that the

scatter paths

72 and 74 are junction-free paths.

In yet another embodiment of the present invention, the chest DICOM digital imaging and communications in medicine file has a topogram image information in step S301.Reference position data 31 is acquired from the spacer image information. After the locating piece is worn on the chest of the patient, the chest DICOM medical digital imaging and communication file is obtained by scanning the chest of the patient through a radiation or imaging device. The locating plate is located in the middle of the two lung parts.

In still another embodiment of the present invention, step S402 includes:

In step S504, an initial organ three-dimensional model is generated from the organ transverse plane, the organ coronal plane, and the organ sagittal plane. The localizer image is superimposed on the initial organ three-dimensional model from thereference position data 31 and a localizer mark is generated in the initial organ three-dimensional model. The alignment graphic 10 is shown on the alignment sheet.

And generating an initial lung three-dimensional model according to the lung transverse plane, the lung coronal plane and the lung sagittal plane. Topogram markers are generated in the initial three-dimensional model of the lung from thefiducial position data 31.

And generating an initial skeleton three-dimensional model according to the skeleton transverse plane, the skeleton coronal plane and the skeleton sagittal plane. Topogram markers are generated in the initial three-dimensional model of the bone from thefiducial position data 31.

As shown in FIG. 7, the AR/MR display system for the pulmonary argon-helium-scalpel surgical path data includes:

an AR/MR eyewear comprising: anacquisition unit 201, aprocessing unit 202 and a display unit 203.

Theacquisition unit 201 is configured to acquire a current image of thecurrent patient 90 in the lying position through AR/MR glasses. The current image includes an alignment image 10. The alignment image 10 is a planar image on onesetting display surface 20. The patient 90 can now lie on a lyingsurface 91. Lyingsurface 91 can be parallel to the display surface.

Theprocessing unit 202 is configured to obtain position information of the alignment image 10 from the current image.

A set of puncture path data is acquired. The puncture path data includes: areference position data 31, astart position data 32 located in the epidermis of thecurrent patient 90 and anend position data 33 located at the lesion. Thereference position data 31 has a relative positional correspondence relationship with thestart position data 32 and theend position data 33.

Thereference position data 31 is loaded based on the currently acquired registration image 10. Thestart position data 32 and theend position data 33 are loaded based on thereference position data 31.

The display unit 203 is configured to display thestart position data 32 and theend position data 33 with AR/MR glasses.

It should be understood that although the present description is described in terms of various embodiments, not every embodiment includes only a single embodiment, and such description is for clarity purposes only, and those skilled in the art will recognize that the embodiments described herein as a whole may be suitably combined to form other embodiments as will be appreciated by those skilled in the art.

The above-listed detailed description is only a specific description of a possible embodiment of the present invention, and they are not intended to limit the scope of the present invention, and equivalent embodiments or modifications made without departing from the technical spirit of the present invention should be included in the scope of the present invention.

Claims

1. An AR/MR display system of the pulmonary argon-helium-scalpel surgical path data, wherein the AR/MR display system of the pulmonary argon-helium-scalpel surgical path data is realized by AR/MR glasses capable of being worn on an operator;

the AR/MR display system of the pulmonary argon-helium-scalpel surgical path data comprises:

an AR/MR eyewear comprising:

an acquisition unit configured to acquire a current image of a current patient in a lying position through the AR/MR glasses; the current image comprises an alignment image; the alignment image is a plane image on a set display surface; the current patient can lie on a lying surface; the lying surface can be parallel to the display surface;

a processing unit configured to obtain position information of the alignment image from the current image;

acquiring a group of puncture path data; the puncture path data includes: a reference position data, a start position data at the epidermis of the patient at present, and an end position data at the lesion; the reference position data has a corresponding relation of relative positions with the start position data and the end position data;

loading the datum position data according to the currently acquired alignment image; loading the initial position data and the final position data according to the reference position data;

a display unit configured to display the start position data and the end position data by the AR/MR glasses;

the processing unit is further configured to: generating an initial three-dimensional model of the patient's chest from the chest DICOM medical digital imaging and communications file of the patient's chest; scanning a patient's chest with a radiological or imaging device to obtain the chest DICOM digital imaging and communications in medicine file; chest radiological or imaging data includes a lung region of the patient; the outer face of the initial three-dimensional model corresponds to a chest body surface of the patient; the initial three-dimensional model comprises: double lung model, skeleton model and thoracic organ model; the thoracic organ model comprises a plurality of tracheas, artery tissues, vein tissue models and organ models which are positioned between two lungs in the thoracic cavity;

marking a position mark to be treated in the initial three-dimensional model according to a set marked focus three-dimensional position;

in the initial three-dimensional model, the position mark to be treated is used as a scattering origin, the puncture length of a surgical cutter is used as a scattering radius, and the diameter of the surgical cutter is used as a ray width to obtain a plurality of scattering paths of a plurality of surgical cutters to the external surface;

acquiring intersection points of the multiple scattering paths, the bone model and the thoracic organ model on the multiple scattering paths; removing scattering paths with junction points from the multiple scattering paths to obtain one or multiple paths without junction points; and acquiring puncture path data of the one or more surgical tools according to the one or more non-intersection point paths.