CN110269715B - Root canal monitoring method and system based on AR - Google Patents

Abstract

The invention discloses a root canal monitoring method and system based on AR, the whole scheme comprises S1, scanning oral cavity by CT, generating root canal three-dimensional perspective image; s2, carrying out segmentation and three-dimensional reconstruction on the three-dimensional perspective images of the tooth body and the root canal, and generating a root canal perspective model; s3, scanning the three-dimensional appearance model of the tooth by the three-dimensional raster scanner; s4, carrying out point cloud editing and surface reconstruction on the three-dimensional appearance model, and generating a target object model; s5, simultaneously importing the obtained root canal perspective model and the target object model into AR modeling software, and exporting a target object identification and root canal display data packet; s6, acquiring a target object recognition and root canal display data packet and adding a gesture and voice interaction module by interaction design software, and generating an APP; s7, the mobile terminal obtains the APP and displays the root canal and carries out interactive detection. The root canal monitoring method and system based on AR provided by the invention can enable root canal detection to be separated from the traditional mode depending on doctor experience, and can also improve detection precision.

Description

Root canal monitoring method and system based on AR

Technical Field

The invention relates to the technical field of medical detection, in particular to a root canal monitoring method and system based on AR.

Background

Root canal therapy is a common oral medical operation and mainly comprises the steps of root canal enlargement, irrigation, filling and the like. Because of individual differences, the shapes and sizes of root canals are different, and the traditional root canal length measuring instrument can not reflect the difference in the aspect at all, which often causes the operation effect of doctors with different experiences to be greatly different. Being able to accurately reflect the position and shape of the root canal is a necessary stage to get rid of experience dependence and to perform accurate medical treatment.

Patent 201811557205.4 discloses a scanning and shooting device for practical training evaluation of a transparent root canal tooth model, which mainly uses two cameras to obtain shape data of root canals in the whole tooth model required by root canal treatment evaluation, and is used for teaching of a false transparent tooth model. Patent 201810509261.4 discloses a device for endodontic treatment in department of stomatology, and patent 201910041509.3 discloses an endodontic instrument lamp holder with optical fibers, which mainly provide more root canal local conditions through common illumination. Although the conventional literature reports that the importance of root canal information is found and efforts are made, the position and shape of the root canal cannot be accurately reflected in real time in clinical practice. Although CT can clearly photograph the inside of a root canal, it cannot be used for real-time surgery because X-rays are harmful to the human body. To date, no implementation has emerged that can display the actual root canal shape and precise location in real time.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a root canal monitoring method and system based on AR, which can enable root canal detection to be separated from the traditional mode depending on doctor experience and can also improve detection precision.

In order to achieve the purpose, the invention provides the following technical scheme: an AR-based root canal monitoring method comprising the steps of:

s1, scanning the oral cavity by using CT to generate a three-dimensional perspective image of the root canal;

s2, carrying out segmentation and three-dimensional reconstruction on the three-dimensional perspective images of the tooth body and the root canal, and generating a root canal perspective model;

s3, scanning the three-dimensional appearance model of the tooth by the three-dimensional raster scanner to generate a target object model;

s4, performing point cloud editing and surface reconstruction on the three-dimensional appearance model, and generating a tooth model, namely a target object model;

s5, simultaneously importing the obtained root canal perspective model and the tooth model into AR modeling software, and exporting a target object identification and root canal display data packet;

s6, acquiring a target object recognition and root canal display data packet and adding a gesture and voice interaction module by interaction design software, and generating an APP;

s7, the mobile terminal obtains the APP and displays the root canal and carries out interactive detection.

A root canal monitoring system based on AR is characterized by comprising a CT used for obtaining a root canal three-dimensional perspective image, a three-dimensional raster scanner used for scanning teeth in an oral cavity to obtain a tooth three-dimensional appearance model, and an AR modeling platform used for carrying out AR recombination on the root canal three-dimensional perspective image and the tooth three-dimensional appearance model; the AR modeling platform is in communication connection with both the CT and the three-dimensional raster scanner so as to transmit data to each other; and the AR modeling platform outputs an AR model for human observation.

As a further improvement of the system, the system further comprises a three-dimensional segmentation and reconstruction platform for segmenting and three-dimensionally reconstructing the tooth body and the root canal image during the root canal three-dimensional perspective image, wherein the three-dimensional segmentation and reconstruction platform is in communication connection with the CT to acquire the root canal three-dimensional perspective image generated by the CT scanning; and the three-dimensional segmentation and reconstruction platform is used for segmenting the tooth body and the root canal image, performing three-dimensional reconstruction, generating a root canal perspective model and transmitting the root canal perspective model to the AR modeling platform.

As a further improvement of the system, the system also comprises a three-dimensional point cloud processing platform used for performing point cloud editing and surface reconstruction on the tooth three-dimensional appearance model, and the three-dimensional point cloud processing platform is in communication connection with the three-dimensional grating scanner so as to obtain the tooth three-dimensional appearance model; and the three-dimensional point cloud processing platform carries out point cloud editing and surface reconstruction on the tooth three-dimensional appearance model and outputs the tooth model to the AR modeling platform.

As a further improvement of the system, the system also comprises an interactive design platform for editing the APP, wherein the interactive design platform is in communication connection with the AR modeling platform; the AR modeling platform receives positions of the root canal perspective model and the tooth model after receiving the root canal perspective model and the tooth model, outputs target object identification and a root canal display data packet to the interactive design platform, and the interactive design platform edits the data packet and adds one or more of a gesture interaction mode and a voice interaction mode and then derives an APP to the mobile terminal.

As the further improvement of this system, remove the end and have microphone, nine acceleration sensor and touch display at least, touch display is used for showing man-machine interaction under APP and the gesture interaction mode, and nine acceleration sensor are used for the gesture induction control APP through removing the end and show content, and the microphone is used for man-machine interaction under the voice interaction mode.

As the further improvement of this system, the removal end still has the camera, the camera is used for the discernment of tooth target, catches current patient's tooth target image in the testing process to look for the root canal display data that the tooth model of the target that corresponds in the APP corresponds according to current tooth image, when the target discernment that the tooth model corresponds matches successfully, the root canal display data that corresponds shows on the APP, and show with the locking of tooth target position.

The invention has the advantages that,

1. the root canal operation is more accurate;

2. reducing the dependency of the root canal surgery on the experience of the doctor;

3. displaying the shape and the trend of the actual root canal inside the tooth in real time;

4. the virtual three-dimensional root canal image is highly registered with the root canal inside the tooth and is locked at the fixed position of the tooth, so that observation and measurement at different angles are facilitated.

Drawings

FIG. 1 is a flow chart of the steps of the present invention;

FIG. 2 is a system framework diagram of the present invention.

Reference numerals: 1. CT; 2. a three-dimensional raster scanner; 3. a three-dimensional segmentation and recombination platform; 4. a three-dimensional point cloud processing platform; 5. an AR modeling platform; 6. an interactive design platform; 7. and a mobile terminal.

Detailed Description

The invention will be further described in detail with reference to the following examples, which are given in the accompanying drawings.

Referring to fig. 1-2, the AR-based root canal monitoring method according to the present embodiment includes the following steps:

s1, scanning the oral cavity by using CT to generate a three-dimensional perspective image of the root canal;

s2, carrying out segmentation and three-dimensional reconstruction on the three-dimensional perspective images of the tooth body and the root canal, and generating a root canal perspective model;

s3, scanning the three-dimensional appearance model of the tooth by the three-dimensional raster scanner to generate a target object model;

s4, performing point cloud editing and surface reconstruction on the three-dimensional appearance model, and generating a tooth model, namely a target object model;

s5, simultaneously importing the obtained root canal perspective model and the tooth model into AR modeling software, and exporting a target object identification and root canal display data packet;

s6, acquiring a target object recognition and root canal display data packet and adding a gesture and voice interaction module by interaction design software, and generating an APP;

s7, the mobile terminal obtains the APP and displays the root canal and carries out interactive detection.

Based on the steps S1-S4, a root canal three-dimensional perspective image and a tooth three-dimensional appearance model of a patient tooth are obtained through CT, three-dimensional segmentation reconstruction software, a three-dimensional raster scanner and three-dimensional point cloud processing software, and a root canal perspective model and a tooth model in an STL format are generated after three-dimensional reconstruction, so that data support is provided for the subsequent steps; after the two models are obtained, the method enters step S5, position registration is performed on the root canal perspective model and the tooth model through AR modeling software, and a target object identification and root canal display data packet corresponding to the current tooth model and the root canal perspective model is generated, where the data packet may be stored in a subsequently generated APP or in an external server; in step S6, interactive design software acquires this data package, and edit this data package, add gesture interaction module and voice interaction module, generate corresponding APP, this APP can interact with doctor or patient, control the data package in the APP through gesture interaction or voice interaction and show the gesture of content and adjustment display thing, and then accomplish real-time adjustment viewing angle, can assist the doctor to know the whole condition of root canal in real time according to the viewing angle of difference, be convenient for the doctor accomplish accurate treatment, different with traditional method of relying on experience to judge, this scheme can make the operation of most doctors' S root canal treatment of starting to hand fast, break away from the dependency to experience.

Based on the detection method, the root canal monitoring system based on AR is provided as follows, which comprises a CT1 for acquiring a root canal three-dimensional perspective image, a three-dimensional raster scanner 2 for scanning teeth in an oral cavity to acquire a tooth three-dimensional appearance model, and anAR modeling platform 5 for carrying out AR recombination on the root canal three-dimensional perspective image and the tooth three-dimensional appearance model; theAR modeling platform 5 is in communication connection with the CT1 and the three-dimensional raster scanner 2 so as to mutually transmit data; theAR modeling platform 5 outputs an AR model for human observation.

Come to scan patient's oral cavity through CT1 scanning, acquire patient's root canal three-dimensional perspective image, afterwards, scan the tooth in the patient oral cavity through three-dimensional grating scanner 2, acquire the three-dimensional appearance model of tooth,AR modeling platform 5 can carry out AR reorganization preparation according to the root canal three-dimensional perspective image and the three-dimensional appearance model of tooth that acquire, generate AR data image, doctor and disease can directly observe current patient's root canal condition through this AR data image, be convenient for more clear analysis and communication.

As an optimization for facilitating AR reconstruction, the system further comprises a three-dimensional segmentation and reconstruction platform for segmenting and three-dimensionally reconstructing the tooth body and root canal images during the three-dimensional perspective image of the root canal, wherein the three-dimensional segmentation and reconstruction platform is in communication connection with CT1 to obtain the three-dimensional perspective image of the root canal generated by CT1 scanning; the three-dimensional segmentation and reconstruction platform segments the tooth body and the root canal image, reconstructs the root canal image in a three-dimensional mode, generates a root canal perspective model and transmits the root canal perspective model to theAR modeling platform 5.

When theAR modeling platform 5 carries out AR recombination on the three-dimensional perspective image of the root canal, the three-dimensional perspective image of the root canal obtained by scanning the CT1 can be segmented and recombined through the three-dimensional segmentation reconstruction platform in advance, the tooth body part in the three-dimensional perspective image can be segmented and removed, the image of the root canal part is left, and the perspective model in the STL format is generated through recombination, so that theAR modeling platform 5 can directly obtain the perspective model in the format for AR recombination, artificial AR recombination editing according to the three-dimensional perspective image of the root canal is not needed, and the AR modeling efficiency is greatly improved.

Further improving optimization of AR modeling efficiency, the system also comprises a three-dimensional pointcloud processing platform 4 used for performing point cloud editing and surface reconstruction on the tooth three-dimensional appearance model, and the three-dimensional pointcloud processing platform 4 is in communication connection with the three-dimensional grating scanner 2 to obtain the tooth three-dimensional appearance model; the three-dimensional pointcloud processing platform 4 carries out point cloud editing and surface reconstruction on the tooth three-dimensional appearance model and outputs the tooth model to theAR modeling platform 5.

The three-dimensional pointcloud processing platform 4 receives the tooth three-dimensional appearance model obtained after scanning by the grating scanner, carries out point cloud editing and surface reconstruction on the tooth three-dimensional appearance model, and generates the tooth model in the STL format in the same way.

The three-dimensional segmentation and reconstruction platform is the existing three-dimensional segmentation and reconstruction software, the three-dimensional pointcloud processing platform 4 is the existing three-dimensional point cloud processing software, and the operation and the output are carried out on a computer platform which is in communication connection with the CT1 and the three-dimensional raster scanner 2. Of course, the manner in which the computer communicates with the CT1 and the three-dimensional raster scanner 2 can be transferred via a data medium, such as a usb disk, hard disk, or the like. TheAR modeling platform 5 is existing AR modeling software.

As a specific embodiment of the improvement, the system further comprises aninteractive design platform 6 for editing the APP, wherein theinteractive design platform 6 is in communication connection with theAR modeling platform 5;AR modeling platform 5 receives the position of registration root canal perspective model and tooth model behind root canal perspective model and the tooth model to output target object discernment and root canal display data package forinteractive design platform 6, export APP after one or more in interactive mode of gesture and the voice interaction mode are edited tointeractive design platform 6 and giveremoval end 7.

Throughinteractive design platform 6 to root canal perspective model and tooth model design of packing, correspond multiunit root canal perspective model and tooth model and integrate and save, make into APP with the data after this integration to add gesture interaction mode and voice interaction mode in this APP, doctor or disease can be interacted with APP through two kinds of interaction modes, position, gesture, size and transparency etc. with the root canal perspective model that changes the demonstration and tooth model, be convenient for user interaction and look over.

To this embodiment, more specifically, themobile terminal 7 at least has a microphone, a nine-axis acceleration sensor and a touch display screen, the touch display screen is used for displaying human-computer interaction in the APP and gesture interaction mode, the nine-axis acceleration sensor is used for controlling APP display content through gesture induction of themobile terminal 7, and the microphone is used for human-computer interaction in the voice interaction mode.

Removeend 7 and can adopt current cell-phone, panel computer, notebook computer etc. gather user's speech information through the microphone, and the voice interaction mode in the cooperation APP carries out speech control, and wherein nine acceleration sensor and touch display screen can be used for showing the state of root canal perspective model and tooth model, also can cooperate the demonstration state of gesture interaction mode control root canal perspective model and tooth model simultaneously.

Further optimization, removeend 7 and still have the camera, the camera is used for tooth target object discernment, catches current patient's tooth target object image in the testing process to look for the root canal display data that the tooth model of the interior corresponding target object of APP corresponds according to current tooth image, when the target object discernment that the tooth model corresponds matches successfully, the root canal display data that corresponds shows on APP, and show with tooth target object position locking.

This camera is used for catching the tooth profile of tooth of current disease, through with tooth model matching in this tooth profile and the APP or in the server the root canal display data that the tooth that can call the treatment of current disease corresponds fast to APP locks the tooth target object that catches in with this root canal display data and the camera, according to the gesture of current tooth target object, shows on patient's tooth target object after the matching, and the image that root canal display data shows on APP changes along with the change of the visual angle that the camera caught. Doctor is when looking over patient's root canal state this moment, can directly come from server or APP inside current patient's root canal state of calling through APP, from this, doctor only need through APP and camera cooperation look over patient's tooth can, the disease arrives corresponding department by oneself and carries out CT1 scanning and three-dimensional grating scanning, after the scanning is accomplished, wait for doctor to look over with APP cooperation camera, the doctor can be more accurate clear understanding user's root canal state, need not judge through the experience, make the analysis result more accurate, supplementary doctor judges the state of an illness, let the operation more accurate.

The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.

Claims (4)

1. A root canal monitoring system based on AR comprises a CT (1) for obtaining a root canal three-dimensional perspective image, a three-dimensional raster scanner (2) for scanning teeth in an oral cavity to obtain a tooth three-dimensional appearance model, and an AR modeling platform (5) for carrying out AR recombination on the root canal three-dimensional perspective image and the tooth three-dimensional appearance model; the AR modeling platform (5) is in communication connection with the CT (1) and the three-dimensional raster scanner (2) to transmit data mutually; the AR modeling platform (5) outputs an AR model for human observation;

the three-dimensional segmentation and reconstruction platform is used for segmenting and reconstructing a tooth body and a root canal image in the process of performing the root canal three-dimensional perspective image, and is in communication connection with the CT (1) so as to obtain the root canal three-dimensional perspective image generated by the CT (1) scanning; the three-dimensional segmentation and reconstruction platform is used for segmenting and three-dimensionally reconstructing the tooth body and the root canal image to generate a root canal perspective model and transmitting the root canal perspective model to the AR modeling platform (5);

the system also comprises an interactive design platform (6) for editing the APP, wherein the interactive design platform (6) is in communication connection with the AR modeling platform (5); the AR modeling platform (5) receives the positions of the root canal perspective model and the tooth model after the root canal perspective model and the tooth model are registered, outputs a target object identification and root canal display data packet to the interactive design platform (6), and the interactive design platform (6) edits the data packet, adds one or more of a gesture interaction mode and a voice interaction mode and then exports an APP to a mobile terminal (7);

remove end (7) and still have the camera, the camera is used for tooth target object discernment, catches current patient's tooth target object image in the testing process to look for the root canal display data that the tooth model of the target object that corresponds in the APP corresponds according to current tooth image, when the target object discernment that the tooth model corresponds matches successfully, the root canal display data that corresponds shows on the APP, and shows with tooth target object position locking.

2. The AR-based root canal monitoring system according to claim 1, further comprising a three-dimensional point cloud processing platform (4) for point cloud editing and surface reconstruction of the tooth three-dimensional appearance model, the three-dimensional point cloud processing platform (4) being communicatively connected to the three-dimensional raster scanner (2) to obtain the tooth three-dimensional appearance model; the three-dimensional point cloud processing platform (4) carries out point cloud editing and surface reconstruction on the tooth three-dimensional appearance model and outputs the tooth model to the AR modeling platform (5).

3. The AR-based root canal monitoring system according to claim 2, wherein the mobile terminal (7) has at least a microphone, a nine-axis acceleration sensor and a touch display screen, the touch display screen is used for displaying the APP and the human-machine interaction in the gesture interaction mode, the nine-axis acceleration sensor is used for controlling the APP display content through the gesture sensing of the mobile terminal (7), and the microphone is used for the human-machine interaction in the voice interaction mode.

4. An AR-based root canal monitoring method applied to the AR-based root canal monitoring system according to claim 2 or 3, comprising the steps of:

s1, scanning the oral cavity by using CT to generate a three-dimensional perspective image of the root canal;

s2, carrying out segmentation and three-dimensional reconstruction on the three-dimensional perspective images of the tooth body and the root canal, and generating a root canal perspective model;

s3, scanning the three-dimensional appearance model of the tooth by the three-dimensional raster scanner to generate a target object model;

s4, performing point cloud editing and surface reconstruction on the three-dimensional appearance model, and generating a tooth model, namely a target object model;

s5, simultaneously importing the obtained root canal perspective model and the tooth model into AR modeling software, and exporting a target object identification and root canal display data packet;

s6, acquiring a target object recognition and root canal display data packet and adding a gesture and voice interaction module by interaction design software, and generating an APP;

s7, the mobile terminal obtains the APP and displays the root canal and carries out interactive detection.

Images