Disclosure of Invention
In order to solve the technical problems, the invention provides a method, a device and computer equipment for generating an arch line image, which concretely comprises the following steps:
In a first aspect, an embodiment of the present application provides a method for generating an arch line image, where the method includes:
converting the three-dimensional tooth model into a binary image, and acquiring a single-layer target image in a DICOM format corresponding to the three-dimensional tooth model;
Determining the centroid coordinates corresponding to each tooth based on the binary image;
Converting each centroid coordinate into a corresponding target coordinate in the target image;
Fitting coordinate points corresponding to the target coordinates into a target dental arch line through an interpolation spline curve algorithm;
And fusing the target dental archwire and the target image into an archwire image.
According to one embodiment of the disclosure, the generating step of the three-dimensional tooth model includes:
acquiring an original cone beam CT image corresponding to an oral cavity, wherein the original cone beam CT image comprises a plurality of layers of first sub-images;
Preprocessing an original cone beam CT image through a two-dimensional convolution network to obtain a basic oral cavity image, wherein the number of filters in the two-dimensional convolution network is the same as the number of layers of the first sub-image;
Inputting the basic oral cavity image into a pre-trained tooth extraction model to obtain the three-dimensional tooth model.
According to one embodiment of the present disclosure, the step of obtaining a single-layer target image in DICOM format corresponding to the three-dimensional dental model includes:
selecting a basic cone beam CT image corresponding to the three-dimensional tooth model in the original cone beam CT image, wherein the basic cone beam CT image comprises a plurality of layers of second sub-images;
And merging the second sub-images of each layer into a target image in a DICOM format by using a maximum intensity projection method.
According to one embodiment of the present disclosure, the step of determining the centroid coordinates corresponding to each tooth based on the binary image includes:
Determining a set of pixel points corresponding to each tooth based on the binary image;
According to the formulaAndCalculating a centroid coordinate corresponding to each tooth, wherein cx is an x-axis coordinate of the centroid, cy is a y-axis coordinate of the centroid, T00 is a sum of pixel values corresponding to all pixels in the pixel point set, T01 is a product of an abscissa in the pixel point set and each corresponding pixel value, and T10 is a product of an ordinate in the pixel point set and each corresponding pixel value.
According to a specific embodiment of the disclosure, the step of determining the set of pixels corresponding to each tooth based on the binary image includes:
Determining a closed pixel contour line based on gray values of all pixel points in the binary image;
judging whether the area of a first closed region corresponding to each closed pixel contour line is larger than a threshold value or not;
If the area of any one of the first closed areas is larger than a threshold value, corroding the closed areas to obtain at least two second closed areas with areas smaller than the threshold value, and determining the number of the first closed areas and the second closed areas with areas smaller than the threshold value as the number of teeth;
if the area of each first closed region is smaller than or equal to the threshold value, determining the number of the first closed regions as the number of teeth;
and determining all pixel points in the first closed area and the second closed area with the areas smaller than the threshold value as a pixel point set of the corresponding teeth.
According to one embodiment of the disclosure, the step of converting each centroid coordinate into a corresponding target coordinate in the target image includes:
Acquiring a pixel interval value corresponding to the target image;
and summing based on the pixel interval value, the centroid coordinates corresponding to each tooth and the origin coordinates preset by the target image to obtain the target coordinates corresponding to each centroid coordinate in the target image.
In a second aspect, an embodiment of the present application provides an apparatus for generating an archwire image, including:
The image acquisition module is used for converting the three-dimensional tooth model into a binary image and acquiring a single-layer target image in a DICOM format corresponding to the three-dimensional tooth model;
the coordinate calculation module is used for determining the centroid coordinate corresponding to each tooth based on the binary image;
the coordinate conversion module is used for converting each centroid coordinate into a corresponding target coordinate in the target image;
The curve fitting module is used for fitting coordinate points corresponding to the target coordinates into a target dental arch line through an interpolation spline curve algorithm;
And the image fusion module is used for fusing the target dental archwire and the target image into an archwire image.
According to one embodiment of the disclosure, the coordinate conversion module is specifically applied to:
Acquiring a pixel interval value corresponding to the target image;
and summing based on the pixel interval value, the centroid coordinates corresponding to each tooth and the origin coordinates preset by the target image to obtain the target coordinates corresponding to each centroid coordinate in the target image.
In a third aspect, an embodiment of the present application provides a computer device, where the computer device includes a processor and a memory, where the memory stores a computer program, where the computer program implements the method for generating an archwire image according to any one of the embodiments of the first aspect when the computer program is executed on the processor.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium storing a computer program, which when executed on a processor implements a method for generating an archwire image according to any one of the embodiments of the first aspect.
Compared with the prior art, the application has the following beneficial effects:
the method comprises the steps of converting a three-dimensional tooth model into a binary image, obtaining a single-layer target image in a DICOM format corresponding to the three-dimensional tooth model, determining a centroid coordinate corresponding to each tooth based on the binary image, converting each centroid coordinate into a target coordinate corresponding to each target coordinate in the target image, fitting coordinate points corresponding to each target coordinate into a target dental arch line through an interpolation spline curve algorithm, and fusing the target dental arch line and the target image into a dental arch line image. The center of mass coordinates corresponding to each tooth are determined through the binary image, and the center of mass coordinates are converted into corresponding target coordinates in a single-layer target image in the DICOM format, so that a target dental archwire can be accurately and quickly generated to generate a dental archwire image.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.
The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present invention.
The terms "comprises," "comprising," "including," or any other variation thereof, are intended to cover a specific feature, number, step, operation, element, component, or combination of the foregoing, which may be used in various embodiments of the present invention, and are not intended to first exclude the presence of or increase the likelihood of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing.
Furthermore, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and should not be construed as indicating or implying relative importance.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the invention belong. The terms (such as those defined in commonly used dictionaries) will be interpreted as having a meaning that is the same as the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein in connection with the various embodiments of the invention.
Some embodiments of the present application are described in detail below with reference to the accompanying drawings. The embodiments described below and features of the embodiments may be combined with each other without conflict.
The method for generating the dental archwire image provided by the application can be divided into the following steps:
1. dividing lower teeth data and/or upper teeth data in the oral cavity image by using a dividing network;
2. edge detection is carried out on the segmented lower tooth data and/or upper tooth data, and the barycenter coordinate of each tooth is found;
3. And inputting coordinate points corresponding to all centroid coordinates to a target image in a DICOM format, and fitting the coordinate points on the target image into a spline curve to obtain a target dental archwire so as to generate a dental archwire image.
Referring to fig. 1, fig. 1 is a flowchart of a method for generating an archwire image according to an embodiment of the present application. As shown in fig. 1, the method for generating an arch line image mainly includes:
step S101, converting the three-dimensional tooth model into a binary image, and acquiring a single-layer target image in a DICOM format corresponding to the three-dimensional tooth model.
Typically, when performing oral medical diagnosis, CBCT images corresponding to the oral cavity of a patient are acquired. Such CBCT images include not only upper or lower tooth images required for generating an archwire, but also other non-tooth images that are not required. Thus, the segmentation network may be used to segment the lower teeth data in the oral cavity image. Wherein the lower teeth data and/or upper teeth data may be presented by different types of images or models, etc., including but not limited to three-dimensional tooth models. In practice, the three-dimensional tooth model may be rotated through different angles to obtain different views suitable for subsequent analysis.
Referring to fig. 2, fig. 2 is a schematic diagram of a three-dimensional tooth model related to a method for generating an archwire image according to an embodiment of the present application. The stereolithography file format (Stereo lithography, STL for short) is a three-dimensional graphic file format. The STL file is composed of a plurality of triangle patch definitions, each triangle patch definition including three-dimensional coordinates of each of the triangle's points and normal vectors of the triangle patches. In the implementation process, the three-dimensional tooth model can be displayed by adopting other three-dimensional graphic file formats according to the actual use requirements and specific application scenes of the user, and the implementation process is not particularly limited.
Because the barycenter coordinate of each tooth needs to be searched, the calculation amount is greatly increased by directly searching on the three-dimensional image corresponding to the three-dimensional tooth model, and finally the generated dental archwire is often on the same plane. Therefore, the three-dimensional tooth model can be converted into a binary image and then the centroid searching can be carried out. Referring to fig. 3, fig. 3 is a schematic diagram of a binary image related to a method for generating an archwire image according to an embodiment of the present application.
The digital imaging transmission protocol for medical treatment (DIGITAL IMAGING AND Communi cations IN MEDICINE, DICOM) format is an international standard for medical images and related information (ISO 12052). DICOM is widely used in radiology, cardiovascular imaging, and radiodiagnosis and diagnosis equipment, including but not limited to X-ray, CT, nuclear magnetic resonance, ultrasound, etc., and is becoming more and more widely used in other medical fields such as ophthalmology and dentistry. The field of oral medicine generally employs the storage of medical images of patients in DICOM file format. This format contains relevant information about the patient, such as name, gender, age, and other image related information, such as device information to capture and generate images, etc. A doctor can conveniently read the medical image and diagnose the problems found in the medical image by using the DICOM reader.
Referring to fig. 4, fig. 4 is a schematic diagram of the composition of a CBCT image related to a method for generating an archwire image according to an embodiment of the present application. In the process of acquiring the CBCT image, a detector with higher sensitivity is used for continuous section scanning around a certain part of a human body, so that the CBCT image obtained after scanning is a multi-layer image. A three-dimensional image may be formed by stacking multiple CBCT images in the Z-axis. At this time, the CBCT image of each layer can be saved in DICOM format.
The generating step of the three-dimensional tooth model comprises the following steps:
acquiring an original cone beam CT image corresponding to an oral cavity, wherein the original cone beam CT image comprises a plurality of layers of first sub-images;
Preprocessing an original cone beam CT image through a two-dimensional convolution network to obtain a basic oral cavity image, wherein the number of filters in the two-dimensional convolution network is the same as the number of layers of the first sub-image;
Inputting the basic oral cavity image into a pre-trained tooth extraction model to obtain the three-dimensional tooth model.
In specific implementation, the lower teeth or the upper teeth can be segmented by using a segmentation network to obtain a three-dimensional tooth model. Because CBCT image data is three-dimensional tomographic sequence data, the size is 640 x 400, the data size is too large, if a three-dimensional convolution network is directly used, overflow of GPU video memory may be caused, and direct conversion of CBCT image into JPG image may lack spatial information, resulting in inaccurate data corresponding to tooth segmentation.
Therefore, in order to ensure that the spatial information is not lost, and the situation that the video memory overflows due to overlarge data volume is avoided, the original cone beam CT image can be preprocessed by using a two-dimensional convolution network, so that a basic oral cavity image is obtained. Specifically, when a two-dimensional convolution network is used, filters, i.e., filters, in the two-dimensional convolution network are used as the number of layers of the CBCT image. For example, the input is fixed to Conv2D (640,640,400). Wherein 640 is tensor, i.e. tensor is consistent with the size of a single CBCT, and 400 is filters, which are consistent with the number of CBCT layers. Through the preprocessing, the three-dimensional data is treated as two-dimensional data, and meanwhile, the technical effect of not losing a large amount of space information can be achieved. In deconvolution output, filters are also output as the number of layers of the two-dimensional matrix. The rest is basically consistent with Unet network structure, and will not be described in detail here.
The training process of the tooth extraction model is described by way of an example of the lower teeth. More than any number of CBCT data, the mandibular teeth may be manually segmented out as label data using Minics software. The technical problem of a relatively large amount of data can be solved by using the data generator. Specifically, batchsize of the data generator may be set to 4, that is, when every 4 batches of data are sent to training, the next batch of data is generated, so as to replace the problem that the memory overflows due to storing the training data into the memory. Compared with the direct use of the three-dimensional convolution network, the method can achieve the technical effect of shortening half of the single reasoning time.
And step S102, determining the centroid coordinates corresponding to each tooth based on the binary image.
Referring to fig. 3, each tooth in the binary image is separated, and belongs to multiple targets in image processing, so that it is necessary to find all the teeth in the binary image. Each tooth is a pile of pixel sets, and the target number can be determined by a method of longitudinally searching pixel contours. The pixel value is a value given by a computer when the image is digitized, and represents the average luminance information or the average reflection density information of a certain small square or pixel point. Although conventional sobel and canny edge detection algorithms can detect edge pixels of an image based on differences in gray values of the image to achieve determination of boundaries, these methods do not take a contour as a whole, and the degree of edge detection is not accurate enough. In the binary image, the contour of each tooth corresponds to a series of pixels. The pixel contour describes a continuous sequence of points with the same pixel values, and the edge pixels can be combined into a pixel contour curve to describe the edge information of the image. Therefore, the closed pixel contour line can be determined based on the gray value of each pixel point in the binary image, and generally, the number of closed pixel contour lines is the number of teeth.
For the special case of tooth arrangement deformity, more than two teeth may be arranged in a close or spatially superimposed relationship, resulting in the closed pixel contours corresponding to two or more teeth being identified as one. To reduce the recognition error in the above case, the accuracy of the subsequent generation of the dental archwire can be improved by:
judging whether the area of a first closed region corresponding to each closed pixel contour line is larger than a threshold value or not;
If the area of any one of the first closed areas is larger than a threshold value, corroding the closed areas to obtain at least two second closed areas with areas smaller than the threshold value, and determining the number of the first closed areas and the second closed areas with areas smaller than the threshold value as the number of teeth;
if the area of each first closed region is smaller than or equal to the threshold value, determining the number of the first closed regions as the number of teeth;
and determining all pixel points in the first closed area and the second closed area with the areas smaller than the threshold value as a pixel point set of the corresponding teeth.
Corrosion is an image processing means in morphology, and structural elements with a certain shape are used for measuring and extracting corresponding shapes in images so as to achieve the purposes of analysis and identification. Referring to fig. 5, fig. 5 is a schematic diagram of an erosion process related to a method for generating an archwire image according to an embodiment of the present application. The overlapping part in the binary image is subjected to corrosion treatment, and the highlight area or the white part in the image is reduced and thinned to obtain a plurality of non-overlapping pixel sets, so that tooth identification is realized. In the implementation, the threshold value and the area value of the second closed region may be customized according to a historical experience value, an actual use requirement of a user, or a specific application scenario, which is not further limited herein.
After the pixel point set corresponding to each tooth is determined based on the binary image, the formula is adoptedAndAnd calculating the corresponding barycenter coordinates of each tooth. Wherein cx is the x-axis coordinate of the centroid, cy is the y-axis coordinate of the centroid, T00 is the sum of the pixel values corresponding to all the pixels in the pixel set, T01 is the product of the abscissa and the corresponding pixel values in the pixel set, and T10 is the product of the ordinate and the corresponding pixel values in the pixel set.
In specific implementation, a centroid is found for the pixel set corresponding to each tooth. Since the cross section of the tooth is not a regular pattern, the centroid and the centroid are not necessarily equal, and since each shape is formed by pixel points in the image, the centroid is a weighted average of pixel values corresponding to the pixel points forming all the shapes.
Step S103, converting each centroid coordinate into a corresponding target coordinate in the target image.
As can be seen from step S101, the dental archwire is generally required to be displayed on a DICOM reader, and therefore, the centroid coordinates of each tooth detected in the JPG format binary image need to be converted into corresponding target coordinates in the target image, i.e., DICOM coordinates.
Specifically, the step of obtaining a single-layer target image in DICOM format corresponding to the three-dimensional tooth model includes:
selecting a basic cone beam CT image corresponding to the three-dimensional tooth model in the original cone beam CT image, wherein the basic cone beam CT image comprises a plurality of layers of second sub-images;
And merging the second sub-images of each layer into a target image in a DICOM format by using a maximum intensity projection method.
Referring to fig. 4, since the CBCT image itself has multiple layers, in order to be able to be easily viewed and diagnosed by a doctor, the CBCT image including the lower arch or the professional upper arch can be fused into a single CBCT image by the maximum intensity projection method. And then converting the barycenter coordinates in the binary image in the JPG format to a single CBCT image. Therefore, the influence of the Z axis of DICOM data on the coordinate conversion process is avoided, the calculated amount in the coordinate conversion process is reduced, and the generating speed of dental archwires is improved.
The step of converting each centroid coordinate into a corresponding target coordinate in the target image comprises the following steps:
Acquiring a pixel interval value corresponding to the target image;
and summing based on the pixel interval value, the centroid coordinates corresponding to each tooth and the origin coordinates preset by the target image to obtain the target coordinates corresponding to each centroid coordinate in the target image.
The main component of the DICOM file is a data set, and the DICOM file is composed of DICOM data elements which are sequentially arranged according to a specified sequence. For DICOM files, explicit transmission is typically used, with data elements arranged in Tag order from small to large. DICOM tags can be largely classified into Patient, study, series and Image four types of Image information or related parameter information, each DICOM Tag being clear determined by a combination of two hexadecimal numbers. Therefore, in the implementation, the corresponding pixel interval value can be obtained through the Tag information in the DICOM file.
And step S104, fitting coordinate points corresponding to the target coordinates into a target dental arch line through an interpolation spline curve algorithm.
After each centroid coordinate is converted into a corresponding target coordinate in a DICOM-format target image, fitting is carried out on coordinate points corresponding to each target coordinate through an interpolation spline curve algorithm, and a target dental arch line accurately predicted is obtained.
In the implementation, other fitting algorithms except the interpolation spline algorithm can be selected for fitting according to the actual use requirement of a user and the specific application scene, and the method is not limited further.
Step S105, fusing the target dental archwire and the target image into an archwire image.
Referring to fig. 6, fig. 6 is a schematic diagram of an arch line image related to a method for generating an arch line image according to an embodiment of the present application. After the target dental archwire is obtained by fitting, the target dental archwire and the target image can be fused to obtain a dental archwire image comprising the target dental archwire, so that a doctor can observe the oral cavity state of a patient more conveniently according to the dental archwire image, and the diagnosis efficiency is improved.
The method for generating the dental arch line image determines the closed pixel contour line based on the gray value of each pixel point in the binary image, and if the area of a first closed region corresponding to any closed pixel contour line is larger than a threshold value, the closed region is corroded to obtain accurate tooth quantity and further obtain the centroid coordinates of each tooth. By converting each centroid coordinate to a corresponding target coordinate in a single-layer target image in DICOM format, a target archwire can be accurately and quickly generated to generate an archwire image.
Corresponding to the above method embodiment, referring to fig. 7, the present invention further provides an apparatus 700 for generating an arch line image, where the apparatus 700 for generating an arch line image includes:
the image acquisition module 701 is configured to convert a three-dimensional tooth model into a binary image, and acquire a single-layer target image in DICOM format corresponding to the three-dimensional tooth model;
the coordinate calculation module 702 is configured to determine centroid coordinates corresponding to each tooth based on the binary image;
a coordinate conversion module 703, configured to convert each centroid coordinate into a corresponding target coordinate in the target image;
A curve fitting module 704, configured to fit coordinate points corresponding to each of the target coordinates to a target dental archwire through an interpolation spline curve algorithm;
an image fusion module 705, configured to fuse the target dental archwire and the target image into an archwire image;
In specific implementation, the coordinate conversion module 703 is specifically applied to:
Acquiring a pixel interval value corresponding to the target image;
and summing based on the pixel interval value, the centroid coordinates corresponding to each tooth and the origin coordinates preset by the target image to obtain the target coordinates corresponding to each centroid coordinate in the target image.
In addition, a computer device is provided, the computer device comprises a processor and a memory, the memory stores a computer program, and the computer program realizes the generating method of the dental arch line image when being executed on the processor.
Further, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program implementing the above mentioned method of generating an archwire image when executed on a processor.
The specific implementation process of the device, the computer device and the computer readable storage medium for generating the dental arch line image required by the accessory provided by the application can be referred to the specific implementation process of the method for generating the dental arch line image provided by the above embodiment, and will not be described in detail herein.
The application provides a generating device, computer equipment and a computer readable storage medium of an arch line image required by accessories, which are used for determining closed pixel contours based on gray values of pixel points in a binary image, and if the area of a first closed region corresponding to any closed pixel contour is larger than a threshold value, corroding the closed region to obtain accurate tooth quantity and further obtain mass center coordinates of each tooth. By converting each centroid coordinate to a corresponding target coordinate in a single-layer target image in DICOM format, a target archwire can be accurately and quickly generated to generate an archwire image.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are merely illustrative, for example, of the flow diagrams and block diagrams in the figures, which illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules or units in various embodiments of the invention may be integrated together to form a single part, or the modules may exist alone, or two or more modules may be integrated to form a single part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a smart phone, a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention.