CN111445575A

Movatterモバイル変換

Info

Publication number: CN111445575A
Application number: CN202010251323.3A
Authority: CN
Inventors: 常云翔; 李丙生
Original assignee: Shenyang Advanced Medical Equipment Technology Incubation Center Co ltd
Current assignee: Neusoft Medical Systems Co Ltd
Priority date: 2020-04-01
Filing date: 2020-04-01
Publication date: 2020-07-24
Anticipated expiration: 2040-04-01
Also published as: CN111445575B

Abstract

The specification provides an image reconstruction method, an image reconstruction device, electronic equipment and a storage medium of a Wiriss ring, wherein the method comprises the following steps: obtaining a CTA image of the head and neck of the subject; identifying bounding boxes of a williams' circle in a CTA image and sampling a sub-image containing the bounding boxes from the CTA image; inputting the sub-images into a key point positioning model, wherein the key point positioning model is obtained by training a neural network by adopting a CTA (computed tomography angiography) image sample, and the CTA image sample is marked with key point position information positioned on a Weilisi ring; intercepting a display area of the Wicress ring from the sub-image according to a plurality of key point positions on the Wicress ring output by the key point positioning model; and reconstructing the image of the display area, adjusting the display area to a target observation angle, and outputting the adjusted display area. The automatic and accurate identification and display of the display area of the Weilisi ring are realized.

Description

Image reconstruction method and device of Wirisi ring, electronic device and storage medium

Technical Field

The present disclosure relates to the field of medical image processing technologies, and in particular, to a method and an apparatus for reconstructing an image of a william, an electronic device, and a storage medium.

Background

The Willis ring area is an important structure of an intracranial blood vessel and is also a high-haired area of intracranial arterial vascular diseases, and doctors need to observe the position, the structure, the trend and other characteristics of the blood vessel in the Willis ring area according to the three-dimensional drawing result of local CTA three-dimensional volume data so as to realize the diagnosis of the intracranial arterial vascular diseases.

Since the wilis ring area is completely blocked by the high-density skull, in order to achieve the purpose of observing blood vessels in the wilis ring in a three-dimensional rendering result in clinical practice, doctors usually need to manually cut the wilis ring area from blood vessels segmented from a CTA image, and adjust an observation angle, and read and print the blood vessels. In particular, the doctor often needs to determine a display area of the williams ring with a better viewing angle through multiple attempts, and the process is very tedious and time-consuming.

Disclosure of Invention

To overcome the problems in the related art, the present specification provides a method, an apparatus, an electronic device, and a storage medium for reconstructing an image of a williams ring.

According to a first aspect of embodiments herein, there is provided an image reconstruction method of a williams ring, the image reconstruction method including:

obtaining a CTA image of the head and neck of the subject;

identifying bounding boxes of a williams' circle in a CTA image and sampling a sub-image containing the bounding boxes from the CTA image;

inputting the sub-images into a key point positioning model, wherein the key point positioning model is obtained by training a neural network by adopting a CTA (computed tomography angiography) image sample, and the CTA image sample is marked with key point position information positioned on a Weilisi ring;

intercepting a display area of the Wicress ring from the sub-image according to a plurality of key point positions on the Wicress ring output by the key point positioning model;

and reconstructing the image of the display area, adjusting the display area to a target observation angle, and outputting the adjusted display area.

Optionally, identifying bounding boxes of the williams ring in the CTA image comprises:

identifying the region information of the artery blood vessel in the CTA image;

and inputting the CTA image and the region information into a target detection model to identify a bounding box region of the Weilisi ring in the CTA image, wherein the target detection model is obtained by training a neural network by using a training CTA image marked with the bounding box of the Weilisi ring.

Optionally, identifying information of a region in the CTA image where the arterial blood vessel is located includes:

inputting the CTA image into a tissue segmentation model, wherein the tissue segmentation model is obtained by training a neural network by using the CTA image marked with tissue part information;

segmenting the tissue part contained in the CTA image according to the tissue part to which each pixel point in the CTA image predicted by the tissue segmentation model belongs;

and identifying the region information of the artery blood vessel in the CTA image according to the segmentation result.

Optionally, intercepting a display area from the sub-image according to a plurality of keypoint locations output by the keypoint location model, including:

determining the opening orientation and the central position of the Weilisi ring according to the plurality of key point positions;

determining a central plane passing through the central position with the opening orientation as a normal vector;

and determining an area between two planes which are parallel to the central plane and are a preset distance away from the central plane in the sub-image as the display area.

Optionally, the number of the key point positions is 4;

determining an opening orientation of the willis ring from the plurality of keypoint locations, comprising:

selecting 3 first key points from 4 key point positions, and determining a first normal vector of a plane where the 3 first key points are located;

selecting 3 second key points from the 4 key point positions, and determining a second normal vector of a plane where the 3 second key points are located; wherein the 3 first keypoints are not identical to the 3 second keypoints;

determining the direction of the sum vector of the first normal vector and the second normal vector as the opening orientation.

Optionally, the image reconstruction method further includes:

acquiring display parameters, wherein the display parameters comprise imaging quantity and/or rotation angle and/or imaging mode;

performing image reconstruction on the display area, including:

and reconstructing an image of the display area based on the display parameters.

According to a second aspect of embodiments herein, there is provided an image reconstruction apparatus of a williams ring, the image reconstruction apparatus including:

an acquisition module for acquiring a CTA image of a head and neck of a subject;

a sampling module for identifying bounding boxes of a williams' ring in a CTA image and sampling a sub-image containing the bounding boxes from the CTA image;

the input module is used for inputting the sub-images into a key point positioning model, wherein the key point positioning model is obtained by training a neural network by adopting a CTA image sample, and the CTA image sample is marked with key point position information positioned on a Weilisi ring;

the intercepting module is used for intercepting a display area of the Weilisi ring from the sub-image according to a plurality of key point positions on the Weilisi ring output by the key point positioning model;

and the reconstruction module is used for reconstructing the image of the display area, adjusting the display area to a target observation angle and outputting the adjusted display area.

Optionally, in identifying bounding boxes of a willis's ring in a CTA image, the sampling module is to:

identifying the region information of the artery blood vessel in the CTA image;

Optionally, in identifying information of a region in the CTA image where the arterial vessel is located, the sampling module is configured to:

Optionally, the intercept module is specifically configured to:

Optionally, the number of the key point positions is 4;

when determining the opening orientation of the willis ring according to the plurality of keypoint locations, the truncation module is specifically configured to:

According to a third aspect of embodiments herein, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the image reconstruction method of a williams ring according to any one of the above when executing the computer program.

According to a fourth aspect of embodiments herein, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the image reconstruction method of williams' ring of any one of the above.

The technical scheme provided by the embodiment of the specification can have the following beneficial effects:

in the embodiment of the description, the bounding box region of the Willis ring in the CTA image is determined, then a plurality of key point positions on the Willis ring are determined from the bounding box of the Willis ring based on the key point positioning model, then the accurate display region of the Willis ring in the bounding box is determined according to the plurality of key point positions, and the display region is output after being adjusted to the target observation angle, so that the Willis ring in the CTA image can be automatically and accurately displayed at the target observation angle, and medical staff does not need to determine the target observation angle of the Willis ring through continuous attempts.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the specification.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present specification and together with the description, serve to explain the principles of the specification.

FIG. 1 is a schematic diagram of a Willis ring configuration shown in accordance with an exemplary embodiment of the present disclosure.

Fig. 2 is a flowchart illustrating an image reconstruction method of a williams ring according to an exemplary embodiment.

FIG. 3a is a slice view of coronal orientation resulting from resampling a CTA image as shown in this specification, according to an exemplary embodiment.

FIG. 3b is a sagittal view of a CTA image resampled with a slice view according to an exemplary embodiment.

Fig. 3c is a schematic diagram of a Willis ring with a viewing angle of 0 ° shown in accordance with an exemplary embodiment.

Fig. 3d is a schematic diagram of a Willis ring with a viewing angle of 30 ° shown in accordance with an exemplary embodiment.

Fig. 3e is a schematic diagram of a Willis's ring with a viewing angle of 120 ° shown in accordance with an exemplary embodiment.

Fig. 3f is a schematic diagram of a Willis ring with a viewing angle of 150 ° shown in accordance with an exemplary embodiment.

Fig. 3g is a schematic diagram illustrating a Willis's ring with a viewing angle of 180 ° according to an exemplary embodiment.

FIG. 4 is aflowchart illustrating step 201 of FIG. 2 according to an exemplary embodiment.

FIG. 5 is aflowchart illustrating step 203 of FIG. 2 according to an exemplary embodiment of the present description.

Fig. 6a is a schematic diagram illustrating the orientation of the openings of the Willis's rings in the bounding box according to an exemplary embodiment.

Figure 6b is a schematic diagram illustrating the determination of the orientation of the opening of a williams ring according to an exemplary embodiment.

FIG. 6c is a first schematic diagram illustrating the display area of the circle of Willis taken from the bounding box of the circle of Willis according to an exemplary embodiment of the present description.

FIG. 6d is a second schematic diagram illustrating the display area of the circle of Willis taken from the bounding box of the circle of Willis according to an exemplary embodiment of the present description.

Fig. 7 is a block diagram of an image reconstruction apparatus of a willis ring according to an exemplary embodiment shown in the present specification.

Fig. 8 is a schematic structural diagram of an electronic device shown in the present specification according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings identify the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the specification, as detailed in the appended claims.

The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present specification. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

Cerebrovascular diseases have the characteristics of high morbidity, mortality, disability rate, complications and the like, and are main diseases which harm the life and health of middle-aged and elderly people, wherein ischemic cerebrovascular diseases account for 75-85% of all cerebrovascular diseases, and ischemic cerebrovascular diseases caused by carotid artery stenosis or occlusion account for 20-30%.

The Willis (Willis) ring circulates as the primary collateral of the brain, can compensate blood flow to the narrow side through the anterior and posterior traffic arteries in a short time, improves blood flow supply, protects brain tissues from being damaged, and is an important structure of intracranial blood vessels.

Fig. 1 is a schematic diagram of a Willis loop structure shown in accordance with an exemplary embodiment of the present disclosure, the Willis loop including the following vessels:

anterior segment of the right and left Anterior Cerebral Arteries (ACA) (a 1);

anterior segment of the right and left Posterior Cerebral Arteries (PCA) (P1);

the ends of the Internal Carotid Arteries (ICAs);

anterior communicating artery (ACom);

posterior communication artery (PCom);

a well-developed circle of Willis has a very strong collateral circulation potential to ensure adequate blood redistribution once blood flow in the nearby vessels is reduced. However, the Willis loop is a high-incidence region of intracranial arterial vasculopathy, and has a high mutation rate, and only 27% -45.2% of individuals have complete Willis loop structures according to research. An autopsy study based on 1000 patients showed that the variation rate of the Willis loop structure was as high as 54.8%. Wherein the anterior blood ring variation rate is 23.4%, and the posterior circulation variation rate is 31.4%. Therefore, analytical studies on the Willis loop, whether they be basic studies or clinical applications, have been the focus of the cerebrovascular field.

Cta (computed Tomography imaging) is a non-invasive vascular imaging technique. The method comprises the steps of injecting an angiographic contrast medium into a subject, scanning the head and neck of the subject by using a CTA device, acquiring X-ray image data, and carrying out image reconstruction on the X-ray image data to obtain a CTA image to display vascular tissues of the head and neck of the subject for diagnosing vasculopathy.

Since the Willis ring region is completely shielded by the high-density skull, in order to observe blood vessels in the Willis ring in the three-dimensional rendering result in clinical practice, doctors usually need to manually cut the Willis ring region from the blood vessel image segmented from the CTA image, adjust the observation angle, and read and print the slice. In particular, a doctor often needs to determine a display area of the williams ring with a better observation angle through multiple attempts, and the manual planning process is tedious and time-consuming.

Based on the above situation, embodiments of the present invention provide an image reconstruction method for a Willis ring, which can accurately identify a display area of the Willis ring from a CTA image, and adjust the display area to an optimal target observation angle or to a target observation angle according to an observation requirement of a user.

The following provides a detailed description of examples of the present specification.

Fig. 2 is a flowchart illustrating an image reconstruction method of a williams ring according to an exemplary embodiment, including the following steps:

step 201, identify the bounding box of the willis's ring in the CTA image and sample a sub-image containing the bounding box from the CTA image.

The CTA image is image data acquired by scanning the head and neck of a subject with CTA equipment after injecting an angiographic contrast medium into the subject.

The bounding box, which is a region of interest (ROI) that a user wants to observe, is a minimum hexahedron or a minimum cuboid or a minimum sphere, etc. containing a williams ring, and the location area of the bounding box can be characterized by a center point coordinate (x, y, z) in combination with a length, width, height (or radius).

For ease of understanding, the CTA images shown in fig. 3a and 3b are taken as an example, fig. 3a is a coronal slice obtained by resampling the CTA images, and fig. 3b is a sagittal slice obtained by resampling the CTA images. If a rectangular parallelepiped bounding box is used, the region S in FIG. 3a₁The region S in FIG. 3b is the position of one side of the bounding box of the WirosisRing, which faces the coronal direction, as identified in the CTA image₂Is the position of the bounding box of the willis's ring identified in the CTA image toward one side in the sagittal direction. Instep 201, a bounding volume containing region is sampled from a CTA imageI.e., sub-image data corresponding to the region enclosed by the six sides of the bounding box sampled from the CTA image. It is understood that the CTA image is a three-dimensional image, and the sub-image sampled from the CTA image is also a three-dimensional image, and the sub-image data includes a williams ring.

Since the bounding box of the wisconsin ring contains, in addition to the arterial blood vessels containing the wisconsin ring, also other vessel information, for example, curved parts of the middle artery, which would interfere with the display of the wisconsin ring, would be displayed in the bounding box. Therefore, the following steps are required to be performed, the display area of the williams ring is cut out from the sub-image, and the interference of other blood vessel information is cut off.

Step 202, inputting the sub-images into the key point positioning model.

The key point positioning model is used for identifying a plurality of key point positions on the Weilisi ring in the sub-image. The plurality of keypoint locations may be a plurality of point locations distributed dispersedly on the willis ring selected on the willis ring, which can represent the profile information of the willis ring in the sub-image.

In order to enable the key point positioning model to identify the key point positions on the Weilisi ring in the input sub-image, a CTA image which contains the Weilisi ring and marks the key point positions on the Weilisi ring is used as a training sample, the neural network is trained, the training is stopped until the loss function of the trained neural network reaches a threshold value or the iteration times reaches the threshold value, and the trained neural network is used as the key point positioning model.

The deep convolutional neural network is the most common machine learning method with the best performance in the image vision field of segmentation, classification, positioning and the like at present. The deep convolutional neural network directly learns the characteristics of training data from the images in an end-to-end training mode, and the highly abstract characteristics can greatly improve the precision of the traditional method in the tasks of medical image segmentation, classification, positioning and the like. In this embodiment, the method may be, but is not limited to, a network architecture that employs a deep convolutional neural network as a key point localization model. Deep learning is a method of adjusting deep machine learning network parameters and features, usually in combination with neural networks.

In one embodiment, a point location in the portion of the anterior communicating artery that constitutes the willis loop, a point location in the portion of the left middle cerebral artery that constitutes the willis loop, a point location in the portion of the right middle cerebral artery that constitutes the willis loop, and a point location in the portion of the basilar artery that constitutes the willis loop may be selected as the keypoint of the willis loop, but not limited to.

Taking the willis's loop shown in fig. 3C as an example, point a in the figure is a keypoint selected on the anterior communicating artery, point B is a keypoint selected on the left middle cerebral artery, point C is a keypoint selected on the right middle cerebral artery, and point D is a keypoint selected on the basilar artery. For a training sample used for training a neural network, the key point positions of the williams ring in the CTA image need to be labeled in each CTA image with reference to the key point positions. The corresponding key point position on the Weilisi ring in the subimage can be identified based on the model obtained by training the neural network by the CTA image labeled with the key point position of the Weilisi ring.

And step 203, intercepting a display area from the sub-image according to a plurality of key point positions on the Weilisi ring output by the key point positioning model.

The sub-image containing the Weilisi ring is input into the key point positioning model, the key point positioning model identifies the sub-image and outputs a plurality of key point positions of the Weilisi ring in the sub-image, and each key point position can be represented by coordinates (x, y, z).

Since the plurality of key point positions represent the contour information of the willis ring in the sub-image, the exact position (display range) and the opening orientation of the willis ring in the sub-image can be determined according to the plurality of key point positions, and the willis ring can be intercepted from the sub-image to be used as the display area of the willis ring.

And 204, carrying out image reconstruction on the display area, adjusting the display area to a target observation angle, and outputting.

The target observation angle may be an angle defined by the user, or may be an optimal observation angle capable of displaying the most information of the williams ring. The so-called optimum viewing angle may be the angle at which the orientation of the willis ring opening is shown in the forward direction. The data corresponding to the display area may be output to an electronic device for display, or may be printed on a film.

Therefore, the bounding box region of the Willis ring in the CTA image is determined, then a plurality of key point positions on the Willis ring are determined from the bounding box of the Willis ring based on the key point positioning model, then the accurate display region of the Willis ring in the bounding box is determined according to the plurality of key point positions, and the display region is adjusted to the target observation angle and then output, so that medical staff can diagnose diseases of the blood vessel.

In another embodiment, the user may define the display parameters by himself, wherein the display parameters may include, but are not limited to, at least one of the following: number of images, viewing angle, imaging mode, etc.

If the user defines the display parameters, the user-defined display parameters are acquired beforestep 204 is executed, and instep 204, the image reconstruction is performed on the display area based on the user-defined display parameters.

For example, if the user defines the imaging mode as VR (virtual reality), image reconstruction is performed on the display area to generate a VR image of a williams ring; if the user defines the imaging mode as MIP (maximum intensity projection), the MIP image of the Weilisi ring is generated by carrying out image reconstruction on the display area.

If the user defines five observation angles of 0 °, 30 °, 120 °, 150 ° and 180 °, which are also the target observation angles, see fig. 3c, 3d, 3e, 3f and 3g,step 204 is executed to reconstruct 5 williams ring images with observation angles of 0 °, 30 °, 120 °, 150 ° and 180 °, respectively, so as to realize batch display of the williams ring images according to the observation angles required by the user. Wherein, the image with the observation angle of 0 ° shown in fig. 3e is the optimum observation angle of the williams ring, and 30 °, 120 °, 150 ° and 180 ° are the angles after rotating the williams ring on the basis of the optimum observation angle.

If the imaging number n is defined by the user, and n is a positive integer greater than 0, n Weilisi ring images with the best observation angle can be reconstructed, batch processing display of the Weilisi rings is realized, and more references can be provided for disease diagnosis of medical staff. Wherein, the angle of the Weilisi ring in each Weilisi ring image is also the target observation angle. The angle of the corresponding wisdom ring in the n wisdom ring images with the best viewing angle may be determined empirically or by fitting to historical data.

If the user simultaneously defines that the imaging quantity is 10 and the rotation angle is 360 degrees, 10 images of the Weilisi ring can be reconstructed, and the difference of the rotation angles of the Weilisi ring in two adjacent images is 36 degrees.

In the embodiment, the accurate display area of the Willis ring in the bounding box can be determined, the display area can be displayed in batch processing according to the display parameters defined by the user, a plurality of Willis ring images with different rotation angles are provided, richer reference is provided for medical diagnosis, and the Willis ring state judgment is facilitated.

The specific implementation ofstep 201 is further described below.

FIG. 4 is aflowchart illustrating step 201 of FIG. 2 according to an exemplary embodiment of the present description, including the steps of:

step 201-1, identify the region information where the artery blood vessel is located in the CTA image.

In one embodiment, since the subject is injected with an angiographic contrast agent, the artery in the CTA image is highlighted, and the region information of the artery in the CTA image can be identified in a conventional manner in step 201-1, for example, a threshold segmentation method is used in combination with morphological features to segment tissue portions such as skull, artery, and other tissues in the CTA image, and the region information of the highlighted artery is identified from the CTA image.

In another embodiment, the information of the region where the artery is located may be identified based on a machine learning method in step 201-1, specifically: and segmenting the tissue part contained in the CTA image based on the tissue segmentation model, and identifying the region information of the blood vessel in the CTA image according to the segmentation result.

In order to enable the tissue segmentation model to segment the tissue part in the CTA image, the CTA image marked with the tissue part information is used as a training sample to train the neural network, the training is stopped until the loss function of the training neural network reaches a threshold value or the iteration number reaches the threshold value, and the trained neural network is used as the tissue segmentation model.

The deep convolutional neural network is the most common machine learning method with the best performance in the image vision field of segmentation, classification, positioning and the like at present. The deep convolutional neural network directly learns the characteristics of training data from the images in an end-to-end training mode, and the highly abstract characteristics can greatly improve the precision of the traditional method in the tasks of medical image segmentation, classification, positioning and the like. In this embodiment, the deep convolutional neural network may be used as a network architecture of the tissue segmentation model. Deep learning is a method of adjusting deep machine learning network parameters and features, usually in combination with neural networks.

The CTA image to be segmented is input into a trained tissue segmentation model, and the tissue segmentation model can predict the probability (credibility) of each pixel point in the input CTA image being skull, artery vessel and other tissues. For each pixel in the CTA image, determining the tissue part corresponding to the maximum confidence coefficient in the prediction result as the tissue part of the pixel point, or determining the tissue part with the confidence coefficient larger than the confidence coefficient threshold value as the tissue organ corresponding to the pixel point, and identifying the tissue part to which the pixel point belongs, thereby realizing the segmentation of the tissue part of the CTA image and determining the region of the artery blood vessel in the CTA image.

Of course, before the CTA image is input into the tissue segmentation model, it needs to be preprocessed, for example, the CTA image is normalized to change its size to meet the input requirement of the tissue segmentation model, and/or the CTA image is subjected to gray scale change to make the gray scale value of each pixel point in the CTA image within a preset range, so as to meet the input requirement of the tissue segmentation model.

Step 201-2, inputting the CTA image and the region information of the artery blood vessel into the target detection model to identify the bounding box region of the Williams' ring in the CTA image.

The target detection model is used for identifying a bounding box of the Weilisi ring from the CTA image according to the regional information of the artery blood vessel, the bounding box is the minimum region containing the Weilisi ring, and the interference information of the skull in the CTA image is eliminated.

In order to enable the target detection model to accurately identify the bounding box of the Weilisi ring, a CTA image marked with information of the bounding box of the Weilisi ring is used as a training sample to train the neural network, the training is stopped until a loss function of the training neural network reaches a threshold value or the number of iterations reaches the threshold value, and the trained neural network is used as the target detection model.

The deep convolutional neural network is the most common machine learning method with the best performance in the image vision field of segmentation, classification, positioning and the like at present. The deep convolutional neural network directly learns the characteristics of training data from the images in an end-to-end training mode, and the highly abstract characteristics can greatly improve the precision of the traditional method in the tasks of medical image segmentation, classification, positioning and the like. In this embodiment, the present invention may be, but is not limited to, a network architecture that employs a deep convolutional neural network as a target detection model. Deep learning is a method of adjusting deep machine learning network parameters and features, usually in combination with neural networks.

Therefore, the subimage containing the bounding box can be accurately sampled from the CTA image based on machine learning, the interference of high-density skull on Willis ring display is eliminated, and a foundation is laid for intercepting an accurate Willis ring display area from the bounding box subsequently.

The specific implementation ofstep 203 is further described below.

FIG. 5 is aflowchart illustrating step 203 of FIG. 2 according to an exemplary embodiment of the present description, including the steps of:

step 203-1, determining the opening direction and the central position of the Wirosiss ring according to the positions of the plurality of key points.

The orientation of the aperture of the willis ring is determined before the display area is cut out from the sub-image, because the orientation of the aperture is such that it has an angle of inclination in the bounding box, see figure 6a, in which the orientation P of the aperture of the willis ring has an angle of inclination theta to the horizontal.

The following description will be made of a specific implementation process for determining the center position of the william ring, taking the 4 key points shown in fig. 3c as an example:

assuming that coordinates of 4 keypoint positions output by the keypoint location model are respectively expressed as A (a1, a2, a3), B (B1, B2, B3), C (C1, C2, C3) and D (D1, D2, D3),

the coordinates of the center position are

The following is also an example of 4 key points shown in fig. 3c, and a specific implementation process for determining the opening orientation of the williams ring is described as follows:

s1, selecting 3 first key points from the 4 key point positions, and determining a first normal vector of a plane where the 3 first key points are located.

Referring to fig. 6B, taking 3 first key points as the key point a selected on the anterior artery, the key point B selected on the left middle cerebral artery, and the key point C selected on the right middle cerebral artery as an example, the normal vector of the plane where the three points A, B, C are located, that is, the first normal vector, can be determined based on the point normal equation

Specifically, the method comprises the following steps:

the coordinates of the 3 first key points can be obtained

First normal vector

And vector

Sum vector

Are all vertical, then have:

by solving the above equation, the first normal vector can be determined

S2, selecting 3 second key points from the 4 key point positions, and determining a second normal vector of the plane where the 3 second key points are located.

Wherein the three first key points are not identical to the three second key points.

Referring to fig. 6B, taking 3 second key points as the key point B selected from the left middle cerebral artery, the key point C selected from the right middle cerebral artery, and the key point D selected from the basilar artery as an example, the normal vector of the plane where the three points B, C, D are located, that is, the second normal vector, can be determined based on the equation of the point normal form

Specifically, the method comprises the following steps:

the coordinates of the 3 second key points can be obtained

Second normal vector

And vector

Sum vector

Are all vertical, then have:

by solving the above equation, the second normal vector can be determined

And S3, determining the direction of the sum vector of the first normal vector and the second normal vector as the opening orientation of the Weilisi ring.

Suppose that the first normal vector is obtained by solving through the steps S1 and S2

Is shown as

Second normal vector

Is shown as

First normal vector

And the second normal vector

Is represented as a sum vector of

Will vector

Direction or vector of

Is determined as the opening orientation of the willis ring.

Step 203-2, determine the central plane of the willis's ring with the opening orientation as a normal vector and passing through the center position.

Referring to FIGS. 6c and 6d, the dotted line L₁Represents a position where a central plane passing through the central position and having the opening of the williams ring oriented normal to the central plane is located.

And step 203-3, determining an area between two planes which are parallel to the central plane and are away from the central plane by a preset distance in the sub-image as a display area.

Referring to FIGS. 6c and 6d, the dotted line L₂And dashed line L₃Representing the position of two planes at a predetermined distance d from the central plane, determined empirically, of about 2cm, along the dotted line L₂And dashed line L₃And performing plane interception to obtain an area, namely a display area.

As can be seen from fig. 6c, the display area of the circle of Willis cut from the bounding box has an oblique angle, and the circle of Willis shown at this angle is not beneficial to observing the circle of Willis, so instep 204, before outputting the display area of the circle of Willis, the display area can be rotated according to the opening direction of the circle of Willis, and then the display area is adjusted to the target observation angle, and then outputted, wherein, as described above, the target observation angle may be the optimal observation angle or the angle defined by the user.

In correspondence with the aforementioned embodiments of the image reconstruction method of the wiliss ring, the present specification also provides embodiments of the image reconstruction apparatus of the wiliss ring.

Fig. 7 is a block diagram of an image reconstruction apparatus of a willis ring according to an exemplary embodiment, the image reconstruction apparatus including: anacquisition module 71, asampling module 72, aninput module 73, atruncation module 74 and areconstruction module 75.

Anacquisition module 71 for obtaining a CTA image of the head and neck of the subject;

thesampling module 72 is configured to identify bounding boxes of a willis's ring in a CTA image and sample a sub-image containing the bounding boxes from the CTA image;

theinput module 73 is configured to input the sub-images into a keypoint localization model, where the keypoint localization model is obtained by training a neural network using CTA image samples, and the CTA image samples are labeled with keypoint location information located on a williams ring;

the interceptingmodule 74 is configured to intercept a display area of the wisdom ring from the sub-image according to a plurality of key point positions on the wisdom ring output by the key point positioning model;

thereconstruction module 75 is configured to perform image reconstruction on the display area, and adjust the display area to a target observation angle and output the adjusted display area.

identifying the region information of the artery blood vessel in the CTA image;

and inputting the CTA image and the region information into a target detection model to identify a bounding box region of the Weilisi ring in the CTA image, wherein the target detection model is obtained by training a neural network by using the CTA image marked with the bounding box of the Weilisi ring.

Optionally, the intercept module is specifically configured to:

Optionally, the number of the key point positions is 4;

For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, wherein the modules described as separate parts may or may not be physically separate, and the parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in the specification. One of ordinary skill in the art can understand and implement it without inventive effort.

Fig. 8 is a schematic diagram of an electronic device according to an exemplary embodiment of the present invention, showing a block diagram of an exemplaryelectronic device 80 suitable for use in implementing any of the embodiments of the present invention. Theelectronic device 80 shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.

As shown in fig. 8, theelectronic device 80 may be embodied in the form of a general purpose computing device, which may be, for example, a server device. The components of theelectronic device 80 may include, but are not limited to: the at least oneprocessor 81, the at least onememory 82, and abus 83 connecting the various system components including thememory 82 and theprocessor 81.

Thebus 83 includes a data bus, an address bus, and a control bus.

Thememory 82 may include volatile memory, such as Random Access Memory (RAM)821 and/orcache memory 822, and may further include Read Only Memory (ROM) 823.

Memory 82 may also include a program tool 825 (or utility tool) having a set (at least one) ofprogram modules 824,such program modules 824 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Theprocessor 81 executes various functional applications and data processing, such as the methods provided by any of the above embodiments, by running a computer program stored in thememory 82.

Theelectronic device 80 may also communicate with one or more external devices 84 (e.g., keyboard, pointing device, etc.), such communication may be through input/output (I/O) interfaces 85, and the model-generatedelectronic device 80 may also communicate with one or more networks (e.g., local area network (L AN), Wide Area Network (WAN) and/or a public network, such as the Internet) through anetwork adapter 86. As shown, thenetwork adapter 86 communicates with other modules of the model-generatedelectronic device 80 through abus 83.

It should be noted that although in the above detailed description several units/modules or sub-units/modules of the electronic device are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

The present specification also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the multimodal image registration method described in any one of the above.

The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.

Claims

1. An image reconstruction method of a willis ring, the image reconstruction method comprising:

obtaining a CTA image of the head and neck of the subject;

identifying bounding boxes of a williams' circle in the CTA image and sampling a sub-image containing the bounding boxes from the CTA image;

2. The method of image reconstruction of a wilisi ring in accordance with claim 1, wherein identifying bounding boxes of the wilisi ring in the CTA image comprises:

identifying the region information of the artery blood vessel in the CTA image;

3. The image reconstruction method of a williams ring according to claim 2, wherein identifying information of a region in the CTA image where the artery blood vessel is located comprises:

4. The method of image reconstruction for a william according to claim 1, wherein intercepting a display area from said sub-image based on a plurality of keypoint locations output by said keypoint location model comprises:

5. The image reconstruction method of a willis ring according to claim 4, wherein the number of said keypoint locations is 4;

6. The image reconstruction method of a william according to claim 1, further comprising:

performing image reconstruction on the display area, including:

7. An image reconstruction apparatus of a willis ring, characterized by comprising:

a sampling module to identify bounding boxes of a willis's ring in the CTA image and to sample a sub-image containing the bounding boxes from the CTA image;

8. The image reconstruction apparatus for a wilisi ring of claim 7, wherein in identifying the bounding boxes of the wilisi ring in the CTA image, the sampling module is to:

identifying the region information of the artery blood vessel in the CTA image;

9. The image reconstruction apparatus of a william's loop according to claim 8, wherein in identifying information of a region in which an arterial blood vessel is located in the CTA image, the sampling module is configured to:

10. The image reconstruction apparatus of a willis ring according to claim 7, wherein said truncation module is specifically configured to:

11. The image reconstruction apparatus of a willis ring according to claim 10, wherein the number of said keypoint locations is 4;

12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the image reconstruction method of a willis ring according to any one of claims 1 to 6 when executing the computer program.

13. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the image reconstruction method of a williams ring according to any one of claims 1 to 6.