US20120184851A1 - Reference image display method for ultrasonography and ultrasonic diagnosis apparatus - Google Patents

Abstract

An ultrasonic image is captured by an ultrasonic probe. A reference image is obtained by extracting a tomographic image corresponding to the scan plane of the ultrasonic image from volume image data that is pre-obtained by a diagnostic imaging apparatus and that is stored in a volume-data storing unit. The ultrasonic image and the reference image are displayed on the same screen. In this case, of the reference image, a portion corresponding to the view area of the ultrasonic image is extracted and the resulting reference image having the same region as the ultrasonic image is displayed as a fan-shaped image.

Description

CROSS-REFERENCE TO RELATED APPLICATION
• The present application is a Continuation of U.S. application Ser. No. 10/556,032, filed Nov. 8, 2005, which is a National Stage Entry of PCT/JP04/006238, filed May 10, 2004, which claims priority from Japanese Patent Application Nos. JP 2003-130490 and JP 2003-130600, both filed on May 8, 2003, the contents of which are hereby incorporated by reference into this application.
TECHNICAL FIELD
• The present invention relates to reference-image display methods for ultrasonography and ultrasonic diagnosis apparatuses using the methods. More specifically, the present invention relates to a technology preferably used for reconstructing, using multi-slice-image data of a patient obtained by a diagnostic imaging apparatus, a reference image of the same cross section as an ultrasonic scan plane in real time and for displaying the reference image and an ultrasonic image on the same screen. Examples of the diagnostic imaging apparatus include an ultrasonic diagnosis apparatus, a magnetic resonance imaging (MRI) apparatus, and an X-ray computed tomography (X-ray CT) apparatus.
BACKGROUND ART
• Ultrasonic diagnosis apparatuses, which are one type of diagnostic imaging apparatuses, are frequently used for diagnosis, since they are easy to handle and are capable of performing noninvasive observation of arbitrary cross sections in real time. On the other hand, ultrasonic images captured by the ultrasonic diagnosis apparatuses are generally inferior in image quality to tomographic images captured by X-ray CT apparatuses or the like. Thus, comprehensive diagnosis may be performed while performing comparison with a tomographic image captured by another diagnostic imaging apparatus, such as an X-ray CT apparatus or an MRI apparatus (the tomographic image will hereinafter be referred to as a “reference image”). For example, when hepatophyma or the like is treated by radiofrequency ablation under the guidance of an ultrasonic image, it is conceived that a treatment portion is pre-located by CT diagnosis and a CT image thereof is used as a reference image to perform the guidance with the ultrasonic image.
• However, when a CT image or MR image is merely rendered as the reference image to recognize an association relationship between the images is a great burden on the operator. This is because the reference image provided by a CT image or MR image is typically a tomographic image of a cross section perpendicular to a body axis, whereas the ultrasonic image is a tomographic image of an arbitrary cross section specified by the operator.
• Non-patent Document 1 describes an approach to facilitate the recognition of an association relationship between a reference image and an ultrasonic image. In the approach, a position sensor is attached to an ultrasonic probe to determine an ultrasonic scan plane and a reference image of the same cross section as the ultrasonic scan plane is reconstructed from multi-slice image data (hereinafter referred to as “volume image data”) of a CT image or MR image and is rendered on a display screen. Similarly,Patent Document 1 also proposes a technology in which a reference image of the same cross section as an ultrasonic scan plane is reconstructed from the volume image data of a CT image or MR image and the reference image and an ultrasonic image are rendered on a display screen in an aligned or superimposed manner or in an alternately switched manner.
• Patent Document 2 proposes a technology to aid manipulation for introducing a puncture needle into a body. That is, an ultrasonic scan plane is controlled so as to include the puncture needle and a reference image corresponding to the ultrasonic scan plane is cut out and is displayed. In the technology, two markers are attached to a body surface at a position corresponding to a patient's diseased area, into which the puncture needle is to be inserted, to obtain the volume image data of a reference image. Further, an ultrasonic probe is provided with an introducing portion for the puncture needle, so that the position and the angle of the puncture needle relative to the probe is fixed, and a sensor for detecting the position and the angle of the probe is attached to the probe to determine the ultrasonic scan plane. In this manner, a coordinate system for the volume image data and a coordinate system for the ultrasonic scan plane are associated with each other and a reference image corresponding to the ultrasonic scan plane is cut out and is displayed.
• Non-patent Document 1: “Radiology” RNSA issued in 1996, page 517, K. Oshio
• Patent Document 1: Japanese Unexamined Patent Application Publication No. 10-151131
• Patent Document 2: Japanese Unexamined Patent Application Publication No. 2002-112998
DISCLOSURE OF INVENTION
• However, in the prior art, although a reference image of a cross section corresponding to the scan plane of an ultrasonic image is cut out and is displayed on the same screen, no consideration is given to a scheme for matching display regions and the magnifications of a reference image and an ultrasonic image and. For example, an ultrasonic image is a fan-shaped image obtained by capturing one part of a living body of a patient, whereas a CT image or MR image is typically a circular image obtained by capturing the entire body of the patient. Thus, when the reference image and the ultrasonic image are merely displayed in an aligned manner, there is also a problem in that it is difficult to recognize an association relationship of portions he or she desires to observe.
• In addition, in order to obtain an ultrasonic image of a region including a target (e.g., a diseased area) arbitrary specified on a reference image by the operator, he or she must manipulate the ultrasonic probe to search the region including the target. However, in the prior art, there is a problem in that no consideration is given to a scheme for facilitating the recognition of the positional relationship between the current ultrasonic scan plane and the target.
• Accordingly, a first object of the present invention is to facilitate the recognition of an association relationship between an ultrasonic image and a reference image which are displayed on the same screen, the reference image being obtained by another diagnosis apparatus.
• A second object of the present invention is to facilitate the recognition of the positional relationship between a target specified on an arbitrary reference image by an operator and the current ultrasonic scan plane.
• In order to achieve the first object, an ultrasonic diagnosis apparatus of the present invention includes ultrasonic image generating means for reconstructing an ultrasonic image from reflection echo signals output from an ultrasonic probe, storing means for storing volume image data pre-obtained by a diagnostic imaging apparatus; reference-image generating means for extracting tomographic image data corresponding to a scan plane of the ultrasonic wave from the volume image data stored in the storing means and reconstructing a reference image, controlling means for causing the reference image and the ultrasonic image to be displayed on a screen, and displaying means for displaying the reference image and the ultrasonic image. In accordance with the tomographic image data and a positional relationship between the ultrasonic probe and a patient, the reference-image generating means extracts tomographic image data of a portion corresponding to a view area of the ultrasonic image to generate the reference image.
• Thus, according to the present invention, since the reference image of the same region corresponding to the fan-shaped view-area of the ultrasonic image is displayed as a fan-shaped image, it is possible to easily recognize an association relationship between both the images. In this case, it is preferable that, of the reference image, the region corresponding to the view area be displayed with the same magnification as the ultrasonic image, since the recognition of an association relationship between both the images is further facilitated. It is also preferable that, of the reference image, brightness of a portion out of the view area of the ultrasonic image be reduced to perform display. With this arrangement, it is possible to perform comparison and observation without losing information of the reference image.
• Further, displaying an acoustic shadow of the ultrasonic image on the reference image in a simulated manner further facilitates the recognition of an association relationship between both the images. Also, the ultrasonic image and the reference image can be displayed on the screen in an aligned manner, but the configuration is not limited thereto. A composite image of the ultrasonic image and the reference image can be displayed on the screen. The composite image can be an image obtained by superimposing a transparent image of the reference image on the ultrasonic image. Also, the composite image can be a difference image between the reference image and the ultrasonic image.
• It is preferable that the reference-image generating means change an image size of the reference image in accordance with a speed of movement of the ultrasonic probe. This makes it possible to display the reference image according to a quick movement of the ultrasonic image, thus enhancing the freedom of manipulating the probe during the comparison and observation.
• In order to achieve the second object, the ultrasonic diagnosis apparatus of the present invention includes a 3D body-mark determining unit for determining a positional relationship between the scan plane and a target set in the volume image data to cause a direction and a distance of the target relative to the scan plane to be displayed on the screen.
• Additionally, the ultrasonic diagnosis apparatus of the present invention may further includes a cine-memory for storing the ultrasonic image reconstructed by the ultrasonic-image generating means, a position sensor for detecting a position and an inclination of the ultrasonic probe, scan-plane-coordinate determining means for determining scan-plane coordinates of the ultrasonic image in accordance with an output from the position sensor, and scan-plane-coordinate-system storing means for storing the determined scan-plane coordinates. The reference-image generating means reads the scan-plane coordinates of the ultrasonic image from the scan-plane-coordinate-system storing means, reads the tomographic-image data corresponding to the read scan-plane coordinates, and reconstructs the reference image. The image processing means reads the ultrasonic image from the cine-memory and causes the reference image corresponding to the read ultrasonic image, the reference image being output from the reference-image generating means, to be displayed. With this arrangement, since ultrasonic images are sequentially read from the cine-memory and displayed and reference images corresponding to the ultrasonic images are sequentially cut out and displayed, comparison and observation can be performed using moving images.
• It is also preferable that the ultrasonic diagnosis further includes at least one of a posture sensor for detecting a change in the posture of the patient and a sensor for detecting breathing and further has correcting means for correcting the scan-plane coordinates in accordance with an amount of internal-organ movement caused by a posture change or the breathing of the patient during ultrasonic diagnosis. With this arrangement, a displacement between the reference-image coordinate system and the ultrasonic-image coordinate system, the displacement being resulting from internal-organ movement caused by the breathing or a posture change of the patient, can be corrected. Thus, the accuracy of comparison and observation of both the images can be improved.
• In addition or instead, the configuration can be such that, after the scan plane of the ultrasonic probe is scanned and one of an ultrasonic image and a reference image which has a distinctive point is searched for and frozen, the ultrasonic probe is manipulated, an image that is other than the frozen ultrasonic image or reference image and that matches the frozen one of the images is displayed and frozen, and a coordinate difference between scan-plane coordinates for the frozen one of the images and the other image is determined, so that the scan plane coordinates can be corrected based on the determined coordinate difference.
• Further, in addition to the above-described configuration, the ultrasonic diagnosis apparatus can include: a position sensor for detecting a position and an inclination of the ultrasonic probe in association with a reference coordinate system; scan-plane-coordinate determining means, for determining scan-plane coordinates of an ultrasonic image captured by the ultrasonic probe in association with the reference coordinate system, in accordance with an output from the position sensor; reference-point inputting means for setting a reference point on a reference image displayed on the screen based on the volume image data obtained in association with the reference coordinate system; volume-data-coordinate determining means for determining coordinates, of tomographic data of the volume image data associated with the scan-plane coordinates, by determining a coordinate relationship between the position of the ultrasonic probe and a region that corresponds to the reference point and that exists on an ultrasonic image obtained by bringing the ultrasonic probe in contact with a body surface of the patient; and volume-data-coordinate storing means for storing the tomographic-image-data coordinates determined by the volume-data-coordinate determining means. The reference-image reconstructing means can read the coordinates of the tomographic image data, associated with the scan-plane coordinates determined by the scan-plane-coordinate determining means, from the volume-data-coordinate storing means and can extract the reference image. With this arrangement, the reference point for aligning the coordinate systems can be set inside the body of the patient. Thus, compared to the prior art in which a reference point is set on the body surface, the freedom of setting the reference point is increased and thus the accuracy of comparison and observation can be further improved.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a block diagram of a basic diagnostic imaging system to which an ultrasonic diagnosis apparatus of one embodiment of the present invention is applied.
• FIG. 2 is a block diagram of a specific diagnostic imaging system to which an ultrasonic diagnosis apparatus of another embodiment of the present invention is applied.
• FIG. 3 is a flow chart showing a sequence of a drawing procedure for an ultrasonic image and a reference image in one embodiment of the present invention.
• FIG. 4 is a view showing a display example of an ultrasonic image, a reference image, a composite image, and a 3D body mark according to a feature of the present invention.
• FIG. 5 is a view showing a display example of an ultrasonic image, a reference image, a composite image, and a 3D body mark which are preferable for navigation according to a feature of the present invention.
• FIG. 6 is a block diagram of a specific diagnostic imaging system to which an ultrasonic diagnosis apparatus of still another embodiment of the present invention is applied.
• FIG. 7 are block diagrams of a position-sensor equipped probe in one embodiment according to the present invention.
• FIG. 8 show the configuration and the processing procedure of breathing-amount determining means according to the present invention.
• FIG. 9 is a detailed block diagram of a scan-plane-coordinate determining unit and a scan-plane-coordinate storing unit in the embodiment shown inFIG. 2.
• FIG. 10 is a flow chart of initialization processing for coordinate associating processing in the embodiment shown inFIG. 6.
• FIG. 11 is a flow chart of embodiment of a reference-image display processing, during ultrasonic diagnosis, in the embodiment shown inFIG. 6.
• FIG. 12 are diagrams illustrating an association relationship between volume image data and a scan-plane coordinate system.
• FIG. 13 is a flow chart of one embodiment for correcting a reference-coordinate-system displacement caused by breathing or the like of a patient.
• FIG. 14 shows one example of a coordinate adjustment screen for assisting processing for correcting scan plane coordinates.
• FIG. 15 is a view illustrating a method for correcting a coordinate-system displacement resulting from internal-organ movement caused by the breathing of the patient.
BEST MODE FOR CARRYING OUT THE INVENTION
• Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.
First Embodiment
• FIG. 1 is block diagram of a basic diagnostic imaging system to which an ultrasonic diagnosis apparatus of one embodiment of the present invention is applied. As shown, the diagnostic imaging system includes anultrasonic diagnosis apparatus101 according to one embodiment of the present invention and a medicaldiagnostic imaging apparatus102 for obtaining volume image data that provides as a reference image. The volume image data refers to the data of multi-slice images obtained by capturing the inside of the body of a patient along multiple slice planes. The data of the volume images captured by the medicaldiagnostic imaging apparatus102 is input to theultrasonic diagnosis apparatus101. A computed tomography apparatus (X-ray CT apparatus) or a magnetic resonance imaging apparatus (MRI apparatus) can be used as the medicaldiagnostic imaging apparatus102. CT images and MR images have higher image qualities than ultrasonic images, as is known, and thus are suitable as reference images for ultrasonic images, which are inferior in image quality. However, when a temporal change in tissues of a patient is diagnosed with ultrasonic waves, the volume image data of a pre-obtained ultrasonic image can be drawn as the reference image.
• InFIG. 1, descriptions of functions commonly included in theultrasonic diagnosis apparatus101 are omitted to avoid complexity, and only the functions of major units associated with displaying the reference image according to a feature of the present invention are described. As shown, theultrasonic diagnosis apparatus101 can broadly be divided into a section for reconstructing an ultrasonic image and a section for reconstructing the reference image. The former ultrasonic-image reconstructing section has aprobe104 and an ultrasonic-image determining unit105. The latter reference-image reconstructing section has a volume-data storing unit107 and a reference-image determining unit111.
• The ultrasonic-image determining unit105 provides ultrasonic-image generating means for reconstructing an ultrasonic image in accordance with a reflection echo signal output from theprobe104. The ultrasonic-image determining unit105 is adapted to associate signals output from aposition sensor108 with the reconstructed ultrasonic image. On the other hand, a controllingunit120 is adapted to determine the scan-plane coordinates of theprobe104 in accordance with signals output from theposition sensor108 and to output the determined scan-plane coordinates to the reference-image determining unit111. The reference-image determining unit111 provides a reference-image generating means for extracting tomographic image data, corresponding to the scan-plane coordinates input from the controllingunit120, from the volume-data storing unit107 and reconstructing the reference image. Thus, the ultrasonic image reconstructed by the ultrasonic-image determining unit105 and the reference image reconstructed by the referenceimage determining unit111 are adapted to be displayed on amonitor114.
• In particular, the reference-image determining unit111 is configured such that it extracts tomographic-image data-of a region corresponding to the view area of an ultrasonic image, in accordance with the scan-plane coordinates that are input from the controllingunit120 and that are based on the positional relationship between theprobe114 and the patient, and generates a reference image.
• According to the present embodiment configured as described above and shown inFIG. 1, a reference image corresponding to the fan-shaped view area of an ultrasonic image, the reference image and the ultrasonic image captured from the same region, is displayed as a fan-shaped image. This makes it possible to easily recognize an association relationship between both the images. In this case, displaying, of the reference image, a region corresponding to the view area of the ultrasonic image with the same magnification as the ultrasonic image can further facilitate the recognition of an association relationship between both the images. Also, displaying, of the reference image, a region that is out of the view area of the ultrasonic image, with reduced brightness, allows comparison and observation without loosing the information of the reference image.
Second Embodiment
• FIG. 2 shows the configuration of a specific diagnostic imaging system to which an ultrasonic diagnosis apparatus of the present invention is applied. In the figure, means having the same functional configurations as those inFIG. 1 are denoted with the same reference numerals and the descriptions thereof are omitted. InFIG. 2, a scan-plane-coordinate determiningunit109 and a scan-plane-coordinatestoring unit110 correspond to the configuration of the major unit of the controllingunit120. A cine-memory106 stores an ultrasonic image reconstructed by the ultrasonic-image determining unit105. A 3D body-mark determining unit112 is provided in connection with the reference-image determining unit111. Anadder113 is configured as image processing means for appropriately combining images generated by the cine-memory106, the reference-image determining unit111, and the 3D body-mark determining unit112. Themonitor114 is adapted to display images generated by the cine-memory106, the reference-image determining unit111, and the 3D body-mark determining unit112 and the image processed by theadder113.
• Theprobe104 transmits/receives ultrasonic waves to/from apatient103 and has built-in multiple transducers that generate ultrasonic waves and that receive reflection echoes. The ultrasonic-image determining unit105 receives reflection echo signals output from theprobe104 and converts the received signals into digital signals to create anultrasonic image302, such as a tomographic image (B-mode image) or a color flow mapping image (CFM image), of a diagnosis region, as shown inFIG. 3 and so on. Thecine memory106 receives ultrasonic images created by the ultrasonic-image determining unit105 and stores ultrasonic images for multiple frames.
• The volume-data storing unit107 receives the volume image data of a reference image, captured by the medicaldiagnostic imaging apparatus102, through a network or via a portable storage (MO) medium, such as a magneto-optical disk, and stores the volume image data in theultrasonic diagnosis apparatus101.
• Theposition sensor108 is attached to theprobe104 to detect the three-dimensional position and inclination of the probe. Asource116 for a coordinate system including thepatient103 is placed in the vicinity of abed115 on which thepatient103 lies. The principle of detecting the three-dimensional position and inclination of theprobe104 is that magnetic signals generated in a three-dimensional space by thesource116 is detected by theposition sensor108 and the three-dimensional position and inclination in a reference coordinate system formed by thesource116 are detected. A position sensor system constituted by theposition sensor108 and thesource116, is not only limited to a magnet-based system but also may employ, for example, a known position sensor system, such as a system utilizing light.
• In accordance with signals output from theposition sensor108 and thesource116, the scan-plane-coordinate determiningunit109 obtains the position and inclination information of theprobe104 in the reference coordinate system to determine scan-plane coordinates including the position and the inclination of a ultrasonic scan plane relative to thepatient103. The scan-plane-coordinate determiningunit109 is also adapted to calculate scan-plane coordinates in a reference-image coordinate system, in accordance with the determined scan-plane coordinates. That is, the scan-plane-coordinate determiningunit109 is adapted to determine scan-plane coordinate data including, for example, x, y, and z coordinate data of one corner of a scan plane and rotation angles about x, y, and y axes of the scan plane in a volume-image-data coordinate system. The scan-plane-coordinate data determined by the scan-plane-coordinate determiningunit109 is input to the scan-plane-coordinatestoring unit110 and scan plane coordinates for multiple frames are stored therein. In this case, it is preferable that the number of frames for scan plane coordinates stored be substantially the same as the number of frames of ultrasonic images captured in real time and stored in the cine-memory106. The reference-image determining unit111 provides reference-image reconstructing means, and receives scan-plane coordinate data and reconstructs a reference image of the same cross section as an ultrasonic scan image.
• Next, a detailed configuration of theultrasonic diagnosis apparatus101 according to the present embodiment will be described in conjunction with the operation thereof.FIG. 3 is a flow chart for rendering an ultrasonic image and a reference image of the same cross section. The drawing processing is broadly classified into an ultrasonic-image processing sequence201 for rendering an ultrasonic image and storing the scan-plane coordinates in a storage unit and a reference-image processing sequence202. These twosequences201 and202 are executed in such a manner that starts and freezes are synchronized.
• First, when an operator starts the twosequences201 and202, a determination is made as to whether a freeze instruction is input (51). When freeze is not performed, the ultrasonic-image processing sequence201 drives theprobe104 to transmit/receive ultrasonic waves to/from the patient103 (S2). The ultrasonic-image determining unit105 reconstructs an ultrasonic image in accordance with the reflection echo signals output from the probe104 (S3). The reconstructed ultrasonic image is stored in the cine-memory106 (S4) and is drawn on the monitor114 (55).
• At this point, theposition sensor108 obtains the position and inclination of theprobe104 in synchronization with the transmission/reception of the ultrasonic waves (S12). In accordance with the position and inclination information input from theposition sensor108, the scan-plane-coordinate determiningunit109 determines scan-plane coordinates (S13). The determined scan-plane coordinates are sequentially written to the scan-plane-coordinate storing unit110 (S14). In this case', the processing of steps51 to55 in the ultrasonic-image processing sequence201 and the processing of steps S12 to S14 are executed in synchronization with each other.
• On the other hand, in the reference-image processing sequence202, a determination about freezing is made (S21). When freezing is not performed, scan-plane coordinates are read from the scan-plane-coordinate storing unit110 (S22). Based on volume image data, the reference-image determining unit111 reconstructs a reference image of the same cross section as the ultrasonic image (S25). The reconstructed reference image is drawn on the monitor114 (S26). The processing of steps523 and524 will be described below.
• Next, when the operator inputs an instruction for freezing the processing, the ultrasonic-image processing sequence201 and the reference-image processing sequence202 are adapted to execute cine playbacks in S31 and532, respectively, based on the determination in steps Si and S21. The cine playback of an ultrasonic image is executed by referring to the ultrasonic image data stored in thecine memory106. In contrast, the cine playback of a reference image is executed by using the scan-plane coordinate data stored in the scan-plane-coordinatestoring unit110 and by reconstructing a reference image corresponding to the scan plane based on the volume image data. The ultrasonic image data stored in the cine-memory106 and the scan-plane coordinate data stored in the scan-plane-coordinatestoring unit110 are stored in synchronization with each other, it is possible to render an ultrasonic image and a reference image whose time phases are the same. The cine playback of an ultrasonic image is performed by referring to the ultrasonic-image data stored in the cine-memory106, whereas the cine-playback of a reference image is performed by referring to the scan-plane-coordinate data stored in the scan-plane-coordinatestoring unit110. Thus, it is sufficient for the memory of the scan-plane-coordinatestoring unit110 to store only scan-plane-coordinate data, the memory capacity can be reduced. Similarly, for storing a moving image, merely storing scan-plane coordinates corresponding to volume image data makes it possible to play back the moving picture while reconstructing it from the volume image data. Thus, a moving-image file having a small file size can be created.
• Now, an image-display processing method according to a feature of the present invention will be described with reference toFIG. 4. First, in accordance with the enlargement factor (magnification) of theultrasonic image302, the reference-image determining unit111 enlarges or reduces a reference image and displays it at the same magnification, as shown in areference image301 Shown inFIG. 4. The reference-image determining unit111 also extracts an out-of-view area312 corresponding to a fan-shapedviewing angle311 of theultrasonic image302 and reduces the brightness of a reference image corresponding to theregion312. As a ˜result, the reference image is displayed in the same display format and with the same magnification as those of theultrasonic image302, thus making it easy to recognize an association relationship between theultrasonic image302 and the reference image. This arrangement also makes it possible to perform display without losing the information of a reference image in the out-of-view area of the ultrasonic image. Also, anacoustic shadow307, such as a bone313 (or air), appears on theultrasonic image302. It is preferable that a region corresponding to theacoustic shadow307 be extracted based on determination, for example, using CT values of a CT image; and the brightness of anarea308 that is deeper than that region be reduced. Similarly, anarea310 is extracted, using CT values, from a region where a blood vessel exists and the region is displayed, for example, in red, like an ultrasonic CFM (color flow mapping)image309. This makes it possible to display thereference image301, which allows easy comparison with theultrasonic image302.
• On the other hand, the 3D body-mark determining unit112 extracts a three-dimension visualized image, such as a3D body mark304 inFIG. 4, by using volume image data, superimposes ascan plane314 in a translucent color on the three-dimension visualized image, and displays the resulting image. As the three-dimensional visualized image, for example, a known method, such as volume rendering or surface rendering, can be used. Displaying the3D body mark304 allows the positional relationship between the patient103 and thescan plane314 to be recognized in three dimensions. The 3D body-mark determining unit112 may be provided with a function for extracting a region of interest, specified by the operator, from the volume image data and determining the distance and the direction from the scan plane to the region of interest.
• Theadder113, which provides the image processing means, is intended to determine acomposite image303 of thereference image301 and theultrasonic image302. Theadder113, for example, converts thereference image301 into a translucent-color image and superimposes it on theultrasonic image302. Instead, a difference image between thereference image301 and theultrasonic image302 may be obtained and drawn. This can facilitate that thereference image301 and theultrasonic image302 are compared with each other using one image. With the difference image, for example, when an ultrasonic volume image data obtained in advance is used as the reference image, it is useful to diagnose a temporal change in living-body tissues of the patient.
• Thus, as shown inFIG. 4, theultrasonic image302, thereference image301, thecomposite image303, and the 3D body-mark304 of the same cross section are drawn on themonitor114. This allows the operator to perform effective diagnosis while comparing those images.
• For example, using the medicaldiagnostic imaging apparatus102 to obtain volume image data centering at a treatment region before medical treatment, causing theultrasonic diagnosis apparatus101 to capture an image of the treatment region after the medical treatment, and displaying a reference image before the medical treatment and an ultrasonic image after the medical treatment, for example, in an aligned manner can facilitate determination of the effect of the medical treatment. Also, synthesizing an image of a difference between the reference image before the medical treatment and the ultrasonic image after the medical treatment and displaying the difference image further facilitates the determination of the effect of the medical treatment. In particular, performing display in added color according to the degree of the difference can further facilitate viewing.
• By reducing the image size and changing the frame rate, the reference-image determining unit111 can increase the speed of reconstructing a reference image in accordance with the motion of theprobe104. That is, the reference-image determining unit111 determines the movement speed and the rotation speed of the scan plane, based on the scan-plane coordinate data. When the speed is greater than a certain threshold, the reference-image determining unit111 reduces the image size to reconstruct the reference image at a high speed. That is, when the movement of theprobe104 is fast, priority is given to the frame rate over the image quality to draw the reference image at a high speed. and when the movement of theprobe104 is slow, priority is given to the image quality over the frame rate to reconstruct and draw the reference image. This makes it possible to draw the reference image so as to correspond to the ultrasonic image that varies according to the motion of theprobe104.
• An image-display processing method with a navigation function will further be described with reference toFIG. 5. This ultrasonic diagnosis apparatus is adapted to allow navigation for guiding the scan plane of theprobe104 to atarget405 that the operator pre-set on a reference image in the volume image data. Thetarget405 can be set by designating. a region with a mouse, for example, on an axial image, sagittal image, coronal image, and three-dimensional visualized image. The 3D body-mark determining unit112 calculates the distance and direction from the current scan plane to the center of thetarget405 and displays a three-dimensional arrow image and numeric values in adisplay region407 on the screen of a3D body mark404. The boundary of the region of thetarget405 is also rendered in areference image401 and anultrasonic image402. This allows the operator to visually recognize the distance from the currentultrasonic scan plane314 to thetarget405. When thetarget405 enters thescan plane314, a boundary determined from thereference image401 is also displayed in theultrasonic image402. Consequently, it is easy to recognize an association relationship between thereference image401 and theultrasonic image402.
• Additionally, a region of interest (ROI)406 that the operator set on any of theultrasonic image402, thereference image401, and acomposite image403 is displayed on all the images. This facilitates the recognition of an association relationship of the region of interest.
Third Embodiment
• FIG. 6 shows the configuration of a diagnostic imaging system to which an ultrasonic diagnosis apparatus of another embodiment of the present invention is applied. InFIG. 6, what are different from the embodiment shown inFIG. 2 are that abreathing sensor117 for detecting the amount of breathing of thepatient103 and aposture sensor118 for detecting the body movement are provided and outputs of the detections are input to the scan-plane-coordinate determiningunit109. Although processing for associating a volume-image-data coordinate system with a scan-plane coordinate system was omitted in the embodiment inFIG. 2, details thereof will be described.
• In the present embodiment, as shown inFIG. 7A, theposition sensor108 is attached to one surface of theprobe104 to make it possible to detect the position and inclination of theprobe104, i.e., the position and inclination of the ultrasonic scan plane, in a coordinate system formed by thesource116. InFIG. 7A, transducers are arranged on a circular arc surface of theprobe104 and the distance between acenter point201 of the transducers and acenter point202 of theposition sensor108 has been accurately determined. The relationship between theprobe104 and theposition sensor108 is not limited to what is shown in the figure and can be configured as shown inFIG. 7B. That is, the arrangement can be such that a bar-shapedpointer203 is detachably attached in association with theposition sensor108 and anend point204 of thepointer203 is used as a reference point relative to thecenter point202. With this arrangement, theprobe104 of the present embodiment can also be utilized as a pointing device.
• In the same manner as theposition sensor108, theposture sensor118 is attached to the body surface of thepatient103 so as to measure the position and inclination of thepatient103 in the reference coordinate system formed by thesource116. Thebreathing sensor117 measures the amount of breathing of thepatient103. For example, as shown inFIG. 8A, thebreathing sensor117 having a function similar to theposition sensor108 is attached to the body surface of thepatient103, lying on thebed115, so as to detect the amount of body-surface movement caused by the breathing. As shown inFIG. 8B, the measured amount of movement can be converted into an amount of breathing.
• While the scan-plane-coordinate determiningunit109 and the scan-plane-coordinatestoring unit110 are configured to be essentially the same as those in the second embodiment, features and functions according to the present embodiment will be specifically described. The scan-plane coordinate determiningunit109 has a function for correcting scan-plane coordinates in accordance with the posture information of thepatient103 and the amount of breathing of thepatient103. The scan-plane coordinates as used herein refer to the coordinates of an ultrasonic scan plane imaged by theprobe104. As shown inFIG. 9, the scan-plane-coordinate determiningunit109 and the scan-plane-coordinatestoring unit110 include a scan-plane coordinate-system storing unit211 a volume-image-data coordinate-system storing unit212, a posture-change-amount determining unit213, an internal-organ-movement-amount determining unit214, a correctingunit215, and a corrected-scan-plane-coordinate determiningunit216. As in the embodiment shown inFIG. 2, the reference-image determining unit111 receives the scan-plane coordinates, extracts same-cross-section image data corresponding to the scan-plane coordinates from the volume-data storing unit107, and reconstructs. a reference image. Theadder113 then draws a reference image, output from the reference-image determining unit111, and an ultrasonic image, read from the cine-memory106, on themonitor114. The ultrasonic image and the reference image are typically displayed on the same screen in an aligned manner, but instead, can be displayed in a superimposed manner. When they are displayed in a superimposed manner, it is desired that the reference image be translucent.
• Now, processing for associating volume-image-data coordinates with scan-plane coordinates will be described with reference toFIGS. 10,11, and12. The coordinate association processing in the present embodiment can be broadly classified into an initialization stage shown inFIG. 10 and a diagnosis stage shown inFIG. 11.
• First, a description is given of the initialization stage shown inFIG. 10, i.e., processing during the imaging of volume image data. In step S101, as shown inFIG. 12B, the x axis of thesource116 for the position sensor is oriented in the lateral direction of thebed115, the y axis is oriented in the longitudinal direction of thebed115, and the z axis is oriented in the vertical direction of thebed115 to thereby place the source. Thus, the x, y. and z axes of a source coordinate system that has the origin at, for example, acenter224 of thesource116 are aligned parallel to the x, y, and z axes of a coordinate system having an original225 at one corner of volume image data shown inFIG. 12A. That is,volume data221 shown inFIG. 12A is obtained by typically laying thepatient103 on thebed115 and capturing a tomographic image perpendicular to the body axis (the y-axis direction) of thepatient103. Placing thesource116 so as to be aligned with thebed115 allows the x, y, and z axes of the reference coordinate system of thesource116 and the coordinate system of thevolume data221 to be substantially parallel to each other.
• Next, in step S102, areference point223 is set in thevolume data221. Thereference point223 is set on an operation screen by using a pointing device, such as a mouse. The operation screen is displayed with a reference image obtained by' imaging volume image data. The operation screen may include an axial image, sagittal image, coronal image, or three-dimensional visualized image. Designating thereference point223 on any of the images makes it possible to set thereference point223 on the body surface or inside the body in the volume image data.
• In contrast, in step S103, areference point222 in the scan-plane coordinate system is set, for example, by locating theprobe104 with theposition sensor108 at a position corresponding to thereference point223 of thevolume data221 and holding theprobe104. For example, when thereference point222 of the volume data is designated on the body surface, thecontact point201 of theprobe104 is placed at a body-surface position of theactual patient103, the body-surface position corresponding to thereference point222, to set thereference point222 in the scan-plane coordinate system. In this case, since the position of thereference point222 and the position of thereference point223 match each other, it is possible to match the coordinate system of the volume image data and the coordinate system of the scan plane. In this case, theprobe104 is used as a pointing device. Here, in order to facilitate the work of placing the probe at the body-surface position of the actual patient, the body-surface position being corresponding to the position of the reference point specified in the volume image data, it is preferable that a distinctive point (e.g., a xiphoid process or a blood vessel branch) that can be easily found on the body surface from the external view be selected as thereference point223 specified in the volume image data.
• On the other hand, when the reference point in the volume image data is specified inside the body, the probe is manipulated, an ultrasonic image containing a region containing the in-vivo reference image223 is displayed, and a region corresponding to the in-vivo reference point223 is specified on the ultrasonic image by using a pointing device, such as a mouse. Then, the distance between the specified point and thecenter202 or thecontact point201 of theprobe104 is determined and the coordinates of the tworeference points222 and223 are associated with each other. In this case, in order to easily identify the in-vivo reference point223 on the ultrasonic image, it is preferable that an easy-to-find distinctive point on the ultrasonic image be selected as the reference point specified in the volume image data, as described above.
• Next, in step S104, relationship data for associating the scan-plane coordinate system with the reference coordinate system of thesource116 is determined. First, the origin of thepatient103 in the real space is set at thereference point222. The coordinate axes of the scan plane coordinate system are set parallel to the coordinate axes of the source coordinate system. Then, the position (X, Y, and Z) of thereference point222 of theprobe104, the position being detected by theposition sensor108, is determined, the scan-plane coordinate system and the source coordinate system are associated with each other, and the resulting association data is stored in the scan-plane coordinate-system storing unit211 shown inFIG. 9. In this manner, the volume-image-data coordinate system and the scan-plane coordinate system can be associated with each other via the reference coordinate system of thesource116. In step S105, data for associating the volume-image-data coordinate system with the scan-plane coordinate system is created and is stored in the volume-image-data coordinate-system storing unit212 shown inFIG. 9.
• Since thesource116 is placed in step S101 such that the coordinate axes of the volume-image-data coordinate system are parallel to the coordinate axes of the source coordinate system. Setting only onereference point223 in the volume-image-data coordinate system can facilitate that those two coordinate systems are associated. That is, placing thesource116 in an appropriate direction according to the direction of the body axis of the patient can readily align the coordinate systems. However, in the present invention, threereference points223 can also be set. In this case, the accuracy of associating the coordinate systems can be improved. For example, when the coordinate systems are determined with three reference points, one of the three points is designated as the origin of the coordinate system, vectors from the origin to the remaining two points are designated as an X axis and a Y axis, and an axis perpendicular to the x and y axes is designated as a Z axis to thereby achieve the alignment. This makes it possible to associate the coordinate systems without caring about the direction of the placedsource116. The remaining two points can be automatically set on the screen by causing a measurement tool for performing measurement on image data to perform the above-described processing.
• The association data between the volume-image-data coordinate system and the scan-plate coordinate system, the association data being created as described above, is used during ultrasonic diagnosis to determine scan-plane coordinates according to a procedure shown inFIG. 11. The scan-plane-coordinate determiningunit109 determines the scan plane coordinates, in accordance with the position and inclination of theprobe104 which are detected by theposition sensor108 attached to the probe104 (step S106). Next, the reference-image determining unit111 cuts a reference image, corresponding to the scan-plane coordinates, out from the volume image data and causes the reference image to be displayed on themonitor114 via the adder113 (step S107). This allows the operator to draw a reference image that matches an ultrasonic image corresponding to an arbitrary set position and direction of the probe, thereby improving the accuracy of diagnosis.
• Next, a feature and a function of the present embodiment for correcting the scan plane coordinates in accordance with a change in the posture and so on will be described. That is, as diagnosis proceeds, a displacement may occur between the coordinate systems of the volume image data and the scan plane, due to factors, such as a change in the posture of the patient and internal-organ movement caused by the breathing of the patient. Such a displacement may make it impossible to draw a reference image that matches the ultrasonic scan plane. Accordingly, in the present embodiment, in a diagnosis stage, the scan plane-coordinate determiningunit109 is adapted to correct a displacement in the scan-plane coordinate system.
• Correction for a change in the posture of the patient will be described first. The posture of the patient can be detected with theposture sensor118 shown inFIG. 6. Thus, a difference between the posture during initialization and the posture during diagnosis is determined by the posture-change-amount determining unit213, and in accordance with difference, the scan-plane coordinate system is shifted and rotated to perform correction.
• Next, means for correcting a coordinate-system displacement due to internal-organ movement caused by the breathing of the patient will be described with reference toFIG. 13. The operator performs this correction processing, while viewing a rendered reference image and ultrasonic image. The operator first pays attention to the reference image. The operator displays, for example, a distinctive cross section, including the patient blood vessel such as a portal vein or superior mesenteric artery, and performs freezing, while manipulating the probe104 (step S201). Next, the operator pays attention to the ultrasonic image, and renders the same cross section as the frozen image of the reference image and performs adjustment, while performing visual comparison with the frozen image (step S202). Here, a difference (the amount of change) between the scan-plane coordinates during the freezing and the scan plane-coordinates during the adjustment corresponds to the amount of internal-organ movement. Thus, the scan-plane coordinate determiningunit109 determines a difference (the amount of change) between the scan-plane coordinates during the freezing and the scan-plane coordinates during the adjustment (step S203). The scan-plane coordinate system is shifted and rotated by an amount corresponding to the difference to thereby perform correction (step S204). Consequently, even when the depth of breathing of the patient differs from that in the initialization stage, the scan-plane coordinates and the volume-image-date coordinate system can be correctly associated with each other.
• During the correction processing inFIG. 13, a coordinateadjustment screen231 shown inFIG. 14 is displayed so as to allow afreeze key232 and an adjust key234 to be operated on the screen. Also, the amount of movement and the amount of rotation which are related to the correction can also be displayed inparameter edit boxes232. The operator can perform correction by directly inputting numeric values to theedit boxes232.
• In addition, an input amount of breathing is measured by thebreathing sensor117, the correction is repeated at multiple depths of breathing, and the correlation between the amount of breathing and the amount of internal-organ movement is determined. With this approach, a displacement caused by internal-organ movement can be automatically corrected in accordance with an input from the breathing sensor.
• Hereinabove, a volume-image-data reference image having a distinctive point is frozen and an ultrasonic scan plate corresponding to the reference image has been determined. Conversely, the arrangement may also be such that an ultrasonic scan plane having a distinctive point is frozen and image processing is performed to automatically determine a reference image corresponding to the ultrasonic scan plane. Specifically, an ultrasonic scan plane displaying a distinctive point of the patient is frozen and the ultrasonic scan plane is recorded in a storage medium, such as the cine-memory106. By using a known pattern matching method, the reference-image determining unit111 extracts a reference image corresponding to the distinctive point of the frozen ultrasonic scan plane from the volume image data stored in the volume-data storing unit107. The reference image extracted as a result of matching is displayed on themonitor114. In the reference-image matching processing, there is no need to search all the volume image data. Thus, only data regarding the scan-plane side, viewed in the scan direction of theultrasonic probe104, may be extracted to match distinctive regions. Magnification may also be adjusted such that a distinctive point on the ultrasonic scan plane and a distinctive point on the reference image have the same magnification. In this manner, extracting a reference image from image information having a distinctive point can enhance the accuracy of aligning the ultrasonic scan plane and the reference image.
• In addition, another example of the means for correcting a coordinate-system displacement due to internal-organ movement caused by the breathing of the patient will be described with reference toFIG. 15. As in the case ofFIG. 13, the operator performs this correction processing while viewing a rendered reference image and an ultrasonic image. First, theprobe104 is placed on the patient so that cross sections perpendicular to a direction in which the internal organs move due to breathing are displayed. The direction in which the internal organs move due to breathing is typically the body axis direction of the patient. Theprobe104 placed on the patient is then moved in the internal-organ movement direction, and distinctive ultrasonic tomographic images, including a blood vessel such a portal vein or superior mesenteric artery, are rendered. At this point, due to the occurrence of a displacement between the volume-image-data coordinate system and the scan-plate coordinate system, a cross-section displacement in the body axis direction occurs between the ultrasonic image and the reference image. In this state, the ultrasonic image and the reference image are frozen, cine playback is performed, and the operator specifies a corresponding combination of a reference image and an ultrasonic image. For example, in the example shown inFIG. 15, since blood vessels can be most clearly rendered on areference image1 displayed at time t1 and anultrasonic image2 displayed at time t2, it is determined that they are corresponding images. It can be understood from this that, although thereference image1 corresponding to theultrasonic image2 is supposed to be displayed at the position of time t2, the corresponding reference image is displayed at time t1. At this point, a difference (the amount of change) between scan-plane coordinates1 and scan-plane coordinates2 which are stored in the scan-plane coordinate storingunit110 corresponds to the amount of internal-organ movement. Accordingly, determining the difference between the scan-plane coordinates1 and the scan-plane coordinates2 and correcting the scan plane coordinates makes it possible to correct a coordinate-system displacement caused by the internal-organ movement.

Claims (13)

1. An ultrasonic diagnosis apparatus comprising:

an ultrasonic image generating unit for generating an ultrasonic image based on reflection echo signals output from an ultrasonic probe;

a storing unit for storing volume image data pre-obtained by a diagnostic imaging apparatus;

a reference-image generating unit for extracting tomographic image data corresponding to a scan plane of the ultrasonic image from the volume image data stored in the storing unit, and reconstructing a reference image;

a displaying unit for displaying the reference image and the ultrasonic image; and

a 3D body-mark determining unit for generating a three-dimension visualized image of an object using the volume image data stored in the storing unit, superimposing an image representing the scan plane in a translucent color on the three-dimension visualized image, and displaying the superimposed image on the displaying unit.

2. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the displaying unit displays a target on the ultrasonic image and the reference image when the target pre-set in the volume image data enters the scan plane of the ultrasonic probe.

3. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the displaying unit displays a target on the ultrasonic image and the reference image when the target pre-set in the reference image of the volume image data enters the scan plane of the ultrasonic probe.

4. The ultrasonic diagnosis apparatus according toclaim 2,

wherein the 3D body-mark determining unit determines a positional relationship between the scan plane and the target set in the volume image data, and

the displaying unit displays a direction and a distance of the target relative to the scan plane.

5. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the displaying unit displays a portion of the reference image corresponding to a view area of the ultrasonic image with the same magnification as the ultrasonic image.

6. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the displaying unit displays a portion of the reference image out of a view area of the ultrasonic image with reduced brightness.

7. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the displaying unit displays an acoustic shadow of the ultrasonic image on the reference image in a simulated manner.

8. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the displaying unit displays the ultrasonic image aligned with the reference image.

9. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the displaying unit displays a composite image of the ultrasonic image and the reference image.

10. The ultrasonic diagnosis apparatus according toclaim 9,

wherein the composite image is an image superimposing a translucent-color image of the reference image on the ultrasonic image.

11. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the composite image is a difference image between the reference image and the ultrasonic image.

12. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the reference image generating unit changes an image size of the reference image in accordance with a speed of movement of the ultrasonic probe.

13. The ultrasonic diagnosis apparatus according toclaim 1,

wherein the displaying unit displays a region of interest set on at least one of the ultrasonic image, the reference image, and a composite image of the ultrasonic image and the reference image on the other images.

Images