US20170143294A1 - Ultrasound diagnostic system, ultrasound image generation apparatus, and ultrasound image generation method - Google Patents

Abstract

The ultrasound diagnostic apparatus, ultrasound image generation apparatus and method transmit ultrasound waves to a subject into which a puncture tool is inserted, receive reflected waves reflected from the subject and the puncture tool, and generate echo signals of time-sequential frames based on the received reflected waves, and generate an ultrasound image of the subject based on the generated echo signals. These apparatus and method generate a differential echo signal between time-sequential frames from the echo signals, perform a tip detection process based on the differential echo signal to thereby detect at least one tip candidate including a tip end of the puncture tool, highlight a tip candidate of the puncture tool detected to thereby generate a tip image, and display the tip image of the highlighted puncture tool so as to be superimposed on the generated ultrasound image.

Description

CROSS-REFERENCE TO RELATED APPLICATION
• This is a Divisional of U.S. application Ser. No. 13/246,471 filed Sep. 27, 2011, which claims priority from Japanese Application No. 2010-216512 filed Sep. 28, 2010, Japanese Application No. 2010-216764 filed Sep. 28, 2010, and Japanese Application No. 2010-274872 filed Dec. 9, 2010, the disclosures of which are hereby incorporated by reference in their entireties.
BACKGROUND OF THE INVENTION
• The present invention relates to an ultrasound diagnostic apparatus, an ultrasound image generation apparatus, and an ultrasound image generation method. More particularly, the invention relates to an ultrasound diagnostic apparatus, an ultrasound image generation apparatus, and an ultrasound image generation method used when displaying a puncture tool on a screen together with body tissues, specifically when displaying the tip end of a puncture tool such as a puncture needle in an ultrasound image in an accurate and appropriate manner when performing paracentesis.
• In medical fields, an ultrasound image generation apparatus for generating ultrasound images and an ultrasound diagnostic apparatus using the generated ultrasound images have been put into practical use and are used widely for diagnosis and examination. In general, this kind of ultrasound image generation apparatus and ultrasound diagnostic apparatus (hereinafter referred collectively to as an ultrasound diagnostic apparatus) includes an ultrasound probe including an array of vibrators therein and a diagnostic apparatus main body connected to the ultrasound probe. In the ultrasound diagnostic apparatus, the ultrasound probe transmits ultrasound waves toward a subject to be examined, that is to say, a patient to thereby irradiate the patient with the ultrasound waves. Then, the ultrasound probe receives echoes (reflection sound) of the ultrasound waves reflected from the patient, namely signals (hereinafter referred to as echo signals) originating from ultrasound echoes from the subject. The received echo signals are electrically processed by the diagnostic apparatus main body whereby ultrasound tomographic images of the patient, called ultrasound images are generated and displayed on a monitor or the like.
• Moreover, the ultrasound diagnostic apparatus is also used when a physician performs a paracentesis which involves inserting a puncture tool, for example, a puncture needle, in a desired site to collect a tissue sample for the purpose of diagnosing cell tissues.
• In the paracentesis, it is important to enable the puncture needle to be observed on a monitor, namely as an image and to allow the puncture needle to reach a target object. To allow the puncture needle to certainly reach a target object or a target site, the physician inserts the puncture needle along a predetermined insertion path (a path along which the puncture needle is inserted within the body of a patient) while observing ultrasound images. Moreover, when establishing a definitive cancer diagnosis such as cell tissue diagnosis based on biopsy or the like, it is also important to capture and store an image in which the puncture needle is inserted into a target object as an evidence image.
• When performing such paracentesis, it is necessary to allow the puncture needle to reach a treatment target such as a target site or a target object and to perform drainage of excess fluid from the treatment target or perform injection (PEIT) of a substance to the treatment target. Thus, it is important to observe the puncture needle, particularly, the tip end thereof, on a monitor (ultrasound image) in a reliable, accurate, and appropriate manner.
• To solve this problem, according to an ultrasound imaging technique disclosed in Patent Document 1 (JP 4030644 B), in order to detect the accurate tip position of a puncture needle inserted into a subject to be examined, the puncture needle is inserted while being mechanically vibrated by a vibration imparting mechanism attached to the base end thereof, whereby a Doppler image of the puncture needle is obtained based on a Doppler signal and displayed so as to be superimposed on a B-mode image to thereby obtain the image of the puncture needle.
• According to an ultrasound diagnostic apparatus disclosed in Patent Document 2 (JP 2001-269339 A), B-mode ultrasound tomographic image frame data obtained by an ultrasound probe are stored in a memory, a difference between the previous frame data and the presently obtained frame data is calculated to obtain a spatial variation as digital data, and the digital data is added to the presently obtained frame data, whereby a desired puncture needle image is displayed in a B-mode ultrasound image. As a result, the technique ofPatent Document 2 can display a clear puncture needle image based on only the B-mode ultrasound tomographic image frame data without using the vibration imparting mechanism and Doppler-mode processing which were necessary in the ultrasound imaging technique disclosed inPatent Document 1.
• According to an ultrasound-guided puncture system disclosed in Patent Document 3 (JP 2000-107178 A), signals received by an ultrasound probe, transmitted from a subject to be examined into which a puncture needle is inserted are processed to generate B-mode image signals, and a B-mode ultrasound tomographic image is displayed on a display device based on the B-mode image signals. Moreover, a portion which has a higher luminance than the ultrasound image being displayed and in which the luminance varies abruptly is extracted and colored, and the colored extracted portion is displayed so as to be superimposed on an updated ultrasound tomographic image. As a result, the technique ofPatent Document 3 can provide an inexpensive ultrasound-guided puncture system capable of performing treatment and examination in a reliable and satisfactory manner without missing the tip position of a puncture needle inserted once into a patient.
• Moreover, Patent Document 4 (JP 2006-346477 A) discloses an ultrasound diagnostic apparatus as an apparatus for displaying a puncture guide line serving as a guide for inserting a puncture needle, in which an advancing angle of the puncture needle is calculated from a linear ultrasound echo signal of a predetermined length or longer, and a puncture guide line corresponding to the advancing angle is displayed so as to be superimposed on a B-mode image which is an ultrasound image generated from echo signals.
• Furthermore, Patent Document 5 (JP 8-299344 A) discloses an ultrasound diagnostic apparatus in which the amount of shift between an insertion path along which a puncture needle is inserted and a predetermined puncture guide line is detected, and the guide line is moved from a reference trajectory corresponding to a reference position of the puncture needle so as to match the ultrasound image of the puncture needle with the guide line to thereby display a corrected puncture guide line.
• In paracentesis, since the burden on a patient and the degree of invasiveness decrease as the puncture needle becomes narrower, a puncture needle which is as narrow as possible is used depending on a risk or the like. However, as the puncture needle becomes narrower, the ability to draw it on an ultrasound image also decreases, and the puncture needle is displayed in a disconnected manner.
• To solve this problem, Patent Document 6 (JP 2006-320378 A) discloses an ultrasound diagnostic apparatus in which a plurality of images are acquired by irradiating ultrasound waves in a direction where strong echo signals are obtained, and the images are combined and displayed so as to suppress a puncture needle from being displayed in a disconnected manner. Moreover, Patent Document 7 (JP 2007-222264 A) discloses an ultrasound diagnostic apparatus which displays a clear image of a tissue structure while suppressing speckles by adaptively changing image processing conditions in accordance with a local property of the tissue.
SUMMARY OF THE INVENTION
• However, in the technique disclosed inPatent Document 1, a special mechanism for mechanically vibrating the puncture needle is needed, which increases the size and cost of an apparatus. Moreover, when performing ultrasound-guided central venous puncture, there is a problem in that it is difficult to separate the Doppler signal from blood vessels and the Doppler signal from the puncture needle.
• Here, in paracentesis, since the burden on a patient and the degree of invasiveness decrease as the puncture needle becomes narrower, a puncture needle which is as narrow as possible is used depending on the risks or the like. However, as the puncture needle becomes narrower, the ability to draw it on an ultrasound image also decreases, and the puncture needle is displayed in a disconnected manner. Thus, there is a problem in that it is difficult to display the position or shape of the puncture needle precisely.
• Moreover, in the technique disclosed inPatent Document 2, it is described that it is possible to solve the problem associated with the technique disclosed inPatent Document 1 and display a clear puncture needle image based on only the B-mode ultrasound tomographic image frame data.
• However, as described above, when a narrow puncture needle is used, the B-mode ultrasound tomographic image frame data itself is data with which it is difficult to display the position or shape of the puncture needle precisely. Thus, there is a problem in that it is difficult to obtain accurate spatial variation data caused by insertion of the puncture needle from the differential data and to obtain an accurate puncture needle image.
• Furthermore, when inserting a puncture needle into a subject to be examined such as a patient or the like, the spatial variation may occur not only when only the position or shape of the puncture needle varies spatially but also when the subject moves with the movement of a patient or the like or when the position or shape of the subject itself varies with insertion of the puncture needle. In this case, the differential data inPatent Document 2 includes spatial variation data associated with not only the insertion of the puncture needle but also the movement of the subject or a variation in the subject itself. Thus, there is a problem in that it is difficult to separate only the spatial variation data associated with the insertion of the puncture needle and to separate only the puncture needle image. Moreover, since the B-mode ultrasound tomographic image frame data itself includes noise, the differential data also includes data resulting from noise. Thus, there is a problem in that unless a process of separating data resulting from noise is performed, it is difficult to obtain only the puncture needle image.
• Furthermore, in the technique disclosed inPatent Document 3, similarly to the technique disclosed inPatent Document 2, the latest image data is compared with image data one frame before to extract a varying portion in the high luminance portion of the ultrasound tomographic image, and the extracted portion is colored and displayed as the tip end portion of the puncture needle. However, in some cases, the varying portion of the high luminance portion may be present not only as a variation in the high luminance portion caused by the insertion of the puncture needle but also as a variation in the high luminance portion caused by the movement of the subject, a variation in the subject itself, or noise. In this case, there is a problem in that it is difficult to separate only the movement of the tip end portion of the puncture needle caused by the insertion and to separate only the tip end portion of the puncture needle.
• Moreover, in the technique disclosed inPatent Document 3, although the high luminance portion of the ultrasound tomographic image is a puncture needle, it is difficult to determine which high luminance portion corresponds to the puncture needle. As described above, when a narrow puncture needle is used, there is a problem in that it is difficult to extract only a high luminance portion corresponding to the puncture needle within an ultrasound tomographic image in which it is difficult to display the position or shape of the puncture needle precisely.
• However, in the ultrasound diagnostic apparatus of the related art, there is a problem in that it is difficult to draw a puncture needle on an ultrasound image, and the puncture needle is displayed in a disconnected manner, so that the accurate position of the puncture needle is not clear. Various reasons can be considered as the cause of this problem. For example, this problem may be caused due to the fact that since the puncture needle has a smooth surface, and scattering of ultrasound waves barely occurs, the intensity of echoes returning to a probe from the puncture needle which is inserted obliquely with respect to the direction of irradiating ultrasound waves decreases.
• Moreover, the ultrasound diagnostic apparatuses disclosed inPatent Documents 4 and 5 are designed to display the puncture guide line but do not address the problem in which the puncture needle is drawn in a disconnected manner. In particular, the ultrasound diagnostic apparatus ofPatent Document 4 does not have a function of correcting the ultrasound image of the puncture needle and the guide line, and the ultrasound diagnostic apparatus ofPatent Document 5 does not have a function of updating a reference position. Thus, even when the puncture guide line is displayed, the puncture needle may be bent when inserting it into a stiff tissue, and thus, the puncture needle may not be drawn along the puncture guide line. Moreover, although both ultrasound diagnostic apparatuses display the guide line of the insertion path, none of the systems takes a case of storing an evidence image into consideration and suggests which luminance on the ultrasound image corresponds to a luminance indicative of the puncture needle.
• Furthermore, although the ultrasound diagnostic apparatus disclosed inPatent Document 6 can display the puncture needle in a smoothly connected manner to some extent, there is a problem in that if the disconnected portions of the puncture needle in a plurality of images to be combined occur at the same position of the images, it is difficult to eliminate the disconnection even when the images are combined. Moreover, when puncturing into a stiff tissue, since the puncture needle is likely to be bent, and strong echo signals may not always be received, there is a problem in that the technique disclosed inPatent Document 6 is difficult to use. Furthermore, in the ultrasound diagnostic apparatus ofPatent Document 7, when the puncture needle is drawn in a disconnected manner in a state where the luminance thereof is lower than that of other tissues, although it is possible to display the tissue structure precisely, there is a problem in that it is difficult to display the position or shape of the puncture needle precisely.
• The invention has been made in view of the above problems, and a first object of the invention is to solve the problems of the related art and to provide an ultrasound diagnostic apparatus and an ultrasound image generation method capable of displaying the tip end of a puncture tool on an ultrasound image in an accurate, appropriate, and easily visible manner without using a special tool for mechanically vibrating the puncture tool such as a puncture needle and Doppler-mode processing when performing paracentesis.
• A second object of the invention is to provide an ultrasound image generation apparatus and an ultrasound image generation method capable of generating an image for presenting users with the position of a puncture tool such as a puncture needle in an accurate and reliable manner.
• A third object of the invention is to provide an ultrasound image generation apparatus and an ultrasound image generation method capable of generating an ultrasound image in which a puncture tool is displayed to be easily visible to users.
• According to a first aspect of the invention, it is possible to display the tip end of a puncture tool on an ultrasound image in an accurate, appropriate, and easily visible manner without using a special tool for mechanically vibrating the puncture tool such as a puncture needle and Doppler-mode processing when performing paracentesis.
• As a result, according to this aspect, even when a puncture tool such as a narrow puncture needle is used, it is possible to enable the tip end of the puncture tool to reach a target site in a reliable manner.
• According to a second aspect of the invention, it is possible to generate an image for presenting users with the position of a puncture tool such as a puncture needle in an accurate and reliable manner and to specify the accurate position of a puncture tool such as a puncture needle in a reliable manner.
• Moreover, according to this aspect, it is possible to detect feature points on a puncture needle which is a puncture tool and to connect these feature points into a line such as a straight line or a curve. Thus, even when the puncture needle is displayed in a disconnected manner, the connection of the puncture needle can be made easily understood.
• Furthermore, according to this aspect, the line of a puncture needle which is a puncture tool can be corrected in a time-sequential manner. Thus, even when an insertion path changes due to bending of the puncture needle, the presence of stiff tissues, shift of a probe, or the like, it is possible to draw the line of the puncture needle with high precision.
• Furthermore, according to this aspect, the connected line can be displayed with gradation by referencing the luminance of an ultrasound image. Thus, the connection of the puncture needle which is a puncture tool can be made easily understood while displaying the puncture needle with the luminance information of the ultrasound image.
• According to a third aspect of the invention, it is possible to generate an ultrasound image in which a puncture tool is displayed to be easily visible to users.
• Moreover, according to this aspect, it is possible to apply a puncture tool enhancement processing in accordance with the shape or insertion angle of the puncture tool.
• Furthermore, according to this aspect, the precision of the puncture tool enhancement processing can be increased by applying preprocessing such as speckle removal.
• Furthermore, according to this aspect, a process for enhancing the line of the puncture tool such as a puncture needle can be applied after the puncture tool enhancement filter is applied.
• Furthermore, according to this aspect, preprocessing for removing a layer structure other than the puncture tool such as a puncture needle can be performed.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a block diagram schematically showing a configuration of an embodiment of an ultrasound diagnostic apparatus according to a first aspect of the invention.
• FIG. 2 is a block diagram showing the details of an example of a puncture needle tip detector of a combined image generator of the ultrasound diagnostic apparatus shown inFIG. 1.
• FIG. 3 is a view showing an example of a differential image obtained by a differential image generator of the combined image generator of the ultrasound diagnostic apparatus shown inFIG. 1.
• FIG. 4 is a view showing an example of an ultrasound image showing an insertion trajectory of a puncture needle.
• FIGS. 5A and 5B are views showing an example of a differential image used by the puncture needle tip detector shown inFIG. 2 and an example of a tip enhancement filter applied to the differential image, respectively.
• FIG. 6 is a block diagram showing an example of a time-sequential frame differential image generator of the combined image generator shown inFIG. 1.
• FIG. 7 is a functional block diagram showing an example of a puncture needle processor of the time-sequential frame differential image generator shown inFIG. 6.
• FIGS. 8A, 8B, and 8C are views showing an example of one frame of ultrasound image processed by a filter application processor of the puncture needle processor shown inFIG. 7, an example of a puncture needle enhancement filter applied, and an example of a puncture needle enhanced ultrasound image after processing, respectively.
• FIG. 9 is a functional block diagram showing another example of the puncture needle processor of the time-sequential frame differential image generator shown inFIG. 6.
• FIG. 10 is a functional block diagram showing a detailed configuration of a puncture needle region specifying unit and a puncture needle tip position specifying unit of the puncture needle processor shown inFIG. 9.
• FIG. 11A shows an example of a B-mode image of the invention,FIG. 11B shows an example of an edge image obtained after performing threshold processing on the B-mode image shown inFIG. 11A,FIG. 11C shows an example of an edge image in which a line specified through Hough transform is displayed so as to be superimposed on the edge image shown inFIG. 11B, andFIG. 11D shows an example of an edge image in which a puncture needle presence region is displayed so as to be superimposed on the edge image shown inFIG. 11C.
• FIG. 12A is a schematic diagram showing a method of specifying a tip position in the puncture needle presence region shown inFIG. 11D, andFIG. 12B is a schematic diagram showing a line which connects the tip position of the puncture needle specified by the method ofFIG. 12A and the starting point of the puncture needle.
• FIG. 13 is a flowchart showing an example of main parts of an ultrasound image generation method according to the first aspect of the invention.
• FIGS. 14A and 14B are examples of two time-sequential frames of ultrasound images,FIG. 14C shows a differential image of the ultrasound images shown inFIGS. 14A and 14B,FIG. 14D shows an example of a tip enhancement filter for lookup table processing (LUT processing) to be applied to the differential image shown inFIG. 14C, andFIG. 14E shows an example of a LUT processed image of the differential image shown inFIG. 14C.
• FIG. 15 is a view showing an example of a Gaussian filter used as the base that determines a filter coefficient of the tip enhancement filter shown inFIG. 14D.
• FIG. 16 is a view showing an example of the distribution of tip candidate points of a puncture needle after binarization of the differential image shown inFIG. 14C.
• FIG. 17 is a flowchart showing another example of main parts of the ultrasound image generation method according to the first aspect of the invention.
• FIG. 18 is a block diagram schematically showing a configuration of another embodiment of a diagnostic apparatus main body of the ultrasound diagnostic apparatus according to the first aspect of the invention.
• FIG. 19 is a functional block diagram schematically showing a configuration of an embodiment of an ultrasound image generation apparatus according to a second aspect of the invention.
• FIG. 20 is a flowchart showing an example of the flow of a series of processes related to an operation of displaying a puncture needle image so as to be superimposed on an ultrasound image in an ultrasound image generation method according to the second aspect of the invention.
• FIG. 21 is a flowchart showing an example of the flow of a process of extracting puncture needle candidate points in the ultrasound image generation method shown inFIG. 20.
• FIG. 22A is a schematic view showing an example of a puncture guide line within a B-mode image,FIG. 22B is a schematic view showing an example of a predicted insertion region when a predicted insertion region is set to a narrow range,FIG. 22C is a schematic view showing an example of a predicted insertion region when a predicted insertion region is set to an average range, andFIG. 22D is a schematic view showing an example of a predicted insertion region when a predicted insertion region is set to a wide range.
• FIG. 23 is a schematic view showing an example of a region of interest within a B-mode image.
• FIG. 24A is a view of a B-mode image on which a line representing a puncture needle is displayed in a superimposed manner,FIG. 24B is a view of a B-mode image on which a line representing a puncture needle is displayed semitransparently in a superimposed manner,FIG. 24C is a view of a B-mode image on which needle candidate points are colored and displayed in a superimposed manner, andFIG. 24D is a view of a B-mode image on which the outline of a puncture needle is displayed in a superimposed manner.
• FIG. 25A is a view showing a case of determining a predicted insertion region based on past three tip positions of a puncture needle, andFIG. 25B is a view showing a case of determining a predicted insertion region based on past four tip positions of a puncture needle.
• FIG. 26A is a view of a puncture needle presence region created by vertically expanding a puncture needle candidate line by the same width,FIG. 26B is a view of a puncture needle presence region shifted upward,FIG. 26C is a view of a puncture needle presence region shifted downward, andFIG. 26D shows a case where the slope of a puncture needle candidate line is different from the slope of a puncture needle presence region.
• FIG. 27 is a functional block diagram of an embodiment of an ultrasound image generation apparatus according to a third aspect of the invention.
• FIG. 28 is a functional block diagram showing a detailed configuration of a puncture tool enhancement data generator of the ultrasound image generation apparatus shown inFIG. 27.
• FIG. 29 is a flowchart showing an example of an operation of the ultrasound image generation apparatus according to the third aspect of the invention and an example of the ultrasound image generation method according to the invention.
• FIG. 30 is a flowchart showing the details of a step of generating puncture tool enhancement data in the ultrasound image generation method shown inFIG. 29.
• FIG. 31A shows an example of a puncture tool enhancement filter according to a first embodiment of the present aspect when an insertion angle is 10°,FIG. 31B is an enlarged view of a region in which a puncture needle is disconnected within a B-mode image, andFIG. 31C shows an example of a puncture tool enhancement filter according to the first embodiment when an insertion angle is 30°.
• FIG. 32A shows filter coefficients which are uniformly allocated to respective elements, andFIG. 32B shows filter coefficients of a Gaussian filter.
• FIG. 33A shows an example of an ultrasound image before a puncture tool enhancement processing, andFIG. 33B shows a combined ultrasound image after a puncture tool enhancement processing.
• FIG. 34 shows an example of a puncture tool enhancement filter according to a second embodiment when an insertion angle is 10°.
• FIG. 35A shows an example of an ultrasound image before a puncture tool enhancement processing, andFIG. 35B shows a combined ultrasound image after a puncture tool enhancement processing.
• FIG. 36 is a functional block diagram of another embodiment of the ultrasound image generation apparatus according to the third aspect of the invention.
• FIG. 37 is a functional block diagram showing a detailed configuration of a puncture tool enhancement data generator of the ultrasound image generation apparatus shown inFIG. 36.
• FIG. 38 shows an example of an ultrasound image rotated so that an image representing a puncture tool becomes horizontal.
• FIG. 39 is a view showing an example of a Gaussian filter serving as the base that determines a filter coefficient of the puncture tool enhancement filter according to a third embodiment of the present aspect.
DETAILED DESCRIPTION OF THE INVENTION
• An ultrasound diagnostic apparatus, an ultrasound image generation apparatus, and an ultrasound image generation method according to the invention will be described in detail based on preferred embodiments shown in the accompanying drawings.
• FIG. 1 is a block diagram schematically showing a configuration of an embodiment of an ultrasound diagnostic apparatus according to a first aspect of the invention, which performs an ultrasound image generation method according to the first aspect of the invention.FIG. 2 is a block diagram showing the details of an embodiment of a puncture needle tip detector of a combined image generator of the ultrasound diagnostic apparatus shown inFIG. 1.
• An ultrasounddiagnostic apparatus10 of the present aspect is an apparatus which irradiates (transmits) ultrasound waves to a subject to be examined, in particular, the subject into which a puncture tool (not shown) such as a puncture needle is inserted, generates and displays ultrasound images obtained by receiving ultrasound waves (echoes) reflected from the subject and the puncture tool, in particular, ultrasound images in which the tip end portion of the puncture tool is combined, and provides the ultrasound images for diagnosis of ultrasound images. The ultrasounddiagnostic apparatus10 includes anultrasound probe12 and a diagnostic apparatusmain body14 to which theultrasound probe12 is connected. In the following description, although a puncture needle is described as a representative example of the puncture tool, the puncture tool is not limited to this.
• Theultrasound probe12 is also called a probe and used by being pressed against a subject to be examined. Theultrasound probe12 is configured to transmit and receive ultrasound waves and output received echo signals to the diagnostic apparatusmain body14 and includes aprobe body16, acommunication cable18, and apuncture adapter20.
• Theprobe body16 is a transducer which converts electrical signals into ultrasound waves to irradiate (transmit) the ultrasound waves and receives ultrasound waves reflected from the subject to convert the ultrasound waves into electrical signals (echo signals). Basically, theprobe body16 is a known ultrasound probe and may be any scanning-type ultrasound probe of such as, for example, a linear scanning type, a convex scanning type, or a sector scanning type.
• A detailed configuration of theprobe body16 will be described later.
• Theprobe body16 includes an ultrasound wave transceiving surface (not shown) for transmitting and receiving ultrasound waves. Thecommunication cable18 is connected to a surface of theprobe body16 opposite to the ultrasound wave transceiving surface, and thepuncture adapter20 is disposed on a side surface of the ultrasound wave transceiving surface.
• Thecommunication cable18 is configured to transmit the echo signals from theprobe body16 to the diagnostic apparatusmain body14.
• Thepuncture adapter20 serves as a guide for inserting a puncture tool such as a puncture needle into a subject to be examined when performing paracentesis using the ultrasounddiagnostic apparatus10. A guide groove (not shown) for inserting the puncture needle into the subject at a predetermined angle is formed in thepuncture adapter20, and the puncture needle is inserted along the guide groove whereby it is inserted into the subject at a predetermined angle. That is, the angle (hereinafter referred to as an insertion angle) at which the puncture needle is inserted into the subject is determined by the angle of the guide groove of thepuncture adapter20 with respect to the subject. The guide groove of thepuncture adapter20 is configured to be able to change the angle with respect to the subject and to adjust the insertion angle.
• Thepuncture adapter20 is preferably connected physically and electrically to theprobe body16. In this case, it is possible to store information on the insertion angle in thepuncture adapter20. When thepuncture adapter20 is physically connected to theprobe body16, it is possible to output a signal representing the insertion angle to theprobe body16 since thepuncture adapter20 is also electrically connected to theprobe body16. Moreover, whenever the angle of the groove with respect to the subject changes, thepuncture adapter20 is capable of outputting the signal representing the present insertion angle to theprobe body16.
• As shown inFIG. 1, theprobe body16 includes a plurality ofultrasound transducers34, a plurality ofreception signal processors36, atransmission driver38, atransmission controller40, areception controller42, and aprobe controller44.
• The plurality ofultrasound transducers34 form a 1D or 2D array of vibrators, and each of the plurality ofreception signal processors36 is connected so as to correspond to each of the plurality oftransducers34. Moreover, thetransmission controller40 is connected to the plurality oftransducers34 through thetransmission driver38, thereception controller42 is connected to the plurality ofreception signal processors36, and thetransmission controller40 and thereception controller42 are connected to theprobe controller44.
• Moreover, thereception signal processors36 are connected to adata storage unit46 of the diagnostic apparatusmain body14 through the communication cable, and theprobe controller44 is connected to amain body controller54 through the communication cable.
• Each of the plurality oftransducers34 transmits ultrasound waves toward the subject in accordance with a driving signal supplied from thetransmission driver38, receives ultrasound echoes from the subject, and outputs the reception signals. Eachtransducer34 is formed of a vibrator in which an electrode is formed on both ends of a piezoelectric body that is formed of piezoelectric ceramics represented by PZT (lead zirconate titanate) or a polymer piezoelectric element represented by PVDF (polyvinylidene fluoride), for example.
• When a pulsating or continuous voltage is applied to the electrodes of each vibrator, the piezoelectric body of the vibrator expands and contracts, pulsating or continuous ultrasound waves are generated from each vibrator, and an ultrasound beam is formed by the combination of these ultrasound waves. Moreover, each vibrator receives a propagating ultrasound wave and expands and contracts to generate electrical signals, and these electrical signals are output as reception signals of ultrasound waves.
• Thetransmission driver38 includes a plurality of pulsers each generating a high-voltage electrical signal serving as a driving signal for generating ultrasound waves, for example. Thetransmission driver38 supplies the driving signals to the plurality oftransducers34 by adjusting the amount of delay thereof based on a transmission delay pattern selected by thetransmission controller40 so that the ultrasound waves transmitted from the plurality oftransducers34 form an ultrasound beam having a large width covering the area of tissues within the subject.
• Thereception signal processor36 of each channel performs various processing on the reception signal output from the correspondingtransducer34 under the control of thereception controller42 to thereby generate a complex baseband signal. The processing includes amplification by an amplifier, rejection of high-frequency components by a low-pass filter, A/D conversion by an A/D converter, and quadrature detection or quadrature sampling. Thereception signal processor36 samples the complex baseband signal to thereby generate sample data including information on the area of tissues. Thereception signal processor36 may perform data compression processing for high-efficiency encoding on the data obtained by sampling the complex baseband signal to thereby generate the sample data.
• In this specification, signals may be used as those which mainly represent signal levels (signal values) in hardware, and data may be used as those which are processed by software and represent magnitude (data values).
• Theprobe controller44 controls respective parts of theprobe body16 based on various control signals transmitted from the diagnostic apparatusmain body14.
• As shown inFIG. 1, the diagnostic apparatusmain body14 includes thedata storage unit46, a combinedimage generator48, adisplay controller50, adisplay unit52, themain body controller54, anoperation unit56, and astorage unit58.
• In the diagnostic apparatusmain body14, thedata storage unit46 is connected to the combinedimage generator48 and the plurality ofreception signal processors36 of theprobe body16. The combinedimage generator48 is connected to thedisplay unit52 through thedisplay controller50. Themain body controller54 is connected to the combinedimage generator48 and thedisplay controller50. Moreover, themain body controller54 is connected to theoperation unit56 and thestorage unit58.
• Thedata storage unit46 is formed of a memory, a hard disk, or the like, and stores at least one frame of sample data which are time-sequentially transmitted from thereception signal processors36 of theultrasound probe12 through thecommunication cable18.
• The combinedimage generator48 performs reception focusing processing on sample data of each frame read from thedata storage unit46 to thereby generate image data (B-mode image data or signal) of an ultrasound image which is a B-mode image of one frame. In particular, the combinedimage generator48 generates image data (image signal) of an ultrasound diagnostic image such as a combined ultrasound image in which a tip image which is the image of a highlighted puncture needle tip is combined. Here, the B-mode image data is so-called ultrasound image data and means image data which represents the amplitude of an acoustic-ray signal by luminance, and the B-mode image means a so-called ultrasound image.
• In this specification, since an image can be considered as a collection of image data or image signals of respective pixels, a collection of image data or image signals representing an image is also simply referred to as an image.
• The details of the combinedimage generator48 will be described later.
• Thedisplay controller50 performs control based on the ultrasound image signal generated by the combinedimage generator48 so as to cause thedisplay unit52 to display an ultrasound image, in particular, an ultrasound diagnostic image in which the puncture needle tip is highlighted. Thedisplay controller50 includes a DSC (Digital Scan Converter). In thedisplay controller50, the DSC converts (rasterizes) the ultrasound image signal into an image signal corresponding to a general television signal scanning format and performs necessary image processing such as gradation processing to thereby convert the rasterized image signal into a display image signal for display on thedisplay unit52.
• Thedisplay unit52 displays an ultrasound image based on the display image signal converted by thedisplay controller50, and includes a display device or a monitor such as an LCD, for example. Thedisplay unit52 displays the highlighted tip end portion of the puncture needle so as to be superimposed on the ultrasound image under the control of thedisplay controller50.
• Theoperation unit56 enables an operator to input instructions for operating the ultrasounddiagnostic apparatus10 and is a unit that sets imaging menus, imaging conditions, and the like and instructs imaging of a subject to be examined. Theoperation unit56 includes various input means such as input keys, dial buttons, a trackball, a touch panel, and the like for setting imaging menus, imaging conditions, and the like.
• Moreover, theoperation unit56 also has a function of inputting and setting the position of a target (target site) and inputting instructions regarding the settings on an insertion angle. Furthermore, theoperation unit56 may also include a function of inputting instructions regarding an insertion position of a puncture needle. Theoperation unit56 supplies the input instructions regarding the settings on the target position, the insertion angle, and the insertion position to themain body controller54.
• Themain body controller54 controls the respective parts within the diagnostic apparatusmain body14 including the combinedimage generator48 and thedisplay controller50. Themain body controller54 is connected to theprobe controller44 of theprobe body16 through thecommunication cable18 and supplies a control signal for controlling the operation of theprobe body16 to theprobe controller44.
• Thestorage unit58 is formed of a memory, a hard disk, or the like, and stores an operation program for operating the respective parts within the diagnostic apparatusmain body14 including the combinedimage generator48 and thedisplay controller50 which are controlled by themain body controller54. Themain body controller54 reads the operation program for operating the respective parts within the diagnostic apparatusmain body14 from thestorage unit58 as necessary and operates the respective parts within the diagnostic apparatusmain body14 in accordance with the read operation program.
• The combinedimage generator48 includes animage generator64, a time-sequential frame image storage unit (hereinafter also referred to simply as an image storage unit)66, a time-sequential frame differential image generator (hereinafter also referred to simply as a differential image generator)68, a puncture needle tip detector (hereinafter also referred to simply as a tip detector)70, and animage combiner72.
• Theimage generator64 generates a B-mode image signal which is tomographic image information from the sample data of each frame read from thedata storage unit46 and includes aphasing addition unit60 and animage processor62.
• Thephasing addition unit60 selects one reception delay pattern from a plurality of reception delay patterns stored in advance in accordance with a reception direction set in themain body controller54, gives respective delays to the plurality of complex baseband signals represented by the sample data of each frame read from thedata storage unit46 based on the selected reception delay pattern, and adds (phasing addition) the complex baseband signals after matching phases, thereby performing reception focusing processing (beam forming). By the reception focusing processing in thephasing addition unit60, a baseband signal (acoustic-ray signal) in which ultrasound echoes are well focused is generated for each frame, that is, a so-called echo signal of each frame is generated.
• Theimage processor62 generates a B-mode image signal which is tomographic image information on a tissue within a subject to be examined based on the echo signal (acoustic-ray signal) of each frame generated by thephasing addition unit60. Theimage processor62 includes a band-pass filter, a high-frequency amplifier including a STC (Sensitivity Time Control) unit, a logarithmic amplifier, a luminance converter, and the like.
• Here, the band-pass filter varies a pass band in accordance with a propagation time of an ultrasound echo to improve the S/N ratio. The STC unit of the high-frequency amplifier controls an amplification gain in accordance with the propagation time to correct attenuation of the echo signal (acoustic-ray signal) based on a distance in accordance with the depth of the reflection position of the ultrasound wave. The logarithmic amplifier amplifies the echo signal by limiting a variation range of the amplitude which varies over a wide range. The luminance converter converts the amplitude into luminance to thereby generate a B-mode image signal for each frame which is tomographic image information, and which makes the echo signal displayed as one luminance-modulated emission line.
• The time-sequential frameimage storage unit66 is a memory that time-sequentially stores the B-mode image signals representing images of a plurality of frames as time-sequential frame images. The time-sequential frameimage storage unit66 is formed of a memory, a hard disk, or the like, similarly to thedata storage unit46.
• The time-sequential framedifferential image generator68 calculates a difference between two time-sequential frame images (B-mode image signals) stored in theimage storage unit66 to thereby generate a differential image (differential image signal).
• A preferred detailed configuration of thedifferential image generator68 will be described later.
• The punctureneedle tip detector70 which is the most characteristic portion of the first aspect of the invention performs a process of detecting a tip end from the differential image generated by thedifferential image generator68. For example, the punctureneedle tip detector70 detects at least one tip candidate including a puncture needle tip using a difference in luminance of respective pixels of the differential image, highlights the detected tip candidates, and generates a highlighted tip image.
• In thetip detector70, as shown inFIG. 3, when apuncture needle162 which is inserted is generated in adifferential image160, since the amount of displacement of atip end164 of thepuncture needle162 is obtained as a difference, it is possible to detect only thetip end164 which is the most important in the puncturing of thepuncture needle162. It is most preferable for thetip detector70 to detect only onetip candidate166 corresponding to thetip end164 of thepuncture needle162. However, since thedifferential image160 includes a movement of a puncture target or noise other than the displacement of thepuncture needle162, a plurality oftip candidates166 are detected. Thus, as described above, it is not always possible to detect only one tip candidate corresponding to thetip end164 of thepuncture needle162.
• Therefore, in the present aspect of the invention, when the number of tip candidates detected by thetip detector70 is one or a predetermined number and for example, six or less, it is preferable to color these tip candidates and/or increase the luminance thereof as a highlighting process to thereby generate a color and/or high-luminance tip image. When a lot of tip candidates, for example, exceeding six, are detected by thetip detector70, it is preferable to eliminate tip candidates of low reliability to narrow the tip candidates down to one to six tip candidates. Naturally, it is most preferable to narrow to one tip candidate. Similarly, it is preferable to perform a highlighting process on the narrowed tip candidates to generate a color and/or high-luminance tip image.
• The detailed configuration of thetip detector70 will be described later.
• Theimage combiner72 combines the tip image (needle tip enhanced image) generated by thetip detector70 with an ultrasound image which is the B-mode image generated by theimage processor62 to thereby generate a combined ultrasound image.
• The combined ultrasound images (image signals) generated by theimage combiner72 are transmitted to thedisplay controller50.
• In the diagnostic apparatusmain body14 of the ultrasounddiagnostic apparatus10 of the shown example, thedisplay controller50 includes the DSC for converting the combined ultrasound images (image signals) combined by theimage combiner72 into display image signals for display on thedisplay unit52. However, the invention is not limited to this. Theimage combiner72 may include the DSC, and the DSC may convert (rasterize) the combined ultrasound images (image signals) combined by theimage combiner72 into image signals corresponding to a general television signal scanning format and perform necessary image processing such as gradation processing to thereby generate B-mode image signals for display on a monitor.
• Moreover, theimage processor62 may include the DSC, and the DSC may convert (rasterize) the echo signals (acoustic-ray signals) corrected by the STC unit or the luminance-modulated B-mode image signals into image signals corresponding to a general television signal scanning format and perform necessary image processing such as gradation processing to thereby generate B-mode image signals for display on a monitor. In this case, if the display image signals are used for detecting the tip end portion of the puncture needle, thedisplay controller50 and theimage combiner72 may not include the DSC.
• Next, the detailed configuration of the punctureneedle tip detector70 which is the most characteristic portion of the present aspect of the invention will be described.
• As shown inFIG. 2, the punctureneedle tip detector70 includes atip candidate detector74, atip candidate processor76, a tipcandidate storage unit78, and a puncture needle information and condition storage unit (hereinafter referred to simply as an information storage unit)80.
• Thetip candidate detector74 performs a tip detection process on the differential image generated by thedifferential image generator68 to detect at least one tip candidate including the puncture needle tip. Thetip candidate detector74 includes a candidate point extraction processor (hereinafter also referred to as an extraction processor)82 and a tip candidate specifying processor (hereinafter also referred to as a specifying processor)84.
• As an example of the tip detection process, first, the candidatepoint extraction processor82 extracts points having luminance difference values satisfying predetermined conditions based on a luminance difference or a luminance value of the differential image as tip candidate points of the puncture needle. For example, the candidatepoint extraction processor82 performs binarization, filter processing, and LUT (lookup table) processing for gradation processing or the like to extract regions or portions (a collection of pixels) in which the luminance difference in relation to the luminance of neighboring portions is equal to or larger or smaller than a predetermined value, or regions of which the luminance values are equal to or larger or smaller than a predetermined value as the tip candidate points of the puncture needle.
• In theextraction processor82, it is preferable to select and extract the tip candidates of the puncture needle based on the density or size of the tip candidate points or regions detected in accordance with the luminance difference or the like of the differential image. Specifically, median filtering may be used, or alternatively, the sum of the luminance values of pixels near a predetermined point may be calculated, and locations having a large luminance sum or portions of which the luminance sum exceeds a predetermined threshold may be extracted as tip candidate points or detected as tip candidates.
• Moreover, in theextraction processor82, it is preferable to perform LUT processing after setting a tip candidate extraction region in advance by referencing the positions of tip candidates detected in the past, stored in the tipcandidate storage unit78. In particular, theextraction processor82 may extract or detect regions of which the luminance difference is equal to or larger than a predetermined value from regions near a line that connects two or more tip candidates of the puncture needle detected in the past, as the tip candidate points of the puncture needle or the tip candidates.
• By doing so, it is possible to find the tip candidate points or the tip candidates taking the advancing movement or direction of the puncture needle into consideration. Thus, it is possible to extract and detect the tip candidate points and the tip candidates more accurately.
• When a plurality of tip candidate points of the puncture needle are extracted by theextraction processor82, the tipcandidate specifying processor84 performs a process of eliminating tip candidate points of low reliability as one of the tip detection processes. For example, the tipcandidate specifying processor84 specifies only the central points of regions having a high correlation as tip candidates to thereby narrow the number of tip candidate points down to a predetermined number, for example, one to six, and most preferably to one, and specifies the narrowed tip candidate points as tip candidates to be detected.
• As the process of eliminating tip candidate points of low reliability, when a plurality of tip candidate points remain after the LUT processing by theextraction processor82, it is preferable for the specifyingprocessor84 to narrow the tip candidate points so as to include points of which the distance to the positions of a plurality of tip candidates detected in the past is minimized. That is, a plurality of past detection results for the tip candidates of the puncture needle may be stored in the tipcandidate storage unit78. For example, as shown inFIG. 4, candidate points among the presently detected tip candidate points, of which the distance to detection result points (tip candidates) of five frames is minimized, may be specified as present detection results. Such a method is effective when the specifyingprocessor84 narrows the tip candidate points when a plurality of tip candidate points are extracted in the extraction of simple tip candidate points by theextraction processor82 as shown inFIG. 4.
• When the number of tip candidate points of the puncture needle extracted by theextraction processor82 is a predetermined number, for example, one to six, as described above, the specifyingprocessor84 may specify all the candidate points as tip candidates of the puncture needle to be detected, and may further narrow the candidate points until the number reaches a smaller number, and preferably one.
• When the number of tip candidate points extracted by theextraction processor82 is always smaller than a predetermined number of tip candidates to be detected by thetip candidate detector74, for example, one to six, the specifyingprocessor84 may not be provided.
• As the result of the LUT processing by theextraction processor82, when it was not possible to extract any tip candidate point of the puncture needle, the tip candidates detected previously or in the past, stored in the tipcandidate storage unit78 may be extracted as the tip candidate points to be detected presently or detected as the tip candidates, or new tip candidate points or new tip candidates may be estimated from the tip candidates detected in the past. The tip candidate points extracted or estimated by theextraction processor82 are specified by the specifyingprocessor84 as tip candidates to be detected presently. Moreover, the tip candidates themselves detected in the past or the tip candidates estimated may directly be specified by the specifyingprocessor84 as tip candidates to be detected presently.
• In such a case, since it can be considered that the tip end was not moved due to the insertion of the puncture needle, it can be understood that the tip candidates detected previously or in the past can be used.
• Moreover, in thetip candidate detector74, even when it was not possible to detect optimal tip candidates in the present frame, past detection results stored in the tipcandidate storage unit78 may be displayed on thedisplay unit52. For example, when there are a plurality of past detection results, since it is possible to calculate the equation of a line and the insertion speed of the puncture needle, points predicted from these two factors may be displayed as tip candidates. Alternatively, points which were successfully detected at the last time by thetip candidate detector74 may continue to be displayed as tip candidates as they were.
• In this way, thetip candidate detector74 detects a predetermined number of, for example, one to six, tip candidates.
• In the invention, it is preferable for thetip candidate detector74 to detect at least six tip candidates. This is because the present inventors have confirmed that when at least six tip candidates are detected, it is highly probable that the puncture needle tip is included within the six tip candidates.
• In the above example, theextraction processor82 and the specifyingprocessor84 perform binarization, filtering, and LUT processing such as gradation processing as a tip detection process. However, the invention is not limited to this. In order that theextraction processor82 extracts the predetermined number of tip candidate points of the puncture needle, LUT processing such as gradation processing may be performed after binarization, and a predetermined number of tip candidate points may be selected in descending order of luminance difference.
• Moreover, the LUT used for the LUT processing in theextraction processor82 or the specifyingprocessor84 may be adjusted in accordance with one or both of the ultrasound images (B-mode images) generated by theimage generator64 and the differential images generated by thedifferential image generator68. That is, in the LUT processing, it is preferable to use a tip enhancement filter which is weighted in the insertion direction of the puncture needle and has a size such that the puncture needle tip is included. Such a tip enhancement filter can be referred to as a filter that detects the movement of the puncture needle since it is applied to a differential image.
• Examples of such a tip enhancement filter include a filter which has a step shape and uses pixels located in the insertion direction of the puncture needle in weighted addition and a filter which has a rectangular shape and performs weighted addition so that pixels located in the insertion direction of the puncture needle have a large filter coefficient.
• Here, an example of a differential image including the image of the puncture needle tip and an example of a tip enhancement filter used when thetip detector70 performs LUT processing on the differential image are shown inFIGS. 5A and 5B, respectively.
• A tip enhancement filter having a size of 81×81 pixels shown inFIG. 5B is a rectangular filter which is weighted in the insertion direction of the puncture needle in the differential image shown inFIG. 5A, has a size such that the puncture needle tip is included, and performs weighted addition so that pixels located in the insertion direction have a large filter coefficient.
• Such a tip enhancement filter is made up of an odd number of pixels in both vertical and horizontal directions so that a target pixel is located at the center, and the filter coefficients of respective pixels can be determined by applying a Gaussian function expressed byEquation 1 below. That is, the tip enhancement filter shown inFIG. 5B is a filter created by applying the Gaussian function expressed byEquation 1 below.
• $\begin{array}{cc}f\left(x,y\right)=\frac{1}{2{\mathrm{\pi \sigma }}_x{\sigma }_y\sqrt{1-{\mathrm{<figure-callout\; label="\rho \; xy"\; filenames="US20170143294A1-20170525-D00007.png,US20170143294A1-20170525-D00014.png"\; state="\{\{state\}\}">\rho </figure-callout>}}_{\mathrm{xy}}^{}}}\exp \left(-\frac{1}{2\left(1-{\rho }_{\mathrm{xy}}^2\right)}\left\{\frac{{\left(x-{\mu }_x\right)}^2}{{\sigma }_x^2}+\frac{{\left(y-{\mu }_x\right)}^2}{{\sigma }_y^2}-\frac{2{\rho }_{\mathrm{xy}}\left(x-{\mu }_x\right)\left(y-{\mu }_y\right)}{{\sigma }_x{\sigma }_y}\right\}\right)& \left(1\right)\end{array}$
• Here, f(x,y) is a filter coefficient, μ_xand μ_yare averages in x and y directions, σ_xand σ_yare variances in x and y directions, and ρ is a correlation value. When μ_x=μ_y=0, σ_x²=σ_y²=40, and ρ=0.9, it is possible to create a filter having the size of 81×81 pixels schematically shown inFIG. 5B. FIG.5B shows the magnitudes of the filter coefficients as the thickness of density. In this way, it is possible to create filter coefficients so that pixels located in the insertion direction of the puncture needle have a large filter coefficient.
• InFIG. 5B, the filter coefficient increases as it approaches the center, and the filter coefficients on an ellipse concentric about the central pixel are the same. InFIG. 5B, for better understanding of the invention, the concentric ellipse is depicted with a boundary, although there is actually no boundary as shown. The longitudinal direction of the ellipse is identical to the insertion direction of the puncture needle. That is, by increasing the filter coefficients located in the insertion direction of the puncture needle, weighted addition is performed so that the pixels located at the position where the possibility of presence of the puncture needle is high have a large filter coefficient.
• The tip enhancement filter, having an aspect ratio corresponding to the insertion angle, created in this way is stored in theinformation storage unit80.
• In the present aspect of the invention, a tip enhancement filter to be used is determined in accordance with the insertion angle and stored in theinformation storage unit80. Thetip candidate detector74 can detect tip candidates which are candidates of high reliability for the puncture needle tip by performing a tip enhancement processing of performing weighted addition on the differential image with neighboring pixels using the tip enhancement filter stored in theinformation storage unit80.
• When calculating the density or size of a region detected in the differential image, a filter having such a size that the puncture needle tip is included like the tip enhancement filter shown inFIG. 5B can be created in the following way.
• A plurality of filter sizes in which the puncture needle tip is included may be prepared in the manner of “Large,” “Medium,” and “Small” in accordance with the thickness of the puncture needle, for example, and a plurality of sizes may be prepared between the angles of 10° and 60° in the manner of “10°,” “30°,” and “60°” in accordance with the insertion angle of the puncture needle. Moreover, the size may be determined based on the size (G) of the puncture needle, the purpose of puncture (FNA (Fine Needle Aspiration Cytology), CNB (Core Needle Biopsy), RFA (Radio-Frequency Ablation), and the like), and puncture information such as an insertion angle.
• Moreover, as described above, weights of the filter may use a Gaussian function or the like. In this case, the proportions of weights in averaging, variance, correlation, and the like may be prepared as parameters which can be changed by a user.
• Moreover, the size of the filter and the weights of the weighted filter may be changed by a user selecting in advance on a setting screen, or may be changed during scanning of theultrasound probe12 by allocating a function to a function key or the like.
• Examples of the tip enhancement filter used in the present aspect of the invention include various puncture tool enhancement filters which are used in a puncture tool enhancement processing applied to an ultrasound image generation apparatus and an ultrasound image generation method of the third aspect of the invention described later.
• Moreover, when theextraction processor82 or the specifyingprocessor84 performs LUT processing to extract the tip candidate points of the puncture needle or detect the tip candidates, a region of the differential image, which is located around the tip candidate points and has a size obtained by applying the tip enhancement filter may be colored. By doing so, it is possible to detect tip candidate points within the colored region as tip candidates, and to increase the reliability or probability of the detected tip candidates being the puncture needle tip.
• Furthermore, preferably, theextraction processor82 and/or the specifyingprocessor84 of thetip candidate detector74 searches a region of the differential image based on a frame displayed on thedisplay unit52, located near the tip candidates of the puncture needle detected at a point in time earlier than the displayed frame, preferably a region near a line connecting tip candidates of the puncture needle detected at two points in time earlier than the displayed frame to thereby detect tip candidates of the puncture needle from the differential image based on the displayed frame. By doing so, it is possible to detect tip candidates having a high probability of being the puncture needle tip taking the movement, or displacement, of the puncture needle inserted into a target site into consideration.
• Thetip candidate processor76 performs a process of highlighting a predetermined number of tip candidates, that is to say, at least one tip candidate detected by thetip candidate detector74 or the coordinates thereof to generate the image of the highlighted tip candidate of the puncture needle, namely a tip image in which the tip candidate of the puncture needle is highlighted, in order to make the image easily identified by an operator when it is displayed on thedisplay unit52.
• In thetip candidate processor76, it is preferable to color the tip candidate of the puncture needle detected by thetip candidate detector74 to generate a color tip image for color display as the highlighting process. Alternatively, it is preferable to increase the luminance of the tip candidate to generate a high-luminance tip image. It is also preferable to increase the luminance after coloring the tip candidate to generate a high-luminance color tip image.
• In addition, although the number of tip candidates or coordinates thereof which are colored or brightened is preferably one, the number may be two or more, and the tip candidates detected in the past or the coordinates thereof may be used. Moreover, as shown inFIG. 4, thetip candidate processor76 may highlight and display the tip image of the puncture needle detected in the past on thedisplay unit52 in a superimposed manner so that the trajectory of the puncture needle is displayed.
• The colored or brightened region of the tip candidate of the puncture needle or the coordinates thereof may be the region of the tip candidate itself, and may be an optional region of the coordinates of the tip candidate, for example, a rectangular, elliptical, circular, or square region including the coordinates. Moreover, the size of the region can be set in advance, and may be changed during scanning of theultrasound probe12 by allocating a function to a function key or the like.
• The colored or brightened region may be weighted by the tip enhancement filter used when thetip candidate detector74 detects the tip candidates.
• Here, preferably, thetip candidate detector74 determines the positive/negative sign of the luminance difference of the differential image, and thetip candidate processor76 highlights the tip candidate of the puncture needle detected by thetip candidate detector74 in accordance with the positive/negative sign of the luminance difference of the differential image determined by thetip candidate detector74. More preferably, thetip candidate processor76 changes the color and luminance used for highlighting the tip candidate of the puncture needle detected by thetip candidate detector74. In this way, by displaying the highlighted tip candidate on thedisplay unit52 by changing the color and luminance thereof, it is possible to display the puncture needle tip so as to be visible as it is inserted into a target site when the luminance difference of the differential image is positive, for example. Moreover, it is possible to display the puncture needle tip so as to be visible as it is pulled out when the luminance difference is negative.
• Moreover, thetip candidate detector74 may not determine the positive/negative sign of the luminance difference of the differential image, but thetip candidate processor76 may generate a color tip image which is colored or brightened based on the absolute value of the luminance difference and display the tip image on thedisplay unit52.
• The tipcandidate storage unit78 is a storage unit which is formed of a memory, a hard disk, or the like and stores the tip candidates (the positions (coordinates) or sizes thereof) detected by thetip candidate detector74. The tipcandidate storage unit78 may store a plurality of past tip candidates.
• The tipcandidate storage unit78 outputs the plurality of past tip candidates stored therein to theextraction processor82 and the specifyingprocessor84 of thetip candidate detector74 in order to allow them to be used for the detection in thetip candidate detector74 and outputs the same to thetip candidate processor76 in order to allow them to be used for the highlighting process in thetip candidate processor76 or the display on thedisplay unit52.
• Theinformation storage unit80 is a storage unit which is formed of a memory, a hard disk, or the like and stores information on the puncture needle, detection conditions for detecting the tip candidates in thetip candidate detector74, processing conditions for the detection, processing conditions for the highlighting process in thetip candidate processor76, and the like. Here, the information on the puncture needle may be the kind, the thickness, the insertion position (the position where the puncture needle is inserted into a patient), the insertion angle (the angle at which the puncture needle is inserted into a patient), and the insertion path of the puncture needle, a puncture target (target object or site), and the like. Moreover, specifically, the detection and processing conditions include the extraction conditions of the tip candidate points extracted by thetip candidate detector74, such as the magnitude of the luminance difference or luminance value of the differential image, the threshold of the density or size of extracted points, as well as the type, size, and weighting conditions of various LUTs and filters including the tip enhancement filter used for extraction, and such processing conditions as the contents of the tip candidate highlighting process such as the coloring and brightening of the tip candidates in thetip candidate processor76, and the size and shape of the target region of the tip candidate.
• Theinformation storage unit80 acquires and stores the information on the puncture needle and the detection and processing conditions for the tip candidates through themain body controller54 in accordance with the input or the like on theoperation unit56 from the user and outputs the information and conditions stored therein to theextraction processor82 and the specifyingprocessor84 of thetip candidate detector74 and thetip candidate processor76.
• Thetip detector70 basically has the configuration described hereinabove.
• Next, a preferred configuration of thedifferential image generator68 shown inFIG. 1 will be described.
• FIG. 6 is a block diagram showing an example of the time-sequential frame differential image generator of the combined image generator shown inFIG. 1.
• As shown inFIG. 6, thedifferential image generator68 includes apreprocessor86 and adifferential processor88. Thepreprocessor86 includes aspeckle noise remover90, alayer structure remover92, and apuncture needle processor94.
• Thepreprocessor86 performs preprocessing on B-mode image signal (frame image) of at least one frame out of two time-sequential frames subjected to differential processing in order to increase the reliability of the detection of a puncture needle tip in a differential image before the generation of the differential image between the B-mode image signals of two time-sequential frames, generated by theimage generator64 and stored in the frameimage storage unit66. Examples of the preprocessing include a process of removing noise or a layer structure, a puncture needle enhancement processing, and a puncture needle image connecting process. The signal processing such as the puncture needle enhancement processing and the puncture needle image connecting process is performed on the B-mode image signal after noise removal in order to make the puncture needle clearly visible.
• Thepreprocessor86 preferably includes all of thespeckle noise remover90, thelayer structure remover92, and thepuncture needle processor94, but may include at least one of them.
• Thedifferential processor88 calculates the difference between the B-mode image signals (images) of two time-sequential frames, in which speckle noise and a layer structure are removed, to thereby generate a differential image (differential image signal).
• Here, since the insertion speed of the puncture needle varies depending on the technique or the like of an examiner or an operator, thedifferential processor88 preferably has a function of adjusting a time difference between the two time-sequential frames used for creating the differential image so that the frame interval of the two time-sequential frames can be optionally set. Moreover, it is preferable to use a plurality of past frames two frames or more before when creating the differential image. The differential image (pixel signal) may include the absolute value of the difference value.
• Thedifferential image generator68 preferably includes both thepreprocessor86 and thedifferential processor88, but may not include thepreprocessor86 if it includes thedifferential processor88.
• As preprocessing of the differential processing on the B-mode image signals (frame images) of two time-sequential frames generated by theimage generator64 and stored in theimage storage unit66, thespeckle noise remover90 performs signal processing for reducing a speckle pattern in the B-mode image signals to thereby remove speckle noise. Although a median filter, for example, is preferably applied to the process of removing speckle noise, a spatial compounding method, a frequency compounding method, morphology processing, or the like may be applied.
• Thelayer structure remover92 performs a layer structure removal processing on the B-mode image signals in which speckle noise is removed by the speckle noise remover90 to thereby remove a bright line extending in the direction of the puncture needle. For example, CFAR (Constant False Alarm Rate) processing and MIP (Maximum Intensity Projection) processing are performed. As for the CFAR processing, a method disclosed in JP 2006-305337 A can be used.
• In this way, by the layer structure removal processing by thelayer structure remover92, connected portions other than the puncture needle can be removed in later signal processing by thepuncture needle processor94.
• Moreover, in order to make the puncture needle clearly visible in a frame image, thepuncture needle processor94 performs signal processing of causing defocusing in the direction of the puncture needle to make the puncture needle continuous or signal processing of making the puncture needle continue from its feature points. Thus, thepuncture needle processor94 includes at least one of a punctureneedle enhancement processor94a(seeFIG. 7) that performs signal processing of causing defocusing in the direction of the puncture needle to make the puncture needle continuous and a punctureneedle connection processor94b(seeFIG. 9) that performs signal processing of making the puncture needle continue from its feature points.
• FIG. 7 shows a puncture needle enhancement processor which is an example of the puncture needle processor of the time-sequential frame differential image generator shown inFIG. 6.
• The punctureneedle enhancement processor94ashown inFIG. 7 includes afilter application processor96 and anedge enhancement processor98.
• Thefilter application processor96 applies a defocus filter to the B-mode image signal in which noise such as speckle noise or a layer structure is removed by thelayer structure remover92 or thespeckle noise remover90. That is, thefilter application processor96 specifies a defocus filter to be used based on the insertion angle stored in theinformation storage unit80 and reads the specified defocus filter from theinformation storage unit80. For example, when the insertion angle is 10°, thefilter application processor96 reads a defocus filter for the insertion angle of 10° and applies the read defocus filter to the B-mode image data after noise removal. Since the defocus filter used herein is a filter corresponding to the insertion angle of the puncture needle, it is possible to defocus the image in the insertion direction of the puncture needle to make a discontinuous puncture needle image continuous.
• Theedge enhancement processor98 performs a process of enhancing the edges of the B-mode image with respect to the B-mode image data to which the defocus filter has been applied. After that, a 1D edge enhancement processing may be performed in the vertical direction to the puncture needle to thereby enhance the edges of the puncture needle. Before outputting to thedifferential processor88, the edge-enhanced B-mode image (image data) which has been made continuous in the insertion direction of the puncture needle may be superimposed on the original B-mode image (image data) to thereby generate a combined frame image (image data). In this way, a frame image in which the whole image of the puncture needle within the tissue is clearly visible can be output to thedifferential processor88.
• Similarly to the tip enhancement filter used by thetip detector70, examples of the defocus filter used by thefilter application processor96 for the signal processing of causing defocusing in the direction of the puncture needle include a filter which has a step shape and uses pixels located in the insertion direction of the puncture needle in weighted addition and a filter which has a rectangular shape and performs weighted addition so that pixels located in the insertion direction of the puncture needle have a large filter coefficient. Here, when such a filter is used as the defocus filter, it is necessary to create the tip enhancement filter used by thetip detector70 as a puncture needle enhancement filter which is weighted in the insertion direction of the puncture needle in the frame image and has a size such that the puncture needle is included.
• That is, in thefilter application processor96, such a filter as shown inFIG. 5B, which has a rectangular shape and performs weighted addition so that pixels located in the insertion direction of the puncture needle have a large filter coefficient, can be used as the defocus filter.
• An example in which such a puncture needle enhancement filter is applied to one frame of B-mode image (frame image) so as to correspond to the insertion direction or the insertion angle of the puncture needle is shown inFIGS. 8A to 8C.
• FIGS. 8A, 8B, and 8C show a frame image (B-mode image), a puncture needle enhancement filter, and a puncture needle-enhanced image, respectively.
• Here, the frame image shown inFIG. 8A is one frame of ultrasound image (B-mode image) which is not subjected yet to the puncture needle enhancement processing performed by thefilter application processor96 of thepuncture needle processor94ashown inFIG. 7.
• The puncture needle enhancement filter shown inFIG. 8B is a puncture needle enhancement filter which has a size of 55×7 pixels, applied to the one-frame image shown inFIG. 8A.
• The puncture needle-enhanced image shown inFIG. 8C is a puncture needle enhanced ultrasound image obtained by applying the puncture needle enhancement filter to the frame image so as to correspond to the insertion direction or the insertion angle of the puncture needle, and is an image in which the B-mode image before the puncture needle enhancement processing is defocused in the direction of the insertion angle. Thus, as shown inFIG. 8C, the puncture needle is displayed in a continuous manner in the obtained puncture needle-enhanced image.
• Here, inFIGS. 8A and 8C, for better understanding of the effect of the puncture needle enhancement filtering, speckle noise removal processing, layer structure removal processing, and edge enhancement processing were not performed.
• As above, the puncture needle enhancement filter shown inFIG. 8B is a puncture needle enhancement filter which is weighted in the insertion direction of the puncture needle in one-frame image shown inFIG. 8A, and has a size of 55×7 pixels such that the puncture needle is included. The puncture needle enhancement filter shown inFIG. 8B can generate filter coefficients used in the puncture needle enhancement filter by linearly interpolating the tip enhancement filter shown inFIG. 5B so as to have the size of the puncture needle enhancement filter. That is, by linearly interpolating the filter having the size of 81×81 pixels shown inFIG. 5B, it is possible to create the puncture needle enhancement filter having the size of 55×7 pixels shown inFIG. 8B. The puncture needle enhancement filter having the size of 55×7 pixels obtained in this way is a puncture needle enhancement filter used if the insertion angle is 10°. The aspect ratio of the linear interpolation is determined by the insertion angle. In this puncture needle enhancement filter, the filter coefficient is the largest at the center, with the magnitude of the filter coefficient widely varying along the insertion direction of the puncture needle. A target pixel at the center is subjected to weighted addition using the 55×7 pixels around the target pixel. The values of the respective pixels are multiplied by the filter coefficients of the puncture needle enhancement filter to perform weighted addition, whereby the value of the target pixel is obtained. The puncture needle enhancement filter created in this way, having the aspect ratio corresponding to the insertion angle is stored in theinformation storage unit80.
• In the present embodiment, the puncture needle enhancement filter to be used is determined based on the insertion angle, and the puncture needle enhancement processing of performing weighted addition with neighboring pixels is performed on all pixels using the determined puncture needle enhancement filter, whereby an image in which the puncture needle is enhanced can be generated.
• In this example, although a case of converting into a size of 55×7 pixels like the puncture needle enhancement filter shown inFIG. 8B has been described, instead of this, a puncture needle enhancement filter shown inFIG. 14D described later may be used, for example. In puncture needle enhancement filters having different sizes like the puncture needle enhancement filter having the size of 15×27 pixels shown inFIG. 14D, filter coefficients corresponding to the sizes of the respective puncture needle enhancement filters can be generated by linearly interpolating the base filter having the size of 81×81 pixels shown inFIG. 5B. The puncture needle enhancement filter shown inFIG. 14D, having the size of 15×27 pixels obtained in this way is also a puncture needle enhancement filter used if the insertion angle is 10° like the puncture needle enhancement filter shown inFIG. 8B, having the size of 55×7 pixels. The puncture needle enhancement filter created in this way, having the aspect ratio corresponding to the insertion angle is stored in theinformation storage unit80.
• A defocus filter such as the puncture needle enhancement filter preferably has a size such that the discontinuance interval of the puncture needle is included. A plurality of sizes of the defocus filter may be prepared in the manner of “Large,” “Medium,” and “Small” in accordance with the thickness of the puncture needle, for example, and a plurality of sizes may be prepared between the angles of 10° and 60° in the manner of “10°,” “30°,” and “60°” in accordance with the insertion angle of the puncture needle. Moreover, the size may be determined based on the size (G) of the puncture needle, the purpose of the puncture (FNA, CNB, RFA, and the like), and puncture information such as an insertion angle.
• Moreover, a Gaussian filter or the like may be used as a weighting filter used for the signal processing of causing defocusing in the direction of the puncture needle. In this case, the proportions of weights in averaging, variance, correlation, and the like may be prepared as parameters which can be changed by a user.
• Moreover, the size of the defocus filter and the weights of the weighting filter may be changed by a user selecting in advance on a setting screen, or may be changed during scanning of theultrasound probe12 by allocating a function to a function key or the like.
• Examples of the defocus filter include various puncture tool enhancement filters which are used in a puncture tool enhancement processing applied to an ultrasound image generation apparatus and an ultrasound image generation method of the third aspect of the invention described later.
• Examples of the signal processing by thefilter application processor96 causing defocusing in the direction of the puncture needle include a puncture needle enhancement processing which uses a puncture needle enhancement filter as the defocus filter. For example, a puncture tool enhancement processing applied to an ultrasound image generation apparatus and an ultrasound image generation method of the third aspect of the invention described later can be applied.
• That is, a puncture tool enhancement filter which has a step shape and uses pixels located in the insertion direction of the puncture needle in weighted addition can be used as the defocus filter. That is, a puncture needle enhancement filter that performs weighted addition between the value (image data) of a pixel (hereinafter referred to as a target pixel) subjected to the puncture needle enhancement processing and the value (image data) of a specific pixel neighboring the target pixel can be used. In this case, thefilter application processor96 sequentially changes the position of the target pixel and performs the puncture needle enhancement processing on the image data of all pixels in the B-mode image using the puncture needle enhancement filter determined based on the insertion angle.
• Examples of such a puncture needle enhancement filter include a filter in which a plurality of lines of filter elements are connected in a step shape while shifting the lines in accordance with the insertion angle, a filter element at the center or almost at the center of the connected filter element lines is used as a target pixel, and the filter coefficients of pixels increase or decrease as the pixels are away from the target pixel.
• The filter coefficient of such a puncture needle enhancement filter is generated using a Gaussian filter expressed byEquation 2 below.
• $\begin{array}{cc}f\left(x\right)=\frac{1}{\sqrt{2\pi }\sigma }\exp \left(-\frac{{\left(x-\mu \right)}^2}{2{\sigma }^2}\right)& \left(2\right)\end{array}$
• Here, μ is an average, σ²is a variance, and x represents the position of a pixel in the vertical direction of the drawing when a central element is at 0. For example, x=−1 and x=1 correspond to the positions of adjacent pixels on the front/rear or left/right sides of the central pixel. In this way, by determining the filter coefficients of respective pixels using the Gaussian filter, it is possible to perform weighted addition so that the pixels located closer to the target pixel have a larger filter coefficient.
• FIG. 9 shows a puncture needle connection processor which is an example of the puncture needle processor of the time-sequential frame differential image generator shown inFIG. 6.
• The punctureneedle connection processor94bshown inFIG. 9 performs signal processing of making the puncture needle continue from its feature points. Examples of such signal processing include a process of displaying a line using the feature points of the puncture needle through Hough transform or the like, a processing of fitting a line using the feature points of the puncture needle through a minimum mean square error method, and a process of storing the position coordinates of the feature points indicating the puncture needle tip in a memory for a certain period and connecting all the points by a line or a curve.
• As shown inFIG. 9, the punctureneedle connection processor94bincludes a puncture needlecandidate point extractor100, a candidate pointposition storage unit102, a puncture needleregion specifying unit104, a puncture needle tipposition specifying unit106, a punctureneedle image generator108, and a puncture needle connectedimage generator110.
• The puncture needle candidate point extractor (hereinafter referred to as a candidate point extractor)100 extracts needle candidate points as the feature points of the puncture needle using the information on the puncture needle stored in theinformation storage unit80 and the B-mode image data in which the speckle noise or noise such as a layer structure is removed by thelayer structure remover92 and thespeckle noise remover90. Specifically, thecandidate point extractor100 performs threshold processing on the B-mode image data using an edge extraction filter to thereby create edge image data and extracts candidate points of the puncture needle from the edge image data as the feature points on the puncture needle. Since the puncture needle has a smooth surface, and scattering of ultrasound waves barely occurs, the puncture needle is displayed in the B-mode image in a discontinuous manner. Thus, by performing the threshold processing on the B-mode image data where the puncture needle is present, it is possible to extract feature points indicating parts of the discontinuous puncture needle. The time interval of extracting the feature points of the puncture needle can be changed by the user. Since high-luminance points originating from tissues or the like other than the puncture needle are also present in the B-mode image, the feature points extracted by the threshold processing are not limited to those originating from the puncture needle. The feature points of the puncture needle originating from tissues or the like become noise when specifying the position of the puncture needle based on the edge image.
• The candidate point position storage unit (hereinafter referred to as a position storage unit)102 stores the positions of all feature points (candidate points) of the puncture needle extracted by thecandidate point extractor100 and outputs the positions of the puncture needle feature points to the puncture needle region specifying unit (hereinafter referred to as a region specifying unit)104.
• Theregion specifying unit104 generates a line (puncture needle candidate line) indicating the puncture needle and an extension line of the puncture needle based on the distribution of a plurality of puncture needle feature points stored in theposition storage unit102. Theregion specifying unit104 specifies a region including the generated line as a region where the puncture needle is present.
• The puncture needle tip position specifying unit (hereinafter referred to as a position specifying unit)106 specifies the tip position of the puncture needle based on luminance information of the region which is specified by theregion specifying unit104 to be a region where the puncture needle is highly likely to be present and outputs the specified tip position to the punctureneedle image generator108.
• The punctureneedle image generator108 generates an image representing the puncture needle based on the line indicating the puncture needle and the extension line of the puncture needle generated by theregion specifying unit104 and the tip position of the puncture needle specified by theposition specifying unit106 and outputs the image to the puncture needle connected image generator (hereinafter referred to as a connected image generator)110. The image representing the puncture needle can be displayed in various modes as selected by the user. For example, the puncture needle may be displayed in a line, the outline of the puncture needle may be displayed, and the puncture needle may be displayed as a collection of dots. Here, when displaying the outline of the puncture needle, the outline of the puncture needle is generated by reading the puncture needle information from theinformation storage unit80. Moreover, the luminance or the like representing the puncture needle can be set by the user. For example, the luminance of the image representing the puncture needle may be a favorite luminance of the user and may be the same luminance as the luminance of a portion which is considered to be the puncture needle in the B-mode image.
• Theconnected image generator110 generates combined B-mode image data in which the image representing the continuous puncture needle and generated by the punctureneedle image generator108 is superimposed on the B-mode image data output from thelayer structure remover92. Theconnected image generator110 outputs the combined B-mode image data to thedifferential processor88.
• FIG. 10 is a functional block diagram showing a more detailed configuration of the puncture needleregion specifying unit104 and the puncture needle tipposition specifying unit106 as shown inFIG. 9.
• Theregion specifying unit104 includes a punctureneedle line generator112 and a punctureneedle region generator114.
• The puncture needle line generator (hereinafter referred to as a line generator)112 performs a Hough transform on the puncture needle feature points distributed within the B-mode image output from theposition storage unit102 to thereby generate a puncture needle candidate line. The puncture needle candidate line is a line that passes the largest number of puncture needle feature points. Theline generator112 outputs the position coordinates of points located on the generated puncture needle candidate line to the puncture needle region generator (hereinafter referred to as a region generator)114.
• Theregion generator114 expands the puncture needle candidate line generated by theline generator112 to a predetermined width and specifies a region included in the puncture needle candidate line as a region (puncture needle presence region) where the puncture needle is present. Theregion generator114 outputs the position coordinates of points included in the specified puncture needle presence region to theposition specifying unit106.
• Theposition specifying unit106 includes anaverage luminance calculator116, a maximumluminance specifying unit118, a minimumluminance specifying unit120, and a puncture needletip position calculator122.
• By referring toFIGS. 11A to 11D andFIGS. 12A and 12B, a method of calculating the puncture needle tip position from a B-mode image and a method of generating an image representing the puncture needle will be described in detail. Moreover, the functions of respective constituent elements of theposition specifying unit106 will be described.
• InFIG. 11A, a case where an image is positioned in an XY orthogonal coordinate system in which the top left corner of a B-mode image is at the origin, a horizontal axis extending from the top left corner to the top right corner is an X axis, and a vertical axis extending from the top left corner to the bottom left corner is a Y axis will be considered. A direction from the top left corner of the B-mode image to the top right corner is defined as the positive direction of the X axis, and a direction from the top left corner of the B-mode image to the bottom left corner is defined as the positive direction of the Y axis. In the following description, the same definition of the XY orthogonal coordinate system in relation to an image will be applied unless otherwise defined.
• FIG. 11A shows a B-mode image of a subject to be examined including a puncture needle.
• The puncture needle inFIG. 11A is displayed in a discontinuous manner, and it is difficult to understand the accurate position of the puncture needle. Thus, first, the punctureneedle connection processor94bspecifies a region where the puncture needle is highly likely to be present from the B-mode image as shown inFIG. 11A and specifies the tip position of the puncture needle from the intensity distribution on a line including the puncture needle within the specified region. Moreover, the punctureneedle connection processor94bgenerates an image representing the puncture needle based on the tip position of the puncture needle, superimposes the image on the B-mode image, and outputs the B-mode image to thedifferential processor88.
• Thecandidate point extractor100 applies an edge extraction filter (weighted addition filter) corresponding to the insertion angle of the puncture needle to the B-mode image shown inFIG. 11A to make the image continuous in the direction of the insertion angle of the puncture needle. Moreover, thecandidate point extractor100 performs threshold processing on the B-mode image to which the edge extraction filter is applied to thereby create an edge image (seeFIG. 11B) so that only the feature points (needle candidate points) having a luminance not lower than the threshold appear white. Theline generator112 calculates the position of a puncture needle candidate line from the distribution of the feature points of the puncture needle within the edge image.
• In the edge image shown inFIG. 11B, a plurality of puncture needle feature points displayed with a high luminance within the B-mode image are distributed. Among the respective puncture needle feature points distributed within the edge image, the points appearing at the positions where the puncture needle is present are those mainly originating from the puncture needle, whereas the points appearing over the entire screen regardless of the positions where the puncture needle is present are those originating from tissues or the like other than the puncture needle. The puncture needle feature points originating from tissues or the like become noise when specifying the position of the puncture needle based on the edge image. Thus, theline generator112 performs Hough transform on the edge image including noise shown inFIG. 11B to thereby generate a puncture needle candidate line which passes the largest number of puncture needle feature points originating from the puncture needle. Even when the edge image includes noise, since the puncture needle feature points originating from the puncture needle show a linear connection, it is possible to generate a line which extends along the linear connection of the puncture needle feature points originating from the puncture needle through the Hough transform.FIG. 11C shows an image in which the generated punctureneedle candidate line130 is displayed so as to be superimposed on the edge image data. The punctureneedle candidate line130 shown inFIG. 11C shows the puncture needle and the extension line of the puncture needle.
• The punctureneedle candidate line130 which represents the puncture needle and the extension line of the puncture needle has an unclear boundary between the puncture needle and a non-puncture needle region. Thus, the boundary position on the punctureneedle candidate line130 between the puncture needle and the non-puncture needle region, namely the tip position of the puncture needle is calculated.FIG. 11D shows an edge image on which aregion132 is superimposed, in which theregion132 is obtained by theregion generator114 expanding the punctureneedle candidate line130 generated through the Hough transform to a predetermined width. Theregion132 is a region which includes the punctureneedle candidate line130. Theregion generator114 specifies theregion132 as a puncture needle presence region where the puncture needle is present. Since the punctureneedle candidate line130 extracted by the Hough transform is a line which passes the largest number of puncture needle feature points originating from the puncture needle, a region including a number of puncture needle feature points is specified as the punctureneedle presence region132. The punctureneedle connection processor94bcreates theregion132 in this way and narrows theregion132 down to a region where the puncture needle is highly likely to be present. Here, the predetermined width to which the punctureneedle candidate line130 is expanded may be the thickness of the puncture needle read from theinformation storage unit80 and may be set by the user while seeing the B-mode image or the edge image.
• Subsequently, theaverage luminance calculator116 of theposition specifying unit106 rotates the edge image shown inFIG. 11D until the longitudinal direction of theregion132 becomes horizontal and defines an X′Y′ orthogonal coordinate system so that the longitudinal direction of theregion132 corresponds to an X′ axis, and the lateral direction of theregion132 corresponds to a Y′ axis. Theaverage luminance calculator116 averages the luminance values at the points (points having the same X′ coordinate value) within theregion132 arranged in the Y′-axis direction.FIG. 12A is a view showing theregion132 within the edge image so as to correspond to a graph showing the distribution of average luminance values on a line (theregion132 reduced to 1D as the result of averaging the luminance values at the points with the same X′ coordinate value within the region132) including the puncture needle within theregion132 in order to illustrate a method of specifying the tip position from theregion132. The graph ofFIG. 12A is a graph showing the distribution of average luminance values in theregion132 as viewed in the scanning direction (X′-axis direction), in which the X′ coordinate is shown on the horizontal axis, and the average luminance on the vertical axis. The maximumluminance specifying unit118 and the minimumluminance specifying unit120 calculate the maximum and minimum values of the average luminance based on the graph shown inFIG. 12A. The graph shown inFIG. 12A is depicted in a simplified manner for better understanding of the invention.
• The puncture needle tip position calculator (hereinafter referred to as a position calculator)122 scans the average luminance values from the maximum side of the X′ coordinate to the origin side in the graph ofFIG. 12A showing the relationship between the X′ coordinate and the average luminance within theregion132 to thereby specify the tip position of the puncture needle. Specifically, theposition calculator122 specifies apoint134 at which the average luminance which had a value near the minimum value due to the non-presence of the puncture needle increases greatly to reach a luminance corresponding to 80% of the difference between the maximum and minimum values for the first time as the tip position of the puncture needle. Here, the reason why the average luminance values are scanned from the maximum side of the X′ coordinate is because by scanning the average luminance values from a side where the puncture needle is expected not to be present, a point at which the luminance varies abruptly can be specified as the tip position of the puncture needle. If the average luminance values are scanned from the origin side, the average luminance at a position where the puncture needle is discontinuous has a value near the minimum value, and after that, a point where the puncture needle is detected is likely to be specified as the tip end. In this example, a point corresponding to 80% of the difference between the maximum and minimum values, which is the point empirically preferable as a point that can substantially specify the puncture needle tip, is set as the tip position of the puncture needle, although the proportion may not always be 80%. However, since a point originating from noise may be specified as the tip position if a point too close to the minimum side of the average luminance values is set as the tip position, it is preferable to set the tip position to the point corresponding to not less than 50% of the difference between the maximum and minimum values of the average luminance.
• Within theregion132, the average luminance values in a region where the puncture needle is not present are approximately 0. Thus, if the average luminance values are scanned toward the 0 side from the maximum side of the X′ coordinate where the puncture needle is not present, values near 0 appear for a considerable amount of time. When the scanning advances into a region where the puncture needle is present from the region where the puncture needle is not present, the average luminance values increase abruptly. This is because the puncture needle is detected in high luminance. Theposition calculator122 specifies thepoint134 which is the tip position of the puncture needle based on a change in the average luminance values due to the presence/absence of the puncture needle. After specifying thepoint134 which is the tip position of the puncture needle, the puncture needletip position calculator122 converts the X′Y′ orthogonal coordinate system into the XY orthogonal coordinate system to calculate the X and Y coordinates of thepoint134.
• The punctureneedle image generator108 generates a line representing the puncture needle based on the punctureneedle candidate line130 and thetip position134 of the puncture needle. Specifically, asegment140 which is a line representing the puncture needle is generated using the punctureneedle candidate line130 as a segment which extends from the position of the X coordinate0 to apoint136 on the punctureneedle candidate line130 having the same X coordinate as the X coordinate of thetip position134 of the puncture needle.FIG. 12B is a schematic view showing thesegment140 generated by the punctureneedle image generator108. Apoint138 inFIG. 12B is a point at which the punctureneedle candidate line130 meets the left side of the edge image.
• The punctureneedle connection processor94bperforms the process of detecting the tip position of the puncture needle and generating an image representing the puncture needle at predetermined time intervals, superimposes an image representing the puncture needle generated latest on the B-mode image, and outputs the B-mode image to thedifferential processor88. The punctureneedle connection processor94bspecifies the tip position of the puncture needle and superimposes the generated puncture needle image (the image representing the puncture needle) on the B-mode image, whereby a frame image in which the position of the puncture needle is clearly visible can be output to thedifferential processor88.
• As the puncture tool connection process of making the puncture needle continuous, a puncture needle connection process of extracting and connecting needle candidate points to thereby generate a puncture needle image, applied in an ultrasound image generation apparatus and an ultrasound image generation method according to the second aspect of the invention can be applied.
• The puncture needle connection process performed by the punctureneedle connection processor94bmay be performed as the LUT processing performed by thetip detector70.
• The ultrasound image diagnostic apparatus according to the first aspect of the invention basically has the configuration described hereinabove.
• Next, the operation of the ultrasound image diagnostic apparatus and the ultrasound image generation method according to the first aspect of the invention will be described.
• FIG. 13 is a flowchart showing an example of main parts of the ultrasound image generation method according to the present aspect, in which the flowchart starts from a step of generating a differential image and ends with a step of generating a combined image in which a tip image including colored puncture needle tip candidates is superimposed on a B-mode image.
• First, an operator brings the ultrasound wave transceiving surface of theultrasound probe12 into contact with the surface of a subject to be examined. In this state, ultrasound waves are transmitted from the plurality oftransducers34 in accordance with the driving signal supplied from thetransmission driver38 of theprobe body16, and the reception signals output from therespective transducers34 having received the ultrasound echoes from the subject are supplied to the correspondingreception signal processors36, whereby the sample data are generated. The sample data are transmitted to the diagnostic apparatusmain body14 through thecommunication cable18 and stored in thedata storage unit46. Moreover, the sample data of each frame are read from thedata storage unit46, and B-mode image data of each frame are generated by theimage generator64 of the combinedimage generator48, and B-mode image data of time-sequential frames are stored in the frameimage storage unit66.
• Subsequently, in accordance with the flow ofFIG. 13, a combined image in which the tip image in which puncture needle tip candidates are colored as the highlighting process is superimposed on the B-mode image is generated.
• As shown inFIG. 13, first, in step S10, thedifferential image generator68 performs differential processing to generate a frame differential image shown inFIG. 14C from the B-mode image data of two time-sequential frames, a frame earlier than the present frame and the present frame shown inFIGS. 14A and 14B, read from the frameimage storage unit66.
• Subsequently, in step S12, thecandidate point extractor82 of thetip candidate detector74 of thetip detector70 performs LUT processing using a tip enhancement filter shown inFIG. 14D to extract tip candidate points. The tip enhancement filter shown inFIG. 14D is a filter for causing defocusing in the insertion direction of the puncture needle which has a rectangular shape and a size of15 x27 pixels, and which has filter coefficients weighted along the insertion direction of the puncture needle by a Gaussian filter shown inFIG. 15.FIG. 14E shows a LUT-processed image obtained through the LUT processing using the tip enhancement filter.
• FIG. 16 shows the distribution of puncture needle tip candidate points in a rectangular region including a puncture needle tip in an image which is obtained by binarizing the frame differential image shown inFIG. 14C based on a luminance difference. As can be understood fromFIG. 16, after the binarization, a plurality of puncture needle tip candidate points are present at positions other than the vicinity of the puncture needle tip.
• Subsequently, in step S14, the tipcandidate specifying processor84 specifies only the central point of tip candidate points in a region having a high correlation with the puncture needle, in particular, the tip end thereof as the tip candidate.
• In step S16, it is determined whether the tipcandidate specifying processor84 has finished the process of extracting and specifying the tip candidates within the differential image. If the process has not finished, the flow returns to step S14 and the process of extracting and specifying the tip candidates is continued. If the process has finished, the flow proceeds to step S18.
• In step S18, thetip candidate processor76 colors the tip candidates specified in step S14 to generate a tip image.
• Subsequently, in step S20, theimage combiner72 combines the tip image generated in step S18 with the ultrasound image (B-mode image) of the present frame to thereby generate a combined image in which the tip candidates colored by thetip candidate processor76 are superimposed on the ultrasound image of the present frame.
• Subsequently, in step S22, it is determined whether theimage combiner72 has finished the process of generating the combined image. If the process has not finished, the flow returns to step S10, and the process which starts from the step of generating the differential image and ends at step S20 of generating the combined image is repeated. If the process has finished, the processes ofFIG. 13 end.
• After that, the combined image combined by theimage combiner72 is sent to thedisplay controller50 and converted into a combined image signal for display, and a combined image in which the colored tip candidates are superimposed on the ultrasound image of the present frame is displayed on thedisplay unit52.
• The ultrasound image generation method of the first aspect of the invention is performed in the above-described manner.
• The example shown inFIG. 13 shows an example in which a tip candidate is always detected from the differential image. However, since the tip candidate may not be detected, a process taking a case where no tip candidate is detected into consideration will be described.
• FIG. 17 is a flowchart showing an example of the ultrasound image generation method according to the first aspect of the invention, taking a case where no tip candidate is detected into consideration.
• First, in step S30, B-mode image data of the present frame is subtracted from B-mode image data of a frame earlier than the present frame to generate frame differential image data.
• Subsequently, in step S32, the LUT processing or the tip candidate specifying process is performed on the differential image data to thereby detect puncture needle tip candidates.
• In step S34, it is determined whether a puncture needle tip candidate is detected in step S32. If the puncture needle tip candidate is detected, the flow proceeds to step S36 by determining that the detection was successful. If the puncture needle tip candidate is not detected, the flow proceeds to step S46.
• In step S36, detection information including the information on the puncture needle tip candidate detected in step S32 is stored in a memory (the tip candidate storage unit78).
• In step S38, the detected tip candidate is read from the memory, and the read tip candidate is colored, whereby a tip image of the puncture needle is generated.
• In step S40, the tip image generated in step S38 is superimposed on the ultrasound image (B-mode image) of the present frame, whereby a combined image in which the colored tip candidate is superimposed on the ultrasound image of the present frame is generated.
• In step S42, the combined image in which the colored tip candidate is superimposed on the ultrasound image of the present frame is displayed on thedisplay unit52.
• Subsequently, in step S44, it is determined whether the process of detecting puncture needle tip candidates and displaying the combined image will be continued. If the process is to be continued, the flow returns to step S30, the process which starts from the step of generating the differential image data and ends at step S42 of displaying the combined image is repeated. If the process is to be finished, the processes ofFIG. 17 end.
• On the other hand, when it is determined in step S34 that the detection of the puncture needle tip candidate was not successful, it is determined in step S46 whether the detection information on the past puncture needle tip candidates in the memory will be used. If the information is to be used, the flow proceeds to step S48. If the information is not to be used, the flow proceeds to step S50.
• In step S48, the past puncture needle tip candidates are colored using the detection information read from the memory to thereby generate a tip image of the puncture needle, and the flow proceeds to step S40. In step S40, the tip image of the puncture needle generated in step S48 is superimposed on the ultrasound image (B-mode image) of the present frame, whereby a combined image is generated.
• In step S50, since there is no detection information and no tip candidate, the ultrasound image of the present frame is displayed on thedisplay unit52, and the flow proceeds to step S44. In step S44, as described above, it is determined whether the detection process will be continued, and the detection process is continued or finished in accordance with the determination result.
• In the embodiment shown inFIG. 1, as described above, the processes of generating the differential image of two frames, detecting the tip candidates, generating the tip image, and superimposing the tip image on the ultrasound image (B-mode image) of the present frame are performed based on the B-mode image signals generated by theimage processor62 of theimage generator64. However, the invention is not limited to this, and instead of the B-mode image signals, as shown inFIG. 18, the processes of generating the differential image of two frames, detecting the tip candidates, generating the tip image, and superimposing the tip image on the ultrasound image of the present frame may be performed based on baseband signals (acoustic-ray signals) having been subjected to beam forming processing by thephasing addition unit60 of theimage generator64, namely echo signals of each frame. Thus, in the invention, any one of the frame echo signals and the B-mode image signals may be used.
• FIG. 18 is a block diagram schematically showing a configuration of another embodiment of the diagnostic apparatus main body of the ultrasound diagnostic apparatus according to the first aspect of the invention.
• A diagnostic apparatusmain body14ashown inFIG. 18 includes thedata storage unit46, a combinedimage generator48a, and thedisplay unit52, and although not shown, further includes the main body controller, the operation unit, and the storage unit similarly toFIG. 1. Moreover, the combinedimage generator48aincludes theimage generator64, a time-sequential frame echo signal storage unit (hereinafter also referred to as an echo signal storage unit)66a, a time-sequential frame differential echo signal generator (hereinafter also referred to simply as a differential echo signal generator)68a, a puncture needle tip detector (tip detector)70a, and animage combiner72a.
• The echosignal storage unit66a, the differentialecho signal generator68a, thetip detector70a, and theimage combiner72aof the combinedimage generator48aof the diagnostic apparatusmain body14ashown inFIG. 18 are the same as theimage storage unit66, thedifferential image generator68, thetip detector70, and theimage combiner72 of the combinedimage generator48 of the diagnostic apparatusmain body14 shown inFIG. 1, except that the elements shown inFIG. 18 perform processes based on echo signals whereas the elements shown inFIG. 1 perform processes based on B-mode image signals. That is, the contents of the processes performed by the two groups of elements are approximately the same, except that the signals (data) to be processed are different. Thus, detailed description of the contents of the same processes performed by the respective elements will not be provided.
• The echosignal storage unit66ais a memory that time-sequentially stores echo signals representing images of a plurality of frames.
• The differentialecho signal generator68acalculates a difference between echo signals of two time-sequential frames stored in the echosignal storage unit66ato thereby generate a differential echo signal.
• Thetip detector70awhich is the most characteristic portion of the present embodiment performs a process of detecting a tip end from the differential echo signal generated by the differentialecho signal generator68a, detects at least one tip candidate including a puncture needle tip, highlights the detected tip candidates, and generates a highlighted tip image. The tip candidates detected from the differential echo signal are preferably subjected to the same conversion process as used by theimage processor62 to generate B-mode image signals, and the tip image is preferably expressed by B-mode image signals.
• Theimage combiner72acombines the tip image (needle tip enhanced image) generated by thetip detector70awith an ultrasound image of the present frame, which is the B-mode image generated by theimage processor62, to thereby generate a combined ultrasound image.
• When the tip image generated by thetip detector70ais not expressed by B-mode image signals, theimage combiner72a performs the same conversion process as used by theimage processor62 on the tip image to generate B-mode image signals and then combines the tip image so as to be superimposed on the ultrasound image of the present frame, which is a B-mode image.
• When the ultrasound image of the present frame, which is the B-mode image generated by theimage processor62 is expressed by a display image signal which is scan-converted by the DSC, it is necessary to perform scan conversion with the DSC so that the tip image converted into the B-mode image is also expressed by a display image signal.
• The conversion from signals or data subjected to processing into B-mode image signals and the scan conversion may be performed in any step as long as the signals or data to be combined have the same format when they are combined by theimage combiner72a. The scan conversion may not be performed in the combinedimage generator48awhen it is performed by thedisplay controller50.
• The ultrasound diagnostic apparatus and the ultrasound image generation method according to the first aspect of the invention have the above-described configuration.
• Next, an ultrasound image generation apparatus and an ultrasound image generation method according to the second aspect of the invention will be described.
• FIG. 19 is a functional block diagram schematically showing a configuration of an embodiment of an ultrasound image generation apparatus according to the second aspect of the invention.
• An ultrasound image generation apparatus (hereinafter referred to as a generation apparatus)200 shown inFIG. 19 includes atransceiving controller202, animage generator204, a puncture needlecandidate point extractor206, a puncture needleinformation storage unit208, a candidate pointposition storage unit210, a puncture needleregion specifying unit212, a puncture needle tipposition specifying unit214, a punctureneedle image generator216, animage combiner218, and animage display controller220. Thegeneration apparatus200 is used by being electrically connected to aprobe222 and animage display unit224. Thegeneration apparatus200, theprobe222, and theimage display unit224 form an ultrasound diagnostic apparatus10a, and thegeneration apparatus200 and theimage display unit224 form a diagnostic apparatusmain body14a. Some constituent elements of thegeneration apparatus200 are the same as those of the ultrasounddiagnostic apparatus10 shown inFIGS. 1, 2, and9, and detailed description thereof will not be provided.
• Although not shown, theprobe222 includes a plurality of piezoelectric elements, transmits ultrasound waves from the plurality of piezoelectric elements toward a patient, and receives echoes reflected from the patient. The piezoelectric element generates an echo signal upon receiving the echoes. Theprobe222 is connected to thegeneration apparatus200 through a communication cable not shown and outputs the echo signal to thetransceiving controller202 through the communication cable.
• Theprobe222 of the present embodiment may be one which includes the plurality oftransducers34, the plurality ofreception signal processors36, and thetransmission driver38 of theprobe body16 of theultrasound probe12 shown inFIG. 1.
• Although not shown, thetransceiving controller202 includes a pulser that generates a high-voltage electrical signal, an amplifier that amplifies the echo signal reflected from the patient and input from theprobe222, a low-pass filter that rejects high-frequency components of the echo signal, and an A/D converter that converts analog signals into digital signals. The high-voltage pulse signal generated by the pulser is applied to the piezoelectric elements (not shown) in theprobe222, whereby ultrasound waves are transmitted from theprobe222.
• In thetransceiving controller202, the echo signal output from theprobe222 is amplified by the amplifier, high-frequency components of the echo signal are rejected by the low-pass filter, the resulting signal is A/D converted by an A/D converter, and the resulting signal is output to theimage generator204.
• Thetransceiving controller202 of the present embodiment may be one which includes thereception signal processors36, thetransmission driver38, thetransmission controller42, thereception controller42, and theprobe controller44 of theprobe body16 of theultrasound probe12 shown inFIG. 1.
• In the present embodiment, although theprobe222 does not include thetransceiving controller202, theprobe222 may include thetransceiving controller202 similarly to theprobe body16 of theultrasound probe12 shown inFIG. 1. Moreover, theprobe222 may include a puncture adapter similarly to theultrasound probe12 shown inFIG. 1.
• Although not shown, theimage generator204 includes a delay circuit, an addition circuit, and a STC circuit. Theimage generator204 adds the echo signals output from thetransceiving controller202 by delaying in accordance with the position of the piezoelectric element to thereby form an acoustic-ray signal. In theimage generator204, the STC circuit corrects attenuation of the acoustic-ray signal based on a distance in accordance with the depth of the reflection position of the ultrasound wave. After that, theimage generator204 generates B-mode image data.
• Theimage generator204 may be configured by theimage generator64 of the diagnostic apparatusmain body14 shown inFIG. 1, which includes thephasing addition unit60 and theimage processor62. Moreover, a storage unit (data storage unit) which temporarily stores the digital echo signal may be provided between theimage generator204 and thetransceiving controller202.
• The puncture needle information storage unit (hereinafter also referred to as an information storage unit)208 is a storage unit that stores information on the puncture needle which is a puncture tool. Here, theinformation storage unit208 may have the same configuration and function as theinformation storage unit80 shown inFIG. 2 in that it stores information on the puncture needle. Theinformation storage unit208 acquires and stores information on the puncture needle through the input or the like from the user.
• The puncture needlecandidate point extractor206 extracts needle candidate points as the feature points of the puncture needle using the information on the puncture needle stored in theinformation storage unit208 and the B-mode image data generated by theimage generator204. Specifically, an edge extraction filter is applied to the B-mode image data, which are then subjected to CFAR processing and threshold processing, whereby edge image data are created. Then, candidate points of the puncture needle are extracted from the edge image data as the feature points on the puncture needle. Here, the puncture needlecandidate point extractor206 may have a configuration different from the puncture needlecandidate point extractor100 shown inFIG. 9 in that it performs speckle noise removal processing, CFAR processing, MIP processing, or the like. However, the puncture needlecandidate point extractor206 may have the same configuration and function as the puncture needlecandidate point extractor100 shown inFIG. 9 with regard to extraction of needle candidate points. Thus, the same description as that of the puncture needlecandidate point extractor100 is applied with regard to the display of the puncture needle and extraction of feature points, and redundant description thereof will not be provided. The candidate points (feature points) of the puncture needle may be detected from one frame and may be detected from a plurality of frames. When detecting the candidate points from a plurality of frames, it is preferable to use about five frames while updating the frames and the candidate points.
• The candidate pointposition storage unit210 has the same configuration and function as the candidate pointposition storage unit102 shown inFIG. 9. That is, the candidate pointposition storage unit210 stores the positions of all needle candidate points extracted by the puncture needlecandidate point extractor206 and outputs the positions of the puncture needle candidate points to the puncture needleregion specifying unit212.
• The puncture needleregion specifying unit212 has the same configuration and function as the puncture needleregion specifying unit104 shown inFIG. 9. That is, the puncture needleregion specifying unit212 generates a line (puncture needle candidate line) representing the puncture needle and the extension line of the puncture needle based on the distribution of a plurality of needle candidate points stored in the candidate pointposition storage unit210. The puncture needleregion specifying unit212 specifies a region including the generated line as a region where the puncture needle is present.
• The puncture needle tipposition specifying unit214 has the same configuration and function as the puncture needle tipposition specifying unit106 shown inFIG. 9. That is, the puncture needle tipposition specifying unit214 specifies the tip position of the puncture needle based on luminance information of a region, in which the puncture needle is highly likely to be present, specified by the puncture needleregion specifying unit212 and outputs the specified tip position to the punctureneedle image generator216.
• The punctureneedle image generator216 has the same configuration and function as the punctureneedle image generator108 shown inFIG. 9. That is, the punctureneedle image generator216 generates an image representing the puncture needle based on the line representing the puncture needle and the extension line of the puncture needle, generated by the puncture needleregion specifying unit212 and the tip position of the puncture needle specified by the puncture needle tipposition specifying unit214 and outputs the image to theimage combiner218. Thus, the same description as that of the punctureneedle image generator108 is applied with regard to the form selection and generation of the image representing the puncture needle and setting of the color, luminance, and the like of the image representing the puncture needle, except that the puncture needle information is read from theinformation storage unit208. Therefore, redundant description thereof will not be provided.
• The puncture needlecandidate point extractor206, theinformation storage unit208, the candidate pointposition storage unit210, the puncture needleregion specifying unit212, the puncture needle tipposition specifying unit214, and the punctureneedle image generator216 described above form a puncture needle detection circuit.
• Theimage combiner218 generates combined B-mode image data in which the image representing the puncture needle generated by the punctureneedle image generator216 is displayed so as to be superimposed on the B-mode image data output from theimage generator204. Theimage combiner218 outputs the combined B-mode image data to theimage display controller220.
• Theimage display controller220 has a DSC and has the same configuration and function as thedisplay controller50 of the diagnostic apparatusmain body14 shown inFIG. 1. In theimage display controller220, the DSC converts the combined B-mode image data combined by theimage combiner218 into display image data corresponding to a general television signal scanning format, performs necessary image processing such as gradation processing, and outputs the processed display image data to theimage display unit224. Thegeneration apparatus200 specifies the tip position of the puncture needle from the B-mode image to create an image representing the puncture needle, combines the image with the B-mode image, and causes the combined image to be displayed on theimage display unit224.
• Thetransceiving controller202, theimage generator204, the puncture needlecandidate point extractor206, the candidate pointposition storage unit210, the puncture needleregion specifying unit212, the puncture needle tipposition specifying unit214, the punctureneedle image generator216, theimage combiner218, and theimage display controller220 may be realized by a combination of a central processing unit (CPU) and software (programs) for causing the CPU to execute various processes.
• The puncture needleregion specifying unit212 and the puncture needle tipposition specifying unit214 shown inFIG. 19 have the same configuration and function as the puncture needleregion specifying unit104 and the puncture needle tipposition specifying unit106 shown inFIG. 10, respectively, and detailed description thereof will not be provided.
• Although not shown, the puncture needleregion specifying unit212 has a puncture needle line generator and a puncture needle region generator similarly to the puncture needleregion specifying unit104 shown inFIG. 10. These constituent elements have the same configuration and function as the punctureneedle line generator112 and the punctureneedle region generator114 shown inFIG. 10, respectively, and detailed description thereof will not be provided.
• Although not shown, the puncture needle tipposition specifying unit214 has an average luminance calculator, a maximum luminance specifying unit, a minimum luminance specifying unit, and a puncture needle tip position calculator similarly to the puncture needle tipposition specifying unit104 shown inFIG. 10. These constituent elements have the same configuration and function as theaverage luminance calculator116, the maximumluminance specifying unit118, the minimumluminance specifying unit120, and the puncture needletip position calculator122 shown inFIG. 10, respectively, and detailed description thereof will not be provided.
• Next, with reference toFIGS. 11A to 11D andFIGS. 12A and 12B used for description of the first aspect of the invention, a method in the ultrasoundimage generation apparatus200 of the present embodiment, of calculating the tip position of the puncture needle from the B-mode image and a method of generating the image representing the puncture needle will be described in more detail. Moreover, the functions of the respective constituent elements of the puncture needle tipposition specifying unit214 of the present embodiment will be described. The description is the same as that of the method of detecting the tip position of the puncture needle and the method of generating the puncture needle image made with reference toFIGS. 11A to 11D andFIGS. 12A and 12B and the description of the functions of the respective constituent elements of theposition specifying unit106 shown inFIG. 9. Thus, detailed description thereof will not be provided.
• When the B-mode image of the patient including the puncture needle is displayed as shown inFIG. 11A, since the puncture needle in the drawing is displayed in a discontinuous manner, it is difficult for the user to understand the accurate position of the puncture needle. Thus, first, thegeneration apparatus200 specifies a region in which the puncture needle is highly likely to be present from the B-mode image as shown inFIG. 11A and specifies the tip position of the puncture needle from the intensity distribution on a line including the puncture needle within the region. Moreover, thegeneration apparatus200 generates an image representing the puncture needle based on the tip position of the puncture needle and displays the image on theimage display unit224 together with the B-mode image. That is, thegeneration apparatus200 displays the image of the puncture needle generated by specifying the position of the puncture needle together with the B-mode image in which the puncture needle is hardly visible to the user, so that the user can understand the accurate position of the puncture needle.
• The puncture needlecandidate point extractor206 applies an edge extraction filter corresponding to an insertion angle of the puncture needle to the B-mode image shown inFIG. 11A to thereby make the image continuous in the direction of the insertion angle of the puncture needle. Moreover, the puncture needlecandidate point extractor206 performs a layer structure removal processing such as CFAR processing to thereby remove a bright line extending in the X-axis direction in the lower part of the drawing. Furthermore, the puncture needlecandidate point extractor206 performs threshold processing on the B-mode image to which the edge extraction filter has been applied to thereby create an edge image (seeFIG. 11B) so that only the feature points (needle candidate points) having a luminance not lower than the threshold appear white. The puncture needle line generator calculates the position of a puncture needle candidate line from the distribution of the puncture needle candidate points within the edge image.
• The puncture needle line generator performs Hough transform on the edge image including noise shown inFIG. 11B to thereby generate a puncture needle candidate line which passes the largest number of needle candidate points originating from the puncture needle.FIG. 11C shows an image in which the generated punctureneedle candidate line130 is displayed so as to be superimposed on the edge image data.
• The punctureneedle candidate line130 which represents the puncture needle and the extension line of the puncture needle has an unclear boundary between the puncture needle and a non-puncture needle region. Thus, the tip position of the puncture needle, which is the boundary position on the punctureneedle candidate line130 between the puncture needle and the non-puncture needle region, is calculated.
• The puncture needle region generator expands the punctureneedle candidate line130 to a predetermined width and specifies theregion132 shown inFIG. 11D as a puncture needle presence region. Thegeneration apparatus200 creates theregion132 in this way and narrows the region in which the puncture needle is highly likely to be present. Here, the predetermined width for expanding the punctureneedle candidate line130 may be the thickness of the puncture needle read from theinformation storage unit208 and may be set by the user while seeing the B-mode image or the edge image.
• Subsequently, the average luminance calculator of the puncture needle tipposition specifying unit214 rotates the edge image shown inFIG. 11D until the longitudinal direction of theregion132 becomes horizontal and defines an X′Y′ orthogonal coordinate system so that the longitudinal direction of theregion132 corresponds to an X′ axis, and the lateral direction of theregion132 corresponds to a Y′ axis. The average luminance calculator averages the luminance values at the points (points having the same X′ coordinate value) within theregion132 arranged in the Y′-axis direction. The maximum luminance specifying unit and the minimum luminance specifying unit calculate the maximum and minimum values of the average luminance based on the graph shown inFIG. 12A.
• The puncture needle tip position calculator scans the average luminance values from the maximum side of the X′ coordinate to the origin side in the graph ofFIG. 12A showing the relationship between the X′ coordinate and the average luminance within theregion132 to thereby specify the tip position of the puncture needle. Specifically, the puncture needle tip position calculator specifies apoint134 at which the average luminance which had a value near the minimum value due to the non-presence of the puncture needle increases greatly to reach a luminance corresponding to 80% of the difference between the maximum and minimum values for the first time as the tip position of the puncture needle.
• Within theregion132, the average luminance values in a region where the puncture needle is not present are approximately 0. When the scanning advances into a region where the puncture needle is present, the average luminance values increase abruptly. The puncture needle tip position calculator specifies thepoint134 which is the tip position of the puncture needle based on a change in the average luminance value due to the presence/absence of the puncture needle. After specifying thepoint134 which is the tip position of the puncture needle, the puncture needle tip position calculator converts the X′Y′ orthogonal coordinate system into the XY orthogonal coordinate system to calculate the X and Y coordinates of thepoint134.
• The punctureneedle image generator216 generates a line representing the puncture needle based on the punctureneedle candidate line130 and thetip position134 of the puncture needle. Specifically, as shown inFIG. 12B, asegment140 which is a line representing the puncture needle is generated using the punctureneedle candidate line130 as a segment which extends from the position of the X coordinate 0 to apoint136 on the punctureneedle candidate line130 having the same X coordinate as the X coordinate of thetip position134 of the puncture needle.FIG. 12B is a schematic view showing thesegment140 generated by the punctureneedle image generator216.
• Thegeneration apparatus200 performs the process of detecting the tip position of the puncture needle and generating an image representing the puncture needle at predetermined time intervals and displays the image representing the puncture needle generated latest on theimage display unit224. Thegeneration apparatus200 specifies the tip position of the puncture needle and displays the generated puncture needle image (the image representing the puncture needle) so as to be superimposed on the B-mode image, so that the position of the puncture needle can be displayed so as to be easily understood by the user.
• As the process in thegeneration apparatus200, of specifying the tip position of the puncture needle and superimposing the generated puncture needle image on the B-mode image to thereby generate a combined image, the puncture needle connection process applied to the ultrasound diagnostic apparatus and the ultrasound image generation method according to the first aspect of the invention can be applied, for example.
• Next, the operation of the ultrasound image generation apparatus and the ultrasound image generation method according to the invention will be described with reference toFIG. 20.
• FIG. 20 is a flowchart showing the flow of a series of processes related to the operation of displaying the puncture needle image in a superimposed manner in the ultrasoundimage generation apparatus200 that performs the ultrasound image generation method of the invention. After ultrasound images are acquired, in step S100, the user sets the time interval for detecting needle candidate points. In step S102, the puncture needlecandidate point extractor206 extracts needle candidate points. In step S104, the coordinates of the puncture needle candidate points are stored in the candidate pointposition storage unit210. In step S106, the puncture needleregion specifying unit212 specifies the puncture needle candidate line representing the puncture needle and the extension line of the puncture needle based on the distribution of the puncture needle candidate points. In step S108, the puncture needle tipposition specifying unit214 specifies the tip position of the puncture needle. In step S110, the punctureneedle image generator216 generates the image representing the puncture needle. In step S112, theimage combiner218 combines the B-mode image and the image representing the puncture needle, and theimage display controller220 causes theimage display unit224 to display the image representing the puncture needle so as to be superimposed on the B-mode image.
• In step S114, the user selects whether or not to change a superimposition method. If the user selects to change the superimposition method, the flow returns to step S110, and an image representing the puncture needle of which the superimposition method is changed is displayed so as to be superimposed on the B-mode image. If the user selects not to change the superimposition method in step S114, the flow proceeds to step S116. In step S116, the user selects whether or not to change the time interval for extracting needle candidate points. If the user selects to change the time interval, the flow returns to step S100, a new extraction time interval is set, and the puncture needle candidate points are extracted again. If the user selects not to change the time interval for extracting needle candidate points in step S116, the flow proceeds to step S118. In step S118, the user selects whether or not to process the next frame. If the user selects to process the next frame, the flow returns to step S100. If the user selects not to process the next frame in step S118, the processes are terminated.
• FIG. 21 is a flowchart illustrating the operation of step S102 in more detail. In step S200, the user selects whether or not to set a predicted insertion region (a region in which the puncture needle is predicted to be inserted). If the user selects to set the predicted insertion region, the flow proceeds to step S202, the predicted insertion region is set, and then, the flow proceeds to step S204. If the user selects not to set the predicted insertion region in step S200, the flow proceeds to step S204. In step S204, the user selects whether or not to enhance the image quality of a region (hereinafter referred to as a target region) which the user wants to see in particular detail. If the user selects to do so, the flow proceeds to step S206, and the user sets a target region. In step S208, the image quality of the target region is enhanced, and then, the flow proceeds to step S210. If the user selects not to enhance the image quality of the target region in step S204, the flow proceeds to step S210. In step S210, an edge extraction filter is applied. In step S212, threshold processing is performed. In step S214, needle candidate points are extracted from the edge image which has been subjected to threshold processing, and the flow proceeds to step S104.
• The operation of setting the predicted insertion region in step S202 will be described with reference toFIGS. 22A to 22D. InFIGS. 22A to 22D, a case where a B-mode image231 is positioned in an XY orthogonal coordinate system in which the top left corner of the image is at the origin, a horizontal axis extending from the top left corner to the top right corner is an X axis, and a vertical axis extending from the top left corner to the bottom left corner is a Y axis will be considered. InFIG. 22A, abroken line230 is a puncture guide line. The puncture needlecandidate point extractor206 sets a predicted insertion region based on thepuncture guide line230. When a probe is attached to a puncture adapter, since the insertion path of the puncture needle is determined by the puncture adapter to some extent, it is possible to display the puncture guide line.
• In the present embodiment, the predicted insertion region can be set by the user selecting one of three regions having different widths. The predicted insertion region is assumed to have a shape that the puncture guide line is expanded to a predetermined width in the Y-axis direction. For example, when performing puncturing on a shallow portion such as a breast cancer, a narrow region (a region having approximately the same width as the puncture guide line), an average region (a region obtained by expanding the puncture guide line by +0.5 cm in both the positive and negative Y-axis directions), and a wide region (a region obtained by expanding the puncture guide line by +1 cm in both the positive and negative Y-axis directions) are used. The information on a plurality of lines displaced by a predetermined distance from the puncture guide line in both the positive and negative Y-axis directions of the puncture guide line is stored, and an image region interposed by the plurality of lines is generated as the predicted insertion region. Since the likelihood of the puncture needle being shifted from the puncture guide line increases as puncturing is performed on a deeper portion, and the amount of shift increases, the area of the predicted insertion region may be increased. For example, the wide region may be a region obtained by expanding the puncture guide line by1.5 cm in both the positive and negative Y-axis directions.
• FIG. 22B shows a case when the predicted insertion region is set to a narrow range. InFIG. 22B, aregion232 represents the predicted insertion region.
• FIG. 22C shows a case when the predicted insertion region is set to an average range. InFIG. 22C, aregion234 represents the predicted insertion region. Theregion234 is wider than theregion232.
• FIG. 22D shows a case when the predicted insertion region is set to a wide range. InFIG. 22D, aregion236 represents the predicted insertion region. Theregion236 is wider than theregion234.
• When the predicted insertion region is set, the puncture needlecandidate point extractor206 performs threshold processing or the like on only the inside of the predicted insertion region to extract needle candidate points from the inside of the predicted insertion region. When the puncture needle candidate points are extracted from only the inside of the predicted insertion region, the number of needle candidate points originating from tissues or the like in the edge image generated from the predicted insertion region decreases. Thus, the proportion of needle candidate points originating from the puncture needle in relation to all needle candidate points increases. Therefore, it is possible to generate the puncture needle candidate line with higher precision.
• The operation of setting the target region which will be subjected to image quality enhancement, described in step S206 will be described with reference toFIG. 23.
• The user sets the target region by checking the position of apuncture needle238 on a B-mode image242. Specifically, the user checks the position of thepuncture needle238 with naked eyes and sets a region in which theentire puncture needle238 is included as atarget region240. InFIG. 23, for better understanding of the invention, thepuncture needle238 is depicted in a continuous manner. In this example, although the user sets the target region, the target region may not always be set by the user. For example, the ultrasoundimage generation apparatus200 may automatically set a region in which the entire image representing the puncture needle, generated by the punctureneedle image generator216 is included as the target region.
• Thetransceiving controller202 performs an image quality enhancing process on the target region. Specifically, thetransceiving controller202 causes theprobe222 to perform an acoustic ray increasing process of increasing the number of acoustic rays in the target region to thereby perform the image quality enhancing process so that the target region has higher image quality than region other than the target region. The acoustic ray increasing process is a process of narrowing the distance from which ultrasound waves are irradiated to thereby obtain echo signals more finely. Thus, a high-precision image can be obtained from the region subjected to the acoustic ray increasing process. In this example, although the acoustic ray increasing process is performed on the target region, the image quality enhancing process on the target region is not limited to the acoustic ray increasing process. For example, a steer beam process (a process of irradiating ultrasound waves in a direction vertical to the longitudinal direction of the puncture needle to increase echoes reflected from the puncture needle) or a frequency compound process (a process of increasing the band of transmission and reception signals, dividing the band into several parts to form a plurality of images, averaging these images to thereby decrease speckle noise) may be performed.
• When the target region is set in advance, and needle candidate points are extracted from the target region having been subjected to the image quality enhancing process, since Hough transform can be performed using needle candidate points in which noise is reduced, it is possible to improve the precision in specifying the position of the puncture needle. For example, since speckle noise can be reduced when the frequency compound process is performed, the proportion of the puncture needle candidate points originating from the puncture needle increases when threshold processing is performed. InFIG. 23, although the frame border of thetarget region240 is depicted so as not to overlap the frame border of the B-mode image242 for better understanding of the invention, thetarget region240 naturally does not include a region on the outer side of the B-mode image242.
• A modified example of a method of displaying the puncture needle image so as to be superimposed on the B-mode image will be described with reference toFIGS. 24A to 24D. InFIGS. 24A to 24D, the puncture needles superimposed on the B-mode image are displayed in different manners. The user can select a desired display mode from the modes of displaying the image representing the puncture needle shown inFIGS. 24A to 24D. In the present modified example, the line, segment, dots representing the puncture needle superimposed on the B-mode image are displayed in a predetermined color, for example, green. However, inFIGS. 24A to 24D, the colored line, segment, dots representing the puncture needle superimposed on the B-mode image are depicted by a black line, a black segment, and black dots.
• InFIG. 24A, the puncture needle is displayed as a segment (inFIG. 24A, a deep black segment)244 having saturation unlike the grayscale used for the B-mode image, and is superimposed on the B-mode image. By displaying in this way, it is possible to display a line representing the puncture needle in a manner clearly different from the B-mode image. Thus, the user can easily understand the position of the puncture needle.
• InFIG. 24B, aline244 representing the puncture needle inFIG. 24A is made transparent to form a semitransparent line246 (inFIG. 24B, a thin black line246), theline246 is superimposed on the B-mode image. The display transparency is 50%, for example. As above, when the puncture needle is displayed so as to be superimposed on the B-mode image as thesemitransparent line246, since the B-mode image is not concealed by theline246 representing the puncture needle, the user can see both the puncture needle and the B-mode image.
• InFIG. 24C, the image representing the puncture needle is not in line form, but needle candidate points248 (inFIG. 24C, thin black dots248) are displayed in a color different from the B-mode image and superimposed on the B-mode image. The superimposed needle candidate points248 are only the puncture needle candidate points of which the X coordinates are equal to or smaller than the X coordinate of thetip position point136 on the line representing the puncture needle, among the puncture needle candidate points present in the puncture needle presence region. As above, when the puncture needle is not displayed as a line, but only the puncture needle candidate points248 are colored and displayed so as to be superimposed on the B-mode image, there will be no line which impairs the visualization of the B-mode image.
• InFIG. 24D, the outline representing the shape of the puncture needle is fitted to the line representing the puncture needle to form an image250 (inFIG. 24D, a deep black segment250) representing the puncture needle. When the outline representing the shape of the puncture needle is displayed so as to be superimposed on the B-mode image, the user can immediately understand the shape of a needle. If the thickness or shape of the puncture needle is known as the puncture needle information stored in theinformation storage unit208, the outline of the puncture needle may be an outline having the thickness or shape. Alternatively, the outline may be set by the user.
• As described above, according to the ultrasoundimage generation apparatus200 of the present embodiment, by extracting the puncture needle candidate points from the B-mode image and performing Hough transform, it is possible to specify the puncture needle candidate line representing the puncture needle and the extension line of the puncture needle. Moreover, thegeneration apparatus200 can specify the tip position of the puncture needle within the B-mode image by specifying the region including the specified puncture needle candidate line as the puncture needle presence region and specifying the tip position of the puncture needle based on the luminance information within the puncture needle presence region. Furthermore, thegeneration apparatus200 can display the accurate position of the puncture needle so as to be easily understood by the user by displaying the image representing the puncture needle so as to be superimposed on the B-mode image based on the specified position of the puncture needle.
• In the present embodiment, although when setting the predicted insertion region, the position of the puncture guide line is determined based on the puncture adapter, the puncture adapter may not always be used in the invention. For example, the puncture guide line may be created by the user, and the puncture guide line may be created by calculating the frame difference of the ultrasound image to calculate the angle or position at which the puncture needle is inserted into the patient. In this case, the predicted insertion region is determined along the newly created puncture guide line. Moreover, a storage unit that stores the past tip position of the puncture needle may be provided to store the past tip position of the puncture needle and the detection timing thereof, and the predicted insertion region may be set based on the past tip position and detection timing of the puncture needle. Moreover, the image representing the puncture needle may be stored in addition to the tip position of the puncture needle.
• FIGS. 25A and 25B are views showing a method of determining the predicted insertion region using a plurality of past tip positions of the puncture needle. InFIGS. 25A and 25B, a case where an image is positioned in an XY orthogonal coordinate system in which the top left corner of the image is at the origin, a horizontal axis extending from the top left corner to the top right corner is an X axis, and a vertical axis extending from the top left corner to the bottom left corner is a Y axis will be considered. A direction from the top left corner of the image to the top right corner is defined as the positive direction of the X axis, and a direction from the top left corner of the image to the bottom left corner is defined as the positive direction of the Y axis.
• InFIG. 25A, a case in which a plurality ofpoints254,256, and258 representing the past tip positions of the puncture needle are present in a B-mode image252 will be considered. In this example, thepoint254 is the tip position of the puncture needle specified earliest. Thepoint256 is the tip position of the puncture needle detected next to thepoint254, and thepoint258 is the latest tip position of the puncture needle. Since the threepoints254,256, and258 are arranged in aline262a, it can be expected that the next tip position of the puncture needle will be on abroken line262bwhich is the extension line of the line passing the three points.
• Thus, aregion260 which is disposed under thepoint258 having the largest Y coordinate withinFIG. 25A and includes thebroken line262b,and of which the size is set taking the bending width of the puncture needle into consideration is determined as a predicted insertion region. Here, theregion260 has a parallelogram shape obtained by expanding thebroken line262bin the X-axis direction. The distance (the difference in Y coordinate) between thepoints258 and261 at which thebroken line262bmeets the upper and lower sides of theregion260 is set to be 2 to 3 times the distance between the past tip positions of the puncture needle (for example, the distance between the points265 and258). Here, the reason why the size of theregion260 is determined based on the distance (position) of the past tip positions of the puncture needle is to predict the advancing direction of the puncture needle after checking the speed and angle at which the puncture needle was inserted in the past. By checking the past insertion speed and angle of the puncture needle, it is possible to predict the advancing direction of the puncture needle since it can be predicted that at the next detection timing of the tip position, the puncture needle will be inserted at a speed and an angle which are not greatly different from the past insertion speed and angle (that is, the tip position will be displaced by an almost identical distance to the distance between the past tip positions). Although the distance is set with a small margin to be 2 to 3 times the distance between the past tip positions of the puncture needle, the distance may not always be 2 to 3 times larger.
• The puncture needlecandidate point extractor206 extracts the puncture needle candidate points within theregion260. When the predicted insertion region is determined using a plurality of past tip positions of the puncture needle, it is possible to further decrease the size of the predicted insertion region. Thus, the proportion of needle candidate points originating from the puncture needle within the predicted insertion region can be increased further. Therefore, it is possible to generate the puncture needle candidate line with high precision. The shape of theregion260 is not limited to a parallelogram, but may be a trapezoidal shape that expands in the X-axis direction as the Y coordinate decreases, for example.
• FIG. 25B is a view showing a method of determining the predicted insertion region when detecting the tip position of the puncture needle at the point in time occurring next toFIG. 25A. InFIG. 25B, apoint264 representing the latest tip position of the puncture needle in a B-mode image270 is added in comparison withFIG. 25A. In this case, aregion266 including abroken line268bwhich is an extension line of aline268athat passes thepoints254,256,258, and264 is set as the predicted insertion region. Apoint267 is a point at which thebroken line268bmeets the lower side of theregion266. As above, even when the number of stored past tip positions of the puncture needle increases, it is possible to set the predicted insertion region based on these positions. Moreover, even when the puncture needle deviates from the original predicted insertion region during puncturing due to bending of the puncture needle caused by the presence of stiff tissues, shift of a probe, or the like, it is possible to detect the puncture needle candidate line with high precision.
• InFIGS. 25A and 25B, although a case in which a plurality of past tip positions of the puncture needle are arranged on a line has been described as an example, the plurality of past tip positions of the puncture needle may not always be arranged on a line. For example, a plurality of past tip positions of the puncture needle may be connected like a line graph, and the next predicted insertion region may be set based on the slope of a line connecting the latest two points. Moreover, the predicted insertion region may be set by calculating the slope of a line by performing a least squares method or the like on the latest three points.
• Moreover, the width of the predicted insertion region may be determined in accordance with a shift width of the puncture needle with respect to the puncture guide line, which is measured in advance. Alternatively, the width of the predicted insertion region may be adjusted within the range of widths prepared in advance. Moreover, the width of the predicted insertion region may be adjusted during acquisition of ultrasound images. In this case, it is preferable to allocate a function of adjusting the width of the predicted insertion region to a function key or the like of the main body.
• Moreover, in the present embodiment, although the width of the predicted insertion region is set from three width steps, the number of width steps is not limited to three. By setting the width steps more finely, the user can set a desired range as finely as possible.
• In the present embodiment, when the predicted insertion region is set, the threshold processing is performed on the inside of the predicted insertion region, although the threshold processing may be performed on the entire image, and the Hough transform may be performed using only the puncture needle candidate points within the predicted insertion region. In this case, it is easy to perform the Hough transform.
• Moreover, in the present embodiment, the puncture needle candidate line generated by the puncture needle line generator of the puncture needleregion specifying unit212 is expanded to a predetermined width, and a region on the puncture needle candidate line is determined as the puncture needle presence region. However, the method of determining the puncture needle presence region is not limited to this method, and a region including the puncture needle candidate line may be manually determined by the user. For example, the puncture needle presence region may be determined as shown inFIGS. 26A to 26D.
• FIG. 26A is a view showing a case in which a punctureneedle candidate line274 is present in a B-mode image272, and a punctureneedle presence region276 is created by expanding the width of the punctureneedle candidate line274. InFIGS. 26A to 26D, a case where the B-mode image272 is positioned in an XY orthogonal coordinate system in which the top left corner of the B-mode image272 is at the origin, a horizontal axis extending from the top left corner to the top right corner is an X axis, and a vertical axis extending from the top left corner to the bottom left corner is a Y axis will be considered. A direction from the top left corner of the B-mode image272 to the top right corner is defined as the positive direction of the X axis, and a direction from the top left corner of the B-mode image272 to the bottom left corner is defined as the positive direction of the Y axis.FIG. 26B shows a case in which the user creates a punctureneedle presence region278 by moving the punctureneedle presence region276 shown inFIG. 26A in a direction of decreasing the Y coordinate.FIG. 26C shows a case in which the user creates a punctureneedle presence region280 by moving the punctureneedle presence region276 shown inFIG. 26A in a direction of increasing the Y coordinate.FIG. 26D shows a case in which the user creates a punctureneedle presence region282 having a slope different from the punctureneedle candidate line274. As above, the puncture needle presence region may be adjusted by the user. If the user can adjust the puncture needle presence region, the user can manually set a region in which a large number of needle candidate points which are considered to represent the puncture needle are included as the puncture needle presence region while seeing the edge image. By calculating the average luminance value in the puncture needle presence region set by the user and detecting the tip position of the puncture needle, it is possible to improve the precision in detecting the tip position of the puncture needle.
• In the present embodiment, the direction of scanning the change in the average luminance value is determined based on the graph showing the relationship between the X′ coordinate and the average luminance. However, since in many cases, puncturing is performed along the direction of a probe mark (a mark indicating a scanning direction) attached to the probe, the target region may be set in the direction of the probe mark.
• In the present embodiment, although the target region is set by specifying the tip position of the puncture needle, it is not always necessary to set the target region by specifying the tip position of the puncture needle. For example, the user may check the position of the puncture needle within the B-mode image with naked eyes and set a region including the puncture needle as the target region.
• In the present embodiment, although the line representing the puncture needle is generated using the punctureneedle candidate line130 and thepoint136 on the punctureneedle candidate line130, having the same X coordinate as thepoint134 specified as the tip position of the puncture needle, the line representing the puncture needle may not always be generated in this way. For example, thepoint134 which is the tip position of the puncture needle may be used as the ending point, the user may determine the starting point, and a segment may be generated from the starting and ending points.
• In the present embodiment, although four variations are illustrated as the display modes of displaying the image representing the puncture needle so as to be superimposed on the B-mode image, the display modes are not limited to these variations. For example, both the line representing the puncture needle and the puncture needle candidate points may be displayed at the same time, and the puncture needle candidate points may be made transparent. Moreover, it is not necessary to display the line representing the puncture needle in a uniform color. For example, the line may be displayed with a gradation or the like, and may be expressed with two colors.
• Moreover, it is preferable to allow the user to select a desired thickness of the line representing the puncture needle. For example, the user may select a thickness from various thicknesses set in advance, for example, narrow, medium, bold, and the like, and the user may input the thickness. When the thickness of the puncture needle used is known, the line may be displayed with the same thickness as the thickness of the puncture needle used.
• Moreover, it is preferable to allow the user to set the transparency of the line representing the puncture needle. For example, a function may be allocated to an input button prepared on a control panel or the like so that the user can freely set the transparency.
• Moreover, the luminance of the line representing the puncture needle may be automatically calculated and set by the ultrasound image generation apparatus. For example, the luminance of the line is determined by normalizing it by a gradation value (for example, 100) using the maximum luminance in the B-mode image before the threshold processing and adding the normalized luminance to the luminance representing the B-mode image.
• Moreover, the luminance of the image representing the puncture needle may be automatically set by the ultrasound image generation apparatus. For example, the luminance of the image representing the puncture needle can be set automatically using the average luminance value of several pixels of the B-mode image that are around the image representing the puncture needle which is superimposed on the B-mode image.
• Moreover, it is preferable to allow the user to set the predetermined time interval of detecting the tip position of the puncture needle and generating the image representing the puncture needle. The time interval is selected from several time intervals, for example, narrow (1 second), medium (2 seconds), and coarse (3 seconds). These time intervals may be set in advance by system configuration, and a function may be allocated to a button prepared in a control panel or the like. Since the speed at which the puncture needle is inserted into a patient is different depending on the user, the user may set the time interval so that the detection of the tip position of the puncture needle and the generation of the image representing the puncture needle are performed at time intervals corresponding to the insertion speed of the puncture needle. By allowing the user to set a desired time interval so that the detection of the tip position of the puncture needle and the generation of the image representing the puncture needle are performed at time intervals corresponding to the insertion speed of the puncture needle, the process of detecting the tip position of the puncture needle is performed less frequently when the position of the puncture needle is not changed much. Thus, it is possible to decrease the processing load on the apparatus.
• In the present embodiment, although the threshold processing or the like is performed on the B-mode image in which the echo signal is expressed by luminance information to thereby specify the tip position of the puncture needle, it is not always necessary to perform the threshold processing or the like on the B-mode image. For example, the threshold processing or the like may be performed on images of other modes or the echo signal itself to thereby specify the tip position of the puncture needle.
• The ultrasound image generation apparatus and the ultrasound image generation method according to the second aspect of the invention have the configuration described hereinabove.
• Next, an ultrasound image generation apparatus and an ultrasound image generation method according to the third aspect of the invention will be described.
First Embodiment
• FIG. 27 is a functional block diagram showing a configuration of main parts of an example of an ultrasound image generation apparatus according to the first embodiment of the third aspect of the invention.
• An ultrasoundimage generation apparatus300 shown inFIG. 27 includes atransceiving controller302, an echosignal storage unit304, anultrasound image generator306, a puncture toolenhancement data generator308, a puncture toolinformation storage unit310, anultrasound image combiner312, and an ultrasoundimage display controller314. Thegeneration apparatus300 is used by being electrically connected to amonitor316 and aprobe318. Moreover, theprobe318 is used together with apuncture adapter320. Thegeneration apparatus300, themonitor316, theprobe318, and thepuncture adapter320 form an ultrasound diagnostic apparatus10b, and thegeneration apparatus300 and themonitor316 form a diagnostic apparatus main body14b. Some constituent elements of thegeneration apparatus300 are the same as those of the ultrasounddiagnostic apparatus10 shown inFIGS. 1, 2, 6, and 7, and those of thegeneration apparatus200 shown inFIG. 19. In this case, the constituent elements have the same configuration and function, and detailed description thereof will not be provided.
• Theprobe318 has the same configuration and function as theprobe222 of thegeneration apparatus200 shown inFIG. 19. That is, theprobe318 transmits ultrasound waves toward a patient, receives an echo signal reflected from the patient, and outputs the echo signal to thetransceiving controller302. Moreover, theprobe318 outputs an insertion angle output from thepuncture adapter320 to thetransceiving controller302.
• Thepuncture adapter320 is used by being physically and electrically connected to theprobe318, and serves as a guide along which a puncture needle (not shown) is inserted into the patient. Thepuncture adapter320 has the same configuration and function as thepuncture adapter20 of the ultrasounddiagnostic apparatus10 shown inFIG. 1 when thepuncture adapter20 is used by being physically and electrically connected to theprobe body16.
• Thepuncture adapter320 has a guide groove (not shown) whose angle with respect to the patient is variable, and which enables the user to change the insertion angle. Thepuncture adapter320 stores information on the insertion angle. When thepuncture adapter320 is physically connected to theprobe318, since thepuncture adapter320 is also electrically connected to theprobe318, a signal indicating the insertion angle is output to theprobe318. Moreover, thepuncture adapter320 outputs a signal indicating the present insertion angle to theprobe318 whenever the angle of the groove with respect to the patient changes.
• Thetransceiving controller302 has the same configuration and function as thetransceiving controller202 shown inFIG. 19. That is, thetransceiving controller302 applies ultrasound transmission pulses to piezoelectric elements (not shown) in theprobe318 and causes ultrasound waves to be generated from theprobe318.
• Thetransmission controller302 amplifies the echo signal output from theprobe318, rejects high-frequency components of the echo signal, performs A/D conversion on the resulting echo signal, and then outputs the digital echo signal to the echosignal storage unit304. Moreover, thetransmission controller302 outputs the insertion angle output from thepuncture adapter320 to the puncture toolinformation storage unit310.
• The echosignal storage unit304 temporarily stores the digital echo signal. The echosignal storage unit304 has the same configuration and function as thedata storage unit46 of the diagnostic apparatus main body10aof the ultrasounddiagnostic apparatus10 shown inFIG. 1.
• Theultrasound image generator306 has the same configuration and function as theimage generator204 shown inFIG. 19. That is, theultrasound image generator306 generates an acoustic-ray signal from the echo signal stored in the echosignal storage unit304, corrects the attenuation of the acoustic-ray signal in accordance with the depth of the reflection position of the ultrasound wave, and generates the B-mode image data.
• The puncture toolinformation storage unit310 has the same configuration and function as theinformation storage unit208 shown inFIG. 19. That is, the puncture toolinformation storage unit310 stores information on the puncture tool such as a puncture needle.
• The puncture toolenhancement data generator308 performs various noise removal processes on the B-mode image data generated by theultrasound image generator306. Moreover, the puncture toolenhancement data generator308 performs a puncture tool enhancement processing using a filter (hereinafter referred to as a puncture tool enhancement filter) that enhances the puncture tool on the B-mode image data after noise removal based on the insertion angle stored in the puncture toolinformation storage unit310. The puncture toolenhancement data generator308 outputs image data enhanced by the puncture tool enhancement filter, namely puncture tool enhancement data to theultrasound image combiner312. Detailed configuration of the puncture toolenhancement data generator308 will be described later.
• Theultrasound image combiner312 combines the B-mode image data stored in theultrasound image generator306 and the puncture tool enhancement data generated by the puncture toolenhancement data generator308 to thereby generate combined B-mode image data. Theultrasound image combiner312 outputs the combined B-mode image data to the ultrasoundimage display controller314.
• The ultrasoundimage display controller314 has the same configuration and function as theimage display controller220 shown inFIG. 19. That is, the ultrasoundimage display controller314 converts the combined B-mode image data combined by theultrasound image combiner312 into display image data and outputs the image data to themonitor316. In this way, the combined B-mode image (combined ultrasound image) is displayed on themonitor316.
• FIG. 28 is a functional block diagram showing a more detailed configuration of the puncture tool enhancement data generator of the ultrasound image generation apparatus shown inFIG. 27. The puncture toolenhancement data generator308 includes aspeckle noise remover322, alayer structure remover324, afilter storage unit326, afilter application processor328, and anedge enhancement processor330.
• Thespeckle noise remover322 has the same configuration and function as thespeckle noise remover90 shown inFIG. 6. That is, thespeckle noise remover322 removes speckle noise in the B-mode image data generated by theultrasound image generator306. For example, a median filter is applied.
• Thelayer structure remover324 has the same configuration and function as thelayer structure remover92 shown inFIG. 6. That is, thelayer structure remover324 performs a layer structure removal processing on the B-mode image data in which the speckle noise is removed by thespeckle noise remover322.
• Thefilter storage unit326 stores a plurality of puncture tool enhancement filters applied to the B-mode image data. Thefilter storage unit326 stores six puncture tool enhancement filters with intervals of 10°, corresponding to insertion angles of 10° to 60°. The plurality of puncture tool enhancement filters are provided from the puncture toolinformation storage unit310 to thefilter storage unit326. In the present aspect of the invention, thefilter storage unit326 may not be provided, and these puncture tool enhancement filters may be stored in the puncture toolinformation storage unit310 so that a puncture tool enhancement filter applied is read directly from the puncture toolinformation storage unit310.
• Thefilter application processor328 specifies a puncture tool enhancement filter to be used based on the insertion angle stored in the puncture toolinformation storage unit310 and reads the specified puncture tool enhancement filter from thefilter storage unit326. For example, when the insertion angle is 10°, a puncture tool enhancement filter for the insertion angle of 10° is read. Thefilter application processor328 applies the read puncture tool enhancement filter to the B-mode image data after layer structure removal. Since the puncture tool enhancement filter used therein is a filter corresponding to the insertion angle of the puncture needle, it is possible to defocus the image in the insertion direction of the puncture needle to make a discontinuous puncture needle image continuous. That is, the puncture tool enhancement filter is the defocus filter used in the first aspect of the invention. Thus, thefilter application processor328 has the same configuration and function as thefilter application processor96 shown inFIG. 7.
• Theedge enhancement processor330 has the same configuration and function as theedge enhancement processor98 shown inFIG. 7. That is, theedge enhancement processor330 performs a process of enhancing the edges of the B-mode image with respect to the B-mode image data to which the puncture tool enhancement filter has been applied. For example, a 1D edge enhancement processing is performed in the vertical direction to the puncture needle to thereby enhance the edges of the puncture needle.
• Theultrasound image combiner312 combines the B-mode image (image data) which has been made continuous in the insertion direction of the puncture needle so as to be superimposed on the original B-mode image (image data) to thereby generate a combined B-mode image (image data). In this way, the whole image of the puncture needle within the tissue can be displayed on themonitor316 in an easily visible manner.
• FIG. 29 is a flowchart showing an example of the operation of the ultrasound image generation apparatus and the ultrasound image generation method according to the present aspect of the invention.
• First, B-mode image (ultrasound image) data is generated in step S300, and puncture tool enhancement data is generated in step S302. In step S304, the puncture tool enhancement data is combined with the B-mode image data to thereby generate combined B-mode image. In step S306, the combined B-mode image data is subjected to scan conversion. In step S308, combined B-mode image is displayed on themonitor316 using the scan-converted combined B-mode image data. In this way, the process ends.
• FIG. 30 is a flowchart illustrating the operation of generating puncture tool enhancement data in the step (step S302) of generating puncture tool enhancement data in the ultrasound image generation method shown inFIG. 29 in more detail.
• In step S400, speckle noise in the B-mode image (ultrasound image) data is removed. In step S402, a layer structure in the B-mode image data is removed. In step S404, the insertion angle of the puncture tool is specified. In step S406, a puncture tool enhancement filter is applied to the B-mode image data. In step S408, a puncture tool edge enhancement processing is performed to generate puncture tool enhancement data, and the flow proceeds to step S304.
• FIGS. 31A and 31C are views each showing the shape of a puncture tool enhancement filter used for generating the puncture tool enhancement data. When the B-mode image is considered as a collection of pixels on a 2D coordinate system, the puncture tool enhancement filter is a filter that performs a weighted addition between the value (image data) of a pixel (hereinafter referred to as a target pixel) subjected to the puncture tool enhancement processing and the value (image data) of a specific pixel around the target pixel. Thefilter application processor328 sequentially changes the position of the target pixel and performs the puncture needle enhancement processing on the image data of all pixels in the B-mode image using the puncture needle enhancement filter determined based on the insertion angle.
• FIG. 31A shows the shape of a puncturetool enhancement filter332 used when the insertion angle is 10°.Elements336ato336tof the puncturetool enhancement filter332 represent pixels subjected to weighted addition, and the positions of theelements336ato336trepresent the positions of the pixels subjected to weighted addition. The puncturetool enhancement filter332 has a size of 21×3 pixels. The puncturetool enhancement filter332 has a shape such that, as seen downwardly, a line of every seven pixels arranged in the vertical direction is shifted rightward by one pixel. When the shape of the puncturetool enhancement filter332 is considered as a 2D matrix with21 rows and3 columns, the elements at the first column and 1st to 7th rows (elements336ato336g), the elements at the second column and 8th to 14th rows (elements336hto336m), and the elements at the third column and 15th to 21st rows (elements336nto336t) have their respective filter coefficients. That is, the puncturetool enhancement filter332 has a shape such that elements at every specified number of rows starting from the first row are shifted from the first column sequentially to an adjacent right column.
• Anelement334 depicted in black and located at the center of the puncturetool enhancement filter332 is a target pixel, the position of theelement334 is the position of the target pixel, and the target pixel is located at the center of the puncturetool enhancement filter332. A hatchedelement336arepresents one pixel used for the weighted addition of the target pixel. Each of theelements336bto336trepresents one pixel used for the weighted addition of the target pixel similarly to theelement336aalthough theelements336bto336tare not hatched, and the positions of theelements336ato336trepresent the positions of the pixels used for the weighted addition. For example, anelement336jadjacent on the upper side of theelement334 represents that weighted addition is performed using the value of a pixel adjacent on the upper side of the target pixel. That is, the shape of the puncture tool enhancement filter represents the positions of neighboring pixels used when a puncture tool enhancement processing is performed on the target pixel. In the puncturetool enhancement filter332, the target pixel is subjected to weighted addition using the values of two sets of three adjacent pixels (elements336hto336m) on the upper and lower sides of the target pixel, the values of pixels (elements336ato336g) 4 to 10 pixels above at the left adjacent column, the values of pixels (elements336nto336t) 4 to 10 pixels below at the right adjacent column, and the value of the target pixel itself. That is, the target pixel is subjected to weighted addition using the values of neighboring 20 pixels and the value of the target pixel itself.
• The reason why the puncturetool enhancement filter332 has a shape such that elements at every specified number of rows starting from the first row are shifted from the first column sequentially to an adjacent right column is to perform weighted addition between the target pixel and pixels located in the insertion direction of the puncture needle based on the insertion angle and to make the puncture needle image within the B-mode image continuous in the insertion direction of the puncture needle. In this example, although elements at every seven rows (every seven pixels) are shifted as a predetermined length rightward when the insertion angle is 10°, the predetermined length is different depending on the insertion angle. In the present embodiment, since the puncture needle is inserted from the top left corner of the drawing toward the bottom right corner, it is assumed that the puncturetool enhancement filter332 has a shape such that elements at every specified number of rows starting from the first row are shifted from the first column sequentially to an adjacent right column.
• Moreover, since the puncture needle is displayed in a linear shape on the B-mode image (ultrasound image), the puncture needle is highly likely to be present in the pixels located in the insertion direction of the puncture needle. Thus, by performing weighted addition using the pixels located at the positions based on the insertion angle, it is possible to perform weighted addition using not only pixels near the target pixel but also pixels which are located at positions away from the target pixel and at positions where the puncture needle is highly likely to be present.
• Moreover, the puncture tool enhancement filter has a size based on the discontinuance interval of the puncture needle within the B-mode image.FIG. 31B is an enlarged view of aregion339 in which the puncture needle within the B-mode image is discontinuous in order to describe the size of the puncture tool enhancement filter. In this example, the puncturetool enhancement filter332 is taken as an example, and, in apuncture tool region339, the puncture needle is displayed with a discontinuance interval ideal for applying the puncturetool enhancement filter332.Pixels337a,337b,337c,and337ddepicted in black inFIG. 31B are pixels representing the puncture needle. In the drawing, apixel338 is a pixel located at the center of theregion339. The puncture needle in theregion339 is discontinuous in pixels (19×1 pixels) between thepixels337band337c.The puncturetool enhancement filter332 has a size such that the pixel region where the puncture needle is discontinuous is expanded by one pixel in upward and downward, as well as rightward and leftward directions. That is, the puncturetool enhancement filter332 has a size of 21×3 pixels. As above, the puncture tool enhancement filter has a size greater than the interval with which the puncture needle is discontinuous within the B-mode image.
• By setting the size of the puncture tool enhancement filter so as to be greater than the discontinuance interval of the puncture needle, even when the puncture tool enhancement filter is applied to the pixel at the center of the region where the puncture needle is discontinuous, pixels representing the puncture needle will be included at both ends of the puncture tool enhancement filter. That is, the puncture tool enhancement filter is a filter that performs weighted addition using at least one of the pixels representing the puncture needle when it is applied to a region where the puncture needle is discontinuous within the B-mode image. By doing so, even when the target pixel is located in a region where the puncture needle is discontinuous within the B-mode image, it is possible to perform weighted addition using pixels representing the puncture needle, located away from the target pixel. Thus, it is possible to make the target pixel have a luminance close to that of the puncture needle image surrounding the target pixel.
• FIG. 31C shows the shape of a puncturetool enhancement filter344 used when the insertion angle is 30°. The puncturetool enhancement filter344 has a size of 21×2 pixels. When the shape of the puncturetool enhancement filter344 is considered as a 2D matrix with 21 rows and 2 columns, the elements at the first column and 1st to 11th rows, and the elements at the second column and 12th to 21st rows have their respective filter coefficients. The puncturetool enhancement filter344 also has a shape such that elements at every specified number of rows starting from the first row are shifted from the first column sequentially to an adjacent right column. Similarly toFIG. 31A, a hatchedelement342 represents one pixel, and apixel340 depicted in black represents a target pixel. When the insertion angle is 30°, the target pixel is subjected to weighted addition using the values of 10 pixels on the upper side of the target pixel, the values ofpixels 1 to 10 pixels below at the right adjacent column, and the value of the target pixel itself. As above, if the insertion angle is different, the shape of the puncture tool enhancement filter applied, namely the position of the pixel used for the weighted addition is also different. Thefilter storage unit326 stores a plurality of puncture tool enhancement filters having a shape corresponding to the insertion angle.
• The puncture tool enhancement filter designates pixels used for weighted addition and performs addition by applying weights based on the filter coefficients to the respective designated pixels.FIGS. 32A and 32B are views showing the filter coefficients of the respective elements used for weighted addition in the puncturetool enhancement filter344. The numbers within the respective elements of the puncturetool enhancement filter344 represent filter coefficients. For example, inFIG. 32A, theelement340 has a filter coefficient of 0.091. The filter coefficients described within the respective elements of the puncturetool enhancement filter344 are rounded off to the third decimal place.
• FIG. 32A shows a case where filter coefficients are evenly allocated to elements on the upper and lower sides of the position of theelement340 which is located at the center of the puncturetool enhancement filter344. That is, the element340 (sample point number11) at the center of the puncturetool enhancement filter344 has the maximum filter coefficient, and the filter coefficient decreases in proportion to the distance in the vertical direction from theelement340. The sum of the filter coefficients of the respective elements is normalized to1. InFIG. 32A, the graph shown to the right of the puncturetool enhancement filter344 shows the numbers within the respective elements of the puncturetool enhancement filter344, in which the vertical axis represents a sample point number, and the horizontal axis represents a filter coefficient. The sample point number on the vertical axis of the graph ofFIG. 32A represents the row number of the element when the puncturetool enhancement filter344 is considered as a 2D matrix. For example,sample point number1 represents that the element is on the first row of the first column. The sample point number corresponding to the element340 (on the 11th row of the 1st column) is 11.
• Another method of determining the filter coefficients of the respective elements will be described. For example, the filter coefficients can be generated using a Gaussian filter expressed byEquation 2 which is applied in the first aspect of the invention.
• InEquation 2, x represents the position of a pixel in the vertical direction of the drawing when the central element indicated bysample point number11 is at 0 as shown inFIG. 32A. For example, x=−1 corresponds to the pixel atsample point number10, and x=1 corresponds to the pixel atsample point number12.FIG. 32B is a graph in which the vertical axis represents the sample point number and the horizontal axis represents a filter coefficient f(x) when the average μ=0 and the variance σ²=1 inEquation 2. InFIG. 32B, the numbers within the respective elements of the puncturetool enhancement filter344 shown to the left of the graph are the values of the filter coefficients of the respective elements. For example, the element340 (sample point number11) has a filter coefficient of 0.080. By determining the filter coefficients of the respective pixels in this way, it is possible to perform weighted addition so that the filter coefficients of the pixels located closer to the target pixel are increased.
• The puncture tool enhancement filter is a filter that designates pixels used for weighted addition by the shape (the position of an element) thereof, and performs weighted addition which involves multiplying the values of the designated pixels by the filter coefficients, to thereby obtain the value of the target pixel. Thefilter application processor328 performs the puncture tool enhancement processing on all pixels using the puncture tool enhancement filter determined based on the insertion angle.
• FIG. 33A shows a B-mode image which has not been subjected to the puncture tool enhancement processing, andFIG. 33B shows a combined image of the B-mode image ofFIG. 33A and a B-mode image after the puncture tool enhancement processing.
• The combined image shown inFIG. 33B is obtained by causing defocusing of the B-mode image before the puncture tool enhancement processing shown inFIG. 33A in the direction of the insertion angle so that the puncture needle is displayed in a continuous manner. InFIGS. 33A and 33B, for better understanding of the effect of the puncture tool enhancement processing by the puncture tool enhancement filter, speckle noise removal processing, layer structure removal processing, and edge enhancement processing were not performed. Thegeneration apparatus300 applies the puncture tool enhancement filter to the image (image data) in which the speckle noise and the layer structure are removed, and combines a B-mode image after edge enhancement processing with the B-mode image before the puncture tool enhancement processing.
• As described above, according to the ultrasoundimage generation apparatus300 according to the first embodiment of the present aspect, the puncture tool enhancement filter used is determined based on the insertion angle of the puncture needle, and the puncture tool enhancement processing is performed using the puncture tool enhancement filter so that the B-mode image is made continuous in the insertion direction of the puncture needle. Thus, it is possible to generate an image in which the puncture needle displayed in a discontinuous manner is made continuous. Moreover, since the B-mode image after the puncture tool enhancement processing is combined with the B-mode image before the puncture tool enhancement processing, it is possible to generate an ultrasound image in which the puncture needle is displayed so as to be easily visible to the user.
• Moreover, removal of the speckle noise may not always be performed. However, when the puncture tool enhancement filter is applied to the B-mode image in which the speckle noise is removed, it is possible to increase the effect of application of the puncture tool enhancement filter without increasing the size of the puncture tool enhancement filter more than necessary.
• In the present embodiment, although six kinds of shapes of the puncture tool enhancement filters are stored in thefilter storage unit326, a larger number of shapes may be stored. Moreover, although puncture tool enhancement filters having shapes corresponding to the range of insertion angles between 10° and 60° are prepared, puncture tool enhancement filters having shapes corresponding to insertion angles of 10° or less and 60° or more or a puncture tool enhancement filter having a shape corresponding to the insertion angle of 15° or 25° may be prepared.
• Moreover, the thickness of the puncture needle may be stored as the puncture tool information, and the shape of the puncture tool enhancement filter may be changed in accordance with the thickness of the puncture needle used. For example, since a region where the puncture needle is likely to be present broadens when the puncture needle is thick, weighted addition may be performed using a wider range of pixels in accordance with the insertion angle.
• In the present embodiment, although the puncture tool enhancement filter used is determined based on the insertion angle by the puncture adapter, it is not always necessary to determine the puncture tool enhancement filter based on the insertion angle by the puncture adapter. For example, the puncture tool enhancement filter may be determined based on the insertion angle acquired from an image. That is, high-luminance points corresponding to the tip end of a puncture needle may be extracted from a plurality of B-mode images, the insertion angle of the puncture needle may be acquired from the plurality of tip positions, and the puncture tool enhancement filter may be determined based on the insertion angle.
Second Embodiment
• In the first embodiment described above, although a case where the pixels located in the insertion direction of the puncture needle are used for weighted addition using a puncture tool enhancement filter having a step shape has been described as an example, the invention is not particularly limited to this. In the second embodiment below, an aspect in which a puncture tool enhancement filter has a rectangular shape, and weighted addition is performed so that the pixels located in the insertion direction of the puncture needle have a large filter coefficient will be described. Since an ultrasound image generation apparatus according to the second embodiment of the present aspect has the same basic configuration as thegeneration apparatus300 described in the first embodiment, the functional block diagram thereof will not be illustrated. Moreover, since the basic operation thereof is the same as that of thegeneration apparatus300, illustration thereof will not be provided.
• FIG. 34 shows an example of a puncturetool enhancement filter402 used in the ultrasound image generation apparatus (not shown) according to the second embodiment of the present aspect. The puncturetool enhancement filter402 is a puncture tool enhancement filter used when the insertion angle is 10°. The puncturetool enhancement filter402 has a size of 21×3 pixels and has a rectangular shape. InFIG. 34, apixel404 is a pixel located at the center of the puncturetool enhancement filter402. Thefilter storage unit326 stores a plurality of rectangular filters having an aspect ratio corresponding to the insertion angle. The plurality of puncture tool enhancement filters have different aspect ratios depending on the insertion angle of the puncture needle. Six puncture tool enhancement filters are stored with intervals of 10° between 10° and 60°. Moreover, the puncture tool enhancement filter has a size based on the interval with which the puncture needle is discontinuous within the B-mode image. In the first embodiment, the weighted addition has been performed using pixels located in the insertion direction of the puncture needle using a puncture tool enhancement filter having a step shape. However, in the present embodiment, neighboring pixels are used regardless of whether the pixels are located in the insertion direction of the puncture needle, whereupon the pixels located in the insertion direction of the puncture needle have a large filter coefficient.
• The puncture tool enhancement filter according to the second embodiment is created in advance by the user and stored in the filter storage unit. The puncture tool enhancement filter according to the second embodiment is made up of an odd number of pixels by an odd number of pixels so that the target pixel is located at the center of the filter.
• A method of determining the filter coefficients of the respective pixels of the puncture tool enhancement filter according to the second embodiment will be described.
• In the second embodiment, for example, the puncture tool enhancement filter can be generated by applying a Gaussian function expressed byEquation 1 applied in the first aspect of the invention.
• InEquation 1, as described in the first aspect of the invention, when μx=μy=0, σ_x²=σ_y²=40, and ρ=0.9, it is possible to create a filter having the size of 81×81 pixels schematically shown inFIG. 5B.
• By linearly interpolating the filter shown inFIG. 5B so as to have the sizes of the respective puncture tool enhancement filters, the filter coefficients used for the respective puncture tool enhancement filters are generated. The filter having the size of 81×81 pixels shown inFIG. 5B can be linearly interpolated so as to become a puncture tool enhancement filter having a size of 15×27 pixels (seeFIG. 14D), for example. A puncture tool enhancement filter having the size of 15×27 pixels obtained in this way is a puncture tool enhancement filter used when the insertion angle is 10°. The aspect ratio obtained by the linear interpolation is determined based on the insertion angle. In the puncture tool enhancement filter, the filter coefficient is the largest at the center, with the magnitude of the filter coefficient widely varying along the insertion direction of the puncture needle. A target pixel at the center is subjected to weighted addition using the neighboring 15×27 pixels around the target pixel. The value of the target pixel is obtained by performing weighted addition which involves a multiplication of the values of the respective pixels by the filter coefficient of the puncture tool enhancement filter. A puncture tool enhancement filter having an aspect ratio corresponding to the insertion angle created in this way is stored in the filter storage unit.
• The ultrasound image generation apparatus according to the present embodiment can determine the puncture tool enhancement filter to be used based on the insertion angle, perform the puncture tool enhancement processing of performing weighted addition with neighboring pixels on all pixels using the determined puncture tool enhancement filter, and generate an image in which the puncture tool is enhanced.
• FIG. 35A shows a B-mode image which has not been subjected to the puncture tool enhancement processing, andFIG. 35B shows a combined image in which a B-mode image after application of the puncture tool enhancement filter is combined with the B-mode image before the puncture tool enhancement processing. The combined image shown inFIG. 35B is obtained by causing defocusing of the B-mode image before the puncture tool enhancement processing in the direction of the insertion angle. InFIGS. 35A and 35B, for better understanding of the effect of the filter processing, speckle noise removal processing, layer structure removal processing, and edge enhancement processing were not performed. The ultrasound image generation apparatus according to the second embodiment of the invention performs the puncture tool enhancement processing on the image in which the speckle noise and the layer structure are removed, and combines a B-mode image after edge enhancement processing with the B-mode image before the puncture tool enhancement processing.
• In this example, a case of converting into a size of 15×27 pixels has been described as an example, although, as described above, in puncture tool enhancement filters having different sizes, filter coefficients corresponding to the sizes of the respective puncture tool enhancement filters can be generated by linearly interpolating the base filter having the size of 81×81 pixels shown inFIG. 5B.
• As described above, according to the ultrasound image generation apparatus according to the second embodiment of the present aspect, weighted addition is performed on all pixels using a rectangular puncture tool enhancement filter having an aspect ratio corresponding to the insertion angle so that pixels located in the insertion direction of the puncture needle have a large filter coefficient. Thus, it is possible to make the B-mode image continuous in the insertion direction of the puncture needle and to generate an image in which the puncture needle displayed in a discontinuous manner is made continuous. Moreover, since an image after application of the puncture tool enhancement filter is combined with the B-mode image before application of the puncture tool enhancement filter, it is possible to generate an ultrasound image in which the puncture needle is displayed so as to be easily understood by the user.
• Moreover, since the puncture tool enhancement filter according to the second embodiment has a sufficiently large size, the puncture tool enhancement filter is not likely to be affected by speckle noise. Thus, the speckle noise removal processing may not be performed.
• In the present embodiment, although six kinds of shapes of the puncture tool enhancement filters are used, a larger number of shapes may be used. Moreover, although puncture tool enhancement filters having shapes corresponding to the range of insertion angles between 10° and 60° are prepared, puncture tool enhancement filters having shapes corresponding to insertion angles of 10° or less and 60° or more may be prepared.
• In the present embodiment, although the puncture tool enhancement filter to be used is determined based on the insertion angle, it is not always necessary to determine the puncture tool enhancement filter based on the insertion angle. For example, the puncture tool enhancement filter may be determined based on an insertion angle acquired from an image. That is, high-luminance points corresponding to the tip end of a puncture needle may be extracted from a plurality of B-mode images, the insertion angle of the puncture needle may be acquired from the plurality of tip positions, and the puncture tool enhancement filter may be determined based on the insertion angle.
Third Embodiment
• In the first and second embodiments of the present aspect, the puncture tool enhancement processing was performed on the B-mode image using the puncture tool enhancement filter having a shape corresponding to the insertion angle of the puncture needle. However, in the third embodiment, the B-mode image is rotated in accordance with the insertion angle, and the puncture tool enhancement processing is performed using the same puncture tool enhancement filter.
• FIG. 36 is a block diagram showing a configuration of main parts of an ultrasoundimage generation apparatus500 according to the third embodiment of the present aspect. The same constituent elements as the ultrasoundimage generation apparatus300 described in the first embodiment will be denoted by the same reference numerals, and description thereof will not be provided.
• The ultrasoundimage generation apparatus500 shown inFIG. 36 is different from the ultrasoundimage generation apparatus300 shown inFIG. 27, mainly in that a puncture toolenhancement data generator502 has a different configuration.
• FIG. 37 is a functional block diagram showing a more detailed configuration of a puncture tool enhancement data generator of the ultrasound image generation apparatus shown inFIG. 36.
• The puncture toolenhancement data generator502 shown inFIG. 37 includes thespeckle noise remover322, thelayer structure remover324, afirst image rotator504, afilter storage unit506, afilter application processor508, asecond image rotator510, and theedge enhancement processor330.
• Thefirst image rotator504 rotates a B-mode image (image data) in which a layer structure is removed by thelayer structure remover324 by an amount corresponding to the insertion angle stored in the puncture toolinformation storage unit310. Specifically, theimage rotator504 performs an image rotation process on the B-mode image data so that the puncture needle is displayed horizontally.
• Thefilter storage unit506 stores a puncture tool enhancement filter ideal when the puncture needle is displayed horizontally. The puncture tool enhancement filter has a size based on the interval with which the puncture needle is discontinuous within the B-mode image. Specifically, the puncture tool enhancement filter has a size such that a horizontal width is slightly larger than the interval with which the puncture needle is discontinuous within the B-mode image. That is, when the puncture tool enhancement filter is applied to a region where the puncture needle is discontinuous within the B-mode image after rotation, a part of the puncture needle is always included. The discontinuance interval of the puncture tool is measured in advance by the user based on the B-mode image.
• Thefilter application processor508 performs the puncture tool enhancement processing on the B-mode image (image data) after rotation using the puncture tool enhancement filter stored in thefilter storage unit506.
• Thesecond image rotator510 rotates the B-mode image data so that the B-mode image after the puncture tool enhancement processing is displayed with the angle before the B-mode image is rotated by thefirst image rotator504.
• FIG. 38 shows a B-mode image after rotation by thefirst image rotator504. The B-mode image is rotated by an angle based on the insertion angle so that the puncture needle becomes horizontal as shown inFIG. 38. For example, when the puncture needle is inserted from the top left corner to the bottom right corner, and the insertion angle is 30°, the B-mode image is rotated counterclockwise by 30°.
• The puncture tool enhancement filter according to the third embodiment is created in advance by the user and stored in thefilter storage unit506 similarly to the second embodiment. The puncture tool enhancement filter stored in thefilter storage unit506 is a filter ideal when the puncture needle is displayed horizontally.
• FIG. 39 is an example of a Gaussian filter serving as the basis when allocating filter coefficients to the respective pixels in the puncture tool enhancement filter used in the third embodiment. The 2D coordinates on the lower side represent positions, and the vertical axis represents a filter coefficient. This Gaussian filter is a 2D Gaussian filter in which the average μ=0, the variance σ²=25, and the correlation value ρ=0. The filter coefficient of the Gaussian filter reaches its maximum at the center of the 2D coordinate system, and decreases as the distance from the center increases. Pixels on a concentric circle about the center of the 2D coordinate system have the same filter coefficient. By linearly interpolating the Gaussian filter, a puncture tool enhancement filter which has a size with a horizontal width slightly larger than the interval with which the puncture needle is discontinuous within the B-mode image, and which is ideal when the puncture needle is displayed horizontally.
• The ultrasoundimage generation apparatus500 applies the puncture tool enhancement filter ideal when the puncture needle is displayed horizontally to the B-mode image (image data) rotated so that the puncture needle is displayed horizontally. That is, the puncture tool enhancement filter used is a filter ideal when the puncture needle is displayed horizontally regardless of the insertion angle. The ultrasoundimage generation apparatus500 rotates the B-mode image (image data) after the puncture tool enhancement processing so that the puncture needle has the original angle, and then performs edge enhancement processing. The ultrasoundimage generation apparatus500 combines the B-mode image before the puncture tool enhancement processing with the B-mode image to which the puncture tool enhancement filter has been applied and which has been subjected to the edge enhancement processing and displays a combined image.
• As described above, according to the ultrasoundimage generation apparatus500 according to the third embodiment of the invention, the B-mode image is rotated in accordance with the insertion angle, and the puncture tool enhancement filter is applied to the B-mode image in which the puncture needle is displayed horizontally, whereby the B-mode image is made continuous in the insertion direction of the puncture needle. Thus, it is possible to generate an image in which the puncture needle displayed in a discontinuous manner is made continuous. Moreover, since the B-mode image is rotated in accordance with the insertion angle to generate an image in which the puncture needle is displayed horizontally, it is only necessary to prepare just a puncture tool enhancement filter ideal when the puncture needle is displayed horizontally. Thus, it is not necessary to prepare a plurality of puncture tool enhancement filters corresponding to the insertion angle. Moreover, since the B-mode image to which the puncture tool enhancement filter has been applied is combined with the original B-mode image, it is possible to generate an ultrasound image in which the puncture needle is displayed so as to be easily understood by the user.
• The speckle noise removal processing may not always be performed. When the puncture tool enhancement filter is applied to a B-mode image in which the speckle noise is removed, it is possible to decrease the size of the puncture tool enhancement filter.
• As described above, according to the third aspect of the invention, the puncture tool enhancement processing is performed on the B-mode image using the puncture tool enhancement filter corresponding to the insertion angle of the puncture needle, and the B-mode image after the puncture tool enhancement processing is combined with the B-mode image before the puncture tool enhancement processing. Thus, it is possible to generate an image in which the puncture needle displayed in a discontinuous manner is made continuous.
• In the respective embodiments, although the puncture tool enhancement processing and the image combination process have been performed on the B-mode image before scan conversion, the processes may be performed after scan conversion. That is, the scan conversion may be performed before the puncture tool enhancement data is generated. Thus, the scan conversion may be performed by theultrasound image generator306 and may be performed by theultrasound image combiner312. Moreover, in the respective embodiments, the B-mode image after the puncture tool enhancement processing has been superimposed on the B-mode image before the puncture tool enhancement processing to generate the combined B-mode image. However, the B-mode image after the puncture tool enhancement processing and the B-mode image before the puncture tool enhancement processing may be subjected to scan conversion and combined so as to be arranged in a parallel arrangement to generate a combined B-mode image of a parallel arrangement.
• Moreover, in the respective embodiments, although the insertion angle was output from the puncture adapter, it is not always necessary to output the insertion angle from the puncture adapter. For example, the user may measure and input the insertion angle while seeing an ultrasound image, and may check the setting of the puncture adapter and store the insertion angle in advance in the puncture toolinformation storage unit310.
• Moreover, in the respective embodiments, although the puncture tool enhancement filter or filters have been stored in advance in the filter storage unit, the user may create a puncture tool enhancement filter ideal for a patient. For example, the user may create a new puncture tool enhancement filter by inputting the filter size, variance, average, correlation value, and the like as the setting items of the puncture tool enhancement filter. The newly created puncture tool enhancement filter is preferably stored in the filter storage unit so as to be used as necessary.
• Moreover, in the respective embodiments, the layer structure removal processing may not always be performed. However, the layer structure removal processing has the following advantage. That is, when the puncture tool enhancement filter is applied to a B-mode image in which the layer structure is removed, it is possible to remove connected portions other than the puncture needle and to improve the effect of application of the puncture tool enhancement filter.
• In the respective embodiments, the edge enhancement processing may not always be performed. However, the edge enhancement processing has the following advantage. That is, since it is possible to enhance the edges between the puncture needle and the other portions, the user can easily recognize the position of the puncture needle. Moreover, although a 1D edge enhancement processing in the vertical direction to the puncture needle is performed as the edge enhancement processing, the edge enhancement processing is not limited to the 1D edge enhancement processing but may be a 2D edge enhancement processing in the vertical direction to the puncture needle, for example.
• In the respective embodiments, although a process using a median filter has been performed as the process of removing the speckle noise, the process of removing the speckle noise is not limited to the process using the median filter. For example, a spatial compounding method, a frequency compounding method, morphology processing, or the like may be performed.
• Moreover, in the respective embodiments, a plurality of images after application of the puncture tool enhancement filter may be generated at different points in time, and the respective images may be averaged to generate a time-averaged B-mode image. Moreover, a 3D filter that appropriately changes the filter coefficient in accordance with time may be applied.
• Moreover, in the respective embodiments, a puncture tool connection process may be performed on an image after application of the puncture tool enhancement filter so that the parts of the discontinuous puncture needle within the image are connected together. For example, the image after application of the puncture tool enhancement filter is binarized to extract high-luminance points, and the extracted high-luminance points are subjected to Hough transform, whereby a line that connects the parts of the discontinuous puncture needle is generated. When the generated line is displayed so as to be superimposed on the puncture needle, it appears to the user that the parts of the discontinuous puncture tool are connected together. The puncture tool connection process may not always be performed on the image after application of the puncture tool enhancement filter, but may be performed on an image after the edge enhancement processing, for example.
• The ultrasound image generation apparatus and the ultrasound image generation method of the third aspect of the invention have the above-described configuration.
• Although the respective configurations in the respective aspects of the invention are realized by a combination of a central processing unit (CPU) and software for causing the CPU to execute various processes, the configurations may be realized by digital circuits or analog circuits. The software is stored in an internal memory and is not shown.
• Moreover, when an algorithm of the ultrasound image generation method according to the invention is described in a program language and compiled as necessary, and an ultrasound image generation program is stored in a memory (recording medium) and executed by an information processor of another apparatus, the same functions as the ultrasound diagnostic apparatus and the ultrasound image generation apparatus according to the respective aspects of the invention can be realized. That is, a program for causing a computer (CPU) to execute the ultrasound image generation method of the invention and a recording medium with the program recorded thereon are also included in the embodiment of the invention.
• While the ultrasound diagnostic apparatus, the ultrasound image generation apparatus, and the ultrasound image generation method according to the invention have been described by way of various embodiments and examples, the invention is not limited to these embodiments and examples, and various improvements and changes can be made without departing from the scope of the invention.
• The ultrasound diagnostic apparatus, the ultrasound image generation apparatus, and the ultrasound image generation method according to the invention can be used when acquiring tomographic images of a subject to be examined into which a puncture tool is inserted using ultrasound waves.

Claims (28)

What is claimed is:

1. An ultrasound diagnostic apparatus comprising:

an ultrasound transceiving unit that transmits ultrasound waves toward a subject to be examined into which a puncture tool is inserted, receives reflected waves of the ultrasound waves reflected from the subject and the puncture tool, and generates echo signals of time-sequential frames based on the received reflected waves;

an ultrasound image generation unit that generates an ultrasound image of the subject based on the echo signals generated by the ultrasound transceiving unit;

an image display unit that displays the ultrasound image generated by the ultrasound image generation unit;

a differential image generation unit that generates a differential image between time-sequential frames from the ultrasound images of time-sequential frames generated by the ultrasound image generation unit;

a tip candidate detection unit that performs a tip detection process based on the differential image generated by the differential image generation unit to thereby detect at least one tip candidate including a tip end of the puncture tool; and

a tip candidate processing unit that highlights a tip candidate of the puncture tool detected by the tip candidate detection unit to thereby generate a tip image in which one or more tip candidates of the puncture tool are highlighted,

wherein the image display unit displays the tip image of the puncture tool highlighted by the tip candidate processing unit so as to be superimposed on the ultrasound image generated by the ultrasound image generation unit.

2. An ultrasound image generation method comprising:

transmitting ultrasound waves to a subject to be examined into which a puncture tool is inserted;

receiving reflected waves of the ultrasound waves reflected from the subject and the puncture tool;

generating echo signals of time-sequential frames based on the received reflected waves;

generating an ultrasound image of the subject based on the generated echo signals;

generating a differential image between time-sequential frames from the generated ultrasound images of the time-sequential frames;

performing a tip detection process based on the generated differential image to thereby detect at least one tip candidate including a tip end of the puncture tool;

highlighting a tip candidate of the puncture tool detected; and

displaying the tip candidate of the puncture tool on a display unit so as to be superimposed on the generated ultrasound image.

3. An ultrasound image generation apparatus in which ultrasound waves are irradiated from a probe toward a subject to be examined into which a puncture tool is inserted, echoes reflected from the subject and the puncture tool are received by the probe, and an ultrasound image is generated from the echo signals output from the probe, the apparatus comprising:

a candidate point extraction unit that extracts plural candidate points of the puncture tool based on the ultrasound image;

a puncture tool presence region specifying unit that specifies a puncture tool presence region in which the puncture tool is likely to be present in the ultrasound image from a distribution of the plural candidate points extracted by the candidate point extraction unit; and

a puncture tool tip position specifying unit that specifies a tip position of the puncture tool based on an intensity distribution on a line that includes the puncture tool in the puncture tool presence region specified by the puncture tool presence region specifying unit.

4. The ultrasound image generation apparatus according toclaim 3, further comprising:

a puncture tool image generation unit that generates a puncture tool image representing the puncture tool based on the tip position of the puncture tool specified by the puncture tool tip position specifying unit; and

an image combination unit that displays the puncture tool image generated by the puncture tool image generation unit so as to be superimposed on the ultrasound image.

5. The ultrasound image generation apparatus according toclaim 4,

wherein the puncture tool presence region specifying unit generates a puncture tool candidate line from the plural candidate points extracted by the candidate point extraction unit and specifies a region including the puncture tool candidate line as the puncture tool presence region, and

wherein the puncture tool image generation unit generates the puncture tool image based on the tip position of the puncture tool and the puncture tool candidate line.

6. The ultrasound image generation apparatus according toclaim 4, further comprising:

a puncture tool information storage unit that stores information on the puncture tool,

wherein the puncture tool image generation unit determines a shape of the puncture tool image using the information on the puncture tool stored in the puncture tool information storage unit.

7. The ultrasound image generation apparatus according toclaim 3,

wherein the puncture tool tip position specifying unit specifies the tip position of the puncture tool based on the maximum and minimum values of the intensity distribution on the line that includes the puncture tool in the puncture tool presence region.

8. The ultrasound image generation apparatus according toclaim 3,

wherein the puncture tool presence region specifying unit performs Hough transform on the plural candidate points extracted by the candidate point extraction unit to thereby generate the puncture tool candidate line.

9. The ultrasound image generation apparatus according toclaim 3,

wherein the candidate point extraction unit performs threshold processing on the ultrasound image to thereby extract the candidate points of the puncture tool.

10. The ultrasound image generation apparatus according toclaim 3, further comprising:

a predicted insertion region setting unit that sets a predicted insertion region to which the puncture tool is highly likely to be inserted in the ultrasound image,

wherein the candidate point extraction unit extracts the candidate points of the puncture tool from the predicted insertion region set by the predicted insertion region setting unit.

11. The ultrasound image generation apparatus according toclaim 3, further comprising:

a predicted insertion region setting unit that sets a predicted insertion region to which the puncture tool is highly likely to be inserted in the ultrasound image,

wherein the puncture tool presence region specifying unit specifies the puncture tool presence region from candidate points included in the predicted insertion region set by the predicted insertion region setting unit among the plural candidate points of the puncture tool extracted by the candidate point extraction unit.

12. The ultrasound image generation apparatus according toclaim 10,

wherein the predicted insertion region setting unit sets the predicted insertion region based on an angle or a position at which the puncture tool is inserted into the subject.

13. The ultrasound image generation apparatus according toclaim 10, further comprising:

a tip position storage unit that stores plural past tip positions of the puncture tool specified by the puncture tool tip position specifying unit,

wherein the predicted insertion region setting unit estimates an advancing direction of the puncture tool from the plural tip positions of the puncture tool stored in the tip position storage unit and sets a region located in the advancing direction of the puncture tool as the predicted insertion region.

14. The ultrasound image generation apparatus according toclaim 3, further comprising:

a target region setting unit that sets a target region where the puncture tool is present; and

an image quality enhancing unit that enhances image quality of the target region set by the target region setting unit in relation to regions other than the target region.

15. An ultrasound image generation method in which ultrasound waves are irradiated from a probe toward a subject to be examined into which a puncture tool is inserted, echoes reflected from the subject and the puncture tool are received by the probe, and an ultrasound image is generated from the echo signals output from the probe, the method comprising:

a candidate point extraction step of extracting plural candidate points of the puncture tool based on the ultrasound image;

a puncture tool presence region specifying step of specifying a puncture tool presence region in which the puncture tool is likely to be present in the ultrasound image from a distribution of the plural candidate points extracted in the candidate point extraction step; and

a puncture tool tip position specifying step of specifying a tip position of the puncture tool based on an intensity distribution on a line that includes the puncture tool in the puncture tool presence region specified in the puncture tool presence region specifying step.

16. An ultrasound image generation apparatus in which ultrasound waves are irradiated from a probe toward a subject to be examined into which a puncture tool is inserted, echo signals reflected from the subject and the puncture tool are received by the probe, and an ultrasound image is generated from the echo signals and displayed on a display unit, the apparatus comprising:

an insertion angle acquisition unit that acquires an insertion angle at which the puncture tool is inserted into the subject;

a puncture tool enhancement processing unit that applies a puncture tool enhancement filter based on the insertion angle to thereby enhance the puncture tool within the ultrasound image;

an image combination unit that combines images before and after the puncture tool enhancement processing to thereby generate a combined image; and

a display controller that causes the combined image to be displayed on the display unit.

17. The ultrasound image generation apparatus according toclaim 16,

wherein the puncture tool enhancement filter has an aspect ratio which is determined by the insertion angle.

18. The ultrasound image generation apparatus according toclaim 16,

wherein the puncture tool enhancement filter has a size which is determined based on an interval on the ultrasound image, at which an image representing the puncture tool is disconnected.

19. The ultrasound image generation apparatus according toclaim 16, further comprising:

a speckle noise removal unit that removes speckle noise in the ultrasound image,

wherein the puncture tool enhancement processing unit performs the puncture tool enhancement processing using the puncture tool enhancement filter on the ultrasound image in which the speckle noise is removed by the speckle noise removal unit.

20. The ultrasound image generation apparatus according toclaim 16, further comprising:

a layer structure removal unit that removes a layer structure in the ultrasound image,

wherein the puncture tool enhancement processing unit performs the puncture tool enhancement processing using the puncture tool enhancement filter on the ultrasound image in which the layer structure is removed by the layer structure removal unit.

21. The ultrasound image generation apparatus according toclaim 16, further comprising:

a speckle noise removal unit that removes speckle noise in the ultrasound image; and

a layer structure removal unit that removes a layer structure in the ultrasound image,

wherein the puncture tool enhancement processing unit performs the puncture tool enhancement processing using the puncture tool enhancement filter on the ultrasound image in which the speckle noise is removed, and subsequently, the layer structure is removed.

22. The ultrasound image generation apparatus according toclaim 16, further comprising:

a puncture tool connection processing unit that performs a puncture tool connection process of connecting disconnected parts of an image representing the puncture tool on the ultrasound image.

23. The ultrasound image generation apparatus according toclaim 16, further comprising:

a filter storage unit that stores plural puncture tool enhancement filters corresponding to the insertion angle,

wherein the puncture tool enhancement processing unit specifies the puncture tool enhancement filter to be used from the plural puncture tool enhancement filters stored in the filter storage unit.

24. The ultrasound image generation apparatus according toclaim 16,

wherein the insertion angle acquisition unit acquires the insertion angle from a puncture adapter which serves as a guide that inserts the puncture tool into the subject.

25. The ultrasound image generation apparatus according toclaim 16,

wherein the insertion angle acquisition unit extracts plural tip positions of the puncture tool from the ultrasound image and acquires the insertion angle from the plural extracted tip positions.

26. An ultrasound image generation apparatus in which ultrasound waves are irradiated from a probe toward a subject to be examined into which a puncture tool is inserted, echo signals reflected from the subject and the puncture tool are received by the probe, and an ultrasound image is generated from the echo signals and displayed on a display unit, the apparatus comprising:

an insertion angle acquisition unit that acquires an insertion angle at which the puncture tool is inserted into the subject;

an image rotation unit that rotates the ultrasound image based on the insertion angle so that an angle of the puncture tool displayed on the display unit becomes horizontal;

a puncture tool enhancement processing unit that applies a puncture tool enhancement filter to thereby enhance the puncture tool within the rotated ultrasound image;

an image combination unit that combines images before and after the puncture tool enhancement processing to thereby generate a combined image; and

a display controller that causes the combined image to be displayed on the display unit.

27. An ultrasound image generation method in which ultrasound waves are irradiated toward a subject to be examined into which a puncture tool is inserted, and an ultrasound image is generated from echo signals reflected from the subject and the puncture tool and is displayed on a display unit, the method comprising:

an insertion angle acquisition step of acquiring an insertion angle at which the puncture tool is inserted into the subject;

a puncture tool enhancement processing step of applying a puncture tool enhancement filter based on the insertion angle to thereby enhance the puncture tool within the ultrasound image;

an image combination step of combining images before and after the puncture tool enhancement processing to thereby generate a combined image; and

a displaying step of causing the combined image to be displayed on the display unit.

28. An ultrasound image generation method in which ultrasound waves are irradiated toward a subject to be examined into which a puncture tool is inserted, and an ultrasound image is generated from echo signals reflected from the subject and the puncture tool and is displayed on a display unit, the method comprising:

an insertion angle acquisition step of acquiring an insertion angle at which the puncture tool is inserted into the subject;

an image rotation step of rotating the ultrasound image based on the insertion angle so that an angle of the puncture tool displayed on the display unit becomes horizontal;

a puncture tool enhancement processing step of applying a puncture tool enhancement filter to thereby enhance the puncture tool within the rotated ultrasound image;

an image combination step of combining images before and after the puncture tool enhancement processing to thereby generate a combined image; and

a displaying step of causing the combined image to be displayed on the display unit.

Images