US20190380790A1 - Methods, systems, and computer readable media for image guided ablation - Google Patents

Abstract

The subject matter described herein includes methods, systems, and computer readable media for image guided ablation. One system for image guided ablation includes an ultrasound transducer for producing a real-time ultrasound image of a target volume and of surrounding tissue. The system further includes an ablation probe for ablating the target volume. The system further includes a display for displaying an image to guide positioning of the ablation probe during ablation of the target volume. The system further includes at least one tracker for tracking position and orientation of the ablation probe during the ablation of the target volume. The system further includes a rendering and display module for receiving a pre-ablation image of the target volume and for displaying a combined image on the display, where the combined image includes a motion tracked, rendered image of the ablation probe and an equally motion tracked real-time ultrasound image registered with the pre-ablation image.

Description

PRIORITY CLAIM
• This application is a continuation of U.S. patent application Ser. No. 15/041,868, filed Feb. 11, 2016, which is a continuation of U.S. patent application Ser. No. 12/842,261, filed Jul. 23, 2010 (U.S. Pat. No. 9,265,572), which is a continuation of PCT International Patent Application No. PCT/US2009/032028, filed Jan. 26, 2009 (Expired), which claims the benefit of U.S. Provisional Patent Application Ser. No. 61/023,268, filed Jan. 24, 2008 (Expired); the disclosures of each which are incorporated herein by reference in their entireties.
GOVERNMENT INTEREST
• This invention was made with government support under Grant Number CA101186 awarded by the National Institutes of Health. The government has certain rights in the invention.
TECHNICAL FIELD
• The subject matter described herein relates to image guided medical treatment systems. More particularly, the subject matter described herein relates to methods, systems, and computer readable media for image guided ablation.
BACKGROUND
• Ablation, such as radio frequency ablation (RFA), microwave ablation, and cryo-ablation, is a first-line treatment for non-resectable hepatic and other types of tumors. RFA is a minimally invasive intervention (MII) uses high-frequency electrical current, introduced—under 2D ultrasound guidance—via a percutaneous needle-like probe, to heat the targeted tissues to physiologically destructive levels. RFA probes are characterized by manufacturer-specified ablation zones that are typically spheres or ellipsoids. The interventional radiologist who performs the procedure must place the probe such that the entire tumor as well as a safety boundary of several millimeters thickness are contained within the predicted ablation area. Frequent tumor recurrence on the periphery of the original tumor [1] indicates that probe placement accuracy may be a major cause for the low 5-year survival rates of hepatic carcinoma patients.
• It is believed that physicians will more accurately target RFA to hepatic and other tumors using a contextually correct 3D visualization system than with standard 2D ultrasound alone. If proven beneficial, 3D guidance could decrease the high post-RFA tumor recurrence rate [3]. Prior experience in developing and evaluating a guidance system for breast biopsy [5] yield results that support this hypothesis.
• Accordingly, there exists a long-felt need for methods, systems, and computer readable media for image guided ablation.
SUMMARY
• The subject matter described herein includes methods, systems, and computer readable media for image guided ablation. One system for image guided ablation includes an ultrasound transducer for producing a real-time ultrasound image of a target volume to be ablated and surrounding tissue. The system further includes an ablation probe for ablating the target volume. The system further includes a display for displaying an image to guide position of the ablation probe during ablation of the target volume. The system further includes at least one tracker for tracking position of the ablation probe during the ablation of the target volume. The system further includes a rendering and display module for receiving a pre-ablation image of the target volume and for displaying a combined image on the display, where the combined image includes a motion tracked, rendered image of the ablation probe and the real-time ultrasound image registered with the pre-ablation image of the target volume.
• The subject matter described herein for image guided ablation may be implemented using a computer readable medium comprising computer executable instructions that are executed by a computer processor.
• Exemplary computer readable media suitable for implementing the subject matter described herein includes disk memory devices, programmable logic devices, and application specific integrated circuits. In one implementation, the computer readable medium may include a memory accessible by a processor. The memory may include instructions executable by the processor for implementing any of the methods described herein for image guided ablation. In addition, a computer readable medium that implements the subject matter described herein may be distributed across multiple physical devices and/or computing platforms.
BRIEF DESCRIPTION OF THE DRAWINGS
• The subject matter described herein will now be explained with reference to the accompanying drawings of which:
• FIG. 1A is an image of an RFA guidance system with a see-through head-mounted display.
• FIG. 1B is a view from inside a head mounted display (HMD) with 3D guidance graphics indicating relationship between needle-like RFA probe and ultrasound image plane;
• FIG. 2A is a see through head mounted display (ST-HMD) view of an ultrasound transducer with infrared LEDs for motion tracking. The dark rectangle below the transducer is an ultrasound image produced by the ultrasound transducer.
• FIG. 2B, from top to bottom, illustrates images of a motion-tracked RFA probe with deployable tines and real-time registered guidance graphics, the ablation region (in this example a sphere) is scaled based on current tine deployment.
• FIG. 3 displays modalities under consideration for the image ablation 3D guidance system, both using optoelectronic tracking (overhead). Left: ST-HMD provides virtual image inside of and registered with the patient (cf.FIGS. 1A and 1B). Right: fish tank VR system shows 3D virtual image above patient (cf.FIGS. 5 through 8).
• FIG. 4 is a block diagram illustrating an exemplary system for image guided ablation according to an embodiment of the subject matter described herein (this is a fish tank VR system as illustrated inFIG. 3, right).
• FIG. 5 is Left: RFA guidance system in use on a woodchuck with liver tumors. The interventional radiologist wears polarized glasses and a large but lightweight head tracker with infrared LEDs. He holds a tracked ultrasound transducer (left hand) and a tracked RFA probe (right hand). The stereoscopic display (a commercial unit consisting of two LCD panels and a half-silvered mirror) is also equipped with an LED tracking panel on the right side. Right: View inside the stereoscopic display shows the transducer, the echography image, and the RFA probe (cf.FIG. 1B). The ablation region (cf.FIG. 2B) is also shown (wireframe sphere). The target volume (tumor) is visible as a partially hollowed out spherical object.
• FIG. 6 is a diagram of a rendered image of a target volume and an ultrasound transducer prior to an ablation pass according to an embodiment of the subject matter described herein.
• FIG. 7 is a rendered image of a target volume, an ultrasound transducer, an RFA probe, and a predicted treatment volume according to an embodiment of the subject matter described herein.
• FIG. 8 is a rendered image of a target volume with the region treated by a prior ablation pass subtracted from the target volume, a predicted treatment volume, and the RFA probe in ultrasound transducers according to an embodiment of the subject matter described herein.
• FIG. 9 is a view of a head-tracked virtual environment suitable for use with an image guided ablation guidance system according to an embodiment of the subject matter described herein. InFIG. 9, the tracked medical instruments (ultrasound transducer and RFA probe are not shown. A tracked hand-held pointer used for eye calibration can be seen.FIG. 10 illustrates an eye calibration setup and sequence, shown for the left eye only.
• FIG. 10 illustrates an eye calibration setup and sequence, shown for the left eye only.
• FIG. 11 is a series of images that illustrate as the user moves about the display, the virtual imagery in the display (in this case a human head for illustrative purposes) is shown from the proper perspective (i.e., from the user's eyes. The three images were photographed with the display's stereo mirror in place (cf.FIG. 5 left and9) and show both eyes' views simultaneously (the stereo mirror reflects the right eye view from the top LCD monitor.
• FIG. 12 is an image of an ultrasound transducer, an ultrasound image, a target volume, and an anatomical context that may be produced by a rendering and display module according to an embodiment of the subject matter described herein.
DETAILED DESCRIPTION
• The subject matter described herein includes methods, systems, and computer readable media for image guided ablation. The following paragraphs describe how an exemplary implementation of the present subject matter was designed, comparing the two designs introduced inFIG. 3: a see through head mounted display and a fish tank virtual reality display.
1. Choosing a Display System
• Our research team has developed 3D guidance for Mils since the mid-1990s; all our systems were based on see-through head-mounted displays (ST-HMDs) [6]. We demonstrated superior targeting accuracy in breast lesions when comparing ST-HMD guidance with the standard 2D method [5]. In addition to stereoscopy and head-motion parallax, the system based on motion-tracked ST-HMDs provided a view of the patient that included a synthetic opening into the patient, showing live echography data and 3D tool guidance graphics in registration with the “real world,” and therefore also with the patient (FIG. 1B) as well as with the motion-tracked instruments (noteFIG. 2A, which shows the ultrasound transducer in an early RFA guidance system prototype based on a video see-through HMD).
• Stereoscopic visualization with head-motion parallax can also be implemented with fixed displays, i.e. without mounting the display on the user's head. Such “fish tank” displays may use CRT monitors and frame-sequential shutter glasses [2], or (at a larger scale) projection displays and passive polarized glasses, for example. Recently, devices based on LCD panels and a semi-transparent mirror have become available from Planar Systems, Inc. [4]; these use passive linearly polarized glasses.
• While we obtained encouraging results in the past with ST-HMD systems, we are disappointed with the bulky and uncomfortable, low-resolution devices resulting from today's state of the art in HMDs. Moreover, since there are no satisfactory video see-through devices on the market, we always constructed our own, with rather modest resources [6]. For these reasons, when designing the RFA 3D guidance system, we considered both an ST-HMD approach and a commercial fish tank system (FIG. 3). With respect to the “augmented reality” (AR) view provided by the ST-HMD, we noted that in Mils—our driving problem—the “interface” between the relevant components of the real world (in our case, the patient, the RFA probe and the ultrasound transducer) and the virtual display (in our case, the echography image, the RFA probe representation inside the patient, and the 3D guidance graphics) is essentially limited to the location where the RFA probe penetrates the skin (FIG. 1A). Furthermore, once the probe pierces the skin, it is moved only lengthwise through this entry point, which is no longer under constant observation by the radiologist. The radiologist then focuses on internal anatomy as he guides the probe into the tumor. From this we conclude that MII (our driving problem) may in fact not derive much benefit from exact registration between real and virtual imagery as provided by an ST-HMD, at least not during the most critical final phase of the probe targeting approach, as the probe tip is introduced into the tumor.
• The above considerations led us to favor a fish tank type display even though it does not offer registration between virtual display and internal patient anatomy. Since our display metaphor proposes life-size representations of the ultrasound image and of the ablation probe, projection displays are unsuitable; and CRT-based stereo has disadvantages such as the requirement for active stereo glasses, which can exhibit flicker. The Planar SD1710 display [4] was almost ideally suited: its small 17-inch 1280×1024 display can fully contain our 3D display elements at life size. Furthermore, it does not exhibit flicker and has manageable bulk.
• FIG. 4 is a block diagram illustrating an exemplary system for image guided ablation according to an embodiment of the subject matter described herein. Referring toFIG. 4, the system includes anultrasound transducer400 for producing a real-time ultrasound image of a target volume to be ablated and surrounding tissue.Ultrasound transducer400 may be any suitable ultrasound transducer, such as the type commonly used for surgery, diagnosis, and patient monitoring. Such a transducer produces a real time ultrasound image of the area of the patient near the contact point of the ultrasound transducer with the patient. One disadvantage associated with ultrasound images is that they are usually two-dimensional and they lack the detail of other image types, such as CT and MRI images.
• InFIG. 4, the system further includes anablation probe402 for ablating the target volume. An exemplary ablation probe suitable for use with embodiments of the subject matter described herein includes the LeVeen RFA needle electrode probe available from Boston Scientific.Probe402 may also include atine deployment tracker404 for tracking deployment of the probe tines (for RFA ablation probes). In one implementation,tine deployment tracker404 may include a light emitting diode attached to the probe to measure movement of the plunger within the probe that indicates the length of the portions of the tines that are deployed. The LED may be mounted on the plunger and its motion relative the probe handle can be observed by a tracking system. In an alternate embodiment, a linear potentiometer may be used to track tine deployment.
• The system illustrated inFIG. 4 further includes aheadband406 to be worn by the user ofablation probe402.Headband406 may include a cluster of infrared or other suitable LEDs for tracking purposes. The purpose ofheadband406 is to track position and orientation of the user's head to be used in rendering a combined image from the viewpoint of the user onto the stereoscopic fish tank display. Atracker408 tracks position of theultrasound transducer400,ablation probe402,headband406, anddisplay410. Any suitable tracking system for tracking such components can be used. In one implementation,tracker408 includes a sensor that senses infrared signals from LEDs mounted toultrasound transducer400,ablation probe402,headband406, anddisplay410 and computes the positions and orientations of these elements from the signals detected from the LEDs. Atriangular LED arrangement501 suitable for trackingdisplay410 is shown inFIG. 5 (left) andFIG. 9. Commercially available trackers suitable for use with the subject matter described herein include infrared trackers available from PhaseSpace or Northern Digital, Inc.
• The subject described herein is not limited to using a fish tank VR display. As stated above, a virtual see through head mounted display may be used without departing from the scope of the subject matter described herein. In an embodiment that uses a virtual see through head mounted display,tracker408 can track both the display and the user'shead using headband406, since the display is worn on the user's head.
• A rendering anddisplay module412 receives the real-time ultrasound image, pre-ablation image data, tracking data fromtracker408, produces combined, stereoscopic, head tracked imagery and displays the imagery ondisplay410. The combined imagery may include a motion tracked, rendered image of the RFA probe, the real-time ultrasound image registered with the pre-ablation image of the target volume, shown from a viewpoint of the user. Exemplary images that may be computed and displayed by rendering anddisplay module412 will be illustrated and described in detail below.
2. Display System Implementation Details
• In one exemplary implementation of the present subject matter, a motion tracker is mounted on the display as in handheld augmented reality applications. Thus, both the tracker base and the stereoscopic display can be moved relative to each other at any time without recalibration to adjust for space and/or line-of-sight constraints within the operating environment; this aims to improve visibility of the tracked system components by the tracker and thereby tracking accuracy and/or reliability. The control software, i.e., rendering anddisplay module412, ensures that the 3D display preserves orientation; e.g., the virtual representations of tracked devices such as the RFA probe in the display are always shown geometrically parallel to the actual devices, in this case thehandheld ablation probe402. The same applies to theultrasound transducer400. In other words, as opposed to the registration in both position and orientation provided by the ST-HMD, this technique maintains only orientation alignment; it introduces a translational offset between the location of the instruments in the real world on the one hand, and their virtual counterparts in the 3D display on the other hand. The interface implemented by rendering anddisplay module412 has three presentation modes that differ in how these user-induced translational movements of the instruments are echoed in the 3D display (orientation changes are always fully shown, as mentioned):
• • A. Centered mode: The ultrasound image is always shown in the center of the 3D display. It is not possible to move the ultrasound transducer such that it leaves the display area.
  • B. Free mode: The user can interactively define the position offset between an area within the patient and the 3D space seen inside the display. Translational motion of the instruments is shown fully within the display, and it is possible to move the ultrasound transducer such that it leaves the display area.
  • C. Delayed mode: This is a combination of the above two modes. The ultrasound image is initially centered as in (A), but the user may move the ultrasound transducer, even outside the display. However after a short lag, the system “catches up” and re-centers the ultrasound image. This allows the user to perceive high-speed translational motion of the ultrasound transducer and image; at low speeds or statically, this is equivalent to (A), at high speeds, to (B).
    For all three modes above, rendering anddisplay module412 continually calculates the appropriate transformations for the RFA probe, in order to always show the correct pose relationship between it and the ultrasound image.
• Given the small size of the display, it is important for the system to accurately track the user's eyes, in order to minimize geometric distortions. A fast and accurate method to calibrate the user's eyes to the head tracker is referenced in the context of which is set forth below [7].
• Table 1 summarizes the principal characteristics of the two display techniques we have considered using for the RFA guidance system (ST-HMD and fish tank VR system).

• TABLE 1
  Characteristics of the two display technologies under consideration

  	See-through HMD	“Fish tank” VR system

  Availability	Custom-designed and	Commercially available
  	built
  Display	Fixed to user's head,	Fixed to room, but
  configuration	motion-tracked with	motion-tracked (can be
  	head	moved)
  Head gear	ST-HMD, tracker	Lightweight glasses,
  		tracker
  Resolution
  	800 × 600 in our	1280 × 1024 in current
  	recent build; higher	device, available at
  	resolution yields	higher resolutions
  	bulkier device
  Registration between	Yes (“true”	Partial only: orientation
  patient and ultrasound	augmented reality)	alignment but offset in
  image (and between		position
  RFA probe and its
  virtual representation)

3. Using the Head-Tracked Fish Tank Stereoscopic Display
• At present there is no controlled study comparing the performance of the head-tracked fish tank display to an ST-HMD device. An interventional radiologist (Charles Burke, MD, UNC Radiology) who has used the head-tracked fish tank display extensively, reports that depth perception is good and that the display correctly portrays three-dimensional relationships during RFA probe targeting. A depth perception study conducted with this display revealed that most subjects (a randomly selected group of 23) were able to determine which of two objects located only a few millimeters apart in depth was closer, based solely on stereoscopic and motion parallax cues provided by the fish tank display.
• The present 3D RF ablation guidance system has been tested on specially constructed liver phantoms; the completed system is currently used in a controlled animal study to ablate liver carcinomas in woodchucks (FIG. 5, left). The study randomizes each woodchuck to either the ultrasound-only conventional guidance method or to the present ultrasound-with-3D-guidance technique.
• According to one aspect of the subject matter described herein, rendering anddisplay module412 may display the target volume, such as the tumor, with successively smaller size as ablated regions are eliminated from display with each ablation pass. Such an image is useful for multiple pass techniques that are aimed to treat a large tumor with multiple overlapping ablations. In one embodiment, an initial target volume to be ablated may be shown as a three dimensional structure on a display screen. The initial target volume may be rendered from the pre-ablation image data, such as MRI or CT image data.FIG. 6 illustrates an example of an initial target volume that may be displayed by rendering anddisplay module412. InFIG. 6, threedimensional region600 represents the initial target volume. Rendering602 representsultrasound transducer400 and its real time orientation. Rendering604 represents the real time ultrasound image continuously produced byultrasound transducer400. Rendering602 is a virtual representation ofultrasound transducer400 or, in other words, the ultrasound transducer's avatar.
• After a first ablation pass, the volume affected by the first ablation pass may be subtracted from the displayed representation of the initial target volume. The volume affected by the first ablation pass may be determined mathematically based on the position of the ablation probe at the time of the first ablation pass, the geometry of the ablation probe, and the tine deployment and power settings of the ablation probe during the first ablation pass. For example, if the probe is the above-referenced LeVeen needle electrode probe, the affected volume for an ablation pass may be determined based on manufacturers specifications. In one current implementation, a constant ellipsoid based on what the probe data sheet indicates is used as the affected ablation volume may be subtracted from the image of the target volume. In alternate implementations, pre-calibrated volumes (shapes measured in a test ablated human-organ-like phantom) or varying the shape based on time deployment can be used to determine the affected sub volume. However, the probes are usually specified to be used with fully deployed times, and manufacturers do not give partial deployment information. Additional bio-chemo-thermo-geometric calibration and simulation work, possibly taking into account fluid flow through blood vessels, may be utilized to increase the accuracy of the affected ablation volume estimates.
• FIG. 7 illustratesinitial target volume600 with the region affected by the first ablation pass shown surrounded by awireframe region700 that represents a predicted ablation area. Arendering702 representingablation probe402 and its current orientation is also shown. Rendering704 represents 3D guidance graphics produced by rendering anddisplay module412. In the illustrated example, the 3D guidance graphics include a calculated projection ofablation probe402 onto the plane defined by the current position and orientation of theultrasound image604; projection lines connectablation probe402's needle to the calculated projection at regular intervals. This is useful in assessing the spatial relationship betweenablation probe402 and the ultrasound image; the probe only appears in the ultrasound image when the two are coplanar, that is, whenablation probe402's needle coincides with its projection and the projective lines have collapsed to zero length. The predicted ablation area may also be determined based on the current position of the probe, the power settings, tine deployment, and the manufacturer's specification. Once the volume determined by the first ablation pass is determined, that volume can be subtracted and the remaining volume can be displayed to the user in the next ablation pass.FIG. 8 illustrates an example of subtracting the volume affected by the first ablation pass frominitial target volume600 illustrated inFIG. 6. InFIG. 8,concave surface area800 illustrates the results of subtracting the volume ofinitial target volume600 affected by the first ablation pass from the initially displayed target volume. This subtracting can be repeated for successive ablation passes untiltarget volume600 is eliminated. The display ofsurfaces600 and800 can be formed in real-time using polygon rendering of the isosurface calculated from the affected volume, for example, using the well known marching cubes isosurface extraction technique. As another example of the display of the target volume affected by successive ablation passes, the image on the right-hand side ofFIG. 5 is another example of volume carving visualization that may be rendered by rendering anddisplay module412 after multiple ablation passes. InFIG. 5 (right),region500 represents the rendering of the target volume from pre-treatment data, such as MRI or CT data.Region604 represents the real-time ultrasound image produced byultrasound transducer400, which is also represented in the display inFIG. 5 by three-dimensional rendering602. It should be noted that in the illustrated example,pre-treatment image500 is registered with real-time ultrasound image604. Further, the combined image is shown from the viewpoint of the user.
• Region504 illustrated inFIG. 5 represents the portion oftarget volume500 that is affected by multiple ablation passes.Wireframe mesh700 represents the treatment volume that will be affected based on the current position of the RFA probe. Such rendering may be particularly suitable for treating large lesions to inform the interventional radiologist of portions of a large tumor that have already been ablated as well as of those that still remain to be treated.
• As stated above, rendering anddisplay module412 may both calculate and display in real-time the amount of tumor and background tissue that would be ablated for the momentary location of the ablation probe, in order to illustrate on the display the impact of probe position. The calculation and display of the amount of tumor and background tissue that would be affected by an ablation can be performed in real-time or may use a lookup table based on the geometry and location of the probe. As stated above, the affected volume can be determined using the data from the probe manufacturer or using experimental data. The volume that would be affected by the ablation can be super imposed about the ablation probe position and displayed to the user.FIG. 7 illustrates an example of displaying the amount of tumor and background tissue that would be ablated for a particular location of the ablation probe. InFIG. 7, relative amounts of healthy and tumor tissue that would be affected by the ablation pass are shown as vertical bars in the lower left hand corner in different shading. Such a display may be useful in probe positioning to maximize the proportion of tumor tissue that is treated with respect to healthy tissue.
• According to another aspect of the subject matter described herein, the guidance system will benefit from accurate registration of the user's eyes for precise head tracked stereoscopic visualization. An exemplary method for accurate registration of the user's eyes for precise head tracked stereoscopic visualization will now be described.
• The high accuracy is achieved in the same calibrated, stereoscopic head-tracked viewing environment used by the guidance system. While the current implementation requires a head-mounted tracker, future embodiments may use un-encumbering tracking, such as vision-based head pose recovery. It is important to note that the technique described here does not require pupil tracking; it uses only head pose, which can generally be obtained less intrusively, with higher reliability, and from a greater distance away than camera-based pupil tracking. An additional pupil tracker is not required unless the system must know the user's gaze direction, for example in order to record user behavior in training-related applications [14].
2. Calibration System for Exact Eye Locations
• The calibration system uses the following main components (FIG. 9):
• • a Planar Systems SD1710 (“Planar”) stereoscopic display with two 17″ LCD monitors and a semi-transparent mirror that reflects the upper monitor's image onto the lower monitor. The user wears linearly polarized glasses that restrict viewing of the lower monitor to the left eye and viewing of the upper monitor's reflection to the right eye. The LCDs' native resolution is 1280×1024.
  • a sub-millimeter precision Northern Digital Optotrak Certus optoelectronic tracking system (“Certus”). Both the Planar and the user's head are tracked by the Certus in all six degrees of freedom with clusters of infrared (IR) LEDs (11 on the head, 4 on the Planar). As mentioned, the advantage of tracking the display as in handheld augmented reality applications [15] is that both the display and the tracker can be moved with respect to each other while the system is running, for example, to improve LED visibility. The Certus also provides a calibration stylus for precise measurements (visible inFIG. 9).
    The user dons the head tracker and performs a simple, fast eye calibration procedure.
2.1. Projection Origin and Eye Calibration
• In fish tank VR systems, the calibration between the head tracker and the eyes is usually obtained from measurements such as the user's inter-pupillary distance (IPD, measured with a pupillometer) [8], the location of the tracker on the user's head, as well as from assumptions about the most suitable location of the projection origin inside the eye. Popular choices for the latter include the eye's 1st nodal point [2], the entrance pupil [9], and the center of the eye [10]. Our method uses the eye center [10] because it is easy to calibrate and yields exact synthetic imagery in the center of the field of view regardless of the user's gaze. However, the 1st nodal point and the entrance pupil are better approximations for the actual optics within the eye. Therefore, by rendering stereo images from the eye centers, i.e. from a few mm too far back, and thus with a slightly exaggerated separation, the EEC system deforms the stereoscopic field [11] ever so slightly. For higher accuracy, a pupil tracker could detect the user's gaze directions, and assuming that the user converges onto the virtual object found along those directions, the rendering and display module could move the projection origins forward to the 1st nodal point, or all the way to the pupil.
Calibration.
• The eye calibration technique (FIG. 10) was inspired by previous methods [12][13] and modified for the additional display tracker. A small panel with a circular hole is temporarily mounted in front of the bottom LCD panel. Both the hole and the bottom LCD monitor are pre-calibrated (one-time only) to the Planar's tracker with the Certus calibration stylus. The eye calibration program shows a circular disk on the display. Using a “mirror image” of the user's head as a guide, the user moves and orients his head to line up the disk through the hole, twice through each eye, under different head orientations. To avoid confusion, users wear frames with one eye masked off, as shown by the “mirror” guides at the top ofFIG. 10. The program collects four line equations in head tracker coordinates. In pairs of two, these four lines define the eye centers at their intersections- or rather, at the closest points between them. The entire task takes 1-2 minutes except for inexperienced first-time users, which take longer mostly because they must receive and follow instructions.
• Since the current head band tracker (FIG. 9) does not guarantee repeatable positioning on the user's head, the user should not remove it between calibration and the following interactive phase (i.e., using the system to guide an ablation). User-specific head-conforming gear equipped with IR LEDs—or with passive markers for camera-based head tracking—could eliminate this restriction and could thus reduce each user's eye calibration to a one-time procedure.
Application of Eye Calibration to Image Guided Ablation
• As stated above, the user's head or eyes can be tracked during image guided ablation and the combined display shown by the rendering anddisplay module412 can adjust the combined display of the treatment volume based on the current position of the user's head and/or eyes. For example, in the images illustrated inFIG. 5, as the user is conducting an RFA procedure, the display illustrated inFIG. 5 may be continually updated based on the viewpoint of the user. As the user's head moves during treatment, the viewpoint of the display will be updated.FIG. 11 shows the stereoscopic display (in this case depicting a photorealistic human head) as seen by a head-tracked user moving around it. Note that the user is able to naturally look around the subject (“head-motion parallax”) as if the display contained actual three-dimensional structures. Together with stereoscopic rendering and exact eye calibration, the illusion is almost perfect.
• Exact eye calibration in an ablation procedure can be used to produce the same 3D effect illustrated inFIG. 11 in the combined image displayed by rendering anddisplay module412 as the user moves aboutdisplay410. For example, the exact eye calibration method described herein is used to determine whether the user's eyes are with respect toheadband406, which is tracked. When the user moves aboutdisplay410, rendering anddisplay module412 uses the position data fromheadband406 and the offset for each eye produced during eye calibration to determine the positions of each of the user's eyes. Rendering anddisplay module412 produces left and right eye images based on the tracked position of headband and the eye calibration data. The left and right eye images appear as a single image ondisplay412 because the user wears stereoscopic glasses. An example of such a combined image viewed through stereoscopic glasses is shown inFIG. 5 (right).
• According to another aspect of the subject matter described herein, rendering anddisplay module412 may render preoperative data, including an anatomical context for the ablation of the target volume. For example, rendering anddisplay module412 may render organs or anatomical structures such as bones or blood vessels adjacent to the target volume.FIG. 12 illustrates an example of such a context.
• The disclosure of each of the following references is hereby incorporated herein by reference in its entirety.
REFERENCES
• [1] O. Catalano et al. “Multiphase helical CT findings after percutaneous ablation procedures for hepatocellular carcinoma.” Abdom. Imaging, 25(6), 2000, pp. 607-614.
• [2] Michael Deering. “High Resolution Virtual Reality.” Proceedings of SIGGRAPH '92, Computer Graphics, 26(2), 1992, pp. 195-202.
• [3] G. D. Dodd et al. “Minimally invasive treatment of malignant hepatic tumors: at the threshold of a major breakthrough.” Radiographics 20(1), 2000, pp. 9-27.
• [4] http://www.planar.com/products/flatpanel_monitors/stereoscopic/[5]
• [5] Michael Rosenthal,AndreiState, Joohi Lee, Gentaro Hirota, Jeremy Ackerman, Kurtis Keller, Etta D. Pisano, Michael Jiroutek, Keith Muller and Henry Fuchs. “Augmented reality guidance for needle biopsies: An initial randomized, controlled trial in phantoms.” Medical Image Analysis 6(3), September 2002, pp. 313-320.
• [6]AndreiState, Kurtis P. Keller, Henry Fuchs. “Simulation-Based Design and Rapid Prototyping of a Parallax-Free, Orthoscopic Video See-Through Head-Mounted Display.” Proc. International Symposium on Mixed & Augmented Reality 2005 (Vienna, Austria, Oct. 5-8, 2005), pp. 28-31.
• [7] Andrei State. “Exact Eye Contact with Virtual Humans.” Proc. IEEE International Workshop on Human Computer Interaction 2007 (Rio de Janeiro, Brazil, Oct. 20, 2007), pp. 138-145.
• [8] Meehan, M., Razzaque, S., Whitton, M., Brooks, F.: Effects of Latency on Presence in Stressful Virtual Environments. Proceedings of IEEE Virtual Reality 2003, IEEE Computer Society, 141-148 (2003).
• [9] Rolland, J. P., Burbeck, C. A., Gibson, W., Ariely, D.: Towards Quantifying Depth and Size Perception in 3D Virtual Environments. Presence: Teleoperators and Virtual Environments 4(1), 24-48 (1995).
• [10] Holloway, R.: Registration Error Analysis for Augmented Reality. Presence: Teleoperators and Virtual Environments 6(4), 413-432 (1997).
• [11] Lipton, L.: Foundations of the Stereoscopic Cinema. Van Nostrand Reinhold (1982).
• [12] Azuma, R., Bishop, G.: Improving Static and Dynamic Registration in an Optical See-Through HMD. Proceedings of SIGGRAPH '94, Computer Graphics, Annual Conference Series, 1994, 197-204 (1994).
• [13] Fuhrmann, A., Splechtna, R., Pikryl, J.: Comprehensive calibration and registration procedures for augmented reality. Proc. Eurographics Workshop on Virtual Environments 2001, 219-228 (2001).
• [14] Raij, A. B., Johnsen, K., Dickerson, R. F., Lok, B. C., Cohen, M. S., Duerson, M., Pauly, R. R., Stevens, A. O., Wagner, P., Lind, D. S.: Comparing Interpersonal Interactions with a Virtual Human to Those with a Real Human. IEEE Transactions on Visualization and Computer Graphics 13(3), 443-457 (2007).
• [15] Billinghurst, M., Henrysson, A.: Research Directions in Handheld AR. Int. J. of Virtual Reality 5(2), 51-58 (2006).
• Although the examples described above relate primarily to RFA, the subject matter described herein is not limited to image guided RFA. The image guided techniques and systems described herein can be used with any type of ablation, including microwave ablation and cryo-ablation. In microwave ablation, a needle delivers microwave energy to the target volume. In cryo-ablation, a needle delivers cold fluid to the target volume.
• The tracking, rendering, and display techniques and systems described above can be used to track, render, and display microwave and cryo-ablation needles in the same manner described above. In addition, the techniques and systems described above for displaying predicted ablation volumes and ablated volumes for successive ablation passes can be applied to microwave and cryo-ablation probes by configuring rendering anddisplay module412 with manufacturer's specifications for these types of probes.
• It will be understood that various details of the subject matter described herein may be changed without departing from the scope of the subject matter described herein. Furthermore, the foregoing description is for the purpose of illustration only, and not for the purpose of limitation, as the subject matter described herein is defined by the claims as set forth hereinafter.

Claims (20)

What is claimed is:

1. A system for image guided ablation, the system comprising:

an ultrasound transducer configured to produce a real-time 2D ultrasound image slice of a target volume and surrounding tissue;

an ablation probe configured to ablate the target volume;

a display configured to display an image to guide positioning of the ablation probe; and

at least one tracker configured to track orientations of the ablation probe, the ultrasound transducer, and a user's head; and

a rendering and display module configured to:

receive the real-time 2D ultrasound image slice from the ultrasound transducer;

receive data regarding the tracked orientation of the ablation probe, the ultrasound transducer, and the user's head produced by the at least one tracker;

determine a perspective view of the real-time 2D ultrasound image slice in a virtual 3D space based at least in part on the tracked orientations of the ultrasound transducer and the user's head;

determine a perspective view of a virtual 3D ultrasound transducer in the virtual 3D space based at least in part on the tracked orientations of the ultrasound transducer and the user's head, wherein the virtual 3D ultrasound transducer corresponds to the ultrasound transducer;

determine a perspective view of a virtual 3D ablation probe in the virtual 3D space based at least in part on the tracked orientations of the ablation probe and the user's head, wherein the virtual 3D ablation probe corresponds to the ablation probe; and

cause the display to concurrently display the perspective view of the real-time 2D ultrasound image slice, the perspective view of the virtual 3D ablation probe, and the perspective view of the virtual 3D ultrasound transducer in the virtual 3D space based at least in part on a location of the ultrasound transducer and the ablation probe.

2. A system for image guided medical care, the system comprising:

a rendering and display module executing on one or more computer processors and configured to:

receive a real-time 2D ultrasound image from an ultrasound transducer;

receive data regarding a tracked orientation of a medical instrument, a tracked orientation of the ultrasound transducer, and a tracked orientation of a user's head;

calculate an orientation of a virtual medical instrument based at least in part on the tracked orientation of the medical instrument, wherein the virtual medical instrument corresponds to the medical instrument;

determine a perspective view of the real-time 2D ultrasound image in a virtual 3D space based at least in part on the tracked orientation of the ultrasound transducer and the tracked orientation of the user's head;

determine a perspective view of the virtual medical instrument in the virtual 3D space based at least in part on the calculated orientation of the virtual medical instrument and the tracked orientation of the user's head; and

cause a display to concurrently display the perspective view of the real-time 2D ultrasound image and the perspective view of the virtual medical instrument based at least in part on a relative location of the ultrasound transducer and the medical instrument.

3. The system ofclaim 2, wherein the medical instrument is an ablation probe and the virtual medical instrument is a virtual ablation probe.

4. The system ofclaim 3, wherein the ablation probe comprises at least one of: a radio frequency ablation (RFA) probe, a microwave ablation probe, or a cryo-ablation probe.

5. The system ofclaim 4, wherein the rendering and display module is further configured to:

determine a perspective view of a virtual ultrasound transducer in the virtual 3D space based at least in part on the tracked orientation of the ultrasound transducer and the tracked orientation of the user's head; and

cause the display to further display the perspective view of the virtual ultrasound transducer concurrently with the display of the perspective view of the real-time 2D ultrasound image and the perspective view of the virtual medical instrument.

6. The system ofclaim 2, wherein the virtual medical instrument is displayed to maintain a parallel orientation with respect to the medical instrument.

7. The system ofclaim 2, wherein the rendering and display module is further configured to determine a trajectory of the medical instrument and cause the display to display a trajectory cue indicating the trajectory of the medical instrument.

8. The system ofclaim 2, wherein the rendering and display module is further configured to cause the display to display a volume that will be affected by an ablation pass for a current position and orientation of the medical instrument and for its operational specifications.

9. The system ofclaim 2, wherein the rendering and display module is further configured to render guidance graphics including:

schematic 3D structures to emphasize a spatial relationship of the real-time 2D ultrasound image and the medical instrument; and

guidance cues between the virtual medical instrument and a plane defined at least in part by the orientation of the real-time 2D ultrasound image.

10. The system ofclaim 9, wherein the guidance cues comprise projection lines connecting the virtual medical instrument to the plane defined at least in part by the orientation of the real-time 2D ultrasound image.

11. The system ofclaim 2, further comprising a tine deployment tracker configured to track deployment of tines of the medical instrument and to provide tine deployment tracking data to the rendering and display module, wherein the rendering and display module is further configured to cause the display to display medical instrument tine deployment.

12. A method for producing an image suitable for image guided ablation, the method comprising:

receiving an ultrasound image from an ultrasound transducer;

receiving data regarding a tracked orientation of an medical instrument, a tracked orientation of the ultrasound transducer, and a tracked orientation of a user's head;

calculating an orientation of a virtual medical instrument based at least in part on the tracked orientation of the medical instrument;

determining a perspective view of the ultrasound image in a virtual 3D space based at least in part on the tracked orientation of the ultrasound transducer and the tracked orientation of the user's head;

determining a perspective view of the virtual medical instrument in the virtual 3D space based at least in part on the tracked orientation of the medical instrument and the tracked orientation of the user's head, wherein the virtual medical instrument corresponds to the medical instrument; and

causing a display to concurrently display the perspective view of the ultrasound image and the perspective view of the virtual medical instrument based at least in part on a relative location of the ultrasound transducer and the medical instrument.

13. The method ofclaim 12, wherein the medical instrument comprises one of: a radio frequency ablation (RFA) probe, a microwave ablation probe, and a cryo-ablation probe.

14. The method ofclaim 12, further comprising:

determining a perspective view of a virtual ultrasound transducer in the virtual 3D space based at least in part on the tracked orientation of the ultrasound transducer and the tracked orientation of the user's head; and

causing the display to concurrently display the perspective view of the ultrasound image, the perspective view of the virtual medical instrument, and the perspective view of the virtual ultrasound transducer based at least in part on a relative location of the ultrasound transducer and the medical instrument.

15. The method ofclaim 12, wherein the virtual medical instrument is displayed to maintain a parallel orientation with respect to the medical instrument.

16. The method ofclaim 12, further comprising determining a trajectory of the medical instrument and causing the display to display a trajectory cue indicating the trajectory of the medical instrument.

17. The method ofclaim 12, further comprising rendering and causing the display to display a volume that will be affected by an ablation pass for a current position and orientation of the medical instrument and for its operational specifications.

18. The method ofclaim 12, further comprising rendering guidance graphics including:

schematic 3D structures to emphasize a spatial relationship of the ultrasound image and the medical instrument; and

guidance cues between the virtual medical instrument and a plane defined by the orientation of the ultrasound image.

19. The method ofclaim 18, wherein the guidance cues comprise projection lines connecting the virtual medical instrument to the plane defined at least in part by the orientation of the ultrasound image.

20. The method ofclaim 12, further comprising tracking deployment of tines of the medical instrument and causing the display to display medical instrument tine deployment.

Images