US20060079745A1 - Surgical navigation with overlay on anatomical images - Google Patents

Abstract

A system and method are provided for control of a navigation system for deploying a medical device within a subject, and for enhancement of a display image of anatomical features for viewing the projected location and movement of medical devices, and projected locations of a variety of anatomical features and other spatial markers in the operating region. The display of the X-ray imaging system information is augmented in a manner such that a physician can more easily become oriented in three dimensions with the use of a single-plane X-ray display. The projection of points and geometrical shapes within the subject body onto a known imaging plane can be obtained using associated imaging parameters and projective geometry.

Description

FIELD OF THE INVENTION
• This invention relates to a system and methods for interventional medicine, and more specifically to computer assisted navigation and imaging of medical devices within a subject body.
BACKGROUND OF THE INVENTION
• Interventional medicine is the collection of medical procedures in which access to the site of treatment is made through one of the subject's blood vessels, body cavities or lumens. For example, angioplasty of a coronary artery is most often performed using a catheter which enters the patient's arterial system through a puncture of the femoral artery in the groin area. Other interventional medical procedures include the assessment and treatment of tissues on the inner surface of the heart (endocardial surfaces) accessed via peripheral veins or arteries, treatment of vascular defects such as cerebral aneurysms, removal of embolic clots and debris from vessels, treatment of tumors via vascular access, endoscopy of the intestinal tract, etc.
• Interventional medicine technologies have been applied to manipulation of instruments which contact tissues during surgical procedures, making these procedures more precise, repeatable and less dependent of the device manipulation skills of the physician. Some presently available interventional medical systems for directing the distal tip of a medical device from the proximal end of the medical device use computer-assisted navigation and a display means for providing a visual display of the medical device along with anatomical images obtained from a separate imaging apparatus. Such systems can provide a visual display of blood vessels and tissues, obtained from a Fluoroscopy (X-ray) imaging system for example, and can display a projection of the medical device being navigated to a target destination using a computer that controls the orientation of the distal tip of the medical device.
• In some cases, it may be difficult for a physician to become oriented in a three dimensional setting using a display of a single-plane X-ray image projection. Enhancement or augmentation of the single-plane X-ray image may be required to aid the physician in visualizing the orientation of the medical device and blood vessels. A method is therefore desired for enhancing a display image of the anatomical surfaces and the orientation of a medical device in real time to improve navigation through the blood vessels and tissues.
SUMMARY OF THE INVENTION
• According to the principles of the present invention, a system and method are provided for control of a navigation system for deploying a medical device within a subject, and for enhancement of a display image of anatomical features for viewing the current location and orientation of a medical device moving through the subject body. The display of the X-ray imaging system information is augmented in a manner such that a physician can more easily become oriented in three dimensions with the use of a single-plane X-ray display. A typical X-ray imaging system comprises a source for emitting a beam through a three dimensional space and onto a plane, where a point within a subject body in the three dimensional space is projected onto the plane. The projection of a point within the subject body onto the imaging plane can be obtained using an orthographic projection matrix derived from the point-to-image plane distance and the source-to-image plane distance. Thus, a point location within the subject body having known coordinates, properly registered to the frame of reference of the X-ray system, can be projected onto the X-ray image plane of the live X-ray image in the same manner.
• In accordance with one aspect of the invention, a method of projection can be used to graphically overlay a representation of the actual medical device location and orientation onto the X-ray image. One or more desired target points within the subject can also be projected onto the X-ray image, as well as one or more reference markers on the subject to track patient movement. A graphical representation of a virtual medical device can be overlaid to show a visual reference of a predicted new location and orientation of the actual medical device that corresponds to a desired navigational configuration. A mathematical model of the medical device can be used to define the configuration of the virtual medical device, which can model the behavior of the device corresponding to a change in navigation control variables to predict deflection and rotation of the medical device. A desired direction for steering the medical device within the plane of the X-ray image can be graphically represented, and surface shapes within the subject may also be rendered and graphically represented on the X-ray image display. All the graphically overlaid information is also updated in real time as the X-ray imaging system is rotated or moved, to augment the image display and enhance visualization of the orientation of a medical device in a three dimensional space using a single-plane X-ray image displayed on the control system.
• It is thus an object of the invention to provide a system and method for augmenting the displayed anatomical image of a subject with graphically overlaid objects to provide enhanced visualization of medical devices, anatomical locations, shapes, markers, and other objects and annotations in a three dimensional space for aiding in the orientation and navigation of the medical device through the subject body.
• It is a further object of the invention to provide a system and method for enabling virtual representation of the medical device, for providing a visual reference of a predicted orientation and location of the medical device corresponding to a desired configuration or movement to a desired target.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a schematic diagram of an automated system for navigating a medical device through the lumens and cavities in the operating regions in a patient in accordance with the principles of this invention;
• FIG. 2 is an illustration of the projection geometry for projecting a location point onto an imaging plane in accordance with the principles of the present invention;
• FIG. 3 is an illustration of an anatomical image display comprising images of the actual medical device, graphically overlaid images of a virtual medical device configuration and a series of target locations according to the principles of the present invention;
• FIG. 4A is a schematic diagram of a navigation system and imaging system combination, in which the navigation system determines the where objects in the operating region should appear based upon information from the imaging system;
• FIG. 4B is a schematic diagram of a navigation system and imaging system combination, in which the imaging system determines where objects in the operating region should appear based upon position information from the navigation system;
• FIG. 5 is a view of the screen of a magnetic navigation system, with imported images from an imaging system in accordance with the principles of this invention;
• FIG. 6A is an x-ray image of an anatomic model of a human heart along an axis 26° on the LAO side, showing objects overlaid on the image in accordance with the principles of operation;
• FIG. 6B is an x-ray image of the anatomic model of the human heart along an axis 26° on the RAO side;
• FIG. 7A is an x-ray image of an anatomic model of a human heart along an axis 25° on the LAO side, showing objects overlaid on the image in accordance with the principles of operation;
• FIG. 7B is an x-ray image of the anatomic model of the human heart along an axis 27° on the RAO side;
• FIG. 8A is an x-ray image of an anatomic model of a human heart along an axis 25° on the LAO side, showing objects overlaid on the image in accordance with the principles of operation;
• FIG. 8B is an x-ray image of the anatomic model of the human heart along anaxis 8° on the RAO side;
• FIG. 8C is an x-ray image of the anatomic model of the human heart along an axis 27° on the RAO side;
• FIG. 9A is an x-ray image of an anatomic model of a human heart along an axis 25° on the LAO side, showing objects overlaid on the image in accordance with the principles of operation;
• FIG. 9B is an x-ray image of an anatomic model of a human heart along an axis 16° on the LAO side;
• FIG. 9C is an x-ray image of an anatomic model of a human heart along an axis 6° on the LAO side;
• FIG. 9D is an x-ray image of an anatomic model of a human heart along an axis 17° on the RAO side;
• FIG. 9E is an x-ray image of an anatomic model of a human heart along an axis 27° on the RAO side.
DETAILED DESCRIPTION OF THE INVENTION
• An automated system for navigating a medical device through the lumens and cavities in an operating region in a patient in accordance with the principles of this invention is indicated generally as20 inFIG. 1. The system20 comprises an elongatemedical device22, having a proximal end and a distal end adapted to be introduced into the operating region in a subject. The system20 also comprises animaging system30 for displaying an image of the operating region on a display32, including a representation of the distal end of themedical device22 in the operating region.
• The system also includes a navigation system for manipulating the distal end of themedical device22. In this preferred embodiment the navigating system is amagnetic navigation system50. Of course, the navigation system could alternatively be a piezoelectric or electrostrictive system or a mechanical control system with pull wires or servo motors, or other suitable system for orienting the distal tip of the medical device. Themagnetic navigation system50 orients the distal end of themedical device22 in a selected direction through the interaction of magnetic fields associated with themedical device22 inside the operating region and at least one external source magnet outside the subject's body. The catheter may then be advanced in the selected direction, to reach the target destination through the successive reorientation stepwise process and advancement.
• A preferred embodiment of the present invention describes a method for a navigation system associated with an elongate flexible catheter or medical device and an X-ray imaging system, for providing a suitable projection of a graphic overlay of the medical device and target locations within the subject body. The control or actuation means used to steer or navigate the medical device with a computer controlled navigation system may be any of a variety of method known to those skilled in the art, such as mechanical, magnetic, electrostrictive, hydraulic, or others. One preferred embodiment is one where an externally applied magnetic field is used to steer the device, while device advancement and retraction is mechanically driven. Such a navigation system is typically used in conjunction with an X-ray system such as a Fluoroscopy Imaging system, with a mutually known registration between the systems. Other anatomical features such as curves, ridge lines, ablation lines, surface portions, landmark locations, marker locations as reference, and so on, possibly including data from preoperative or intraoperative three dimensional images, can be overlaid on the live X-ray display. Past device configurations can also be displayed as a reference so that any changes in configuration such as patient shift can be monitored during the course of the procedure.
• Likewise reconstructed features such as blood vessels reconstructed from contrast agent injection and subsequent imaging and image processing, or other path reconstructions as defined by a user to produce a three dimensional path could be overlaid on the live X-ray display.
• A typical X-ray imaging system comprises a source for emitting a beam through a three dimensional space and onto an imaging plane, where a point within a subject body in the three dimensional space is projected onto the plane. In a preferred embodiment, the X-ray imaging system is preferably a Fluoroscopy imaging system capable of providing images on at least two separate planes, which together can provide the three dimensional coordinates for a location displayed in the two separate planes.FIG. 2 shows a geometric illustration of an X-ray source point oforigin60 for emitting a beam towards the subject and theimaging plane62. The projection of {right arrow over (x)}, apoint64 in a three dimensional space, onto theimaging plane62 as a perspective projection {right arrow over (x)}_p, can be obtained using an orthographic projection matrix. The orthographic projection matrix can be derived from h the point-to-image plane distance72, and d the source-to-image plane distance70, or distance to the center {right arrow over (x)}_cof theplane62. A vector {right arrow over (q)} from a point in space {right arrow over (x)} to the center of the plane {right arrow over (x)}_cmay be defined as {right arrow over (q)}=({right arrow over (x)}−{right arrow over (x)}_c):. The source-to-image distance70 is defined as d. The orthographic projection of {right arrow over (q)} onto theimaging plane72 is:
  {right arrow over (y)}=(I−nn^T){right arrow over (q)} or
  {right arrow over (y)}=A{right arrow over (q)}=A({right arrow over (x)}−{right arrow over (x)}_c)
  where nn^Tis the 3×3 outer product constructed from the normal {right arrow over (n)} to the X-ray image plane, I is the 3×3 identity matrix, and (I−nn^T) is the orthographic projection matrix. FromFIG. 2, it can be seen that:$\begin{array}{cc}\frac{{\overrightarrow{x}}_P-{\overrightarrow{x}}_C}{d}=\frac{\overrightarrow{y}}{\left(d-h\right)},\mathrm{where}h=\left(\overrightarrow{q}·\overrightarrow{n}\right)& \left(1\right)\\ \mathrm{Since}\left({\overrightarrow{x}}_P-{\overrightarrow{x}}_C\right)={\overrightarrow{x}}_P-{\overrightarrow{x}}_C·\frac{\overrightarrow{y}}{\overrightarrow{y}},& \left(2\right)\end{array}$
  Equation (1) may be rewritten as:$\begin{array}{cc}\left({\overrightarrow{x}}_P-{\overrightarrow{x}}_C\right)=\frac{d}{\left(d-h\right)}\overrightarrow{y}=\frac{d}{\left(d-\overrightarrow{n}·\left(\overrightarrow{x}-{\overrightarrow{x}}_C\right)\right)}A\left({\overrightarrow{x}}_P-{\overrightarrow{x}}_C\right)& \left(3\right)\end{array}$
  Equation (3) above defines the perspective projection {right arrow over (x)}_pof point {right arrow over (x)} onto the imaging plane, so {right arrow over (x)}_pmay be rewritten in the form:$\begin{array}{cc}{\overrightarrow{x}}_P={\overrightarrow{x}}_C+\frac{d}{\left(d-\overrightarrow{n}·\left(\overrightarrow{x}-{\overrightarrow{x}}_C\right)\right)}A\left({\overrightarrow{x}}_P-{\overrightarrow{x}}_C\right)& \left(4\right)\end{array}$
• For any given point {right arrow over (x)} in a three dimensional space, a corresponding perspective projection point {right arrow over (x)}_pon the X-ray image plane can be determined using equation (4) above. Thus, for any point location within the imaging volume, a corresponding graphic overlay object may be suitably projected onto the X-ray image display. Such graphic overlay objects that may be suitably projected onto a display as illustrated inFIG. 3 may include objects such as the actualmedical device100 andtarget locations102,104,106 and108 within the operating region of the subject. Other objects that can be usefully overlaid on the live X-ray display include anatomical features such as curves, ridge lines, ablation lines, surface portions, landmark locations, marker locations used as reference, and so on, possibly including data from preoperative or intraoperative three dimensional images. Likewise, previously marked or identified device configurations can also be displayed as a reference so that any changes in configuration due to factors such as patient shift can be monitored during the course of the procedure. Additionally, reconstructed features such as blood vessels reconstructed from contrast agent injection and subsequent imaging and image processing, or other path reconstructions as defined by a user to produce a three dimensional path, or a variety of path-like or other features extracted from three dimensional image data could be overlaid on the live X-ray display.
• As the Fluoroscopic imaging system is moved or rotated about the subject, the graphically overlaid objects may be continuously updated and displayed along with the continuously updated X-ray images to provide projection images in real time to improve visualization of the orientation of the medical device and target locations.
• Other graphic overlay objects that can be suitably projected onto the display may include one ormore reference markers110 on the subject body to provide a reference for the movement of themedical device100. In the preferred embodiment, themedical device100 is preferably deployed from the distal end of a relatively stiff sheath inserted within the subject body. The distal end of such a sheath functions as a base for the distal end of amedical device100 deployed therefrom. One efficient method to mark the pivot or base of the medical device as areference marker110 is to position the tip of themedical device110 at the intended base, for example at the distal tip of a sheath, and then record the current location of the tip as a reference marker, as illustrated inFIG. 3 at110. Reference markers could also be used to indicate target locations for the tip of the medical device to access, and text or other graphic annotations could be used to distinguish and identify various locations. A pre-operative anatomical three-dimensional data set, of an endocardial surface for example, could also be graphically rendered and projected onto the display at118, after a suitable registration of the coordinates to the frame of reference of the X-ray is performed. Likewise, an intra-operative three-dimensional data set could also be graphically rendered and projected onto the image display.
• In the preferred embodiment where a magnetic navigation system is employed for controlling the orientation of the distal tip of the medical device, a graphic annotation of the currentmagnetic field direction116 could be projected onto the live Fluoroscopy Image display as a steering reference. Where a localization system for determining the position of the medical device in a frame of reference translatable to the displayed image of the Fluoroscopy system is also included, a graphical rendition of portions of the medical device as determined from the localization information obtained from the localization system can be overlaid on the X-ray image display. Rates of change of control variables such as the magnetic field, or the rate of movement of the medical device may also be determined and displayed on the X-ray image display.
• A graphical representation of a virtualmedical device112 can be overlaid to show a visual reference themedical device100 being rotated or moved before initiating actual movement of the medical device. A mathematical model of the medical device can be used to define the configuration of the virtualmedical device112, which can model the behavior of the device relative to a desired navigation rotation to predict movement of themedical device110. Thus, a desired rotation or re-orientation of the tip of themedical device110 may be evaluated through a visual representation of a virtualmedical device112 in advance of re-orientation of the actualmedical device110. The model of the virtualmedical device112 can account for the deflective behavior of themedical device110 relative to changes in navigation control variables such as the applied magnetic field direction, and can provide a representation of the resulting changes in configuration of the device. Agraphic indication114 of a direction for steering the medical device within the plane of the X-ray image may also be graphically overlaid onto the display for coordination with a joystick that is mapped to the X-ray plane. Likewise, a desired target such aslocation point102 may be entered, and the model of the virtualmedical device112 configuration can be used to determine the appropriate change in navigation control variables to steer the tip of the medical device to the desiredtarget destination102.
• The imaging display of the present invention may be further augmented by the use of gated location data, for example where the gating is performed with respect to ECG (electro cardiograph) data, so that the device location is always measured at the same phase of a periodic cycle of anatomical motion such as the cardiac cycle. In a preferred embodiment, this data is input into the navigation system together with the real-time location data in a manner such that the location data may be projected onto the X-ray image display.
• It should be noted that the overlay of the medical device and various objects could be controlled by a user input from an input device such as a joystick, mouse, or hand-held localized stylus, or it could automatically be controlled by a computer. Alternatively, a joystick could also be used to control the direction or steering of the medical device within the subject body. Additional design considerations such as the above modifications may be incorporated without departing from the spirit and scope of the invention. More particularly, the system and method may be adapted to medical device guidance and actuation systems other than magnetic navigation systems, including electrostrictive, mechanical, hydraulic, or other actuation technologies. Likewise, a variety of medical devices such as catheters, cannulas, guidewires, microcatheters, endoscopes and others known to those skilled in the art can be remotely guided according to the principles taught herein. Accordingly, it is not intended that the invention be limited by the particular form described above, but by the appended claims.
• Operation
• In operation, the imaging system of the various embodiments of the present invention display an image of an operating region together with an overlay of representations of various objects in the operating region to facilitate the user's orientation within the image. For example these objects can include points that the user has identified or marked, or which have been identified or marked for the user. The objects can alternatively or additionally include shapes, for example closed loops identifying openings in the operating region. The objects can also be reconstructions of medical devices in the operating region, based upon mathematical models of the devices or position information from a localization system. The positions and shapes of the representations automatically change as the imaging plane changes when the imaging beam and imaging plate move about the operating region. Thus the user does not lose the points of reference and landmarks that he or she may have been using prior to the reorientation of the imaging system. This reorientation can occur frequently during medical procedures as the imaging system is moved to accommodate other equipment in the procedure room (e.g. a magnetic navigation system), or when the user desires a different imaging angle to better observe the procedure.
• In one embodiment the imaging system consists of an imaging beam source, an imaging plate, an imaging processor, for processing the imaging data collected by the imaging plate, and a display for displaying the image from the processed imaging data. This imaging system can be used in conjunction with another system, such as a navigation system for orienting the a medical device in the operating region in the subject, or a medical localization system for determining the location of a medical device in the operating region in the subject. Whether using the navigation system or the localization system, the user can generally identify points of interest, for example anatomical land marks or points of physiological interest. Representations of these points can be displayed on the image of the operating region from the imaging system, to help the user visualize the operating region and the procedure. However, in addition to overlaying the representation on a static image from the imaging system, the overlay can be dynamically adjusted as the imaging plane changes so that the objects not only remain on the display, but the remain in the correct position and orientation relative to the displayed image and the displayed image changes.
• The method can be implemented in several ways as illustrated byFIGS. 4A and 4B. In one embodiment, shown schematically inFIG. 4A, the navigation or localization system receives information about the location of the imaging beam source and the imaging receiver, and uses this information to determine where objects of known locations in the operating region should appear on the image generated by the imaging system. More specifically, theimaging system100 can provide the navigation system (or localization system)102 with information about the position/orientation of theimaging beam source104 and theimaging receiver106. (This is represented by arrow108). Using this information the navigation system (or localization system)104 can determine where an object of known position in the operating region should appear in an image generated by the imaging system. This information can be communicated back to theimaging system100 so that the selected objects can be overlaid in the proper location and orientation on the image generated by the imaging system, and displayed on thedisplay110. (This is represented by arrow112)
• As theimaging beam source104 andimaging receiver106 move, the information provided by the imaging system to the navigation system (or the localization system), and the resulting information provided by the navigation system (or the localization system) to the imaging system is updated. (This is represented by arrow114). So that representations of the selected objects can be overlaid on the images from the imaging system are updated as the imaging system moves about the operating region.
• As shown inFIG. 4B, the navigation system (or the localization system)102 can provide the imaging system with the positions of objects in the operating region. Theimaging system100 can use this information to determine where the objects should appear in an image generated by the imaging system, using the known position of theimaging beam source104 andimaging receiver106, and then overlay representations of the objects on the image generated by the imaging system on thedisplay110.
• As the positions of the imaging beam source and imaging receiver change, the imaging system can redetermine where the objects should appear in an image generated by the system in the new configuration, and overlay the representations of the object on the image generated by the imaging system, so that the representations of the objects are updated as the imaging system moves about the operating region.
• An example of a display from a graphical user interface from a magnetic navigation system is shown inFIG. 5. The interface inFIG. 5 allows the user to import images from an x-ray imaging system, and display them in windows in the display. The magnetic navigation system allows the user to identify points in the operating region and show these points on an overlay on the image from the imaging system. The overlay becomes “persistent” such that as the imaging system is moved about the operating region, and another image is made of the operating region, the overlay is adjusted in position and/or orientation so that it correctly shows the points on the new image. This is illustrated inFIG. 5 in which two images from the operating region in different directions are depicted in side by side panes on the interface, and the overlaid objects are properly positioned and oriented in each,
• FIG. 6A shows an x-ray image of an anatomical model of a human heart taken in a direction 26° to the left anterior side. An object, and more specifically a representation of aring200 constructed from a plurality ofpoints202 is overlaid on the x-ray image.FIG. 6B shows an x-ray image of the anatomical model taken in a direction 26° to the right anterior side (i.e., rotated 52° fromFIG. 6A). The representation of thering200 andconstituent points202 inFIG. 6B have been rotated fromFIG. 6A in accordance with the principles of this invention, to remain in the proper orientation with respect to the image in the new imaging direction.
• FIG. 7A shows an x-ray image of an anatomical model of a human heart taken in a direction 25° to the left anterior side. Objects, and more specifically a plurality of annotations including an “E”204, an “F”206, and a “G”208 are overlaid on the x-ray image.FIG. 7B shows an x-ray image of the anatomical model taken in a direction 27° to the right anterior side (i.e., rotated 52° fromFIG. 7A). The representation of the annotations “E”204, “F”206, and “G”208 inFIG. 7B have been rotated fromFIG. 7A in accordance with the principles of this invention, to remain in the proper orientation with respect to the image in the new imaging direction.
• FIG. 8A shows an x-ray image of an anatomical model of a human heart taken in a direction 25° to the left anterior side. An object. and more specifically acatheter210 is overlaid on the x-ray image. The representation ofcatheter210 can be generated from localization data of one or more points on the corresponding real catheter in the operating region. Alternatively, the representation of thecatheter210 can be generated from a mathematical model of the actual catheter in the operating region (for example using the control variable from the navigation system).
• FIG. 8B shows an x-ray image of the anatomical model taken in adirection 8° to the right anterior side (i.e., rotated 33° fromFIG. 8A). The representation of thecatheter210 inFIG. 8B has been rotated fromFIG. 8A in accordance with the principles of this invention, to remain in the proper orientation with respect to the image in the new imaging direction.FIG. 8C shows an x-ray image of the anatomical model taken in a direction 27° to the right anterior side (i.e., rotated 19° fromFIG. 8B). The representation of thecatheter210 inFIG. 8C has been rotated fromFIG. 8B in accordance with the principles of this invention, to remain in the proper orientation with respect to the image in the new imaging direction.
• FIG. 9A shows an x-ray image of an anatomical model of a human heart taken in a direction 25° to the left anterior side. Objects, and more specifically representations ofpoints212,214, and216 in the operating region are overlaid on the x-ray image.FIG. 9B shows an x-ray image of the anatomical model taken in a direction 16° to the right anterior side (i.e., rotated 9° fromFIG. 9A). The representation of thepoints212,214 and216 inFIG. 8B have been rotated fromFIG. 9A in accordance with the principles of this invention, to remain in the proper orientation with respect to the image in the new imaging direction.FIG. 9C shows an x-ray image of the anatomical model taken in a direction 6° to the left anterior side (i.e., rotated 3° fromFIG. 9B). The representation of thepoints212,214, and216 inFIG. 9C has been rotated fromFIG. 9B in accordance with the principles of this invention, to remain in the proper orientation with respect to the image in the new imaging direction.FIG. 9D shows an x-ray image of the anatomical model taken in a direction 17° to the right anterior side (i.e., rotated 23° fromFIG. 9C). The representation of thepoints212,214, and216 inFIG. 9D has been rotated fromFIG. 9C in accordance with the principles of this invention, to remain in the proper orientation with respect to the image in the new imaging direction.FIG. 9E shows an x-ray image of the anatomical model taken in a direction 27° to the right anterior side (i.e., rotated 10° fromFIG. 9D). The representation of thepoints212,214, and216 inFIG. 9E has been rotated fromFIG. 9D in accordance with the principles of this invention, to remain in the proper orientation with respect to the image in the new imaging direction.

Claims (72)

1. A medical imaging system for imaging an operating region in a subject, the system comprising an imaging beam source, an imaging plate, and a display for displaying an image generated by the imaging beam on the imaging plate together with an overlay of at least one object in the operating region, the overlay generated from the position of the object, the position of the imaging beam source, and the position and orientation of the imaging plate.

2. The medical imaging system according toclaim 1 wherein the object is at least one point in the operating region.

3. The medical imaging system according toclaim 1 wherein the object is a loop comprising a plurality of points in the operating region.

4. The medical imaging system according toclaim 1 wherein the object is a medical device in the operating region.

5. The medical imaging system according toclaim 1 wherein the position of the medical device in the operating region is determined by a localization system.

6. The medical imaging system according toclaim 1 wherein the position of the medical device is calculated using a mathematical model of the medical device.

7. The medical imaging system according toclaim 1 wherein the imaging beam source and the imaging plate are movable about the operating region, and wherein the display updates both the displayed image and the overlay as the positions of the imaging beam and imaging plane change.

8. A method of providing medical images of an operating region in a subject's body, the method comprising:

generating an image of the operating region using an imaging beam source and an imaging plate; and

displaying the generated image together with an overlay of at least one object in the operating region, the overlay generated from the position of the object, the position of the imaging beam source, and the position and orientation of the imaging plate.

9. The medical imaging system according toclaim 8 wherein the object is at least one point in the operating region.

10. The medical imaging system according toclaim 8 wherein the object is a loop comprising a plurality of points in the operating region.

11. The medical imaging system according toclaim 8 wherein the object is a medical device in the operating region.

12. The medical imaging system according toclaim 8 wherein the position of the medical device in the operating region is determined by a localization system.

13. The medical imaging system according toclaim 8 wherein the position of the medical device is calculated using a mathematical model of the medical device.

14. The method according toclaim 8 wherein the imaging beam source and the imaging plate are movable about the operating region, and further comprising updating the both the displayed image and the overlay as the positions of the imaging beam and imaging plane change.

15. A medical imaging and display system for displaying medical images of an operating region in a subject, the system comprising:

a localization system for determining the location of an object in the operating region;

an imaging system having an imaging beam source, an imaging plate, and a display for displaying an image of the operating region generated by the imaging beam source and the imaging plate; together with an overlay of at least one object in the operating region, the overlay generated from the position of the object, the position of the imaging beam source, and the position and orientation of the imaging plate.

16. The medical imaging and display system according toclaim 15 wherein the imaging system communicates the position of the imaging beam source and the imaging plate to the localization system, and the localization system calculates the position of the object on the display.

17. The medical imaging and display system according toclaim 15 wherein the localization system communicates the position of the object in the operating region to the imaging system, and the imaging system calculates the position of the object on the display.

18. The medical imaging system according toclaim 15 wherein the object is at least one point in the operating region.

19. The medical imaging system according toclaim 15 wherein the object is a loop comprising a plurality of points in the operating region.

20. The medical imaging system according toclaim 15 wherein the object is a medical device in the operating region.

21. The medical imaging system according toclaim 15 wherein the position of the medical device in the operating region is determined by a localization system.

22. The medical imaging system according toclaim 15 wherein the position of the medical device is calculated using a mathematical model of the medical device.

23. A method of displaying medical images of an operating region in a subject, the method comprising:

generating an image of the operating region using an imaging beam source and an imaging plate; and

displaying the generated image;

determining the location relative to the generated image of an object in the operating region based upon the position of the object in the operating region, the location of the imaging beam source, and the location of the imaging beam plate;

overlaying a representation of the object on the and the on the display to position of an object in the operating region together with an overlay of at least one object in the operating region;

24. A system for navigating a medical device in a body, having a display means for displaying an image of an operating region in the body, the navigation system comprising:

a means of remotely actuating the medical device for navigational purposes;

a connection to an anatomical imaging system, wherein image information can be transferred to the navigation system for viewing on the display means;

transfer of imaging parameters associated with the imaging system to the navigation system;

a means of transformation of three dimensional geometrical information based on the imaging parameters; and

a means of transferring the suitably transformed geometrical information to the imaging system for display on its live imaging monitor.

25. The system ofclaim 24, where the imaging parameters include positions of mechanisms of an X-ray C-arm system.

26. The system ofclaim 24, where the geometrical information is the real-time location and orientation of a localized medical device.

27. The system ofclaim 24, further comprising a graphic overlay of at least one target location that is suitably projected onto the anatomical image shown on the live imaging display.

28. The system ofclaim 24, further comprising a graphic overlay of at least one reference location that is suitably projected onto the anatomical image shown on the live imaging display.

29. The system ofclaim 23, wherein the current location and orientation of the medical device is continuously updated and graphically overlaid onto the anatomical image shown on the live imaging display.

30. The system ofclaim 24, further comprising a mathematical model of a predicted configuration of the medical device that is suitably projected onto the live imaging display, wherein the mathematical model provides a visual reference of a predicted location and orientation of the medical device that corresponds to a desired change in control variables.

31. The system ofclaim 24, further comprising a mathematical model of a predicted configuration of the medical device that is suitably projected onto the live imaging display, wherein the mathematical model provides a visual reference of a predicted location and orientation of the medical device that corresponds to a change in control variables.

32. The system ofclaim 29, wherein the current location and orientation of the medical device is graphically displayed together with other geometrical information in the live imaging display.

33. The system ofclaim 29, wherein the geometrical information transferred to and displayed on the live imaging display is obtained from a three dimensional pre-operative image dataset.

34. The system ofclaim 29, wherein the current location and orientation of the medical device is graphically displayed in the live imaging display together with data from a three dimensional intra-operative surface image rendering.

35. The system ofclaim 24, further comprising data input from a source of anatomical surface normal information, wherein the surface normal information is transformed to the frame of reference of the displayed live image, and is graphically overlaid onto the anatomical image shown on the live imaging display.

36. The system ofclaim 24, wherein the surface normal information is transformed using an orthographic projection matrix incorporating a source to image distance variable.

37. The method ofclaim 24, where the geometrical information comprises anatomical landmarks.

38. The method ofclaim 24, where the geometrical information comprises anatomical curvilinear features.

39. The method ofclaim 24, where the geometrical information comprises anatomical surface portions.

40. The method ofclaim 24, where the geometrical information comprises reconstructed vascular path data.

41. An improved system for remotely navigating a medical device in a body, having a display device for displaying an image of the medical device in the operating region, the improvement comprising:

a means of remote actuation of the medical device;

a means of controlling the remote actuation of the medical device from the navigation system;

transfer of real-time location data to the navigation system from a localization system for determining the position of the medical device;

a connection to an X-ray imaging system, wherein imaging parameters can be transferred to the navigation system from the imaging system, and geometrical information can be transferred from the navigation system to the imaging system for overlaid viewing on the live imaging display device, and a continuously updated current location and orientation of the medical device as determined by the localization system are suitably projected and graphically overlaid onto the live X-ray image

42. The system ofclaim 41, wherein a graphic overlay of at least one target location is suitably projected and overlaid onto the live X-ray image display.

43. The system ofclaim 41, wherein a graphic overlay of at least one reference location is suitably projected and overlaid onto the live X-ray image display.

44. The system ofclaim 41, wherein at least one user-defined device marker is projected and graphically overlaid onto the live X-ray image display.

45. The system ofclaim 41, further comprising a mathematical model of the configuration of the medical device that is suitably projected onto the live X-ray image display, wherein the mathematical model provides a visual reference of a predicted location and orientation of the medical device that corresponds to a desired change in control variables.

46. The system ofclaim 41, further comprising a mathematical model of the configuration of the medical device that is suitably projected onto the live X-ray image display, wherein the mathematical model provides a visual reference of a predicted location and orientation of the medical device that corresponds to a change in control variables.

47. The system ofclaim 41, wherein the current location and orientation of the medical device is graphically displayed on the live X-ray image display together with data obtained from a three dimensional pre-operative image dataset.

48. The system ofclaim 41, wherein the current location and orientation of the medical device is graphically displayed on the live X-ray image display together with data obtained from a three dimensional intra-operative surface image dataset.

49. The system ofclaim 41, further comprising data input from a source of anatomical surface normal information, wherein the surface normal information is graphically overlaid onto the live X-ray image display.

50. The system ofclaim 49, wherein the surface normal information is transformed using an orthographic projection matrix incorporating a source to image distance variable.

51. The method ofclaim 41, where the geometrical information comprises anatomical landmarks.

52. The method ofclaim 41, where the geometrical information comprises anatomical curvilinear features.

53. The method ofclaim 41, where the geometrical information comprises anatomical surface portions.

54. The method ofclaim 41, where the geometrical information comprises reconstructed vascular path data.

55. A system for remotely navigating a medical device in a body with a remote navigation system having a display for displaying an image of operating region, the system comprising:

a means of controlling the remote actuation of the medical device from the navigation system;

a connection to an X-ray imaging system, wherein X-ray imaging parameters can be transferred to the navigation system from the imaging system;

a means of transforming three dimensional geometrical information for appropriate projection and overlay using the X-ray imaging parameters corresponding to the orientation of the current live X-ray image; and

displaying the transformed geometrical information on the live X-ray imaging display.

56. The system ofclaim 55, where the geometrical information comprises anatomical landmarks.

57. The system ofclaim 55, where the geometrical information comprises reference positions.

58. The system ofclaim 55, where the geometrical information comprises target locations.

59. The system ofclaim 55, where the geometrical information comprises predicted device configurations.

60. The system ofclaim 55, where the geometrical information comprises real-time device location and orientation data.

61. The system ofclaim 55, where the geometrical information comprises anatomical curvilinear features.

62. The system ofclaim 55, where the geometrical information comprises anatomical surface portions.

63. The system ofclaim 55, where the geometrical information comprises identified device markers.

64. The system ofclaim 55, where the geometrical information comprises surface normal data.

65. The system ofclaim 55, where the geometrical information comprises geometrical data obtained from three dimensional image data.

66. The system ofclaim 55, where the geometrical information comprises reconstructed vascular path data.

67. A method for overlaying geometrical information onto an X-ray imaging display for use in remotely navigating a medical device in a body, the method comprising the steps of:

connecting a navigation system to an X-ray imaging system, wherein X-ray imaging parameters can be transferred to the navigation system from the imaging system;

transforming three dimensional geometrical information for appropriate projection and overlay using the X-ray imaging parameters corresponding to the orientation of the current live X-ray image;

displaying the transformed geometrical information on the live X-ray imaging display; and

controlling the remote actuation of the medical device from the navigation system using the geometrical information overlaid on the X-ray imaging display.

68. The method ofclaim 67, further comprising the step of providing predicted device configurations for use in controlling the actuation of the medical device.

69. The method ofclaim 67, wherein the geometrical information comprises anatomical landmarks.

70. The method ofclaim 67, wherein the geometrical information comprises reference positions.

71. The method ofclaim 67, wherein the geometrical information comprises target locations.

72. The method ofclaim 67, wherein the geometrical information comprises real-time device location and orientation data.

Images