US20090290768A1 - Remote interpretation of medical images - Google Patents

Abstract

A remote view station is communicatively coupled to an image server and receives a compressed version of source medical images. The remote view station uncompresses and displays the received medical image. A medical professional, such as a pathologist, can select a region of the displayed medical image. Region information is transmitted back to the image server that applies image analysis operations on a region of the source medical image that corresponds to the selected region of the compressed medical image. In this manner, the data loss that occurs during image compression does not effect the image analysis operations. As such, the image analysis operations produce more accurate results than if the operations were applied by remote view station to the uncompressed image.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• This application is a continuation of application Ser. No. 10/803,194 filed Mar. 16, 2004, which in turn is a continuation-in-part of and claims priority to U.S. application Ser. No. 09/542,091, filed on Apr. 3, 2000, which is hereby fully incorporated herein by reference.
BACKGROUND
• Pathologists typically use microscopes when diagnosing physiological conditions such as cancer, infectious disease and prenatal disorders. Typically, tissue samples on a slide are loaded onto the microscope, the microscope objective or lens focuses on an area of the sample, and the sample is scanned for particular features or objects of interest. In this manner, the microscope helps the pathologist to visually determine the presence of abnormal numbers or types of cells, organelles, organisms or biological markers.
• Recently, automated microscopes have been integrated into medical imaging systems that include a variety of networked components. The medical imaging system provides an environment for storing and retrieving the medical images produced by the microscopes. The components of the medical imaging system are spread throughout the department or hospital, or even located remotely, and connected by a communication network.
• An image acquisition device is coupled to the microscope and captures images produced by the microscope. The image acquisition device can include a variety of components. For example, the image acquisition device can include a video camera coupled to a high-speed frame grabber for capturing the stream of video produced by the video camera and generating a series of digital images. Alternatively, the electronic camera can be a megapixel digital camera. The microscope and the image acquisition device can acquire images for a number of different color planes and at several different focal planes. These images can be stitched together to form a two-dimensional or three-dimensional composite image. As a result of the combination, and because the images are typically in color and at high-resolution, the composite images place significant storage and bandwidth requirements on the medical imaging system. For example, a composite image for a single tissue slide can often exceed a gigabyte in size.
• Image storage and archival devices provide a central library for storing the medical images captured by the image acquisition device. Image storage devices include one or more databases and image servers for fast access to recently acquired images. Archival devices, such as optical disc jukeboxes and tape backup systems, provide long-term storage. When a pathologist wishes to view an archived image, the image is automatically “migrated” from the corresponding archival device to one of the image storage devices.
• Diagnostic quality view stations display the images captured by the image acquisition system. In order to assist the pathologist in interpreting a medical image, a view station is able to perform a variety of image analysis operations on the medical image for the purposes of diagnosis. Unlike other types of image processing, image analysis operations are not used to manipulate or produce another image from a subject image, but rather to analyze the information in the subject image to produce a “non-image” result, such as a fixed number (or “score”), often falling within a predetermined range. For example, the pathologist at the view stations may invoke algorithms to perform densitometry on selected regions of the medical image in order to identify concentration of a particular analyte within the tissue sample. Other image analysis operations are useful for finding objects within the image such as the nuclei of the cells, computing an integrated optical density for the nuclei of the cells and reporting the number of molecules per cell.
SUMMARY
• In general, the invention facilitates the remote interpretation of medical images. In order to facilitate the timely diagnosis of a tissue sample, it is desirable that a medical professional, such as a pathologist, be able to remotely view and interpret a medical image. The immense size of a medical image for a single tissue sample typically makes remote viewing unworkable due to bandwidth constraints. Compression algorithms can produce an image suitable for transmission, but the data lost during compression can lead to inaccurate results from the image analysis operations.
• According to one aspect, the invention is directed to a system in which a remote view station is communicatively coupled to an image server and receives a compressed version of a source medical image. The remote view station uncompresses and displays the received medical image. The remote view station selects a region of the displayed medical image as a function of input received from a medical professional, such as a pathologist. Based on the input, the remote view station transmits region information, such as a series of pixel coordinates, back to the image server. The image server applies image analysis operations to a region of the source medical image that corresponds to the selected region of the compressed medical image. In this manner, the data loss that occurs during image compression does not effect the image analysis operations. As such, the image analysis operations produce more accurate results than if the operations were applied by the remote view station on the compressed image.
• In another aspect, the invention is directed to a method for remotely interpreting medical images. According to the method, a compressed medical image is generated from a source medical image and transmitted from an image server to a remote view station for display. In one implementation, the compressed medical image is transmitted over a global packet-switched network such as the Internet. A region of the medical image displayed by the remote view station is selected in response to input from a medical professional. Region information, defining the selected region of the displayed medial image, is transmitted from the remote view station back to the image server. Based on the region information, image analysis operations are applied to a corresponding region of the source medical image. A resulting score is communicated to the remote view station for display. A diagnosis is received from the remote view station and associated with the source medical image in a database maintained by the image server.
• Various embodiments of the invention are set forth in the accompanying drawings and the description below. Other features and advantages of the invention will become apparent from the description, the drawings, and the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a block diagram providing a high-level illustration of the various components of the invention.
• FIG. 2 is a flowchart illustrating one embodiment of a process for remotely interpreting medical images.
• FIG. 3 is a block diagram illustrates one embodiment of a computer suitable for implementing the various embodiments of the invention.
DETAILED DESCRIPTION
• FIG. 1 is a block diagram illustrating asystem10 that facilitates remotely viewing and interpreting medical images.System10 includesremote view station26 this is communicatively coupled to medical imaging system (MIS)12 bynetwork20, which represents any packet-switched network such as a local area network or the Internet. MIS12 includesimage acquisition devices14 that represent any medical imaging device that generates digital medical images, such as an electronic camera used in conjunction with an automated microscope. Other image acquisition devices include computed tomography (CT), nuclear medicine, magnetic resonance imaging (MRI), ultrasound and X-ray devices.Image server24 stores the images that are generated byimage acquisition device14 and, upon request, communicates the images to viewstations16 for display. Usingview stations16, a medical professional, such as a pathologist, can perform a variety of image analysis techniques on selected regions to assist in rendering a diagnosis.
• As described in detail below,image server24 ofMIS12 communicates compressed medical images toremote view station26 for interpretation by a medical professional. Using a network software application, such as a web browser, the medical professional interacts withremote view station26 to select various regions of interest within the image. Based on the selection,image server24 applies image analysis functions directly to the source medical image stored onimage server24, thereby generating a more accurate score than if applied byremote view station26 to the medical image after compression. The resultant score produced by the image analysis operations is communicated toremote view station26 to assist the medical professional in interpreting the medical image and rendering a diagnosis.
• FIG. 2 is a flow chart illustrating one implementation of aprocess28 that facilitates the interpretation of medical images viaremote view station26. Using a network software application the medical professional interacts withremote view station26, accessesMIS12 and receives a list of cases that are marked for review, i.e., cases in which images have been acquired byimage acquisition devices14 but have not been reviewed by a medical professional (30). Upon reviewing the list displayed byremote view station26, the medical professional selects one of the cases for review (32).
• Based on the selection,image server24 compresses medical images associated with the case and communicates the compressed images toremote view station26 via network20 (34).Remote view station26 decompresses the images and displays the uncompressed images for review by the medical professional (36). Theimage server24 may use a lossy compression technique, e.g., JPEG. Such techniques attempt to eliminate redundant or unnecessary information. Consequently, in lossy compression, some information is lost, and the displayed image is not a pixel-by-pixel duplicate of the original source image stored onimage server24. Several algorithms, however, are known that can achieve a compression ratio, such that the compressed image uses relatively low bandwidth, without significantly changing the visual representation of the image. Therefore, compressing the image can be effectively lossless with respect to human vision. For example, it has been found that both JPEG compression and fractal image compression, when set for to moderate compression, result in images suitable for transmission without resulting in data loss perceptible with human vision.
• Using a pointing device, such as a light pen, mouse or track ball, the medical professional selects one or more regions of interest. The medical professional can tag a region for image analysis or can request thatimage server24 transmit image data for the selected region using a lower compression setting. For example, the medical professional can directimage server24 to transmit an image using low compression or even to transmit an uncompressed pixel-by-pixel duplicate of the selected region.Remote view station26 encodes the shape and size of the selected region and transmits the region information toMIS12 by network20 (38). The region information defines the boundaries of the selected regions and, in one implementation, is a set of pixel coordinates defining the outlines of the selected regions. As such, the region information typically comprises a small amount information and can be quickly transmitted to imageserver24.
• If the medical professional requests thatimage server24 transmit image data for a region of interest using a lower compression setting, or no compression, then imageserver24 extracts the corresponding pixel data from the source image and communicates the pixel data to remote view station26 (40). In this fashion, the medical professional can view regions of interest at higher resolution, or even a pixel-per-pixel duplicate, without requiring thatimage server24 transmit the entire source image acrossnetwork20. Upon viewing the region of interest, the medical professional can tag the region for image analysis or can select sub-regions of interest. In an alternative embodiment, theimage server24 may calculate “difference data” for the selected region, e.g., data lost during the lossy compression operation for the selected region. The image server may then transmit the difference data to theremote view station26. Theremote view station26 may then use the difference data and compressed (and/or decompressed) image file to reconstruct an image of the selected region of higher resolution for display at theremote view station26. By sending only the difference data, the amount of data transmitted over the network may be significantly reduced, thereby reducing network traffic and transmission times.
• If the medical professional tags a selected region for image analysis,image server24 executes the requested image analysis operation on the corresponding source medical image (42). More specifically,image server24 analyzes the region information received fromremote view station26 and applies the image analysis operation to a subset of the pixel data of the source image. The subset pixel data is selected based on the boundaries defined by the region information received fromremote view station26. In this manner, the image analysis operation produces a more accurate score than if the operation were applied byremote view station26 to the image that has been compressed for transmission and then uncompressed for display at the remote view station. As such, the data loss that occurs during compression does not effect the image analysis.
• Image server24 communicates the scores for each region toremote view station26 for display to the medical professional (44). Based on the visual display of the medical image as well as the scores associated with regions of interest, the medical professional interprets the medical image and renders a diagnosis. For example, the medical professional may determine that a particular tissue sample is cancerous.Remote view station26 communicates the diagnosis toMIS12 for association with the appropriate case within a database maintained by image server24 (46).
• The invention can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations thereof. Furthermore, the invention can be implemented in one or more computer programs that are executable within an operating environment of a programmable system embodied and tangibly stored in a machine-readable storage device.
• FIG. 3 illustrates an example of acomputer100 suitable for use asview station26 in order to implement or perform various embodiments of the invention. As shown inFIG. 3, thecomputer100 includes aprocessor112 that in one embodiment belongs to the PENTIUM® family of microprocessors manufactured by the Intel Corporation of Santa Clara, Calif. However,computer100 can be implemented on computers based upon other microprocessors, such as the MIPS® family of microprocessors from the Silicon Graphics Corporation, the POWERPC® family of microprocessors from both the Motorola Corporation and the IBM Corporation, the PRECISION ARCHITECTURE® family of microprocessors from the Hewlett-Packard Company, the SPARC® family of microprocessors from the Sun Microsystems Corporation, or the ALPHA® family of microprocessors from the Compaq Computer Corporation.
• Computer100 includessystem memory113, including read only memory (ROM)114 and random access memory (RAM)115, which is connected to theprocessor112 by a system data/address bus116.ROM114 represents any device that is primarily read-only including electrically erasable programmable read-only memory (EEPROM), flash memory, etc.RAM115 represents any random access memory such as Synchronous Dynamic Random Access Memory.
• Within thecomputer100, input/output bus118 is connected to the data/address bus116 viabus controller119. In one embodiment, input/output bus118 is implemented as a standard Peripheral Component Interconnect (PCI) bus. Thebus controller119 examines all signals from theprocessor112 to route the signals to the appropriate bus. Signals between theprocessor112 and thesystem memory113 are merely passed through thebus controller119. However, signals from theprocessor112 intended for devices other thansystem memory113 are routed onto the input/output bus118.
• Various devices are connected to the input/output bus118 includinghard disk drive120,floppy drive121 that is used to readfloppy disk151, andoptical drive122, such as a CD-ROM drive that is used to read anoptical disk152. Thevideo display124 or other kind of display device is connected to the input/output bus118 via avideo adapter125 and preferably is a high-resolution display suitable for viewing medical images.Computer100 also includes amodem129 and network interface53 for communicating overnetwork20 via either a wired or wireless connection.
• A medical professional enter commands and information into thecomputer100 by using akeyboard140 and/or pointing device, such as amouse142, which are connected tobus118 via input/output ports128. Other types of pointing devices (not shown inFIG. 1) include track pads, track balls, joysticks, data gloves, head trackers, and other devices suitable for positioning a cursor on thevideo display124.
• Software applications136 and data are typically stored via one of the memory storage devices, which may include thehard disk120,floppy disk151, CD-ROM152 and are copied to RAM115 for execution.Operating system135 executessoftware applications136 and carries out instructions issued by the user. The Basic Input/Output System (BIOS)117 for thecomputer100 is stored inROM114 and is loaded intoRAM115 upon booting. BIOS117 is a set of basic executable routines that help transfer information between the computing resources within thecomputer100.
• A number of embodiments have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. For example, blocks in the flowchart may be skipped or performed out of order and still produce desirable results. Accordingly, other embodiments are within the scope of the following claims.

Claims (24)

1. A method comprising: generating a compressed medical image from a source medical image at a first location using a lossy compression operation;

transmitting the compressed medical image to a remote view station at a second location for display; decompressing the compressed image file at the remote view station;

selecting a region of the decompressed medical image at the second location; and

at the first location, applying image analysis operations to a region of the source medical image corresponding to the selected region of the decompressed medical image.

2. The method ofclaim 1, wherein transmitting the compressed medical image includes transmitting the compressed medical image over a global packet-switched network.

3. The method ofclaim 1, further comprising:

transmitting region information separate from the compressed medical image from the remote view station to an image server at the first location, wherein the region information defines the selected region of the displayed medical image.

4. The method ofclaim 3, wherein the region information comprises pixel coordinates.

5. The method ofclaim 3, further comprising:

at the first location, receiving from the remote view station a request for improved resolution of the selected region;

determining image data to send to the remote view station to provide improved resolution of the selected region; and sending said image data to the remote view station.

6. The method ofclaim 5, wherein said determining the image data comprises:

identifying pixel data in the source image corresponding to the selected region in the displayed medical image.

7. The method ofclaim 5, wherein said determining the image data comprises:

calculating image data lost in the lossy compression operation.

8. The method ofclaim 1, wherein applying the image analysis operations includes outputting a score and communicating the score to the remote view station for display.

9. A system comprising: an image server at a first location to store a source medical image and to generate a compressed medical image from the source medical image using a lossy compression operation;

a remote view station at a second location communicatively coupled to the image server to receive the compressed medical image, said remote view station including a decoder operative to decompress the compressed medical image to generate a decompressed medical image, a display to display the decompressed medical image, and an input device to enable selection of a region of the decompressed medical image; and

wherein the image server is operative to perform an image analysis operation on a region of the source medical image that corresponds to a selected region of the decompressed medical image.

10. The system ofclaim 9, the remote view station is communicatively coupled to the image server via a global packet-switched network.

11. The system ofclaim 9, wherein the remote view station is operative to transmit region information separate from the compressed medical to the image server, wherein the region information defines the selected region of the decompressed medical image.

12. The system ofclaim 11, wherein the region information comprises pixel coordinates.

13. The system ofclaim 11, wherein the image server is operative to:

receive from the remote view station a request for improved resolution of the selected region;

determine image data to send to the remote view station to provide improved resolution of the selected region; and

send said image data to the remote view station.

14. The system ofclaim 13, wherein said determining the image data comprises:

identifying pixel data in the source image corresponding to the selected region in the displayed medical image.

15. The system ofclaim 13, wherein said determining the image data comprises:

calculating image data lost in the lossy compression operation.

16. The system ofclaim 9, wherein the image server is further operative to:

output a score; and

communicate the score to the remote view station for display.

17. A computer program comprising:

generating a compressed medical image from a source medical image at a first location using a lossy compression operation;

transmitting the compressed medical image to a remote view station at a second location for display;

decompressing the compressed image file at the remote view station; selecting a region of the decompressed medical image at the second location; and

at the first location, applying image analysis operations to a region of the source medical image corresponding to the selected region of the decompressed medical image.

18. The computer program ofclaim 17, wherein transmitting the compressed medical image includes transmitting the compressed medical image over a global packet-switched network.

19. The computer program ofclaim 17, further comprising:

transmitting region information separate from the compressed medical image from the remote view station to an image server at the first location, wherein the region information defines the selected region of the displayed medical image.

20. The computer program ofclaim 19, wherein the region information comprises pixel coordinates.

21. The computer program ofclaim 19, further comprising:

at the first location, receiving from the remote view station a request for improved resolution of the selected region;

determining image data to send to the remote view station to provide improved resolution of the selected region; and

sending said image data to the remote view station.

22. The computer program ofclaim 21, wherein said determining the image data comprises:

identifying pixel data in the source image corresponding to the selected region in the displayed medical image.

23. The computer program ofclaim 21, wherein said determining the image data comprises:

calculating image data lost in the lossy compression operation.

24. The computer program ofclaim 17, wherein applying the image analysis operations includes outputting a score and communicating the score to the remote view station for display.

Images