ELECTRONIC VIDEO ENDOSCOPE WITH NON-SYNCHRONOUS EXPOSURE
Notice of Related Applications This Application is a continuation-in-part of pending United States Patent Application Serial No. 07/988,183 for "Electronic Video Endoscope And Method Of Use", filed December 9, 1992.
FIELD OF THE INVENTION
This invention relates to endoscopes for viewing inside a human patient's body cavity during laproscopic or minimally invasive surgical procedures and, more particularly, to a small-diameter color video endoscope that contains at its distal end both a CCD chip and an immediate low power light source which may be either a miniature white-light, light emitting diode (LED) and/or fiber-optic elements providing white light illumination from an internal low power light source.
BACKGROUND OF THE INVENTION Endoscopes have been used in medicine over the past 40 years to examine internal body organs. These devices were historically a rigid or flexible tube of 10mm or more in diameter. The distal end of the tube was placed inside the patient adjacent to an organ or other bodily innards chosen as an object to be viewed.
The tube usually contained a series of lenses at its distal end, i.e. the end next to the object. These lenses relayed an object's image to a bundle of optical fibers or a series of Hopkins rod lenses. The image was then conveyed along the tube via the fibers or rod lenses and magnified by a series of lenses at its proximal end, i.e. the end closest to the observer. The image projected by the proximal lenses was then viewed with the eye or relayed to a miniature camera for viewing on a monitor.
These classical endoscopes also contained numerous optical fibers to transmit light from a high-intensity light source to illuminate the area being viewed. The light source most frequently used is a 300-watt xenon light. This xenon light source is quite expensive, and it has a characteristic blue color compatible with color CCD camera elements which may be used for viewing.
Due to the tortuous light delivery and recovery paths, and because of the inefficiency of the optical elements in transmitting the light, these classic endoscopes required intense illumination. As the light was delivered and as the image was relayed through the optical system, particularly a fiber-optic transport system, an immense amount of the light was lost. By the time the image passed through the system and returned to the proximal viewing end, over 95% of the light intensity was lost.
To account for this loss, more intensive and powerful xenon light sources were used. However, when the intensity of power of these light sources were increased to account for the light loss, a higher intensity beam was focused upon the object, and the object became vulnerable to being burned. This was especially the case since a large part of the light loss occurred during the recovery transmission when an image of the object was being conveyed for viewing. This loss had to be accounted for at the light source, so the additional light was necessarily first focused upon the object.
If the eye was used for viewing, less intense light was required because of the eye's ability to adapt to the lower light intensity. However, if the image was viewed on a video monitor, higher intensity light was required for illumination. The sensing elements were not sensitive enough to accommodate a low light image. Even though the image quality of these conventional systems has been adequate, the medical community has demanded even higher- quality optical images from their endoscopic systems.
To cater to this demand, electronic endoscopes have been developed to improve image quality. These devices, as discussed within U.S. Patent No. 4,253,447 to Danna et al., U.S. Patent No. 4,854,302 to Allred, U.S. Patent No. 4,742,388 to Cooper et al., U.S. Patent No. 4,667,229 to Cooper et al. and U.S. Patent No. 5,006,928 to Kawajiri et al., have a basic configuration as illustrated in Figure 1.
As depicted in Figure 1, the body 1 of the electronic endoscope is introduced into a patient's body to view an object O. Within external light unit 2a, an external high-intensity light source 2 emits light which passes through a filter 3 to remove infrared light. The filtered light is then collimated by a collimating lens 4 and guided into and through light cable 5, which includes a bundle of fiber-optic elements. The light cable 5 feeds the illumination light through the endoscopic body 1 and delivers it to an illuminating lens 6 which then focuses the light upon the object 0 for viewing.
The illumination light then reflects an image of the object 0. The image is focused by objective lens 7 onto a solid state image pickup device 8. Solid state pickup device 8 converts the image into electrical signals to be amplified, decoded, and passed by control unit 9 to video monitor 10 for viewing.
These electronic videoscopes provide a better quality image, but at a very large expense. Furthermore, some of these systems include a complex color image generation designs. In these color designs, the light source generates a synchronized red-green-blue (RGB) component output timed with the CCD chip electronics in order to generate a color image. Additionally, the need for a separate external light source adds considerable expense to the system and the timing circuit required.
In these synchronized systems, the synchronized red, green, and blue (RGB) lighting is projected onto a black and white CCD chip to produce a timed color image. However, this technique generates a color image that is prone to smearing. An anti-smearing mechanism is discussed in U.S. Patent No. 5,032,913 to Hattori et al. However, the image produced by the system described in U.S. Patent No. 5,032,913 is still less than optimal.
An electronic video endoscope has been developed which incorporates a CCD chip into the distal end of the endoscopic body. An example is produced by Medical Dynamics of Englewood, Colorado. In these systems, the outer diameter of the endoscope must be larger than practical for human medical applications in order to accommodate the CCD chip, and the high-intensity external light source still requires complex and optically lossy couplings. Furthermore, the use of the high-intensity light source still makes the system vulnerable to tissue burning.
Attempts are continuously being conducted to create a color electronic video endoscope which has a solid state image pickup installed within the head of the endoscopic body. However, due to the need to provide a light source and adequately expose the image pick-up, problems have been encountered in the provision of a unit small enough to be practical within a human cavity. These problems have led to various complicated and expensive systems which attempt to compensate for the recognized problems.
One such attempt at advancing the technical development of electronic video endoscopes is shown in U.S. Patent No. 4,602,281 to Nagasaki et al (The 281 Patent) . Theλ28l Patent attempts to create a color endoscopic body which includes both a tri-color image pick-up device and a luminous tri-color (RGB) LED or incandescent light source. However, the device described in theλ281 Patent is extremely complex electronically, costly to produce and technically impossible to create in a miniaturized version which is practically capable of being used in human endoscopic procedures. Theλ281 Patent applies a high cost color CCD system with synchronization circuitry to activate image acceptance. The image is synchronously accepted from the solid state element which is continuously illuminated. The synchronization circuit determines when to download image information from the continuously illuminated solid state pixels. The information is downloaded from the CCD and the entire system cycle is synchronized with the vertical drive (VD) signal of the television monitor system. This limits the system exposure options as disclosed below with respect to U.S. Patent No. 5,187,572.
However, the system described by theΛ281 Patent cannot be adequately miniaturized for practical application in human procedures using current technology. This is due to the excessive electronics which must be utilized to synchronously access each pixel.
In order to operate properly, theΛ281 Patent must use either an "interline" or "frame transfer" CCD. These types of CCDs each have storage sections on the CCD neighboring each pixel and require that the surface geography between the pixels be large. Thus, the system described by the 281 Patent requires that the CCDs have large surface areas for high resolution imaging.
Additionally, further problems are recognized in practical applications of these prior systems. Examples of the problems include the "Blooming" effect discussed within the '281 patent, the low light output of certain color LEDs (Blue) which yield image distortion, and the excessive heat generated by certain LEDs (Red) when they are continuously operated at their maximum intensity and which leads to thermal noise, system damage and tissue burning.
A further system which attempts to provide a color electronic video endoscope is shown in U.S. Patent No. 5,187,572 to Naka ura et al. The '572 Patent system is timed by the TV monitor "VD" (vertical drive) signal, which has a fixed time period by TV standards. The fixed period of the "VD" signal forces the system to operate at the fixed rate. Hence, the CCD must generate a timed full color image (i.e., 3 separate images of Red, Green and Blue) during this fixed time period (e.g. 1/30 second) .
To generate each separate color component image (Red, Green or Blue) the entire content of the CCD must be digitized and transferred to the proper frame memory. Since this process takes a fixed duration and the "VD" signal also has a fixed period, the period of time that a particular color of light can illuminate the scene becomes fixed (at the most 1/3 of the period of VD) . This fixed timing severely limits exposure variations such as when the system is changed from lighter or darker imaging or the object distance changes.
To overcome the exposure problem caused by this fixed period of illumination, the system must provide a motorized diaphragm to control the amount light passing through the color filters by continuously monitoring the image brightness. Thus, the system must provide a specialized light source having motorized diaphragms for illumination control and an elaborate set of position sensors and motor control electronics which increase the system cost. Furthermore, the image is susceptible to blurring and loss of color fidelity due to separation of colors from the motion of the image on the focal plane array (CCD) . These problems are illustrated within Figures 13 and 14. Since the process is time sequenced for each separate RGB component, the system assumes that the relative position of the object and CCD remain fixed from one color frame to the next. However, if the image formed on the focal plane is moved from one primary color exposure to the next (due to relative motion of object and CCD) the three primary color images will be formed on different parts of the CCD (mis-aligned) , and the information in the three frame memories will be mis-aligned and the image is distorted.
Furthermore, these errors increase with an increase in the motion of the image on the focal plane (CCD) . This limits the possible applications of the apparatus. The limiting speed at which the apparatus can be moved is the range of a few centimeter per second. The image quality is highly sensitive to vibrations and other sudden movements.
The costs of the lenses and other complex components internal to the invasive bodies of all of these prior systems also require that they be cleaned and reused from patient to patient. In the medical environment in which these devices are used, this sterilization is of the utmost importance. However, the delicate lenses and components included within the housing make medical sterilization difficult and costly.
Furthermore, the environment in which these electronic endoscopes are used makes them susceptible to being damaged during the course of an operation or procedure. Each of these endoscopes contemplate incorporating at least some portion of the electronics within the distal end of the endoscope which is placed in close proximity to a medical procedure being undertaken within the body.
The electronics are vulnerable to being damaged during the course of that procedure. A scalpel or other sharp or abrasive medical instrument may penetrate the endoscope body, allowing fluid leakage into the endoscope or shorting out the system and causing the system to arc or burn up within the patient.
OBJECTS OF THE INVENTION In view of the above-discussed prior art, it is an object of the present invention to provide a low-cost, high-quality video endoscope system.
It is yet a further object of the invention to provide a video endoscope system which eliminates the requirement for a high-intensity, external light source and avoids tissue burning.
It is yet a still further object of the invention to provide a low-cost, high-quality video endoscope system which can be easily sterilized or is disposable.
It is yet even a still further object of the present invention to provide a low-cost video endoscope system which includes all exposure components within the invasive endoscope body, and which can be manufactured as a unit small enough to be practical for human surgical procedures.
It is yet even a still further object of the present invention to provide an electronic video endoscope which prevents damage to a patient from the electronics being penetrated during the course of a medical procedure.
It is yet even a still further object of the present invention to provide a low-cost video endoscope system which yields exceedingly high-quality optical resolution.
It is yet even a still further object of the present invention to provide an improved method for electronic video endoscopy.
SUMMARY OF THE INVENTION These and other objects are provided by the present invention, which includes an electronic endoscope, otherwise known as a videoscope, that incorporates a miniature CCD chip which enables small-diameter, endoscopes which are medically-applicable to humans to be produced. An aspect of the invention provides an illumination means built directly into the invasive endoscope body.
In the preferred embodiment, the illumination means is a white-light LED built directly into the distal end of the invasive endoscopic body. A white-light LED is used to allow color imaging information to be obtained by allowing the various color components to be detected during a single exposure at the image pickup. In the preferred embodiment, the white-light LED is simply strobed on and off for all color components simultaneously under the control of a CCD timing generator unit. By strobing the white light LED to create an exposure, all of image pixels on the CCD receive imaging light concurrently, and the complicated electronics and synchronization circuitry used by prior systems is eliminated.
Furthermore, by providing an exposure control within an optical path defined by the path the light travels between the light source to the object and the object to the CCD, the geographical structure of the solid-state image element is simplified and can be provided in a miniaturized version capable of invasive procedures on humans. In a first preferred embodiment, the exposure control simply strobes the light source on and off so that the CCD is only illuminated during the allotted exposure time.
In another preferred embodiment, an electro-optical shutter is placed within the optical path. The shutter is strobed on and off so that it is transparent when the CCD is to be illuminated.
In accordance with these embodiments, the invention enables the use of a full frame CCD which does not have on-the-chip storage and is much simpler to manufacture, less costly, easier to use and can be designed in a unit small enough to obtain high resolution color images within an endoscope small enough to be practical for human surgical procedures. It is an aspect of the preferred embodiment of the invention to define the imaging electronics into two independent systems. An image capture system obtains an image of the object and creates image information signals. An image display system controls the display of the image upon a video display.
Since these two systems are time independent and are not synchronized, the image capture system is able to have an unlimited exposure time variation, thereby allowing it to cater to any exposure demands created by the human bodily environment into which it is placed. Furthermore, since the two systems are time independent, there is no need for extensive and complicated synchronization circuitry found in prior art systems.
The preferred embodiment uses a bufferred transfer system to store and transfer the imaging information signals between the two imaging systems in a non- synchronous or non-periodic manner.
The image capture system includes the light source, the CCD, an exposure control unit which sets an exposure time and provides an exposure signal during the exposure time and a timing generator unit which provides timing signals to drive the CCD according to a timing cycle for exposure and downloading. The timing generator unit sets the CCD for image capture when the exposure control unit outputs the exposure signal and provides a signal directing the CCD to transfer the image information signals for display. In the preferred embodiment, the exposure control unit is programmable by a default exposure time at power up, is programmable by an automatic feedback signal changing the exposure time based upon dynamic exposure requirements, or is programmable by a user controlled input.
In the preferred embodiment, the CCD timing generator unit CCDTGU also provides download sequencing signals to a multiplexing circuit. The multiplexing circuit includes respective color component sets of analog to digital converters, buffer memories and conditioning circuits. The sequencing signals allocate serially fed image information signals between the respective color component circuits for processing color component information for display. The preferred embodiment thereby eliminates the need for multiple color component exposures and the synchronization circuitry, complexity, chip geography allocation and color smearing which accompanies multiple color exposures.
A small objective lens focuses the reflected light from the object onto the CCD. Thus, in the preferred embodiment of the invention an external, high-cost high- intensity light source is eliminated, and the videoscope of the claimed invention may be manufactured and sold at a considerably lower cost than conventional electronic endoscopes. Such low-cost endoscopes can be easily discarded to maintain sterilization.
In another aspect of the preferred embodiment, by providing the low-power illuminator, the CCD pickup in the manner used within the preferred embodiment, and a low- cost objective lens directly above the CCD, the system is able to simplify the timing circuitry and incorporate much of the simple electronics directly into the endoscopic head at the distal end of the invasive endoscopic body in a manner practical for human procedures.
In addition to lowering cost, the placement of the components within the endoscopic head allows a quick connection electrical coupling to be included so that the endoscopic housing may be sterilized or even discarded, and which further eases sterilization and fabrication.
An additional aspect of the present invention is to provide an electronic warning system and emergency shut- off which detects leakage into the endoscopic body or if the endoscopic body has been ruptured during the course of the medical procedure.
The system of the present invention further allows a simplified method of endoscopy.
These aspects of the invention are simplified by providing both the light source and imaging planes within the endoscopic head.
BRIEF DESCRIPTION OF THE DRAWINGS The present invention, both as to its organization and manner of operation, together with further objects and advantages, may be understood by reference to the following drawings. Figure 1 is a block diagram of a conventional electronic video endoscope system;
Figure 2 is a block diagram of one preferred embodiment of the video endoscope of the present invention;
Figure 3 is a block diagram of another preferred embodiment for the video endoscope of the present invention;
Figure 4 is a block diagram of another preferred embodiment for the video endoscope of the present invention;
Figure 5 shows a block diagram of the image capture unit of the preferred embodiment of the present invention;
Figure 6 shows a block diagram of the image display unit of the preferred embodiment of the present invention;
Figure 7 shows a timing diagram of the control signals transmitted within the image capture unit of Figure 5;
Figure 8 shows an end-wise cross sectional view of the distal end of the walls of the endoscope body of the preferred embodiment of the present invention with an emergency shut-off wire;
Figure 9 shows a block diagram of the emergency shut- off system of the preferred embodiment of the present invention; Figure 10 shows a longitudinal cross-section of the endoscopic body illustrated in Figure 8;
Figure 11 shows a block diagram of a further image capture system of another preferred embodiment of the invention;
Figure 12 shows a structural cross-sectional diagram of the distal end of a video endoscope including the image capture system of Figure 11;
Figure 13 is an illustration which shows a representation of blurring and color separation occurring from an image travelling across the CCD focal plane;
Figure 14 is an illustration of color smearing;
Figure 15 is an illustration of color component combination for white light exposure of the CCD according to the preferred embodiment of the present invention; and
Figure 16 is an illustration of the "full frame" CCD used in the preferred embodiment of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following description is provided to enable any person skilled in the art to make and use the invention and sets forth the best modes contemplated by the inventors of carrying out their invention. Various modifications, however, will remain readily apparent to those skilled in the art, since the generic principles of the present invention have been defined herein. Figure 2 is a diagram showing the basic components of a video endoscope system constructed according to a first preferred embodiment of the present invention. As shown, the endoscope body housing 11 is an elongated tube that houses the exposure components of the endoscope and their signal conditioning circuits. The housing tube 11 may be flexible or rigid, depending upon the components from which the tube is constructed.
At the distal end of the endoscope body, windows 13 allow light to be transmitted and recovered from the object 0. A movable lens holder 12 positions and holds an objective lens 14 at a predetermined focal distance from the solid state imaging element 15. The lens holder 12 is movable to provide proper focusing by the objective lens 14 of the image upon the imaging element 15. In a further embodiment, not shown in the figures, the lens is mounted directly upon the solid state imaging element 15 during fabrication, and no further components are required.
The imaging element 15 is anchored to the substrate 16 along a first radial end on one side. At the same radial end, and on the opposing side of the substrate 16, a signal conditioning circuit 17 is mounted.
The substrate 16 is secured in place, and fits within a groove G running along the internal circumference of the endoscope body housing 11. Also secured to the substrate 16, along a second opposing radial end, is a miniature white-light LED (light-emitting diode) 18 that provides illumination to the object O. White light is provided to allow the various color component imaging signals to be detected by the imaging element 15 during a single exposure time period Texp as discussed below.
In operation, the imaging system of the preferred embodiment has two separate control systems which each have a separate time base of operation. They are labelled the "image capture system" and the "image display system" for illustration of the interactive exposure aspects of the preferred embodiment of the invention. Figure 5 shows a block diagram of the image capture system and Figure 6 shows a block diagram of the image display system.
These two control systems are not synchronized with each other. Each system has its own time base and clock. They each function independently from the other to allow practically unlimited exposure time variations, as discussed below, and eliminate exposure synchronization circuitry which increases system size and cost.
The image capture system of the preferred embodiment of the invention, as shown in Figure 5, operates based upon the control timing diagram of Figure 7. The image capture system has its own independent system clock CSC which provides the time base by which the entire image capture system operates. In the image capture system, the clock signal CS is fed from the system clock CSC into the CCD timing generator unit CCDTGU and the exposure control unit ECU.
At initial power-up, the image capture system provides an exposure time Texp which determines how long the CCD imaging element is exposed to the object 0 for image capturing. Upon power-up, a reset circuit (monostable multivibrator) generates a pulse. This pulse (on its positive edge i.e., transition from "ground" to "5" volts) does the following:
1. Loads a predetermined value from an exposure time latch (not shown) into the exposure control unit ECU. Hence, setting up the default exposure time Texp; and
2. Resets all of the counters on both CCD timing generator unit CCDTGU and the Display timing generator unit DTGU to "0", hence, initializing the system.
After the initial power-up, the clock pulse signal CS is sent to the exposure control unit ECU and the CCD timing generator unit CCDTGU to provide timing signals for the image capture system operation.
In the preferred embodiment, the exposure time value
Texp is preset and stored in memory. However, it is also capable to allow either the user to alter the exposure time Texp or to provide an automatic feedback system whereby the exposure time Texp is varied automatically based upon the lighting requirements of an object when the endoscope of the embodiment is in use.
It is an aspect of the preferred embodiment of the invention that the exposure time Texp may be altered in a manner which is practically unlimited without affecting the timing or synchronization of any of the other image capture or display functions of the system. The preferred embodiment uses a 16 bit counter to provide over 64,000 time settings. However, by changing the size of the counter, the number of digital settings can be increased. The exposure time Texp is completely independent of any other system timing.
In response to the receipt of a clock pulse signal CS, the exposure control unit ECU begins timing the exposure time Texp and turns on the LEDs by providing signal SI to the LED switches 10R, 10G and 10B, respectively. When the switches 10R, 10G and 10B are turned on current is allowed to flow through resistors 9R, 9G, and 9B and diodes 11R, 11G and 11B and white light is emitted to the object to be reflected back to the CCD for image capture. The exposure control unit ECU continues sending the signal SI to the LED switches 10R, 10G, 10B until the exposure time Texp has expired. When the exposure time Texp expires, the signal SI is discontinued, the LEDs are shut down, and no further light is emitted to the object. Thereby the exposure of the CCD is ended within the optical path of the imaging light at the light source 18 (Figure 2) .
The exposure control unit ECU also sends the signal SI to the CCD timing generator unit CCDTGU to set the timing of the CCD image exposure and image downloading operations. Upon receiving the signal SI, the CCD timing generator unit places the CCD in an image capture mode.
The CCD timing generator unit CCDTGU controls the operation of the CCD through a set of control signals S4 sent to the CCD driving unit CCDDU. The CCD driving unit CCDDU in turn drives the CCD using a corresponding set of driving signals S4A. As a group, these signals S4A, illustrated in Figures 5 and 7, are simply amplified and conditioned representations of control signals S4 delivered from the CCD timing generator unit CCDTGU.
The preferred embodiment of the invention uses a three-phase, full frame charge coupled device. This CCD uses a set of control signals configured in different manners to drive the CCD for its different functions.
By using a full frame CCD in the manner of the preferred embodiment of the invention, the system is able to provide color imaging, yet still eliminate the necessity of a storage section within the CCD chip geography. This in turn allows the CCD element to provide high resolution upon a small enough CCD chip to be placed directly within the distal end of an endoscopic body used for human medical procedures.
Prior electronic endoscopes have applied CCD elements with an on board storage section to allow the CCD to properly yield any recorded imaging information. In prior electronic video endoscopes which have attempted to apply CCD imaging elements within the distal end of the endoscope, the CCDs have been "frame transfer CCDs" or "interline transfer CCDs" which include a pixel storage site adjacent each CCD pixel.
These devices allow each pixel's imaging information to be immediately stored following pixel exposure. The imaging information is then prevented from being disturbed or molested by any further light which impinges upon the CCD pixel element after exposure is complete.
However, CCDs with on-board storage sections require greater chip surface area and extreme control circuit complexity. Furthermore, the surface area and complexity of these CCDs increase geometrically as higher resolution devices are used. The CCDs become too complex, costly and large to be applied within an electronic video endoscope with a resolution necessary for human procedures. The endoscopes simply become too large to provide high resolution color images and still be placed within a human body cavity.
The preferred embodiment of the present invention eliminates any on-the-chip storage area for its CCD, and thereby increases image production yield and reduces CCD circuit complexity, cost and size. By allowing for a CCD absent on-the-chip storage, the preferred embodiment of the invention is able to provide a high resolution color endoscope practically suitable in size and cost for human procedures.
As shown in the illustration of Figure 16, the preferred embodiment of the invention applies a CCD where the image section I is directly connected to a read out register R. The read out register R is horizontally laid across the CCD chip, and provides horizontal access from the pixels P vertically relieving illumination L.
In a "FULL FRAME CCD" the photo site structure I (imaging section of the CCD) is made up of contiguous CCD elements P with no voids or inactive (shielded from light) areas anywhere across the horizontal surface of the chip. In addition to sensing light L, these elements P are used to shift image data vertically to the horizontal read out register R for downloading the created image information. The lack of a on-the-chip storage area (area which is shielded from the light L) therefore requires that the surface of the CCD to be kept dark during readout. Any amount of extra or post-exposure light L present will be fed directly to the read out register R and destroy the image captured by the CCD. To avoid this imaging destruction, the prior art must use on-the-chip storage devices which are not practical in size, cost and complexity for human medical procedures.
The preferred embodiment of the present invention uses a "full frame CCD" and incorporates a system for controlling the CCD exposure along the optical path. The preferred embodiment incorporates an optical exposure control into the distal end of the endoscope body 11 (Figure 2) to control the illumination which is allowed to impinge upon the solid state imaging element 15.
In one preferred embodiment of the invention, this exposure control non-synchronously strobes the illumination source 18 within the distal end of the endoscope 11 (Figure 2) to expose and darken the solid state imaging element 15. The control system is described in detail in Figure 5.
In another preferred embodiment, the exposure control opens and closes an electro-optical shutter LCS within the distal end of the endoscope body 141 (Figure 12) . This embodiment is described in detail in accordance with Figure 11.
Furthermore, in the preferred embodiment of the invention shown in Figure 16, the color filter F is printed directly on the CCD during fabrication. A single horizontal read out register R is used to readout the entire CCD. The information about color sequencing is provided by the signal "S5" from the CCD timing generator unit CCDTGU.
As discussed below, the color sequence information signal S5 is used by a multiplexing circuit which is external to the CCD, to recognize whether the serially fed pixel information S6 being output from the readout register R is for a Red or Green or Blue pixel. This external multiplexing circuit (Figure 5) consists of the conditioning units AMP-R, AMP-G, and AMP-B, the analog to digital converters ADC-R, ADC-G, ADC-B, the buffer memories BM-R, BM-G, BM-B and the signal S5 from the CCD timing generator unit CCDTGU.
The preferred embodiment of the invention additionally reduces the size, cost and complexity of the image capture circuitry by providing the multiplexing circuit as discussed. In prior art systems, CCDs used three separate registers, one each for Red, Green and Blue. An on-the-chip multiplexing circuit routes the different color pixel information to the correct register.
The preferred embodiment overcomes this chip complexity by providing a single horizontal read out register R which accesses all of the adjacent tri-color pixels P. The pixels P are dedicated to one of the three colors Red, Green or Blue by a filter F fabricated over the image section I. The pixel information S6 is sequentially fed during the serial downloading as discussed below. The preferred embodiment of the present invention also resolves any problems of "color separation" and "blurring" seen in prior art systems. Color separation occurs in the prior art when various image color components RGB are exposed at separate exposure times.
The preferred embodiment has a single white light exposure divided of all tri-color (RGB) adjacent pixels P. All pixels feed into a single register R which is downloaded as discussed above.
Figures 13 and 14 show the blurring and loss of color fidelity caused when the three primary color images are formed in three separate exposures at three different times. If the position of the image on the focal plane FP changes from one exposure to the next, the primary color images will be registered on different locations P of the CCD. Hence, the full color image that is formed from this set of misregistered primary color images will have blurred edges, and loss of color fidelity.
This misregistration also leads to smearing of the image and causes the image to appear fuzzy. Streaking of the image is caused by linear motion of the image on the focal plane which causes the image to appear out of focus and smeared in linear bands. An analysis of the calculation of the speed at which color separation occurs is discussed in the following chart.
O: Object to center of the lens distance.
F: Focal plane to center of the lens distance. S: Length of side of the CCD pixel (assumed square pixel for simplicity) .
N: Displacement of the same pixel in two consecutive primary color exposures in terms of number pixels.
D: Displacement of the object in two consecutive primary color exposures.
T: Time interval between two consecutive primary color exposures.
N x S D
F 0
S
D = 0 x X N F
D
Speed = T
Using the typical parameters:
F = 3 mm O = 4 cm S = 12 microns T = 1/90 sec
Based upon these system parameters discussed in this chart, the speed at which each pixel P of Figure 13 can be misregistered by at least 3 pixels is about 4.5 cm/sec.
Figure 15 illustrates the formation of a full color image in accordance with the preferred embodiment and without any distortions caused by misalignment. Each color element is produced by adding the corresponding primary color values, and no color separation or blurring of the edges is allowed.
As discussed above in accordance with Figures 5 and
7, the exposure and downloading of the CCD is driven in accordance with the timing cycle dictated by the CCD timing generator unit. When the CCD timing generator unit CCDTGU receives signal SI from the exposure control unit ECU, the CCD timing generator unit CCDTGU sets the control signals S4, and in turn the CCD, for exposure. The CCD is then maintained in an image capture mode for the entire exposure time Texp, the time the signal SI is being sent to the CCD timing generator unit CCDTGU. For this entire exposure time Texp period, the control signals S4 are held in an exposure configuration.
When the exposure control unit ECU discontinues sending the signal SI, the CCD timing generator unit CCDTGU begins cycling the image capture system through an exposed image downloading mode. The signals S4 are activated to pulse at their various timing periods to control the CCD in a read out state. The information generated at each pixel of the CCD is serially fed as signal S6 to be stored in image memory buffers BM-R, BM-G, BM-B.
The color component signals from the CCD pixels are fed through an RGB data multiplexing clock system to be separately stored as red, green and blue components of the image pixels. The RGB data multiplexing clock system includes image memory buffers BM, analog to digital converters ADC and amplifiers/signal conditioners AMP, which are controlled by the CCD timing generator unit CCDTGU to operate one set at a time in a cycle upon the serially fed pixel information S6 being downloaded from the CCD.
The CCD timing generator unit CCDTGU provides the signals S4 and S5 to download the information from the CCD and activate a one of the three sets of A/D convertors (ADC-R, ADC-G, ADC-B) and memory buffers (BM-R, BM-G, BM- B) depending on whether the data from Red, Green or Blue CCD picture element is being accessed at the time. In this manner the CCD timing generator unit CCDTGU will cause the entire content of the CCD to be serially read and stored in the respective buffer memories BM-R, BM-G, BM-B for further use.
When the last pixel of the CCD is read and stored in the various memory buffers BM, the CCD timing generator unit CCDTGU sends a signal S2 to the exposure control unit ECU signaling the end of CCD image downloading cycle. The exposure control unit ECU then again issues the signal SI beginning the exposure time Texp of the CCD and again beginning the image capture cycle illustrated in Figures 5 and 7.
The image capture system again begins the exposure time Texp and exposes the CCD with the LED illumination and then downloads the recorded information and begins again. The process is repeated indefinitely unless either the power is turned off or the user interrupts the operation. The preferred embodiment also contemplates that the exposure time Texp can be altered during the course of the system operation. When a new exposure time is selected to be used by the system, whether the new exposure time Texp is selected by the user or by a contemplated automatic exposure feedback unit, this value is placed in the exposure time latch (not shown) . The contents of this latch are loaded into the exposure control unit ECU each time the signal "SI" goes to ground. Hence, the exposure control unit ECU is updated after each exposure without affecting the operation of the CCD timing generator unit CCDTGU.
The only other time the system of the preferred embodiment of the invention is discontinued from its continuous cycling operation, other than through a normal shut-off, is if the emergency shutoff unit ESU senses a problem. The emergency shut-off unit ESU disconnects power to the image capture system illustrated in Figure 5 and housed within the distal end of the video endoscope of the preferred embodiment as shown in Figures 1-4.
Since the CCD and the light source are placed inside the body of the patient when the distal end of the endoscope is placed within a body cavity during a procedure, precaution must be taken to guard against subjecting the patient to electric shock in case of damage to the endoscope. The Emergency Shut-Off System of the preferred embodiment of the present invention is illustrated in Figures 8-10. As shown in FIG.9, a continuous, long, thin strand of conductive wire 149 is coiled and placed inside the wall of the endoscopic body near the distal end of the endoscope. The ends of this strand of wire 149 are connected to the emergency shut-off unit (ESU) . In the preferred embodiment, the wire is gauge 30-31 copper wire.
Figures 8 and 10 show cross-sectional views of the endoscopic body 150 of the preferred embodiment of the invention including the emergency shut-off wire 149 wound therethrough. Small holes 151 are placed within the walls of the circumference of the endoscope body 150 at the distal end. The hole is a continuous coiled tunnel and the shut-off wire 149 is wound therethrough.
As shown in FIG. 5, the emergency shut-off unit ESU is connected to both the CCD driving unit CCD DU which provides all the necessary power and signals to drive the CCD and the exposure control unit ECU which controls the operation of the light source through switches 10R, 10G and 10B. The emergency shut-off unit ESU monitors the conductivity of the strand of wire 149 by monitoring a minute electrical current flowing along the wire 149. When current ceases to flow through the wire 149, or the conductivity otherwise changes, a breach in the wire 149 is detected by the emergency shut-off unit. The emergency shut-off unit ESU then issues a signal S3 to both the exposure control unit ECU and the CCD driving unit CCD DU.
Upon receiving this signal S3, the exposure control unit ECU turns off the switches 10R, 10G and 10B and the CCD driving unit CCD DU shuts off all power to the CCD 5. When the power is disconnected from the CCD and the illumination source, the image capture system shuts off and an electrical shock is prevented from occurring to the patient.
The image display system of the preferred embodiment is illustrated in block diagram in Figure 6. In the endoscope system of Figure 2, the image display system is placed within the video processor unit 23 external to the endoscope body housing 11. This placement minimizes the amount of electronics placed within the endoscopic body
11, and thereby decreases the cost of the endoscopic body 11. This placement in turn increases the capability for miniaturization of the endoscopic body 11 to a size practical for human procedures and make the economics of the disposeability of the endoscopic body 11 more favorable.
An important aspect of the image display system of the preferred embodiment of the invention illustrated in Figure 6, is the presence of an independent clock oscillator within the Display System Clock DSC. The image display system operates on a completely separate timing pattern than the image capture system illustrated in Figures 5 and 7. This in turn, eliminates the need for costly and complex synchronization electronics and systems which must be applied to the CCD to synchronize the CCD and its image capture function to video displays. Again, this further decreases per unit cost and enhances miniaturization and disposeability.
In the image display system of Figure 6, the display timing generator unit DTGU receives continuous clock pulses from the display system clock DSC and provides a set of display timing signals Sll in video synchronous format. The timing signals Sll correspond with the TV Sync signals TV Horizontal Drive, TV Vertical Drive and the TV Frame signals.
Unlike the image capture system, the image display system has a constant frame rate, which is necessary to conform to television standards. The timing signals Sll are then converted by amplification and signal conditioning by the synchronization generating unit SGU to provide an exact composite television synchronization control signal SYNC in common NTSC format.
Other common formats may also be used. It is necessary to provide the signal SYNC in a commonly used video format to control the display monitor and synchronize the transmission of the down-loaded image signals to the monitor for display.
The timing signals Sll are provided in TTL format by the display timing generator unit DTGU to both the Sync generator unit SGU and the memory address counter MAC to ensure that the entire image display system of the figure is synchronized with the monitor operation for image display.
Upon receiving the timing signals Sll, the memory address counter MAC sequentially loads the contents of the three frame memories FM-R, FM-G and FM-B simultaneously into the corresponding Digital To Analog Convertors, DAC- R, DAC-G and DAC-B, the output of these Digital To Analog Convertors DAC-R, DAC-G, DAD-B and the output of the Sync Generator Unit SGU together form the output signal to the TV monitor.
The memory address counter MAC sequentially provides identical memory addresses to all of the frame memories FMR, FM-G, FM-B along a common bus network to access information for each of the red, green, blue components from identical pixels in the CCD which are to be displayed on the monitor. The Memory Address Counter MAC sequentially clocks the content of the three frame memories FM-R, FM-G, FM-B simultaneously into the respective Digital to Analog D/A Converters DAC-R, DA-G, DAC-B.
The Memory Address Counter MAC also generates a signal S10 that transfers the contents of the three buffer memories BM-R, BM-G, BM-B shown in Figure 5 onto the corresponding frame memories FM-R, FM-G, FM-B of Figure 6.
In accessing the information for downloading, the image capture system is always given access priority to refresh the pixel information being stored in the Buffer Memories BM-R, BM-G, BM-B from the CCD. If the signal S10 is received during CCD downloading cycle, the information transfer to the frame memories FM-R, FM-B, FM-G are delayed until the CCD downloading cycle is complete.
The provision of separate Buffer Memories BM-R, BM-G, BM-B and Frame Memories FM-R, FM-G, FM-B is a further aspect of the system of the preferred embodiment of the invention which allows the image capture system to have a non-synchronized and periodically changing system cycle yet provide an output to an image display system which must be synchronized to a television/video monitor.
As shown in Figure 6, the frame memories FM-R, FM-G, FM-B accepts pixel information from each of the buffer memories BM-R, BM-G, BM-B as respective signals S7, S8, S9 when access signal SIO is provided from the memory address counter MAC. The memory address counter MAC also cycles through the pixel information output from the frame memories FM-R, FM-G, FM-B in accordance with a standard television time-base set by signal Sll.
The output of the frame memories FM-R, FM-G, FM-B is fed to respective digital to analog converters DAC-R, DAC- G, DAC-B for conversion to cathode ray tube control signals. The output of the three D/A Convertors DAC-R, DAC-G, DAC-B are transmitted to the three corresponding output amplifiers AMP-R, AMP-G, AMP-B. The output of the three amplifiers AMP-R, AMP-G, AMP-B and the synchronization signal SYNC are then transmitted to a TV monitor for viewing. The SYNC signal provides CRT raster timing coordination and the R,G,B signals provide color component information for each pixel.
Figures 11 and 12 illustrates a further image capture system within another preferred embodiment of the present invention. In this embodiment, instead of turning a light source off and on, an electro-optical, liquid crystal shutter (LCS) is placed between the image forming lens 107 and the CCD 108. The electro-optical shutter LCS is attached via cable 143 to the switch 131. As shown in Figure 11, switch 131 is controlled by the exposure control unit ECU. In this embodiment, the illumination source may be removed from the distal end of the scope and light may be supplied through a light guide 140. The remainder of this embodiment remains similar to the previously discussed image capture system installed in Figure 5.
In the electro-optical shutter embodiment, the exposure to the CCD is controlled by the electro-optical shutter LCS. In normal state, when no voltage is applied to the electro-optical shutter LCS, the shutter LCS is transparent, allowing light to pass therethrough. The light reflected by the object O becomes focused by the lens 107 and passes through the shutter LCS and forms the image of the object O on the focal plane.
When a specified voltage is applied to the shutter LCS, the shutter LCS becomes opaque, interrupting the beam of light from reaching the focal plane. In this embodiment, the voltage across the shutter LCS is controlled by the exposure control unit ECU allowing the use of a continuous light source.
The electro-optical shutter configuration of Figure
11 can also accept the emergency shut-off system of Figures 8-10. As shown in Figure 11, to mate the emergency shut-off system with the electro-optical shutter system, the emergency shut-off unit ESU is connected to both the CCD driving unit CCD DU which provides all the necessary power and signals to running the CCD and the exposure control unit ECU. The exposure control unit ECU in turn controls the switch 131 that controls the power driving the electro-optical shutter LCS. The emergency shut-off unit ESU monitors the conductivity of the strand of wire 149 by monitoring the minute electrical current flowing in the strand 149. When a breach in wire 149 is detected by the emergency shut-off unit ESU, it issues a signal to both the exposure control unit ECU and CCD driving unit CCD DU. Upon receiving this signal, the exposure control unit ECU turns the switch 131 off and the CCD driving unit CCD DU shuts off all the power to the CCD preventing an electrical shock to the patient.
The preferred structure of these endoscopes are depicted in Figures 2-4. Viewing the distal side of the endoscope as the end closest the object O, signal lines 19 and 20 are connected to the proximal side of the substrate 16 and provide electrical connections to the solid state (CCD) element 15, the signal conditioning circuit 17, and the LED 18.
The electrical harness and connections 19, 20 terminate at the proximal end of the endoscope body 11 at a quick lock connector 21a. The quick lock connector 21a is separable from the mating connector 21b. Thus, the entire assembly housed in endoscope body housing 11 can be disposed of and replaced with a new endoscopic body housing element containing items 12-21a when required or as necessary to maintain medical sterilization. Quick lock connector 21b receives connector 21a.
Electrical cable 22 is coupled to quick lock connector 21b to communicate with video processor unit 23.
The video processor unit 23 generates the signals to operate the images and processes the video signal which is preamplified and conditioned by, and sent from, the signal conditioning circuit 17. The video processor unit 23 is coupled via cable 24 to a video or TV monitor 25 that displays the image picked up by the solid state image sensor 15.
Figure 3 depicts another preferred embodiment of the present invention. As shown therein, and in a similar manner to the embodiment depicted in Figure 2, the endoscope body 26 houses the distal windows 27. The lens 28 is held in a movable lens housing 29 that positions the lens at the correct focal length from the solid state imager 30. The imager 30 is fastened, along with the signal conditioning circuit 31, to the substrate 32, which is anchored to the inside of the endoscope body 26.
Arranged through and alongside the substrate 32 is a bundle of fiber-optic elements 33 that traverse the shaft 26 and terminate in a fiber-optic fixture that positions the fiber in relation to a fiber-optic bule or condenser lens 34. Condenser lens 34 focuses white light emitted from bulb 35 into the fiber-optic elements 33. Condensing the light provides greater intensity of light emission at the distal end of the optical fibers 33 and provides an adequate amount of light to assure a good quality image.
Surrounding the illumination source 35 is a mirrored shield 36 that helps to reflect and focus the light onto the condensing lens 34. The reflector 36 also serves as a heat sink to help cool the illuminator 35.
Emanating from the illumination source 35 is signal line 37 that terminates into quick lock connector 39a. Signal line 38 from the solid state imager 30 also feeds into quick lock connector 39a. The quick lock connector 39a is separable from the mating connector 39b, and the entire assembly housed in the endoscope body 26 may be disposed of and replaced with a new endoscope body 26 when required.
Quick lock connector 39b receives quick lock connector 39a. Coupled to quick lock connector 39b is an electrical cable 40 that carries the wire from the video processor unit 41. The video processor unit 41 is coupled via cable 40 to a video or TV monitor 41 that displays the image picked up by the solid state image sensor 30.
Figure 4 depicts a further preferred embodiment for constructing the video endoscope of the present invention. For ease of manufacture and cost savings, in this embodiment, all of the imaging, illumination, and electrical connections are fabricated in the most distal section 45 of the endoscope body 44.
The distal section 45 is separable from the shaft 44 through female quick connect 52 to the wire harness 54 and 55 that emanate from the preamplifier (signal conditioning circuit) 57 and its circuit board 56, both of which are connected to the main electrical connection 59a by wire harness 58.
At the distal end of the endoscope body, windows 46 allow light to be transmitted and recovered from the object O. The lens 48 within the tip of the distal assembly 45 is affixed to lens holder 47 in order to focus an image on the solid state image sensor 49. The image sensor 49 and LED 50 are mounted to a substrate 51 that is electrically connected to female connector 52 via a male electrical connector 54.
With this means of construction, distal section 45 and its associated components can be quickly and easily connected and disconnected from the endoscope shaft 44 and its components, allowing each segment to be fabricated independently.
Endoscope body 44 and its contents, along with distal section 45 and its contents, are connected to the video processor 61 via cable 60. Cable 60 is connected to quick lock connector 59b, which attaches to quick lock mating connector 59a at the proximal end of the endoscope shaft 44. Video processor 61 is connected to video monitor 63 via cable 62, which displays the video images picked up by the solid state sensor 49.
Those skilled in the art will appreciate that various adaptations and modifications of the just-described preferred embodiment can be configured without departing from the scope and spirit of the invention. Therefore, it is to be understood that, within the scope of the appended claims, the invention may be practiced other than as specifically described herein.