US20220361737A1 - Fluorescence Imaging Scope With Dual Mode Focusing Structures - Google Patents

Abstract

Improved fluoresced imaging (FI) and other sensor data imaging processes, devices, and systems are provided to enhance use of endoscopes with FI and visible light capabilities. A first optical device is provided for endoscopy imaging in a white light and a fluoresced light mode with an image sensor assembly including one or more image sensors. A mechanism in the first optical device to automatically adjust the focus of the first optical device wherein the automatic focus adjustment compensates for a chromatic focal difference between the white light image and the fluoresced light image caused by the dispersive or diffractive properties of the optical materials or optical design employed in the construction of the first or second optical devices, or both. Adjustment mechanisms are provided using liquid lenses or repositioning sensors. The design may be integrated with a scope or detachable.

Description

CROSS REFERENCE TO RELATED PATENT APPLICATIONS
• This application is a continuation of U.S. patent application Ser. No. 15/636,345, filed Jun. 28, 2017, and entitled “Fluorescence Imaging Scope with Dual Mode Focusing Structures,” which is incorporated herein by reference.
TECHNICAL FIELD OF THE INVENTION
• The invention relates generally to the field of image capture and more specifically to a medical imaging camera head and camera designs for attaching or integrating with endoscopes capable of fluorescent imaging and visible color imaging.
BACKGROUND OF THE INVENTION
• Endoscopes and other medical scopes often use fluorescing agents or autofluorescence to better examine tissue. A fluorescing agent such as a dye may be injected or otherwise administered to tissue, and then an excitation light is directed toward the tissue. Responsive to the excitation light, the fluorescing agent fluoresces (emits light typically at a longer wavelength than the excitation light), allowing a sensor to detect the light, which is often not in a wavelength visible to the human eye. The detected light is formatted to images, and examining the images can indicate the concentration of fluorescing agent in the observed tissue. Further, a phenomenon known as autofluorescence may occur in which tissue fluoresces light under certain conditions without a fluorescing agent. Such light can be detected as well. Images based on detected fluoresced light, known as “fluorescence imaging” (FI), are therefore useful in medical diagnosis, testing, and many scientific fields.
• A typical prior art endoscope2, as illustrated inFIG. 1, usually includes a first imaging lens (e.g., an objective) followed by a series of carrier lenses (e.g., relays) which capture and transmit an optical image from inside an enclosedarea1 to the outside. The proximal end of the endoscope2 may be attached, via direct coupling or an adaptor, to a camera head3 or an eye-piece for viewing. The camera head3 usually includes lenses for receiving the optical image and forming a real optical image onto the image sensor. The digital image captured by the image sensor can then be transmitted to a camera control module (CCM) or other similar modules for analysis and display.
• Frequently, endoscopes used for FI applications, and particularly indocyanine green (ICG) applications are mainly designed and deployed for visible light imagery. To perform FI imaging, such scopes often employ an appropriate optical filter to block the stimulus light and transmit the fluoresced light. Since these endoscopes are designed for use with visible light (approximately 450-650 nm wavelengths), the infrared fluorescence (800-900 nm) is focused in a different plane than the visible light. Therefore, in addition to adding a filter, the user must refocus when switching between visible light mode and fluorescence mode unless the camera compensates. The focal differences exist because the endoscopes are not chromatically corrected for the infrared where certain fluorescence bands (particularly ICG) are located. Such differences, even in the face of common correction techniques, often result in a signal to noise ratio of FI imaging being low, resulting in poor quality FI images. Given the dispersion characteristics of optical elements used in the endoscope's optical channel, such as rod lenses, correcting these issues may be difficult or expensive. For example, when autofocus algorithms are employed, the algorithm is frequently slow to apply the correction.
• What is needed are devices and methods to enable an endoscopic camera to compensate for the endoscope characteristics when the user switches between white light mode and fluorescence mode. What is further needed are devices allowing the use of varied existing endoscopes for fluorescence imaging applications without expensive and slow optical elements such as autofocus mechanisms or lenses and processing systems for chromatic aberration correction.
SUMMARY OF THE INVENTION
• It is an object of the invention to provide improved endoscopic camera devices and methods to enable the camera to compensate for the endoscope characteristics when the user switches between white light mode and fluorescence mode. Another object of the invention is to enable such switching in an automated manner without the user inserting filters or manually refocusing a device to achieve a switch between FI to visible light modes. Another object of the invention is to enable the use of existing endoscopes in simultaneous FI and visible light modes. Is it a further object of the invention to provide devices and methods allowing the use of varied existing endoscopes for fluorescence imaging applications without expensive optical elements such as autofocus mechanisms or lenses and processing systems for chromatic aberration correction.
• In order to achieve these objects, various aspects of the invention provide improved fluoresced imaging (FI) and other sensor data imaging processes, devices, and systems are provided to enhance use of endoscopes with FI and visible light capabilities. A first optical device is provided for endoscopy imaging in a white light and a fluoresced light mode with an image sensor assembly including one or more image sensors. A focus adjustment mechanism in the first optical device to automatically adjust the focus of the first optical device wherein the automatic focus adjustment compensates for a chromatic focal difference between the white light image and the fluoresced light image caused by the dispersive or diffractive properties of the optical materials or optical design employed in an assembly construction of the first or second optical devices, or both. Adjustment mechanisms are provided using liquid lenses or repositioning sensors. The design may be integrated with a scope or detachable.
• According to a first aspect of the invention, a first optical device is provided for use in endoscope procedures and operable to image in a first mode with white light and a second mode with fluoresced light outside the visible band. The first optical device includes an image sensor assembly including one or more image sensors operable for capturing white light images and fluoresced light images. A focus adjustment mechanism is positioned in the first optical device to automatically adjust the focus of the first optical device wherein the automatic focus adjustment compensates for a chromatic focal difference between the white light image and the fluoresced light image caused by the dispersive or diffractive properties of the optical materials or optical design employed in an assembly construction of the first optical device or an optional second optical device connected to the first optical device, or both.
• According to a second aspect of the invention, a first optical device is provided for use in endoscope procedures and operable to image in a first mode with white light and a second mode with fluoresced light outside the visible band. The first optical device includes an image sensor assembly including one or more image sensors operable for capturing white light images and fluoresced light images. An optical assembly is positioned in the first optical device, or in the second optical device, the optical assembly including one or more optical lens elements, all either fixed in place or adjustable by manual movements, and is configured when attached to direct light received from a subject scene toward the image sensor assembly. A liquid-based deformable lens is provided in the first optical device optically arranged to be between the optical assembly and the image sensor and operable to automatically adjust the focus of the first optical device wherein the automatic focus adjustment compensates for a chromatic focal difference between the white light image and the fluoresced light image caused by the dispersive or diffractive properties of the optical materials or optical design employed in an assembly construction of the first optical device or the second optical device, or both.
• According to a third aspect of the invention, a first optical device is provided for use in endoscope procedures and operable to image in a first mode with white light and a second mode with fluoresced light outside the visible band. The first optical device includes an image sensor assembly including one or more image sensors operable for capturing white light images and fluoresced light images. An optical assembly is positioned in the first optical device, or in a second optical device that attaches to the first optical device, the optical assembly including one or more optical lens elements, all either fixed in place or adjustable by manual movements, and is configured when attached to direct light received from a subject scene toward the image sensor assembly. The first optical device includes a mechanical device which shifts the position of at least one of the one or more image sensors when the mode is switched from white light mode to fluoresced light mode and operable to automatically adjust the focus of the first optical device wherein the automatic focus adjustment compensates for a chromatic focal difference between the white light image and the fluoresced light image caused by the dispersive or diffractive properties of the optical materials or optical design employed in an assembly construction of the first optical device or the second optical device, or both.
• In some implementations of the various aspects of the invention, the first optical device is a videoendoscope. In some other implementations, the first optical device is camera head that attaches to and detaches from the second optical device, and the second optical device is an endoscope. The camera head may configured to automatically recognize the second optical device and, in response, automatically apply a known, prescribed adjustment based on known characteristics of the second optical device.
• In some implementations of the various aspects of the invention, the adjustment made is a known, prescribed adjustment based on the known characteristics of the first optical device and/or the second optical device. The focus adjustment mechanism may include a liquid-based deformable lens. The device may be adapted to toggle the liquid-based deformable lens between two pre-determined states when the mode is switched from white light mode to fluoresced light mode. The sensor position may toggle between two pre-determined positions when the mode is switched from white light mode to fluoresced light mode. In some implementations of the various aspects of the invention, the focus adjustment includes a mechanical device which shifts the position of at least one of the one or more image sensors when the mode is switched from white light mode to fluoresced light mode. The mechanical device may be a motor, such as a piezoelectric motor.
• In some implementations of the various aspects of the invention, the device is adapted to trigger the automatic focus adjustment when the device is toggled between white light mode and fluorescence mode. The first device may include an electronic controller having tangible, non-transitory computer memory holding program code executable to control the various adjustments, and to trigger the automatic focus adjustment when the device is toggled.
• Implementations of the invention may also be embodied according to a fourth aspect of the invention as software or firmware, stored in a suitable medium, and executable to perform various versions of the methods herein. The program code is executable to control the various adjustments, and to trigger the automatic focus adjustment when the device is toggled. According to a fifth aspect, the functionality described herein for the implementations of the first aspect may be embodied as a method or process for operating an imaging scope to perform the automatic focus adjustment. The method may include automatically recognizing the connection of an endoscope device to a camera head embodying the first optical device. The method may retrieve focus settings for the different modes from memory, and may store focus settings for multiple devices. The method applies the retrieved focus settings and allows procedures to be conducted in the white light imaging mode. In response to a user changing to a fluorescence imaging mode, the method applies retrieved focus settings for the fluorescence imaging mode according to any of the different device aspects of the invention. Then the method allows procedures in the fluorescence imaging mode, and allows the modes to be toggled as needed with the appropriate focus settings applied for each toggle.
• These and other features of the invention will be apparent from the following description of the preferred embodiments, considered along with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
• The present invention will become more fully understood from the detailed description given herein and the accompanying drawings which are given by way of illustration only, and thus are not imitative of the present invention, and wherein:
• FIG. 1 is diagram of a prior art endoscopic system;
• FIG. 2 shows cross section diagram of a video endoscope device according to an example embodiment of the invention;
• FIG. 3 is a cross section diagram of an optical channel having a deformable lens according to another embodiment for use in a video endoscopic system;
• FIG. 4 is a cross section diagram of a partial optical channel having a light splitter and a focal adjustment after the optical channel is split according to another embodiment for use in a video endoscopic system;
• FIG. 5 is a hardware block diagram of system including an example image capture device according to an example embodiment of the invention; and
• FIG. 6 is a flowchart of a process for applying device adjustments according to an example embodiment of the invention.
DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS
• As used herein, first elements (e.g., sensors and lenses) that are “optically arranged” in relation to other elements, refers to the first elements' position along a common optical path that includes first and other elements. For example, a lens group optically arranged between an image sensor and an objective, means that the lens group occupies a portion of the optical path that light travels (e.g., from the objective to the image sensor) for capturing images or video.
• Because digital cameras and FI sensors and related circuitry for signal capture and processing are well-known, the present description will be directed in particular to elements forming part of, or cooperating more directly with, a method and apparatus in accordance with the invention. Elements not specifically shown or described herein are selected from those known in the art. Certain aspects of the embodiments to be described are provided in software. Given the system as shown and described according to the invention in the following materials, software not specifically shown, described or suggested herein that is useful for implementation of the invention is conventional and within the ordinary skill in such arts.
• FIG. 2 shows cross section diagram of a video endoscope device100 according to an example embodiment of the invention. This device100 includes a first optical device108 (in this version “camera head108”) connected in a detachable manner to endoscope107 (“second optical device”) which is connected to in a detachable manner as known in the art.Endoscope107 includes ashaft106. Other versions may, of course, include a scope integrated with the camera head or in a single first optical device. Various structural components supporting the depicted elements are omitted in the diagram, as well as other components such as illumination lights sources, fluorescent excitation light sources, and controls, which are known in the art and are not shown in order to avoid obscuring the relevant details of the example embodiments of the invention.
• Toward the left side of the drawing, at the distal tip of theendoscope shaft106 iscover glass202, which in this version faces directly along the longitudinal axis of theshaft106, but may also be positioned at an angle relative to the longitudinal axis as is known in the art. Behind thecover glass202 is shown a preferred position for thewide angle lens204, set against or verynear cover glass202 and preferably assembled together with the cover glass in construction. While a wide angle lens is shown, this is not limiting, and any suitable lens may be used in various embodiments. Further, the particular number and arrangement of lenses in theendoscope shaft106 will vary widely depending on the application. Optically arranged or attached at the proximal side oflens204 is a second lens orlens group208 to focus the incoming light to an appropriate size for the imaging process receiving the incoming light downstream in the proximal direction. The directed light then passes alongendoscope shaft106 and may be guided by other optical elements such as rod lenses. Next the light is received at adoublet lens210, and directed toward firstoptical device108, where it passes through theproximal window212 of theendoscope107 and thedistal window213 of firstoptical device108. One or more additional lens groups or rod lenses may be included optically positioned betweendoublet lens210 and focusinglens group214. In versions with a unified device, such windows may not be used. Next in the optical path is a focusinglens group214 which in this version includes a plano-convex lens and a doublet lens including a biconvex lens and a biconcave lens. Many other suitable lenses and combinations of lenses may be used. Focusinglens group214 focuses the image light toward theimage sensor assembly215 which may include acover glass221 andimage sensor222.
• As shown,image sensor222 is repositionable along the focal path by anadjustment mechanism224, which may be a piezo-electric motor or other suitable adjustment mechanism.Adjustment mechanism224 is shown mechanically coupled tosensor222 with arod225, which pushes or pullsimage sensor222 to achieve the adjustment movement.Image sensor222 may be held in a frame having guide rails or other suitable structure to stabilize the orientation during movement. In this version,image sensor222 is a single sensor capable of detecting both visible light images and fluoresced light images, for example visible light imagery at approximately the 450-650 nm wavelengths, and the infrared fluorescence imagery at 800-900 nm. Additionally or alternatively, the sensor may detect other fluorescent wavelengths commonly used in endoscopic imagery in addition to the visible light wavelengths. Because the fluorescent imagery is focused in a different plane than the visible light, the device has the capability of adjusting the optical path focus when switching between visible light mode, as shown in theimage sensor222 first white light image position depicted as a solid box, and the second fluorescence mode image position depicted by the dotted box. The chromatic focal difference in the optical path is depicted by the two ray trace lines shown on the diagram, with the visiblelight ray trace226 shown focused on thesensor222 in the first depicted position in visible light mode, and the FIlight ray trace227 shown focused onsensor222 in the FI mode position. The slight chromatic focal difference can be seen as the rays diverge slightly over the course of the optical path. The distance thatimage sensor222 is moved, shown as D, is for adjusting the focal length of the optical path between white light imaging mode and fluorescent imaging mode. Typically, this distance may be around 2 mm of movement, but this will vary depending on the total channel length and the chromatic characteristics of the optical elements in the optical channel. Theadjustment mechanism224 is positioned in the firstoptical device108 to automatically adjust the focus of the first optical device where the automatic focus adjustment compensates for a chromatic focal difference between the white light image and the fluoresced light image caused by the dispersive or diffractive properties of the optical materials or optical design employed in the assembly construction of the first or second optical devices, or both. This may be accomplished even for endoscope designs in which the optical assembly includes optical lenses all either fixed in place or adjustable by manual movements. Preferably the act of a user changing modes to toggle between white light mode and fluorescence mode triggers an autofocus algorithm controlling the movement ofadjustment mechanism224. Preset positions may be employed for both modes to achieve the desired position, as further described below, avoiding the need for autofocus processing to achieve the known desired focal length for each of the modes.
• FIG. 3 is a cross section diagram of anoptical channel300 for use in a video endoscopic system according to another embodiment, which employs a deformable lens to adjust the focus. The depictedoptical channel300 may be embodied in first and secondoptical devices107 and108, such as the camera head and endoscope discussed above, or a single integrated optical device. The assembly construction of the second optical device, or the endoscope and shaft portion of an integrated device, may vary. As shown, the optical channel includes an objective passing the observed image light to one ormore lens groups208. The light is directed along the channel through a suitable lens arrangement such asdoublet lens210, and in detachable dual-device embodiments passes throughwindows212 and213 to the firstoptical device108. Next in the optical path is thedeformable lens302, which is deformable to adjust the optical channel focal length for the same purpose of changing modes between white light imaging fluorescence imaging as discussed with regard to the other embodiments. The deformable lens can be made of, for example, birefringent liquid crystal, a transparent elastic membrane filled with fluid, or a two fluid interface. Preferably such a liquid-based deformable lens is employed, but other suitable deformable lens technologies may be used. Depicted are two different deformable surface configurations for the deformable lens. In the first white light imaging mode,deformable lens302 is configured with the deformable surface atposition306, which generally has more negative lens power than the second fluoresced light configuration position of the deformable lens surface shown atdotted line308. The deformable lenses can be tuned at least in part by a suitable adjustment mechanism (not shown) such as an electrostatic actuator, an electromagnetic actuator, a piezo-motor, a magneto-strictive actuator, a stepper motor, or an electroactive polymer actuator for a high focus tuning range. As depicted, the whitelight configuration position306 is a negative power lens and the fluorescedlight confirmation position308 is a positive power lens, however this is not limiting, and they may both be negative or both may be positive depending on the optical channel design. Typically, because the chromatic focal difference between the white light image and the fluoresced light image causes a shorter focal distance for the white light, the white light mode will have a comparatively negative lens power than the fluoresced light mode. Thedeformable lens302 with its adjustment mechanism is arranged in the firstoptical device108 to automatically adjust the focus of the first optical device where the automatic focus adjustment compensates for a chromatic focal difference between the white light image and the fluoresced light image caused by the dispersive or diffractive properties of the optical materials or optical design employed in the assembly construction of the first or second optical devices, or both. Preferably the act of a user changing modes to toggle between white light mode and fluorescence mode triggers an autofocus algorithm controlling the movement ofadjustment mechanism224. This design also may be accomplished even for endoscope designs in which the optical assembly includes optical lenses all either fixed in place or adjustable by manual movements. Preset positions may be employed for both modes to achieve the desired position, as further described below, avoiding the need for autofocus processing to achieve the known desired focal length for each of the modes.
• Note that while the position of thedeformable lens302 is shown before the focusinglens group214 in this version, this is not limiting and the deformable lens may be placed in any suitable location in the optical channel where the channel construction can accommodate the varied focal lengths resulting from changing the deformable lens configuration. For example,deformable lens302 may be part of the focusinglens group214.
• After the deformable lens in the optical channel, the image light passes to the focusinglens assembly214, which is similar to that described with respect toFIG. 2, and then through the imagesensor cover glass221 and finally to theimage sensor222. Note that in this version the image sensor is fixed in position relative to the optical channel.
• FIG. 4 is a cross section diagram of a partial optical channel according to another embodiment. The diagram shows only the portion of the optical channel in firstoptical device108, which is preferably a camera head device. In devices where the camera is combined with the endoscope, the diagram represents the proximal portion of the optical channel including the focusing lens group and the image sensor.
• The depicted system uses alight splitter402 to direct the visible image light to imagesensor222 and the fluoresced light tofluorescence image sensor223. This is illustrated with the depicted light bundle traces230 in which the fluorescence bundle traces231 light enters and is passed through the dichroic prism interface oflight splitter402 and hitsfluorescence image sensor223, while the visible light bundle traces232 are reflected downward and hitimage sensor222.Light splitter402 is preferably a dichroic prism or other suitable dichroic optical element, having a high-pass reflective surface at a 45 degree angle allowing higher frequencies to pass through tofluorescence image sensor223 and lower frequencies to reflect tosensor222. The cutoff frequency may be positioned at or near the top wavelength of visible light, that is near 650 nm, or higher as long as it is below the lowest frequency (higher than longest wavelength) required for detection of the fluoresced light. While this version reflects the visiblelight wavelengths232 and passes the fluorescedlight wavelengths231, this is not limiting and other versions may pass the visible light and reflect the fluoresced light (with a low-pass dichroic prism, for example), or may reflect both in differing directions.
• Image sensor222 is capable of detecting the visible light wavelengths commonly used for endoscopic examination, for example visible light imagery at approximately the 450-650 nm wavelengths. In this version,image sensor223 is a single sensor capable of detecting fluoresced light images, infrared fluorescence imagery at 800-900 nm. Additionally, or alternatively, the sensor may detect other fluorescent wavelengths commonly used in endoscopic imagery.
• As shown,fluorescence imaging sensor223 is repositionable along the focal path by anadjustment mechanism224 in order to adjust the focal plane for the fluoresced light based on characteristics of the attached endoscope, such as the aberration characteristics or the use of a differing fluoresced wavelengths in differing scopes for use with various fluoresced imaging techniques. This design also may be accomplished even for endoscope designs in which the optical assembly includes optical lenses all either fixed in place or adjustable by manual movements. The adjustment mechanism may be a piezo-electric motor or other suitable adjustment mechanism as discussed above with regard to the embodiment ofFIG. 2. Similarly to the embodiment ofFIG. 2,adjustment mechanism224 is shown mechanically coupled toimage sensor223 with arod225, which pushes or pullsimage sensor223 to achieve the adjustment movement.Image sensor223 may be held in a frame having guide rails or other suitable structure to stabilize the orientation during movement. The distance adjusted, shown as distance D, in this version will typically be less than that required to adjust for differing focal lengths between visible and fluoresced light, because this embodiment adjusts only for different fluoresced light characteristics of the optical channel such as dispersion and chromatic focal differences.
• Referring toFIG. 5, a block diagram of system including an image capture device according to an example embodiment of the invention is shown. The invention is applicable to more than one type of device enabled for image capture, such as endoscopes incorporating solid state imagers, digital microscopes, digital cameras, mobile phones equipped with imaging sub-systems, and automotive vehicles equipped with imaging sub-systems, for example. The preferred version is an imaging scope system, such as an endoscope.
• A light source8 illuminates subject scene9 with visible light and/or fluorescent excitation light, which may be outside the visible spectrum in the ultra-violet range or the infra-red/near infrared range, or both. Light source8 may include a single light emitting element configured to provide light throughout the desired spectrum, or a visible light emitting element and a one or more fluorescent excitation light emitting elements. Further, light source8 may include fiber optics passing through the body of the scope, or other light emitting arrangements such as LEDs or laser diodes positioned at or near the front of the scope.
• As shown in the drawing, light10 reflected from (or, alternatively, as in the case of fluorescence, excitation light8 absorbed and subsequently emitted by) the subject scene is input to anoptical assembly11, where the light is focused to form an image at a solid-state image sensor(s)222 and/or fluoresced light sensor(s)223.
• Optical assembly11 includes at least one lens, which may be a wide-angle lens element such thatoptical assembly11 focuses light which represents a wide field of view. In embodiments that employ a deformable lens as described above, thedeformable lens302 is part of the optical assembly. As discussed above, portions of the optical assembly may be embodied in a camera head or other first optical device, while other portions are in an endoscope or other scope device, or theoptical assembly11 may contained in a single imaging device. Image sensor222 (which may include separate R, G, and B sensor arrays) and fluorescedlight sensor223 convert the incident visible and invisible light to an electrical signal by integrating charge for each picture element (pixel). It is noted that fluorescedlight sensor223 is shown as an optional dotted box because embodiments may use theRGB image sensor222 to detect only white light images or to also detect fluoresced light (e.g., NIR, ICG, Fl). The latter scheme may be used when the fluoresced light is in a spectrum detectable byimage sensor222 that is in or near the visible light spectrum typically detected by a RGB sensor arrays.
• Theimage sensor222 and fluorescedlight sensor223 may be active pixel complementary metal oxide semiconductor sensor (CMOS APS) or a charge-coupled device (CCD).
• The total amount of light10 reaching theimage sensor222 and/or fluorescedlight sensor223 is regulated by the light source8 intensity, theoptical assembly11 aperture, and the time for which theimage sensor222 and fluorescedlight sensor223 integrates charge. An exposure controller40 responds to the amount of light available in the scene given the intensity and spatial distribution of digitized signals corresponding to the intensity and spatial distribution of the light focused onimage sensor222 and fluorescedlight sensor223.
• Exposure controller40 also controls the emission of fluorescent excitation light from light source8 and may control the visible and fluorescent light emitting elements to be on at the same time, or to alternate to allow fluoresced light frames to be captured in the absence of visible light if such is required by the fluorescent imaging scheme employed. Exposure controller40 may also control theoptical assembly11 aperture, and indirectly, the time for which theimage sensor222 and fluorescedlight sensor223 integrate charge. The control connection from exposure controller40 totiming generator26 is shown as a dotted line because the control is typically indirect.
• Typically, exposure controller40 has a different timing and exposure scheme for each ofsensors222 and223. Due to the different types of sensed data, the exposure controller40 may control the integration time of thesensors222 and223 by integratingsensor222 up to the maximum allowed within a fixed60Hz or50Hz frame rate (standard frame rates for USA versus European video, respectively), while the fluorescedlight sensor223 may be controlled to vary its integration time from a small fraction ofsensor222 frame time to many multiples ofsensor222 frame time. The frame rate ofsensor222 will typically govern the synchronization process such that images frames based onsensor223 are repeated or interpolated to synchronize in time with the 50 or 60 fps rate ofsensor222.
• Analog signals from theimage sensor222 and fluorescedlight sensor223 are processed byanalog signal processor22 and applied to analog-to-digital (A/D) converter24 for digitizing the analog sensor signals. The digitized signals each representing streams of images or image representations based on the data, are fed to imageprocessor30 asimage signal27, and firstfluorescent light signal29. For versions in which theimage sensor222 also functions to detect the fluoresced light, fluoresced light data is included in theimage signal27, typically in one or more of the three color channels.
• An adjustment control circuit20 may be provided for supplying the driving signals to operate the adjustment mechanism according to the various embodiments herein. For versions in which image sensor positions are adjusted, the adjustment control circuit sends appropriate driving signals to the mechanical adjustment mechanism, such as the piezo-electric motor, and may also receive position signals back from the adjustment mechanism. For versions in which adeformable lens302 is employed, adjustment control circuitry20 sends appropriate drive signals to the deformable lens adjustment mechanism, such as an actuator or piezo-electric motor, and may also receive position signals from the adjustment mechanism.Image processing circuitry30 includes circuitry performing digital image processing functions to process and filter the received images as is known in the art. Image processing circuitry may include separate, parallel pipelines for processing the visible light image data and the FI image data separately. Such circuitry is known in the art and will not be further described here. In some embodiments,image processing circuitry30 may also perform known autofocus algorithms to allow feedback control of adjustment control circuitry20 to compensate for chromatic focal difference between the white light image and the fluoresced light image. However, in preferred embodiments, such adjustments are predetermined and stored insystem memory56 to allow quick and reliable focus adjustment. In some versions, the predetermined settings may be stored in memory in the first optical device itself rather than a CCM or other attached controller.
• Image processing circuitry30 may provide algorithms, known in the art, for combining visible light imagery with FI imagery in a combined image display, and further highlighting or emphasizing the FI imagery for easily distinguishing the presence of fluorescing features in the image.
• Timinggenerator26 produces various clocking signals to select rows and pixels and synchronizes the operation ofimage sensor222 andfluorescent sensor223,analog signal processor22, and A/D converter24.Image sensor assembly28 includes theimage sensor222 andfluorescent sensor223, adjustment control20, theanalog signal processor22, the A/D converter24, and thetiming generator26. The functional elements of theimage sensor assembly28 can be fabricated as a single integrated circuit as is commonly done with CMOS image sensors or they can be separately-fabricated integrated circuits.
• Thesystem controller50 controls the overall operation of the image capture device based on a software program stored in program memory54. This memory can also be used to store user setting selections and other data to be preserved when the camera is turned off.
• System controller50 controls the sequence of data capture by directing exposure controller40 to set the light source8 intensity, theoptical assembly11 aperture, and controlling various filters inoptical assembly11 and timing that may be necessary to obtain image streams based on the visible light and fluoresced light. In some versions,optical assembly11 includes an optical filter configured to attenuate excitation light and transmit the fluoresced light. Adata bus52 includes a pathway for address, data, and control signals.
• Processed image data are continuously sent tovideo encoder80 to produce a video signal. This signal is processed bydisplay controller82 and presented onimage display88. This display is typically a liquid crystal display backlit with light-emitting diodes (LED LCD), although other types of displays are used as well. The processed image data can also be stored insystem memory56 or other internal or external memory device.
• The user interface60, including all or any combination ofimage display88,user inputs64, andstatus display62, is controlled by a combination of software programs executed onsystem controller50. User inputs typically include some combination of typing keyboards, computer pointing devices, buttons, rocker switches, joysticks, rotary dials, or touch screens. Thesystem controller50 manages the graphical user interface (GUI) presented on one or more of the displays (e.g., on image display88).System controller50 may receive inputs from buttons or other external user interface controls on the scope itself (or software controls through the GUI) to receive inputs to control the process for automatically adjusting the focus according to the present invention. In particular, thesystem controller50 will typically have a mode toggle user input (typically through a button on the endoscope or camera head itself, but possibly through a GUI interface), and in response transmit commands to adjustimage processing circuitry30 based on predetermined setting stored in system memory. Such settings may include different settings for different models of scopes that may be attached to a camera head or other imaging device containingimage sensor assembly28.
• Image processing circuitry30 is one of three programmable logic devices, processors, or controllers in this embodiment, in addition to asystem controller50 and the exposure controller40.Image processing circuitry30,controller50, exposure controller40, system andprogram memories56 and54,video encoder80 anddisplay controller82 may be housed within camera control module (CCM)70.
• CCM70 may be responsible for powering and controlling light source8,image sensor assembly28, and/oroptical assembly11. In some versions, a separate front end camera module may perform some of the image processing functions ofimage processing circuitry30.
• Although this distribution of imaging device functional control among multiple programmable logic devices, processors, and controllers is typical, these programmable logic devices, processors, or controllers can be combinable in various ways without affecting the functional operation of the imaging device and the application of the invention. These programmable logic devices, processors, or controllers can comprise one or more programmable logic devices, digital signal processor devices, microcontrollers, or other digital logic circuits. Although a combination of such programmable logic devices, processors, or controllers has been described, it should be apparent that one programmable logic device, digital signal processor, microcontroller, or other digital logic circuit can be designated to perform all of the needed functions. All of these variations can perform the same function and fall within the scope of this invention.
• FIG. 6 is a flowchart of a process for controlling the automatic focus adjustment according to one embodiment of the present invention. The process may include attaching a new endoscope or other scope device to a first optical device such as a camera head atblock602, or powers on the first optical device which may have a new scope attached. Some versions may not include the steps ofblocks602 and604. Atblock604, the first optical device, alone or in cooperation with an attached CCM, recognizes the model of new scope has been attached, or the user selects through the CCM GUI the model of scope attached. The recognition may happen with any suitable means such as an RFID tag being recognized, or a serial number or other identifier being read from the second optical device, the scope, by the first optical device. Next atblock606, the process retrieves focus settings for the different available visible light and FI modes from memory as shown byarrows605 and607. The memory may hold focus settings for multiple scope models, including known, prescribed adjustments to provide the desired focal settings for each scope. The focus settings typically include control information for the automatic adjustment mechanism, such as actuator or piezo-electric motor positions for controlling the deformable lens or the sensor positioning mechanism as described above. For integrated scopes, such memory may be connected to a controller aboard the first optical device. For the preferred version in which a detachable camera head is used, the memory will typically besystem memory56 of the system CCM as shown, however onboard memory may be used in this case as well. The step of retrieving the settings may be performed by the CCM or the first optical device. Next atblock608, the first optical device applies the focus settings for the mode in which the first optical device is currently set, in this example the visible light mode is shown applied first. Then the first optical device may be used to conduct examination procedures as shown atblock610. Next atblock612, the user changes the mode to an alternate mode, in this case selecting the FI mode. This is typically done through a button or switch, or a foot pedal, on the first optical device but may be done through the CCM GUI. Then atblock614 the process applies the settings for FI mode, adjusting the sensor position or deformable lens as described above. This may involve another retrieval from memory or all mode settings may be held in working memory on the device or CCM. Then the device may be used to conduct examination procedures atblock616. The process may also include applying settings to change between different FI modes with different fluoresced light wavelengths. The FI mode may include simultaneous dual imaging with visible light images as discussed above. As shown atblock618, the modes may be toggled as needed with each change of mode applying the appropriate focus settings. As discussed above, the process may be conducted by program code running onsystem controller50 on a system CCM or may be conducted under control of program code or digital logic such as FPGA circuitry embodied in adjustment control circuitry20 on the first optical device.
• As used herein the terms “comprising,” “including,” “carrying,” “having” “containing,” “involving,” and the like are to be understood to be open-ended, that is, to mean including but not limited to. Any use of ordinal terms such as “first,” “second,” “third,” etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another, or the temporal order in which acts of a method are performed. Rather, unless specifically stated otherwise, such ordinal terms are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term).
• The foregoing has outlined rather broadly the features and technical advantages of the invention in order that the detailed description of the invention that follows may be better understood. It should be appreciated by those skilled in the art that the conception and specific embodiments disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the scope of the invention as set forth in the appended claims.
• Although the invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the scope of the invention as defined by the appended claims. The combinations of features described herein should not be interpreted to be limiting, and the features herein may be used in any working combination or sub-combination according to the invention. This description should therefore be interpreted as providing written support, under U.S. patent law and any relevant foreign patent laws, for any working combination or some sub-combination of the features herein.
• Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the invention, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the invention. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.

Claims (20)

1. An endoscopic camera operable to image in white light and fluoresced light, where a fluoresced light spectrum is outside of a wavelength spectrum of the imaged white light, wherein the endoscopic camera can be removably attached to an endoscope comprising a plurality of relay lenses including rod lenses, comprising:

a focusing lens group to receive and condition a relayed light from the endoscope;

a light splitter to split the conditioned, relayed light into a first beam and a second beam; and

an image sensor assembly including a first image sensor operable to capture white light images, a second image sensor operable to capture fluoresced light images, and a focus adjustment mechanism configured to automatically adjust the position of the second image sensor a distance D along a focal path orthogonal to an image sensitive surface thereof,

wherein the automatic focus adjustment compensates for a chromatic focal difference between the white light image and the fluoresced light image caused by dispersive or diffractive properties of optical materials or optical design employed in the attached endoscope.

2. The endoscopic camera ofclaim 1, wherein there are no further focusing optics between the light splitter and the second image sensor.

3. The endoscopic camera ofclaim 1, wherein the light splitter is a dichroic prism, directing the first beam to the first image sensor and the second beam to the second image sensor.

4. The endoscopic camera ofclaim 3, wherein the first beam includes light spectra below a cutoff wavelength of about 650 nm.

5. The endoscopic camera ofclaim 1, wherein the endoscopic camera is configured to automatically recognize the attached endoscope, and, in response, automatically move the second image sensor a known, prescribed distance D, such that images captured by the second image sensor are in focus.

6. The endoscopic camera head ofclaim 5, further comprising a memory, wherein the memory contains focal settings for multiple endoscopes, each endoscope having different dispersive or diffractive properties of optical materials or optical design employed therein.

7. The endoscopic camera ofclaim 1, wherein the focus adjustment mechanism comprises a motor.

8. The endoscopic camera ofclaim 7, wherein the motor is a piezoelectric motor.

9. The endoscopic camera ofclaim 1, wherein the position of the second image sensor toggles between two pre-determined positions when the endoscopic camera is switched from a white light mode to a fluoresced light mode.

10. The endoscopic camera ofclaim 1, wherein the distance D is smaller than would be required to adjust for differing focal lengths between visible and infrared light.

11. An endoscopic system, comprising

The endoscopic camera ofclaim 1;

a plurality of endoscopes that may be detachably attached to the endoscopic camera, each endoscope comprising relay lenses and an identifying means indicating a model of the endoscope, where each endoscope has a chromatic focal difference between white light with a first wavelength band and fluoresced emission light with a second wavelength band longer than that of the first wavelength band, where the chromatic focal difference is caused by the dispersive or diffractive properties of optical materials or optical design employed therein;

a light source able to provide white light illumination in a first mode and fluorescence excitation illumination in a second mode; and

a memory on which is stored focus settings for multiple endoscope models, including known, prescribed adjustments to provide desired focal settings required for each endoscope to provide in-focus images at the image sensors.

12. The endoscopic system ofclaim 11, further comprising a camera control module and wherein the memory is an element of the camera control module.

13. The endoscopic system ofclaim 12, wherein the camera control module further comprises image processing circuitry.

14. A method for providing an automatic focus adjustment of an endoscopic camera, comprising the steps of

providing an endoscope comprising relay lenses and an identifying means indicating a model of the endoscope, where the endoscope has a chromatic focal difference between white light with a first wavelength band and fluoresced emission light with a second wavelength band longer than that of the first wavelength band, where the chromatic focal difference is caused by the dispersive or diffractive properties of optical materials or optical design employed therein;

providing the endoscopic camera to which the endoscope may be removably attached, the endoscopic camera comprising a focusing lens group to receive and condition a relayed light from the endoscope and an image sensor assembly comprising a first image sensor and a focus adjustment mechanism;

providing a memory on which is stored focus settings for multiple endoscope models, including known, prescribed adjustments to provide desired focal settings required for each endoscope to provide in-focus images at the image sensor;

attaching the endoscope to the endoscopic camera;

recognizing, by means of the endoscope identifying means, the model of the endoscope; and

retrieving from the memory the focal settings for the attached endoscope.

15. The method ofclaim 14, comprising the further step of collecting image light in a visible light mode by illuminating a subject scene with white light, collecting image light with an objective lens of the endoscope, relaying light by means of the relay lenses from the objective lens to the focusing lens group, focusing the relayed visible light, and collecting an in-focus white light image with the first image sensor.

16. The method ofclaim 15, comprising the further step of subsequently collecting a fluoresced light image, comprising the steps of

applying the automatic focus adjustment to the endoscopic camera;

illuminating the subject scene with a fluorescence excitation light;

collecting fluorescence emission image light with the objective lens of the endoscope, relaying light by means of the relay lenses, from the objective lens to the focusing lens group, focusing the relayed fluorescence emission light; and;

collecting an in-focus fluoresced light image with the first image sensor or a second image sensor.

17. The method ofclaim 16 wherein the automatic focus adjustment comprises the step of adjusting the position of image sensor collecting the fluoresced light image a distance D along a focal path orthogonal to an image sensitive surface thereof.

18. The method ofclaim 16, wherein the automatic focus adjustment comprises the step of adjusting the curvature of a liquid-based deformable lens positioned along an optical path within the endoscopic camera.

19. The method ofclaim 14, comprising the further steps of

collecting image light by illuminating a subject scene, collecting image light with an objective lens of the endoscope, relaying light by means of the relay lenses from the objective lens to the focusing lens group, focusing the relayed visible light,

splitting the relayed light downstream from the focusing lens group such that the first image sensor receives light in the visible spectrum; and

providing a second image sensor that receives fluorescence image light.

20. The method ofclaim 19, wherein the relayed light is split by a dichroic prism.

Images