US8956396B1 - Eye-tracking visual prosthetic and method - Google Patents

Abstract

An improved prosthesis and method for stimulating vision nerves to obtain a vision sensation that is useful for the patient that has lost vision due to AMD, RP, and other diseases. The invention utilizes infrared light to cause action potentials in the retinal nerves similar to those which result from rods and cones stimulated by visible light in healthy retinas. In some embodiments, the invention provides a prosthesis that generates a stimulation pattern of infrared light from an external stimulator array through the eye and focusing the stimulation pattern of infrared light on the retina, especially the fovea. Some embodiments the invention provides improved resolution down to a group of nerves, or even the individual nerve level, with sufficient energy density so as to cause a desired action potential.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority benefit under 35 U.S.C. §119(e) of U.S. Provisional Patent Application No. 61/514,894 filed Aug. 3, 2011 by Michael E. Friend et al., titled “SIGHT-RESTORING VISUAL PROSTHETIC AND METHOD USING INFRARED NERVE-STIMULATION LIGHT”, which is incorporated herein by reference in its entirety.

This invention is related to the following prior applications and patents:

• U.S. patent application Ser. No. 13/525,268 filed on even date herewith (Jun. 15, 2012), titled “SIGHT-RESTORING VISUAL PROSTHETIC AND METHOD USING INFRARED NERVE-STIMULATION LIGHT”;
• U.S. Provisional Patent Application No. 60/715,884 filed Sep. 9, 2005, titled “Apparatus and Method for Optical Stimulation of Nerves”;
• U.S. patent application Ser. No. 11/257,793 filed Oct. 24, 2005, titled “Apparatus for Optical Stimulation of Nerves and Other Animal Tissue” (now U.S. Pat. No. 7,736,382 issued Jun. 15, 2010);
• U.S. Provisional Patent Application No. 60/826,538 filed Sep. 21, 2006, titled “Miniature Apparatus and Method for Optical Stimulation of Nerves and Other Animal Tissue”;
• U.S. patent application Ser. No. 11/536,639 filed Sep. 28, 2006, titled “Miniature Apparatus and Method for Optical Stimulation of Nerves and Other Animal Tissue” (now U.S. Pat. No. 7,988,688 issued Aug. 2, 2011);
• U.S. patent application Ser. No. 11/536,642 filed Sep. 28, 2006, titled “Apparatus and Method for Stimulation of Nerves and Automated Control of Surgical Instruments”;
• U.S. Provisional Patent Application No. 60/884,619 filed Jan. 11, 2007, titled “Vestibular Implant Using Infrared Nerve Stimulation”;
• U.S. patent application Ser. No. 11/971,874 filed Jan. 9, 2008, titled “Method and Vestibular Implant using Optical Stimulation of Nerves” (now U.S. Pat. No. 8,012,189 issued Sep. 6, 2011);
• U.S. Provisional Patent Application No. 60/964,634 filed Aug. 13, 2007, titled “VCSEL Array Stimulator Apparatus and Method for Light Stimulation of Bodily Tissues”;
• U.S. patent application Ser. No. 12/191,301 filed Aug. 13, 2008, titled “VCSEL Array Stimulator Apparatus and Method for Light Stimulation of Bodily Tissues” (now U.S. Pat. No. 8,475,506 issued Jul. 2, 2013);
• U.S. Provisional Patent Application No. 61/015,665 filed Dec. 20, 2007, titled “Laser Stimulation of the Auditory System at 1.94 μm and Microsecond Pulse Durations”;
• U.S. Provisional Patent Application No. 61/147,073 filed Jan. 23, 2009, titled “Optical Stimulation Using Infrared Lasers (or In Combination with Electrical Stimulation) of the Auditory Brainstem and/or Midbrain”;
• U.S. Provisional Patent Application 61/081,732 filed on Jul. 17, 2008, titled “Method and Apparatus for Neural Signal Capture to Drive Neuroprostheses or Bodily Function,”
• U.S. Provisional Patent Application 61/226,661 filed on Jul. 17, 2009, titled “Method and Apparatus for Neural-Signal Capture to Drive Neuroprostheses or Control Bodily Function,”
• U.S. Patent Publication No. 2010/0016732 (U.S. patent application Ser. No. 12/505,462) filed on Jul. 17, 2009, titled “Method and Apparatus for Neural-Signal Capture to Drive Neuroprostheses or Control Bodily Function”;
• U.S. patent application Ser. No. 12/693,427 filed Jan. 25, 2010, titled “Optical Stimulation of the Brainstem and/or Midbrain, including Auditory Areas” (now U.S. Pat. No. 8,744,570 issued Jun. 3, 2014);
• U.S. Provisional Patent Application No. 61/349,813 filed May 28, 2010, by Jonathon D. Wells et al., titled “Laser-Based Nerve Stimulators for, e.g., Hearing Restoration in Cochlear Prostheses”;
• U.S. Provisional Patent Application No. 61/381,933 filed Sep. 10, 2010, by Jonathon D. Wells et al., titled “Laser-Based Nerve Stimulators for, e.g., Hearing Restoration in Cochlear Prostheses and Method”;
• U.S. patent application Ser. No. 12/890,602 filed Sep. 24, 2010, by Jonathon D. Wells et al., titled “Laser-Based Nerve Stimulators for, e.g., Hearing Restoration in Cochlear Prostheses and Method” (now U.S. Pat. No. 8,792,978 issued Jul. 29, 2014);
• U.S. Provisional Patent Application No. 61/349,810 filed May 28, 2010, by Jonathon D. Wells et al., titled “Implantable Infrared Nerve Stimulation Devices for Peripheral and Cranial Nerve Interfaces”;
• U.S. Provisional Patent Application No. 61/386,461 filed Sep. 24, 2010, by Jonathon D. Wells et al., titled “Implantable Infrared Nerve Stimulation Devices for Peripheral and Cranial Nerve Interfaces”;
• U.S. patent application Ser. No. 13/117,121 filed May 26, 2011, by Jonathon D. Wells et al., titled “Implantable Infrared Nerve Stimulation Devices for Peripheral and Cranial Nerve Interfaces”;
• U.S. patent application Ser. No. 13/117,122 filed May 26, 2011, by Jonathon D. Wells et al., titled “Cuff Apparatus and Method for Optical and/or Electrical Nerve Stimulation of Peripheral Nerves” (now U.S. Pat. No. 8,652,187 issued Feb. 18, 2014);
• U.S. patent application Ser. No. 13/117,125 filed May 26, 2011, by Jonathon D. Wells et al., titled “Nerve-Penetrating Apparatus and Method for Optical and/or Electrical Nerve Stimulation of Peripheral Nerves”;
• U.S. patent application Ser. No. 13/117,118 filed May 26, 2011, by Jonathon D. Wells et al., titled “Optical Bundle Apparatus and Method for Optical and/or Electrical Nerve Stimulation of Peripheral Nerves” (now U.S. Pat. No. 8,864,806 issued Oct. 21, 2014);
• U.S. patent application Ser. No. 13/204,610 filed Aug. 5, 2011, by Michael E. Friend et al., titled “OCULAR IMPLANT AND METHOD FOR TRANSMISSION OF NERVE-STIMULATION LIGHT” (now U.S. Pat. No. 8,709,078 issued Apr. 29, 2014);
• U.S. Provisional Patent Application 61/511,020 filed Jul. 22, 2011 by Ryan C. Stafford, titled “METHOD AND APPARATUS FOR OPTIMIZING AN OPTICALLY STIMULATING COCHLEAR IMPLANT”,
• U.S. Provisional Patent Application 61/511,048 filed Jul. 22, 2011 by Ryan C. Stafford, titled “BROAD WAVELENGTH PROFILE TO HOMOGENIZE THE ABSORPTION PROFILE IN OPTICAL STIMULATION OF NERVES”, and
• U.S. Provisional Patent Application 61/511,050 filed Jul. 22, 2011 by Ryan C. Stafford et al., titled “OPTICAL COCHLEAR IMPLANT WITH ELECTRODE(S) AT THE APICAL END FOR STIMULATION OF APICAL SPIRAL GANGLION CELLS OF THE COCHLEA”, each of which is incorporated herein by reference in its entirety.

FIELD OF THE INVENTION

The invention relates generally to methods and apparatus for nerve stimulation, and more particularly to image processing and emission of optical signals (such as with pulsed infrared light) for stimulation of nerves in a human's eyes to obtain a sensation of vision.

BACKGROUND OF THE INVENTION

For many patients suffering from retinal degenerative diseases such as advanced or age-related macular degeneration (AMD) and retinitis pigmentosa (RP) there has been little hope for maintaining vision. Every year, 700,000 new cases of AMD in the U.S. are diagnosed and 10% of those patients will become legally blind. There are presently no cures for these debilitating diseases, and, at best, current treatments only slow the disease progression. The overall social and economic impact of AMD and RP is immense and the importance of treating blindness is profound as this is a problem of significant scope and breadth. There is an unmet need to treat this ailment by developing a visual prosthetic with a large number (e.g., thousands) of stimulation channels to realistically restore sight using infrared light to stimulate the retinal nerves. Advanced macular degeneration and retinitis pigmentosa are both degenerative diseases that degrade vision in patients and eventually can lead to blindness.

Researchers have artificially stimulated various parts of the human nervous system for many years as a way to restore lost or damaged neural function of various systems in the human body. Neuroprosthetic devices circumvent non-functioning physiological structures (hair cells in the ear, rods and cones in the eye) which would normally transduce an external stimulus (sound, light) into an action potential. Presently, there are numerous efforts underway to develop neuroprostheses to restore sight at various interventional anatomical locations: in the subretina, the epiretina, the optic nerve and in the visual cortex. These devices apply an electric current pulse to stimulate the neurons of the visual system which is inherently hindered by a lack of spatial selectivity. Electrical current spread leads to imprecise nerve stimulation and limits the ability of the neuroprosthesis to restore function. The limitation of spatial selectivity is based on fundamental physical principles of electrical stimulation. To date, after 20 years of development, electrical implants are just now hoping to make the jump to 64-channel systems from 16-channel systems. This is far less than the thousands of channels estimated to be needed for a good vision prosthetic. The technology is further limited by the fact that physical contact is required with tissue, which can lead to damage over time. Implantation of a complex powered device in very close proximity to sensitive neural tissue forms a significant drawback to this approach, making it impossible to update the technology without further risky surgeries.

There have been rudimentary attempts to stimulate the retinal nerves with electrical signals, which are being conducted by various groups globally. For example, the Argus™ II implantable device, by Second Sight Medical Products, Inc., 12744 San Fernando Road—Building 3, Sylmar, Calif. 91342, U.S.A., which is intended to treat profoundly blind people suffering from degenerative diseases such as RP. The Second Sight Medical Products, Inc. Argus™ II system works by converting video images captured from a miniature camera, housed in the patient's glasses, into a series of small electrical pulses that are transmitted wirelessly to an epiretinal prosthesis array of electrodes implanted inside the eye on the retina. These pulses then stimulate the retina's remaining cells resulting in the corresponding perception of patterns of light in the brain. Patients supposedly learn to interpret these visual patterns thereby gaining some functional vision.

U.S. Pat. No. 7,079,900 issued Jul. 18, 2006 to Greenberg et al., titled “Electrode Array for Neural Stimulation,” is incorporated herein by reference. Greenberg et al. describe a retinal prosthesis to restore color vision by electrically stimulating areas of undamaged retinal cells, which remain in patients with lost or degraded visual function. There are three main parts: one is external to the eye, the second part is internal to the eye, and the third part communicates between those two parts. The external part has color imaging means (CCD or CMOS video camera), an eye-tracker, a head-motion tracker, a data processor, a patient's controller, a physician's local controller, a physician's remote controller, and a telemetry means. The color data is processed in the video data processing unit and encoded by time sequences of pulses separated by varying amounts of time, and also with the pulse duration being varied in time. The basis for the color encoding is the individual color code reference. Direct color stimulation is another operational basis for providing color perception. The electrodes stimulate the target cells so as to create a color image for the patient, corresponding to the original image as seen by the video camera. The physician's test unit can be used to set up or evaluate and test the implant during or soon after implantation.

U.S. Pat. No. 7,914,842 issued Mar. 29, 2011 to Greenberg et al., titled “Method of Manufacturing a Flexible Circuit Electrode Array,” is incorporated herein by reference. Greenberg et al. describe polymer materials and electrode array bodies for neural stimulation, especially for retinal stimulation to create vision. The method lays down a polymer layer, applies a metal layer to the polymer and pattern to create electrodes and leads, and applies a second polymer layer over the metal layer and pattern to leave openings for electrodes. The array and its supply cable are a single body.

Electrical stimulation represents a major challenge in developing implantable devices with long-term system performance while reducing their overall size. The Boston Retinal Implant Project has identified long-term biocompatibility as one of the most significant challenges to be met in order to develop a successful retinal prosthesis. For example, U.S. Pat. No. 6,324,429 issued Nov. 27, 2001 to Shire et al., titled “Chronically Implantable Retinal Prosthesis,” is incorporated herein by reference. Shire et al. describe a chronically implantable retinal prosthesis for the blind which will restore some useful vision to patients over at least several degrees of their former field of view. These thin, strong, and flexible epiretinal devices are constructed of or encapsulated in known biocompatible materials which will have a long working life in the eye's saline environment. The function of the implants is to electrically stimulate the ganglion cell layer at the surface of the retina using controlled current sources. Due to the exceptionally low mass of the implant and its flexible, nearly planar form, patient discomfort and fluid drag caused by the implant minimized. These physical attributes also substantially reduce the potential of harm to the most delicate structure of the eye, the retina, and therefore enhance the long term safety and biocompatibility of the device. Since no micro-cables are required to be attached to the device, and its overall form and edges are rounded, the device is not expected to stress the retina during chronic implantation. A provision is also made for nutrients to reach the retinal cells underneath the device for their long-term health.

U.S. Pat. No. 7,908,010 issued Mar. 15, 2011 to Greenberg et al., titled “Retinal Prosthesis with Side Mounted Inductive Coil,” is incorporated herein by reference. Greenberg et al. describe a retinal prosthesis with an inductive coil mounted to the side of the eye by means of a strap around the eye. This allows for close coupling to an external coil and movement of the entire implanted portion with movement of the eyeball.

Electrical stimulation, as described in the above devices and patents, is limited since the spread of electricity does not allow separate or independent stimulation of individual retinal nerve cells or even small-enough groups of nerve cells for sharp or clear vision. This electrical-stimulation technology is severely limited, as electricity spreads in human tissue and thus will severely limit the number of stimulation sites. Electrical stimulation thus greatly limits the number of sites that could be separately stimulated. Additionally, the electrical-stimulation approach will require implantation of a powered (e.g., an electrically powered) device, which has significantly difficult issues associated with obtaining power into the eye and using the power by devices in the eye.

An implant has been developed which refocuses the light normally aimed at the fovea region of the retina to the periphery of the retina. This approach is limited as there is a high degree of integration on the periphery of the fovea that, again, will severely limit resolution.

Other work is being done in the area of optogenetics wherein a virus is used to genetically sensitize nerve cells to certain wavelengths of light, e.g., PCT publication WO 2010/011404 A2 titled “Vectors for Delivery of Light-Sensitive Proteins and Methods of Use,” which is incorporated herein by reference. Attempts have been made to sensitize nerve cells to optical light via injection of foreign DNA (optogenetic engineering). This area may have some potential, however it will require significant development work, it involves injecting a virus into nerve tissue (which may have significant side effects and FDA-approval issues), and the virus is only partially taken up by nerve cells.

Materials that are compatible with the eye are described in U.S. Pat. No. 6,254,637 to Jin Hak Lee et al., titled “Artificial Cornea and Implantation Thereof”; U.S. Pat. No. 6,391,055 to Yoshito Ikada et al., titled “Artificial Cornea”; U.S. Pat. No. 6,976,997 to Noolandi et al., titled “Artificial Cornea”; U.S. Pat. No. 7,857,849 to David Myung et al., titled “Artificial corneal implant”; and U.S. Pat. No. 7,909,867 to David Myung et al., titled “Interpenetrating Polymer Network Hydrogel Corneal Prosthesis”; each of which is incorporated herein by reference in its entirety.

Numerous digital-light-projection micro-electro-mechanical-system (MEMS) devices exist. For example, U.S. Pat. No. 4,566,935 issued to Hornbeck on Jan. 28, 1986, titled “Spatial Light Modulator and Method,” is incorporated herein by reference in its entirety. Hornbeck described methods of fabrication of spatial light modulators (SLMs) with deflectable beams by plasma etching after dicing of a substrate into chips, each of the chips an SLM. Various architectures available with such plasma etching process were disclosed and include metal cloverleafs for substrate addressing, metal flaps formed in a reflecting layer over a photoresist spacer layer, and torsion-hinged flaps in a reflecting layer.

As another MEMS-display example, U.S. Pat. No. 7,776,631 issued to Miles on Aug. 17, 2010, titled “MEMS Device and Method of Forming a MEMS Device,” and is incorporated herein by reference in its entirety. Miles described light in the visible spectrum being modulated using an array of modulation elements, and control circuitry connected to the array for controlling each of the modulation elements independently, each of the modulation elements having a surface which is caused to exhibit a predetermined impedance characteristic to particular frequencies of light.

U.S. Pat. No. 7,177,081 issued to Tomita et al. on Feb. 13, 2007, titled “High Contrast Grating Light Valve Type Device,” and is incorporated herein by reference in its entirety. Tomita et al. describe a grating light valve with a plurality of spaced reflective ribbons that are spatially arranged over a substrate with reflective surfaces. The grating light valve is configured to optimize the conditions for constructive and destructive interference with an incident light source having a wavelength λ. The grating light valve preferably has a set of movable active ribbons alternating between the set of stationary bias ribbons. In operation, active ribbons are moved by a multiple of λ/4 to switch between the conditions for constructive and destructive interference.

U.S. Pat. No. 4,720,189 issued Jan. 19, 1988 to Heynen et al., titled “Eye-Position Sensor,” is incorporated herein by reference in its entirety. Heynen et al. describe an eye-position sensor for use in an eye-activated optical transducer in which a spatial filter is used to modify light reflected from the eye to form a substantially rectangular pattern on a quadrennial array of contiguous sensors. This arrangement provides a substantially linear change in the output signal from the sensors in response to an equivalent movement of the eye.

U.S. Pat. No. 6,055,110 issued Apr. 25, 2000 to Kintz et al., titled “Compact Display System Controlled by Eye Position Sensor System,” is incorporated herein by reference in its entirety. Kintz et al. describe a virtual image display system is provided which is made thinner through the use of an immersed beam splitter, and in one embodiment, total internal reflection. The display system includes an imaging surface on which a source object is formed, a first optical element having a reflective function and a magnification function, a second optical element having a magnification function and an immersed beam-splitting element positioned between the first and second optical elements, the immersed beam-splitting element including a beam splitter surrounded by an optically transparent material having a refractive index greater than air. An illumination source projects the source object formed at the imaging surface through the optically transparent material to the beam splitter. The beam splitter reflects the projected source object to the first optical element. The first optical element magnifies the projected source object and reflects a magnified virtual image of the projected source object to the beam splitter. The magnified virtual image traverses the beam splitter to the second optical element which magnifies the magnified virtual image to produce a compound magnified virtual image of the source object.

Yet, there remains a need in the art for an improved prosthesis and method for stimulating vision nerves to obtain a vision sensation that is more useful for the person.

BRIEF SUMMARY OF THE INVENTION

The present invention uses infrared nerve-stimulation (INS) technology that exploits infrared light (and/or other optical wavelengths) to cause action potentials in nerve cells in the eye. In recent years, optical-stimulation technology has been developed to stimulate nerves. This INS technology can achieve much higher precision and selectivity of stimulation than is possible using electrical current to trigger nerve action potentials. In some embodiments, the present technology uses pulsed, infrared lasers to excite the neural tissue next to the retina directly and without tissue damage. The advent of this technology represents a paradigm shift in artificial nerve stimulation because it allows a high degree of spatial selectivity of neural stimulation without the need for tissue contact.

The present invention provides a prosthetic system and method for restoring vision that was lost to advanced macular degeneration (AMD), retinitis pigmentosa (RP), or other causes. Infrared-light nerve stimulation (INS) is a technology that uses infrared light to trigger nerve action potentials (NAPs) in various nerves of an animal. The cells being stimulated are not necessarily the retina cone and rod cells themselves, which retina cone and rod cells are normally associated with receiving visible light (having wavelengths in the range of about 400 nm to about 700 nm) and that emit NAPs based on the visible light received to these cells. Instead, the cells stimulated by the present invention are typically cells in and/or near the nerves leading from the retina. The present invention leverages INS but applies it in a new manner. For the present invention, INS is used to stimulate the retinal nerves (those nerves between the light-sensitive cones and rods of the retina (the usual source of nerve action potentials for sight) and the optic nerve that exits the eye) of a person (the patient). These retinal nerves are essentially insensitive to visible light; however, nerve-action potential signals can be triggered by application of pulsed light, particularly light that has an infrared wavelength in the range of about 1800 nm to about 1900 nm. In some embodiments, images are first captured with a miniaturized camera and translated into infrared-light stimulation signals (i.e., changed from signals representing visible-light colors and intensities at each of an array of pixel positions, into a series of signal pulses having durations, repetition frequencies, and positions that may be different than the positions and temporal intensities that would directly correspond to the original image positions and intensities) that are then transmitted into the eye from a two-dimensional (2D) vertical-cavity surface-emitting laser (VCSEL) array (e.g., a grid of rows and columns of independently operable vertical-cavity surface-emitting lasers) or other suitable light source(s) and imaging modalities.

In the present invention, the wavelength(s), power(s) and/or energy(s) of the light and/or light pulses emitted from the laser elements in the VCSEL array, and the locations toward which the light is directed are optimized in order to most efficiently stimulate the retinal nerves (i.e., to obtain the desired nerve response) without damaging the nerves being stimulated and without drawing too much power from the power supply of the prosthesis. In some embodiments, the camera(s), processor(s), and VCSEL array(s) are all packaged into a pair of eyeglasses-like device or other headwear that is worn by the person. In some embodiments, in order to facilitate propagation of the infrared stimulation light to the retinal nerves, a passive light-focussing-and-transmitting implant (e.g., a polymer conduit) is surgically implanted within the eye of the patient. In some embodiments, this implant is made of an acrylic, or other suitable polymer, glass, or other material, that transmits well in the infrared (IR) region, and which is commonly used for certain ocular implants or specifically developed (i.e., having the desired specific gravity, focussing, transparency, bio-compatibility characteristics and the like) for use with the present invention. In some embodiments, this ocular implant also includes one or more lenses, and in order to appropriately focus and direct the path of the infrared stimulation light. In some embodiments, the implant length extends a considerable distance through the posterior chamber of the eye such that the optical path distance of the stimulating light within vitreous humor is minimized (in some embodiments, while maintaining a sufficient layer or film of vitreous humor at the back of the eye for the proper physiological functioning of the retina and optic nerves). In some embodiments, the implant is anchored in the region of the lens, while in other embodiments, a mount on one or more other surfaces of the eye is included.

In some embodiments, the sources of infrared (for stimulating nerves from the light-sensing cells that no longer function) and visible light (for sending conventional visible-light image information to the light-sensing cells that continue to function) are physically configured in a display that moves with the eye (e.g.,FIGS. 6A-6C and8, which include electro-mechanical actuators or the like to position the display(s) and camera(s) in different locations and angular directions, orFIGS. 7A-7D, which include light-beam scanners to sweep the infrared simulation sources and visible light sources); adjusting the geometry of image acquisition and image projection (left/right, angle, up/down, longitudinal (focus)); sensing a direction of gaze and adjusting stimulation light geometries accordingly. In some embodiments, cylinder focus (astigmatism) is also corrected by the displays of the present invention (e.g., in some embodiments, by inserting a lens or other optics that provide the required correction).

In some embodiments, the present invention provides one or more devices for focusing stimulation light. In some such embodiments, the device(s) project, onto and/or into the eye, visible points of light (fiducials) that are then imaged by a camera or other imager from outside the eye to assist the focusing of the stimulation light source to the desired nerves to be stimulated. In some such embodiments, the device(s) project stimulation light onto the sclera of the outer surface of the eye and correlate this projection to infer where light would focus on the back of the eye, and/or to determine the direction of gaze.

In some such embodiments, the device(s) project the stimulation light toward the back of the eye, and the camera or other imager obtains images of the tissue at the back of the eye, with the stimulation light projected thereon, from outside the eye, and these images are used to assist the focusing of the stimulation light source to the desired nerves to be stimulated, or use a camera located on the headwear and imaging the side of the eye to observe and/or follow some feature of the eye or fiducial device to assist the focusing of the stimulation light and/or to determine the direction of gaze. In some such embodiments, the device(s) project ultrasound signals (e.g., chirps or pulses) toward the eye and sense the return signal to assist the focusing of the stimulation light.

The infrared visual prosthetic (IVP) system of the present invention is an improved approach over conventional electrical stimulation or other methods conventionally used to stimulate vision nerves since the IVP increases the possible number of stimulation sites by at least an order of magnitude over electrical stimulation, and the IVP does not require implantation into the patient of a powered device. Additionally, if optogenetic technology is eventually successful, technology of the IVP stimulation device may be directly applicable.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A demonstrates animage101 of lily pads and flowers using 48 channels, 432 channels, and 6912 channels.

FIG.1B1 is a much-enlarged schematic representation of a small portion of anarray102 of stimulation sources having both IR-nerve-stimulation VCSELs and visible light emitters.

FIG.1B2 is an enlarged schematic representation of a larger portion ofarray102 of stimulation sources having both IR-nerve-stimulation VCSELs and visible light emitters.

FIG. 1C is a schematic representation of asystem103 with a hardware- and operating-environment having an implanted passive optical-transmission device200, an optional externallyworn device111 and acustomization console computer20.

FIG. 2 is a cut-away perspective view of a nerve-stimulation system101 for aneye90 that illustrates an implantedocular unit200, according to some embodiments of the invention.

FIG. 3 is a perspective view of a nerve-stimulation system301, according to some embodiments of the invention.

FIG. 4A is a high-level flow diagram of theimage generating system401, according to some embodiments of the invention.

FIG. 4B is a high-level flow diagram of theimage generating system402, according to some embodiments of the invention.

FIG. 4C is a high-level data-flow diagram of theimage generating system402, according to some embodiments of the invention.

FIG. 5A is a simplified flow diagram ofimage processing501 according to some embodiments of the invention.

FIG. 5B is a flow diagram ofimage processing502 according to some embodiments of the invention.

FIG. 5C is a schematic representation of asystem503 having an implanteddevice110, an externallyworn device111 and atransceiver71 of customization console computer such as shown inFIG. 1C.

FIG. 6A is a top cross-section view of a movable-external-light-source subsystem601 in a first position relative to the eye of a patient, according to some embodiments of the present invention.

FIG. 6B is a top cross-section view of a movable-external-light-source subsystem601 in a second position relative to the eye of a patient.

FIG. 6C is a top cross-section view of a movable-external-light-source subsystem601 in a third position relative to the eye of a patient.

FIG. 7A is a top cross-section view of a scannable-external-light-source subsystem701 emitting light toward a first position and toward a second position of a reflectingarray751 relative to the eye of a patient, according to some embodiments of the present invention.

FIG. 7B is a top cross-section view of a scannable-external-light-source subsystem702 emitting light toward a first position and toward a second position of a holographic ordiffractive unit752 based on movement of the eye of a patient, according to some embodiments of the present invention.

FIG. 7C is a top cross-section view of a MEMS-array scannable-external-light-source subsystem703 emitting light toward a first position and toward a second position of aMEMS reflector unit753 based on movement of the eye of a patient, according to some embodiments of the present invention.

FIG. 7D is a top cross-section view of a MEMS-array scannable-external-light-source subsystem704 having a plurality of emitters (such as diode or fiber lasers), according to some embodiments of the present invention.

FIG. 8 is top cross-section view of a multiple element externallight source subsystem801.

DETAILED DESCRIPTION OF THE INVENTION

Although the following detailed description contains many specifics for the purpose of illustration, a person of ordinary skill in the art will appreciate that many variations and alterations to the following details are within the scope of the invention. Very narrow and specific examples are used to illustrate particular embodiments; however, the invention described in the claims is not intended to be limited to only these examples, but rather includes combinations and subcombinations of features shown in the various figures and descriptions. The invention includes the full scope of the attached claims. Accordingly, the following preferred embodiments of the invention are set forth without any loss of generality to, and without imposing limitations upon the claimed invention. Further, in the following detailed description of the preferred embodiments, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the invention may be practiced. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.

The leading digit(s) of reference numbers appearing in the Figures generally corresponds to the Figure number in which that component is first introduced, such that the same reference number is used throughout to refer to an identical component which appears in multiple Figures. Signals and connections may be referred to by the same reference number or label, and the actual meaning will be clear from its use in the context of the description.

In recent years, a pioneering technology has been developed to stimulate nerves, which can achieve much more precision and selectivity of stimulation than electric current. This technology uses pulsed, infrared lasers to excite the neural tissue directly without tissue damage. The advent of this technology represents a paradigm shift in artificial nerve stimulation because it allows a high degree of spatial selectivity of neural stimulation without the need for tissue contact.

Research is in process of using infrared-light nerve stimulation (INS) technology to provide hearing restoration using an implantable prosthetic in the cochlea and vestibular systems. In cochlear implants, the higher spatial selectivity of INS allows the implant to better mimic the natural stimulation of the neurons which results in superior hearing fidelity. This approach also has added benefits over existing technologies including residual hearing preservation and faster healing from surgery because light can be directed through the round window. The enhancement achievable over electrical stimulation has only been demonstrated in animals. Additional work is necessary to provide safety and efficacy data and develop the device hardware and software for nerve stimulation such that it will be ready for clinical trials.

Transitioning INS technology to the retina has some unique challenges not found in the cochlea, however along with those challenges are some unique opportunities. The higher level of specificity enabled by INS technology can potentially result in a prosthetic containing tens of thousands of channels. For the retina, high quantities of channels are especially important for realistic sight restoration.

FIG. 1A demonstrates animage101 of lily pads and flowers using 48 channels, 432 channels, and 6912 channels. Most of today's electrical optical stimulators are working in the range of 64 to 256 channels. Higher numbers of channels are needed to restore vision to more realistic levels.

FIG.1B1 is a much-enlarged schematic representation of a small portion of anarray102 of stimulation sources having both IR-nerve-stimulation light source(s) (in some embodiments, VCSELs that emit infrared light of one or more wavelengths that are effective to stimulate nerve action potential signals in nerves when applied as a pulse of sufficient energy and properly focussed) and visible-light source(s) (in some embodiments, VCSELs that emit visible light of one or more wavelengths (such as red, green and blue)), used by some embodiments of the present invention. In some embodiments,array102 is fabricated as a VCSEL array on one or more chips, wherein emitters of the various wavelengths are arranged in an array such as shown in this example, which shows three IR wavelengths and three visible wavelengths interleaved with one another. In some embodiments, more or fewer different IR wavelengths are used (in some embodiments, a single IR wavelength is used), and in some embodiments, more or fewer visible wavelengths are used. In some other embodiments, such as described below in the description ofFIG. 7D a plurality of emitters (such as diode or fiber lasers) are used in a scanned projection system (e.g., in some embodiments, one, two, or threeIR lasers711 and one, two, or three visiblelight lasers711′ emit collimated light toward asystem704 of MEMS-deflection mirrors that create the spatial pattern of each wavelength that is projected into ocular implant200 (seeFIG. 2) and then onto the retinal nerves and/or retina of the patient). In such a system that combines both visible light emitters (to stimulate those areas of the light-sensitive rods and cones in the retina that remain functional in a patient with partial loss of vision) and IR nerve-stimulation light emitters (that stimulate the nerves leading from the non-functional rods and cones, which nerves generally overlie rods and cones that may or may not still be functional), these embodiments of the present invention provide the novel capability of stimulating both the remaining functional rod and cone cells that underlie the layer of nerve cells that conduct signals toward the optical nerve, as well as the nerve cells leading from non-functional rod and cone cells. In some embodiments, the present invention provides the ability to test the effectiveness of each emitter or group of emitters, and using the results of the testing to reprogram the controller to recalibrate the algorithms that convert image into stimulation signals and change which areas of the eye receive visible-light signals and which areas of the eye receive IR light signals. These visible-stimulation and IR-stimulation areas can overlap (to provide additional stimulation capabilities when desired) or not overlap (selectively turning off various of the stimulation sources, particularly to save electrical power by not using emitters that provide no function (providing little or no IR nerve stimulation to nerves from functioning visible-light-sensitive cells and little or no visible light to cells that have been found by testing to be non-functioning), and/or to prevent overheating of tissue by reducing the frequency of nerve-stimulation pulses to areas that have recently been stimulated to near a limit of the tissue's ability to dissipate local heating, and/or to turn off the stimulation when the patient blinks or closes their eyes). In some embodiments,array102 is as described further in the description of FIG.1B2 below.

FIG.1B2 is an enlarged schematic representation of a larger portion ofarray102 of stimulation sources having both IR-nerve-stimulation VCSELs and visible light emitters. In some embodiments,array102 includes a first plurality of IR-nerve-stimulation pixels (IR1 pixels, wherein theIR1 emitters131 are indicated with a clear background and are always “OFF” until configuration software enables them, while theIR1 emitters141 are selectively enabled and indicated with a hashed background inFIG. 1C), a second plurality of IR-nerve-stimulation pixels (IR2 pixels, wherein theIR2 emitters132 are indicated with a clear background and are always “OFF” until configuration software enables them, while theIR2 emitters142 are selectively enabled and indicated with a hashed background inFIG. 1C), and a third plurality of IR-nerve-stimulation pixels (IR3 pixels, wherein theIR3 emitters133 are indicated with a clear background and are always “OFF” (inactivated for stimulation and not powered for extended periods of time) until configuration software enables them, while theIR3 emitters143 are “ON” (activated for use and selectively enabled based on the input image) and indicated with a hashed background inFIG. 1C). In some embodiments,array102 includes a plurality ofIR1 pixels131 that are turned OFF (inactivated for stimulation and not emitting regardless of the received image for extended periods of time, e.g., until a recalibration and/or change in stimulation mode) and a plurality ofIR1 pixels141 that are turned ON (activated for use and selectively emitting, depending on the received image and the image processing that may temporarily inhibit emitting based on heuristics in the image-processing algorithm that enhance certain features such as edges and motion, and/or to prevent local overheating). In some embodiments,array102 includes a plurality ofIR2 pixels132 that are turned OFF (not emitting regardless of the received image) and a plurality ofIR2 pixels142 that are turned ON (selectively emitting, depending on the factors described above). In some embodiments,array102 includes a plurality ofIR3 pixels133 that are turned OFF (not emitting regardless of the received image) and a plurality ofIR3 pixels143 that are turned ON (selectively emitting, depending on the factors described above). In some embodiments,array102 includes a plurality of visible-red-light pixels136 that are turned OFF (not emitting regardless of the received image), a plurality of visible-red-light pixels146 that are turned ON (selectively emitting, depending on the factors described above), a plurality of visible-green-light pixels137 that are turned OFF (not emitting regardless of the received image), a plurality of visible-green-light pixels147 that are turned ON (selectively emitting, depending on the factors described above), a plurality of visible-blue-light pixels138 that are turned OFF (not emitting regardless of the received image), and a plurality of visible-blue-light pixels148 that are turned ON (selectively emitting, depending on the factors described above). Note that in the example shown in FIG.1B2, the processor has been programmed to turn off most of the visible-light pixels in the center of this portion of the array102 (such that these turned-off pixels do not emit regardless of the received image), and turn on (selectively enable for light emission depending on the received image) most of the visible-light pixels around the edges of this portion of thearray102, and the processor has been programmed to turn on (selectively enable for light emission depending on the received image) most of the IR-light pixels in the center of this portion of thearray102, and turn off (such that these turned-off pixels do not emit regardless of the received image) most of the IR-light pixels around the edges of this portion of thearray102. Such a configuration may be used for a patient with macular degeneration where the macula has stopped functioning. Note that even in such a situation, there may be some cells in the mostly IR-light-stimulation area that are still functional as to visible light and thus selective ones of the visible-light emitters are enabled, and there may be nerves over the mostly visible-light stimulation areas that one desired to stimulate and thus some of the IR emitters around the edges are then enabled for selective emission to stimulate those nerves.

In some embodiments, the device of the present invention has a plurality of stimulation modes that can be selectively activated, and different subsets of theemitters102 are activated for use in the different modes of operation. For example, in some embodiments, a first set of modes of operation can be optimized for reading and enhance the contrast of the image, change the received text to black-and-white or other contrasting colors, enhance edges and smooth edges, and similar image processing that makes text images more readable and understandable (in some embodiments, the device can also be set to a text-reading calibration mode and generate various test patterns, and elicit and receive user input to customize the image-processing algorithm to best suit the individual user for one or more different text-reading situations, such that the user can later select one of one-or-more different text-reading modes based on the text to be read). In some embodiments, one or more additional modes of operation can be calibrated and selectively activated based on the environment and/or user input, such that the image processing can pre-process the video images to suit a particular activity, such as recognizing various colors useful for cooking food and identifying types of food or defects in the food, or for recognizing different denominations of money, or for walking and bicycling outdoors, or for participating in various social or business activities, or for manipulating and assembling parts as part of a job or hobby. In some embodiments, two or more image-processing modes can be selectively activated at the same time, such that the device of the present invention can simultaneously be suited for combined activities (such as card playing, which can benefit from both shape and text enhancement as well as color differentiation). In some embodiments, the device can receive user input in conventional ways (such as from voice commands or manual mode selection) to select its operational mode, and/or by automatically recognizing the need for a particular mode (such as by sensing when the eye-direction (gaze) of the user is directed towards an area of the field of view that has text or particular symbols, or that faces of people are in the field of view), and the device then automatically changes its mode of operation to enhance image processing to suit that environment. Thus, in some of the plurality of image-processing modes, different subsets of the visible-light emitters146,147, and/or148 and of the infrared-light emitters136,137, and/or138 can be activated or de-activated, based on the content of the environmental video images received and on the various areas of nerves to be stimulated (where rods and cones are not properly functioning) and areas of rod and cones to receive visible wavelengths of light.

In some embodiments, the present invention provides a test-and-recalibration system for the patient to use by themselves, or with the help of a care-giver, that provides an interactive test (e.g., a program that goes through a set of images displayed on a personal computer (such as a desktop computer, iPad®-like tablet, smart phone or the like), and based on patient feedback of the perceptions of those images, recalibrates which pixels are to be IR-stimulation pixels and which are to be visible-light emitters, and also frequently recalibrates the image-processing algorithms to optimize the perceived vision each week, each day, or each hour, or on an activity-by-activity basis (e.g., in some embodiments, different algorithms are used for reading, for household chores, or for outdoor walking and the like). In some embodiments, this recalibration program determines which nerves (or areas of visual perception) are stimulated by which IR-stimulation pixel position, and stores a set of mapping parameters that are used by the image processor (for example, in some embodiments, a system and algorithm such as illustrated inFIG. 5B described below) to assist in mapping the received environmental image to particular ones of the nerves to be stimulated. For example, in some embodiments, the test program displays a set of images on the screen of a smart phone, which is programmed to receive user input (e.g., by touch screen input or voice input) as to what the patient user perceives, and the smart phone (system103 ofFIG. 1C) then wirelessly transmits mapping parameters to the externally worn device (device111 ofFIG. 1C), which uses the mapping parameters to convert the data from the received environmental image to particular ones of the stimulation sources to stimulate the desired nerves to achieve the vision sensations for the patient in subsequent use of the now-recalibrateddevice111. In other embodiments, thedevice111 itself generates the series of test stimulation signals that include both visible-light stimulation signals and IR-light stimulation signals, anddevice111 itself receives feedback input from the patient (either directly (such as by voice input, motion input or touch input) or indirectly (by wireless signals from another device such as a smart phone or personal computer), anddevice111 performs the necessary recalibration and re-mapping functions and stores the parameters needed for future mapping in thedevice111 as used in a particular environment. In yet other embodiments, thedevice111 itself generates the series of test stimulation signals that include both visible-light stimulation signals and IR-light stimulation signals, anddevice111 itself receives feedback input from imaging the patient's eye (e.g., optically sensing physiological nerve signals that are generated in various nerves as a result of the stimulation, such as by the systems and methods described in U.S. Patent Publication No. 2010/0016732 (U.S. patent application Ser. No. 12/505,462) filed on Jul. 17, 2009, titled “Method and Apparatus for Neural-Signal Capture to Drive Neuroprostheses or Control Bodily Function,” which is incorporated herein by reference), and does the needed recalibration periodically or in real time (on a periodic (weekly, daily, hourly, or based on a given activity, such as reading versus walking outside) or continuous basis throughout the day of usage).

In other embodiments, (not shown) two or more chips of VCSEL arrays or other emitters (for example, one chip with an array of VCSELs all emitting just one wavelength of IR-stimulation light signal, and one chip with an array of MEMS mirrors used to reflect three visible wavelengths (e.g., red, green, and blue), or one chip with an array of VCSELs all emitting just one wavelength of IR-stimulation light signal, another chip with an array of VCSELs all emitting red-stimulation light signal, another chip with an array of VCSELs all emitting green-stimulation light signal, and another chip with an array of VCSELs all emitting blue-stimulation light signal) are provided (e.g., side-by-side) and projection optics are used to superimpose pixel images from the plurality of sources each including an array of one or more wavelengths of emitters. In some such embodiments, the sources include both array emitters (such as a two-dimensional VCSEL array, wherein each pixel of light of a given wavelength from that “object” (the 2D source of light) is imaged into the back of the eye, such that each pixel of the “image” (the 2D image of the “object” as projected into the back of the eye) comes from a different VCSEL in the array) and projection emitters (wherein a plurality of the pixels of light imaged into the back of the eye all come from a single emitter (e.g., a diode or fiber laser, or a wavelength-conversion device such as a non-linear frequency doubler of 1064-nm IR light that is frequency doubled to 532-nm green light), wherein the light from that single emitter is scanned to different pixel positions at the back of the eye by a system of MEMS mirrors, for example, as described below).

FIG. 1C is an overview schematic diagram of a hardware- and operating-environment (or system)103 that is used in conjunction with embodiments of the invention. The description ofFIG. 1C is intended to provide a brief, general description of suitable computer hardware and a suitable computing environment in conjunction with which the invention may be implemented. In some embodiments, aspects of the invention are described in the general context of computer-executable instructions, such as program modules, that are stored on computer-readable media and that are executed by a computer, such as a microprocessor residing in an implanted device (located within a patient) and/or in an external device worn by the patient and/or personal computer that is/are wirelessly linked to the implanted device. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types.

In some embodiments,system103 includes an optometrist- and/or user-control console computer20 that is programmable and that has awireless transceiver71 that allows wireless control (i.e., reprogramming of the remote microprocessors) of the externally worn device111 (which includes a programmed microcontroller), that transmits pulsed laser nerve-stimulation laser-light signals130 to the passive implanted device (ocular unit200). In some embodiments,application programs36 stored on a computer-readable storage device (e.g., optical disk31 (CDROM, DVD, Blu-ray® Disc (BD), or the like), magnetic or FLASH storage device29 (e.g., floppy disk, thumb drive, SDHC (Secure Digital High Capacity, one type of FLASH memory) memory card or the like), and/or astorage device50 connected to aremote computer49 that connects tocomputer20 across a local-area network51 or a wide-area network52 such as the internet) contain instructions and/or control structures (such as look-up tables, control parameters, databases and the like) that are processed and/or transmitted into the externally worndevice111 to control its operation by methods of the present invention described herein. In some embodiments, theapplications programs36 are partially executed in thecomputer20 and/or the externally worndevice111.

Accordingly, in some embodiments, an optometrist and/or user can adjust parameters of the externally worndevice111 to customize its operation to a much greater extent than is possible with a conventional electrical-stimulation ocular implant, because externallyworn device111 has a far greater number of parameters that can be finely adjusted (e.g., pulse width, amplitude, frequency, wavelength, polarization, wavelength profile, beam profile, beam angle, and, the like). In some embodiments, theapplications programs36 contain a substantial amount of safety control code that runs incomputer20 to guide the optometrist and/or user to adjust the parameters of the externally worndevice111 and to help prevent operation that might harm the patient or damage the externally worn device111 (such as what might occur if too much optical energy were applied in a concentrated small area of the nerve layers of the retina or within too short a period of time, or if overheating occurred in thedevice111 due to too many vertical-cavity surface-emitting lasers (VCSELs) next to one another being activated in a short period of time).

Although many of the embodiments herein have light-emitting elements that include VCSELs (vertical-cavity surface emitting lasers) implemented as electrically pumped semiconductor diode lasers, other embodiments of the present invention use edge-emitting semiconductor diode lasers, optically pumped semiconductor lasers, optically pumped optical-fiber lasers, light-emitting diodes, superluminescent devices, or any other suitable light source. Some embodiments use wavelengths in the range of 1.75 microns to 2 microns, other embodiments use any other suitable wavelengths.

Moreover, those skilled in the art will appreciate that the invention may be practiced with other computer-system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. The invention may also be practiced in distributed computer environments where tasks are performed by remote processing and input-output (I/O) devices that are linked through a communications network. In a distributed-computing environment, program modules may be located in both local and remote storage devices.

As shown inFIG. 1C, in some embodiments, the hardware- and operating-environment includes optometrist- and/or user-control console computer20, or aserver20, including aprocessing unit21, asystem memory22, and asystem bus23 that operatively couples various system components including thesystem memory22 to theprocessing unit21. In some embodiments, there may be only one, or in other embodiments, there may be more than oneprocessing unit21, such that the processor ofcomputer20 comprises a single central-processing unit (CPU), or a plurality of processing units, commonly referred to as a multiprocessor or parallel-processing environment. In various embodiments,computer20 may be implemented using a conventional computer, a distributed computer, or any other type of computer including those embedded in cell phones, personal-data-assistant devices or other form factors.

Thesystem bus23 can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The system memory can also be referred to as simply the memory, and includes read-only memory (ROM)24 and random-access memory (RAM)25, a basic input/output system (BIOS)26, containing the basic routines that help to transfer information between elements within the computer (or server)20, such as during start-up, may be stored inROM24. Thecomputer20 further includes ahard disk drive27 for reading from and writing to a magnetic hard disk, a removable-media drive or FLASH controller28 for reading from or writing to a removable magnetic floppy-disk orFLASH storage device29, and anoptical disk drive30 for reading from or writing to a removable optical disk31 (such as a CDROM, DVD, Blu-ray Disc (BD) or other optical media).

Thehard disk drive27, magnetic disk drive28, andoptical disk drive30 couple with a harddisk drive interface32, a magneticdisk drive interface33, and an opticaldisk drive interface34, respectively. The drives and their associated computer-readable media provide non-volatile, non-ephemeral storage of computer-readable instructions, data structures, program modules and other data for thecomputer20. It should be appreciated by those skilled in the art that any type of computer-readable media which can store data that is accessible by a computer, such as magnetic cassettes, FLASH memory cards, digital video disks, Bernoulli cartridges, random-access memories (RAMs), read-only memories (ROMs), redundant arrays of independent disks (e.g., RAID storage devices) and the like, can be used in the exemplary operating environment.

A plurality of program modules that implement the optimization methods of the present invention can be stored on the hard disk, magnetic orFLASH storage device29,optical disk31,ROM24, orRAM25, including anoperating system35, one ormore application programs36,other program modules37, andprogram data38. A plug-in program containing a security transmission engine for the present invention can be resident on any one, or on a plurality of these computer-readable media.

In some embodiments, a user (e.g., the optometrist or the patient) enters commands and perception information into thecomputer20 through input devices such as a keyboard40, pointingdevice42 or other suitable device such as a microphone (not shown). Other input and/or output devices (not shown) can include a microphone, joystick, game pad, satellite dish, scanner, speaker, headphones or the like. These other input and output devices are often connected to theprocessing unit21 through aserial port interface46 that is coupled to thesystem bus23, but can be connected by other interfaces, such as a parallel port, game port, or a universal serial bus (USB), a monitor47 or other type of display device can also be connected to thesystem bus23 via an interface, such as avideo adapter48. The monitor47 can display a graphical user interface for the optometrist and/or user. In addition to the monitor47, computers typically include other peripheral output devices (not shown), such as speakers and printers.

In some embodiments, the monitor47 or other image-output device outputs calibration images, andsystem20 elicits and receives user input (e.g., as to perception sensations) into thecomputer20 through keyboard40 or any other suitable input device, for example such as described above. In some embodiments, the calibration images include images having lines, edges, object movement, colors and/or other features at various angles, positions and motions in the field of vision, and the user input is used to map the regions of nerves that provide various visual sensations to the patient. This retinal map (i.e., a map that indicates which sensations result from optical stimulation of each small area of the nerve layers of the retina) is then stored in a computer-readable data structure (CRDS) and used in the transformation of image information (intensities and/or colors at each pixel location) into pulsed IR stimulation light at each of a plurality of areas on the nerve layers of the retina.

In some embodiments,computer20 operates in a networked environment using logical connections to one or more remote computers or servers, such asremote computer49. These logical connections are achieved by a communication device coupled to or a part of thecomputer20; the invention is not limited to a particular type of communications device. Theremote computer49 can be another computer, a server, a router, a network PC, a client, a peer device or other common network node, and typically includes many or all of the elements described above relative to thecomputer20, although onlymemory storage device50 andapplication programs36 have been illustrated inFIG. 1C. The logical connections depicted inFIG. 1C include local-area network (LAN)51 and wide-area network (WAN)52. Such networking environments are commonplace in office networks, enterprise-wide computer networks, intranets and the Internet, which are all types of networks. In some embodiments, thelocal computer20 has merely a user interface (e.g., a browser) to aremote system49 located remote from the user and accessed across the internet, wherein thesystem49 performs the control and calculations of the calibration process. In some embodiments, the method of the calibration process is embodied as instructions and/or data structures stored on a computer-readable storage medium that causes the method to be performed when the instructions are read and executed by a suitable information-processing system.

When used in a local-area networking (LAN) environment, thecomputer20 is connected to theLAN51 through a network interface, modem or adapter53, which is one type of communications device. When used in a wide-area networking (WAN) environment such as the internet, thecomputer20 typically includes an adaptor or modem54 (a type of communications device), or any other type of communications device, e.g., a wireless transceiver, for establishing communications over thewide area network52, such as the internet. Themodem54, which may be internal or external, is connected to thesystem bus23 via theserial port interface46. In a networked environment, program modules depicted relative to thepersonal computer20, or portions thereof, (or those stored in the externally worndevice111 or the implanted ocular unit200) can be stored in the remotememory storage device50 of remote computer, orserver49 and accessed over the internet or other communications mechanism. Note that the transitory signals on the internet may move stored program code from a non-transitory storage medium at one location to a computer that executes the code at another location by the signals on one or more networks. The program instructions and data structures obtained from a network or the internet are not “stored” on the network itself, but are stored in non-transitory storage media that may be connected to the internet from time to time for access. It is appreciated that the network connections shown are exemplary and in some embodiments, other mechanisms of, and communications devices for, establishing a communications link between the computers may be used including hybrid fiber-coax connections, T1-T3 lines, DSL's, OC-3 and/or OC-12, TCP/IP, microwave, WAP (wireless application protocol), and all other electronic media through standard switches, routers, outlets and power lines, as the same are known and understood by one of ordinary skill in the art.

The hardware and operating environment in conjunction with which embodiments of the invention may be practiced has been described. Thecomputer20 in conjunction with which embodiments of the invention can be practiced can be a conventional computer, a distributed computer, or any other type of computer; the invention is not so limited. Such acomputer20 typically includes one or more processing units as its processor, and a computer-readable medium such as a memory. Thecomputer20 can also include a communications device such as a network adapter or a modem, so that it is able to communicatively couple to other computers, servers, or devices.

In some embodiments, one or more parts ofsystem103 elicits and receives input from a user, and based on the input, modifies, adjusts or executes one or more of the methods of the present invention as described herein.

The INS technology as disclosed in the invention herein also enables the use of an external stimulator, thus eliminating the need to implant a powered device (only a passive implant device) and allowing for easier upgrades as the stimulator technology advances since the active stimulator device is not implanted. Because the tissue of the eye normally absorbs at infrared light wavelengths, a passive implant will have to be surgically placed in the eye to create a “light pipe” from the external stimulator array to the retina nerve cells. Infrared light from an external infrared stimulating array passes through an implanted “light pipe” that allows it to be imaged or focused on the retina, especially the macula and fovea. In some embodiments, the main light-transmission material of the passive implant is an acrylic such as PMMA, which is used today for ocular implants lenses. The implant allows the infrared light to reach the retina which would normally be highly absorbed by the fluid of the eye.

In some embodiments, the system provides sight restoration to patients blinded by retinitis pigmentosa (RP) and age-related or advanced macular degeneration (AMD). It utilizes infrared-nerve-stimulation (INS) technology to cause action potentials in retinal nerve cells in the in the retina, especially the macula and fovea regions. In some embodiments, two 2D (2-dimensional) VCSEL arrays (one for each eye) are used to generate the stimulation patterns at the stimulating infrared wavelength (1400-1900 nm). In some embodiments, images are captured with miniaturized cameras and converted into stimulation patterns in an image processor. In order for the stimulating light to be able to reach the retinal nerves, an ocular implant is placed in the eyeball. The ocular implant is made of a material that is both biocompatible and highly transmissive to the stimulating light. In some embodiments, the ocular implant contains optics that serve to focus the light on the back on the retina. In some embodiments, the VCSEL arrays, camera, processor, and power supply are all packaged in a headset or set of eyeglasses to be worn by the patient. In some other embodiments, portions of the system (e.g., the power supply and/or processor) are packaged separate from the headset or glasses that include the VCSEL arrays.

FIG. 2 is a cut-away perspective view of a nerve-stimulation system201 for aneye90 that illustrates an implantedocular unit200, according to some embodiments of the invention. Appendix A., titled “Ocular Implant and Method for Transmission of Nerve-Stimulation Light” of related U.S. Provisional Patent Application No. 61/514,894 filed Aug. 3, 2011 by Michael E. Friend et al., titled “SIGHT-RESTORING VISUAL PROSTHETIC AND METHOD USING INFRARED NERVE-STIMULATION LIGHT”, incorporated herein by reference, describes various ocular implants and methods for transmission of nerve-stimulation light through the eye.

In some embodiments, theocular unit200 includes a light-transparent pathway or “image pipe”110 (which includes anoptional lens system116 and a transparent body117) for transmitting a stimulation pattern of infrared light130 from anexternal stimulator array180 through theeye90 alongoptical path199, theocular unit200 having a light-receivinganterior end112 closest to the eye's anterior surface and extending to aposterior end114 ofimage pipe110 closer to thefovea96 than to the eye's anterior surface, transmitting light and/or projectingimage122 onto theretina97, including onto themacula95 andfovea96. In some embodiments, the curved anterior surface ofimage pipe110 acts as the anterior-end focussing element and no separately formedlens116 is needed.

As used herein, an “image pipe” is an optical device that forms an image just beyond its posterior end (e.g., when an image pipe of the present invention is implanted in the eye, the image is formed on the nerves at the anterior surface of the retina) that is based on light130 entering the anterior end. In some embodiments, an image pipe includes internal imaging components such as lenses, holographs, fiber optics or fiber-optic bundles, or the like, which assist in providing a focussed image at the retina. In other embodiments, the image pipe is simply a transparent path that allows external imaging components to form the image on the nerves at the front surface of the retina. Because some embodiments of the present invention use single-wavelength infrared lasers, holographic imagers are well suited to form images through such an image pipe.

In some embodiments, theimage pipe110 is substantially transparent to at least some infrared wavelengths of light between about 1000 nm and about 2000 nm, and in particular, is substantially transparent to those infrared wavelengths output by the source lasers of the stimulation apparatus. In some embodiments, theimage pipe110 has a substantially cylindrical shape such as shown inFIG. 2, such that both ends of theimage pipe110 have substantially the same diameter. In some embodiments, theimage pipe110 is formed from a biocompatible-transparent-thermoplastic material such as poly(methyl methacrylate) (PMMA), or the like. In some embodiments, such as shown inFIG. 2, the light-receivinganterior end112 ofocular unit200 replaces at least a portion of thecornea99 of the eye and thus forms part of the anterior surface of the eye.

Poly(methyl methacrylate) (PMMA) is a transparent thermoplastic. PMMA has been sold under many different names including Plexiglas®, Lucite® and Perspex®. PMMA is substantially transparent (i.e., a given thickness of about a centimeter or more passes a majority of incident light) to visible light (having wavelengths of 400 nm to 700 nm) and infrared light (IR) having wavelengths from about 700 nm to about 2800 nm. Colored or tinted PMMA varieties allow specific IR wavelengths to pass while blocking visible light and/or other IR wavelengths.

In some embodiments,ocular unit200 is surgically secured in place to thecornea99 and/orsclera98 in the eye with anchoringcollar140 andhydrogel skirt150. In some embodiments, the implant is sewn (or stapled or otherwise anchored) to the ciliary muscle or secured to other internal parts of the eye to hold it securely in place.Ocular unit200 extends well into thevitreous humor94, which is less transparent than isimage pipe110 to certain infrared light wavelengths useful for nerve stimulation.

Theposterior end114 of theimage pipe110 is closer to the fovea than the front of the eye. In some embodiments,image pipe110 has a length such that theposterior end114 of theimage pipe110 is near theretina97 in the region of themacula95 andfovea96. In some embodiments, theimage pipe110 does not contact theretina97, in order to leave a pathway for thevitreous humor94 to circulate and nourish the cells of the retina. In some embodiments, theposterior end112 is positioned close enough to theretina97 andfovea96 such that the remaining vitreous humor is thin enough and transparent enough that infrared light output from the posterior end of theimage pipe110 will be sufficiently intense to cause retinal-nerve stimulation (i.e., triggering of nerve action potentials in the nerves of the retina due to impinging pulses of infrared light).

In some embodiments, theocular image pipe110 is solid material. PMMA has a higher density than the vitreous humor. To more closely match the density of the vitreous humor, some embodiments ofimage pipe110 include at least one hollow portion such that the overall density of theimage pipe110 is the same as the density of the surrounding vitreous humor and the center of mass of theimage pipe110 coincides with the center of rotation of the eye, in order that theimage pipe110 does not tend to move relative to the eye with movement. In some embodiments, the hollow portion is filled with an inert gas. In some embodiments, the hollow portion is filled with a low-pressure gas having a pressure of no more than about 1000 Torr. In some embodiments, the hollow portion is in the light path of the light path and at least one end of the hollow portion is shaped to form a lens to focus the infrared light on nerves of the retina.

The placement, size, and shape of the hollow portion in theimage pipe110 is used in some embodiments to not only match the density of the vitreous humor but to also control the center of gravity to help provide a more stable implant the is resistant to movement of the head or eyeball. In some embodiments, the light-transmitting portion ofimage pipe110 is solid material and the hollow portion is formed in a peripheral portion outside and surrounding the light-transmitting path. This configuration reduces the number of optical interfaces in the light path. In some embodiments, the light-transmitting portion ofimage pipe110 is solid material and the hollow portion is formed symmetrically around a peripheral portion outside and surrounding the light-transmitting path, such that regardless of whether the person's head is upright or is lying on one side, there is no rotational or other force acting to move the implant (i.e., image pipe110) relative to the eye. In other embodiments, the hollow portion is formed in (or is very slightly larger in) a top portion ofimage pipe110, in order to help keep theimage pipe110 upright and in the desired position when the patient's head is upright.

In some embodiments, one or both ends of theimage pipe110 are shaped to focus the external stimulator array signals130 on the retina and fovea. In some embodiments, there is an externallight source180 that emits IR-wavelength stimulation light130. For example, in some embodiments,source180 includes a two-dimensional array of vertical-cavity surface-emitting lasers (VCSEL-array) that form an IR stimulator, which provides IR light130 into theanterior end112 of theocular implant200. In some embodiments, the user has anocular unit200 implanted in each eye, and the system provides there is a separate external two-dimensional arrayIR stimulator source180 for each eye, wherein the two separate images help provide three-dimensional images to the brain through each eye'socular unit200. In some embodiments,image pipe110 includes a lens orlens system116, with a different index of refraction than the rest ofimage pipe110, to focus the image on theretina97. In some embodiments, thelens system116 inverts the incoming image and focuses the image on the retina. In some other embodiments, thelens system116 is non-inverting and directs diverging, collimated, or converging light on the nerve-tissue layer of theretina97. In some embodiments, theimage pipe110 and itslens116, in combination with an external laser image-generation device180 and its image processor(s)182 and one or more cameras incamera system181, produce an infrared image on the retina, similar to the inverted optical-wavelength image a normal human eye. Since the human brain will automatically accustom itself to any image consistently formed on the retina whether or not the image is inverted, thecamera system181,image processor182 and stimulationlight sources180 can be configured to form the image as inverted or not according to the preferences of the user. In some embodiments,camera system181 includes at least one camera directed toward the user's eye (e.g., to determine the locations ofindicia118 and/or119) to determine the location and/or direction of movement of the gaze of the user, and this image of the eye (or of theindicia118/119) is processed byimage processor system182 in order to control the position of the stimulation light sources that are generating the stimulation light signals130, in order to position the projected pattern of stimulation light onto the desired locations onretina122. In some embodiments, the array oflight sources180 themselves are physically moved to the desired position, while in other embodiments, different ones of thelight sources180 that are already in the desired positions are activated.

In some embodiments, theocular unit200 has at least one indicia mark to facilitate detection of the eye's position. In some embodiments, the ocular unit has at least oneindicia mark118 on the anterior end to facilitate external detection of the position of the eye and the pointing directions. In some embodiments, theocular unit200 has at least oneindicia mark119 on the posterior end to facilitate external detection of the position of the eye and the pointing directions. In some embodiments, one or more indicia marks are placed on both the anterior end posterior end, and/or on one or more other locations on theocular unit200. In some embodiments the location and/or orientation of the implant is determined, for example, by obtaining an image of, or detecting reflected or fluorescent light from, the indicia mark or marks118 and/or119 and the external stimulator array signals are adjusted to compensate for the position of the eye (e.g., the image or pattern is moved such that the desired nerve tissue continues to be stimulated). In some such embodiments, an eye-position processor in theexternal image processor182 uses an “inward-pointing” camera in camera system181 (i.e., a camera pointed toward the user to obtain an image of the eye and/orindicia118/119) to detect movement or position of the user's eye(s), and generates control signals that direct an external camera view (i.e., the direction in which thecamera system181 is pointing, or if a very-wide-angle lens and/or multiple cameras are used, which of the images obtained bycamera system181 is used), providing a more realistic sensation of “looking around” to the user, instead of requiring movement of the user's entire head to obtain different images. In some embodiments, a plurality of “outward-pointing” cameras is included in camera system181 (i.e., a plurality of cameras pointed toward different directions in the environment surrounding user to obtain a plurality of images from which to select based on the detected direction of the user's gaze).

In some embodiments, (e.g.,FIG. 2) thestimulation system101 includes acamera181, animage processor182, and alaser output device180 that outputs stimulation light signals intoocular implant200. In some embodiments, theimage processor182 receives images signals fromcamera181 that represent pixel-intensity values for light received at each of a plurality of pixels in a rectangular array of pixels. In some embodiments, the rectangular array of pixels detected bycamera181 includes a plurality of pixels for each of a plurality of colors (each having a range of wavelengths).

In some embodiments, such asFIG. 2, theimage processor182 receives thecamera181 output and converts the image information into electrical signals that drive the IR stimulationlight sources180. The brightness of a particular pixel in the field of vision is not directly converted to the brightness of a particular IR stimulation signal. The individual stimulation signals that are transmitted into the eye are converted by the image processor from a series of image brightness signals for each pixel into pulse repetition patterns that represent brightness wherein the pulse repetition patterns stimulate the nerves of the retina to produce actions potentials. In some embodiments, and individual stimulating laser signal is off, or on at a below-threshold level, and then pulsed on in order to trigger the nerve action signal in the retina. The stimulation signals generated are pulse representations of the brightness of a pixel, and the pulse intensity is not a representation of the brightness, similar to how the nerves in a normal eye work. The image processor converts image pixel brightness, and in some embodiments converts the image pixel color, into what will become IR stimulating nerve pulse patterns, where the pulse patterns are of substantially the same height and the same length sufficient to generate nerve axons but at different pulse rate frequencies and directed to different locations on the retina in order to accomplish the perception of vision to the patient. In some embodiments, the perception of different colors is reproduced for the patient by stimulating particular nerve axons of the retina rather than by modifying the frequency of the IR stimulating light.

In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about 1 to 2 pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about two to five (2 to 5) pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about five to ten (5 to 10) pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about 10 to 20 pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about 20 to 50 pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about 50 to 100 pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about 100 to 200 pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about 200 to 500 pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about 500 to 1000 pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is about 1000 to 2000 pulses per second. In some embodiments, the nerve stimulating pulse repetition rate of the optical signal is more than about 2000 pulses per second.

In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of no more than 4 J/cm²per nerve-action-potential (NAP) response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of no more than 3 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of no more than 2 J/cm²per NAP response generated.

In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 5 J/cm²and about 6 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 4 J/cm²and about 5 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 3 J/cm²and about 4 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 3 J/cm²and about 3.5 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 2.5 J/cm²and about 3 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 2 J/cm²and about 2.5 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 1.5 J/cm²and about 2 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 1 J/cm²and about 1.5 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 0.5 J/cm²and about 1 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 0.2 J/cm²and about 0.5 J/cm²per NAP response generated. In some embodiments of the apparatus, the one or more lasers output an infrared signal having a radiant exposure of the retina of between about 0.1 J/cm²and about 0.2 J/cm²per NAP response generated.

In some embodiments, the one or more lasers output an infrared signal having an energy of less than about 2 mJ per pulse.

In some embodiments, the one or more lasers output an infrared signal providing an energy of less than about 2 mJ per pulse to the retina.

In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about ten microseconds (10 μs) and about five milliseconds (5 ms).

In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 1 μs and about 10 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 10 μs and about 20 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 20 μs and about 50 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 20 μs and about 40 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 40 μs and about 80 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 80 μs and about 160 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 50 μs and about 100 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 100 μs and about 200 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 200 μs and about 500 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 200 μs and about 400 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 400 μs and about 800 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 800 μs and about 1600 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 500 μs and about 1000 μs. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 1 millisecond (ms) and about 2 ms. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 2 ms and about 5 ms. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 2 ms and about 4 ms. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 4 ms and about 8 ms. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 8 ms and about 16 ms. In some embodiments, the one or more lasers output an infrared signal having a pulse width of between about 5 ms and about 10 ms.

FIG. 3 is a perspective view of a nerve-stimulation system301, according to some embodiments of the invention. Nerve-stimulation system301 includes aheadset310 with one ormore cameras181 and one ormore displays180. In some embodiments, camera(s) capture images that are conveyed to the image processor (not shown inFIG. 3). The image processor converts the images to electrical signals that drive the display(s)180. In some embodiments, the displays include VCSEL arrays to provide an array of infrared laser signals. In some embodiments, as demonstrated inFIG. 3, one camera and one display is used for each eye to restore stereoscopic vision with depth perception. In some embodiments, thedisplays180 convert electrical signals from the image processor into a plurality of infrared laser signals that are transmitted through the eye(s)90 using implanted ocular unit(s)200 to stimulate NAPS in the patient's retina.

In some embodiments, the image processor and associated power supply is mounted on or in theheadset310. In some other embodiments, one or both of the image processor and associated power supply are remotely mounted, e.g., on a belt worn around the patient's waist.

In some embodiments, eye position sensors detect the position or movement of eye(s)90 and the image processor or other mechanisms (e.g., seeFIG. 6A,FIG. 6B,FIG. 6C,FIG. 7A,FIG. 7B,FIG. 7C, andFIG. 8 described below) are used to adjust the display(s)180 to compensate for the eye movement. In some embodiments, the eye position sensors are mounted or imbedded on the display(s)180.

FIG. 4A is a high-level flow diagram of theimage generating system401, according to some embodiments of the invention. In some embodiments, camera(s)181 capture images that are processed by theimage processor182 into signals driving the stimulationlight sources180. Additional information provided to the image processor, in some embodiments, includes the positions of the eyes detected by eye position sensor(s)183.

FIG. 4B is a high-level flow diagram of theimage generating system402, according to some embodiments of the invention. Patients with a partially damaged retina that is still largely light sensitive (such as when a detached retina has been reattached but the edges could not be reconnected without a gap or missing or relocated pieces, or in the case of starting macular degeneration), it is sometimes beneficial to obtain image information with camera(s)181, detect eye position with sensor(s)183, remap the locations of various pieces of the image information usingimage processor184, and generate a visible-light image using visible-light sources185 (using configurations such as shown inFIGS. 6A,6B,6C,7A,7B,7C, and/or8 or other suitable image sources), where various pieces of the image have been re-arranged in their positions, such that when projected or imaged onto the patient's remaining light-sensitive retina areas, the image is perceived by the patient's eye and brain to be a normal, or at least improved vision sensation. As one particular example, some persons with a gap in their reattached retina would normally not see the portions of the scene that land in the gap, but when the image of the scene is remapped such that parts of the scene are moved to the sensitive areas on one side of the gap and other parts of the scene are moved to the sensitive areas on the other side of the gap, a continuous scene is perceived. As another particular example, some persons with macular degeneration have lost light-sensitivity in the detail-sensing central part of the retina, but could perceive sentences in a book or newspaper if the successive letters are enlarged and scanned across the remaining light-sensitive peripheral retina area, so some embodiments of the present invention focus on printed text, acquire an image of the text, and then imageprocessor184 enlarges the letters andlight sources185 scan the test across the peripheral field(s) of remaining vision so that the patient can read the text. As yet another example, the eye may require changes in eye fixation direction (either microsaccades, or more large-scale changes in gaze direction), and for patients unable to provide this movement of the eye, some embodiments of the present invention move the displayed or projected image to compensate for the lack of eye movement. As still another example, binocular fixation disparity (e.g., perhaps caused by strabismus, also known as heterotropia), is a condition in which the eyes are not properly aligned with each other, and in some embodiments, the present invention independently moves the positions of the displayed or projected visible images based on the two different directions of gaze of the two eyes of the patient.

FIG. 4C is a high-level data-flow diagram of theimage generating system402, according to some embodiments of the invention. In some embodiments, camera(s)181 acquire image data481 (shown in this example as areas A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P). A map of the sensitive areas of theretina490 is obtained in a calibration procedure to determine where the various light-sensitive areas (e.g., thelocations491 of the retina on either side of gap484 (there may be more than one gap, and each sensitive area is mapped), and an image-retina mapping process is used to remap the positions of various regions of the image (in some embodiments, each region is a set of one or more pixels) so that the appropriate regions of the retina are illuminated with the corresponding regions from the image. In some embodiments, eye-position detector184 detects theposition483 locating where in the field of possible views the gaze of the patient is directed, and this is used to further map the positions of the various regions from the original camera image. In other embodiments, the order of mapping operations are rearranged or combined. The visible light sources (e.g., display screen or image projector)185 then create the visible-light image485 that is imaged into the eye of the patient, and is perceived as an improved sensedvision480 with reduced or eliminated gaps. In some embodiments, thelight sources485 comprise a display screen that moves within a range of possible X and Y positions. In other embodiments, the display is in a fixed position relative to the eye, and the image is moved to various X and Y positions on the display screen.

FIG. 5A is a flow chart of amethod501, according to some embodiments of the present invention. In some embodiments ofmethod501, at block510 a video signal is obtained from one or more cameras configured to obtain signals of a scene representing the visual field the patient would have if the patient could see, and to generate electrical signals that are processed byprocess512 to obtain real-time data representing the image of the visual filed as an array of pixels, wherein the array of pixels representing the scene at the current time (or processing cycle) is commonly referred to as a frame. In some embodiments, the frame pixels of the scene are processed byprocess530 to select which pixels will be combined within each of a plurality of bins. In some embodiments, experimental rules for visual stimulation are derived and a storage-medium having a general-rule computer-readable data structure (CRDS)580 contains general rules for mapping image input to visual stimulation based on physiological considerations.Operation530 takes the pixel information for the current time frame (and optionally from a predetermined number of prior time frames) and using the heat-based anti-tissue-damage-rules CRDS580 (which limit the number of VCSELs that are allowed to be activated in any one time frame and/or within P successive time frames), along with data from VCSEL heat-based anti-device-damage-rules and rules for stimulation pulse shapes (pulse width and/or rise/fall shape) also inCRDS580, andoperation530 generates pulse parameters for the stimulation light for each VCSEL to be activated.Operation595 takes the pulse parameters fromoperation530 and drives the VCSELs to emit stimulation signals596 (a set of infrared optical-stimulation signal pulses) which are transmitted597 through the ocular implant to the retina to triggerNAPs598 providing a vision sensation. In some embodiments, a reloadable computer-readable storage medium590 holds instructions data structures that control the operations described above.

FIG. 5B is a flow chart of amethod502, according to some embodiments of the present invention. In some embodiments ofmethod502, at block510 a video signal is obtained from one or more cameras configured to obtain signals of a scene representing the visual field the patient would have if the patient could see, and to generate electrical signals that are processed byprocess512 to obtain real-time data representing the image of the scene as an array of pixels, wherein the array of pixels representing the scene at the current time (or processing cycle) is commonly referred to as a frame. In some embodiments, the frame pixels of the scene are processed byprocess522 to select which pixels will be combined within each of a plurality of bins, and a history of such values is stored byprocess524 intodata structure586. In some embodiments, experimental rules for visual stimulation are derived and a storage-medium having a general-rule computer-readable data structure (CRDS)584 contains general rules for mapping image input to visual stimulation based on physiological considerations. In addition, some embodiments include a patient-specific CRDS585 that modifies or supplements the general-rule CRDS584. In some embodiments, patient-specific CRDS585 is derived by empirically outputting a set of visual output signals to the implanted system and eliciting and receiving feedback from the patient indicative of the sensations perceived as a result of the optical stimulation applied. In some embodiments, operation520, based on general-rule CRDS584 and patient-specific CRDS585, derives a map of visual input (e.g., pixels)-to-stimulation site(s) based on the patient's feedback. In some embodiments for each successive time frame,operation522 combines the pixel brightness values from image-processing operation512 and the mapping rules from520 into bin-stimulation values. In some embodiments,operation524 stores into CRDS586 a history of the bin and/or pixel values for most-recent P frames.Operation530 then takes the bin information for the current time frame (and optionally from a predetermined number of prior time frames) and using the heat-based anti-tissue-damage-rules CRDS582 (which limit the number of bins that are allowed to be activated in any one time frame and/or within P successive time frames), along with data from VCSEL heat-based anti-device-damage-rules CRDS581 and rules for stimulation pulse shapes (pulse width and/or rise/fall shape) inCRDS583, andoperation530 generates pulse parameters for the stimulation light for each VCSEL to be activated.Operation595 takes the pulse parameters fromoperation530 and drives the VCSELs to emit stimulation signals596 (a set of infrared optical-stimulation signal pulses) which are transmitted597 through the ocular implant to the retina to trigger NAPs. In some embodiments, the resulting physiological response is a set ofNAPs598 that is transmitted to the brain of the patient, andoperation518 optionally measures the nerve response andoperation519 processes a feedback signal that is fed back intooperation530. In some embodiments, a reloadable computer-readable storage medium590 holds instructions data structures that control the operations described above.

In some embodiments, the present invention further includes eye-position cameras514 that track where in the visual field the patient's eye is pointed.Operation515 takes the eye-position camera information from514 and generates control signals516 used inoperation530 to re-map VCSELs altering the portion of the scene transmitted by the VCSELs to the patient's retina. In other embodiments,operation515 takes the eye-position camera information from514 and generates control signals517 to move theVCSEL array513 to keep it positioned in front of the ocular implant as the gaze of the patient's eye changes, as shown inFIG. 6A,FIG. 6B andFIG. 6C.

In some embodiments,process502 includes a “fail-safe” function inoperation530 that immediately (or after a short predetermined amount of time) turns off all stimulation devices (including lasers or other optical sources) if and when communications are lost to the environment camera(s)510.

FIG. 5C is a schematic representation of asystem503 having an implanteddevice110, an externallyworn device111 and atransceiver71 ofcustomization console computer20 such as shown inFIG. 1C. Elements shown here that have identical numbers as elements inFIG. 1C are described above in the description ofFIG. 1C. Elements here having identical numbers as elements inFIG. 5B are described above in the description ofFIG. 5B. In some embodiments, a light-source-moving device575 (such as used in movable externallight source subsystem601 ofFIG. 6A or scannable-external-light-source subsystem701 ofFIG. 7A or scannable-external-light-source subsystem702 ofFIG. 7B or MEMS-array scannable-external-light-source subsystem703 ofFIG. 7C) is coupled tolight source system180 to move theoptical signal light596 as theeye90 moves, in order to keep the infrared stimulation aligned with the nerves that are to be stimulated. In some embodiments, eye-position-determining camera(s)514 obtain images of the eye, and the position of the eye is calculated by theimage processor182 which then controls the light-source-movingdevice575 such that the stimulation signal596 tracks to the correct positions on the nerves at the back of theeye90 at or near theretina96. In some embodiments, theprogramming computer20 communicates with the externally worndevice111 from time-to-time to recalibrate and/or improve theprograms590 stored therein. In some embodiments, the externally worndevice111 communicates back toprogramming computer20 to download device parameters and usage characteristics such that theprogramming computer20 can diagnose issues and appropriately reconfigure externally worndevice111 for best performance.

FIG. 6A,FIG. 6B andFIG. 6C are top views of an externallight source180 which in some embodiments of the present invention is a movable externallight source subsystem601, wherein the external light source moves to follow a patient's eye movements. In some embodiments, the externallight source601 subsystem emits IR-wavelength stimulation light130. The external light source subsystem includes alight emitter686 and, in some embodiments, acarrier688 upon which thelight emitter686 moves. For example, in some embodiments,light emitter688 includes a two-dimensional array of vertical-cavity surface-emitting lasers (VCSEL-array) that form an IR stimulator, which provides IR light130 into theeye90. In some embodiments, an eye tracking camera (for example,514 inFIG. 5B, but not shown here) senses the side to side direction to which the eye is pointed. Thelight emitter686 moves and is kept in substantially the same location relative to the front of the eye. That is, as the eye looks ahead, or side-to-side, the light emitter moves to stay in front of the eye.FIG. 6A shows the eye oriented to look toward the right, and thelight emitter686 is positioned at the right side of the carrier.FIG. 6B shows the eye oriented to look straight ahead, and thelight emitter686 is at the middle of thecarrier688.FIG. 6C shows the eye oriented to look toward the left, and thelight emitter686 is at the left side of thecarrier688. In some embodiments, thecarrier688 is a curved track in which the light emitter moves, and in some embodiments, the light emitter is moved along the track by anactuator681. In other embodiments, the carrier is a set of mechanical linkages the move the light emitter in such a way that it stays located in front of the eye, and substantially perpendicular to the long axis of theocular implant200. In some embodiments,carrier688 is configured such thatlight emitter686 can be moved in a left-right x-direction, an up-down y-direction, and a forward-backward (e.g., toward the eye and away from the eye) direction, and such that the angle in which light emitted fromlight emitter686 contacts the eye can be modified.

In some embodiments of the present invention, theimage processor182 further includes an actuator controller682. The actuator controller causes theactuator681 to move thelight emitter686 from side to side, keeping the light emitter in front of theocular implant200 such that image projected by light emitter stays in substantially the same position on the retina of the patient's eye as the eye looks from side to side.

In some embodiments, theimage processor182 of the nerve stimulation system controls thelight emitter686 such that the nerve stimulation light signals emitted correspond with the external scene in the direction the eye is pointing. This allows a patient to look in different directions without the need to actually rotate one's head. A nerve stimulation system with capability to allow a patient to see side-to-side simply by changing the direction in which one's eyes are pointing better simulates natural vision and provides an improved experience for the patient.

In some embodiments, the externallight source180 further includes anoptical element687 between thelight emitter686 and theeye90. The optical element corrects for distortions in the IR light130 from thelight emitter686 for further improve the accuracy of the nerve stimulation in the eye. In some embodiments, the optical element moves with thelight emitter686 along thecarrier688 to provide consistent optical correction in all directions of viewing.

FIG. 7A is a top cross-section view of a scannable-external-light-source subsystem701 emitting light toward a first position and toward a second position of a reflectingarray751 relative to the eye of a patient, according to some embodiments of the present invention. In some embodiments,system701 includes a laser unit711 (e.g., an optically pumped fiber-optic laser, VCSEL, or VCSEL array) that emits one ormore laser beams710 that are deflected in a first direction by angularly movable scanning minor712 (e.g., in some embodiments, the first direction being one of a plurality of angles in a plane perpendicular to the figure sheet (e.g., in some embodiments, this moves the beam in an X direction)), and that are then that are deflected in a second direction by angularly movable scanning mirror713 (e.g., in some embodiments, the second direction being one of a plurality of angles in a plane parallel to the figure sheet (e.g., in some embodiments, this moves the beam in an Y direction)). This type of scanning is generally called raster scanning. In some embodiments, the effective extent of the raster scanned light pattern during any one time frame is scanned across only a small portion of reflecting array751 (the field of vision that is being acquired in the position that the eye is in during that time frame). For example, when theeye90 is looking at theminor array717, thelight beam710 is only scanned across the area ofreflectors717; since no stimulation of action potentials will occur for light coming from other directions, this saves power. This eye tracking and moving of the raster pattern also allows the patient to move their eyes in different directions to control the image being acquired, andsystem111 will move the cameras toward the direction of gaze and generate the raster-scanned pulsed light pattern such that the pattern reaches the desired area of the retina. Similarly, if the patient's gaze is directed toward the area ofreflectors719, the system's mirrors will raster scan the pattern of IR nerve-stimulation light only towards the area ofreflectors719. In some embodiments, each area ofreflectors717,718,719 (and other areas of reflectingarray751 not shown here for clarity) has pixilated arrays of very small reflectors where each reflector is configured to reflect incoming light from scanningmirror713 toward a direction intoocular implant200. In some embodiments, a grating light valve (such as described

FIG. 7B is a top cross-section view of a scannable-external-light-source subsystem702 emitting light toward a first position and toward a second position of a holographic ordiffractive unit752 based on movement of the eye of a patient, according to some embodiments of the present invention.

FIG. 7C is a top cross-section view of a MEMS-array scannable-external-light-source subsystem703 emitting light toward a first position and toward a second position of aMEMS reflector unit753 based on movement of the eye of a patient, according to some embodiments of the present invention.

FIG. 7D is a top cross-section view of a MEMS-array scannable-external-light-source subsystem704 having a plurality of emitters (such as diode or fiber lasers), according to some embodiments of the present invention, which are used in a scanned projection system (e.g., in some embodiments, one, two, or threeIR lasers711 and one, two, or three visiblelight lasers711′ emit collimated light toward asystem751 of MEMS-deflection mirrors that create the spatial pattern of each wavelength that is projected into ocular implant200 (seeFIG. 2) and then onto the retinal nerves and/or retina of the patient). Other than the visiblelight lasers711′ and theirlight output710′, the rest of the elements ofsubsystem704 are as described for subsystem701 (above) and as shown inFIG. 7A.

FIG. 8 is a top cross-section view of an externallight source180, which in some embodiments of the present invention is a multiple-light-emitter subsystem801. In some embodiments, the multiple-light-emitter subsystem includes a plurality oflight emitters880. In some embodiments, each of the plurality oflight emitters880 includes a two-dimensional array of vertical-cavity surface-emitting lasers (VCSEL-array) that forms an IR stimulator, which provides IR light130 into theeye90. In some embodiments, the plurality of light emitters is arranged as a single horizontal row of emitters, wherein the row is located so that images projected from the emitters fall on the desired portion of the retina of the patient's eye.

In some embodiments, the present invention provides a vision system to aid in the treatment of a vision problem of an eye of a person, wherein the eye has a retina having at least one functional nerve layer, wherein an ocular implant is implanted in the eye of the person, and wherein at least part of the vision system is immediately in front of the eye of the person, the vision system including: at least one imager that obtains a temporal sequence of images and converts the sequence of images into digital-image signals; an image-processing unit that generates a series of laser-driving signals based on the digital-image signals; and a laser unit that receives the laser-driving signals and generates pulsed infrared-light (IR) nerve-stimulation laser signals, and directs the pulsed IR nerve-stimulation laser signals into the ocular implant in the eye of the person, wherein the ocular implant transmits the IR nerve-stimulation laser signals through an inner portion of the eye and onto the at least one functional nerve layer of the retina, and wherein the IR nerve-stimulation laser signals produce optical nerve action potentials directly in nerve cells other than rods or cones, in order to produce a vision sensation to the person. In some embodiments, the ocular implant is used to provide a transparent path for the nerve-stimulation signals that are at wavelengths that are substantially absorbed by the cornea, aqueous humor, lens and/or vitreous humor in the eye. In some other embodiments, the present invention omits the ocular implant and generates pulsed light nerve-stimulation laser signals at one or more wavelengths at which the cornea, aqueous humor, lens and vitreous humor are all relatively highly transparent but at which the nerve tissue at the back of the eye is sufficiently absorbent to be optically stimulated to trigger a nerve action potential (in some embodiments, the wavelength(s) are infrared (e.g., between about 700 nm and about 1400 nm), while in other embodiments, one or more visible or ultraviolet wavelengths are used to trigger nerve action potential signals in nerve cells in the eye that are between the rod cells and the optic nerve and/or to trigger nerve action potential signals in nerve cells in the eye that are between the cone cells and the optic nerve.

In some embodiments of the system, the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm, and wherein the at least one array of VCSELs are included in a wearable headpiece.

In some embodiments of the system, the at least one image-gathering unit obtains a temporal sequence of pairs of images of three-dimensional (3-D) scenes, wherein the image-processing unit generates a different series of laser-driving signals for each eye of a person, and wherein the at least one array of lasers generates different infrared-light (IR) nerve-stimulation signals directed into each of both eyes of the person (i.e., one set of IR nerve-stimulation signals that are projected into the user's left-hand eye trigger nerve signals that convey a left-hand-eye visual scene to the user's brain, while the other set of IR nerve-stimulation signals that are projected into the user's right-hand eye trigger nerve signals that convey a right-hand-eye visual scene to the user's brain), in order to produce a 3-D vision sensation to the person.

Some embodiments of the system further include an eye-position sensor, wherein the image-processing unit adjusts the generated series of laser-driving signals to adjust positions of the stimulation emitters based on the current position(s) of the person's eye(s) for each of a plurality of successive times and positions of the person's eye(s).

In some embodiments of the system, the image-processing unit preprocesses the sequence of images (i.e., the sequence of frames) to perform at least part of the image processing that normally occurs in the layers of the retina before image-processing unit generates the series of laser-driving signals based on the digital-image signals. In some such embodiments, the image-processing unit preprocesses the sequence of images to determine edges on objects and generates the series of laser-driving signals to stimulate edge-detecting nerves in the retina. In some embodiments, the image-processing unit preprocesses the sequence of images to determine areas of increased brightness and generates the series of laser-driving signals to stimulate increased-brightness-detecting nerves in the retina. In some embodiments, the image-processing unit preprocesses the sequence of images to determine areas of decreased brightness and generates the series of laser-driving signals to stimulate decreased-brightness-detecting nerves in the retina.

In some embodiments of the system, the image-processing unit generates a series of laser-driving signals based on motion of an object in a series of the digital-image frames received over a period of time and generates the series of laser-driving signals that stimulate the nerves of the retina that detect motion.

In some embodiments of the system, the image-processing unit that generates a series of laser-driving signals based on the digital-image signals received over a period of time and generates the series of laser-driving signals modified to ensure nerves in the retina are not overheated by IR laser signals.

Some embodiments of the system further include the ocular implant implanted in the eye of the person.

In some embodiments, the present invention includes a vision system to aid in the treatment of a vision problem of an eye of a person, where the eye has a retina having at least one functional nerve layer, an ocular implant is implanted in the eye of the person and at least part of the vision system is immediately in front of the eye of the person. The vision system includes at least one imager that obtains a temporal sequence of images of a scene and converts the sequence of images into digital-image signals, an image-processing unit that is operatively coupled to the imager, that processes the digital-image signals, and that generates a series of optical-source-driving signals based on the digital-image signals, and an optical source that receives the optical-source-driving signals and generates pulsed infrared-light (IR) nerve-stimulation optical signals, and directs the pulsed IR nerve-stimulation optical signals into the ocular implant in the eye of the person. In some embodiments, the ocular implant transmits the IR nerve-stimulation optical signals through an inner portion of the eye and onto the at least one functional nerve layer of the retina, and the IR nerve-stimulation optical signals produce optical nerve action potentials directly in nerve cells other than rods or cones, in order to produce a vision sensation to the person.

Some embodiments omit the ocular implant, and some such embodiments use one or more visible light wavelengths that are directed into the eye of the person, in order to stimulate working portions of the retina to generate nerve action potentials that are sent to the brain for a vision sensation. Such a system can be useful for restoring vision to a person who has had a torn retina reattached but the retina pieces are no longer located at correct places in the eye, in which case the remapping functionality of the present invention can map pieces of a received camera image to locations on the retina pieces in their reattached positions, and project the remapped image into the eye. For example, if a torn retina were reattached to the back of the eye with a gap between two edges, the portions of a normal image formed by the lens of the eye that landed on the gap would not be sensed or seen, but the present invention can split portions of the received camera image and remap to the locations of the retina and avoid losing image in the gap. These embodiments are useful for improving vision for other defects or injuries that would benefit from remapping a received image to other patterns such that the light-sensitive regions of the patients' eye(s) receive the various pieces of the image as remapped by the image processor of the present invention. In some such embodiments, no infrared signals are projected into the eye, but rather a visible image of remapped portions of the scene captured by video cameras of the present invention.

In some embodiments, the optical unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm. In some embodiments, the at least one array of VCSELs are included in a wearable headpiece.

In some embodiments, the at least one image-gathering unit vision system of the present invention obtains a temporal sequence of pairs of images of three-dimensional (3-D) scenes, the image-processing unit generates a different series of laser-driving signals for each eye of a person, and the at least one array of lasers generates different 3-D infrared-light (IR) nerve-stimulation signals directed into each of both eyes of the person, in order to produce a 3-D vision sensation to the person.

The vision system further includes, in some embodiment, an eye-position sensor used by the image-processing unit to adjust the generated series of optical-driving signals based on a current position of the person's eye.

In some embodiments, the image-processing unit preprocesses the sequence of images to perform at least part of the image processing that normally occurs in the layers of the retina before image-processing unit generates the series of optical-driving signals based on the digital-image signals. In other embodiments, the image-processing unit preprocesses the sequence of images to determine edges on objects and generates the series of optical-driving signals to stimulate edge-detecting nerves in the retina.

The present invention further includes the ocular implant implanted in the eye of the person, in some embodiments.

In some embodiments, the image-processing unit generates a series of laser-driving signals based on motion of an object in a series of the digital-image frames received over a period of time and generates the series of laser-driving signals that stimulate the nerves of the retina that detect motion.

In some embodiments, the image-processing unit generates a series of laser-driving signals based on the digital-image signals received over a period of time and generates the series of laser-driving signals modified to ensure nerves in the retina are not overheated by IR laser signals.

In some embodiments, the image-processing unit identifies an edge of a contiguous area in the scene based on a plurality of pixels of the area, and the optical-source-driving signals are pulsed signals configured to directly trigger nerve action potentials on an edge-detection nerve. In some embodiments, the image-processing unit identifies a line in the scene based on a plurality of pixels in the scene, and n the optical-source-driving signals are pulsed signals configured to directly trigger nerve action potentials on a line-detection nerve. In some embodiments, the image-processing unit identifies motion in the scene based on a plurality of pixels in a plurality of frames of the scene, and the optical-source-driving signals are pulsed signals configured to directly trigger nerve action potentials on a motion-detection nerve.

In other embodiments, the image processing unit identifies a first pattern in the scene based on a plurality of pixels in one or more frames in the scene, and based on the first pattern, generates optical-source-driving pulsed signals configured to directly trigger nerve action potentials on one or more excitatory nerves, and the image processing unit identifies a second pattern in the scene based on a plurality of pixels in one or more frames in the scene, and based on the second pattern, generates optical-source-driving pulsed signals configured to directly trigger nerve action potentials on one or more inhibitory nerves.

In some embodiments, the patterns in the scene are determined by a pattern-recognition algorithm such as described in U.S. Pat. No. 4,504,970 issued Mar. 12, 1985, titled “Training Controller for Pattern Processing System”; U.S. Pat. No. 4,541,115 issued Sep. 10, 1985, titled “Pattern Processing System”; U.S. Pat. No. 4,550,431 issued Oct. 29, 1985, titled “Address Sequencer for Pattern Processing System”; U.S. Pat. No. 4,551,850 issued Nov. 5, 1985, titled “Response Detector for Pattern Processing System”; U.S. Pat. No. 5,473,707 issued Dec. 5, 1995, titled “Pattern Processing System with Weighted Training Codes”; and/or U.S. Pat. No. 5,473,708 issued Dec. 5, 1995, titled “Pattern Processing System Using Minimum Length Address Loops”; each of which is incorporated herein by reference in its entirety. In some embodiments, the patterns determined from the scene are used to stimulate particular nerves or sets of nerves that would normally be triggered by visual stimulation by corresponding patterns if the eyes were functioning normally.

In some embodiments of the present invention, the image-processing unit is configured to transmit a sequential set of optical signals onto a patient's retina through the ocular implant, determine the patient's perceived visual response to the optical signals, and empirically produce a retinal map by correlating the perceived responses to the sequential set of optical-signal stimuli.

In some embodiments of the present invention, the image-processing unit is configured to transmit a sequential set of optical signals onto a patient's retina through the ocular implant, determine the patient's nerve-action-potential responses to the optical signals, and empirically produce a retinal map by correlating the perceived responses to the sequential set of optical-signal stimuli. In some such embodiments, the nerve-action-potential responses are detected optically, such as by the systems and methods described in U.S. Patent Publication No. 2010/0016732 (U.S. patent application Ser. No. 12/505,462) filed on Jul. 17, 2009, titled “Method and Apparatus for Neural-Signal Capture to Drive Neuroprostheses or Control Bodily Function,” which is incorporated herein by reference.

In some embodiments, the image-processing unit is configured to transmit a sequential set of optical signals onto a patient's retina through the ocular implant, determine the patient's perceived visual response to the optical signal, and empirically produce a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of a line by the patient. In some embodiments, the image-processing unit is further configured to transmit a sequential set of optical signals onto a patient's retina through the ocular implant, determine the patient's perceived visual response to the optical signal, and empirically produce a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of an edge by the patient. In some embodiments, the image-processing unit is further configured to transmit a sequential set of optical signals onto a patient's retina through the ocular implant, determine the patient's perceived visual response to the optical signal, and empirically produce a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of motion by the patient In some embodiments, the present invention is a method to aid in the treatment of a vision problem of an eye of a person, where the eye has an anteroposterior axis extending from the eye's anterior surface to the eye's fovea. In some embodiments, the method includes forming an infrared (IR)-transparent optical path inside the person's eye, imaging a temporal sequence of images of a scene and converting the sequence of images into digital-image signals, generating a series of optical-source-driving signals based on the digital-image signals, receiving the optical-source-driving signals and generating pulsed infrared-light (IR) nerve-stimulation optical signals, and directing the pulsed IR nerve-stimulation optical signals into the ocular implant in the eye of the person, where the ocular implant transmits the IR nerve-stimulation optical signals through an inner portion of the eye and onto the at least one functional nerve layer of the retina, and the IR nerve-stimulation optical signals produce optical nerve action potentials directly in nerve cells other than rods or cones, in order to produce a vision sensation to the person.

In some embodiments, the method further includes outputting optical signals from a laser unit wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm and including the at least one array of VCSELs in a wearable headpiece.

In other embodiments, imaging the scene includes obtaining a temporal sequence of pairs of images of three-dimensional (3-D) scenes, where the image-processing unit generates a different series of optical-driving signals for each eye of a person and different 3-D infrared-light (IR) nerve-stimulation signals are directed into each of both eyes of the person, in order to produce a 3-D vision sensation to the person.

In some embodiments, further includes adjusting the generated series of laser-driving signals based on a current position of the person's eye, wherein the current position of the person's eye is determined by an eye-position sensor.

The present invention, in some embodiments, includes preprocessing the sequence of images to perform at least part of the image processing that normally occurs in the layers of the retina before image-processing unit generates the series of optical-driving signals based on the digital-image signals. In other embodiments, the method includes preprocessing the sequence of images to determine edges on objects and generating the series of optical-driving signals to stimulate edge-detecting nerves in the retina.

In some embodiments, the method includes implanting the ocular implant in the eye of the person.

In some embodiments, the present invention includes generating a series of optical-driving signals based on motion of an object in a series of the digital-image frames received over a period of time and generating the series of optical-driving signals that stimulate the nerves of the retina that detect motion.

In some embodiments, the method includes generating a series of optical-driving signals based on the digital-image signals received over a period of time and generating the series of optical-driving signals modified to ensure nerves in the retina are not overheated by IR laser signals.

In some embodiments, the present invention includes identifying an edge of a contiguous area in the scene based on a plurality of pixels of the area, and generating optical-source-driving pulsed signals configured to directly trigger nerve action potentials on an edge-detection nerve. In some embodiments, the method further includes identifying a line in the scene based on a plurality of pixels in the scene, and generating optical-source-driving pulsed signals configured to directly trigger nerve action potentials on a line-detection nerve. In some embodiments, the method further includes identifying motion in the scene based on a plurality of pixels in a plurality of frames of the scene, and generating optical-source-driving pulsed signals configured to directly trigger nerve action potentials on a motion-detection nerve.

In some embodiments, the present invention includes identifying a first pattern in the scene based on a plurality of pixels in one or more frames in the scene, generating optical-source-driving pulsed signals based on the first pattern and configured to directly trigger nerve action potentials on one or more excitatory nerves, identifying a second pattern in the scene based on a plurality of pixels in one or more frames in the scene, and generating optical-source-driving pulsed signals based on the second pattern and configured to directly trigger nerve action potentials on one or more inhibitory nerves.

In some embodiments, the method further includes transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the optical signal, and empirically producing a retinal map by correlating the perceived responses to the sequential set of optical-signal stimuli.

In some embodiments, the present invention includes transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the optical signal, and empirically producing a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of an edge by the patient. In some embodiments, the method includes transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the optical signal, and empirically producing a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of a line by the patient. In some embodiments, the method includes transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the optical signal, and empirically producing a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of motion by the patient.

In some embodiments, the present invention includes an apparatus to aid in the treatment of a vision problem of an eye of a person, where the eye has an anteroposterior axis extending from the eye's anterior surface to the eye's fovea. The apparatus includes a means for forming an optical path inside the person's eye, where the optical path is substantially transparent to at least some infrared wavelengths of light between about 1000 nm and about 2000 nm, where the optical path has a light-receiving anterior end closest to the eye's anterior surface and extends to a posterior end, where the posterior end is closer to the fovea than to the eye's anterior surface; and a means for shining the light.

In some embodiments, the means for forming the optical path further includes a means for transmitting to the posterior end more than half of infrared light having wavelengths between about 1800 and about 2000 nm that is incident on the anterior end.

In some embodiments, the means for forming the optical path includes poly(methyl methacrylate) (PMMA).

In some embodiments, the means for forming the optical path includes a means for forming a substantially cylindrical-shaped path of a transparent material from the anterior end to the posterior end, and wherein the posterior end of the transparent material has a diameter substantially equal to a diameter of the anterior end of the transparent material.

In some embodiments, the present invention provides an apparatus to aid in the treatment of a vision problem of an eye of a person, wherein the eye has an anteroposterior axis extending from the eye's anterior surface to the eye's fovea. The apparatus includes an ocular implant inside the person's eye; means for imaging a temporal sequence of images of a scene and for converting the sequence of images into digital-image signals; means for generating a series of optical-source-driving signals based on the digital-image signals; means for receiving the optical-source-driving signals and for generating pulsed infrared-light (IR) nerve-stimulation optical signals; and means for directing the pulsed IR nerve-stimulation optical signals into the ocular implant in the eye of the person, wherein the ocular implant transmits the IR nerve-stimulation optical signals through an inner portion of the eye and onto the at least one functional nerve layer of the retina, and wherein the IR nerve-stimulation optical signals produce optical nerve action potentials directly in nerve cells other than rods or cones, in order to produce a vision sensation to the person. In some embodiments, the means for imaging of the scene includes: means for obtaining a temporal sequence of pairs of images of three-dimensional (3-D) scenes; means for generating a different series of optical-driving signals for each eye of a person; and means for generating different 3-D infrared-light (IR) nerve-stimulation signals directed into each of both eyes of the person, in order to produce a 3-D vision sensation to the person.

Some embodiments further include means for adjusting the generated series of laser-driving signals based on a current position of the person's eye; and means for in the current position of the person's eye is determined by an eye-position sensor.

Some embodiments further include means for preprocessing the sequence of images to perform at least part of the image processing that normally occurs in the layers of the retina before generating the series of optical-driving signals based on the digital-image signals.

Some embodiments further include means for preprocessing the sequence of images to determine edges on objects and generating the series of optical-driving signals to stimulate edge-detecting nerves in the retina.

Some embodiments further include the ocular implant implanted in the eye of the person.

Some embodiments further include means for generating a series of optical-driving signals based on motion of an object in a series of the digital-image frames received over a period of time; and means for generating the series of optical-driving signals that stimulate the nerves of the retina that detect motion.

Some embodiments further include means for generating a series of optical-driving signals based on the digital-image signals received over a period of time; and means for generating the series of optical-driving signals modified to ensure nerves in the retina are not overheated by IR laser signals.

Some embodiments further include means for identifying an edge of a contiguous area in the scene based on a plurality of pixels of the area, and wherein the optical-source-driving signals are pulsed signals configured to directly trigger nerve action potentials on an edge-detection nerve.

Some embodiments further include means for identifying a line in the scene based on a plurality of pixels in the scene, and wherein the optical-source-driving signals are pulsed signals configured to directly trigger nerve action potentials on a line-detection nerve.

Some embodiments further include means for identifying motion in the scene based on a plurality of pixels in a plurality of frames of the scene, and wherein the optical-source-driving signals are pulsed signals configured to directly trigger nerve action potentials on a motion-detection nerve.

Some embodiments further include means for identifying a first pattern in the scene based on a plurality of pixels in one or more frames in the scene; means for generating optical-source-driving pulsed signals based on the first pattern and configured to directly trigger nerve action potentials on one or more excitatory nerves; means for identifying a second pattern in the scene based on a plurality of pixels in one or more frames in the scene; and means for generating optical-source-driving pulsed signals based on the second pattern and configured to directly trigger nerve action potentials on one or more inhibitory nerves.

Some embodiments further include means for transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the optical signal, and empirically producing a retinal map by correlating the perceived responses to the sequential set of optical-signal stimuli.

Some embodiments further include means for transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the optical signal, and empirically producing a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of an edge by the patient.

Some embodiments further include means for transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the optical signal, and empirically producing a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of a line by the patient.

Some embodiments further include means for transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the optical signal, and empirically producing a set of temporal and spatial optical signal patterns that when transmitted through the ocular implant directly trigger nerve action potentials that result in the visual perception of motion by the patient.

Some embodiments further include means for outputting optical signals from a laser unit wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm, wherein the at least one array of VCSELs is in a wearable headpiece.

In some embodiments, the present invention includes a wearable system to aid in the treatment of a vision problem of a first eye of a person, wherein the first eye has a retina having at least one functional nerve layer, and wherein at least part of the system is immediately in front of the first eye of the person. This system includes at least one imager that obtains a temporal sequence of images of a scene and converts the sequence of images into digital-image signals; an eye-position detector that detects a direction of gaze of the first eye; an image-processing unit that is operatively coupled to the at least one imager, that processes the digital-image signals, and that generates a series of optical-source-driving signals based on the digital-image signals; an optical source that receives the optical-source-driving signals and generates optical signals; and an image-position moving unit that directs the optical signals into the first eye of the person, such that directions of the optical signals track changes in the direction of gaze of the first eye, in order to produce a vision sensation to the person.

In some embodiments of this wearable system, the optical source includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm, and wherein the at least one array of VCSELs are included in a wearable headpiece.

In some embodiments of this wearable system, the optical source includes at least one display screen, and the image-position moving unit moves the screen based on the detected direction of gaze of the first eye.

In some embodiments of this wearable system, the optical source includes at least one laser and a MEMS-based projection system that scans light from the least one laser to form a projected image, and wherein the image-position moving unit moves the image based on the detected direction of gaze of the first eye.

Some embodiments of this wearable system further include an ocular implant implanted in the first eye of the person, wherein the optical source includes at least one infrared (IR) laser that generates pulsed IR nerve-stimulation optical signals, and a MEMS-based projection system that scans light from the pulsed IR nerve-stimulation optical signals to form a projected image, and wherein the image-position moving unit moves the image based on the detected direction of gaze of the first eye.

In some embodiments of this wearable system, the image-processing unit has a calibration mode configured to transmit a sequential set of test-and-calibration optical signals onto a patient's retina through the ocular implant and to determine the patient's perceived visual response to the optical signal, and to empirically produce a retinal map by correlating the perceived responses to the sequential set of optical-signal stimuli, and wherein the image-processing unit has a vision mode configured to remap image information from the imager based on the retinal map.

Some embodiments of this wearable system further include an ocular implant implanted in the first eye of the person, wherein the optical source includes at least one infrared-wavelength light source and at least one visible-wavelength light source and is configured to form a projected image having interleaved infrared-wavelength pixels and visible-wavelength light pixels, and wherein the image-processing unit generates a series of optical-source-driving signals to control the at least one infrared-wavelength light source and the at least one visible-wavelength light source based on the digital-image signals.

In some embodiments, the present invention provides a method to aid in the treatment of a vision problem of a first eye of a person, wherein the first eye has an optical path extending along an anteroposterior axis extending from the first eye's anterior surface to the first eye's interior posterior surface. This method includes imaging a temporal sequence of images of a scene and converting the sequence of images into digital-image signals; generating a first series of optical-source-driving signals based on the digital-image signals; detecting a direction of gaze of the first eye and generating a first gaze-direction indication; receiving the first optical-source-driving signals and generating optical signals to form a first image; and moving the first image and directing the optical signals into the optical path in the first eye of the person in a direction based in the first gaze-direction indication, wherein the optical path transmits the optical signals through an inner portion of the first eye and onto the at least one functional nerve layer of the retina, and wherein the optical signals produce optical nerve action potentials in order to produce a vision sensation to the person.

In some embodiments of the method, the receiving of the optical-source-driving signals and the generating of the optical signals further includes outputting pulsed infrared-light (IR) nerve-stimulation optical signals from a laser unit, wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm; and including the at least one array of IR VCSELs in a wearable headpiece.

Some embodiments of this method further include detecting a direction of gaze of a second eye of the person and generating a second gaze-direction indication; generating a second series of optical-source-driving signals based on the digital-image signals; detecting a direction of gaze of the second eye and generating a second gaze-direction indication; receiving the second optical-source-driving signals and generating optical signals to form a second image; and moving the second image and directing the optical signals into the optical path in the second eye of the person in a direction based in the second gaze-direction indication, wherein the optical path transmits the optical signals through an inner portion of the second eye and onto the at least one functional nerve layer of the retina, and wherein the optical signals produce optical nerve action potentials in order to produce a vision sensation to the person.

In some embodiments of the method, the generating of the optical signals further includes adjusting the generated series of optical-source-driving signals based on a current position of the person's eye as indicated by the first gaze-direction indication.

In some embodiments of the method, the generating of the optical signals to form the first image further includes generating laser light and using a MEMS-based projection system to scan light from the laser light to form a first projected image, and wherein the moving of the first image moves the first projected image based on the first gaze-direction indication.

In some embodiments of the method, the generating of the optical signals further includes outputting pulsed infrared-light (IR) nerve-stimulation optical signals from a laser unit, wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm; and the method further includes implanting an ocular implant in the eye of the person, wherein the ocular implant is more transparent to IR wavelengths between about 1800 and about 1900 nm than is aqueous humor in the eye and than is vitreous humor in the eye.

Some embodiments further include transmitting a sequential set of optical signals onto a patient's retina through the ocular implant, determining the patient's perceived visual response to the sequential set of optical signals, and empirically producing a retinal map of retinal sensitivity by correlating the perceived responses to the sequential set of optical-signal stimuli.

In some embodiments, the present invention provides an apparatus to aid in the treatment of a vision problem of a first eye of a person, wherein the first eye has an optical path extending along an anteroposterior axis extending from the eye's anterior surface to the first eye's fovea and retina. This apparatus includes means for imaging a temporal sequence of images of a scene and for converting the sequence of images into digital-image signals; means for generating a series of optical-source-driving signals based on the digital-image signals; means for detecting a direction of gaze of the first eye and generating a first gaze-direction indication; means for receiving the optical-source-driving signals and for generating optical signals to form an image; and means for moving the image and directing the optical signals into the optical path the eye of the person in a direction based in the gaze-direction indication, wherein the optical path transmits the optical signals through an inner portion of the eye and onto the at least one functional nerve layer of the retina, and wherein the optical signals produce optical nerve action potentials in order to produce a vision sensation to the person.

In some embodiments, the means for receiving the optical-source-driving signals and the means for generating the optical signals further include means for outputting pulsed infrared-light (IR) nerve-stimulation optical signals from a laser unit, wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm; and means for including the at least one array of IR VCSELs in a wearable headpiece.

In some embodiments, the means for generating the optical signals further includes means for forming a visible-wavelength image on each of two display screens, wherein the means for directing of the optical signals into the optical path in the eye of the person further includes means for moving each display screen to remain oriented along and substantially perpendicular to the anteroposterior axis of each respective eye, and the apparatus further includes means for adjusting the generated series of laser-driving signals based on a current position of each of the person's eyes, wherein the means for detecting the direction of gaze of the eye includes means for sensing a position of the person's eye.

In some embodiments, the means for receiving the optical-source-driving signals and for generating optical signals to form the first image further include means for generating laser light and MEMS-based means for scanning the laser light from the means for generating laser light to form the first image, and wherein the means for moving moves the first image based on the first gaze-direction indication.

Some embodiments further include means for transmitting a sequential set of optical signals onto the person's retina through the ocular implant, means for determining the patient's perceived visual response to the sequential set of optical signals, and means for empirically producing a retinal map of retinal sensitivity by correlating the perceived responses to the sequential set of optical-signal stimuli.

Some embodiments further include an ocular implant implanted in the eye of the person, wherein the ocular implant is more transparent to IR wavelengths between about 1800 and about 1900 nm than is aqueous humor in the eye and than is vitreous humor in the eye.

Some embodiments further include means for outputting optical signals from a laser unit wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs) having output wavelengths between about 1800 and about 1900 nm, wherein the at least one array of VCSELs is in a wearable headpiece.

As discussed in par. 55 of Ser. No. 11/257,793, in some embodiments, a laser diode emitting light with a 1.87-micron wavelength stimulates nerves. This wavelength is important because devices capable of generating this wavelength are more available than longer mid-IR wavelengths. In some embodiments, laser-diode light of a 2.1-micron wavelength is used for nerve stimulation. Laser diodes that emit 2.1-micron-wavelength light are currently in research and would most likely work as well as other wavelengths, since this wavelength, when generated by a lamp-pumped solid-state laser, has been shown to be effective in stimulating nerves. In some embodiments, a laser-diode device (having one or more emitters) outputs light that is used for nerve stimulation, wherein the light has a wavelength of between about 1.5 microns and about 6 microns; in various embodiments, for example, the wavelength is in the far infrared at about 1.5 microns, or about 1.51 microns, about 1.52 microns, about 1.53 microns, about 1.54 microns, about 1.55 microns, about 1.56 microns, about 1.57 microns, about 1.58 microns, about 1.59 microns, about 1.6 microns, about 1.61 microns, about 1.62 microns, about 1.63 microns, about 1.64 microns, about 1.65 microns, about 1.66 microns, about 1.67 microns, about 1.68 microns, about 1.69 microns, about 1.7 microns, about 1.71 microns, about 1.72 microns, about 1.73 microns, about 1.74 microns, about 1.75 microns, about 1.76 microns, about 1.77 microns, about 1.78 microns, about 1.79 microns, about 1.8 microns, about 1.81 microns, about 1.82 microns, about 1.83 microns, about 1.84 microns, about 1.85 microns, about 1.86 microns, about 1.87 microns, about 1.88 microns, about 1.89 microns, about 1.9 microns, about 1.91 microns, about 1.92 microns, about 1.93 microns, about 1.94 microns, about 1.95 microns, about 1.96 microns, about 1.97 microns, about 1.98 microns, about 1.99 microns, about 2.0 microns, about 2.01 microns, about 2.02 microns, about 2.03 microns, about 2.04 microns, about 2.05 microns, about 2.06 microns, about 2.07 microns, about 2.08 microns, about 2.09 microns, about 2.1 microns, about 2.11 microns, about 2.12 microns, about 2.13 microns, about 2.14 microns, about 2.15 microns, about 2.16 microns, about 2.17 microns, about 2.18 microns, about 2.19 microns, about 2.2 microns, about 2.21 microns, about 2.22 microns, about 2.23 microns, about 2.24 microns, about 2.25 microns, about 2.26 microns, about 2.27 microns, about 2.28 microns, about 2.29 microns, about 2.3 microns, about 2.31 microns, about 2.32 microns, about 2.33 microns, about 2.34 microns, about 2.35 microns, about 2.36 microns, about 2.37 microns, about 2.38 microns, about 2.39 microns, about 2.4 microns, about 2.5 microns, about 2.6 microns, about 2.7 microns, about 2.8 microns, about 2.9 microns, about 3 microns, about 3.1 microns, about 3.2 microns, about 3.3 microns, about 3.4 microns, about 3.5 microns, about 3.6 microns, about 3.7 microns, about 3.8 microns, about 3.9 microns, about 4 microns, about 4.1 microns, about 4.2 microns, about 4.3 microns, about 4.4 microns, about 4.5 microns, about 4.6 microns, about 4.7 microns, about 4.8 microns, about 4.9 microns, about 5 microns, about 5.1 microns, about 5.2 microns, about 5.3 microns, about 5.4 microns, about 5.5 microns, about 5.6 microns, about 5.7 microns, about 5.8 microns, about 5.9 microns, or about 6.0 microns, or, in other embodiments, in ranges between any two of the above values. In other embodiments, an LED having output wavelengths centered in one of these ranges is used as a source of light to stimulate nerves.

As discussed in par. 99 of Ser. No. 12/191,301, in some embodiments, the present invention provides a VCSEL array configured to output light pulses capable of optically stimulating neural tissue (e.g., cochlear nerve tissue, deep brain tissue, white brain matter tissue, gray brain matter tissue, spinal cord tissue, cardial nerve tissue, central nervous system nerve tissue, olfactory nerve tissue, optic nerve tissue, nerve bundles and the like). In some embodiments, the stimulating lights pulses have a wavelength that results in the appropriate penetration depth for effective stimulation of the tissue of interest without causing tissue damage (e.g., in some embodiments, the wavelength of stimulating light pulses is in the range of about 1.8 microns to about 2.2 microns, in some embodiments, the wavelength of stimulating light pulses is in the range of about 1.85 microns to about 2.0 microns, in some embodiments, the wavelength of stimulating light pulses is about 1.87 microns, in some other embodiments the wavelength of stimulating light pulses is in the range of about 4.0 microns to about 5.0 microns, in some other embodiments the wavelength of stimulating light pulses is in the range of about 4.2 microns to about 4.8 microns, in some other embodiments the wavelength of stimulating light pulses is in the range of about 4.4 microns to about 4.6 microns).

As discussed in U.S. patent application Ser. No. 13/204,610, in some embodiments, one or both ends of the image pipe are shaped to focus the externally generated stimulator-array signals on the retina and fovea. In some embodiments, the present invention includes an external two-dimensional array VCSEL-array IR stimulator providing IR light into the anterior end of the ocular implant. In other embodiments of any of the embodiments of the present invention, other IR light sources are used, such as LED array emitters, or one or more single IR light sources that project light to an array modulator such as one or more grating light valves (for example, as described in U.S. Pat. No. 7,177,081 titled “High Contrast Grating Light Valve Type Device,”) and/or one or more digital light projector devices (such as described in U.S. Pat. No. 4,566,935 issued to Hornbeck on Jan. 28, 1986, titled “Spatial light modulator and method,” or U.S. Pat. No. 7,776,631 titled “MEMS Device and Method of Forming a MEMS Device,”), wherein the array light modulator provides a modulated nerve-stimulation signal to each of a plurality of locations on the patient's retina via the ocular implant. In yet other embodiments, any other suitable sources of IR stimulation light are used, including light sources emitting from a plurality of heights from their substrates (such as LED arrays or MEMS minors configured to focus at a plurality of depths in the nerves of the retina).

Although the various figures and embodiments described herein explicitly show some combinations of the individual features of the present invention in combination, other embodiments include any subset or all of the features in combination, and it is sepecifically contemplated that features described in one embodiment herein ma be added to other embodiments, and that not all features of any one described embodiments are required for other variations of that embodiment.

It is to be understood that the above description is intended to be illustrative, and not restrictive. Although numerous characteristics and advantages of various embodiments as described herein have been set forth in the foregoing description, together with details of the structure and function of various embodiments, many other embodiments and changes to details will be apparent to those of skill in the art upon reviewing the above description. The scope of the invention should be, therefore, determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein,” respectively. Moreover, the terms “first,” “second,” and “third,” etc., are used merely as labels, and are not intended to impose numerical requirements on their objects.

Claims (25)

What is claimed is:

1. A wearable system to aid in treating a vision problem of a first eye of a person, wherein the first eye has a retina having at least one functional nerve layer, and wherein at least part of the system is immediately in front of the first eye of the person, the system comprising:

at least one imager that obtains a temporal sequence of images of a scene and converts the sequence of images into digital-image signals, the at least one imager having a field of view facing away from the person;

an eye-position detector that detects a direction of gaze of the first eye;

a field-of-view controller configured to direct the field of view of the at least one imager in response to the direction of gaze of the first eye detected by the eye-position detector;

an image-processing unit that is operatively coupled to the at least one imager, wherein the image-processing unit processes the digital-image signals and generates a series of optical-source-driving signals based on the digital-image signals;

an optical source configured to receive the optical-source-driving signals and to generate and emit optical signals, wherein the optical signals have a wavelength between 1800 nm and 1900 nm and a pulse width between 1 microsecond and 8 milliseconds;

an ocular implant configured to be implanted at least in part within the first eye of the person, wherein the ocular implant has a light-receiving anterior end closest to the first eye's anterior surface, and extends to a posterior end that is closer to the first eye's fovea than to the first eye's anterior surface; and

an image-position moving unit configured to direct the optical signals into the ocular implant, such that directions of the optical signals follow changes in the direction of gaze of the first eye, wherein the ocular implant is configured to transmit the optical signals through an inner portion of the first eye and onto an at least one functional nerve layer of the first eye's retina, and wherein the optical signals produce nerve action potentials in order to produce a vision sensation to the person.

2. The system ofclaim 1, wherein the optical source includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs), and wherein the at least one array of VCSELs are included in a wearable headpiece.

3. The system ofclaim 1, wherein the optical source includes at least one display screen, and the image-position moving unit moves the screen based on the detected direction of gaze of the first eye.

4. The system ofclaim 1, wherein the optical source includes at least one laser and a MEMS-based projection system that scans light from the least one laser to form a projected image, and wherein the image-position moving unit moves the image based on the detected direction of gaze of the first eye.

5. The system ofclaim 1, wherein the optical source includes at least one infrared (IR) laser that generates the optical signals, and a MEMS-based projection system that scans light from the optical signals to form a projected image, and wherein the image-position moving unit moves the image based on the detected direction of gaze of the first eye.

6. The system ofclaim 1,

wherein the image-processing unit has a calibration mode configured to transmit a sequential set of test-and-calibration optical signals onto a patient's retina through the ocular implant, and to determine the patient's perceived visual response to the sequential set of optical signals, and to empirically produce a retinal map by correlating the perceived responses to the sequential set of optical signals, and wherein the image-processing unit has a vision mode configured to remap image information from the imager based on the retinal map.

7. The system ofclaim 1,

wherein the optical source includes at least one infrared-wavelength light source and at least one visible-wavelength light source and is configured to form a projected image having interleaved infrared-wavelength pixels and visible-wavelength light pixels, and wherein the image-processing unit generates a series of optical-source-driving signals to control the at least one infrared-wavelength light source and the at least one visible-wavelength light source based on the digital-image signals.

8. The system ofclaim 1, wherein the field-of-view controller generates control signals configured to cause the at least one imager to move such that the at least one imager points in the direction of gaze of the first eye.

9. The system ofclaim 1, wherein the at least one imager includes a wide-angle lens camera, and wherein the field-of-view controller selects an image signal of the field of view from the wide-angle lens camera based on the direction of gaze of the first eye.

10. The system ofclaim 1, wherein the at least one imager includes a plurality of cameras pointed toward different directions in an environment surrounding the person, and wherein the field-of-view controller selects an image signal of the field of view from the plurality of cameras based on the direction of gaze of the first eye.

11. The system ofclaim 1, wherein the system has a plurality of different modes of operation that are selectable to enhance text recognition in a first mode and face recognition in a second mode.

12. The system ofclaim 1, wherein the posterior end of the ocular implant is within 2 mm of the first eye's retina.

13. A method to aid in treating a vision problem of a first eye of a person, wherein the first eye has an optical path extending along an anteroposterior axis extending from the first eye's anterior surface to the first eye's interior posterior surface, the method comprising:

providing the system ofclaim 1;

imaging a temporal sequence of images of a scene in the field of view and converting the sequence of images into digital-image signals;

generating a first series of optical-source-driving signals based on the digital-image signals;

detecting a direction of gaze of the first eye and generating a first gaze-direction indication, wherein the imaging includes directing the field of view for the scene in response to the first gaze-direction indication;

receiving the first optical-source-driving signals and generating and emitting the optical signals to form a first image;

implanting the ocular implant at least in part within the first eye of the person such that the ocular implant has a light-receiving anterior end closest to the first eye's anterior surface, and extends to a posterior end that is closer to the first eye's fovea than to the first eye's anterior surface:

and moving the first image and directing the optical signals into the implanted ocular implant in the first eye of the person in a direction based on the first gaze-direction indication, wherein the ocular implant transmits the optical signals through an inner portion of the first eye and onto an at least one functional nerve layer of the retina, and wherein the optical signals nerve action potentials in order to produce a vision sensation to the person.

14. The method ofclaim 13, the receiving of the optical-source-driving signals and the generating of the optical signals further comprising:

outputting the optical signals from a laser unit, wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs); and including the at least one array of IR VCSELs in a wearable headpiece.

15. The method ofclaim 13, further comprising:

detecting a direction of gaze of a second eye of the person and generating a second gaze-direction indication;

generating a second series of optical-source-driving signals based on the digital-image signals;

detecting a direction of gaze of the second eye and generating a second gaze-direction indication;

receiving the second optical-source-driving signals and generating optical signals to form a second image; and

moving the second image and directing the optical signals into the optical path in the second eye of the person in a direction based on the second gaze-direction indication, wherein the optical path transmits the optical signals through an inner portion of the second eye and onto the at least one functional nerve layer of the retina, and wherein the optical signals produce nerve action potentials in order to produce a vision sensation to the person.

16. The method ofclaim 13, wherein the generating of the optical signals further includes adjusting the generated series of optical-source-driving signals based on a current position of the person's eye as indicated by the first gaze-direction indication.

17. The method ofclaim 13, wherein the generating of the optical signals to form the first image further includes generating laser light and using a MEMS-based projection system to scan light from the laser light to form a first projected image, and wherein the moving of the first image moves the first projected image based on the first gaze-direction indication.

18. The method ofclaim 13, wherein the generating of the optical signals further includes outputting the optical signals from a laser unit, wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs); and wherein the ocular implant is more transparent to the wavelength of the optical signals than is aqueous humor in the first eye and than is vitreous humor in the first eye.

19. The method ofclaim 18, further comprising:

transmitting a sequential set of optical signals onto a patient's retina through the ocular implant,

determining the patient's perceived visual response to the sequential set of optical signals, and

empirically producing a retinal map of retinal sensitivity by correlating the perceived responses to the sequential set of optical signals.

20. An apparatus to aid in treating a vision problem of a first eye of a person, wherein the first eye has an optical path extending along an anteroposterior axis extending from the first eye's anterior surface to the first eye's fovea and retina, the apparatus comprising:

means for imaging a temporal sequence of images of a scene and for converting the sequence of images into digital-image signals, the means for imaging having a field of view facing away from the person;

means for generating a series of optical-source-driving signals based on the digital-image signals;

means for detecting a direction of gaze of the first eye and generating a first gaze-direction indication;

means for directing a field of view of the means for imaging in response to the first gaze-direction indication;

means for receiving the optical-source-driving signals and for generating and emitting optical signals to form an image, wherein the means for generating and emitting optical signals include an optical source configured to emit the optical signals at a wavelength between 1800 nm and 1900 nm and with a pulse width between 1 microsecond and 8 milliseconds;

an ocular implant configured to be implanted at least in part within the first eye of the person, wherein the ocular implant has a light-receiving anterior end closest to the first eye's anterior surface, and extends to a posterior end, wherein the posterior end is closer to the first eye's fovea than to the first eye's anterior surface; and

means for moving the image and directing the optical signals into the ocular implant in a direction based on the first gaze-direction indication, wherein the ocular implant is configured to transmit the optical signals through an inner portion of the first eye and onto an at least one functional nerve layer of the retina, and wherein the optical signals nerve action potentials in order to produce a vision sensation to the person.

21. The apparatus ofclaim 20, wherein the means for receiving the optical-source-driving signals and the means for generating the optical signals further include

means for outputting the optical signals from a laser unit, wherein the laser unit includes at least one array of infrared (IR) vertical-cavity surface-emitting lasers (VCSELs); and

means for including the at least one array of IR VCSELs in a wearable headpiece.

22. The apparatus ofclaim 20, wherein the means for generating the optical signals further includes means for forming a visible-wavelength image on each of two display screens, wherein the means for directing of the optical signals into the ocular implant in the eye of the person further includes means for moving each display screen to remain oriented along and substantially perpendicular to the anteroposterior axis of each respective eye, the apparatus further comprising:

means for adjusting the generated series of laser-driving signals based on a current position of each of the person's eyes, wherein the means for detecting the direction of gaze of the eye includes means for sensing a position of the person's eye.

23. The apparatus ofclaim 20, wherein the means for receiving the optical-source-driving signals and for generating optical signals to form the first image further include means for generating laser light and MEMS-based means for scanning the laser light from the means for generating laser light to form the first image, and wherein the means for moving moves the first image based on the first gaze-direction indication.

24. The apparatus ofclaim 20, further comprising:

means for transmitting a sequential set of optical signals onto the person's retina through the ocular implant;

means for determining the patient's perceived visual response to the sequential set of optical signals; and

means for empirically producing a retinal map of retinal sensitivity by correlating the perceived responses to the sequential set of optical signals.

25. The apparatus ofclaim 20, wherein the ocular implant is more transparent to the wavelength of the optical signals than is aqueous humor in the first eye and than is vitreous humor in the first eye.

Images