US20140240477A1 - Multi-spectral imaging system for shadow detection and attenuation - Google Patents

Abstract

Systems and methods for detecting and attenuating shadows in a visible light image are disclosed. In various embodiments, shadows on human skin may be detected and attenuated using multi-spectral imaging techniques. Multispectral image data that includes a living subject can be processed to detect live-subject portions of the multispectral image data. Shadows in the detected live-subject portions of the multispectral image data can be identified. The identified shadows in at least part of the multispectral image data can be attenuated.

Description

BACKGROUND OF THE INVENTION
• 1. Field of the Invention
• The present disclosure relates to systems and methods for shadow detection and attenuation. In particular, the disclosure relates to systems and methods for detecting and attenuating shadows on human skin or other living objects using multi-spectral imaging techniques.
• 2. Description of the Related Art
• Imaging systems enable users to take photographs of a variety of different objects and subjects in many different lighting conditions. However, when photographs are captured of objects in daylight or under bring indoor lights, shadows may be cast over the object to be imaged. For example, when a user takes photographs of another person outside in daylight, shadows may be cast on the person's face by intervening objects (e.g., by a tree or structure). These shadows may degrade image quality and obscure rich details of the person's skin that may otherwise be illuminated in the absence of shadows. Accordingly, improved systems and methods for detecting and attenuating shadows cast on objects are desirable.
SUMMARY
• In one implementation, a computer-implemented method for attenuating shadows in an image is disclosed. The method can comprise processing multispectral image data that includes a living subject to detect live-subject portions of the multispectral image data. Further, the method can include identifying shadows in the detected live-subject portions of the multispectral image data. The identified shadows can be attenuated in at least part of the multispectral image data.
• In another implementation, an imaging system for attenuating shadows in a visible image is disclosed. The system can include a live-subject verification module programmed to process multispectral image data that includes a living subject to detect live-subject portions of the multispectral image data. The system can also include a shadow identification module programmed to identify shadows in the detected live-subject portions of the multispectral image data. In addition, the system can include a shadow attenuation module programmed to attenuate the identified shadows in at least part of the multispectral image data.
• In yet another implementation, an imaging system is disclosed. The imaging system can include means for processing multispectral image data that includes a living subject to detect live-subject portions of the multispectral image data. The system can also include means for identifying shadows in the detected live-subject portions of the multispectral image data. Furthermore, the system can include means for attenuating the identified shadows in at least part of the multispectral image data.
• In another implementation, a non-transitory computer-readable medium is disclosed. The non-transitory computer-readable medium can have stored thereon code that when executed performs a method comprising processing multispectral image data that includes a living subject to detect live-subject portions of the multispectral image data. The method can include identifying shadows in the detected live-subject portions of the multispectral image data. Further, the method can include attenuating the identified shadows in at least part of the multispectral image data.
• Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Note that the relative dimensions of the following figures may not be drawn to scale.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1A is a schematic drawing of a user capturing an image of a live subject in daylight using a multispectral imaging system.
• FIG. 1B is a magnified view of the diagram shown inFIG. 1A that illustrates shadow and non-shadow regions in an image captured by the multispectral imaging system.
• FIG. 2 is a schematic diagram of a shadow attenuation system, according to one embodiment.
• FIG. 3 is a flowchart illustrating a method for attenuating shadows in a visible light image, according to one implementation.
• FIG. 4 is a flowchart illustrating a method for detecting live-subject portions of the visible and NIR images, according to some implementations.
• FIG. 5 is a flowchart illustrating a method for identifying shadows in detected live-subject portions, according to some implementations.
• FIG. 6 is a histogram of the measured intensity of non-shadow and shadow pixels.
• FIG. 7 is a flowchart illustrating a method for attenuating the identified shadows, according to some implementations.
• FIGS. 8A-1 through8E are example images at different stages of a shadow attenuation method, according to one implementation.
DETAILED DESCRIPTIONSystem Overview
• Implementations disclosed herein provide systems, methods, and apparatus for identifying and attenuating shadows cast on a live subject, such as on skin of a live human face or another portion of a human subject's body. In particular, the disclosed implementations can identify and attenuate shadows on live human skin using a multispectral imaging system. The multispectral imaging system may include separate visible light and near infrared (NIR) sensors, or a single sensor capable of capturing both visible and NIR images. In various implementations, the multispectral imaging system can be configured to capture both a visible light image and a NIR image of a live subject. For example, the multispectral imaging system can be configured to capture visible and NIR images of a human face during daylight conditions. As explained herein, during daylight operation, shadows may be cast over the human face, which can undesirably interfere with the quality of images taken of the face. For example, the shadows can cast a dark region over the face, which may conceal structural features of the face and/or rich details of the subject's skin. In addition to obscuring details of facial features, shadows cast on a person to be imaged can also obscure details of a live subject's skin in other parts of a person's body.
• In various implementations disclosed herein, shadows cast on portions of a live subject (e.g., shadows cast on a human face) can automatically be detected and attenuated. Such automatic detection and attenuation can advantageously improve image quality by removing the dark regions formed by the shadow and by enabling rich details of a person's skin or facial features to be imaged. Without being limited by theory, rich details of the subject's skin can be imaged even in shadow because of the unique reflective properties of human skin when illuminated by NIR light. Thus, the disclosed multispectral imaging techniques, which include imaging at visible and NIR wavelengths, can enable the automatic detection and attenuation of shadows on a live subject's skin, while preserving the natural look of human skin.
• For example, in some arrangements, a subject can be imaged using a multispectral imaging system configured to capture visible light having a wavelength in a range of about 400 nm to about 750 nm and to also capture NIR light having a wavelength in a range of about 750 nm to about 1100 nm. In implementations where a human face is imaged, automatic face detection techniques can be employed to detect the human face. Further, live-subject portions of the visible and NIR images can be detected. For example, when a human face is imaged, live-skin portions or pixels of the face may be identified by the systems and methods disclosed herein. The multispectral imaging system can detect the live-skin portions of a human subject based at least in part on the unique reflectance properties of human skin when illuminated by light at NIR wavelengths.
• Shadows that are cast over the skin can therefore be identified and attenuated. The resulting visible light image may be substantially free of the artifacts and other undesirable effects induced by shadows cast over the live-subject portions of the image. In some implementations, the methods described herein may be performed substantially automatically by the disclosed systems such that minimal or no user interaction is needed. Such automatic detection and attenuation of shadows cast on living subjects can allow users to capture images on various imaging systems and automatically detect and attenuate shadows in an efficient and simple manner.
• Furthermore, although some of the implementations disclosed herein relate to detecting and attenuating shadows cast on a human face or on live-skin portions of a human subject, it should be appreciated that the principles and details disclosed herein may also be applicable to other types of materials. For example, NIR light may also have unique response characteristics when a piece of vegetation, such as plant matter, is illuminated with NIR light and imaged with a NIR sensor. Indeed, although many of the disclosed implementations result from calibrated and/or theoretical optical responses of human skin when illuminated by various wavelengths of light (e.g., visible and NIR wavelengths), skilled artisans will appreciate that it may also be possible to calibrate and/or calculate the optical responses of other materials such as living vegetation, animal skin, etc. when the other materials are illuminated by various wavelengths of light.
• In the following description, specific details are given to provide a thorough understanding of the examples. However, it will be understood by one of ordinary skill in the art that the examples may be practiced without these specific details. For example, electrical components/devices may be shown in block diagrams in order not to obscure the examples in unnecessary detail. In other instances, such components, other structures and techniques may be shown in detail to further explain the examples.
• It is also noted that the examples may be described as a process, which is depicted as a flowchart, a flow diagram, a finite state diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel, or concurrently, and the process can be repeated. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a software function, its termination may correspond to a return of the function to the calling function or the main function, or a similar completion of a subroutine or like functionality.
• Those of skill in the art will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
• FIG. 1A is a schematic drawing of a user capturing an image of alive subject1 in daylight using ashadow attenuation system10. The illustratedshadow attenuation system10 includes a visible light sensor5 and a NIR sensor7. InFIG. 1A, thelive subject1 is standing near atree2. Light emitted from the sun3 (or other light source) may be partially obscured by thetree2, which can cast a shadow over portions of thelive subject1 as shown in more detail inFIG. 1B.
• FIG. 1B is a magnified view of the diagram shown inFIG. 1A that shows more details of thelive subject1. Because thetree2 is positioned between the sun3 and thelive subject1, ashadow region14 is cast over a portion of the imaged live subject1. Light that is not obscured by thetree2 may be imaged in a non-shadowed, full illumination,region12. The portions of the subject1 captured in theshadow region14 would appear darker in a captured image than the portions of the subject1 captured in thenon-shadow region12. In addition, rich details of the subject'sskin15 captured in theshadow region14 may be lost or degraded due to the shadow cast over theskin15 of thesubject1.
• Thenon-shadow region12 can include non-shadow, live-skin portions11 and non-shadow,non-live skin portions13. The non-shadow, live-skin portions11 can be one or more pixels of the image that correspond to the skin of thelive subject1 that are not obscured by the shadow. The non-shadow,non-live skin portions13 can be one or more pixels of the image that correspond to non-living portions of the live subject, such as clothes or hair, or of a non-living object in the image, that are not obscured by the shadow in the image.
• Similarly, theshadow region14 can include shadow, live-skin portions15 and shadow, non-live-skin portions17. The shadow,non-live skin portions17 can include one or more pixels that correspond to non-living portions in the image, such as the clothes or hair of thelive subject1, or other non-living or non-skin objects. The shadow, live-skin portions15 can include one or more pixels of the image that correspond to portions of the skin of thelive subject1 that are at least partially obscured by the cast shadow, such as portions that image the face or neck of thelive subject1. As described herein, the systems and methods described below attenuate the shadows imaged in the shadow, live-skin portions15 and to improve the quality of the skin captured in the image. The systems and methods disclosed herein can advantageously reduce the effects of the cast shadow in the image such that the image includes the rich details and maintains the natural look of the skin.
• Turning toFIG. 2, a schematic diagram of ashadow attenuation system10 is disclosed, such as theshadow attenuation system10 shown inFIG. 1A above. Theshadow attenuation system10 can include amultispectral imaging system16 in some arrangements. As explained herein, themultispectral imaging system16 can include a visible light sensor5 and a NIR sensor7. For example, the visible light sensor5 can be a CCD/CMOS capable of detecting visible light at least in the range between about 400 nm and about 700 nm. Themultispectral imaging system16 can further include a second sensor, such as a CCD/CMOS that is capable of detecting NIR light in the range between about 700 nm and about 1100 nm. In some implementations, the wavelength ranges for the visible and NIR sensors can overlap or can even be substantially the same. Skilled artisans would understand that other types of sensors are possible, and other wavelength ranges are possible. In some implementations, imaging filters, such as a NIR pass filter, can be used on a suitable CCD/CMOS to detect only the NIR data. Skilled artisans would understand that various other sensors or combinations thereof can be used to capture visible and NIR image data. In some arrangements, a single sensor can be used to capture both visible and NIR wavelengths.
• In other implementations, themultispectral imaging system16 can be configured to include a single multispectral imaging sensor that can sense a broad band of wavelengths, including at least visible light wavelengths and near infrared (NIR) light wavelengths. The multispectral imaging sensor can be configured to detect light at wavelengths between about 400 nm and about 1100 nm (e.g., between about 400 nm and about 700 nm for visible light and between about 700 nm and about 1100 nm for NIR light, in various implementations). Of course, the imaging sensor can also be configured to detect a much broader range of wavelengths as well. In some implementations, a charge-coupled device (CCD) can be used as the multispectral imaging sensor. In other implementations, a CMOS imaging sensor can be used as the multispectral imaging sensor.
• Theshadow attenuation system10 can include amemory18 and aprocessor19. Thememory18 andprocessor19 are configured to electrically communicate with each other and with themultispectral imaging sensor16. Theshadow attenuation system10 also has a storage device4 that is capable of storing various software modules that can be executed by theprocessor19. In some implementations, theprocessor19 can receive and transmit data to and from themultispectral imaging system16, and can operate on that data by executing computer-implemented instructions stored in one or more software modules in the storage device4.
• The storage device4 can be any suitable computer-readable storage medium, such as a non-transitory storage medium. The storage device4 can comprise any number of software modules. For example, the storage device4 can include aface detection module20. Theface detection module20 can include software that can detect a human face in an image. In some implementations, theface detection module20 can use known techniques to detect and verify the geometry of a captured face in an image. In some implementations, theface detection module20 can be configured to detect the outline of a face, while in other implementations, theface detection module20 can detect the general region in which a face is located (e.g., a face located within a particular square or rectangular region). In one implementation, for example, the OKAO Vision Face Sensing Technology, manufactured by OMRON Corporation of Kyoto, Japan, can be used by theface detection module20. Other implementations of theface detection module20 are possible and thus embodiments are not limited to any particular method for detecting faces in an image.
• A live-subject verification module21 can also be stored in the storage device4. As will be described in more detail herein, the live-subject verification module21 can include computer-implemented instructions for identifying live-subject portions in an image. For example, the live-subject verification module21 can include instructions for identifying live human skin, e.g., live-skin portions, of an image that includes thelive subject1. As explained herein, the live-subject verification module21 can be programmed to calculate a binary skin map, which can identify pixels as live- or non-live-skin pixels or portions. Further, the live-subject verification module21 can be programmed to smooth the boundaries of the identified skin calculated in the binary skin map. In other embodiments, the live-subject verification module21 can be configured to identify other types of living subjects or objects, such as vegetation.
• Further, the storage device4 can include ashadow identification module22. Theshadow identification module22 can include computer-implemented instructions for identifying shadow region(s) in an image, such as theshadow region14 ofFIG. 1B. In some embodiments, theshadow identification module22 can be programmed to distinguish shadow regions from dark, non-shadow objects in the captured image. In conjunction with the live-subject verification module21, theshadow identification module22 can be programmed to identify and distinguish shadow, live-skin portions15 from shadow, non-live-skin portions17. For example, theshadow identification module22 can utilize skin's unique reflectance properties when illuminated with NIR and visible light to derive a shadow map of the captured image.
• The storage device4 can also comprise ashadow attenuation module23. Theshadow attenuation module23 can include computer-implemented instructions for attenuating the identified shadows in the image. For example, as explained in more detail herein, theshadow attenuation module23 can be programmed to generate a weight map indicating the amount of luminance shift for each pixel. In some implementations, the luminance value of each pixel can be adjusted to attenuate the shadows, e.g., to reduce the dark regions captured in theshadow regions14 of the image. In addition, in some arrangements, the luminance of the pixels can be adjusted in blocks of multiple pixels by theshadow attenuation module23.
• Acommunications module25 and apre-processing module26 can be stored on the storage device4. Thecommunications module25 can comprise computer-implemented instructions that manage the flow of data between the components of theshadow attenuation system10. Thepre-processing module26 can be configured to pre-process data, such as image data received from themultispectral imaging system16, before other operations are performed on the data. The storage device4 can optionally include auser interface module24. Theuser interface module24 can comprise instructions for implementing anoptional user interface6. For example, theuser interface6 can include a display and/or one or more buttons to actuate themultispectral imaging system16. In some arrangements, theuser interface6 can include features that allow the user to select a shadow attenuation mode, in which the methods disclosed herein may be used to detect and attenuate shadows cast on thelive subject1. Other user interface features, including a graphical user interface (GUI), can be controlled or implemented by instructions stored in theuser interface module24. Also, as shown inFIG. 2,other processing modules27 can be stored in the storage device4 as desired for implementing various other functionalities for thesystem10.
• Theshadow attenuation system10 can also include theoptional user interface6. Theuser interface6 can enable a user of thesystem10 to interact with thesystem10 and to effectively use the various modules to detect and attenuate shadows and/or to activate themultispectral imaging system16. For example, theuser interface6 can include one or more displays to display the captured image and/or other data. The display(s) can also be configured to display a graphical user interface (GUI) to further enhance the usability of thesystem10. In some implementations, theuser interface6 can include various peripheral devices, including, e.g., a keyboard, a mouse, a printer, and other input/output devices.
• Theshadow attenuation system10 can be implemented on a mobile device, including a mobile phone or smartphone, a tablet computer, a laptop computer, a digital camera, or the like. By integrating themultispectral imaging system16, thememory18, theprocessor19, the storage4, and theoptional user interface6 on a mobile device, theshadow attenuation system10 can advantageously be used without requiring the system to remain in a fixed location. In other implementations, however, theshadow attenuation system10 can comprise a desktop computer, server, computer workstation, or other type of computing device. Theshadow attenuation system10 can be integrated with the other computer hardware, or theshadow attenuation system10 can be separate from the computing device, for example as a separate camera or cameras.
Shadow Attenuation Overview
• FIG. 3 is a flowchart illustrating amethod30 for attenuating shadows in a visible light image. Themethod30 begins in ablock31, in which visible and NIR images of a living subject are captured. For instance, image data can be captured by themultispectral imaging system16 over a wavelength range between about 400 nm and about 1100 nm (e.g., between about 400 nm and about 700 nm for visible light and between about 700 nm and about 1100 nm for NIR light, in various implementations). In some arrangements, the visible and NIR images can be captured by separate visible light and NIR sensors. In some implementations, the visible light and NIR images can be initially roughly aligned because the visible light and NIR imaging sensors may be spaced closely together. Thus, a pixel in the visible light image can correspond to a pixel in the NIR image, such that pixels that are aligned between the visible and NIR images can be referred to as aligned pixel pairs. The NIR and visible images can be further aligned based on techniques disclosed in U.S. patent application Ser. No. 13/663,897, filed Oct. 30, 2012, entitled “MULTISPECTRAL IMAGING SYSTEM,” the contents of which are incorporated by reference herein in their entirety and for all purposes. In other arrangements, visible and NIR image data can be captured by a single sensor that can detect visible and NIR image data.
• Themethod30 moves to ablock40 to detect live-subject portions of the visible and NIR images. In the embodiments disclosed herein, the detected live-subject portions comprise human skin on a human face, e.g., live-skin portions or pixels. In addition, the detected live-subject portions can comprise human skin on other parts of the subject's body. In other embodiments, however, the detected live-subject portions can comprise other types of living subjects or objects, such as vegetation, etc.
• In some arrangements, a human face can be detected in the captured image. Any suitable method of face detection can be used to detect the face in the image. For example, in some arrangements, the imaged faces in the visible and NIR images may be roughly aligned because the visible light sensor5 and the NIR sensor7 can be spaced closely together and/or by using other alignment methods as explained above. As explained in more detail below, the alignment of faces in the visible and NIR images may allow the system to calculate color-to-NIR pixel ratios on a pixel-by-pixel basis. Further, theface detection module20 can detect details about the geometry of the captured face. Alternatively, theface detection module20 can detect the general region in which a face lies, such as within a particular box in an image. In addition, in various implementations, one or more regions-of-interest (ROI) may be defined on the face. For example, a weighted face mask can be generated based on the shape of the face and the location of the eyes and/or the mouth from theface detection module20.
• Live-skin portions of the imaged subject can be detected utilizing the difference of human skin's reflectance under visible and NIR illumination. For example, inblock40, a binary skin map can be calculated based on the difference between the pixel value of a NIR image pixel and the pixel value of the green channel of a corresponding visible image pixel in an aligned pixel pair (e.g., a pixel in the visible light image that is at roughly the same location as the pixel in the NIR image, such that the visible and NIR pixels are aligned). The binary skin map may also be based on the difference between the pixel values of the red and green channels of the visible image pixel. The boundaries of the detected skin can be smoothed in various ways, such as smoothing the skin boundaries using sigmoid functions, as explained below.
• Themethod30 then moves to ablock50, in which shadows are identified in the detected live-subject portions. For example, themethod30 can identify shadows in the detected live-skin portions on the face of a live subject. In some implementations, a global dark map is generated, in which shadow candidates are identified by analyzing portions of the visible and NIR images that are generally darker than other portions. Because dark portions of the image can correspond to shadow regions, or merely to dark objects in non-shadow regions, themethod30 can distinguish shadow, live-skin regions from other dark objects based on a ratio of the visible light intensity to the NIR light intensity for each pixel pair. The visible-to-NIR ratio can take advantage of the unique reflectance properties of human skin when illuminated by NIR light.
• A shadow map can be generated to identify live-skin pixels that are located within shadow regions of the image. Indeed, a shadow pixel can be differentiated from a non-shadow pixel in the live-skin portions based on the histogram of the shadow map. In various arrangements, a shadow edge can be calculated using edge detection algorithms, and the shadow boundary can be smoothed by finding the shadow penumbra and adjusting the pixel values based on the penumbra. In addition, shadow and non-shadow anchor pixels can be calculated. The shadow and non-shadow anchor pixels can represent regions of the image that originate from the same human skin. The anchor pixels can be calculated based on the intensity distribution of identified shadow and non-shadow pixels.
• Turning to ablock70, the identified shadows in the live-subject portions can be attenuated. A pixel-wise method can be used to adjust the luminance of each pixel in the visible light image to attenuate the shadows. By shifting the luminance of each pixel, the dark regions generated by the cast shadow can be removed. In some implementations, a block-wise method can be used to adjust the luminance of pixels in a pre-defined block. The pixel- and block-wise attenuations can be blended. The resulting visible light image with attenuated shadows can include the rich details of human skin and geometric features of the face or other live-subject portion of the image.
• Themethod30 moves to adecision block32 to determine whether additional multispectral images are to be captured. If a decision is made that additional images are to be captured, themethod30 returns to theblock31 to capture additional visible and NIR images of the living subject. If a decision is made that no additional images are to be captured, themethod30 ends.
Detection of Live-Subject Portions
• FIG. 4 is a flowchart illustrating amethod40 for detecting live-subject portions of the visible and NIR images, according to some implementations. Themethod40 begins in ablock41 to calculate a binary skin map. The difference of human skin reflectance under visible and NIR illumination can enable the detection of live-skin portions of a living subject. For example, a first normalized reflectance difference, r₁, can be calculated based on the pixel value of the green channel of a visible image pixel i and the pixel value of the corresponding pixel i in the NIR image:
• ${\mathrm{<figure-callout\; label="r"\; filenames="US20140240477A1-20140828-D00007.png"\; state="\{\{state\}\}">r</figure-callout>}}_1=\frac{{\rho }_i\left({\lambda }_{\mathrm{NIR}}\right)-{\rho }_i\left({\lambda }_g\right)}{{\rho }_i\left({\lambda }_{\mathrm{NIR}}\right)+{\rho }_i\left({\lambda }_g\right)},$
• where ρ represents the normalized intensity at pixel i for imaging channels NIR, green, red, blue, etc.
• A second normalized reflectance difference, r₂, can be calculated based on the pixel values of the red and green channels of the visible image pixel i:
• ${\mathrm{<figure-callout\; label="r"\; filenames="US20140240477A1-20140828-D00002.png"\; state="\{\{state\}\}">r</figure-callout>}}_2=\frac{{\rho }_i\left({\lambda }_g\right)-{\rho }_i\left({\lambda }_r\right)}{{\rho }_i\left({\lambda }_g\right)+{\rho }_i\left({\lambda }_r\right)}$
• Thresholds of the histograms of r₁and r₂can enable estimation of threshold values t_n1, t_n2, t_r1, and t_r2. For example, a particular pixel i can be identified as a live-skin pixel if:
• 
  t_n1<r₁<t_n2andt_r1<r₂<t_r2.
• The resulting binary skin map can be used to indicate which portions of the image are live-skin pixels. Additional details of various techniques for detecting live-subject portions, e.g., live-skin portions, can be found in U.S. patent application Ser. No. 13/533,706, filed Jun. 26, 2012, and entitled “SYSTEMS AND METHOD FOR FACIAL VERIFICATION,” the contents of which are incorporated by reference herein in their entirety and for all purposes.
• Themethod40 moves to ablock43, in which the boundary of the skin is smoothed. While the binary skin map calculated inblock41 can distinguish live- and non-live skin portions of the image, the resulting boundary may be choppy or otherwise unsmooth. To smooth the boundary between live- and non-live skin portions, a smoothed skin map, S, can be calculated by:
• 
  S=N(w₁.w₂.w₃.w₄),
• where N represents the min-max normalization function. The sigmoid functions w₁, w₂, w₃, and w₄can be calculated based on the differences between the normalized reflectance differences and their associated thresholds. For example, the sigmoid functions can be calculated by:
• ${\mathrm{<figure-callout\; label="w"\; filenames="US20140240477A1-20140828-D00007.png"\; state="\{\{state\}\}">w</figure-callout>}}_1=\frac{1}{1+{}^{a\left(r_1-t_{n2}\right)}};$${\mathrm{<figure-callout\; label="w"\; filenames="US20140240477A1-20140828-D00002.png"\; state="\{\{state\}\}">w</figure-callout>}}_2=\frac{1}{1+{}^{-a\left(r_1-t_{n1}\right)}};$$w_3=\frac{1}{1+{}^{a\left(r_2-t_{r2}\right)}};$$\mathrm{and}$$w_4=\frac{1}{1+{}^{-a\left(r_2-t_{r1}\right)}},$
• where a is a parameter that controls the rate of the sigmoid functions w.
• Themethod40 moves to adecision block45, in which a decision is made whether additional images are to be processed to identify live-subject portions. If a decision is made that there are additional images, themethod40 returns to theblock41 to calculate the binary skin map. If a decision is made that there are no additional images, themethod40 ends. It should be appreciated that, while themethod40 is based on pixel-wise differences, exact alignment of the visible and NIR images is not required. Indeed, facial skin is typically rather smooth and has a constant color response across the face. Thus, themethod40 can accurately identify live-skin portions of the image even when the visible and NIR images are roughly aligned, e.g., aligned based on a detected face. The identified live-skin portions can be used to detect shadows, as explained in more detail herein.
Identification of Shadows
• FIG. 5 is a flowchart illustrating amethod50 for identifying shadows in detected live-subject portions, according to some implementations. Themethod50 can begin in ablock51 to calculate a global dark map D to identify shadow candidate pixels. The dark map D identifies pixels in both the NIR and visible images that are dark, e.g. those pixels that have a low measured intensity value in both images. Because the dark map D identifies all pixels in the NIR and visible images that are dark, the dark map D can include pixels representing objects that are in shadow regions (e.g., objects that are dark in both images due to the cast shadow) as well as pixels that merely represent dark objects (e.g., black or other dark-colored objects). In some implementations, the global dark map D can be calculated as
• $D=D_{\mathrm{vis}}*D_{\mathrm{NIR}},\mathrm{where}$$D_{\mathrm{vis}}=1-\frac{\rho \left({\lambda }_r\right)+\rho \left({\lambda }_g\right)+\rho \left({\lambda }_b\right)}{3}$$\mathrm{and}$$D_{\mathrm{NIR}}=1-\rho \left({\lambda }_{\mathrm{NIR}}\right).$
• Thus, large values in the dark map D can represent shadow candidate pixels, e.g., pixels that are generally dark in both the visible and NIR images. However, the dark map D alone may not distinguish between dark shadow regions and dark objects (whether in or out of a shadow region).
• Themethod50 moves to ablock52 to calculate a shade image F based on skin reflectance in the NIR and visible images. The shade image F can help to distinguish shadow, live-skin portions from non-shadow, live-skin portions by outlining the shadow regions with non-shadow dark objects. The binary skin map can be used to filter the live-skin portions from the non-live skin portions. It should be appreciated that daylight (e.g., direct sunlight) emits more energy at visible wavelengths than at NIR wavelengths. However, the difference in illumination between visible and NIR light in the shade, e.g., areas that are only exposed under sky light rather than direct daylight, is generally negligible compared to the difference between the reflectance of skin illuminated by visible light and the reflectance of skin illuminated by NIR light. Thus, the ratio of visible light to NIR light can be used to distinguish shadow regions from non-shadow dark objects for regions identified as live-skin portions.
• For example, inblock52, the shade image F can be calculated based on a ratio of the color-to-NIR pixel values in pixel pairs of the NIR and visible images (e.g., pixels in the NIR image that correspond to pixels in the visible image). For each pixel pair in the NIR and visible images, a pixel ratio F_kcan be calculated for a channel k, where the channel k corresponds to the red, green or blue image data when a RGB visible light sensor is used. The pixel ratio F_kcan be defined as
• $F_k=\frac{\rho \left({\lambda }_k\right)}{\rho \left({\lambda }_{\mathrm{NIR}}\right)};k=\left\{r,g,b\right\}.$
• As mentioned above, daylight (e.g., direct sunlight) emits more energy in the visible band than in the NIR band, so it should be appreciated that non-shadow pixels may have a relatively larger F_kthan shadow pixels, for live-skin portions of the image. Thus,
• 
  F_k^shadow<F_k^non-shadow<1.
• For human skin, the shade image F can be calculated based on the pixel ratios F by
• $F=1-\frac{1}{t}\min \left(\underset{k}{\max }\left(F_k\right),t\right).$
• In the calculated shade image F, large pixel values in live-skin portions may represent true shadow pixels, e.g., pixels located within shadow regions on the identified skin. Thus, the pixel ratio F_kcan be used to generate the shade image F to outline actual shadow regions compared to dark objects. Skin's unique reflectance properties under visible and NIR illumination can enable the disclosed systems to detect shadows on live-skin portions of an image.
• Moving to ablock53, the global dark map D (representing shadow candidates) can be combined with the calculated shade image F to derive a shadow map M. The shadow map M may be calculated by
• 
  M=1−D.*F.
• In the calculated shadow map M, small pixel values in the live-skin portions have a high probability of being live-skin, shadow pixels. To classify a region as a shadow or non-shadow region in the visible light image, the calculated shadow map M can be binarised. In particular, the histogram of M can be computed for live-skin pixels. As above, the binary skin map can be used to select the pixels that correspond to live human skin and to exclude non-live-skin portions.
• Themethod50 moves to adecision block54 to compare the value of the shadow map M for a pixel p(i,j) with a threshold value θ. In the computed histogram of M, the threshold θ can correspond to the first valley of the histogram of M, which generally represents the pixel value below which there is a high probability that the pixel is within a shadow region. Thus, whether a particular pixel p(i,j) is a live-skin, shadow pixel can be given by
• $p\left(i,j\right)=\{\begin{array}{cc}\mathrm{shadow\_pixel}& \mathrm{if}(M\left(i,j\right)\le \theta \\ \mathrm{non}\text{-}\mathrm{shadow\_pixel}& \mathrm{otherwise}.\end{array}$
• If a decision is made inblock54 that the value of the dark map M(i,j) for a pixel p(i,j) is less than or equal to the threshold θ, then the method moves to ablock55 to identify the pixel p(i,j) as a shadow pixel within the live-skin portion of the image. If a decision is made that the value of the dark map M(i,j) for a pixel p(i,j) is not less than or equal to the threshold θ, themethod50 moves to ablock56 to identify the pixel p(i,j) as a non-shadow pixel. Upon identifying the pixel as a shadow or non-shadow pixel, themethod50 ends.
• In some arrangements, the boundary of the shadow can be estimated using the shadow map M by implementing a standard edge-detection algorithm. Furthermore, the shadow boundary can be smoothed by locating the penumbra region across the cast shadow boundary. By locating the penumbra region, the accuracy of the shadow attenuation method described below with respect toFIG. 7 can be improved. For example, pixels that are identified as edge or penumbra pixels can be used with a Markov Random Field (MRF) technique to smooth the shadow boundaries.
• Furthermore, to improve the shadow attenuation technique explained below with respect toFIG. 7, shadow and non-shadow anchor pixels can be selected. The intensity distribution of shadow pixels and non-shadow pixels on human skin in the same image may be highly correlated. Indeed, the shadow and non-shadow anchor pixels can represent pixels that are highly likely to have originated from the same human skin in the image. For example, in some arrangements, shadow and non-shadow anchor pixels can be calculated based on the histograms of shadow and non-shadow pixel luminance values.
• In particular, shadow and non-shadow pixels can be selected for pixels in both the shadow and non-shadow regions that have a probability above a pre-defined threshold.FIG. 6 is a histogram of the measured intensity of non-shadow and shadow pixels. In some implementations, pixels having a normalized intensity corresponding to a probability below 0.2 can be discarded, such that shadow and non-shadow anchor pixels can include only those pixels with probabilities greater than or equal to 0.2. The respective anchor pixels with intensities corresponding to probabilities greater than or equal to 0.2 can be used to calculate the luminance distribution explained below with respect toFIG. 7. The resulting anchor pixels for both shadow and non-shadow regions can represent regions that originate from the same human skin regions.
Attenuation of Shadows
• FIG. 7 is a flowchart illustrating amethod70 for attenuating the identified shadows, according to some implementations. In general, the shadows may be attenuated by shifting the luminance distribution of pixels in the detected shadow regions of the image towards that of the non-shadow regions, e.g., to make the shadow regions appear lighter in the attenuated image. In some embodiments the system preserves non-shadow and non-skin regions in the captured image while making the detected shadow region(s) appear lighter (e.g., attenuating the shadow), and while also preserving human skin texture in the shadow regions of the image. Further, embodiments preserve a natural visual perception of human skin, such that the human skin does not appear artificial or otherwise modified.
• To attenuate detected shadows, luminance histograms may be calculated for both shadow and non-shadow pixels. The cumulative distribution functions (CDF) may be calculated for the histograms of the shadow and non-shadow pixels, e.g., C_shadowand C_non-shadow, respectively. The pixels in shadow and non-shadow regions can be matched such that the luma components in the YCbCr space of each pixel i in shadow and non-shadow can be correlated. For example, in some implementations, the CDFs of the shadow and non-shadow pixels can be matched such that, for the luma component Y_iof each pixel i, a corresponding luma component Y_i′ can be identified such that
• 
  C_shadow(Y_i)=C_non-shadow(Y_i′),
• where Y_icorresponds to the luminance in the shadow regions and Y_i′ corresponds to the luminance in the non-shadow regions. Thus, the luminance shift can be estimated as
• 
  Δ=Y′−Y.
• Themethod70 begins in ablock71 to generate a weight map W of pixel luminance shifts. The weight map W can be used to overcome pixel-wise error induced by the live-skin detection method50 and/or the shadow detection method60, and can indicate the amount of luminance shift for each pixel. The weight map W can therefore help to preserve the natural look of human skin. In some implementations, the weight map W can be calculated by
• $W=N\left(\frac{1}{1+{}^{a\left(Y-pt_1\right)}}.*\frac{1}{1+{}^{-a\left(Y-pt_2\right)}}\right){.}^{*}S,\mathrm{where}$$p{\mathrm{<figure-callout\; label="t"\; filenames="US20140240477A1-20140828-D00007.png"\; state="\{\{state\}\}">t</figure-callout>}}_1={\mu }_1-{\mathrm{<figure-callout\; label="\sigma "\; filenames="US20140240477A1-20140828-D00007.png"\; state="\{\{state\}\}">\sigma </figure-callout>}}_1,\mathrm{and}$$p{\mathrm{<figure-callout\; label="t"\; filenames="US20140240477A1-20140828-D00002.png"\; state="\{\{state\}\}">t</figure-callout>}}_2={\mu }_1-3{\mathrm{<figure-callout\; label="\sigma "\; filenames="US20140240477A1-20140828-D00007.png"\; state="\{\{state\}\}">\sigma </figure-callout>}}_1.$
• μ₁and σ₁can indicate the mean and variance of the luminance value of shadow anchor pixels, in some arrangements. In the weight map W, larger weights may be assigned to skin shadow pixels, and smaller weights may be assigned to non-skin and/or non-shadow pixels.
• Themethod70 then moves to adecision block72, in which a decision is made regarding whether or not a pixel-wise attenuation technique will be used to attenuate the detected shadows. If a decision is made not to employ the pixel-wise technique, then themethod70 continues to adecision block74 to attenuate the shadows in a block-wise technique, as explained in more detail below. If a decision is made to use the pixel-wise technique, then themethod70 moves to ablock73 to adjust the luminance of each pixel on a pixel-by-pixel basis. In particular, the luminance of each pixel can be adjusted by
• 
  Y′=Y+W.*Δ.
• Thus, the detected shadows in the live-skin portions can be attenuated by adjusting the original luminance Y by a weighted amount Δ to lighten the shadow regions and improve image quality.
• In some cases, the shadow boundary may be abrupt or choppy. To soften and/or smooth the shadow boundary, and to improve the natural look of the imaged skin, a directional smoothing technique may be applied. For example, in some arrangements, the shadow penumbra map B can be defined as
• $B\left(i,j\right)=\{\begin{array}{ccc}1& \mathrm{if}& \nabla Y^{\prime }\left(i,j\right)<\mathrm{<figure-callout\; label="\tau "\; filenames="US20140240477A1-20140828-D00007.png"\; state="\{\{state\}\}">\tau </figure-callout>}\\ 0& \mathrm{otherwise}& \end{array}p\left(i,j\right)\subseteq \mathrm{penumbra}\bigcap \mathrm{binary\_skin}\mathrm{\_map},$
• where τ is a pre-defined threshold. As ∥∇Y′(i,j)∥ represents the magnitude of the gradient at pixel p(i,j) of luminance image Y′, it should be appreciated that the shadow penumbra map B can represent regions of the image where the image gradient is relatively small, e.g., where the shadow boundaries are relatively smooth. Further, the Y value can be smoothed across the actual penumbra region such that each pixel in the shadow penumbra map B can be smoothed locally in a direction tangent to the shadow edge. The texture of the shadow boundary can thereby be preserved.
• Themethod70 then moves to adecision block74 to determine whether a block-wise attenuation technique is to be performed on the detected shadows. If the answer is no, then themethod70 ends. If a decision is made that the block-wise attenuation techniques is to be performed, then themethod70 moves to ablock75 to adjust the luminance values of one or more blocks of pixels. The block-wise techniques ofblock75 can be employed in various situations, such as those in which the entire face or live-skin region is in a shadow region. Because large portions of the face may be obscured by shadow, the pixel-wise attenuation technique may not fully capture skin texture differences in shadow portions of the skin. To preserve the natural look of human skin, the block-wise approach may be used. For example, the image may be divided into one or more blocks, and a contrast limit function may be applied on the histogram of each block independently. The contrast limit function may be a fixed exponential function having a parameter to control the rate. Histograms of neighboring blocks may be averaged, and the luma of each pixel may be tone mapped using four neighboring histograms. A weight may be assigned to each block. For example, the assigned weight may be calculated from the weight map W, described above. The weight can be mapped to the rate of the contrast limit function. In some arrangements, the weights of all the pixels in a block can be averaged. The mapping function may be determined by
• $f\left(w\right)=\frac{e^w-1}{e-1}*a+b,$
• where a and b are pre-determined parameters (e.g., a=4, b=1 in some arrangements). The blocks having larger weights may have more shadow, such that more contrast enhancement can be applied. Similarly, blocks having smaller weights may have less shadow, such that less contrast enhancement can be applied.
• Themethod70 then moves to ablock76 to blend pixel- and block-wise attenuations in cases where both types of attenuation are employed. For example, alpha blending can be used to incorporate the advantages of both types of shadow attenuation techniques. In particular, the attenuated image Y′ can be calculated by
• 
  Y′=αY′_pixel-wise+(1−α)Y_block-wise′,
• where α varies from 0 to 1. The value of α may be determined by the user or adaptively by the system. In some arrangements α can be about 0.75 for cast shadow cases and about 0.25 for non-cast shadow cases. Further, the chroma components C_band C_rcan be adjusted by
• $C_b^{\prime }=W.*\frac{Y^{\prime }.}{Y}+\left(1-W\right).C_b,\mathrm{and}$$C_r^{\prime }=W.*\frac{Y^{\prime }.}{Y}+\left(1-W\right).C_r.$
• The attenuated image can therefore include attenuated luma and chrominance components. The resulting improvement in image quality can reduce the dark portions that obscure live-skin portions in captured images. Once the image has been attenuate, themethod70 ends.
Examples of Shadow Attenuation
• FIGS. 8A-1 through8E are example images at different stages of a shadow attenuation method, according to one implementation. For example,FIGS. 8A-1 and8A-2 are example NIR and visible light images, respectively, that were captured using a NIR sensor and a separate visible light sensor. Although the visible light image ofFIG. 8A-2 is illustrated as a black and white photograph in the drawings, it should be appreciated that the original image was captured as a color RGB image. As shown in bothFIGS. 8A-1 and8A-2, the live subject was imaged outside in daylight. The hat on the subject cast a substantial shadow over the subject's face such that details of the face are obscured by the cast shadow.
• FIG. 8B-1 illustrates the binary skin map that was calculated according to block41 ofFIG. 4, explained in detail above. The binary skin map can represent live- and non-live skin pixels of the subject. As shown inFIG. 8B-1, themethod40 accurately located the live-skin portions in the image, such as the subject's face and exposed arms.FIG. 8B-2 illustrates the skin map S that was calculated according to block43 ofFIG. 4. The disclosed smoothing functions were successful in smoothing the skin boundaries between live-skin and non-live skin pixels from the binary skin map.
• FIG. 8C-1 illustrates the dark map D that was generated according to block51 ofFIG. 1, explained above. As shown inFIG. 8C-1, large values in the dark map D can represent generally dark objects, corresponding to either shadow regions or merely dark objects. Thus, regions inFIG. 8C-1 that are lighter (representing higher pixel values) can correspond to darker regions in the original visible and NIR images. For example, the regions in the subject's face have high pixel values, which correspond to shadow regions. In addition, in the dark map D, dark objects such as the subject's hair and eyebrows also have high pixel values.
• Turning toFIG. 8C-2, the shade map F outlines shadow regions within the live-skin portions. For example, inFIG. 8C-2, large pixel values in the skin portions represent shadow, live-skin portions, while small pixel values (e.g., darker regions) represent either non-shadow portions or dark object portions of the image. For example, in the shade map F, the face (which is occluded by shadow) is at a higher pixel value, while the eyebrows, hair, and eyes are at lower pixel values. Thus, the shadow regions within the live-skin portions are generally outlined by non-shadow portions, such as the eyebrows, hair, eyes, etc. The shade map F is thereby able to distinguish shadow from non-shadow regions in the live-skin portions.
• InFIG. 8D, the shadow map M is shown for an example image. In the shadow map M, small values within the skin portions (e.g., darker regions) represent regions that have a high probability of being a shadow pixel. As shown inFIG. 8D, for example, the subject's lips, hair, eyebrows, and eyes have higher pixel values (corresponding to non-shadow, dark regions), while the skin on the subject's face, occluded by the subject's hat, have lower pixel values, which correspond to shadow regions on the subject's face.
• FIG. 8E illustrates an example image in which the shadows cast over the subject's face have been detected and substantially attenuated. In particular,FIG. 8E is a visible light image in which the shadow cast over the subject's face has been reduced. Facial features, such as lines that define the subject's nose, mouth, cheek, etc., can be seen, whereas inFIG. 8A-2, the same facial features are largely obscured by the dark, shadow region. As explained above with respect toFIG. 7, for example, the systems and methods disclosed herein can utilize the unique reflectance properties of human skin when illuminated by NIR light to detect and attenuate shadows on human skin. It should be appreciated that the image shown inFIG. 8E may be rendered as a black and white image; however, the original visible light image was captured using a RGB visible light sensor. The illustrated black and white image is presented only for convenience of illustration.
Clarifications Regarding Terminology
• Those having skill in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and process steps described in connection with the implementations disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. One skilled in the art will recognize that a portion, or a part, may comprise something less than, or equal to, a whole. For example, a portion of a collection of pixels may refer to a sub-collection of those pixels.
• The various illustrative logical blocks, modules, and circuits described in connection with the implementations disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
• The steps of a method or process described in connection with the implementations disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. Each module can include various sub-routines, procedures, definitional statements and macros. Each of the modules may be separately compiled and linked into a single executable program. The description of each of the modules herein is used for convenience to describe the functionality of the system. Processes that are undergone by each of the modules may be arbitrarily redistributed to one of the other modules, combined together in a single module, or made available in, for example, a shareable dynamic link library. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of non-transitory storage medium known in the art. An exemplary computer-readable storage medium is coupled to the processor such the processor can read information from, and write information to, the computer-readable storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal, camera, or other device. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal, camera, or other device.
• Headings are included herein for reference and to aid in locating various sections. These headings are not intended to limit the scope of the concepts described with respect thereto. Such concepts may have applicability throughout the entire specification.
• The previous description of the disclosed implementations is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these implementations will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other implementations without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the implementations shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (27)

What is claimed is:

1. A computer-implemented method for attenuating shadows in an image, comprising:

processing multispectral image data that includes a living subject to detect live-subject portions of the multispectral image data;

identifying shadows in the detected live-subject portions of the multispectral image data; and

attenuating the identified shadows in at least part of the multispectral image data.

2. The method ofclaim 1, wherein the multispectral image data comprises visible light image data and near infrared (NIR) image data.

3. The method ofclaim 2, wherein attenuating the identified shadows comprises attenuating the identified shadows in the visible light image data.

4. The method ofclaim 2, wherein the living subject comprises a human subject, and wherein the detected live-subject portions comprise live-skin portions.

5. The method ofclaim 4, wherein identifying the shadows comprises calculating a ratio of visible to NIR light for corresponding pixels in the visible and NIR images.

6. The method ofclaim 4, further comprising attenuating the identified shadows based at least in part on the detected live-skin portions and the identified shadows in the visible light image data and the NIR image data.

7. The method ofclaim 6, wherein attenuating the identified shadows comprises calculating a luminance distribution in the NIR and visible light image data.

8. The method ofclaim 4, further comprising detecting a human face in the visible light and NIR image data, and wherein the detected human face comprises at least some of the detected live-skin portions.

9. The method ofclaim 1, further comprising capturing the multispectral image data with a multispectral imaging system.

10. The method ofclaim 9, wherein capturing the multispectral image data comprises capturing visible light image data and near infrared (NIR) image data.

11. The method ofclaim 10, wherein capturing visible light image data and NIR image data comprises:

capturing a visible light image of the living subject with a visible light sensor; and

capturing a NIR image of the living subject with a NIR sensor.

12. An imaging system for attenuating shadows in a visible image, the system comprising:

a live-subject verification module programmed to process multispectral image data that includes a living subject to detect live-subject portions of the multispectral image data;

a shadow identification module programmed to identify shadows in the detected live-subject portions of the multispectral image data; and

a shadow attenuation module programmed to attenuate the identified shadows in at least part of the multispectral image data.

13. The imaging system ofclaim 12, further comprising a multispectral imaging system configured to capture multispectral image data that includes a living subject.

14. The imaging system ofclaim 13, wherein the multispectral imaging system comprises a visible light sensor and a near infrared (NIR) sensor.

15. The imaging system ofclaim 12, wherein the multispectral image data comprises visible light image data and near infrared (NIR) image data.

16. The imaging system ofclaim 12, wherein the living subject comprises a human subject, and wherein the detected live-subject portions comprise live-skin portions.

17. The imaging system ofclaim 16, wherein the shadow identification module is programmed to calculate a ratio of visible to NIR light for corresponding pixels in the visible and NIR images

18. The imaging system ofclaim 16, wherein the shadow attenuation module is programmed to attenuate the identified shadows based at least in part on the detected live-skin portions and the identified shadows in the visible light image data and the NIR image data.

19. The imaging system ofclaim 18, wherein the shadow attenuation module is programmed to attenuate the identified shadow by calculating a luminance distribution in the NIR and visible light image data.

20. An imaging system, comprising:

means for processing multispectral image data that includes a living subject to detect live-subject portions of the multispectral image data;

means for identifying shadows in the detected live-subject portions of the multispectral image data; and

means for attenuating the identified shadows in at least part of the multispectral image data.

21. The imaging system ofclaim 20, further comprising means for capturing multispectral image data that includes a living subject.

22. The imaging system ofclaim 21, wherein the capturing means comprises a visible light sensor and a near infrared (NIR) sensor.

23. The imaging system ofclaim 20, wherein the processing means comprises a live-subject verification module programmed to process the multispectral image data.

24. The imaging system ofclaim 20, wherein the shadow-identifying means comprises a shadow identification module programmed to identify the shadows.

25. The imaging system ofclaim 20, wherein the shadow-attenuating means comprises a shadow attenuation module programmed to attenuate the identified shadows.

26. A non-transitory computer-readable medium having stored thereon code that when executed performs a method comprising:

processing multispectral image data that includes a living subject to detect live-subject portions of the multispectral image data;

identifying shadows in the detected live-subject portions of the multispectral image data; and

attenuating the identified shadows in at least part of the multispectral image data.

27. The computer-readable medium ofclaim 26, wherein the multispectral image data comprises visible light image data and near infrared (NIR) image data.

Images