CN114270416B - Off-screen optical sensor with large sensing area - Google Patents

Abstract

Optical sensing with large sensing areas in a thin package is provided. For example, embodiments may operate in the context of an off-screen optical fingerprint sensor integrated into an electronic device such as a smartphone. In response to reflected probe light passing through the display module, the reflective structure is configured to redirect the reflected probe light onto the refractive structure, and the refractive structure is configured to concentrate the reflected probe light into an input aperture of the optical sensor for detection. Some embodiments operate in the context of an enhanced panel with microprismatic structures that tend to obscure the reflected probe light. In such an environment, embodiments are configured for off-axis detection to preferentially pass light through only certain micro-prism facets, thereby mitigating blur.

Description

Off-screen optical sensor with large sensing area

Cross Reference to Related Applications

This patent document claims the benefit and priority of U.S. non-provisional patent application No. 16/558,309 filed on month 9 and 2 of 2019. The entire contents of the above-mentioned patent applications are incorporated by reference as part of the disclosure of this patent document.

Technical Field

The present disclosure relates to optical sensors, such as under-screen (underscreen) optical fingerprint sensors, integrated with a display panel arrangement of a mobile computing device and used to provide a large sensing area.

Background

The various sensors may be implemented in an electronic device or system to provide certain desired functions. Sensors that enable user authentication are one example of sensors in a variety of devices and systems to protect personal data and prevent unauthorized access, including portable or mobile computing devices (e.g., notebook, tablet, smart phone), gaming systems, various databases, information systems, or larger computer control systems.

User authentication on an electronic device or system may be performed by one or more forms of biometric identifiers, which may be used alone or in combination with conventional cryptographic authentication methods. One common form of biometric identifier is a fingerprint pattern of a person. The fingerprint sensor may be built into the electronic device to read the fingerprint pattern of the user so that the device can only be unlocked by an authorized user of the device by authenticating the fingerprint pattern of the authorized user. Another example of a sensor for an electronic device or system is a biomedical sensor in a wearable device like a wristband device or a watch, etc., which detects a biometric of the user, e.g. a characteristic of the user's blood, heart beat. In summary, different sensors may be provided in the electronic device to achieve different sensing operations and functions.

The fingerprint may be used to authenticate a user accessing an electronic device, computer control system, electronic database, or information system, may be used as a separate authentication method or in combination with one or more other authentication methods, such as a password authentication method. For example, electronic devices and gaming systems, including portable or mobile computing devices such as notebook computers, tablet computers, smartphones, and the like, may utilize user authentication mechanisms to protect personal data and prevent unauthorized access. As another example, a computer or computer controlled device or system for an organization or business should be protected to allow access only by authorized personnel to protect information or use of the organization or business's device or system. The information stored in the portable device and computer controlled databases, devices or systems may be personal information in nature, such as personal contacts or phone books, personal photos, personal health information or other personal information, or confidential information used exclusively by organizations or businesses, such as business financial information, employee data, business secrets, and other proprietary information. If the security of accessing the electronic device or system is compromised, such data may be accessed by others, resulting in loss of personal privacy or loss of valuable confidential information. In addition to the security of information, secure access to computers and computer-controlled devices or systems also allows for the use of secured computers or computer processor-controlled devices or systems, such as computer-controlled automobiles and other systems such as ATMs.

Secure access to devices (e.g., mobile devices) or systems (e.g., electronic databases and computer-controlled systems) may be achieved in different ways, such as using user passwords. However, the password may be easily propagated or obtained, and this property of the password may reduce the security level of the password. Moreover, because users need to remember passwords to access password-protected electronic devices or systems, if a user forgets a password, the user needs to perform some password recovery procedure to obtain authentication or otherwise regain access to the device or system. These processes can be cumbersome for the user and have various practical limitations and inconveniences. Personal fingerprinting may be used to enable user authentication to enhance data security while mitigating some of the undesirable effects associated with passwords.

An electronic device or system including a portable or mobile computing device may utilize user authentication through one or more forms of biometric identifiers to protect personal or other confidential data and to prevent unauthorized access. The biometric identifier may be used alone or in combination with a cryptographic authentication method to provide user authentication. One form of biometric identifier is a fingerprint pattern of a person. The fingerprint sensor may be built into the electronic device or information system to read the fingerprint pattern of the user so that the device can only be unlocked by an authorized user of the device by authenticating the fingerprint pattern of the authorized user.

Disclosure of Invention

Embodiments provide optical sensing with large sensing areas in thin packages. For example, embodiments may operate in the context of an under-screen (unders-display) optical fingerprint sensor integrated into an electronic device such as a smartphone. In response to reflected probe light passing through the display module, the reflective structure is configured to redirect the reflected probe light onto the refractive structure, and the refractive structure is configured to concentrate the reflected probe light into an input aperture of the optical sensor for detection. Some embodiments operate in the context of an enhanced panel with microprismatic structures that tend to obscure the reflected probe light. In such an environment, embodiments are configured for off-axis detection to preferentially pass light through only certain micro-prism facets, thereby mitigating blur.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure. Together with the description, these drawings serve to explain the principles of the invention.

FIG. 1 is a block diagram of an example of a system with a fingerprint sensing module, which may be implemented to include an optical fingerprint sensor, according to some embodiments.

Fig. 2A and 2B illustrate exemplary implementations of an electronic device having a touch-sensing display screen assembly and an optical fingerprint sensor module positioned below the touch-sensing display screen assembly, according to some embodiments.

Fig. 3A and 3B illustrate examples of devices implementing the optical fingerprint sensor modules illustrated in fig. 2A and 2B, according to some embodiments.

Fig. 4A and 4B illustrate an exemplary implementation of an optical fingerprint sensor module located below a display screen assembly for implementing the design shown in fig. 2A and 2B, according to some embodiments.

Fig. 5A-5C illustrate signal generation of return light from a sensing region on a top sensing surface under two different optical conditions in accordance with some embodiments to facilitate understanding of operation of an off-screen optical fingerprint sensor module.

Fig. 6A-6C, 7, 8A-8B, 9, and 10A-10B illustrate example designs of an off-screen optical fingerprint sensor module according to some embodiments.

Fig. 11A-11C illustrate imaging of a fingerprint sensing area on a top transparent layer by an imaging module under different tiling conditions, wherein the imaging device images the fingerprint sensing area onto an optical sensor array, and the imaging device may be optically transmissive or optically reflective, according to some embodiments.

FIG. 12 is a flow chart illustrating exemplary operation of a fingerprint sensor for reducing or eliminating unwanted contributions from background light in fingerprint sensing, in accordance with some embodiments.

Fig. 13 is a flowchart illustrating an exemplary process for operating an on-screen optical fingerprint sensor module to capture a fingerprint pattern, in accordance with some embodiments.

Fig. 14-16 illustrate exemplary operational procedures for determining whether an object in contact with an LCD display screen is part of a live human finger by illuminating the finger with light of two different light colors, according to some embodiments.

Fig. 17A and 17B show cross-sections of an illustrative portable electronic device and an illustrative display module for such a portable electronic device, respectively, in accordance with various embodiments.

Fig. 18A-18C show views of illustrative portions of a symmetric enhancement layer.

Fig. 19A-19C show views of illustrative portions of asymmetric enhancement layers.

FIG. 20 shows an illustrative fingerprint sensing area overlaid on a single enhanced film layer.

FIG. 21 shows an illustrative fingerprint sensing area overlaid on a reinforced panel having two reinforced film layers stacked such that their respective microprismatic structures extend in a generally orthogonal direction.

FIG. 22 shows an illustrative under-screen optical sensing environment in accordance with various embodiments.

FIG. 23 shows an illustrative under-screen optical sensing environment with off-axis sensing, in accordance with various embodiments.

In the drawings, similar components and/or features may have the same reference numerals. Further, various components of the same type may be distinguished by following the reference label by a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description applies to any one of the similar components having the same first reference label irrespective of the second reference label.

Detailed Description

In the following description, numerous specific details are provided to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without one or more of these details. In other examples, features and techniques known in the art will not be described again for brevity.

An electronic device or system may be equipped with a fingerprint authentication mechanism to improve security of an access device. Such electronic devices or systems may include portable or mobile computing devices, such as smartphones, tablets, wrist-worn devices, and other wearable or portable devices, as well as larger electronic devices or systems, such as portable or desktop personal computers, ATMs, various terminals for commercial or government use to various electronic systems, databases or information systems, and including automobiles, boats, trains, planes, and other motor transportation systems.

Fingerprint sensing is useful in mobile applications and other applications that use or require secure access. For example, fingerprint sensing may be used to provide secure access to mobile devices and secure financial transactions including online shopping. It is desirable to include robust and reliable fingerprint sensing suitable for mobile devices and other applications. In mobile, portable, or wearable devices, it is desirable for the fingerprint sensor to minimize or eliminate the footprint for fingerprint sensing due to limited space on these devices, especially in view of the need for maximum display area on a given device. Many implementations of capacitive fingerprint sensors must be implemented on the top surface of the device due to the near field interaction requirements of capacitive sensing.

The optical sensing module may be designed to alleviate the above and other limitations of capacitive fingerprint sensors and achieve additional technical advantages. For example, in implementing an optical fingerprint sensing device, light carrying fingerprint imaging information may be directed over a distance to an optical detector array of optical detectors for fingerprint detection, without limitation to near field sensing in capacitive sensors. In particular, light carrying fingerprint imaging information may be directed through a top cover glass and other structures commonly used in many display screens, such as touch sensing screens, and may be directed to an optical detector array through folded or complex optical paths, allowing for flexibility in placing an optical fingerprint sensor in a device that is not suitable for a capacitive fingerprint sensor. The optical fingerprint sensor module based on the techniques disclosed herein may be an off-screen optical fingerprint sensor module that is placed under a display screen to collect and detect light from a finger placed on or above the top sensing surface of the screen. As disclosed herein, in addition to detecting and sensing fingerprint patterns, optical sensing may also be used to optically detect other parameters associated with a user or user action, such as whether a detected fingerprint is from a live human finger and is used to provide an anti-spoof mechanism, or to optically detect certain biometric parameters of a user.

I. Off-screen optical sensing module overview

Examples of the optical sensing techniques and implementations described herein provide an optical fingerprint sensor module that uses, at least in part, light from a display screen as illumination probe light to illuminate a fingerprint sensing area on a touch-sensing surface of the display screen to perform one or more sensing operations based on optical sensing of such light. A suitable display screen for implementing the disclosed optical sensor technology may be based on various display technologies or configurations, including: a liquid crystal display (liquid crystal display, LCD) screen that uses a backlight to provide white light illumination to LCD pixels and matched optical filters to implement color LCD pixels; or a display screen having light emitting display pixels without using a backlight, wherein each individual pixel generates light for forming a display image on a screen, e.g., an organic light emitting diode (organic light emitting diode, OLED) display screen or an electroluminescent display screen. While various aspects of the disclosed technology are applicable to OLED screens and other display screens, the specific examples provided below relate to the integration of an off-screen optical sensing module with an LCD screen, and thus contain certain technical details associated with an LCD screen.

A portion of the light generated by the display screen for displaying an image must pass through the top surface of the display screen to be seen by the user. A finger in contact with or near the top surface interacts with the light at the top surface such that the reflected or scattered light at the touch surface area carries the spatial image information of the finger. Such reflected or scattered light carrying the spatial image information of the finger is returned to the display panel below the top surface. In a touch-sensing display device, for example, the top surface is a touch-sensing interface that contacts a user, and such interaction between light used to display an image and a user's finger or hand continues to occur, but such information-carrying light returned to the display panel is wasted in a large amount and is not used in various touch-sensing devices. In various mobile or portable devices with touch-sensing display and fingerprint-sensing capabilities, the fingerprint sensor tends to be a device separate from the display, possibly disposed on the same surface of the display at a location other than the display area, such as in some models of apple iPhone and samsung smartphones, or possibly disposed on the back of smartphones, such as in some models of smart phones like Hua Cheng, hui-Xiang, setaria or Google, to avoid taking up valuable space on the front for disposing a large display. These fingerprint sensors are devices that are separate from the display screen and thus need to be compact to save space for the display screen and other functions while still providing reliable and fast fingerprint sensing with spatial image resolution above some acceptable level. However, in many fingerprint sensors, the need to design the fingerprint sensor to be compact and slim and the need to provide high spatial image resolution when capturing fingerprint patterns are directly conflicting with each other, as the high spatial image resolution when capturing fingerprint images based on various suitable fingerprint sensing technologies (e.g., capacitive touch sensing or optical imaging) requires a large sensor area with a large number of sensing pixels.

Examples of sensor technology and implementations of sensor technology described in this disclosure provide an optical fingerprint sensor module that, in some implementations, uses light from a display screen at least in part as illumination probe light to illuminate a fingerprint sensing area on a touch-sensing surface of the display screen, to perform one or more sensing operations based on optical sensing of such light, or in other implementations, uses specified illumination or probe light from one or more specified illumination light sources that is separate from the display light for optical sensing, or in some implementations, uses backlight for optical sensing.

In the disclosed example for integrating an optical sensing module into an LCD screen based on the disclosed optical sensor technology, an under-LCD optical sensor may be used to detect a portion of light used to display an image in the LCD screen, where such a portion of light for the display screen may be scattered light, reflected light, or some stray light. For example, in some implementations, image light of a backlight-based LCD screen may be reflected or scattered back into the LCD display screen as return light when encountering an object such as a user's finger or palm, or a user pointer device such as a stylus. Such return light may be collected for performing one or more optical sensing operations using the disclosed optical sensor techniques. Since optical sensing is performed using light from the LCD screen, optical fingerprint sensor modules based on the disclosed optical sensor technology are specifically designed to be integrated into the LCD display screen, wherein the way of integration preserves the display operation and functionality of the LCD display screen from interference while providing optical sensing operation and functionality to enhance the overall functionality, device integration, and user experience of an electronic device or system, such as a smart phone, tablet computer, or mobile and/or wearable device.

In addition, in various implementations of the disclosed optical sensing technology, one or more designated detection light sources may be provided to generate additional illumination detection light for optical sensing operations by the underscreen optical sensing module. In such applications, light from the backlight of the LCD screen and probe light from one or more designated probe light sources together form illumination light for optical sensing operations.

With respect to additional optical sensing functions other than fingerprint detection, optical sensing may be used to measure other parameters. For example, in view of the large touch area available on the entire LCD display screen, the disclosed optical sensor technology is capable of measuring the pattern of a person's palm (in contrast, some designated fingerprint sensors, such as the fingerprint sensor in the main button of a apple iPhone/tablet iPad device, have quite small and designated off-screen fingerprint sensing areas that are highly limited in the size of the sensing area, which may not be suitable for sensing large patterns). As another example, the disclosed optical sensor technology may be used not only to collect and detect patterns of a finger or palm associated with a person using optical sensing, but also to detect whether the collected or detected patterns of a fingerprint or palm are from a living person's hand by a "living finger" detection mechanism that is based on, for example, different optical absorption behaviors of blood at different optical wavelengths, in fact, the living person's finger is typically moving or stretching due to the person's natural movement or motion (intentional or unintentional), or the finger is typically pulsed as blood flows through a human body connected to the heartbeat. In one implementation, the optical fingerprint sensor module may detect a change in light returned from the finger or palm due to a change in heart beat/blood flow, thereby detecting whether a living heart beat is present in an object that appears as a finger or palm. User authentication may enhance access control based on a combination of optical sensing of fingerprint/palm patterns and positive determination of the presence of a living person. As another example, the optical fingerprint sensor module may include a sensing function for measuring glucose level or oxygen saturation based on optical sensing of return light from a finger or palm. As another example, when a person touches an LCD display screen, the change in touch force can be reflected in one or more ways, including fingerprint pattern distortion, a change in contact area between a finger and the screen surface, a widening of the fingerprint ridge, or a dynamic change in blood flow. These and other variations may be measured by optical sensing based on the disclosed optical sensor technology and may be used to calculate touch force. Such touch force sensing may be used to add more functionality to the optical fingerprint sensor module than fingerprint sensing.

For useful operational or control features related to touch sensing aspects of an LCD display screen, the disclosed optical sensor technology may provide a trigger function or additional function based on one or more sensing results from an optical fingerprint sensor module to perform certain operations related to touch sensing control on the LCD display screen. For example, the optical properties (e.g., refractive index) of the finger skin are typically different from other artifacts. Based on this, the optical fingerprint sensor module may be designed to selectively receive and detect return light caused by a finger in contact with the surface of the LCD display screen, while return light caused by other objects is not detected by the optical fingerprint sensor module. Such object-selective optical detection may be used to provide useful user control through touch sensing, such as waking up a smartphone or device only via a touch of a person's finger or palm, while touches of other objects do not cause waking up of the device to achieve energy-efficient operation and prolonged use of the battery. This operation may be accomplished by control based on the output of the optical fingerprint sensor module to control the wake-up circuit operation of the LCD display screen by turning off the LCD pixels (and turning off the LCD backlight) to be in a "sleep" mode while turning on one or more illumination sources (e.g., LEDs) for the optical fingerprint sensor module under the LCD panel to be in a flash mode to intermittently flash to the screen surface to sense any touch of a person's finger or palm. Under this design, the optical fingerprint sensor module operates one or more illumination light sources to produce a "sleep" mode wake-up sensing light flicker, such that the optical fingerprint sensor module is able to detect return light of such wake-up sensing light caused by a finger touch on the LCD display screen, and upon detection of the return light, the LCD backlight and the LCD display screen are turned on or "wake-up". In some implementations, the wake-up sensing light may be in a spectral range where infrared light is not visible, so the user does not experience any visual flickering of light. The LCD display screen operation may be controlled to provide improved fingerprint sensing by eliminating background light for optical sensing of fingerprints. For example, in one implementation, one frame fingerprint signal is generated per display scan frame. If two frames of display-related fingerprint signals are generated, wherein one frame of fingerprint signal is generated when the LCD display is on and the other frame of fingerprint signal is generated when the LCD display is off, the difference of the two frames of fingerprint signals may be used to reduce the influence of ambient background light. In some implementations, by operating the fingerprint sensing frame rate to be half the display frame rate, background light noise in fingerprint sensing can be reduced.

An optical fingerprint sensor module based on the disclosed optical sensor technology can be coupled to the back of an LCD display screen without creating a designated area on the surface side of the LCD display screen that would take up valuable device surface space in some smart phones, tablet computers, or wearable devices. This aspect of the disclosed technology may be used to provide certain advantages or benefits in device design and product integration or manufacturing.

In some implementations, an optical fingerprint sensor module based on the disclosed optical sensor technology may be configured as a non-invasive module that can be easily integrated into a display screen without requiring changes to the design of the LCD display screen to provide the desired optical sensing functionality, such as fingerprint sensing. In this regard, an optical fingerprint sensor module based on the disclosed optical sensor technology may be independent of the design of a particular LCD display screen due to the following properties of the optical fingerprint sensor module: the optical sensing of such an optical fingerprint sensor module is performed by detecting light emitted by one or more illumination sources of the optical fingerprint sensor module and returned from the top surface of the display area, and the disclosed optical fingerprint sensor module is coupled to the back surface of the LCD display screen as an off-screen optical fingerprint sensor module for receiving the returned light from the top surface of the display area, thereby eliminating the need for a specific sensing port or sensing area separate from the display screen area. Thus, such an off-screen optical fingerprint sensor module may be used in combination with an LCD display screen to provide optical fingerprint sensing and other sensor functions on the LCD display screen without using a specially designed LCD display screen with hardware specifically designed to provide such optical sensing. This aspect of the disclosed optical sensor technology enables a wide range of LCD displays in smartphones, tablets or other electronic devices to have enhanced functionality from optical sensing of the disclosed optical sensor technology.

For example, for existing handset component designs that do not provide a separate fingerprint sensor, like some apple iPhone or samsung Le Shi Galaxy smartphones, such existing handset component designs may integrate an off-screen optical fingerprint sensor module as disclosed herein without changing the touch sensing display screen component to provide increased on-screen fingerprint sensing functionality. Because the disclosed optical sensing does not require a separate designated sensing area or port, the integration of on-screen fingerprint sensing disclosed herein does not require substantial changes to existing handset component designs or touch sensing display modules with touch sensing layers and display layers, like some apple phone iPhone/samsung Galaxy phones have a front fingerprint sensor outside the display screen area, or some smartphones in models like hua, millet, google or association have a designated rear fingerprint sensor on the back. Based on the optical sensing technology disclosed in this document, there is no need to provide external sensing ports and external hardware buttons outside the device for adding the disclosed optical fingerprint sensor module for fingerprint sensing. The added optical fingerprint sensor module and associated circuitry is located below the display screen within the cell phone housing and fingerprint sensing can be conveniently performed on the same touch sensing surface of the touch screen.

As another example, due to the above-described nature of optical fingerprint sensor modules for fingerprint sensing, smartphones incorporating such optical fingerprint sensor modules can be updated with improved designs, functions, and integration mechanisms without affecting or burdening the design or manufacture of LCD display screens to provide desired flexibility for device manufacturing and improvement/upgrade in product cycles while maintaining the availability of updated versions of optical sensing functions for smartphones, tablet computers, or other electronic devices that use LCD display screens. In particular, by utilizing the disclosed off-screen optical fingerprint sensor module, the touch sensing layer or LCD display layer can be updated at the next product release without adding any significant hardware changes to the fingerprint sensing features. Furthermore, by using a new version of an off-screen optical fingerprint sensor module, improved on-screen optical sensing for fingerprint sensing or other optical sensing functions implemented for such an optical fingerprint sensor module may be added to the new product version without requiring significant changes to the handset design, including adding additional optical sensing functions.

The above and other features of the disclosed optical sensor technology may be implemented to provide improved fingerprint sensing and other sensing functions to new generation electronic devices, particularly for smartphones, tablets and other electronic devices with LCD display screens, to provide various touch sensing operations and functions, and to enhance the user experience of these devices. Features of the optical fingerprint sensor module disclosed herein may be applicable to a variety of display panels based on different technologies, including LCD display screens and OLED display screens. The following specific examples are directed to an LCD display panel and an optical fingerprint sensor module disposed below the LCD display panel.

In an implementation of the disclosed technical features, additional sensing functions or sensing modules may be provided, such as biomedical sensors, for example heartbeat sensors in wearable devices like wrist band devices or watches. In general, different sensors may be provided in an electronic device or system to achieve different sensing operations and functions.

The disclosed technology may be implemented to provide devices, systems, and techniques that perform optical sensing of human fingerprints and authentication for verifying access attempts to a locked computer-controlled device, such as a mobile device, or computer-controlled system, equipped with a fingerprint detection module. The disclosed technology may be used to secure access to a variety of electronic devices and systems, including portable or mobile computing devices such as notebook computers, tablet computers, smartphones, and gaming devices, as well as other electronic devices or systems such as electronic databases, automobiles, bank ATMs, and the like.

II. Design example of an off-screen optical sensing module

As described herein, embodiments provide a large sensing area implementation of an off-screen optical sensing module, such as for an off-screen optical fingerprint module. Examples of various designs for an off-screen optical fingerprint sensor module are described for collecting optical signals to an optical detector and providing the required optical imaging, e.g. with sufficient imaging resolution, for added clarity and context. These and other embodiments of an off-screen optical fingerprint sensing implementation are further described in the following patent documents, which are incorporated herein by reference in their entirety, respectively: U.S. patent application Ser. No. 15/616,856; U.S. patent application Ser. No. 15/421,249; U.S. patent application Ser. No. 16/190,138; U.S. patent application Ser. No. 16/190,141; U.S. patent application Ser. No. 16/246,549; U.S. patent application Ser. No. 16/427,269.

Fig. 1 is a block diagram of an example of asystem 180 having a fingerprint sensing module including afingerprint sensor 181, which may be implemented as an optical fingerprint sensor including optical sensing based on fingerprints disclosed in this document. Thesystem 180 includes a fingerprintsensor control circuit 184 and adigital processor 186, thedigital processor 186 may include one or more processors for processing the fingerprint pattern and determining whether the input fingerprint pattern is that of an authorized user. Thefingerprint sensing system 180 obtains a fingerprint using thefingerprint sensor 181 and compares the obtained fingerprint to a stored fingerprint to enable or disable functions in a device or system 188 that is protected by thefingerprint sensing system 180. In operation, thefingerprint processing processor 186 controls access to the device 188 based on whether the acquired user fingerprint is from an authorized user. As shown, thefingerprint sensor 181 may include a plurality of fingerprint sensing pixels, such as pixels 182A-182E that collectively represent at least a portion of a fingerprint. For example, thefingerprint sensing system 180 may be implemented at an ATM as the system 188 to determine the fingerprint of a customer requesting access to funds or other transactions. Based on a comparison of the customer fingerprint obtained fromfingerprint sensor 181 with one or more stored fingerprints, in response to positive identification,fingerprint sensing system 180 may cause ATM system 188 to grant access to the requested user account, or in response to negative identification, may deny access. As another example, the device or system 188 may be a smart phone or portable device and thefingerprint sensing system 180 is a module integrated into the device 188. As another example, the device or system 188 may be a door or secure portal of a facility or home that uses thefingerprint sensor 181 to grant or deny access. As another example, the device or system 188 may be an automobile or other vehicle that uses thefingerprint sensor 181 to link to the start of the engine and identify whether a person is authorized to operate the automobile or vehicle.

As a specific example, fig. 2A and 2B illustrate one exemplary implementation of an electronic device 200, the electronic device 200 having a touch-sensing display screen assembly and an optical fingerprint sensor module positioned below the touch-sensing display screen assembly. In this particular example, the display technology may be implemented by an LCD display screen having a backlight for optically illuminating LCD pixels or another display screen (e.g., an OLED display screen) having light emitting display pixels without using a backlight. The electronic device 200 may be a portable device such as a smart phone or tablet computer, or may be the device 188 shown in fig. 1.

Fig. 2A illustrates a front side of a device 200 that is similar to some features in some existing smartphones or tablets. The device screen is located on the front of the device 200, occupies all, most or a significant portion of the front space, and provides fingerprint sensing functionality on the device screen, e.g., one or more sensing areas for receiving a finger on the device screen. As an example, fig. 2A illustrates a fingerprint sensing area for finger touch in a device screen that may be illuminated as a clearly identifiable region or area for a user to place a finger for fingerprint sensing. Such a fingerprint sensing area may be used to display an image like the rest of the device screen. As shown, in various implementations, the device housing of the device 200 may have sides that support side control buttons commonly found in various smartphones currently on the market. Also, as shown in one example of the upper left corner of the device housing in FIG. 2A, the device 200 may be provided with one or more optional sensors on the front face outside the device screen.

Fig. 2B shows an example of the structural configuration of the module in the apparatus 200 related to optical fingerprint sensing disclosed in this document. The device screen assembly shown in fig. 2B includes: for example, a touch sensing screen module having a touch sensing layer on top, and a display screen module having a display layer under the touch sensing screen module. An optical fingerprint sensor module is coupled to and below the display screen assembly module to receive and collect return light from the top surface of the touch sensing screen module and direct and image the return light onto an optical sensor array of optical sensing pixels or photodetectors that converts optical images in the return light into pixel signals for further processing. Below the optical fingerprint sensor module is a device electronics structure that contains certain electronic circuitry for the optical fingerprint sensor module and other components in the device 200. The device electronics may be disposed inside the device housing and may include components below the optical fingerprint sensor module as shown in fig. 2B.

In some implementations, the top surface of the device screen assembly may be a surface of an optically transparent layer that serves as a user touch-sensing surface to provide a variety of functions, such as (1) a display output surface through which light carrying a display image passes to the eye of a viewer, (2) a touch-sensing interface through which user touches are received by the touch-sensing screen module for touch-sensing operations, and (3) an optical interface for on-screen fingerprint sensing (and possibly one or more other optical sensing functions). The optically transparent layer may be a rigid layer or a flexible layer such as a glass or crystalline layer.

One example of a display screen is an LCD display screen having an LCD layer, a thin film transistor (thin film transistor, TFT) structure or substrate. The LCD display panel is a multi-layer Liquid Crystal Display (LCD) module that includes an LCD display backlight source (e.g., an LED lamp) that emits LCD illumination light for LCD pixels, an optical waveguide layer that directs the backlight, and an LCD structural layer that may include, for example, a Liquid Crystal (LC) cell layer, LCD electrodes, a transparent conductive ITO layer, an optical polarizer layer, a color filter layer, and a touch sensing layer. The LCD module further includes a backlight diffuser positioned below the LCD structural layer and above the optical waveguide layer for spatially dispersing backlight for illuminating LCD display pixels; and an optical reflector film layer under the optical waveguide layer for recycling backlight to the LCD structural layer to improve light utilization and display brightness. For optical sensing, one or more individual illumination sources may be provided and operated independently with respect to the backlight source of the LCD display module.

Referring to fig. 2B, the optical fingerprint sensor module in this example is located below the LCD display panel to collect return light from the top touch-sensing surface and to acquire a high resolution fingerprint pattern image when a user's finger is in contact with a sensing area on the top surface. In other implementations, the disclosed off-screen optical fingerprint sensor module for fingerprint sensing may be implemented on a device that does not have touch sensing features.

Fig. 3A and 3B illustrate examples of devices implementing the optical fingerprint sensor module of fig. 2A and 2B. Fig. 3A shows a cross-sectional view of a portion of a device including an off-screen optical fingerprint sensor module. Fig. 3B shows on the left side a view of the front of a device with a touch-sensing display screen, indicating a fingerprint sensing area on the lower part of the screen, and on the right side a perspective view of a portion of the device comprising an optical fingerprint sensor module located below the device display screen assembly. Fig. 3B also shows an example of a layout of a flexible tape with circuit elements.

In the design examples of fig. 2A-2B and 3A-3B, the optical fingerprint sensor design is different from some other fingerprint sensor designs that use a display-independent fingerprint sensor structure and have a physical demarcation between the display and the fingerprint sensor on the surface of the mobile device (e.g., a button-like structure in an opening of the top glass cover plate in some mobile phone designs). In the design shown herein, the optical fingerprint sensor for detecting fingerprint sensing signals and other optical signals is located below the top cover glass or layer (e.g., fig. 3A) such that the top surface of the cover glass serves as the top surface of the mobile device as a continuous and uniform glass surface over vertically stacked and vertically overlapping display screen layers and optical detection sensors. Such design examples for integrating optical fingerprint sensing and touch sensitive display screens under a common and uniform surface provide benefits including improved device integration, enhanced device packaging, enhanced resistance of the device to external elements, failures, wear and tear, and enhanced user experience during the holding of the device.

Referring back to fig. 2A and 2B, the illustrated off-screen optical fingerprint sensor module for on-screen fingerprint sensing may be implemented in a variety of configurations. In one implementation, a device based on the above design may be configured to include a device screen that provides touch sensing operations and includes an LCD display panel structure for forming a display image, a top transparent layer formed over the device screen as an interface that is touched by a user to perform the touch sensing operations and transmits light from the display structure to display the image to the user, and an optical fingerprint sensor module under the display panel structure to receive light returned from the top transparent layer to detect a fingerprint.

Such devices and other devices disclosed herein may also be configured to include various features. For example, a device electronic control module may be included in the device to grant a user access to the device when the detected fingerprint matches the fingerprint of the authorized user. Further, the optical fingerprint sensor module is configured to detect, in addition to the fingerprint, a biometric parameter different from the fingerprint by optical sensing to indicate whether a touch at the top transparent layer associated with the detected fingerprint is from a living person, and if (1) the detected fingerprint matches a fingerprint of an authorized user, and (2) the detected biometric parameter indicates that the detected fingerprint is from a living person, the device electronic control module is configured to grant the user access to the device. The biometric parameters may include, for example, whether the finger contains a human blood flow or heartbeat.

For example, the device may include a device electronic control module coupled to the display panel structure to provide power to the light emitting display pixels and control image display of the display panel structure, and in a fingerprint sensing operation, the device electronic control module is operative to turn off the light emitting display pixels in one frame and turn on the light emitting display pixels in a next frame to allow the optical sensor array to capture two fingerprint images with and without light emitting display pixel illumination, thereby reducing background light in fingerprint sensing.

For another example, the device electronic control module may be coupled to the display panel structure to provide power to the LCD display panel and turn off backlight power to the LCD display panel in the sleep mode, and when the optical fingerprint sensor module detects the presence of human skin at a designated fingerprint sensing region of the top transparent layer, the device electronic control module may be configured to wake up the display panel structure from the sleep mode. More specifically, in some implementations, the device electronic control module may be configured to operate one or more illumination sources in the optical fingerprint sensor module to intermittently emit light while powering off the LCD display panel (in sleep mode) for directing the intermittently emitted illumination light to a designated fingerprint sensing area of the top transparent layer to monitor whether there is skin of a person in contact with the designated fingerprint sensing area for waking up the device from sleep mode.

For another example, the device may include a device electronic control module coupled to the optical fingerprint sensor module to receive information of a plurality of detected fingerprints obtained by sensing a touch of a finger, and operative to measure a change in the plurality of detected fingerprints and determine a touch force that causes the measured change. For example, the change may include a change in a fingerprint image due to a touch force, a change in a touch area due to a touch force, or a change in a fingerprint ridge interval.

For another example, the top transparent layer may include a designated fingerprint sensing area for a user to touch by a finger for fingerprint sensing, and the optical fingerprint sensor module under the display panel structure may include a transparent block in contact with the display panel substrate to receive light emitted from the display panel structure and returned from the top transparent layer, and the optical fingerprint sensor module may further include an optical sensor array to receive the light and an optical imaging module to image the light received in the transparent block onto the optical sensor array. The optical fingerprint sensor module may be positioned relative to a designated fingerprint sensing region and configured to: light returned by total internal reflection at the top surface of the top transparent layer is selectively received when in contact with human skin, and light returned from the designated fingerprint sensing region is not received when there is no contact by human skin.

As another example, an optical fingerprint sensor module may be configured to include an optical wedge positioned below a display panel structure to change a total reflection condition on a bottom surface of the display panel structure engaged with the optical wedge to allow light to be extracted from the display panel structure through the bottom surface, the optical fingerprint sensor module may further include an optical sensor array that receives light from the optical wedge extracted from the display panel structure, and an optical imaging module positioned between the optical wedge and the optical sensor array to image light from the optical wedge onto the optical sensor array.

Fig. 4A and 4B illustrate an example of one implementation of an optical fingerprint sensor module located below a display screen assembly for implementing the designs of fig. 2A and 2B. The device shown in fig. 4A and 4B includes adisplay assembly 423 having a toptransparent layer 431 formed over thedevice screen assembly 423 as an interface that is touched by a user for a touch sensing operation and transmits light from a display structure to display an image to the user. In some implementations, the toptransparent layer 431 may be a cover glass or crystalline material. Thedevice screen assembly 423 may include anLCD display module 433 below a toptransparent layer 431. The LCD display layer allows partial optical transmission such that light from the top surface can partially pass through the LCD display layer to the optical fingerprint sensor module under the LCD. For example, the LCD display layer includes electrodes and wiring structures that optically function as an array of apertures and light scattering objects. Adevice circuit module 435 may be disposed below the LCD display panel to control the operation of the device and perform functions for a user to operate the device.

The opticalfingerprint sensor module 702 in this particular implementation example is located below theLCD display module 433. One or more illumination sources, such asillumination source 436 belowLCD display module 433 or/and another illumination source or sources belowtop cover glass 431, are provided for providing illumination or detection light optically sensed by opticalfingerprint sensing module 702 and may be controlled to illuminate at least partially throughLCD display module 433 to illuminatefingerprint sensing region 615 on toptransparent layer 431 in the area of the device screen for a user to place a finger therein for fingerprint recognition. Illumination from one ormore illumination sources 436 may be directed to thefingerprint sensing area 615 on the top surface as if the illumination were from the fingerprintillumination light area 613. Another one or more illumination sources may be located below thetop cover glass 431 and may be placed adjacent to thefingerprint sensing region 615 on the top surface to direct the generated illumination light to thetop cover glass 431 without passing through theLCD display module 433. In some designs, one or more illumination sources may be located above the bottom surface of thetop cover glass 431 to direct the generated illumination light to a fingerprint sensing area above the top surface of thetop cover glass 431 without having to pass through thetop cover glass 431, e.g., directly illuminate a finger above thetop cover glass 431.

As shown in fig. 4A, afinger 445 is placed in an illuminatedfingerprint sensing region 615 as an active sensing region for fingerprint sensing. A portion of the reflected or scattered light inregion 615 is directed into the optical fingerprint sensor module belowLCD display module 433 and a photodetector sensing array within the optical fingerprint sensor module receives such light and collects fingerprint pattern information carried by the received light. The one ormore illumination sources 436 are separate from and operate independent of the backlight for the LCD display module.

In such designs that use one ormore illumination sources 436 to provide illumination light for optical fingerprint sensing, in some implementations, eachillumination source 436 may be controlled to be intermittently turned on at a relatively slow period, thereby reducing the energy source for optical sensing operations. In some implementations, the fingerprint sensing operation may be implemented in a two-step process: first, one or more illuminationlight sources 436 are turned on in a flash mode without turning on the LCD display panel, thereby sensing whether a finger touches thesensing region 615 using a flash light, and then, upon detecting the presence of a touch in theregion 615, the optical sensing module is operated to perform fingerprint sensing based on optical sensing, and the LCD display panel may be turned on.

In the example of fig. 4B, the off-screen optical fingerprint sensor module includes atransparent block 701 coupled to the display panel to receive return light from the top surface of the device assembly and anoptical imaging block 702 to perform optical imaging and imaging acquisition. In one design ofillumination sources 436,illumination sources 436 are positioned to direct illumination light first throughtop cover glass 431 and then to the finger, from which light from one ormore illumination sources 436 is reflected or scattered back after reaching the cover top surface, e.g., the cover top surface at sensingregion 615 where the user's finger touches or when not touching the cover top surface. When the fingerprint ridge is in contact with the top surface of the cover plate in thesensing region 615, the light reflection under the fingerprint ridge differs from the light reflection under the fingerprint valley at another location where there is no skin or tissue of the finger due to the presence of skin or tissue of the finger in contact at that location. This difference in light reflection conditions at the locations of the ridges and valleys in the area touched by the finger on the top surface of the cover sheet forms an image representing the spatial distribution of the ridges and valleys of the touched portion of the finger. The reflected light is directed back to theLCD display module 433 and, after passing through the aperture of theLCD display module 433, reaches the interface of the low refractive index opticallytransparent block 701 of the optical fingerprint sensor module. The low refractive index opticallytransparent block 701 is configured to have a refractive index smaller than that of the LCD display panel so that the return light can be extracted from the LCD display panel into the opticallytransparent block 701. Once the return light is received within opticallytransparent block 701, such received light enters an optical imaging unit that is part ofimaging sensing block 702 and is imaged onto a photodetector sensing array or optical sensing array withinblock 702. The difference in light reflection between the fingerprint ridges and valleys creates the contrast of the fingerprint image. As shown in fig. 4B, a control circuit 704 (e.g., a microcontroller or MCU) is coupled to theimaging sense block 702 and other circuitry on the main circuit board, such as a devicemain processor 705.

In this particular example, the optical light path design is configured such that illumination light enters the top surface of the cover plate over a range of total reflection angles on the top surface between the substrate and the air interface, and thus, reflected light is most effectively collected by the imaging optics and imaging sensor array inblock 702. In this design, the image of the fingerprint ridge/valley region exhibits maximum contrast due to the total internal reflection condition at each finger valley position, where the finger tissue does not touch the top cover surface of thetop cover glass 431. Implementations of such imaging systems may have undesirable optical distortions that can adversely affect fingerprint sensing. Thus, based on the optical distortion profile at the optical sensor array along the optical path of the return light, the acquired image is further corrected by distortion correction during imaging reconstruction when the output signals of the optical sensor array inblock 702 are processed. By scanning the test image pattern of one row of pixels at a time over the entire sensing area of the X-direction lines and Y-direction lines, distortion correction coefficients can be generated from the images acquired at each photodetector pixel. Such a correction process may also use an image from turning on a single pixel at a time and scanning the entire image area of the photodetector array. Such correction factors need only be generated once after the sensor is assembled.

Background light from the environment (e.g., sunlight or room illumination light) may enter the image sensor through the top surface of the LCD panel and further through an aperture in theLCD display assembly 433. Such background light may create a background baseline in the valuable image from the finger and thus may undesirably reduce the contrast of the acquired image. Different methods may be used to reduce this undesirable baseline intensity caused by the background light. One example is to turn theillumination light source 436 on and off at an illumination modulation frequency f, and by phase synchronizing the light source drive pulses with the image sensor frame, the image sensor thus acquires the received image at the same illumination modulation frequency. In this operation only one of the image phases contains light from the light source. In practicing this technique, the imaging acquisition may be timed to turn on illumination light at even (or odd) frames while the illumination light is turned off at odd (or even) frames to acquire an image, and therefore, the even and odd frames may be subtracted for obtaining an image formed primarily of light emitted by a modulated illumination source with significantly reduced background light. Based on this design, one frame of fingerprint signal is generated per display scan frame, and two consecutive signal frames are obtained by turning on illumination light in one frame and turning off illumination light in the other frame. The subtraction of adjacent frames may be used to minimize or substantially reduce the effect of ambient background light. In some implementations, the fingerprint sensing frame rate may be half the display frame rate.

In the example shown in fig. 4B, a portion of the light from the one ormore illumination sources 436 may also pass through the top surface of the cover plate and into the finger tissue. This portion of the illumination light is scattered around, and a portion of this scattered light may eventually be collected by the imaging sensor array in the opticalfingerprint sensor module 702. The intensity of this scattered light is a result of interactions with the internal tissues of the finger and therefore depends on the skin color of the finger, the blood concentration in the finger tissue, or the internal finger tissue. This information of the finger is carried by this scattered light on the finger, is useful for fingerprint sensing, and can be detected as part of a fingerprint sensing operation. For example, the intensities of the regions of the user's finger image may be integrated at the time of detection to measure or observe an increase or decrease in blood concentration associated with or dependent on the phase of the user's heartbeat. Such features may be used to determine the heart rate of the user, to determine whether the user's finger is a live finger, or to provide a spoof device with a fake fingerprint pattern. The latter part of this patent document provides additional examples of using information in the light carrying the internal tissue information of the finger.

In some designs, one ormore illumination sources 436 in fig. 4B may be designed to emit illumination light of different colors or wavelengths, and the optical fingerprint sensor module may collect return light from a person's finger at the different colors or wavelengths. By recording the intensities of the corresponding measured return light at different colors or wavelengths, information associated with the skin tone, blood flow, or internal tissue structures within the finger of the user may be measured or determined. For example, when a user registers a finger for a fingerprint authentication operation, the optical fingerprint sensor may be operable to measure the intensity of scattered light from the finger at two different colors or wavelengths of illumination light associated with light color a and light color B, respectively, noted as intensities Ia and Ib. When a user's finger is placed on the sensing area on the top sensing surface to measure a fingerprint, the ratio of Ia/Ib may be recorded for comparison with later measurements. The method may be used as part of an anti-spoofing system of the device to reject spoofed devices made by using simulated user fingerprints or fingerprints that are the same as the user fingerprints but may not match the user's skin tone or other biometric information of the user.

One ormore illumination sources 436 may be controlled by the same electronics 704 (e.g., MCU) used for the image sensor array incontrol block 702. The one or more illuminationlight sources 436 may be pulsed for a short time (e.g., low duty cycle) to intermittently emit light and provide pulsed light for image sensing. The image sensor array may be operable to monitor the light pattern with the same pulse duty cycle. If there is a human finger touching thesensing area 615 on the screen, the image acquired at the imaging sensing array inblock 702 may be used to detect a touch event. Control electronics orMCU 704 connected to the image sensor array inblock 702 may be operative to determine whether the touch is a human finger touch. If a human finger touch event is determined,MCU 704 may be operable to wake up the smartphone system, turn on one or more illuminationlight sources 436 for optical fingerprint sensing, and acquire a complete fingerprint image using normal mode. The image sensor array inblock 702 sends the acquired fingerprint image to the smartphonemain processor 705, and the smartphonemain processor 705 is operable to match the acquired fingerprint image with the registered fingerprint database. If there is a match, the smartphone unlocks the phone to allow the user to access the phone and initiate normal operation. If the acquired images are not matched, the smart phone feeds back authentication failure to the user and keeps the locking state of the mobile phone. The user may attempt to again perform fingerprint sensing or may enter a password as an alternative to unlocking the handset.

In the example shown in fig. 4A and 4B, the under-screen optical fingerprint sensor module uses an opticallytransparent block 701 and animaging sensing block 702 having a photodetector sensing array, so as to optically image a fingerprint pattern of a touching finger in contact with the top surface of the display screen onto the photodetector sensing array. By way of example, an optical imaging ordetection axis 625 from thesensing region 615 to the photodetector array inblock 702 is shown in fig. 4B. The front ends of the opticallytransparent block 701 andimaging sensing block 702, which are in front of the photodetector sensing array, form a volumetric imaging module to achieve suitable imaging for optical fingerprint sensing. Due to the optical distortion during the imaging process, distortion correction may be used to achieve the desired imaging operation.

In the optical sensing by the off-screen optical fingerprint sensor module and other designs in fig. 4A and 4B disclosed herein, the optical signal from thesensing region 615 on the toptransparent layer 431 to the off-screen optical fingerprint sensor module includes different light components.

Fig. 5A-5C illustrate signal generation for light returned from sensingregion 615 under different optical conditions to facilitate an understanding of the operation of an off-screen optical fingerprint sensor module. Light entering the finger from an illumination source or from other sources (e.g., background light) may produce internal scattered light in tissue below the surface of the finger, such asscattered light 191 in fig. 5A-5C. Such internally scattered light in the tissue below the finger surface may propagate through the finger's internal tissue and then penetrate the finger's skin into the toptransparent layer 431, carrying some information not carried by the light scattered, refracted or reflected by the finger surface, e.g., information about the finger's skin color, blood concentration or flow characteristics within the finger, or the optical transmission pattern of the finger, which contains (1) a two-dimensional spatial pattern of external ridges and valleys of the fingerprint; and (2) an internal fingerprint pattern associated with an internal finger organization that produces external ridges and valleys of the finger.

Fig. 5A shows an example of how illumination light from one or more illuminationlight sources 436 propagates throughLCD display module 433 after passing through toptransparent layer 431 and generates a different return light signal to an off-screen optical fingerprint sensor module, including a light signal carrying fingerprint pattern information. For simplicity, twoillumination rays 80 and 82 at two different locations are directed to the toptransparent layer 431 without undergoing total reflection at the interface of the toptransparent layer 431. In particular, illumination rays 80 and 82 are perpendicular or nearly perpendicular totop layer 431. Thefinger 60 is in contact with asensing region 615 on the toptransparent layer 431. As shown,illumination beam 80, after passing through toptransparent layer 431, reaches the finger ridge in contact with toptransparent layer 431 to generatebeam 183 in finger tissue and anotherbeam 181 back toLCD display module 433. After passing through toptransparent layer 431,illumination beam 82 reaches a finger valley located above toptransparent layer 431 to generate reflectedbeam 185 that returns from the interface of toptransparent layer 431 toLCD display module 433,second beam 189 that enters the finger tissue, andthird beam 187 that is reflected by the finger valley.

In the example of fig. 5A, it is assumed that the equivalent refractive index of the finger skin at 550nm is about 1.44, and the cover glass refractive index of the toptransparent layer 431 is about 1.51. The finger ridge-cover glass interface reflects a portion of thelight beam 80 as reflected light 181 to thebottom layer 524 below theLCD display module 433. In some LCD panels, the reflectivity may be low, for example, about 0.1%. Most of the light inbeam 80 becomes transmitted throughbeam 183 infinger tissue 60, andfinger tissue 60 causes scattering oflight 183, producing scattered light 191 that returns toLCD display module 433 andbottom layer 524. The scattering of the transmittedbeam 189 from the LCD pixels in the finger tissue also contributes to the returnedscattered light 191.

Thelight beam 82 at the fingerskin valley location 63 is reflected by the cover glass surface. In some designs, for example, the reflectivity may be about 3.5% with the reflected light 185 toward thebottom layer 524, and the finger valley surface may reflect about 3.3% (light 187) of the incident light energy to thebottom layer 524, such that total reflection may be about 6.8%. Most of the light 189 is transmitted into thefinger tissue 60. A portion of the light energy in the transmitted light 189 in the finger tissue is scattered by the tissue, contributing into thescattered light 191 toward and into theunderlying layer 524.

Thus, in the example of fig. 5A, the light reflections from the various interfaces or surfaces at the finger valleys and finger ridges of the touching finger are different, and such reflectance differences carry fingerprint information and can be measured to extract the fingerprint pattern of the portion that is in contact with the toptransparent layer 431 and illuminated by the LCD light.

Fig. 5B and 5C illustrate the optical paths of two additional types of illumination light under different conditions and on the top surface at different positions relative to the valleys or ridges of the finger, including the optical paths of illumination light under total reflection conditions at the interface of the toptransparent layer 431. The illustrated illumination light generates different return light signals to the off-screen optical fingerprint sensor module, including light signals carrying fingerprint pattern information. Assuming that thecover glass 431 and theLCD display module 433 are bonded together without any air gap therebetween, illumination light having a large incident angle with respect to thecover glass 431 is totally reflected at the cover glass-air interface. Fig. 5A, 5B and 5C show examples of three different sets of divergent beams: (1) acentral beam 82 having a small angle of incidence relative to coverglass 431 and no total reflection occurs (fig. 5A), (2) high contrast beams 201, 202, 211, and 212, which total reflection occurs atcover glass 431 when no object touches the cover glass surface and can couple into finger tissue when a finger touches cover glass 431 (fig. 5B and 5C), and (3) escape beams having a large angle of incidence, which total reflection occurs atcover glass 431 even at the location of the finger tissue touch.

Forcenter beam 82, in some designs, the cover glass surface reflects approximately 0.1% to 3.5% of the light tobeam 185, which is transmitted intounderlayer 524, and the finger skin may reflect approximately 0.1% to 3.3% of the light tobeam 187, which is also transmitted intounderlayer 524. The difference in reflection depends on whether thelight beam 82 meets thefinger skin ridge 61 orvalley 63. The remaininglight beam 189 is coupled into thefinger tissue 60.

For high contrast beams 201 and 202 that meet the condition of partial total internal reflection, the cover glass surface reflects nearly 100% of the light tobeams 205 and 206, respectively, if no object touches the cover glass surface. When the finger skin ridge touches the cover glass surface and is located at the position ofbeams 201 and 202, a large portion of the light energy can be coupled into thefinger tissue 60 throughbeams 203 and 204.

For high contrast beams 211 and 212 that meet the condition of partial total internal reflection, the cover glass surface reflects nearly 100% of the light tobeams 213 and 214, respectively, if no object touches the cover glass surface. When a finger touches the cover glass surface and the finger skin valleys are just in the location oflight beams 211 and 212, no light energy is coupled intofinger tissue 60.

As shown in fig. 5A, a portion of the illumination light coupled into thefinger tissue 60 will be randomly scattered by the internal finger tissue to formlow contrast light 191, and a portion of suchlow contrast light 191 will pass through theLCD display module 433 to the optical fingerprint sensor module. The portion of light collected by the optical fingerprint sensor module contains additional information about the finger skin color, blood characteristics, and the internal tissue structure of the finger associated with the fingerprint. Additional features for using internally scattered light in tissue below the finger surface in optical sensing will be explained in the latter part of this patent document, e.g. obtaining an optical transmission pattern of the finger, a two-dimensional spatial pattern comprising (1) external ridges and valleys of the fingerprint; and (2) an internal fingerprint pattern associated with an internal finger organization that produces external ridges and valleys of the finger. Thus, in areas illuminated by the high contrast beam, the finger skin ridges and valleys cause different optical reflections, and the reflection difference pattern carries fingerprint pattern information. A high contrast fingerprint signal may be achieved by comparing such differences.

Based on the designs shown in fig. 2A and 2B, the disclosed off-screen optical sensing technology may optically fingerprint in various configurations. For example, the particular implementation of fig. 4B based on optical imaging using a volumetric imaging module in an optical sensing module may be implemented in various configurations.

Fig. 6A-6C illustrate examples of an off-screen optical fingerprint sensor module based on optical imaging via lenses for capturing a fingerprint from afinger 445 pressing adisplay cover glass 423. Fig. 6C is an enlarged view of a portion of the optical fingerprint sensor module shown in fig. 6B. The off-screen optical sensor module as shown in fig. 6B is located below theLCD display module 433, including: opticallytransparent spacers 617 bonded to the bottom surface of theLCD display module 433 for receiving return light from thesensing region 615 on the top surface of the toptransparent layer 431; and animaging lens 621 positioned between thespacer 617 and thephotodetector array 623 to image the return light received from thesensing region 615 onto thephotodetector array 623. Unlike the example of an optical projection imaging system that does not include lenses shown in fig. 4B, the example of the imaging design in fig. 6B uses animaging lens 621 to capture a fingerprint image at aphotodetector array 623, and image reduction is achieved by the design of theimaging lens 621. To some extent similar to the imaging system in the example of fig. 4B, the imaging system for the optical fingerprint sensor module in fig. 6B may experience image distortion and may use appropriate optical correction calibration to reduce such distortion, for example, the distortion correction method described for the system in fig. 4B.

Similar to the assumptions in fig. 5A-5C, it is assumed that the equivalent refractive index of the finger skin at 550nm is about 1.44, and forcover glass 423, the refractive index of bare cover glass is about 1.51. When theLCD display module 433 is bonded to thecover glass 431 without any air gaps, total internal reflection occurs at a large angle equal to or greater than the critical angle of incidence of the interface. If no object touches the top surface of the cover glass, the angle of incidence of total reflection is about 41.8 °, and if the finger skin touches the top surface of the cover glass, the angle of total reflection is about 73.7 °. The corresponding total reflection angle difference is about 31.9 °.

In this design, themicrolens 621 andphotodiode array 623 define a viewing angle θ for capturing an image of a contact finger in thesensing region 615. To detect a desired portion of thesensing region 615 on the cover glass surface, the viewing angle may be properly aligned by controlling a physical parameter or configuration. For example, the viewing angle may be aligned to detect total internal reflection of the LCD display assembly. Specifically, the viewing angle θ is aligned to sense aneffective sensing region 615 on the cover glass surface. The active sensingcover glass surface 615 may be considered a mirror such that the photodetector array effectively detects an image of the fingerillumination light area 613 in the LCD display screen, which is projected onto the photodetector array by the sensingcover glass surface 615. The photodiode/photodetector array 623 can receive an image reflected by the sensingcover glass surface 615 in theregion 613. When a finger touches thesensing region 615, a portion of the light may couple into the ridges of the fingerprint, which may cause the photodetector array to receive light from the ridge locations to appear as a darker image of the fingerprint. Since the geometrical principle of the optical detection path is known, the fingerprint image distortion caused in the optical path in the optical fingerprint sensor module can be corrected.

As a specific example, assume that the distance H from the detection module center axis to the cover glass top surface in fig. 6B is 2mm. This design can directly cover aneffective sensing area 615 of 5mm, theeffective sensing area 615 being Wc in the width of the cover glass. Adjusting the thickness of thespacer 617 can adjust the detector position parameter H and can optimize the effective sensing zone width Wc. Since H includes the thickness ofcover glass 431 anddisplay module 433, the application design should take these layers into account. Thespacer 617, themicrolens 621 and thephotodiode array 623 may be integrated under thecolor coating 619 on the bottom surface of the toptransparent layer 431.

Fig. 7 illustrates an example of another design consideration for the optical imaging design of the optical fingerprint sensor module shown in fig. 6A-6C by usingspecial spacers 618 instead of thespacers 617 in fig. 6B-6C to increase the size of thesensing region 615. Thespacer 618 is designed to have a width Ws, a thickness Hs, a low Refractive Index (RI) ns, and thespacer 618 is placed under theLCD display module 433, for example, attached (e.g., adhered) to the bottom surface of theLCD display module 433. The end surfaces of thespacers 618 are angled or slanted surfaces that engage themicrolenses 621. This relative position of the spacer and the lens is different from fig. 6B-6C, in which fig. 6B-6C the lens is located below thespacer 617. Themicrolens 621 andphotodiode array 623 are assembled into an optical detection module having a detection angular extent θ. Thedetection axis 625 is curved due to optical refraction at the interface between thespacer 618 and thedisplay module 433 and at the interface between thecover glass 431 and air. The local angles of incidence φ 1 and φ 2 are determined by the refractive indices RI, ns, nc, and na of the component materials.

If nc is greater than ns, then φ 1 is greater than φ 2. Thereby, the refraction amplifies the sensing width Wc. For example, assuming that the equivalent refractive index RI of the finger skin is about 1.44 at 550nm and the refractive index RI of the cover glass is about 1.51, the total reflection incident angle is estimated to be about 41.8 ° if no object touches the cover glass top surface, and the total reflection angle is about 73.7 ° if the finger skin touches the cover glass top surface. The corresponding total reflection angle difference is about 31.9 °. If thespacer 618 is made of the same material as the cover glass, the distance from the center of the detection module to the top surface of the cover glass is 2mm, and if the detection angular extent θ=31.9°, the effective sensing area width Wc is about 5mm. The corresponding local incidence angle of the central axis Φ1=Φ2=57.75°. If the material of thespecial spacer 618 has a refractive index ns of about 1.4 and Hs is 1.2mm, the detection module is tilted at Φ1=70°. The effective sensing area width increases above 6.5 mm. Under these parameters, the detection angle extent in the cover glass was reduced to 19 °. Thus, the imaging system for the optical fingerprint sensor module may be designed to desirably enlarge the size of thesensing region 615 on the toptransparent layer 431.

Refractive index R of particular spacer 618I is designed to be low enough (e.g., using MgF₂ 、CaF₂ Or even air to form spacers), the width Wc of theeffective sensing region 615 is no longer limited by the thickness of thecover glass 431 and thedisplay module 433. This property provides the desired design flexibility. In principle, the effective sensing area may even be increased to cover the entire display screen if the detection module has sufficient resolution.

Because the disclosed optical sensor technology may be used to provide a large sensing area to collect patterns, the disclosed under-screen optical fingerprint sensor module may be used not only to collect and detect patterns of fingers, but also to collect and detect patterns of a larger size, such as a person's palm associated with a person, for user authentication.

Fig. 8A-8B illustrate an example of another design consideration for the optical imaging design of the optical fingerprint sensor module shown in fig. 7 by providing a detection angle θ' of the photodetector array relative in the display screen surface and a distance L between thelens 621 and thespacer 618. Fig. 8A shows a cross-sectional view along a direction perpendicular to the surface of the display screen, and fig. 8B shows a view of the device from the bottom or top of the display screen. Afill material 618c may be used to fill the space between thelens 621 and thephotodetector array 623. For example, thefiller material 618c may be the same material as theparticular spacer 618 or a different material. In some designs, thefiller material 618c may be an air gap.

Fig. 9 shows another example of an off-screen optical fingerprint sensor module based on the design of fig. 7, in which one ormore illumination sources 614 are provided to illuminate the topsurface sensing region 615 for optical fingerprint sensing. Theillumination source 614 may be an extended type or a collimated type source such that all points within theeffective sensing region 615 are illuminated. Theillumination source 614 may be a single element light source or an array of light sources.

Fig. 10A-10B illustrate examples of an off-screen optical fingerprint sensor module usingoptical couplers 628 in the shape of thin wedges to improve optical detection at theoptical sensor array 623. Fig. 10A shows a cross section of a device structure with an off-screen optical fingerprint sensor module for fingerprint sensing, and fig. 10B shows a top view of a device screen. An optical wedge 628 (having a refractive index ns) is positioned below the display panel structure to change the total reflection condition on the bottom surface of the display panel structure to which theoptical wedge 628 is engaged, thereby allowing light to be extracted from the display panel structure through the bottom surface. Theoptical sensor array 623 receives the light extracted from the display panel structure from theoptical wedge 628, and anoptical imaging module 621 is positioned between theoptical wedge 628 and theoptical sensor array 623 to image the light from theoptical wedge 628 onto theoptical sensor array 623. In the example shown,optical wedge 628 includes an optical wedge face that is angled toward optical imaging module andoptical sensing array 623. Further, as shown, there is an idle space betweenoptical wedge 628 andoptical imaging module 621.

The reflectance is 100% with the highest efficiency if the light is totally reflected at the sensing surface of thecover glass 431. However, if the light is parallel to the cover glass surface, the light is also totally reflected at theLCD bottom surface 433 b. Thewedge coupler 628 is used to change the local surface angle so that light can be coupled out for detection at theoptical sensor array 623. The micro-holes in theLCD display module 433 provide a desired light propagation path such that light is transmitted through theLCD display module 433 for off-screen optical sensing. The actual light transmission efficiency may gradually decrease if the light transmission angle becomes too large or when the TFT layer becomes too thick. When the angle is close to the total reflection angle, i.e. about 41.8 deg., and the cover glass refractive index is 1.5, the fingerprint image appears to be good. Accordingly, the wedge angle of thewedge coupler 628 may be adjusted to several degrees, so that the detection efficiency may be improved or optimized. If the refractive index of the cover glass is selected to be higher, the total reflection angle becomes smaller. For example, if the cover glass is made of sapphire having a refractive index of about 1.76, the total reflection angle is about 34.62 °. The transmission efficiency of the detection light in the display screen is also improved. Therefore, this design uses thin wedges to set the detection angle higher than the total reflection angle, and/or uses a cover glass material with a high refractive index to improve detection efficiency.

In some under-screen optical fingerprint sensor module designs (e.g., the designs shown in fig. 6A-6C, 7, 8A, 8B, 9, 10A, and 10B), thesensing region 615 on the top transparent surface is not perpendicular or orthogonal to thedetection axis 625 of the optical fingerprint sensor module, such that the image plane of the sensing region is also not vertical or perpendicular to thedetection axis 625. Thus, the plane of thephotodetector array 623 may be tilted with respect to thedetection axis 625 to achieve high quality imaging at thephotodetector array 623.

Fig. 11A-11C show three example configurations for such tilting. Fig. 11A shows thatsensing region 615a is tilted and not perpendicular todetection axis 625. In fig. 11B, sensing region 615B is aligned abovedetection axis 625 such that its image plane will also lie ondetection axis 625. In practice, thelens 621 may be partially cut out to simplify packaging. In various implementations,microlenses 621 may also be transmissive or reflective lenses. For example, a particular method is shown in fig. 11C. Thesensing region 615c is imaged byimaging mirror 621 a. Thephotodiode array 623b is aligned to detect a signal.

In the above-describeddesigns using lenses 621,lenses 621 may be designed to have an effective aperture that is larger than the aperture of the holes in the LCD display layer, allowing light to pass through the LCD display module for optical fingerprint sensing. Such a design may reduce the undesirable effects of wiring structures and other scattering objects in the LCD display module.

Fig. 12 illustrates an example of the operation of a fingerprint sensor for reducing or eliminating unwanted contributions from background light in fingerprint sensing. The optical sensor array may be used to collect various frames and the collected frames may be used to perform differential and average operations between frames to reduce the effects of background light. For example, in frame a, the illumination light source for optical fingerprint sensing is turned on to illuminate the finger touch area, and in frame B, the illumination is changed or turned off. The subtraction of the signal of frame a and the signal of frame B may be used in image processing to reduce the undesirable effects of background light.

Undesired background light in fingerprint sensing may also be reduced by providing suitable optical filtering in the optical path. One or more optical filters may be used to filter ambient light wavelengths, such as near infrared IR light and a portion of red light, etc. In some implementations, such optical filter coatings may be fabricated on surfaces of optical components, including display screen floors, prismatic surfaces, or sensor surfaces, among others. For example, a human finger may absorb a majority of energy below about 580nm in wavelength, and if one or more optical filters or optical filter coatings may be designed to reject light having wavelengths from 580nm to infrared, the undesirable contribution of ambient light to optical detection in fingerprint sensing may be greatly reduced.

Fig. 13 shows an example of an operation procedure for correcting image distortion in an optical fingerprint sensor module. Atstep 1301, one or more illumination sources are controlled and operated to emit light in a particular region, and the light emission of such pixels is modulated by frequency F. Atstep 1302, an imaging sensor under the display panel is operated to acquire images at the same frequency F and a certain frame rate. In the optical fingerprint sensing operation, a finger is placed on the top surface of the display panel cover substrate, and the finger modulates the light reflection intensity of the top surface of the display panel cover substrate. An imaging sensor under the display screen collects the reflected light pattern after fingerprint modulation. Atstep 1303, demodulation of the signal from the image sensor is synchronized at frequency F, and background subtraction is performed. The resulting image reduces the effects of background light and includes images resulting from the emitted light from the pixels. Atstep 1304, the acquired image is processed and calibrated to correct image system distortions. Atstep 1305, the corrected image is used as a human fingerprint image for user authentication.

The same optical sensor used to capture the user's fingerprint may also be used to capture scattered light from an illuminated finger, such asbackscattered light 191 shown in fig. 5A. The detector signals from the backscattered light 191 in fig. 5A in the region of interest may be integrated to produce an intensity signal. The intensity variation of the intensity signal is evaluated to determine other parameters than the fingerprint pattern, such as the heart rate of the user or the internal topological organization of the finger associated with the external fingerprint pattern.

The fingerprint sensor may be hacked by a malicious individual who can obtain the fingerprint of an authorized user and copy the stolen fingerprint pattern on a carrier similar to a human finger. Such an unauthorized fingerprint pattern may be used on a fingerprint sensor to unlock a target device. Thus, while the fingerprint pattern is a unique biometric identifier, it may not itself be a completely reliable or secure identification. The off-screen optical fingerprint sensor module may also be used as an anti-spoof sensor for sensing whether an input object having a fingerprint pattern is a finger from a living person and for determining whether the fingerprint input is a fingerprint spoof attack. Such anti-spoofing sensing functionality is provided without the use of a separate optical sensor. The spoofing can provide a high-speed response without affecting the overall response speed of the fingerprint sensing operation.

Fig. 14 shows an exemplary optical extinction coefficient of a monitored material in blood, wherein the optical absorption is different between a visible spectral range of red light, e.g., 660nm, and an infrared range of infrared IR light, e.g., 940 nm. By illuminating the finger with detection light at a first visible wavelength (color a) and a second, different wavelength (color B), such as an Infrared (IR) wavelength, the difference in optical absorption of the input object can be collected to determine whether the touching object is from a live finger. The one or more illumination sources for providing optical sensing illumination may be used to emit light of different colors, and the emitted light is probe light or illumination light of at least two different optical wavelengths for in vivo finger detection using different optical absorption behaviors of blood. When a human heart beats, pulse pressure pumps blood to flow in an artery, so that the extinction ratio of monitored material in the blood varies with the pulse. The received signal carries a pulse signal. These properties of blood can be used to detect whether the monitored material is a live or fake fingerprint.

Fig. 15 shows a comparison between optical signal behavior in reflected light from inanimate material (e.g., a fake finger or a spoof device with a fake fingerprint pattern) and a live finger. The optical fingerprint sensor may also be operated as a heartbeat sensor to monitor the living body. When two or more wavelengths of probe light are detected, the difference in extinction ratio can be used to quickly determine whether the monitored material is a living body, such as a living body fingerprint. In the example shown in fig. 15, probe light of different wavelengths is used, one being visible wavelength and the other being infrared IR wavelength as shown in fig. 14.

When the inanimate material touches the top cover glass over the fingerprint sensor module, the received signal reveals an intensity level associated with the surface pattern of the inanimate material and the received signal does not contain a signal component associated with a live human finger. However, when a live finger touches the top cover glass, the received signal reveals signal characteristics associated with the live, including significantly different intensity levels, because the extinction ratios of the different wavelengths are different. This method does not take a long time to determine whether the touch material is part of a living person. In fig. 15, the pulse-like signal reflects multiple touches, not blood pulsation. Similar multiple touches of inanimate material do not show differences caused by a living finger.

Such optical sensing of different optical absorption behavior of blood at different optical wavelengths may be performed in a short period for living finger detection and may be faster than optical detection of a person's heartbeat using the same optical sensor.

In an LCD display screen, the LCD backlight illumination light is white light, thereby containing light in the visible spectrum and infrared IR spectral ranges for performing the above-described living finger detection at the optical fingerprint sensor module. The LCD filter in the LCD display module may be used to allow the optical fingerprint sensor module to obtain the measurements in fig. 14 and 15. In addition, the designatedlight source 436 for generating optically sensed illumination light may be operated to emit detection light at selected visible and infrared IR wavelengths at different times, and the reflected detection light of two different wavelengths is collected by theoptical detector array 623 to determine whether the touching object is a living finger based on the above-described operations shown in fig. 14 and 15. It is noted that while the reflected probe light of the selected visible wavelength and the IR wavelength at different times may reflect different optical absorption characteristics of the blood, the fingerprint image is always acquired by both the probe light of the selected visible wavelength and the probe light of the IR wavelength at different times. Thus, fingerprint sensing may be performed at both the visible and IR wavelengths.

Fig. 16 shows an example of an operation procedure for determining whether an object in contact with an LCD display screen is part of a live human finger by operating one or more illumination light sources for optical sensing to illuminate the finger with two different colors.

As another example, the disclosed optical sensor technology may be used to detect whether a pattern of an acquired or detected fingerprint or palm is from a live human hand by a "live finger" detection mechanism by other mechanisms besides the different optical absorption of blood at different optical wavelengths described above. For example, living fingers are typically moving or stretching due to the natural movement or motion (intentional or unintentional) of a person, or are typically pulsed as blood flows through the body in connection with a heartbeat. In one implementation, the optical fingerprint sensor module may detect a change in return light from a finger or palm due to a change in heartbeat/blood flow, thereby detecting whether a living heartbeat is present in an object that appears as a finger or palm. User authentication may enhance access control based on a combination of optical sensing of fingerprint/palm patterns and positive determination of the presence of a living person. As another example, when a person touches an LCD display screen, the change in touch force may be reflected in one or more ways, including fingerprint pattern distortion, a change in contact area between a finger and the screen surface, a widening of the fingerprint ridge, or a dynamic change in blood flow. These and other variations may be measured by optical sensing based on the disclosed optical sensor technology and may be used to calculate touch force. Such touch force sensing may be used to add more functionality to the optical fingerprint sensor module than fingerprint sensing.

In the example above where the fingerprint pattern is acquired on the optical sensor array by the imaging module, as shown in fig. 4B and 6B, optical distortion typically reduces image sensing fidelity. Such optical distortions can be corrected in various ways. For example, an optical image may be generated at the optical sensor array using a known pattern, and image coordinates in the known pattern may be correlated with the optical image generated at the optical sensor array with distortion for calibrating imaging sensing signals output by the optical sensor array for fingerprint sensing. The fingerprint sensing module calibrates the output coordinates with reference to the image of the standard pattern.

In accordance with the disclosure in this patent document, various implementations may be made for the disclosed optical fingerprint sensor module. For example, the display panel may be configured to: each pixel emits light and can be controlled individually; the display panel includes an at least partially transparent substrate; a substantially transparent cover substrate. The optical fingerprint sensor module is placed under the display panel for sensing an image formed on top of the surface of the display panel. The optical fingerprint sensor module may be used to sense an image formed by light emitted from the display panel pixels. The optical fingerprint sensor module may include a transparent block having a refractive index lower than that of the display panel substrate, an imaging sensor block having an imaging sensor array, and an optical imaging lens. In some implementations, the low refractive index block has a refractive index in the range of 1.35 to 1.46 or 1 to 1.35.

As another example, a method may be provided for fingerprint sensing, in which light emitted from a display panel is reflected by a cover substrate, and a finger located on top of the cover substrate interacts with the light to modulate a light reflection pattern of the fingerprint. The imaging sensing module below the display panel is used for sensing the reflected light pattern image and reconstructing a fingerprint image. In one implementation, the emitted light from the display panel is modulated in the time domain and the imaging sensor is synchronized with the modulation of the light emitting pixels, where the demodulation process rejects most of the background light (future light from the target pixel).

As described above, the display screen of a portable electronic device is generally implemented as a component having multiple layers. For example, a display screen implemented as a touch screen may include a display layer for outputting video data, a capacitive touch screen layer for detecting touch events, a hard top layer, and the like. Additional layers are used to integrate an off-screen optical sensing function, such as fingerprint sensing. To get the light to the sensing assembly, the light passes through the various layers between the top surface and the sensor (e.g., photodetector). To this end, the layers are designed to allow transmission of light, and some layers may be designed to enhance, bend, focus, collimate, reflect, and/or otherwise affect the transmission of light through the layers.

Fig. 17A and 17B show cross-sections of an illustrative portableelectronic device 1700 and anillustrative display module 1710 for such a portableelectronic device 1700, respectively, in accordance with various embodiments. Portableelectronic device 1700 is shown as a smart phone. In other implementations, the portableelectronic device 1700 is a notebook computer, tablet computer, wearable device, or any other suitable computing platform. The portableelectronic device 1700 may include adisplay system 423. As described above, thedisplay system 423 may be a touch-sensing display system 423. Thedisplay system 423 has an off-screen optical sensor integrated therein. As shown, the off-screen optical sensor may define asensing region 615 within which optical sensing may be performed. For example, when a user placesfinger 445 on the display screen withinsensing region 615, a fingerprint scan may be performed by an off-screen optical sensor. Such an off-screen optical sensor may be implemented using multiple layers.

Thedisplay module 1710 of fig. 17B may be one implementation of thedisplay system 423 of fig. 17A. As shown, thedisplay module 1710 includes multiple layers. The top cover sheet 1715 (e.g., glass) may be used as a user interface surface for various user interface operations. For example, thecover layer 1715 may facilitate touch sensing operations for a user, display images to a user, serve as an optical sensing interface to receive fingers for optical fingerprint sensing and other optical sensing operations, and so forth. In some embodiments, thedisplay module 1710 includes acover layer 1715. In other implementations, thecover layer 1715 is separate from thedisplay module 1710. For example, thedisplay module 1710 is integrated as a module into the portableelectronic device 1700, and thecover layer 1715 is mounted on top of thedisplay module 1710.

One or more other layers of thedisplay module 1710 form a liquid crystal module (liquid crystal module, LCM) 1720. BelowLCM 1720,display module 1710 includes anenhancement layer 1725. As described herein, theenhancement layer 1725 may include one or more layers of brightness enhancement film, such as enhancement film comprising a trapezoidal prism structure. Thedisplay module 1710 may also include some or all of alight diffuser 1730, alight guide plate 1735, areflector film 1740, and aframe 1745. Additional components are included in some embodiments, such as one or moredisplay light sources 1750, and one or more external light sources 1760 (e.g., for fingerprint sensing and/or other optical sensing).

Implementations of thedisplay light source 1750 may include an LCD display screen backlight source (e.g., LED lamp) that provides a white backlight for thedisplay module 1710. Implementations oflight guide plate 1735 include a waveguide optically coupled to displaylight source 1750 to receive and guide backlight. Implementations ofLCM 1720 include some or all of a Liquid Crystal (LC) cell layer, LCD electrodes, transparent conductive ITO layers, optical polarizer layers, color filter layers, touch sensing layers, and the like. Implementations oflight diffuser 1730 include a backlight diffuser positioned belowLCM 1720 and abovelight guide plate 1735 to spatially disperse the backlight to illuminate LCD display pixels inLCM 1720. Thereflector film 1740 is implemented to be placed under thelight guide plate 1735 to recycle backlight to theLCM 1720, thereby improving light utilization efficiency and display brightness.

When the LCD cell is turned on (e.g., in the sensing region 615), the LCM 1720 (e.g., LC cell, electrode, transparent ITO, polarizer, color filter, touch sensing layer, etc.) may become partially transparent, although the microstructure may interfere with and/or block some of the detected light energy. Embodiments of thelight diffuser 1730,light guide plate 1735,reflector film 1740, andframe 1745 are treated to secure the fingerprint sensor and provide a transparent or partially transparent sensing light path so that a portion of the reflected light from the top surface ofcover plate layer 1715 may reach the sensing elements (e.g., photodetector array) of the off-screen optical sensor. The off-screen optical sensor may include any suitable components, such as a fingerprint sensor component, a photodetector array, an optical collimator array for collimating and directing reflected probe light to the photodetector array, and an optical sensor circuit that receives and conditions detector output signals from the photodetector array. Embodiments of the photodetector array include a CMOS sensor comprised of CMOS sensing pixels, a CCD sensor array, or any other suitable optical sensor array.

Embodiments of theenhancement layer 1725 include one or more enhancement films. Some reinforced film designs include prismatic films having sharp prismatic ridges and sharp prismatic valley profiles (i.e., sharp transitions at each ridge and sharp transitions at each valley). 18A-18C show views of an illustrative portion of thesymmetric enhancement layer 1800. Fig. 18A shows anenlarged view 1810 of a small portion of thesymmetric enhancement layer 1800. Fig. 18B shows a cross-section of a small portion of oneenhancement film layer 1820 of thesymmetric enhancement layer 1800. Fig. 18C shows a cross-section of a small portion of two reinforcedfilm layers 1820a, 1820b stacked in a direction orthogonal relative to each other in a symmetrical reinforcedlayer 1800.

As shown, eachenhancement film layer 1820 is formed with a series of sharp prismatic structures. Each sharp prism structure includessharp ridges 1822 andsharp valleys 1824. Theenlarged view 1810 of fig. 18A shows two enhancement film layers 1820 of fig. 18C stacked in orthogonal directions relative to each other as viewed from the top. As shown, the intersecting sharp prism structures form a grid ofsharp ridges 1812 andsharp valleys 1814, corresponding to thesharp ridges 1822 andsharp valleys 1824, respectively, of each sharp prism structure. As shown in fig. 18B, thesharp ridge 1822 points in the direction of theLCM 1720.

Such enhancement layers 1800 generally attempt to enhance the brightness of light toward the viewer (e.g., toward and/or through the LCM 1720). For example, theenhancement layer 1800 attempts to enhance the brightness of the backlight behind theLCM 1720 and/or the brightness of the probe illumination for off-screen optical sensing. As shown in fig. 18B, light passing through the prismatic structures of theenhancement layer 1800 is bent in different directions, as shown bylight paths 1832a and 1832B. Each optical path 1832 shows two directions of propagation of optical energy. The first direction (i.e., generally toward LCM 1720) may represent a vector of backlight and/or probe illumination energy originating from a light source belowenhancement layer 1800. The second direction (i.e., generally away from the LCM 1720) may represent a vector of backlight and/or detection illumination energy reflected off another layer (e.g., a top transparent layer above the LCM 1720) and traveling back in the direction of the off-screen optical sensor.

Such bending may tend to be beneficial as light passes through theenhancement film layer 1820 in the direction of theLCM 1720. For example, light passing through theenhancement film layer 1820 in a first direction, including an optical path having a large angle of incidence (e.g., path 1832), may be bent toward theLCM 1720 in a generally converging manner, thereby causing brightness enhancement. As a matter of course, light passing through theenhancement layer 1800 in the second direction may tend to bend in a generally divergent manner. If optical sensing is attempted, image blurring may result. In typical display applications, this blurring is insignificant as the blurred light is entering the device rather than being directed toward the viewer. However, as described herein, in the context of off-screen optical fingerprint sensing, such blurring affects light propagating in the direction of the optical sensing component, which can hinder optical sensing by components below theconventional enhancement layer 1800.

For greater clarity, fig. 18B shows three exemplary potential reference positions (e.g., positions and orientations) 1850 for optical sensing. If an optical sensor is placed according toreference position 1850a, the optical sensor may tend to detect light enteringenhancement film layer 1820 through the left and right prism faces, which may tend to cause image blurring. However, if the optical sensor is placed according toreference position 1850b or 1850c, the optical sensor may be prone to detect light enteringenhancement film layer 1820 through only the right or left prism face, respectively. In this case, blurring may be avoided, but at least half of the detection area may not be imaged.

Fig. 19A-19C show views of illustrative portions ofasymmetric enhancement layer 1900 in accordance with various embodiments.Asymmetric enhancement layer 1900 may be another embodiment ofenhancement layer 1725. Fig. 19A shows anenlarged view 1910 of a small portion of anasymmetric enhancement layer 1900. Fig. 19B shows a cross-section of a small portion of one reinforcingfilm layer 1920 of asymmetric reinforcinglayer 1900. Fig. 19C shows a cross-section of a small portion of twoasymmetric layers 1920a, 1920b stacked in a direction orthogonal relative to each other in anasymmetric reinforcement layer 1900.

As shown, each reinforcingfilm layer 1920 is formed with a series of asymmetric prismatic structures. Each asymmetric prismatic structure (microprism structure) is generally defined by a cross section having two angled sides, formingsharp ridges 1922 andsharp valleys 1924. As shown, each of the two angled sides is inclined at a respective different angle of inclination 1926 relative to vertical. Notably, at each limit of the range of possible tilt angles 1926 is an embodiment in which one tilt angle 1926 is substantially zero degrees to effectively form a sawtooth prism structure. In another embodiment, one tilt angle 1926 is 45 degrees and the other is 52 degrees. In another embodiment, one tilt angle 1926 is 45 degrees and the other is 54 degrees. In another embodiment, one tilt angle 1926 is 45 degrees and the other is 56 degrees. In another embodiment, one tilt angle 1926 is 38 degrees and the other is 52 degrees. In another embodiment, one tilt angle 1926 is 36 degrees and the other is 54 degrees. As described herein, the tilt angle 1926 is selected to provide a desired type and/or amount of brightness enhancement (e.g., for backlighting through theenhancement film layer 1920 in the direction of the LCM 1720).

Theenlarged view 1910 of fig. 19A shows two enhancement film layers 1920 of fig. 19C stacked in orthogonal directions relative to each other as viewed from the top. As shown, the intersecting prism structures form a grid ofsharp ridges 1912 andsharp valleys 1914, corresponding to thesharp ridges 1922 andsharp valleys 1924, respectively, of each sharp prism structure. This arrangement makes the top view look similar to the top view of thereinforcement layer 1800 of fig. 18, but provides various features that differ from theconventional reinforcement layer 1800.

Fig. 19B shows light propagating throughenhancement film 1920 in the direction ofLCM 1720, e.g., along optical path 1930. Light that passes throughenhancement film layer 1920 generally in the direction of LCM 1720 (i.e., having an upward directional component with reference to the direction shown), such as those alongoptical paths 1930a and 1930b, is bent toward the vertical direction by the angled surfaces of the microprisms. Thus, although some light paths are affected differently by the asymmetric prism structures than by the microprismatic structures of thesymmetric enhancement layer 1800, the asymmetricenhancement film layer 1920 still provides backlight enhancement features.

Unlike thesymmetric enhancement layer 1800, the asymmetricenhancement film layer 1920 produces less blur for light traveling in a direction opposite the LCM 1720 (i.e., having a downward directional component with reference to the illustrated direction). Fig. 19B shows light propagating throughenhancement film 1920 in such a direction (e.g., the direction of an off-screen optical sensor), for example, along optical path 1940. As shown, three objects 1950 are positioned at different locations relative to asymmetricreinforcement film layer 1920. For example, object 1950 is a fingerprint ridge or valley of a finger placed on a fingerprint sensing area of a device having an asymmetricenhancement film layer 1920, the asymmetricenhancement film layer 1920 disposed betweenLCM 1720 and an off-screen optical fingerprint sensor. Light from the second object 1950B propagates along the refractedlight path 1940a to detection point "B"1955B (e.g., corresponding to a first potential sensor location), while light from the third object 1950C propagates along the refracted light path 1940B to detection point "C"1955C (e.g., corresponding to a second potential sensor location). It is noted that althoughobjects 1950b and 1950c are relatively close, theirrespective detection points 1955b and 1955c are relatively far apart. After exitingasymmetric enhancement film 1920 in a substantially perpendicular direction, light fromfirst object 1950a propagates along refractiveoptical path 1945 to detection point "a"1955a. It can be seen that by configuring the sensor to detect light exiting along path 1945 (e.g., atdetection location 1955 a), relatively clear and bright detection information can be produced. This is further illustrated in fig. 19C, where two stacked asymmetric enhancement film layers 1920 (in a direction orthogonal relative to each other) may provide a clear image light path, for example represented bydetection point 1955a.

In addition, as discussed above with reference to fig. 18A-18C, avoiding blurring may involve positioning the optical sensor to receive from only one of the left or right sides of the microprism structure. For example, a single optical sensor may be directed to detectobject 1950a and object 1950b throughpaths 1945 and 1940a, respectively, as both paths reach the sensor through the right prism face. Furthermore, receiving optical information from only one face of the microprism structure can effectively reduce the sensing area that can be imaged practically and accurately without blurring.

For illustration, FIG. 20 shows an illustrativefingerprint sensing region 615 overlaid on a singleenhancement film layer 2000, such as enhancement film layers 1820 or 1920. As shown, the microprismatic structure of theenhancement film layer 2000 forms parallel ridgelines extending generally in a first direction (labeled "X" axis 2020) and extending in a second direction (labeled "Y" axis 2025) generally orthogonal to the first direction. Although not shown, it may be assumed that the "Z" axis points outward from the page in the direction ofLCM 1720, orthogonal to both the X and Y axes. As described above, while the theoreticalfingerprint sensing area 615 occupies a relatively large sensing area, an optical sensor oriented to sense the entire area may tend to receive reflected probe light from both prism faces of each micro-prism structure, resulting in image blurring. Optical sensors oriented to receive reflected probe light from only one prism face of each microprism structure can be prone to avoid image blurring, but also prone to reduced sensing area (as shown by sensing sub-area 2010). For example, the optical sensor may be oriented to generally point in the negative Y-direction, the positive Z-direction (e.g., {0, -Y, Z }). In this case, theactual sensing sub-region 2010 may be about half or less of the area of thetheoretical sensing region 615.

Fig. 21 shows an illustrativefingerprint sensing area 615 overlaid on a reinforcedpanel 2100 having two reinforced film layers stacked such that their respective microprismatic structures extend in a generally orthogonal direction. Thereinforcement panel 2100 may be implemented as, for example, the reinforcement panel shown in fig. 18A and 19A or a pair of reinforcement film layers shown in fig. 18C and 19C. As shown, the reinforcedpanel 2100 has a first micro-prismatic structure (e.g., belonging to a first reinforced film layer) that forms parallel ridgelines extending generally in a first direction (along the "X" axis 2020) and a second micro-prismatic structure (e.g., belonging to a second reinforced film layer) that forms parallel ridgelines extending generally in a second direction (along the "Y" axis 2025). By two layers of reinforced film, each layer may cause blurring. For example, light passing through one prism face of the microstructure of the lower enhancement film layer is a combination of light passing through both prism faces of the microstructure of the upper enhancement film layer. Thus, to avoid blurring, the optical sensor is oriented to receive reflected probe light from only one prism face of each microprism structure through two enhancement film layers. For example, the optical sensor may be oriented to point generally in the positive X, negative Y, positive Z directions (e.g., { X, -Y, Z }). In this case, theactual sensing sub-region 2110 may be about one-fourth of the area of thetheoretical sensing region 615. For example, when symmetric microprismatic structures (e.g., those shown in fig. 18A-18C) are used, theactual sensing sub-region 2110 may be less than one-fourth of the area of thetheoretical sensing region 615, while when well-designed asymmetric microprismatic structures (e.g., those shown in fig. 19A-19C) are used, theactual sensing sub-region 2110 may be slightly greater than one-fourth of the area of thetheoretical sensing region 615.

Increasing the area of thetheoretical sense region 615 may offset the limitation of theactual sense sub-region 2110 of reduced area. For example, doubling the area of thetheoretical sense region 615 may double the area of theactual sense sub-region 2110. One way to increase the area of thetheoretical sensing region 615 is to increase the distance between the object being detected (e.g., a fingerprint feature) and the lens at the input of the optical sensor. However, directly increasing such distance may involve increasing the thickness of the optical sensor, which is impractical in many applications. For example, in smart phone applications, it is desirable to maintain a thin package while still achieving a large sensing area. Some embodiments attempt to effectively increase thetheoretical sensing area 615 by using refraction and reflection to create a folded optical path without a corresponding increase in sensor thickness.

FIG. 22 shows an illustrative under-screenoptical sensing environment 2200 in accordance with various embodiments. As shown,environment 2200 includes an under-screen optical sensing system disposed belowdisplay module 1710. The optical sensing system includes anoptical sensing module 2210, arefractive structure 2220, and areflective structure 2230. Theoptical sensing module 2210 may include any suitable components for optical sensing. For simplicity, theoptical sensing module 2210 is shown as anoptical detector 2212 and alens 2214. For reference, thedisplay module 1710 is shown as defining adisplay plane 2205. For example, the display surface of thedisplay module 1710 is generally flat (e.g., although it may include a framed edge, a rounded edge, etc.), such that the display surface is generally located in thedisplay plane 2205. Thedisplay module 1710 may be any suitable type of display screen and may include one or more layers. In some embodiments, as described above, thedisplay module 1710 is a Liquid Crystal Display (LCD) module having multiple layers including LCD layers (e.g., including an LCD pixel array, electrode interconnects, etc.).

Theoptical sensing module 2210 is configured to obtain optical information from the received illumination energy. For example, as light entersoptical sensing module 2210 throughinput aperture 2216, the light is focused bylens 2214 ontooptical detector 2212, andoptical detector 2212 may include an array of photodetectors and/or any other suitable components. The received illumination energy may then be converted into optical information. For example, the conversion may include any suitable optical processing (e.g., using lenses, filters, modulators, masks, etc.) and/or any suitable logic processing (e.g., using a computing processor, state machine, software, etc.). As shown, theinput aperture 2216 may be oriented substantially parallel to thedisplay plane 2205. In this configuration, the optical path propagating parallel to thedisplay plane 2205 may enter the center of theinput aperture 2216 coaxially with thelens 2214.

Thereflective structure 2230 can receive reflected probe light that passes through the display module 1710 (e.g., within the optical sensing region 615). In some embodiments, thereflective structure 2230 includes a mirror that is integrated with (e.g., attached to) the topreflective surface 2235. For example, the reflectingstructure 2230 includes structures that support and orient the mirrors at appropriate angles for redirecting the optical path 2240 of the reflected probe light toward therefractive structure 2220. In other embodiments, thereflective structure 2230 is a unitary prism body having an angledtop surface 2235. In such an embodiment, the unitary prism body may be made of a material (e.g., plastic, glass, etc.) having a selected refractive index such that reflected probe light incident on the angledtop surface 2235 after passing through thedisplay module 1710 is reflected toward therefractive structure 2220. As shown, thereflective structure 2230 is sized and oriented to redirect the optical path 2240 of reflected probe light from the entireoptical sensing zone 615 to therefractive structure 2220. In some implementations, thereflective structure 2230 is sized and oriented to redirect the optical path 2240 of reflected probe light from only a portion of theoptical sensing region 615 and/or otherwise from a region outside of theoptical sensing region 615.

An embodiment ofrefractive structure 2220 receives reflected probe light fromreflective structure 2230 and bends optical path 2240 of the reflected probe light to converge oninput aperture 2216 ofoptical sensing module 2210. For example, as shown,optical paths 2240a, 2240b, and 2240c originate from the leftmost, center, and rightmost edges, respectively, ofoptical sensing zone 615, thereby representing optical path 2240 from the entireoptical sensing zone 615. Although covering a largeroptical sensing region 615, the optical path 2240 is redirected all by thereflective structure 2230 to therefractive structure 2220, and is bent all by therefractive structure 2220 to converge on theinput aperture 2216 of theoptical sensing module 2210. In some embodiments,refractive structure 2220 is a unitary prism body having a first refractive surface (e.g., farther from optical sensing module 2210) to receive reflected probe light from the reflective structure and a second refractive surface (e.g., closer to optical sensing module 2210) to transmit the reflected probe light to an input aperture of the optical sensing module, the first refractive surface being at an angle relative to the second refractive surface. For example, as shown, the second refractive surface may be oriented substantially orthogonal to the display plane.

As described above (although not shown in fig. 22), embodiments may include an illumination source and a top transparent layer. An illumination source may be disposed below thedisplay module 1710 to generate detection light and direct the detection light through at least a portion of thedisplay module 1710. For example, the detection light source may include a light emitting diode (light emitting diode, LED), a vertical cavity surface emitting laser (vertical cavity surface emitting laser, VCSEL), or any other suitable light source. A top transparent layer may be disposed over thedisplay module 1710 and configured as an output interface for images produced by thedisplay module 1710. In some implementations, thedisplay module 1710 also includes touch sensitive features, and the top transparent layer can also be configured as an input interface for touch sensitive interactions. Although theoptical sensing region 615 is shown as being located substantially in thedisplay plane 2205, theoptical sensing region 615 may also be defined with reference to the top surface of the top transparent layer. For example, when theoptical sensing module 2210 is configured for off-screen optical fingerprint sensing, the fingerprint is typically placed on the top surface of the top transparent layer such that theoptical sensing region 615 more directly corresponds to the region in the top surface of the top transparent layer where optical fingerprint sensing may be performed. Theoptical sensing region 615 may then be a region configured to receive detection light from an illumination source and reflect a portion of the detection light in response to an interaction between an object (e.g., a fingerprint feature) and the top surface. For example, as described above, the top transparent layer may be made of a material (e.g., treated glass) having a particular refractive index relative to air and/or fingerprint features or other objects above the top transparent layer. Thus, the probe light will be prone to be reflected or not reflected at the top surface of the top transparent layer (e.g., fingerprint ridges will cause the probe light to be reflected, while fingerprint valleys will cause the probe light to be absorbed, scattered, etc.), depending on whether an object is present. In such an embodiment, the reflected probe light redirected by thereflective structure 2230, bent by therefractive structure 2220, and received by theoptical sensing module 2210 is probe light reflected by the top surface of the top transparent layer.

In some embodiments, thereflective structure 2230 andrefractive structure 2220 are configured (e.g., sized, shaped, positioned, and oriented) such that one of theoptical paths 2240b through the center of theoptical sensing zone 615 is redirected by thereflective structure 2230 and bent by therefractive structure 2220 to enter the center of theinput aperture 2216 of the optical sensing module. For example, thelens 2214 is a convex lens having a main optical axis, and theinput aperture 2216 is aligned with thelens 2214 such that the center of theinput aperture 2216 is aligned with the main optical axis of thelens 2214. In some embodiments, as shown, thereflective structure 2230 and therefractive structure 2220 are configured such that one of theoptical paths 2240b through the display module in a first direction substantially perpendicular to the display plane (e.g., perpendicular relative to the illustration) is redirected by thereflective structure 2230 and bent by therefractive structure 2220 to enter theinput aperture 2216 of theoptical sensing module 2210 in a second direction substantially parallel to the display plane (e.g., horizontal relative to the illustration).

FIG. 23 shows an illustrative under-screenoptical sensing environment 2300 with off-axis sensing, in accordance with various embodiments. Similar toenvironment 2200 of fig. 22,environment 2300 includes an under-screen optical sensing system disposed belowdisplay module 1710. The optical sensing system includes anoptical sensing module 2210, arefractive structure 2220, and a reflective structure 2230 (having a reflective top surface 2235). Theoptical sensing module 2210 may include any suitable components for optical sensing, such as anoptical detector 2212 and alens 2214. Unlike fig. 22,environment 2300 of fig. 23 shows an actualoptical sensing sub-region 2310 that is smaller than theoreticaloptical sensing region 615, such as described with reference to fig. 20 and 21.

As described above, embodiments of thedisplay module 1710 may include an enhancement layer having a microprismatic structure. For example, each microprism structure may have at least first and second prism faces, and the first and second prism faces may be symmetrical or asymmetrical with respect to each other. When the reflected probe light passes through the microprism structure, some of the reflected probe light passes through a first prism face of the microprism structure and some of the reflected probe light passes through a second prism face of the microprism structure. For example, a first portion of light path 2340 that reflects probe light passes through a first prism face of the microprism structure and a second portion of light path 2340 that reflects probe light passes through a second prism face of the microprism structure such that the first and second portions of light path 2340 diverge. As described above, ifoptical sensing module 2210 receives both the first and second portions of optical path 2340, such divergence may cause blurring. Instead, embodiments may configure theoptical sensing module 2210 to obtain optical information from received illumination energy corresponding to only a first portion of the optical path 2340 of the reflected probe light. Thus, the actualoptical sensing region 2310 is smaller than the theoretical optical sensing region 615 (e.g., approximately half or one-quarter area).

In some such embodiments, the reinforcement layer includes a first reinforcement film layer and a second reinforcement film layer. The first enhancement film layer has a first portion of microprismatic structure arranged to form a first parallel prism ridge extending in a first direction and the second enhancement film layer has a second portion of microprismatic structure arranged to form a second parallel prism ridge extending in a second direction different from the first direction. For example, as shown in fig. 21, the first reinforcing film layer may be substantially the same as the second reinforcing film layer, and the first reinforcing film layer may be stacked on top of the second reinforcing film layer and oriented such that the first direction is orthogonal to the second direction. In some such embodiments, a first portion of optical path 2340 that reflects probe light passes through a first prism face of a first portion of a microprism structure and a first prism face of a second portion of the microprism structure, thereby passing through the first prism face of the microprism structure; a second portion of optical path 2340 that reflects the probe light passes through the second prism face of the first portion of the microprism structure and the second prism face of the second portion of the microprism structure. There may also be a third portion of optical path 2340 that reflects the probe light that passes through the first prism face of the first portion of the microprismatic structure and the second prism face of the second portion of the microprismatic structure and a fourth portion of optical path 2340 that reflects the probe light that passes through the second prism face of the first portion of the microprismatic structure and the first prism face of the second portion of the microprismatic structure. Further, in such embodiments, the actualoptical sensing sub-area 2310 may be smaller than the theoretical optical sensing area 615 (e.g., approximately only a quarter of the theoreticaloptical sensing area 615 may be available without blurring).

An embodiment of anoptical sensing module 2210 includes alens 2214 and an optical detector 2212 (e.g., including any suitable type of sensing component) oriented such that reflected probe light converging on aninput aperture 2216 of theoptical sensing module 2210 is focused onto theoptical detector 2212 by thelens 2214. Some such embodiments may be configured to preferentially select a first portion of optical path 2340 corresponding to illumination energy that passes only (or primarily) through a particular face of the microprism structure (e.g., only the first prism face). For example, as shown in fig. 23, the off-screen optical sensing system is configured such thatoptical paths 2340a and 2340b are directed intooptical sensing module 2210 byreflective structure 2230 andrefractive structure 2220, whileoptical path 2340a is not directed. In some such embodiments,optical detector 2212 may be positioned off-axis with respect to the lens in accordance with configuration preferences for only the general eccentric optical path resulting from the first portion of optical path 2340. Thus, theoptical detector 2212 may be configured (e.g., adjusted in position and/or size) to receive illumination energy reflected from the actualoptical sensing sub-region 2310, rather than from the rest of the theoreticaloptical sensing region 615.

While this disclosure contains many specifics, these should not be construed as limitations on the scope of any application or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular applications. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Additionally, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and shown in this patent document.

References to "a", "an", or "the" are intended to mean "one or more" unless specifically indicated to the contrary. Ranges may be expressed herein as from "about" one specified value and/or to "about" another specified value. The term "about" is used herein to mean approximately, near, approximately, or about. When the term "about" is used in connection with a range of values, it modifies that range by extending the upper and lower boundaries of the proposed values. In general, the term "about" is used herein to modify a numerical value above and below the stated value by a variance of 10%. When such a range is expressed, another embodiment includes from the one specified value and/or to the other specified value. Similarly, when values are expressed as approximations, by use of the antecedent "about," it will be understood that the particular value forms another embodiment. It is also to be understood that the endpoints of each of the ranges are inclusive of the range.

All patents, patent applications, publications, and descriptions mentioned herein are incorporated by reference in their entirety for all purposes. All of which are not considered prior art.

Claims (21)

1. An electronic device, comprising:

a display module defining a display plane; and

an under-screen optical sensing system located below the display module and comprising:

an optical sensing module having an input aperture oriented parallel to the display plane, the optical sensing module for obtaining optical information from received illumination energy;

a refractive structure; the method comprises the steps of,

a reflective structure for receiving the reflected probe light passing through the display module and redirecting an optical path of the reflected probe light to a refractive surface of the refractive structure,

the refraction structure is used for receiving the reflection detection light from the reflection structure and bending the optical path of the reflection detection light so as to be converged on the input hole of the optical sensing module.

2. The electronic device of claim 1, wherein the refractive structure and the reflective structure are configured such that one of the optical paths through the display module in a first direction perpendicular to the display plane is redirected by the reflective structure and bent by the refractive structure to enter the input aperture of the optical sensing module in a second direction parallel to the display plane.

3. The electronic device of claim 1, wherein the refractive structure is a unitary prism having a first refractive surface for receiving reflected probe light from the reflective structure and a second refractive surface for transmitting the reflected probe light to an input aperture of the optical sensing module, the first refractive surface being angled with respect to the second refractive surface.

4. The electronic device of claim 3, wherein the second refractive surface is oriented substantially orthogonal to the display plane.

5. The electronic device of claim 1, wherein the reflective structure is a unitary prism having an angled top surface, the refractive index of the unitary prism being selected to reflect reflected probe light incident on the angled top surface after passing through the display module.

6. The electronic device of claim 1, wherein the reflective structure comprises a mirror.

7. The electronic device of claim 1, further comprising:

an illumination source disposed below the display module to generate detection light and direct the detection light through at least a portion of the display module; the method comprises the steps of,

A top transparent layer disposed over the display module and configured as an output interface for an image produced by the display module and having an optical sensing region configured to receive the probe light from the illumination source, the top transparent layer configured to cause a portion of the probe light to be reflected by the top surface in response to an interaction between an object and the top surface of the top transparent layer, the reflected probe light being the portion of the probe light reflected by the top surface.

8. The electronic device of claim 7, wherein the refractive structure and the reflective structure are configured such that one of the optical paths through the center of the optical sensing region is redirected by the reflective structure and bent by the refractive structure so as to enter the center of the input aperture of the optical sensing module.

9. The electronic device of claim 1, wherein the optical sensing module comprises a lens and an optical sensor oriented such that the reflected probe light converging on an input aperture of the optical sensing module is focused onto the optical sensor by the lens.

10. The electronic device of claim 1, wherein:

The display module further includes an enhancement layer having a plurality of microprism structures;

a first portion of the optical path of the reflected probe light passes through a first prism face of the plurality of microprisms;

a second portion of the optical path of the reflected probe light passes through a second prism face of the plurality of microprisms such that the first and second portions of the optical path diverge; the method comprises the steps of,

the optical sensing module is configured to obtain the optical information from received illumination energy corresponding to only a first portion of the optical path of the reflected probe light.

11. The electronic device of claim 10, wherein:

the reinforcement layer includes a first reinforcement film layer having a plurality of first microprism structures arranged to form a plurality of first parallel prism ridgelines extending in a first direction, and a second reinforcement film layer having a plurality of second microprism structures arranged to form a plurality of second parallel prism ridgelines extending in a second direction different from the first direction;

a first portion of the optical path of the reflected probe light passes through the first prism face of the multi-microprism structure by passing through the first prism face of the plurality of first microprism structures and the first prism face of the plurality of second microprism structures;

A second portion of the optical path of the reflected probe light passes through the second prism facets of the plurality of first microprisms and through the second prism facets of the plurality of second microprisms;

a third portion of the optical path of the reflected probe light passes through the first prism face of the plurality of first microprisms and the second prism face of the plurality of second microprisms; and

a fourth portion of the optical path of the reflected probe light passes through the second prism face of the plurality of first microprisms and the first prism face of the plurality of second microprisms.

12. The electronic device of claim 11, wherein:

the first reinforcing film layer is substantially the same as the second reinforcing film layer; the method comprises the steps of,

the first reinforcing film layer is stacked on top of the second reinforcing film layer and oriented such that the first direction is orthogonal to the second direction.

13. The electronic device of claim 10, wherein:

the optical sensing module comprising a lens and an optical sensor oriented such that the probe light converging on an input aperture of the optical sensing module is focused by the lens onto the optical sensor;

A first portion of the optical path corresponds to an off-center optical path relative to the lens; the method comprises the steps of,

the optical sensor is positioned off-axis with respect to the lens according to the off-center optical path to receive illumination energy corresponding primarily to a first portion of the optical path.

14. The electronic device of claim 10, wherein the plurality of microprismatic structures comprise symmetrical microprismatic structures.

15. The electronic device of claim 10, wherein the plurality of microprismatic structures comprise asymmetric microprismatic structures.

16. The electronic device of claim 1, wherein the display module is an LCD module having a plurality of layers including a Liquid Crystal Display (LCD) layer.

17. The electronic device of claim 1, wherein the electronic device is a smartphone.

18. An under-screen optical sensing system for mounting under a display module defining a display plane, the system comprising:

an optical sensing module having an input aperture oriented to receive illumination energy, the optical sensing module for obtaining optical information from the received illumination energy;

a refractive structure; the method comprises the steps of,

a reflective structure;

the reflective structure and the refractive structure are arranged relative to each other and relative to the optical sensing module such that an optical path of reflected probe light passing through an optical sensing region of the display module is redirected by the reflective structure to a refractive surface of the refractive structure and bent by the refractive structure to converge on an input aperture of the optical sensing module.

19. The system of claim 18, wherein:

the optical sensing module includes a lens and an optical sensor oriented such that the reflected probe light converging on an input aperture of the optical sensing module is focused by the lens onto the optical sensor.

20. The system of claim 19, wherein:

the optical sensing module is used for obtaining optical information according to received illumination energy corresponding to a part of the light path of the reflected detection light passing through a specific subarea of the optical sensing area; the method comprises the steps of,

the optical sensor is positioned off-axis with respect to the lens to receive the illumination energy through an off-axis optical path with respect to the lens.

21. The system according to claim 20, wherein:

the specific sub-area corresponds to approximately one quarter of the optical sensing area.

Images