CN110493491B

Movatterモバイル変換

Info

Publication number: CN110493491B
Application number: CN201910472689.0A
Authority: CN
Inventors: 范蒙
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2019-05-31
Filing date: 2019-05-31
Publication date: 2021-02-26
Anticipated expiration: 2039-05-31
Also published as: WO2020238806A1; CN110493491A

Abstract

The application relates to an image acquisition device and a camera shooting method, and belongs to the field of monitoring. The image acquisition device includes: the device comprises an image sensor, a light supplementing device and a light filtering component, wherein the image sensor is positioned on the light emergent side of the light filtering component; the device also comprises an image signal processing unit, a coding compression unit and an analysis unit; the image sensor is used for generating and outputting a first image signal and a second image signal through multiple exposure; the light supplementing device comprises a first light supplementing device, and the first light supplementing device is used for performing near-infrared light supplementing in a stroboscopic mode; the filter assembly comprises a first filter, and the first filter allows visible light and part of near infrared light to pass through; the image signal processing unit is used for obtaining a gray image according to the first original image signal and obtaining a color image according to the second original image signal; the coding compression unit is used for carrying out compression coding on the color image; the analysis unit is used for carrying out intelligent analysis on the gray level image. The method and the device can improve the accuracy of the analysis result.

Description

Image acquisition device and camera shooting method

Technical Field

The present disclosure relates to monitoring, and particularly to an image capturing device and a method for capturing an image.

Background

The image acquisition equipment can be installed in places such as buildings and roads, and the image acquisition equipment shoots the places, so that not only is a video code stream obtained, but also intelligent analysis can be carried out based on the shot video. The image acquisition equipment can store the shot video code stream in the storage equipment, and intelligent analysis such as face recognition is carried out on the basis of the shot video.

However, in a scene with low illumination brightness, for example, at night, intelligent analysis is performed based on the shot video, and the accuracy of the obtained analysis result is low.

Disclosure of Invention

The embodiment of the application provides an image acquisition device and a camera shooting method, so that the precision of intelligent analysis on images is improved. The technical scheme is as follows:

in one aspect, the present application provides an image capturing apparatus, comprising: the device comprises an image sensor (01), a light supplementing device (02) and a light filtering component (03), wherein the image sensor (01) is positioned on the light emitting side of the light filtering component (03); the device also comprises an image signal processing unit (04), a coding compression unit (05) and an analysis unit (06);

the image sensor (01) is used for generating and outputting a first image signal and a second image signal through multiple exposures, wherein the first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are two exposures of the multiple exposures;

the light supplement device (02) comprises a first light supplement device (021), wherein the first light supplement device (021) is used for performing near-infrared light supplement in a stroboscopic mode, near-infrared light supplement is performed at least in a part of exposure time period of the first preset exposure, and near-infrared light supplement is not performed in the exposure time period of the second preset exposure;

the filter assembly (03) comprises a first filter (031), and the first filter (031) allows visible light and part of near infrared light to pass through;

the image signal processing unit (04) is used for obtaining a gray image according to the first original image signal and obtaining a color image according to the second original image signal;

the encoding and compressing unit (05) is used for performing compression encoding on the color image;

the analysis unit (06) is used for carrying out intelligent analysis on the gray level image.

On the other hand, the application provides a camera shooting method, is applied to image acquisition equipment, image acquisition equipment includes image sensor (01), light filling ware (02) and filtering subassembly (03), image sensor (01) are located the light-emitting side of filtering subassembly (03), light filling ware (02) include first light filling device, filtering subassembly (03) include first light filter, the method includes:

the image acquisition equipment performs near-infrared light supplement through the first light supplement device, wherein the first light supplement device performs near-infrared light supplement at least in a part of exposure time period of first preset exposure of multiple exposure of the image sensor, near-infrared light supplement is not performed in exposure time period of second preset exposure of the multiple exposure, and the first preset exposure and the second preset exposure are two exposures of the multiple exposure;

the image acquisition equipment generates a first original image signal and a second original image signal through the multiple exposure, wherein the first original image signal is an image signal generated according to the first preset exposure, and the second original image signal is an image signal generated according to the second preset exposure;

the image acquisition equipment passes through the first optical filter to enable visible light and partial near infrared light to pass through;

the image acquisition equipment obtains a gray image according to the first original image signal and obtains a color image according to the second original image signal;

the image acquisition equipment performs compression coding on the color image and performs intelligent analysis on the gray level image.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is a schematic structural diagram of an image capturing device according to an embodiment of the present disclosure;

fig. 2 is a schematic structural diagram of a filter assembly according to an embodiment of the present disclosure;

fig. 3 is a timing chart of an exposure time period of a first preset exposure and an exposure time period of a second preset exposure provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a roller shutter exposure method according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a first preset exposure and a second preset exposure provided in an embodiment of the present application;

FIG. 6 is a schematic diagram of a second first preset exposure and a second preset exposure provided in an embodiment of the present application;

FIG. 7 is a schematic diagram of a third first preset exposure and a second preset exposure provided by an embodiment of the present application;

fig. 8 is a schematic view of a first roller shutter exposure method and near-infrared light supplement provided in an embodiment of the present application;

fig. 9 is a schematic view of a second roller shutter exposure method and near-infrared light supplement provided in an embodiment of the present application;

fig. 10 is a schematic diagram of a third rolling shutter exposure mode and near-infrared light supplement provided in the embodiment of the present application;

FIG. 11 is a schematic diagram of a wavelength band corresponding to near infrared light provided by an embodiment of the present application;

fig. 12 is a schematic diagram illustrating a relationship between a wavelength and a relative intensity of a first light supplement device for performing near-infrared light supplement according to an embodiment of the present disclosure;

fig. 13 is a schematic structural diagram of another image capturing device provided in an embodiment of the present application;

FIG. 14 is a schematic structural diagram of an RGBW sensor provided by an embodiment of the present application;

FIG. 15 is a schematic structural diagram of an RCCB sensor provided by the embodiment of the application;

fig. 16 is a schematic structural diagram of an RGB sensor provided in an embodiment of the present application;

fig. 17 is a schematic structural diagram of an RYYB sensor provided in an embodiment of the present application;

FIG. 18 is a schematic view of a wavelength band corresponding to each photosensitive channel provided in the embodiment of the present application;

FIG. 19 is a schematic diagram of image fusion provided by embodiments of the present application;

fig. 20 is a flowchart of an image capturing method according to an embodiment of the present application.

With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

In an environment with weak light, for example, in an environment at night, noise of an image signal photographed by an image acquisition device is large, and noise in a gray scale image generated based on the image signal is also large, so that the accuracy of an analysis result obtained when intelligent analysis processing is performed based on the gray scale image is low. In order to improve the accuracy of the analysis results, the present application provides the following image capturing device.

Referring to fig. 1, an embodiment of the present application provides an image capturing apparatus, including:

the device comprises animage sensor 01, alight supplementing device 02 and alight filtering component 03, wherein theimage sensor 01 is positioned on the light emergent side of thelight filtering component 03; further comprises an imagesignal processing unit 04, anencoding compression unit 05 and ananalysis unit 06;

animage sensor 01 for generating and outputting a first image signal and a second image signal through multiple exposures, wherein the first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are two exposures of the multiple exposures;

thelight supplement device 02 includes a firstlight supplement device 021, and the firstlight supplement device 021 is configured to perform near-infrared light supplement in a stroboscopic manner, wherein the near-infrared light supplement is performed at least in a partial exposure time period of a first preset exposure, and the near-infrared light supplement is not performed in an exposure time period of a second preset exposure;

thefilter assembly 03 includes afirst filter 031, and thefirst filter 031 passes visible light and a part of near-infrared light;

an imagesignal processing unit 04, configured to obtain a grayscale image according to the first original image signal, and obtain a color image according to the second original image signal;

acode compression unit 05 for compression-coding the color image;

and theanalysis unit 06 is used for intelligently analyzing the gray level image.

Theimage sensor 01 generates and outputs a first image signal and a second image signal through multiple exposures, the first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are two exposures of the multiple exposures. The first image signal is processed by the imagesignal processing unit 04 to output a color image for encoding, compressing and outputting a video code stream; the second image signal is processed by thesignal processing unit 04 to generate a gray image, and theanalysis unit 06 performs analysis processing on the gray image to output an intelligent analysis result; carry out special light filling and image signal collection and optimization to intelligent analysis, under the prerequisite that does not change current video effect, further promote intelligent analysis's effect.

As an example, the image capture device may be a video camera, a snap-shot, a face recognition camera, a code reading camera, a vehicle-mounted camera, a panoramic detail camera, or the like.

As another example, thelight supplement 02 may be located inside the image capturing device or outside the image capturing device. Thelight supplement 02 may be a part of the image capturing device or may be a device independent of the image capturing device. When thelight supplement device 02 is located outside the image acquisition device, thelight supplement device 02 can be in communication connection with the image acquisition device, and it can be ensured that the exposure time sequence of theimage sensor 01 in the image acquisition device and the near-infrared light supplement time sequence of the firstlight supplement device 021 included in thelight supplement device 02 have a certain relationship, if at least near-infrared light supplement is performed in a partial exposure time period of the first preset exposure, near-infrared light supplement is not performed in an exposure time period of the second preset exposure.

Referring to fig. 3, the exposure time periods of the first preset exposure and the second preset exposure are alternately and cyclically occurred, the firstlight supplement device 021 generates and emits near infrared light in the exposure time period of the first preset exposure to implement near infrared light supplement, and stops generating near infrared light in the exposure time period of the second preset exposure to implement no near infrared light supplement. The so-called stroboscopic mode for near-infrared fill-in is to generate and emit near-infrared light during an exposure time period of a first predicted exposure, and to stop generating near-infrared light during an exposure time period of a second preset exposure adjacent to the exposure time period of the first predicted exposure.

Referring to fig. 3, as an example, the multiple exposures may include an odd number of exposures and an even number of exposures, the first preset exposure may be one exposure of the even number of exposures, and the second preset exposure may be one exposure of the odd number of exposures.

As an example, the first preset exposure may be one exposure of odd number exposures, and the second preset exposure may be one exposure of even number exposures; or

As an example, the first preset exposure may be one exposure of a designated odd number of exposures, and the second preset exposure may be one exposure of other exposures other than the designated odd number of exposures; or

As an example, the first preset exposure may be one exposure of a designated even-numbered exposure, and the second preset exposure may be one exposure of other exposures other than the designated even-numbered exposure.

As an example, the number of times of fill-in light of the first fill-inlight device 021 in a unit time length may be lower than the number of times of exposure of theimage sensor 01 in the unit time length, wherein one or more exposures are spaced in each of two adjacent fill-in light periods.

As an example, the first preset exposure is one exposure in a first exposure sequence, and the second preset exposure is one exposure in a second exposure sequence; or

As an example, the first preset exposure is one exposure in a second exposure sequence, and the second preset exposure is one exposure in the first exposure sequence;

the multiple exposure comprises a plurality of exposure sequences, the first exposure sequence and the second exposure sequence are one exposure sequence or two exposure sequences in the exposure sequences, each exposure sequence comprises N times of exposure, the N times of exposure comprise 1 time of first preset exposure and N-1 times of second preset exposure, or the N times of exposure comprise 1 time of second preset exposure and N-1 times of second preset exposure, and N is a positive integer greater than 2.

As an example, theimage sensor 01 may adopt a global exposure mode, and may also adopt a rolling shutter exposure mode. The global exposure mode means that the exposure start time of each row of effective pixels is the same, and the exposure end time of each row of effective pixels is the same. In other words, the global exposure mode is an exposure mode in which all the rows of effective pixels are exposed at the same time and the exposure is ended at the same time. The rolling shutter exposure mode means that the exposure time of different rows of effective pixels is not completely overlapped, that is, the exposure start time of one row of effective pixels is later than the exposure start time of the previous row of effective pixels, and the exposure end time of one row of effective pixels is later than the exposure end time of the previous row of effective pixels. In addition, in the rolling exposure method, data output can be performed after exposure of each row of effective pixels is finished, and therefore, a time from a data start output time of a first row of effective pixels to a data end output time of a last row of effective pixels can be expressed as a readout time.

Illustratively, referring to fig. 4, fig. 4 is a schematic diagram of a rolling shutter exposure mode. As can be seen from fig. 10, theline 1 effective image starts exposure at time T1, ends exposure at time T3, and theline 2 effective image starts exposure at time T2, ends exposure at time T4, and shifts back by a time period from time T2 to time T1, and shifts back by a time period from time T4 to time T3. When the exposure of the 1 st line effective image is finished and the data output is started at the time T3, the data output is finished at the time T5, the exposure of the nth line effective image is finished and the data output is started at the time T6, and the data output is finished at the time T7, the time between the times T3 and T7 is the readout time.

Under the condition that theimage sensor 01 performs multiple exposures in a global exposure mode, for any one time of near-infrared light supplement, the time period of the near-infrared light supplement does not have an intersection with the exposure time period of the nearest second preset exposure, the time period of the near-infrared light supplement is a subset of the exposure time period of the first preset exposure, or the time period of the near-infrared light supplement has an intersection with the exposure time period of the first preset exposure, or the exposure time period of the first preset exposure is a subset of the near-infrared light supplement. Therefore, near-infrared light supplement can be carried out in at least part of the exposure time period of the first preset exposure, and near-infrared light supplement is not carried out in the whole exposure time period of the second preset exposure, so that the second preset exposure cannot be influenced.

In addition, since the intensity of the near-infrared light in the ambient light is lower than the intensity of the near-infrared light emitted by the firstlight supplement device 021, the intensity of the near-infrared light passing through the firstoptical filter 031 when the firstlight supplement device 021 performs the near-infrared light supplement is higher than the intensity of the near-infrared light passing through the firstoptical filter 031 when the firstlight supplement device 021 does not perform the near-infrared light supplement.

The wavelength range of the firstlight supplement device 021 for performing near-infrared light supplement may be a second reference wavelength range, and the second reference wavelength range may be 700nm to 800nm, or 900nm to 1000nm, and the like, which is not limited in this embodiment. In addition, the wavelength range of the near infrared light incident to thefirst filter 031 may be a first reference wavelength range, which is 650nm to 1100 nm, for example, 650nm to 1000 nm. Among them, it should be noted that: the second reference wavelength band range may be 700nm to 800nm, which means that the second reference wavelength band range is a range greater than or equal to 700nm and less than or equal to 800nm, or a range greater than 700nm and less than 800nm, or a range greater than or equal to 700nm and less than 800nm, or a range greater than 700nm and less than or equal to 800 nm. To the extent that similar terms appear elsewhere herein, they are intended to be synonymous, they are not described in detail.

In the exposure time period of the first preset exposure, the near-infrared light passing through the firstoptical filter 031 may include near-infrared light reflected by the object and entering the firstoptical filter 031 when the firstlight supplement device 021 performs near-infrared light supplement, and therefore, the center wavelength and/or the band width of the near-infrared light passing through the firstoptical filter 031 may be matched with the center wavelength of the near-infrared light supplement performed by the firstlight supplement device 021, that is, when the center wavelength of the near-infrared light supplement performed by the firstlight supplement device 021 is the set characteristic wavelength or falls within the set characteristic wavelength range, the center wavelength and/or the band width of the near-infrared light passing through the firstoptical filter 031 reaches the constraint condition.

The firstlight supplement device 021 can have multiple choices for the center wavelength and/or the waveband range of near-infrared light supplement, in this embodiment of the application, in order to make the firstlight supplement device 021 and the firstoptical filter 031 have better cooperation, the center wavelength of near-infrared light supplement can be designed for the firstlight supplement device 021, and the characteristic of the firstoptical filter 031 is selected, thereby make the center wavelength of near-infrared light supplement be for setting for the characteristic wavelength or fall when setting for the characteristic wavelength range at the firstlight supplement device 021, the center wavelength and/or the waveband width of near-infrared light through the firstoptical filter 031 can reach the constraint condition. The constraint condition is mainly used to constrain the center wavelength of the near-infrared light passing through the firstoptical filter 031 to be as accurate as possible, and the band width of the near-infrared light passing through the firstoptical filter 031 to be as narrow as possible, so as to avoid the occurrence of wavelength interference caused by too wide band width of the near-infrared light.

The central wavelength of the near-infrared light supplement by the firstlight supplement device 021 may be an average value in a wavelength range where energy in a spectrum of the near-infrared light emitted by the firstlight supplement device 021 is the maximum, or may be a wavelength at an intermediate position in a wavelength range where energy in the spectrum of the near-infrared light emitted by the firstlight supplement device 021 exceeds a certain threshold.

The set characteristic wavelength or the set characteristic wavelength range may be preset. As an example, the center wavelength of the near-infrared supplementary lighting performed by the firstsupplementary lighting device 021 may be any wavelength within a wavelength range of 750 ± 10 nanometers; or, the center wavelength of the near-infrared supplementary lighting performed by the firstsupplementary lighting device 021 is any wavelength within the wavelength range of 780 ± 10 nanometers; or, the firstlight supplement device 021 supplements light in near-infrared light at any wavelength within a wavelength range of 940 ± 10 nanometers. That is, the set characteristic wavelength range may be a wavelength range of 750 ± 10 nanometers, or a wavelength range of 780 ± 10 nanometers, or a wavelength range of 940 ± 10 nanometers. Illustratively, the center wavelength of the first fill-inlight device 021 for near-infrared fill-in light is 940 nm, and the relationship between the wavelength and the relative intensity of the first fill-inlight device 021 for near-infrared fill-in light is shown in fig. 11. As can be seen from fig. 11, the wavelength band of the firstlight supplement device 021 for performing near-infrared light supplement is 900nm to 1000nm, wherein the relative intensity of the near-infrared light is the highest between 940 nm and 960 nm.

Since most of the near-infrared light passing through the firstoptical filter 031 is near-infrared light entering the firstoptical filter 031 after being reflected by the object when the first fill-indevice 021 performs near-infrared fill-in the exposure time period of the first preset exposure, in some embodiments, the constraint condition may include: the difference between the central wavelength of the near-infrared light passing through the firstoptical filter 031 and the central wavelength of the near-infrared light supplemented by the firstlight supplementing device 021 is within a wavelength fluctuation range, which may be 0 to 20 nm, as an example.

The central wavelength of the near-infrared supplementary light passing through the firstoptical filter 031 may be a wavelength at a peak position in a near-infrared band range in the near-infrared light transmittance curve of the firstoptical filter 031, or may be a wavelength at a middle position in a near-infrared band range in which a transmittance exceeds a certain threshold in the near-infrared light transmittance curve of the firstoptical filter 031.

In order to avoid introducing wavelength interference due to too wide band width of the near infrared light passing through thefirst filter 031, in some embodiments, the constraint conditions may include: the first band width may be less than the second band width. The first wavelength band width refers to the wavelength band width of the near-infrared light passing through thefirst filter 031, and the second wavelength band width refers to the wavelength band width of the near-infrared light blocked by thefirst filter 031. It should be understood that the band width refers to the width of the wavelength range in which the wavelength of the light is located. For example, the wavelength of the near infrared light passing through thefirst filter 031 is in the wavelength range of 700nm to 800nm, and then the first wavelength band width is 800nm minus 700nm, i.e., 100 nm. In other words, the wavelength band width of the near infrared light passing through thefirst filter 031 is smaller than the wavelength band width of the near infrared light blocked by thefirst filter 031.

As can be seen from the above description, the second original image signal generated and output by theimage sensor 01 during the exposure period of the second preset exposure mainly includes the visible light luminance information. However, in the exposure time period of the second preset exposure, the firstoptical filter 031 not only allows visible light in the ambient light to pass, but also allows near-infrared light that enters the firstoptical filter 031 after being reflected by an object in the ambient light to pass. Therefore, in order to prevent the second original image signal generated and output by theimage sensor 01 from containing too much near-infrared brightness information and to improve the quality of the second original image signal, the blocking of thefirst filter 031 to the visible light needs to be reduced, so that the first wavelength band width of thefirst filter 031 can be smaller than the second wavelength band width of thesecond filter 031. The first wavelength band width refers to the wavelength band width of the near-infrared light passing through thefirst filter 031, and the second wavelength band width refers to the wavelength band width of the near-infrared light blocked by thefirst filter 031. It should be understood that the band width refers to the width of the wavelength range in which the wavelength of the light is located. For example, the wavelength of the near infrared light passing through thefirst filter 031 is in the wavelength range of 700nm to 800nm, and then the first wavelength band width is 800nm minus 700nm, i.e., 100 nm. In other words, the wavelength band width of the near infrared light passing through thefirst filter 031 is smaller than the wavelength band width of the near infrared light blocked by thefirst filter 031. In this way, in the exposure time period of the second preset exposure, the intensity of the near infrared light passing through the firstoptical filter 031 is relatively weak, while the intensity of the visible light passing through the firstoptical filter 031 is relatively strong, and the intensity of the near infrared light sensed by theimage sensor 01 is relatively weak, so that the generated and output second original image signal does not contain too much near infrared light brightness information, and the quality of the second original image signal is further higher.

For example, referring to fig. 12, fig. 12 is a schematic diagram illustrating a relationship between a wavelength of light that can pass through thefirst filter 031 and a pass rate. The wavelength range of the near-infrared light incident to the firstoptical filter 031 is 650nm to 1100 nm, the firstoptical filter 031 can allow visible light with a wavelength of 380 nm to 650nm to pass through, near-infrared light with a wavelength of 900nm to 1000nm to pass through, and block near-infrared light with a wavelength of 650nm to 900 nm. That is, the first band width is 1000 nanometers minus 900 nanometers, i.e., 100 nanometers. The second band has a width of 900nm minus 650nm plus 1100 nm minus 1000nm, i.e., 350 nm. 100nm is smaller than 350 nm, that is, the band width of the near infrared light passing through the firstoptical filter 031 is smaller than the band width of the near infrared light blocked by the firstoptical filter 031. The above relation is only an example, and the wavelength range of the near-red light band that can pass through the filter may be different for different filters, and the wavelength range of the near-infrared light that is blocked by the filter may also be different.

In order to make the second original image signal generated and output by theimage sensor 01 not contain too much near-infrared light brightness information, and make the quality of the second original image signal higher, the constraint conditions include: the half-bandwidth of the near infrared light passing through thefirst filter 031 is less than or equal to 50 nm. The half bandwidth refers to the band width of near infrared light with a passing rate of more than 50%.

In order to make the second original image signal generated and output by theimage sensor 01 not contain too much near-infrared light brightness information and make the quality of the second original image signal higher, the constraint conditions are: the third wavelength band width of thefirst filter 031 may be smaller than the reference wavelength band width. The third wavelength band width is a wavelength band width of the near infrared light having a transmittance greater than a set ratio, and as an example, the reference wavelength band width may be any one of wavelength band widths in a wavelength band range of 50nm to 150 nm. The set proportion may be any proportion of 30% to 50%, and of course, the set proportion may be set to other proportions according to the use requirement, which is not limited in the embodiment of the present application. In other words, the band width of the near infrared light having the passing rate larger than the set ratio may be smaller than the reference band width. In this way, in the exposure time period of the second preset exposure, the intensity of the near-infrared light that can be sensed by theimage sensor 01 is relatively weak, so that the generated and output second original image signal does not contain excessive near-infrared light brightness information, and the quality of the second original image signal is further higher.

For example, referring to fig. 12, the wavelength band of the near infrared light incident to thefirst filter 031 is 650nm to 1100 nm, the set ratio is 30%, and the reference wavelength band width is 100 nm. As can be seen from fig. 12, in the wavelength band of the near-infrared light of 650nm to 1100 nm, the wavelength band width of the near-infrared light with the transmittance of more than 30% is significantly less than 100 nm.

Because the firstlight supplement device 021 provides near-infrared light supplement at least in the partial exposure time period of the first preset exposure, the near-infrared light supplement is not provided in the whole exposure time period of the second preset exposure, and the first preset exposure and the second preset exposure are two exposures of multiple exposures of theimage sensor 01, that is, the firstlight supplement device 021 provides near-infrared light supplement in the exposure time period of the partial exposure of theimage sensor 01, and the near-infrared light supplement is not provided in the exposure time period of the other partial exposure of theimage sensor 01. Therefore, the number of light supplement times of the firstlight supplement device 021 in a unit time length can be lower than the number of exposure times of theimage sensor 01 in the unit time length, wherein one or more exposures are spaced in each interval time period of two adjacent light supplement. Therefore, the number of times of light supplement of the firstlight supplement device 021 can be reduced, and the service life of the firstlight supplement device 021 is prolonged.

It should be noted that, since human eyes easily confuse the color of the near-infrared light supplemented by the firstlight supplementing device 021 with the color of the red light in the traffic light, referring to fig. 13, thelight supplementing device 02 may further include a secondlight supplementing device 022, and the secondlight supplementing device 022 is used for supplementing the visible light. Like this, if secondlight filling device 022 provides the visible light filling at the partial exposure time of first preset exposure at least, promptly, carries out near-infrared light filling and visible light filling in the partial exposure time quantum of first preset exposure at least, and the mixed colour of these two kinds of light can be distinguished from the colour of the red light in the traffic light to the colour that the people's eye carries out near-infrared light filling withlight filling ware 02 and the colour of the red light in the traffic light are confused has been avoided. In addition, if the secondlight supplement device 022 provides supplementary lighting for visible light in the exposure time period of the second preset exposure, since the intensity of visible light in the exposure time period of the second preset exposure is not particularly high, the brightness of visible light in the second original image signal can be further improved when the supplementary lighting for visible light is performed in the exposure time period of the second preset exposure, and the quality of image acquisition is further ensured.

In some embodiments, the secondlight supplement device 022 can be configured to supplement the visible light in a normally bright manner, wherein the supplementary lighting of the visible light is performed at least in a partial exposure time period of the first preset exposure, and the supplementary lighting of the visible light can also be performed in the whole exposure time period of the second preset exposure. When the secondlight supplement device 022 is normally on, visible light is supplemented, so that the color of the firstlight supplement device 021 for near-infrared light supplement can be prevented from being mixed up with the color of a red light in a traffic light by human eyes, the brightness of visible light in a second original image signal can be improved, and the quality of image acquisition is ensured. Or, the secondlight supplement device 022 may be configured to perform the light supplement of the visible light in a stroboscopic manner, where the light supplement of the visible light is not performed at least in the whole exposure time period of the first preset exposure, and the light supplement of the visible light is performed in the exposure time period of the second preset exposure. Or, the secondlight supplement device 022 is configured to supplement the visible light in a stroboscopic manner, where the supplementary filling of the visible light is performed at least in a partial exposure time period of the first preset exposure, and the supplementary filling of the visible light is not performed in the entire exposure time period of the second preset exposure.

When secondlight filling device 022 carries out visible light filling with the stroboscopic mode, can avoid the colour that human eye carries out near-infrared light filling with firstlight filling device 021 and the colour of the red light in the traffic light to obscure, perhaps, can improve the luminance of the visible light in the original image signal of second, and then guarantee image acquisition's quality, but also can reduce the light filling number of times of secondlight filling device 022 to prolong the life of secondlight filling device 022.

As an example, with thelens 04 described above, thelens 04 is used to focus light reflected by an object;

the firstoptical filter 031 in theoptical filtering component 03 is configured to filter out the near-infrared light and the visible light and the near-infrared light in the ambient light generated by the firstlight supplement device 021 from the focused light in the exposure time period of the first preset exposure, and filter out the visible light and the near-infrared light in the ambient light from the focused light in the exposure time period of the second preset exposure.

Referring to fig. 2, thefiltering assembly 03 includes afirst filter 031, asecond filter 032, and a switching member 033; thefirst filter 031 and thesecond filter 032 are both connected to the switching member 033.

Afirst filter 031 for passing light in a visible light band and a near-infrared light band, which is light in a range of the first reference filter segment;

asecond filter 032 for passing light in the visible band;

in an external environment with strong light, the firstlight supplement device 021 can stop performing near infrared light supplement. For example, during daytime, when the light is strong, the first fill-inlight device 021 may stop performing the near-infrared fill-in light. In an external environment with weak light, the firstlight supplement device 021 performs near-infrared light supplement in a stroboscopic manner. For example, at night, the firstfill light device 021 performs near-infrared fill light in a stroboscopic manner.

Therefore, the switching component 033 is used to switch thefirst filter 031 to the light-incident side of theimage sensor 01 in the external environment with weak light intensity, and to switch thesecond filter 032 to the light-incident side of theimage sensor 01 in the external environment with strong light intensity.

After thesecond filter 032 is switched to the light incident side of theimage sensor 01, thesecond filter 032 passes light in the visible light band and blocks light in the near-infrared light band, and theimage sensor 01 generates and outputs a third image signal by exposure.

As an example, referring to fig. 1 and 2, the apparatus may further include alens 04, in which case thefiltering component 03 may be located between thelens 04 and theimage sensor 01, and theimage sensor 01 is located at the light emitting side of thefiltering component 03. Alternatively, thelens 04 is located between thefilter assembly 03 and theimage sensor 01, and theimage sensor 01 is located on the light emitting side of thelens 04. As an example, thefirst filter 031 may be a filter film, such that thefirst filter 031 may be attached to a surface of the light-emitting side of thelens 04 when thefilter assembly 03 is positioned between thelens 04 and theimage sensor 01, or attached to a surface of the light-entering side of thelens 04 when thelens 04 is positioned between thefilter assembly 03 and theimage sensor 01.

The switchingmember 03 switches the secondoptical filter 032 to the light incident side of theimage sensor 01 or switches the firstoptical filter 031 to the light incident side of theimage sensor 01; after the secondoptical filter 032 is switched to the light incident side of the image sensor, the firstlight supplement device 021 is in a closed state; the secondoptical filter 032 is used for blocking light in a near infrared light band and allowing light in a visible light band to pass through; and animage sensor 01 for generating and outputting a third original image signal through exposure.

The switching member 033 is configured to switch thesecond filter 032 to the light incident side of theimage sensor 01, and it may be understood that thesecond filter 032 replaces the position of thefirst filter 031 on the light incident side of theimage sensor 01. After thesecond filter 032 is switched to the light incident side of theimage sensor 01, the firstlight supplement device 021 may be in an off state or an on state.

The external environment with stronger light intensity can indicate that the light intensity of the external environment is greater than or equal to a preset intensity threshold, and the external environment with weaker light intensity can indicate that the light intensity of the external environment is less than the preset intensity threshold.

As an example, the switching member 033 switches thefirst filter 031 between thelens 04 and theimage sensor 01 in an external environment (e.g., at night) where the intensity of light is weak, and the light reflected by the object travels through thelens 04, thefirst filter 031, and theimage sensor 01. Since the firstoptical filter 031 can pass through the visible light and the near-infrared light in the ambient light and the near-infrared light generated by the firstlight supplement device 021 reflected by the object from the light, in the exposure time period of the first preset exposure, the firstoptical filter 031 can pass through the visible light and the near-infrared light reflected by the object from the ambient light and the near-infrared light generated by the firstlight supplement device 021 reflected by the object from the light focused by thelens 04, and transmit the passed visible light and the passed near-infrared light to theimage sensor 01. In the exposure time period of the second preset exposure, the firstoptical filter 031 may pass visible light and near-infrared light reflected by an object in ambient light from the light focused by thelens 04, and transmit the visible light and the near-infrared light in the filtered ambient light to theimage sensor 01.

As an example, the switching member 033 switches thesecond filter 032 between thelens 04 and theimage sensor 01 in an external environment (e.g., daytime) where the intensity of light is strong, and the light reflected by the object travels through thelens 04, thesecond filter 032, and theimage sensor 01. Since thesecond filter 032 can pass the visible light from the light, thesecond filter 032 can pass the visible light from the light focused by thelens 04 and transmit the passed visible light to theimage sensor 01.

In some embodiments, the multiple exposure of the image sensor refers to multiple exposure within one frame period, that is, theimage sensor 01 performs multiple exposure within one frame period, thereby generating and outputting at least one frame of the first image signal and at least one frame of the second image signal. For example, theimage sensor 01 performs exposure for a plurality of times in each frame period for 1 second, thereby generating at least one frame of the first image signal and at least one frame of the second image signal, and the first image signal and the second image signal generated in one frame period are referred to as a set of image signals, so that 25 sets of image signals are generated in 25 frame periods. The first preset exposure and the second preset exposure may be adjacent two exposures in multiple exposures within one frame period, or may also be nonadjacent two exposures in multiple exposures within one frame period, which is not limited in this embodiment of the application.

The first image signal is generated and output for a first preset exposure, the second image signal is generated and output for a second preset exposure, and the first image signal and the second image signal may be processed after the first image signal and the second image signal are generated and output. In some cases, the first image signal and the second image signal may be used differently, so in some embodiments, at least one exposure parameter of the first preset exposure and the second preset exposure may be different. As an example, the at least one exposure parameter may include, but is not limited to, one or more of exposure time, analog gain, digital gain, aperture size. Wherein the exposure gain comprises an analog gain and/or a digital gain.

In some embodiments. It can be understood that, when performing the near-infrared light compensation, the intensity of the near-infrared light sensed by theimage sensor 01 is stronger, and the brightness of the near-infrared light included in the first image signal generated and outputted accordingly is higher, compared to the second preset exposure. But the higher brightness near infrared light is not favorable for the acquisition of external scene information. Also, in some embodiments, the larger the exposure gain, the higher the brightness of the image signal output by theimage sensor 01, and the smaller the exposure gain, the lower the brightness of the image signal output by theimage sensor 01, and therefore, in order to ensure that the brightness of the near-infrared light included in the first image signal is within a suitable range, in the case where at least one exposure parameter of the first preset exposure and the second preset exposure is different, as an example, the exposure gain of the first preset exposure may be smaller than the exposure gain of the second preset exposure. Thus, when the firstlight supplement device 021 performs near-infrared light supplement, the brightness of near-infrared light included in the first image signal generated and output by theimage sensor 01 is not too high due to the near-infrared light supplement performed by the firstlight supplement device 021.

In other embodiments, the longer the exposure time, the higher the brightness included in the image signal obtained by theimage sensor 01, and the longer the motion smear of the moving object in the external scene in the image signal; the shorter the exposure time, the lower the brightness included in the image signal obtained by theimage sensor 01, and the shorter the motion smear of the moving object in the external scene in the image signal. Therefore, in order to ensure that the brightness of the near-infrared light contained in the first image signal is within a proper range, and the motion tail of the moving object in the external scene in the first image signal is short. In a case where at least one exposure parameter of the first preset exposure and the second preset exposure is different, as an example, the exposure time of the first preset exposure may be smaller than the exposure time of the second preset exposure. Thus, when the firstlight supplement device 021 performs near-infrared light supplement, the brightness of near-infrared light included in the first image signal generated and output by theimage sensor 01 is not too high due to the near-infrared light supplement performed by the firstlight supplement device 021. And the shorter exposure time makes the motion smear of the moving object in the external scene appearing in the first image signal shorter, thereby facilitating the identification of the moving object. Illustratively, the exposure time of the first preset exposure is 40 milliseconds, the exposure time of the second preset exposure is 60 milliseconds, and so on.

It is noted that, in some embodiments, when the exposure gain of the first preset exposure is smaller than the exposure gain of the second preset exposure, the exposure time of the first preset exposure may be not only smaller than the exposure time of the second preset exposure, but also equal to the exposure time of the second preset exposure. Similarly, when the exposure time of the first preset exposure is shorter than the exposure time of the second preset exposure, the exposure gain of the first preset exposure may be smaller than or equal to the exposure gain of the second preset exposure.

In other embodiments, the first image signal and the second image signal may be used for the same purpose, for example, when both the first image signal and the second image signal are used for intelligent analysis, at least one exposure parameter of the first preset exposure and the second preset exposure may be the same in order to enable the same definition of the human face or the target under intelligent analysis when the human face or the target moves. As an example, the exposure time of the first preset exposure may be equal to the exposure time of the second preset exposure, and if the exposure time of the first preset exposure is different from the exposure time of the second preset exposure, a motion smear may exist in one path of image signals with a longer exposure time, resulting in different resolutions of the two paths of image signals. Likewise, as another example, the exposure gain of the first preset exposure may be equal to the exposure gain of the second preset exposure.

It is noted that, in some embodiments, when the exposure time of the first preset exposure is equal to the exposure time of the second preset exposure, the exposure gain of the first preset exposure may be smaller than or equal to the exposure gain of the second preset exposure. Similarly, when the exposure gain of the first preset exposure is equal to the exposure gain of the second preset exposure, the exposure time of the first preset exposure may be shorter than the exposure time of the second preset exposure, or may be equal to the exposure time of the second preset exposure.

As an example, another implementation manner of generating the first original image signal and the second original image signal is provided in the embodiment of the present application, where the another implementation manner is:

in an external environment with weak light intensity (e.g., at night), the firstlight supplement device 021 is used to continuously generate and emit near infrared light. The switching component 033 drives thefirst filter 031 to be located between thelens 04 and theimage sensor 01 within an exposure time period of a first preset exposure, thelens 04 focuses light reflected by an object, thefirst filter 031 passes visible light and near-infrared light in ambient light and near-infrared light generated by the firstlight filling device 021 reflected by the object from the focused light, and theimage sensor 01 senses the visible light and the near-infrared light passed by thefirst filter 031 according to exposure parameters of the first preset exposure to obtain a first original image signal. And the switching component 033 drives the secondoptical filter 032 to be located between thelens 04 and theimage sensor 01 within an exposure time period of the second preset exposure, thelens 04 focuses light reflected by the object, the secondoptical filter 032 filters visible light from the focused light, and theimage sensor 01 senses the visible light according to exposure parameters of the second preset exposure to obtain a second original image signal.

In the exposure time period of the second preset exposure, the secondoptical filter 032 only allows visible light to pass through and blocks near visible light from passing through, so that the second original image signal obtained by sensing the visible light by theimage sensor 01 cannot be over-exposed, and a color image obtained based on the second original image signal cannot be whitened.

In an external environment with strong light intensity (e.g., in the daytime), the switching component 033 drives thesecond filter 032 between thelens 04 and theimage sensor 01.

With respect to theimage sensor 01 described above, theimage sensor 01 includes a plurality of sensing channels for sensing light in at least two different visible light bands. Wherein each photosensitive channel is used for sensing at least one color of light in a visible light band and light in a near infrared band, and the at least one color of visible light comprises red light, green light, blue light, yellow light and the like.

As an example, each photosensitive channel corresponds to a visible light wavelength range of one color, that is, each photosensitive channel is used for sensing visible light and light in a near infrared band in the visible light wavelength range of the corresponding color. For example, the plurality of photosensitive channels include at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, a Y photosensitive channel, a W photosensitive channel, a C photosensitive channel, and the like; the light sensing device comprises a light sensing channel, a light sensing channel and a light sensing channel, wherein the light sensing channel is used for sensing light of a red light wave band and a near infrared wave band, the light sensing channel is used for sensing light of a green light wave band and a near infrared wave band, the light sensing channel is used for sensing light of a blue light wave band and a near infrared wave band, the light sensing channel is used for sensing light of a yellow light wave band and a near infrared wave band, the light sensing channel is used for sensing light of.

In other embodiments, some of the photosensitive channels may also sense only light in the near infrared band and not in the visible band. As an example, the plurality of photosensitive channels may include at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, and an IR photosensitive channel. The R light sensing channel is used for sensing light of a red light wave band and a near infrared wave band, the G light sensing channel is used for sensing light of a green light wave band and a near infrared wave band, the B light sensing channel is used for sensing light of a blue light wave band and a near infrared wave band, and the IR light sensing channel is used for sensing light of a near infrared wave band.

Illustratively, theimage sensor 01 may be an rgbiir sensor, wherein each IR photosensitive channel in the rgbiir sensor may sense light in the near infrared band, but not light in the visible band.

When theimage sensor 01 is an RGB sensor, compared with other image sensors, such as an rgbiir sensor, the RGB information acquired by the RGB sensor is more complete, and a part of photosensitive channels of the rgbiir sensor cannot acquire visible light, so that the color details of the image acquired by the RGB sensor are more accurate.

It is noted that theimage sensor 01 may include a plurality of photosensitive channels corresponding to a plurality of sensing curves. Illustratively, referring to fig. 18, an R curve in fig. 18 represents a sensing curve of theimage sensor 01 for light in a red wavelength band, a G curve represents a sensing curve of theimage sensor 01 for light in a green wavelength band, a B curve represents a sensing curve of theimage sensor 01 for light in a blue wavelength band, a W (or C) curve represents a sensing curve of theimage sensor 01 for light in a full wavelength band, and an NIR (Near infrared) curve represents a sensing curve of theimage sensor 01 for light in a Near infrared wavelength band.

For the imagesignal processing unit 04, the imagesignal processing unit 04 is configured to perform at least a first sharpening process on the first original image signal to obtain a grayscale image, and perform at least a second sharpening process on the second original image signal to obtain a color image, where an intensity of the first sharpening process is smaller than an intensity of the second sharpening process.

The method comprises the steps of carrying out sharpening processing on an image to obtain a first original image signal, wherein information loss in the image can be caused when the image is sharpened, the intensity of the first sharpening processing is lower than that of the second sharpening processing, so that the intensity of the first sharpening processing can be reduced, carrying out at least the first sharpening processing on the first original image signal to obtain a gray image, the information loss amount in the gray image can be reduced, and the precision of intelligent analysis can be improved when intelligent analysis is carried out based on the gray image.

The first process further includes at least one of black level correction, Gamma correction, color correction, demosaicing, noise reduction, or the like. The second process further includes at least one of a black level correction, a Gamma correction, a color correction, a demosaicing, or a noise reduction process.

Optionally, referring to fig. 19, the imagesignal processing unit 04 is configured to fuse the image signal after the first original image signal is preprocessed and the image signal after the second original image signal is preprocessed to obtain a fused image, and determine the fused image as a color image.

The first original image signal and the second original image signal are preprocessed, so that noise in the image signals can be eliminated. The first original image signal is obtained by exposure under the condition that the near-infrared fill light exists, so that the noise of the first original image signal is lower than that of the second original image signal, the brightness of the image corresponding to the first original image signal is higher than that of the image corresponding to the second original image signal, the two preprocessed image signals are fused, and the noise in the color image can be reduced and the brightness of the color image can be increased.

The first original image signal and the second original image signal are two images, respectively, the imagesignal processing unit 04 may extract a first edge image from the image signal after the first original image signal is preprocessed, where the first edge image includes edges of object images in the image corresponding to the first original image signal, and extract a second edge image from the image signal after the second original image signal is preprocessed, where the second edge image includes edges of object images in the image corresponding to the second original image signal. Then, the fused image is obtained by the following first formula.

The first formula is:

in the first formula: k_i,jFor pixel points, Z, in the image signal after the preprocessing of the first original image signal_i,jFor pixel points, H, in the image signal after the second original image signal has been preprocessed_i,jFor pixel points in the fused image, m_i,jIs a pixel point in the first edge image, n_i,jThe pixel points in the second edge image are (i, j) the positions of the pixel points.

For theanalysis unit 06, theanalysis unit 06 includes an intelligent analysis algorithm, and theanalysis unit 06 performs intelligent analysis on the gray image by using the intelligent analysis algorithm to obtain an analysis result or a target sub-graph. For example, the intelligent analysis algorithm may be a face detection algorithm, and theanalysis unit 06 performs intelligent analysis on the gray-scale image using the face detection algorithm to obtain an analysis result, where the analysis result is whether the gray-scale image has a face image or not, or the analysis result is a detected face image.

The intelligent analysis algorithm can be obtained by training a neural network. For example, FastRCNN is a neural network, and a face detection algorithm can be derived by training FastRCNN.

Theencoding compression unit 05 can compress and encode the color image sequence generated by the imagesignal processing unit 2 by adopting the h.264 standard or the h.265 standard to obtain a video code stream.

The following briefly describes the relevant components in the present embodiment:

light filling ware 021 carries out near-infrared light filling with the stroboscopic mode, does not exclude to produce visible light filling with certain mode simultaneously, specifically is: the light supplement device does not perform near-infrared light supplement in the exposure time period of the second preset exposure, and performs near-infrared light supplement in the exposure time period of the first preset exposure.

Thefiltering component 03 can allow the near-infrared supplementary light generated by thesupplementary light 02 to pass through, and simultaneously allow visible light to pass through, so as to block other light.

Every pixel homoenergetic inimage sensor 01 can respond to near-infrared light, guarantees that the near-infrared image of gathering has clear detail under the light filling, helps promoting intelligent analysis effect.

Each pixel in theimage sensor 01 can sense at least one of the visible light of red light, green light and blue light, or can sense all the three visible lights simultaneously, so that the collected color video has sufficient resolution.

Theimage sensor 01 adopts different exposure parameters in the first preset exposure and the second preset exposure, the exposure parameters include and are not limited to exposure time, analog gain, digital gain and aperture size, and the image acquired for intelligent analysis is ensured to have proper exposure.

Thelight supplement device 02 supplements light in near infrared, energy is concentrated in the range of 700 nm-800 nm or 900 nm-1000 nm, and 800 nm-900 nm is avoided, so that interference caused by common 850nm infrared lamps is reduced.

Thelight supplement device 02 generates visible light supplement light in addition to the near-infrared supplement light, so that the near-infrared supplement light looks less reddish, and is prevented from being mixed with traffic lights.

Thefiltering component 03 can enable the near-infrared supplementary lighting to pass through, specifically, in a near-infrared band of 650nm to 1000nm, the width of the passing band is smaller than the sum of the widths of the blocked bands, so that interference of other light sources is reduced as much as possible on the premise that the infrared supplementary lighting is effectively utilized.

Thefiltering component 03 can enable the near-infrared supplementary lighting to pass through, specifically, in a near-infrared band of 650nm to 1000nm, the band width with the passing rate larger than 30% is smaller than 100nm, and interference of other light sources is reduced as much as possible on the premise that the infrared supplementary lighting is effectively utilized.

Thefilter assembly 03 has a switching member 033, which can be switched to a second state in which the filter assembly blocks near-infrared light and allows visible light to pass therethrough, in addition to the first state, so that the image capturing apparatus can be easily switched to an image capturing mode of an existing camera, and is compatible with an existing use mode.

An imagesignal processing unit 04 that performs different processing steps when processing the first image signal and the second image signal; and when sharpening is carried out, the adopted processing intensities are different, and particularly, the weaker sharpening intensity is adopted for the second image signal, so that the method is more suitable for the requirement of intelligent analysis.

The imagesignal processing unit 04, when generating a color image, may fuse the first image signal and the second image signal to generate a color image with higher quality, and is more suitable for the requirement of the video code stream for human eyes to watch.

In an embodiment of the present application, an image capturing apparatus includes: the device comprises an image sensor, a light supplementing device and a light filtering component, wherein the image sensor is positioned on the light emergent side of the light filtering component; the device also comprises an image signal processing unit, a coding compression unit and an analysis unit; the light supplementing device comprises a first light supplementing device, near-infrared light supplementing is carried out in a stroboscopic mode through the first light supplementing device, namely, the near-infrared light supplementing is carried out in the exposure time period of the first preset exposure of multiple exposure of the image sensor, and the near-infrared light supplementing is not carried out in the exposure time period of the second preset exposure of the multiple exposure; the light filtering component comprises a first light filter, the first light filter is used for allowing visible light and partial near infrared light to pass through, the near infrared light passing through the first light filter in the exposure time period of the first preset exposure comprises near infrared light which is reflected by an object and enters the light filtering component when the first light supplementing device performs near infrared light supplementing, and the near infrared light passing through the first light filter in the exposure time period of the second preset exposure comprises near infrared light which is reflected by the object and enters the light filtering component when the first light supplementing device does not perform near infrared light supplementing; generating and outputting a first original image signal and a second original image signal through multiple exposures by using an image sensor, wherein the first original image signal is an image signal generated according to a first preset exposure, the second original image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are two exposures of the multiple exposures; using an image signal processing unit to perform first processing on a first original image signal to obtain a gray image, and performing second processing on a second original image signal to obtain a color image; the coding compression unit is used for carrying out compression coding on the color image to obtain a video code stream; and intelligently analyzing the gray level image by using an analysis unit to obtain an analysis result. Because the first light supplementing device performs near-infrared light supplementing in the exposure time period of the first preset exposure, the image sensor senses the ambient light and the near-infrared light in the external environment according to the exposure parameters of the first preset exposure in the exposure time period of the first preset exposure to obtain a first original image signal, noise in the first original image signal can be reduced, the first original image signal is processed to obtain a gray-scale image, the noise of the gray-scale image is reduced accordingly, analysis is performed based on the gray-scale image, and the precision of an analysis result can be improved.

Referring to fig. 20, an embodiment of the present application provides an image capturing method, which is applied to an image capturing device, where the image capturing device may be any image capturing apparatus in the embodiment shown in fig. 1, that is, the image capturing device includes animage sensor 01, alight supplement device 02, and afilter assembly 03, theimage sensor 01 is located on a light emitting side of thefilter assembly 03, thelight supplement device 02 includes a first light supplement device, and thefilter assembly 03 includes a first optical filter. The method comprises the following steps:

step 201: the image acquisition equipment carries out near-infrared light supplement through a first light supplement device, wherein the first light supplement device carries out near-infrared light supplement at least in a part of exposure time period of first preset exposure of multiple exposure of the image sensor, does not carry out near-infrared light supplement in the exposure time period of second preset exposure of the multiple exposure, and the first preset exposure and the second preset exposure are two times of exposure in the multiple exposure.

The firstlight supplement device 021 is a device capable of emitting near-infrared light, such as a near-infrared light supplement lamp, and the embodiment of the present application does not limit this. The firstlight supplement device 021 can perform near-infrared light supplement in a stroboscopic manner, and can also perform near-infrared light supplement in other manners similar to stroboscopic. In some examples, when the firstlight supplement device 021 performs near-infrared light supplement in a stroboscopic manner, the firstlight supplement device 021 may be controlled in a manual manner to perform near-infrared light supplement in the stroboscopic manner, or the firstlight supplement device 021 may be controlled in a software program or a specific device to perform near-infrared light supplement in the stroboscopic manner, which is not limited in this embodiment. The time period of the firstlight supplement device 021 for performing near-infrared light supplement may coincide with the exposure time period of the first preset exposure, or may be greater than the exposure time period of the first preset exposure or smaller than the exposure time period of the first preset exposure, as long as the near-infrared light supplement is performed in the whole exposure time period or part of the exposure time period of the first preset exposure, and the near-infrared light supplement is not performed in the exposure time period of the second preset exposure.

For example, referring to fig. 3, the exposure time periods of the first preset exposure and the second preset exposure are alternately and cyclically occurred, the image capturing device generates and emits near-infrared light during the exposure time period of the first preset exposure to implement near-infrared supplementary lighting, and stops generating near-infrared light during the exposure time period of the second preset exposure to implement non-near-infrared supplementary lighting. The so-called stroboscopic mode for near-infrared fill-in is to generate and emit near-infrared light during an exposure time period of a first predicted exposure, and to stop generating near-infrared light during an exposure time period of a second preset exposure adjacent to the exposure time period of the first predicted exposure.

The firstlight supplement device 021 can be controlled to perform near-infrared light supplement in a stroboscopic manner in a manual manner, and the firstlight supplement device 021 can also be controlled to perform near-infrared light supplement in a stroboscopic manner through a software program or specific equipment, which is not limited in the embodiment of the present application. The time period of the firstlight supplement device 021 for performing near-infrared light supplement may coincide with the exposure time period of the first preset exposure, or may be greater than the exposure time period of the first preset exposure or smaller than the exposure time period of the first preset exposure, as long as the near-infrared light supplement is performed in the whole exposure time period or part of the exposure time period of the first preset exposure, and the near-infrared light supplement is not performed in the exposure time period of the second preset exposure.

In the exposure time period of the first preset exposure, the firstlight supplement device 021 generates and emits near infrared light to the external environment, so that the external environment includes ambient light and the near infrared light generated by the firstlight supplement device 021 in the exposure time period of the first preset exposure, the ambient light includes visible light, and may include at least one of near infrared light and infrared light, and the like, so that in the exposure time period of the first preset exposure, light reflected by an object and entering the firstoptical filter 031 in the image acquisition device includes the near infrared light generated by the firstlight supplement device 021 and the ambient light, and the firstoptical filter 031 passes through the visible light and the near infrared light in the near infrared light and the ambient light. In the exposure time period of the second preset exposure, the firstlight supplement device 021 stops generating the near infrared light, so that the external environment includes the ambient light in the exposure time period of the second preset exposure, the light reflected by the object and entering the firstoptical filter 031 includes the ambient light, and the firstoptical filter 031 passes through the visible light and the near infrared light in the ambient light.

Step 202: the image capturing device passes the visible light and a part of the near infrared light through the firstoptical filter 031.

Thefiltering assembly 03 in the image capturing device includes afirst filter 032, asecond filter 032 and a switching member 033, and both thefirst filter 031 and thesecond filter 032 are connected to the switching member 033;

thefirst filter 031 is switched to the light incident side of theimage sensor 01 by the switching member 033; the firstoptical filter 031 is configured to pass visible light and a part of near-infrared light, wherein the near-infrared light passing through the firstoptical filter 031 in the exposure time period of the first preset exposure includes near-infrared light reflected by the object and entering theoptical filter assembly 03 when the firstlight supplement device 021 performs near-infrared light supplement, and the near-infrared light passing through the firstoptical filter 031 in the exposure time period of the second preset exposure includes near-infrared light reflected by the object and entering theoptical filter assembly 03 when the firstlight supplement device 021 does not perform near-infrared light supplement; the intensity of the near-infrared light passing through the firstoptical filter 031 when the firstlight supplement device 021 performs near-infrared light supplement is higher than the intensity of the near-infrared light passing through the firstoptical filter 031 when the firstlight supplement device 021 does not perform near-infrared light supplement.

Switching thesecond filter 032 to the light incident side of theimage sensor 01 by the switching member 033; the secondoptical filter 032 passes light in the visible light band and blocks light in the near-infrared light band, and the image capturing device generates and outputs a third image signal through exposure of theimage sensor 01.

Step 203: the image acquisition device generates a first original image signal and a second original image signal through multiple exposures, wherein the first original image signal is an image signal generated according to a first preset exposure, and the second original image signal is an image signal generated according to a second preset exposure.

Since the firstlight supplement device 021 supplements the near-infrared light in the exposure time period of the first preset exposure, the external environment includes the near-infrared light supplemented by the firstlight supplement device 021 and the ambient light, and the ambient light includes visible light and near-infrared light. An object in an external environment can reflect near-infrared light supplemented by the firstlight supplementing device 021 to image acquisition equipment and reflect ambient light to the image acquisition equipment, and the image acquisition equipment can expose the reflected near-infrared light and visible light and near-infrared light in the ambient light through theimage sensor 01 to generate a first original image signal.

The firstlight supplement device 021 does not supplement the near infrared light in the exposure time period of the second preset exposure, so the external environment includes the ambient light including the visible light and the near infrared light. An object in the external environment may reflect ambient light to the image capture device, and the image capture device may expose the reflected visible light and near-infrared light in the ambient light to generate a second raw image signal via theimage sensor 01.

Theimage sensor 01 includes a plurality of light sensing channels for sensing light in at least two different visible light bands. Wherein each photosensitive channel is used for sensing at least one color of light in a visible light band and light in a near infrared band, and the at least one color of visible light comprises red light, green light, blue light, yellow light and the like.

As an example, during an exposure time period of a first preset exposure, the image capture device performs exposure based on exposure parameters of the first preset exposure to generate a first original image signal; and in the exposure time period of the second preset exposure, the image acquisition equipment performs exposure based on the exposure parameters of the second preset exposure to generate a second original image signal.

For the exposure parameters of the first preset exposure and the exposure parameters of the second preset exposure, the exposure parameters comprise at least one of exposure time, analog gain, digital gain and aperture size.

The exposure parameter of the first preset exposure and the exposure parameter of the second preset exposure may be the same, or at least one of the exposure parameters of the first preset exposure and the exposure parameter of the second preset exposure may be different.

In the case that the exposure parameter of the first preset exposure and the at least one exposure parameter of the second preset exposure are different, the at least one exposure parameter is at least one of exposure time, exposure gain, and aperture size, and the exposure gain includes analog gain, and/or digital gain.

At least one parameter in the exposure parameters of the first preset exposure is different from at least one parameter in the exposure parameters of the second preset exposure. As an example, the exposure gain of the first preset exposure is smaller than the exposure gain of the second preset exposure. In the exposure time period of the first preset exposure, the intensity of the near-infrared light sensed by the image acquisition device is strong, and accordingly, the brightness of the near-infrared light included in the first original image signal generated and output is also high. But the higher brightness near infrared light is not favorable for the acquisition of external scene information. The larger the exposure gain is, the higher the brightness of the image signal generated by the exposure of the image acquisition device is, and the smaller the exposure gain is, the lower the brightness of the image signal generated by the exposure of the image acquisition device is, therefore, in order to ensure that the brightness of the near-infrared light included in the first original image signal is within a proper range, the exposure gain of the first preset exposure may be smaller than the exposure gain of the second preset exposure. Therefore, when the image acquisition equipment performs near-infrared light compensation, the brightness of near-infrared light contained in a first original image signal generated by exposure of the image acquisition equipment is not too high due to the fact that the image acquisition equipment performs near-infrared light compensation. And/or the presence of a gas in the gas,

as an example, the exposure time of the first preset exposure is smaller than the exposure time of the second preset exposure. Wherein the longer the exposure time, the higher the brightness of the image generated by the exposure of the image acquisition device, and the shorter the exposure time, the lower the brightness included in the image signal generated by the exposure of the image acquisition device. Therefore, in order to ensure that the brightness of the near-infrared light included in the first original image signal is within a proper range, the exposure time of the first preset exposure may be made smaller than the exposure time of the second preset exposure. Therefore, when the image acquisition equipment performs near-infrared light compensation, the brightness of near-infrared light contained in a first original image signal generated by exposure of the image acquisition equipment is not too high due to the fact that the image acquisition equipment performs near-infrared light compensation. In addition, when a moving object is shot, the motion tailing phenomenon is not easy to occur in an image signal due to short exposure time, so that high imaging quality is ensured.

As an example, when the exposure gain of the first preset exposure is smaller than the exposure gain of the second preset exposure, the exposure time of the first preset exposure may be not only smaller than the exposure time of the second preset exposure but also equal to the exposure time of the second preset exposure. Similarly, when the exposure time of the first preset exposure is shorter than the exposure time of the second preset exposure, the exposure gain of the first preset exposure may be not only smaller than the exposure gain of the second preset exposure, but also equal to the exposure gain of the second preset exposure.

As an example, the number of times of light supplement by the image capturing device in a unit time length is lower than the number of times of exposure by the image capturing device in the unit time length, wherein one or more exposures are spaced in each interval period of two adjacent light supplements.

In this step, the multiple exposure operation of the image capturing device may include the following two modes:

in the first mode, the image acquisition device performs multiple exposures in a global exposure mode, and for any near-infrared supplementary light, the time period of the near-infrared supplementary light does not intersect with the nearest exposure time period of the second preset exposure, and the time period of the near-infrared supplementary light is a subset of the exposure time period of the first preset exposure, or the time period of the near-infrared supplementary light intersects with the exposure time period of the first preset exposure, or the exposure time period of the first preset exposure is a subset of the near-infrared supplementary light.

In the second mode, the image acquisition equipment adopts a roller shutter exposure mode to carry out multiple exposure, and for any near-infrared supplementary light, the time period of the near-infrared supplementary light does not intersect with the nearest exposure time period of the second preset exposure; the starting time of the near-infrared supplementary lighting is not earlier than the exposure starting time of the last line of effective images in the first preset exposure, and the ending time of the near-infrared supplementary lighting is not later than the exposure ending time of the first line of effective images in the first preset exposure; or the starting time of the near-infrared supplementary lighting is not earlier than the exposure ending time of the last line of effective images of the nearest second preset exposure before the first preset exposure and not later than the exposure ending time of the first line of effective images in the first preset exposure, and the ending time of the near-infrared supplementary lighting is not earlier than the exposure starting time of the last line of effective images in the first preset exposure and not later than the exposure starting time of the first line of effective images of the nearest second preset exposure after the first preset exposure; or the starting time of the near-infrared supplementary lighting is not earlier than the exposure ending time of the last line of effective images of the nearest second preset exposure before the first preset exposure and not later than the exposure starting time of the first line of effective images in the first preset exposure, and the ending time of the near-infrared supplementary lighting is not earlier than the exposure ending time of the last line of effective images in the first preset exposure and not later than the exposure starting time of the first line of effective images of the nearest second preset exposure after the first preset exposure.

The multiple exposure comprises odd exposure and even exposure;

as an example, the first preset exposure is one exposure of odd-numbered exposures, and the second preset exposure is one exposure of even-numbered exposures.

As an example, the first preset exposure is one exposure of even-numbered exposures, and the second preset exposure is one exposure of odd-numbered exposures.

As an example, the first preset exposure is one exposure of a designated odd number of exposures, and the second preset exposure is one exposure of other exposures other than the designated odd number of exposures.

As an example, the first preset exposure is one exposure of a specified even-numbered exposure, and the second preset exposure is one exposure of other exposures except the specified even-numbered exposure.

As an example, the first preset exposure is one exposure in a first exposure sequence and the second preset exposure is one exposure in a second exposure sequence.

It should be noted that, since human eyes easily mix the color of the near-infrared light supplementary lighting performed by the image capturing device with the color of the red light in the traffic light, the image capturing device also performs visible light supplementary lighting. Like this, if image acquisition equipment provides the visible light filling at the partial exposure time of first preset exposure at least, promptly, carries out near-infrared light filling and visible light filling at the partial exposure time quantum of first preset exposure at least, and the mixed colour of these two kinds of light can be distinguished from the colour of the red light in the traffic light to the colour that the human eye carries out near-infrared light filling with image acquisition equipment and the colour of the red light in the traffic light are confused has been avoided. In addition, if the image capturing device provides the visible light supplementary light within the exposure time period of the second preset exposure, since the intensity of the visible light within the exposure time period of the second preset exposure is not particularly high, the brightness of the visible light in the second original image signal can be further improved when the visible light supplementary light is performed within the exposure time period of the second preset exposure, and the image capturing quality can be further ensured.

As an example, the implementation manner of the image capturing device to perform visible light supplementary lighting may be:

the image acquisition equipment performs visible light supplementary lighting in a normally bright mode; or

The image acquisition equipment carries out visible light supplementary lighting in a stroboscopic mode, wherein the visible light supplementary lighting is carried out at least in a part of exposure time period of the first preset exposure, and the visible light supplementary lighting is not carried out in the whole exposure time period of the second preset exposure; or

The image acquisition equipment carries out visible light supplementary lighting in a stroboscopic mode, wherein the visible light supplementary lighting is not carried out at least in the whole exposure time period of the first preset exposure, and the visible light supplementary lighting is carried out in the partial exposure time period of the second preset exposure.

Step 204: the image acquisition equipment obtains a gray image according to the first original image signal and obtains a color image according to the second original image signal.

In this step, the image capturing device performs a first signal processing on the first original image signal to obtain a grayscale image, and performs a second signal processing on the second original image signal to obtain a color image.

The first signal processing includes at least a first sharpening process, and the second signal processing includes at least a second sharpening process, the intensity of the first sharpening process being less than the intensity of the second sharpening process.

The first signal processing further includes at least one of black level correction, Gamma correction, color correction, demosaicing, noise reduction, and the like, and the second signal processing further includes black level correction, Gamma correction, color correction. At least one of demosaicing or noise reduction.

As an example, the image capturing device may fuse the image signal after the first original image signal is preprocessed and the image signal after the second original image signal is preprocessed to obtain a fused image, and perform the second signal processing on the fused image to obtain the color image.

The first original image signal and the second original image signal are two images respectively, the image acquisition device may extract a first edge image from the image signal after the first original image signal is preprocessed, the first edge image includes edges of each object image in the image corresponding to the first original image signal, and extract a second edge image from the image signal after the second original image signal is preprocessed, the second edge image includes edges of each object image in the image corresponding to the second original image signal. And then the image acquisition equipment performs fusion according to the first formula to obtain a fused image, and performs second signal processing on the fused image to obtain a color image.

Step 205: and carrying out compression coding on the color image and carrying out intelligent analysis on the gray image.

In this step, a neural network computing unit in the SoC chip may be used, and a deep learning network, such as FastRCNN, may be used to perform intelligent analysis on the grayscale image, such as face detection.

In this step, an encoding compression module in the SoC chip may be used, for example, the standard of h.264 is used to compress the image sequence into a video code stream and output the video code stream.

In the step, the color image is compressed and coded to obtain a video code stream, and the gray image is intelligently analyzed to obtain an analysis result or obtain a target sub-image.

When the intelligent analysis is target detection or face detection, the obtained target subgraph can be a target image or a face image. For example, when the license plate is detected, the obtained target sub-image is a license plate image.

In the embodiment of the application, the image acquisition equipment performs near-infrared light supplement within the exposure time period of the first preset exposure, and does not perform near-infrared light supplement within the exposure time period of the second preset exposure; in this way, in the exposure time period of the first preset exposure, the image acquisition equipment exposes the near infrared light reflected by the object and the visible light and the near infrared light in the ambient light to obtain a first original image signal, and in the exposure time period of the second preset exposure, the image acquisition equipment exposes the visible light and the near infrared light in the ambient light reflected by the object to obtain a second original image signal; then the image acquisition equipment performs first signal processing on the first original image signal to obtain a gray image, performs second signal processing on the second original image signal to obtain a color image, performs compression coding on the color image to obtain a video code stream, and performs intelligent analysis on the gray image to obtain an analysis result. Because the image acquisition equipment carries out near-infrared light supplement within the exposure time period of the first preset exposure, the image acquisition equipment is exposed according to the exposure parameters of the first preset exposure within the exposure time period of the first preset exposure to obtain a first original image signal, the noise in the first original image signal can be reduced, the first original image signal is processed to obtain a gray image, the noise of the gray image is reduced, the analysis is carried out based on the gray image, and the precision of the analysis result can be improved.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An image acquisition apparatus, comprising: the device comprises an image sensor (01), a light supplementing device (02) and a light filtering component (03), wherein the image sensor (01) is positioned on the light emitting side of the light filtering component (03); the light filtering assembly (03) comprises a first light filter (031), a second light filter (032) and a switching component (033), the first light filter (031) and the second light filter (032) are connected with the switching component (033), the first light filter (031) enables visible light and partial near infrared light to pass through, and the second light filter (032) enables visible light to pass through and blocks the near infrared light; the device also comprises an image signal processing unit (04), a coding compression unit (05) and an analysis unit (06);

the image sensor (01) is configured to generate and output a first image signal and a second image signal through multiple exposures when the first optical filter (031) is located on a light entrance side of the image sensor (01), where the first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are two exposures of the multiple exposures;

the analysis unit (06) is used for carrying out intelligent analysis on the gray level image;

after the second filter (032) is switched to the light entrance side of the image sensor (01) by the switching member (033), the second filter (032) passes light in the visible light band and blocks light in the near-infrared light band, and the image sensor (01) is also configured to generate and output a third image signal by exposure.

2. The image capturing apparatus of claim 1,

the first preset exposure and the second preset exposure are different in at least one exposure parameter, the at least one exposure parameter is one or more of exposure time, exposure gain and aperture size, and the exposure gain comprises analog gain and/or digital gain.

3. The image capture device of claim 2, wherein an exposure gain of the first pre-set exposure is less than an exposure gain of the second pre-set exposure.

4. The image capturing device as claimed in claim 1, wherein at least one exposure parameter of the first and second predetermined exposures is the same, the at least one exposure parameter comprises one or more of exposure time, exposure gain, aperture size, the exposure gain comprises analog gain, and/or digital gain.

5. The image capturing device of claim 4, wherein the exposure time of the first preset exposure is equal to the exposure time of the second preset exposure.

6. The image capturing apparatus according to claim 1, wherein obtaining a gray-scale image from the first original image signal and obtaining a color image from the second original image signal comprises:

and at least carrying out first sharpening on the first original image signal to obtain a gray image, and at least carrying out second sharpening on the second original image signal to obtain a color image, wherein the intensity of the first sharpening is smaller than that of the second sharpening.

7. The image capturing apparatus according to claim 1, wherein said obtaining a color image from the second original image signal comprises:

and fusing the image of which the first original image signal is preprocessed and the image of which the second original image signal is preprocessed to obtain a fused image, and determining the fused image as the color image.

8. The image capturing apparatus according to claim 1, wherein the wavelength band of the near infrared light incident on the first filter (031) is a first reference wavelength band, and the first reference wavelength band is 650nm to 1100 nm.

9. The image capturing apparatus of claim 1,

when the central wavelength of the near-infrared light supplement performed by the first light supplement device (021) is a set characteristic wavelength or falls within a set characteristic wavelength range, the central wavelength and/or the waveband width of the near-infrared light passing through the first optical filter (031) reach a constraint condition.

10. The image capturing device according to claim 9, wherein the first fill-in light device (021) performs near-infrared fill-in light at any wavelength within a wavelength range of 750 ± 10 nm at a center wavelength; or

The center wavelength of the first light supplement device (021) for near-infrared light supplement is any wavelength within the wavelength range of 780 +/-10 nanometers; or

The center wavelength of the first light supplement device (021) for near-infrared light supplement is any wavelength within the wavelength range of 940 +/-10 nanometers.

11. The image capturing apparatus according to claim 9, wherein the constraint condition includes:

the difference value between the central wavelength of the near-infrared light passing through the first optical filter (031) and the central wavelength of the near-infrared light supplemented by the first light supplementing device (021) is within a wavelength fluctuation range, and the wavelength fluctuation range is 0-20 nanometers; or,

the constraint conditions include: the half bandwidth of the near infrared light passing through the first optical filter (031) is less than or equal to 50 nanometers; or,

the constraint conditions include: the first wave band width is smaller than the second wave band width; wherein the first wavelength band width refers to a wavelength band width of near infrared light passing through the first filter (031), and the second wavelength band width refers to a wavelength band width of near infrared light blocked by the first filter (031); or,

the constraint conditions include: the third wave band width is smaller than the reference wave band width, the third wave band width refers to the wave band width of the near infrared light with the passing rate larger than the set proportion, the reference wave band width is any wave band width in the wave band range of 50 nanometers to 150 nanometers, and the set proportion is any proportion in the proportion range of 30 percent to 50 percent.

12. The image capturing device according to claim 1, wherein the image sensor (01) comprises a plurality of photosensitive channels, each photosensitive channel being configured to sense light in at least one visible wavelength band and to sense light in a near infrared wavelength band.

13. The image capturing device of claim 12, wherein the plurality of photosensitive channels are configured to sense light in at least two different visible light bands.

14. An image acquisition device according to any one of claims 1 to 3, characterized in that the fill light (02) further comprises a second fill light device (022);

the second light supplement device (022) is used for supplementing visible light in a normally bright mode; or

The second light supplement device (022) is used for supplementing visible light in a stroboscopic manner, wherein the supplementing visible light is performed at least in a partial exposure time period of the first preset exposure, and the supplementing visible light is not performed in the whole exposure time period of the second preset exposure; or

The second light supplement device (022) is used for supplementing visible light in a stroboscopic mode, wherein the first preset exposure is not supplemented with the visible light in the whole exposure time period, and the second preset exposure is supplemented with the visible light in a partial exposure time period.

15. The image capturing device as claimed in claim 1, wherein the number of fill-in light of the first fill-in light device (021) per unit time length is lower than the number of exposure times of the image sensor (01) per unit time length, wherein the exposure times are spaced one or more times in each interval period of two adjacent fill-in lights.

16. The image capturing apparatus of claim 1,

the image sensor carries out multiple exposure in a global exposure mode, for any one time of near-infrared supplementary lighting, the time period of the near-infrared supplementary lighting does not have an intersection with the nearest exposure time period of the second preset exposure, the time period of the near-infrared supplementary lighting is a subset of the exposure time period of the first preset exposure, or the time period of the near-infrared supplementary lighting has an intersection with the exposure time period of the first preset exposure, or the exposure time period of the first preset exposure is a subset of the near-infrared supplementary lighting.

17. The image capturing apparatus of claim 1,

the image sensor performs multiple exposures in a rolling shutter exposure mode, and for any near-infrared supplementary light, the time period of the near-infrared supplementary light does not intersect with the nearest exposure time period of the second preset exposure;

the starting time of the near-infrared supplementary lighting is not earlier than the exposure starting time of the last row of effective images in the first preset exposure, and the ending time of the near-infrared supplementary lighting is not later than the exposure ending time of the first row of effective images in the first preset exposure;

or,

the starting time of the near-infrared supplementary lighting is not earlier than the exposure ending time of the last line of effective images of the nearest second preset exposure before the first preset exposure and is not later than the exposure ending time of the first line of effective images in the first preset exposure, and the ending time of the near-infrared supplementary lighting is not earlier than the exposure starting time of the last line of effective images in the first preset exposure and is not later than the exposure starting time of the first line of effective images of the nearest second preset exposure after the first preset exposure; or

The starting time of the near-infrared supplementary lighting is not earlier than the exposure ending time of the last line of effective images of the nearest second preset exposure before the first preset exposure and not later than the exposure starting time of the first line of effective images in the first preset exposure, and the ending time of the near-infrared supplementary lighting is not earlier than the exposure ending time of the last line of effective images in the first preset exposure and not later than the exposure starting time of the first line of effective images of the nearest second preset exposure after the first preset exposure.

18. A camera shooting method, applied to an image capturing device, wherein the image capturing device includes an image sensor (01), a light supplement device (02) and a light filtering assembly (03), the light filtering assembly (03) includes a first light filter (031), a second light filter (032) and a switching component (033), the first light filter (031) and the second light filter (032) are both connected to the switching component (033), the first light filter (031) allows visible light and part of near-infrared light to pass through, and the second light filter (032) allows visible light to pass through and blocks near-infrared light; the image sensor (01) is located on the light emergent side of the light filtering component (03), the light supplementing device (02) comprises a first light supplementing device, the light filtering component (03) comprises a first optical filter, and when the first optical filter is located on the light incident side of the image sensor, the method comprises the following steps:

the image acquisition equipment performs compression coding on the color image and performs intelligent analysis on the gray level image;

after the second filter (032) is switched to the light entrance side of the image sensor (01) by the switching component (033), the image acquisition apparatus passes light in the visible light band through the second filter (032), blocks light in the near-infrared light band, and the image sensor (01) also generates and outputs a third image signal by exposure.

19. The method of claim 18, wherein the image capture device obtains a grayscale image from the first original image signal and a color image from the second original image signal, comprising:

20. The method of claim 18, wherein the image capture device deriving a color image from the second raw image signal comprises: