【The content of the invention】
In order to overcome the deficiencies in the prior art, the invention provides at a kind of combination shot system, mobile terminal and imageReason method, realizes high-quality Color Image Acquisition and depth image collection with fewer resource.
A kind of combination shot system, including the first camera head, the second camera head, laser projection device and processor；
The laser projection device is used for, and the total visual field to first camera head and the second camera head projects notThe structured light patterns of visible laser；
First camera head is used for, and gathers the first otherwise visible light color image in the total visual field；
Second camera head is used for, and gathers the visible ray gray level image in the total visual field；
One of first camera head and the second camera head be additionally operable to gather in the total visual field can notSee light image；
The processor is used for, visible using first otherwise visible light color image and the synthesis second of visible ray gray level imageLight coloured image, and depth image is calculated using the black light image.
Preferably, the laser projection device is used for alternately projecting the structured light patterns and pass to the total visual fieldClose the projection structured light patterns；
Gathering one of described black light image in first camera head and the second camera head is used for, describedLaser projection device gathers the first black light image during projecting the structured light patterns, closes in the laser projection deviceThe second black light image is gathered during projecting the structured light patterns；
The processor is used for, using the first black light image and the difference image meter of the second black light imageCalculate the depth image.
Preferably, project in the cycle of the structured light patterns in the laser projection device alternating projection and closing, throwThe duration of the structured light patterns is penetrated more than the duration for closing the projection structured light patterns.
Preferably, the focal length of lens of first camera head is different from the focal length of lens of second camera head.
Preferably, the invisible laser is infrared ray laser.
Preferably, the laser projection device is arranged between first camera head and the second camera head.
Present invention also offers a kind of mobile terminal, including described combination shot system.
Present invention also offers a kind of image acquiring method, comprises the steps：
Projection step：Laser projection device is invisible to the total visual field projection of the first camera head and the second camera headThe structured light patterns of laser；
First acquisition step：First camera head gathers the first otherwise visible light color image in the total visual field,Second camera head gathers the visible ray gray level image in the total visual field；
Second acquisition step：One of first camera head and the second camera head also gather the total visual fieldIn black light image；
Process step：The processor utilizes first otherwise visible light color image and visible ray gray level image synthesis secondOtherwise visible light color image, and depth image is calculated using the black light image.
Preferably, in the projection step：
The laser projection device alternately projects the structured light patterns to the total visual field and closes described in projectionStructured light patterns；
In first acquisition step：
One of described black light image is gathered in first camera head and the second camera head, in the laserProjection arrangement gathers the first black light image during projecting the structured light patterns, closes in the laser projection device and projectsThe second black light image is gathered during the structured light patterns；
In the process step：
The processor utilizes the first black light image and the difference image of the second black light image to calculate instituteState depth image.
Preferably, project in the cycle of the structured light patterns in the laser projection device alternating projection and closing, throwThe duration of the structured light patterns is penetrated more than the duration for closing the projection structured light patterns.
The invention has the beneficial effects as follows：
Compared to prior art, the present invention can realize the high-quality colour to target with less volume, lower power consumptionThe collection of image and depth image.
【Specific embodiment】
Preferred embodiment to inventing is described in further detail below.
As shown in figure 1, a kind of combination shot system of embodiment, including the first camera head, the second camera head, laserProjection arrangement and processor, processor are electrically connected with the first camera head, the second camera head and laser projection device respectively.
The laser projection device is used for, and the total visual field to first camera head and the second camera head projects notThe structured light patterns of visible laser；First camera head is used for, and gathers the first otherwise visible light color in the total visual fieldImage, second camera head are used for, and gather the visible ray gray level image in the total visual field；First camera headIt is additionally operable to gather the black light image in the total visual field with one of the second camera head；The processor is used for,Synthesize the second otherwise visible light color image using first otherwise visible light color image and visible ray gray level image, and using describedBlack light image calculates depth image.
In Fig. 1, FOV represents the visual angle of the first camera head and the second camera head, the projection scope of laser projection deviceCover the total visual field of the first camera head and the second camera head.
As shown in Fig. 2 the typical structure schematic diagram of camera head, including imageing sensor 1, optical filter 2 and lens 3, lightThrough lens 3 convergence again after optical filter 3 by imageing sensor gather.Optical filter 2 is for by the light of specific wavelength, figureAs sensor 1 is used for converting light to digital electric signal.
The Baeyer optical filter that common RGB camera is adopted, optical filter is with identical with image sensor pixel quantity and oneOne corresponding filter unit, Baeyer optical filter have for respectively by the filter unit of HONGGUANG, green glow and blue light, light processThe pixel of imageing sensor corresponding with the filter unit is incided after certain filter unit.It is quicker to green glow in view of human eyeSense, therefore be R (25%) generally by the ratio setting of three:G (50%):B (25%).
As shown in figure 3, in a kind of embodiment, the first camera head is used for gathering the first otherwise visible light color image and can notSee light image, the second camera head is only used for gathering visible ray gray level image.Such as the first camera head adopts RGB-IR cameras,The optical filter which adopts is different from Baeyer optical filter, and the optical filter of the present embodiment is made up of four kinds of different filter units, respectivelyR, G, B and infrared light (IR) component can be passed through, four ratio is R (25%):G (25%):B (25%):IR (25%),By using this optical filter 2, the first camera head can both gather otherwise visible light color image, it is also possible to gather sightless redOuter light image.It is of course also possible to use the filter unit of other black lights substitutes IR filter units, to coordinate laser projection to fillPut other invisible laser of projection.The arrangement mode of each filter unit of the optical filter of the present embodiment and each component accounting are notUniquely, it is possibility to have other arrangements and distribution mode.
After imageing sensor in first camera head obtains the optical information of each component (such as R, G, B, IR), due toEach optical information all only occupies the pixel of part, so that recovering other three kinds points in each pixel by way of interpolationThe intensity information of amount, so that finally realize synchronous acquisition RGB image and IR images.The method of interpolation has various, such as plusWeight average etc., due to for prior art thus do not described in detail here.
The second camera head in the present embodiment is gray scale camera, and its optical filter is transparent, or does not have optical filter.Due to not filtering the light of some wavelength, thus stronger optical signal can be collected, even if under gloomy environmentAlso preferable image effect has been can guarantee that.
As shown in figure 4, in another embodiment, the first camera head is used only for gathering the first otherwise visible light color figurePicture, the second camera head are used for gathering visible ray gray level image and black light image.Fig. 4 second is taken the photograph in a kind of embodimentAs the optical filter that device is adopted, wherein IR represent infrared filtering unit, W represents white light filter element i.e. transparent cell, Ke YitongCross the light of any wavelength.
Laser projection device is different according to the structured light patterns of projection, and its structure is also different.The present embodiment is with speckle particleIllustrate as a example by structured light patterns.Laser projection device is typically by light source, collimating lens and diffraction optical element (DOE) groupInto.In the present embodiment, light source is iraser, can be that single edge emitting LASER Light Source, or vertical-cavity surface-emitting swashOptical arrays light source.As the light that LASER Light Source sends has certain angle of divergence so that using collimating lens collimated withLaunch the light beam of focal length.Laser beam is expanded into multiple laser Jing after DOE, and forms speckle particle pattern in space.
In much using scene, it is to gather image under the irradiation of the light sources such as sunlight, these light sources are usually containedThere is black light identical black light (such as infrared light) projected with laser projection device, the light note of this part is to collectionBlack light image produce impact, particularly in the case of light source is more strong, the black light image for gathering can be causedThe unfavorable situation such as contrast is relatively low, noise is big.In one embodiment, it is spaced to total visual field by controlling laser projection deviceProjection structure light pattern is eliminating this adverse effect.As shown in figure 5, being a kind of interval throwing of laser projection device of embodimentShadow sequential chart, such as closes laser projection device projection in the T1 time periods, and gathers corresponding black light pattern I1, and in phaseThe adjacent T2 time periods open laser projection device projection, and gather corresponding black light pattern I2... by that analogy.When adjacentBetween section spacing frequency be not more than black light image frequency acquisition, you can once can not with only gathering in a period of timeSee light image, it is also possible to gather multiple black light image.To illustrate how to below eliminate with frequency identical situationThe impact of black light image of the black light that light source is produced to gathering.
In the T1 time periods, laser projection device is closed, and the black light image I1 of acquisition is only comprising invisible in light sourceLight composition.In the T2 time periods, the black light image I2 of acquisition black light then simultaneously comprising laser projection device projection intoPoint and light source black light composition.A kind of processing method is, it is assumed that the illumination of two neighboring time period inner light source does not becomeChange and collected object is not moved, now can be eliminated the effects of the act by infrared-difference：Differentiated black light figureAs I2 '=I2-I1, such black light image I2 ' has more high-contrast.In another kind of processing method, it is possible to use in front and backAt least two width black light images doing difference, such as：I2 '=I2- (I3+I1)/2.Concrete calculus of finite differences processing method also has manyKind.
It is as laser projection device is to close within a period of time, therefore under conditions of frequency acquisition is constant, overallThe quantity of the black light image of acquisition is reduced, and causes certain impact to the follow-up depth image that calculates.In order to reduce this impact,The length of the length more than projection shut-in time section of projection opening time section, in the switch periods of laser projection device, is set(such as T2 is more than T1), obtain the collection frame number of higher black light image.
In one embodiment, a kind of image acquiring method, comprises the steps：
S1, laser projection device project invisible laser to the total visual field of the first camera head and the second camera headStructured light patterns.
S2, first camera head gather the first otherwise visible light color image in the total visual field, and described second takes the photographAs device gathers the visible ray gray level image in the total visual field.
One of S3, first camera head and second camera head also gather invisible in the total visual fieldLight image.
S4, the processor utilize first otherwise visible light color image and visible ray gray level image synthesis high-quality theTwo otherwise visible light color images.
In the present embodiment, the first camera head is identical with the size and resolution of the image acquisition device of the second camera head, thisSample visible ray gray level image and the first otherwise visible light color image only need to consider two image acquisition devices due to position in synthesisThe image registration problem that difference causes.The parameter of the first camera head and the second camera head can also in other embodimentsDiffer.Image registration algorithm is known technology more ripe at present, is not elaborated here.Due to visible ray gray-scale mapsSeem not filter through optical filter and directly receive acquired in visible ray, thus its light sensitivitys is stronger, can with first by whichSee that light coloured image carries out synthesis and can realize following effect：
Details strengthens.Under half-light environment, many details in the first otherwise visible light color image relatively will be obscured, nowIt will be seen that after in light gray level image, the details of the part is synthesized with the first otherwise visible light color image, the second visible ray of formationColoured image will possess the image effect for becoming apparent from.
Image zoom.First camera head and the focal length of lens of the second camera head are set to different, are such as respectivelyIt is near burnt with remote Jiao.As the first otherwise visible light color image is obtained under remote focal length, thus the subject image in image nearby comparesIt is fuzzy, and visible ray gray level image is obtained under nearly focal length mode, now with the information in visible ray gray level image mendingThe part obscured in repaying the first otherwise visible light color image, ultimately forms the second relatively sharp otherwise visible light color image.
S5, the processor calculate depth image using the black light image.
May be constructed based on the trigon depth survey unit of structure light by the first camera head and laser projection device.ObtainThe principle for taking depth information is the black light image that will collect to be carried out matching primitives with the reference picture that accesses in system and obtainsCapture element deviation value, then just can be with using the one-to-one relationship of pixel deviation value and actual grade value by principle of triangulationObtain the depth image of object space.Here reference picture is by gathering in the plane of depth camera known depth valueBlack light image and obtain.
As shown in fig. 6, a kind of mobile terminal 7 containing this combination shot system of embodiment, including processor, a shootingDevice 4, laser projection device 5 and the second camera head 6.First camera head 4, laser projection device 5 and the second camera head 6Can be arranged on the same surface of mobile terminal, laser projection device 5 is preferably provided at the first camera head 4 and second and takes the photographAs, between device 6, the first camera head 4, laser projection device 5 and the second camera head 6 are preferably provided on same straight line.Processor etc. can be made up of one or more in the CPU in mobile terminal, application specific processor, microelectronic component etc..It is mobileTerminal can be mobile phone, flat board, computer etc..
Above content is with reference to specific preferred implementation further description made for the present invention, it is impossible to assertThe present invention be embodied as be confined to these explanations.For general technical staff of the technical field of the invention,On the premise of without departing from present inventive concept, some simple deduction or replace can also be made, should all be considered as belonging to the present invention byThe scope of patent protection that the claims submitted to determine.