CN102567968A - Image algorithm based on micro-multispectral optical filter - Google Patents

Abstract

The invention provides an image algorithm based on a micro-multispectral optical filter, which enables a pair of images obtained by an image sensor with a plurality of (N) optical filters in a pasting mode to be classified into a plurality of (N) monochrome images; demarcating optical filtering pixels and image sensor pixels; and obtaining multispectral imaging pixels and multispectral imaging gray value of the monochrome images. The image algorithm comprises an image algorithm formula under ideal conditions and a correction method which can be used after corrected under non-ideal conditions, for example, tolerance exists on the optical filter and errors exist in an optical filter and image sensor assembling process.

Description

Image algorithm based on little multispectral optical filter

Technical field

The present invention relates to a kind of method for correcting image, especially relate to a kind of image algorithm based on little multispectral optical filter.

Background technology

In the biomedical clinical deagnostic test; Biological chromophore and biomarker show the chromophoric optical characteristics of absorption peak of the one group of discrete wavelength of himself; Also be the chromophoric optical information of human fingerprint material; Between pathological tissues and normal tissues, pathological tissues is different with normal tissues on three or four wave bands.Through miniature narrowband light filter and imageing sensor is integrated and this stack features fingerprint material imaging selectively, their optical characteristics of real-time monitored and space distribution are significant to the clinical medicine biological diagnosis of disease.

50 years in the past, the filtering device that multispectral technology is often used in instrument was a kind of simple optical filtering wheel disc, and it remains in many practical applications the multispectral instrument of usefulness at present.Its volume is about 35cm * 35cm * 35cm, and weight is about 2kg.Volume is big, and is heavy, involves great expense, and machinery itself and artificial motion cause the distortion of imaging.Simultaneously, exposure of filtering wheel disc filtering mode can only be obtained the image of a wavelength, obtain the multiple image of different wave length and will constantly adjust the optical filtering wheel disc.When needs are obtained the view data of a plurality of wavelength, need constantly repeat to adjust light filter, all generate up to image corresponding to a plurality of different wave lengths through band centre wavelength.In constantly the obtaining of view data, cause following problem easily: one of which in most of the cases, light filter all has with respect to the mechanical motion of imageing sensor and vibration; It is two in the process of obtaining these a few width of cloth multispectral images, and target object that is captured or part object are also in motion, and especially the active bio body is also motion (breathing heartbeat).Like this, because the mobile actual object that causes of equipment itself and subject exists dislocation and distortion between the multispectral front and back image that obtains.Before formally the diagnosis target being analyzed, need to drop into a large amount of calculating strength and carry out spatial calibration.This just makes and uses by traditional multi-optical spectrum imaging technology complex operation consuming time.

Summary of the invention

Technical matters to be solved by this invention provides a kind of image algorithm based on little multispectral optical filter, its can solve in the prior art since equipment itself and subject move or multispectral front and back image that sensor pixel causes between dislocation and the technical matters of being out of shape.

For solving the problems of the technologies described above, technical scheme of the present invention provides a kind of image algorithm based on little multispectral optical filter, on imageing sensor, pastes the multiple tracks optical filter, makes the corresponding a plurality of monochrome images of an original image, specifically comprises:

Step S1: judge whether optical filter accurately covers an integer pixel of said imageing sensor, if then be judged to be ideal state, the computed image gray-scale value; If not, then be judged as, forward step S2 in non-ideal conditions;

Step S2: obtain pixel index (i_Te, j_Te), estimation X-Y dimension linear displacement is used for revising optical filter and imageing sensor translation that assembling process causes; The estimated angle displacement is used for revising optical filter and imageing sensor rotation error that assembling process causes; Estimation cycle ratio is used to correct the displacement that optical filter does not cover an integer image sensor pixel, is divided into the optical channel of corresponding a plurality of wavelength respectively to the optical lightscreening pixel, obtains the index map of each optical channel, carries out the calculating of gradation of image value again.

The computing method of gradation of image value specifically comprise among the said step S1,

$I(x,y)=\frac{1}{n^2}\underset{i=0,j=0}{\overset{i=3,j=3}{\mathrm{\Σ}}}X(2\mathrm{nx}+i,2\mathrm{ny}+j),\mathrm{wherex}\∈\left[\mathrm{1,160}\right],y\∈[1,\stackrel{15}{128}]$

Wherein, said n is the number of optical filter, and (x y) represents an optics locations of pixels, index (i to each group_Te, j_Te) be the unit picture element of imaging sensor, I (x, y) be optical lightscreening pixel position for (x, the gray-scale value of y) locating, thereby according to the optics locations of pixels (x, y) and index (i_Te, j_Te) obtain the gradation of image value.

The method of calculating angle displacement among the said step S2 is:

Angle displacement is the displacement of the optical lightscreening pixel and the angle between the image sensor pixel of multispectral optical filter, index (i₁, j₁) be meant first pixel index, (i_f, j_f) be meant the pixel of last imageing sensor of light field.

The evaluation method of cycle ratio is among the said step S2,

The quantity ratio of the quantity that said cycle ratio is an imaging pixel and light field grid; Index N is the number of light field.

Among the said step S2, obtain the index map that obtains each optical channel, specifically may further comprise the steps,

S201: the pixel index (i that from S1, obtains_Te, j_Te) as initial starting position, first 4 * 4 optical lightscreening pixel is set belongs to first wavelength;

S202: obtain pixel index (i_Te, j_Te), estimation X-Y dimension linear displacement is obtained angle displacement and the value of cycle ratio among the S2, estimates the position of next light field grid;

S203: around the estimated position of the light field grid that in the S202 step, obtains, seek the brightest pixel index, and these pixel indexs are labeled as the optical channel that belongs to first wavelength;

S204: repeating step S202-S203 is searched for until all original images entirely, all accomplish as the optical channel of first wavelength, and to write down all pixel indexs that belong to first wavelength is the index map of first wavelength;

S205: repeating step S201-S204 for all wavelengths is set up corresponding optical channel respectively, is kept at one 160 * 128 matrix, the corresponding spectrum channel of each matrix with the coordinate in all upper left corners of 3 * 3 pixel grids.

Adopt technique scheme; Multispectral imaging dislocation and the problem of being out of shape in the contrast prior art; The invention has the beneficial effects as follows: owing to adopted the reflection deviation that linear displacement and angle displacement and periodic law are not matched and produced; The maximum probability of that passage that proposition belongs to according to it; Each pixel of original image is categorized into four error compensations in the multispectral image pixel, through obtaining the index map of each optical channel, between the multispectral imaging of the pixel of the multispectral original image that obtains and identical wavelength, has set up position correction.

Description of drawings

Fig. 1 is the process flow diagram of the figure bearing calibration based on little multispectral optical filter provided by the present invention;

Fig. 2 is the optical lightscreening pixel network trrellis diagram that the present invention provides the little multispectral optical filter that is applied to early stage pressure sore detection;

Fig. 3 is the grid chart of multispectral optical filter provided by the invention;

Fig. 4 is row's original image of the monochromatic wavelength light of multispectral optical filter provided by the invention.

Embodiment

By specifying technology contents of the present invention, structural attitude, realized purpose and effect, give explanation below in conjunction with embodiment and conjunction with figs. are detailed.

In the present invention, multispectral optical filter is designed to the checkerboard type grid, and the area of each miniature multispectral optical filter is 7mm * 8mm, and it contains the optical lightscreening pixel of 128 * 160 um magnitudes.Each optical lightscreening pixel size is 20.8um * 20.8um.The optical lightscreening pixel of these um magnitudes becomes two-dimensional array by the checkerboard type layout arrangement.The micro-filtration sheet can directly stick on the commercial monochrome cameras CCD/CMOS imageing sensor through micro production and little assembling mode.Aim between optical lightscreening pixel and the image sensor pixel, make single exposure take a picture or camera system can obtain the multiple image corresponding to different wave length simultaneously in real time at one.The Pixel Dimensions size of monochrome image sensor CCD/CMOS has difference with the optical lightscreening pixel on physical dimension, for example Pixel Dimensions is that 5.2um * 5.2um or Pixel Dimensions are 4.65um * 4.65um.Therefore will obtain pixel and pixel, the multiple image of aiming between image and the image will once be proofreaied and correct demarcation to each multispectral imaging result of optical lightscreening pixel and image sensor pixel.Ideal situation is the integral multiple that grid cell optical lightscreening pixel covers an imaging sensor pixel accurately; But have the displacement of the linear displacement or the angle of X-Y dimension during practical application, some optical lightscreening pixels appear in this displacement in micro-filtration sheet and sensor are integrated may to cover of imaging sensor pixel non-integral multiple.The linear displacement of X and Y dimension and the displacement of the anglec of rotation cause belonging to the dislocation between some neighbors of different spectrum channels.In addition because the error of the optical filter of making or pixel size are not that 5.2um * 5.2um (is the deviation that 4.65um * 4.65um) also can cause imaging to the ccd image sensor pixel for example to the cmos image sensor pixel.

The Pixel Dimensions size of monochrome image sensor CCD/CMOS has difference with the optical lightscreening pixel on physical dimension, for example Pixel Dimensions is that 5.2um * 5.2um or Pixel Dimensions are 4.65um * 4.65um.Therefore will obtain pixel and pixel, the multiple image of aiming between image and the image will once be proofreaied and correct demarcation to each multispectral imaging result of optical lightscreening pixel and image sensor pixel.

See also Fig. 1 and Fig. 2, in one embodiment, in the concrete application that pressure sore detects, relevant chromophore is respectively a melanin, aerobic haemoglobin, anaerobic haemoglobin and water.They comprise one group of four central wavelength lambda 1=650nm, λ 2=577nm, λ 3=540nm, the optical information of λ 4=970nm.The wavelength of 540nm and 577nm are corresponding to the absorption peak of haemoglobin, and 650nm provides melanin or background image, the peak value of 970nm sign water.

In the present embodiment, the optical lightscreening pixel geometry is of a size of 20.8um * 20.8um, and the Pixel Dimensions size of the monochrome image sensor CMOS that is adopted is 5.2um * 5.2um.Therefore will obtain pixel and pixel, the multiple image of aiming between image and the image will once be demarcated each multispectral imaging result of optical lightscreening pixel and image sensor pixel.Ideal situation is the integral multiple that grid cell optical lightscreening pixel covers an imaging sensor pixel accurately, and for example grid 41.6um * 41.6um optical lightscreening pixel comprises 8 * 8CMOS pixel.

In certain embodiments, source of error mainly is:

1, when making miniature multispectral optical filter; As shown in Figure 2; It is not a constant (41.6 μ m * 41.6 μ m) strictly speaking that error in dipping causes the grid cell size; Ideal situation is the integral multiple that grid cell optical lightscreening pixel covers an imaging sensor pixel accurately, and for example grid 41.6um * 41.6um optical lightscreening pixel comprises 8 * 8CCD pixel.In addition because optical filter error or the pixel size made are not that 5.2um * 5.2um also can cause the dislocation between optical lightscreening pixel and the imaging sensor pixel.

2, miniature multispectral optical filter and imageing sensor closely paste integrated in; Have X-Y dimension linear displacement between optical lightscreening pixel and the image sensor pixel, it is non-integral multiple that the result causes some grid cells may cover of imaging sensor pixel.Between the optical filtering pixel that in grid cell, is comprised, form the boundary effect of 1-2nm gap.Can produce the reflection mistake thus.

3, miniature multispectral optical filter and imageing sensor closely paste integrated in, have the displacement of the anglec of rotation between optical lightscreening pixel and the image sensor pixel.

4, in micro-filtration sheet and sensor are integrated, occur the linear displacement and the anglec of rotation displacement of X and Y dimension simultaneously, can introduce the dislocation between some neighbors that belong to different spectrum channels.

For this reason, present embodiment provides a kind of image algorithm based on little multispectral optical filter,

When optical filter accurately covers integer the pixel of said imageing sensor is ideal state, the computed image gray-scale value; Under non-ideal conditions, estimation X-Y dimension linear displacement is used for revising optical filter and imageing sensor translation that assembling process causes; The estimated angle displacement is used for revising optical filter and imageing sensor rotation error that assembling process causes; Estimation cycle ratio is used to correct the displacement that optical filter does not cover an integer image sensor pixel.

Under ideal state, the computed image gray-scale value, following steps,

S1: obtain pixel index (i_Te, j_Te), estimation X-Y dimension linear displacement;

Obtain pixel index (i among the said step S1_Te, j_Te) specifically comprise: search for first 4 * 4 bright, the manual top of operator light field is aimed at left column estimation linear displacement, the index (i in its upper left corner_Te, j_Te) be the linear displacement of unit picture element on the X-Y direction of imaging sensor, (inferior pixel size will produce the linear displacement of um unit's level).Concrete, when first grid of little multispectral optical filter covers the pixel of first imaging sensor, index (i so_Te, j_Te) two all be 0.

See also Fig. 3; An original image upper left corner part of the monochromatic wavelength light that multispectral imaging obtains; The wavelength channel linear displacement of first grid cell upper left corner optical lightscreening pixel of micro-filtration sheet detects application example for early stage pressure sore, and the monochromatic light that is used to calibrate is wavelength 650nm.According to the pixel index of image coordinate system, the linear displacement of selected portion between estimation micro-filtration sheet and the potential imaging sensor.

Wherein, Estimation X-Y dimension linear displacement specifically comprises: owing among this embodiment, have four different wavelengths, it is corresponding with imaging sensor therefore 4 optical filters need to be set; Be n=4; Because the computing method of the gray-scale value that the present invention adopts are, to the demarcation of optical lightscreening pixel and image sensor pixel

$I(x,y)=\frac{1}{n^2}\underset{i=0,j=0}{\overset{i=3,j=3}{\mathrm{\Σ}}}X(2\mathrm{nx}+i,2\mathrm{ny}+j),\mathrm{wherex}\∈\left[\mathrm{1,160}\right],y\∈\left[\mathrm{1,128}\right]$

Because n=4, then above-mentioned formula then is:

Wherein, I is the gradation of image value of corresponding wavelength, (i j) is corresponding pixel index among the step S1, each group (x y) represents an optical lightscreening locations of pixels, promptly the X-Y dimension linear displacement that will obtain.

For example, obtain wavelength 650nm the current pixel index (i j), through above-mentioned formula, adopts mathematical method, just can access (x, value y), thereby obtain the gradation of image value of wavelength 650nm.

Under non-ideal conditions, carry out step S2: obtain pixel index (i_Te, j_Te), estimation X-Y dimension linear displacement is used for revising optical filter and imageing sensor translation that assembling process causes, and estimated angle displacement and cycle ratio, said angle displacement are the displacement of the angle between multispectral optical filter and the imageing sensor,

The quantity ratio of the quantity that said cycle ratio is an imaging pixel and light field grid;

See also Fig. 4; One row's original image of the monochromatic wavelength light that multispectral imaging obtains; First grid optical lightscreening pixel cell upper left corner monochromatic wave long-channel of little multispectral optical filter does not match, and detects instance for early stage pressure sore, and the monochromatic light that is used to calibrate is wavelength 650nm green glow.Can select the displacement of angle between any row (like whole parallel) little multispectral optical filter of estimation and the imageing sensor.

The account form of angle displacement is among the said step S2:

Index (i₁, j₁) refer to first pixel of first grid, (i_f, j_f) be meant last image sensor pixel of light field.

The account form of cycle ratio is among the said step S2:

Index N is the number of light field.

Among the step S2, obtain the index map of each optical channel, specifically may further comprise the steps,

S201: the pixel index (i that from S1, obtains_Te, j_Te) as initial starting position, first 4 * 4 optical lightscreening pixel is set belongs to first wavelength;

S202: obtain pixel index (i_Te, j_Te), estimation X-Y dimension linear displacement is obtained angle displacement and the value of cycle ratio among the S2, estimates the position of next light field grid;

S203: around the estimated position of the light field grid that in the S202 step, obtains, seek the brightest pixel index, and these pixel indexs are labeled as the optical channel that belongs to first wavelength;

S204: repeating step S202-S203 is searched for until all original images entirely, as first wavelength

Optical channel all accomplish, and to write down all pixel indexs that belong to first wavelength be the index map of first wavelength;

S205: repeating step S201-S204 for all wavelengths is set up corresponding optical channel respectively, is kept at one 160 * 128 matrix, the corresponding spectrum channel of each matrix with the coordinate in all upper left corners of 3 * 3 pixel grids.

Through can the uses algorithm formula after the above-mentioned correction step calculating the multispectral gray-scale value of corresponding all wavelengths.

The present invention detects application example except pressure sore, and its image algorithm and imaging correction for drift method based on little multispectral optical filter imaging also has general general applicability.For different application; Number according to survey target fingerprint material property spectral wavelength; Little multispectral optical filter geometry can be different with the optical lightscreening number of pixels of reflection characteristic wavelength; Optical lightscreening pixel geometry size can be that several microns are to tens microns; The optical lightscreening number of pixels of reflection characteristic wavelength can be 2-N, and the optical lightscreening pixel groups of each reflection characteristic wavelength can comprise one group of a plurality of image sensor pixel, and ideal situation is the integral multiple that grid cell optical lightscreening pixel groups covers an imaging sensor pixel accurately.Imageing sensor can be the CMOS or the CCD of any Pixel Dimensions; Imageing sensor for the different pixels size; Little multispectral optical filter is designed the optical lightscreening pixel cell that is complementary with image sensor pixel; Algorithm of the present invention can be demarcated the optical lightscreening pixel and the image sensor pixel of little multispectral optical filter; And calculate gradation of image value I by algorithmic formula, the unmatched imaging deviation of pixel linear displacement, angle displacement and periodicity of little multispectral wave filter and the integrated appearance of imageing sensor height is estimated and proofreaied and correct.

The above is merely embodiments of the invention; Be not so limit claim of the present invention; Every equivalent structure or equivalent flow process conversion that utilizes instructions of the present invention and accompanying drawing content to be done; Or directly or indirectly be used in other relevant technical fields, all in like manner be included in the scope of patent protection of the present invention.

Claims (5)

1. image algorithm based on little multispectral optical filter; It is characterized in that: the image classification that a pair is posted the imageing sensor acquisition of multiple tracks optical filter is a plurality of monochrome images; Demarcate optical lightscreening pixel and image sensor pixel; And obtain the multispectral imaging pixel and the gray-scale value of said a plurality of monochrome images, specifically comprise:

Step S1: judge whether optical filter accurately covers an integer pixel of said imageing sensor, if then be judged to be ideal state, the computed image gray-scale value; If not, then be judged as, forward step S2 in non-ideal conditions;

Step S2: obtain pixel index (i_Te, j_Te), estimation X-Y dimension linear displacement is used for revising optical filter and imageing sensor translation that assembling process causes; The estimated angle displacement is used for revising optical filter and imageing sensor rotation error that assembling process causes; Estimation cycle ratio is used to correct the displacement that optical filter does not cover an integer image sensor pixel, is divided into the optical channel of corresponding a plurality of wavelength respectively to the optical lightscreening pixel, obtains the index map of each optical channel, carries out the calculating of gradation of image value again.

2. the image algorithm based on little multispectral optical filter according to claim 1 is characterized in that: the computing method of gradation of image value specifically comprise among the said step S1,

$I(x,y)=\frac{1}{n^2}\underset{i=0,j=0}{\overset{i=3,j=3}{\mathrm{\Σ}}}X(2\mathrm{nx}+i,2\mathrm{ny}+j),\mathrm{wherex}\∈\left[\mathrm{1,160}\right],y\∈[1,\underset{15}{128}]$

Wherein, said n is the number of optical filter, and (x y) represents an optics locations of pixels, index (i to each group_Te, j_Te) be the unit picture element of imaging sensor, I₁(x, y) be optical lightscreening pixel position for (x, the gray-scale value of y) locating, thereby according to the optics locations of pixels (x, y) and index (i_Te, j_Te) obtain the gradation of image value.

3. the image algorithm based on little multispectral optical filter according to claim 1 is characterized in that, the method for calculating angle displacement among the said step S2 is:

Angle displacement is the displacement of the optical lightscreening pixel and the angle between the image sensor pixel of multispectral optical filter, index (i₁, j₁) be meant first pixel index, (i_f, j_f) be meant the pixel of last imageing sensor of light field.

4. the image algorithm based on little multispectral optical filter according to claim 1 is characterized in that the evaluation method of cycle ratio is among the said step S2,

The quantity ratio of the quantity that said cycle ratio is an imaging pixel and light field grid; Index N is the number of light field.

5. the image algorithm based on little multispectral optical filter according to claim 1 is characterized in that, among the said step S2, obtains the index map that obtains each optical channel, specifically may further comprise the steps,

S201: the pixel index (i that from S1, obtains_Te, j_Te) as initial starting position, first 4 * 4 optical lightscreening pixel is set belongs to first wavelength;

S202: obtain pixel index (i_Te, j_Te), estimation X-Y dimension linear displacement is obtained angle displacement and the value of cycle ratio among the S2, estimates the position of next light field grid;

S203: around the estimated position of the light field grid that in the S202 step, obtains, seek the brightest pixel index, and these pixel indexs are labeled as the optical channel that belongs to first wavelength;

S204: repeating step S202-S203 is searched for until all original images entirely, all accomplish as the optical channel of first wavelength, and to write down all pixel indexs that belong to first wavelength is the index map of first wavelength;

S205: repeating step S201-S204 for all wavelengths is set up corresponding optical channel respectively, is kept at one 160 * 128 matrix, the corresponding spectrum channel of each matrix with the coordinate in all upper left corners of 3 * 3 pixel grids.

Images