CN101807303A - Tracking device based on multiple-target mean shift - Google Patents

Abstract

The invention provides a tracking device based on multiple-target mean shift, which comprises a tracker pipeline manager and at least one MSFG single-target tracker, wherein the tracker pipeline manager is used for application, destroy, data import and data export of the MSFG single-target tracker; and the tracker pipeline manager comprises a target demand analysis module, a data import module, a tracker execution module, a tracker reliability analysis module and a data export module. Each MSFG single-target tracker is used for tracking one object and comprises a modeling/updating module and an MS iteration module. The invention solves the problems of missing, mistaken tracking and the like in the block matching tracker due to illumination and other factors.

Description

A kind of tracking means based on multiple-target mean shift

Technical field

The present invention relates to a kind of tracking means, belong to Flame Image Process, field of video monitoring based on multiple-target mean shift.

Background technology

It is that prospect agglomerate by target and detecting device are provided mates realization that classical piece coupling is followed the tracks of, and its advantage is a precise and high efficiency, and shortcoming is need be based upon on the reliable basis of detecting device.If have middle of target branch gray values of pixel points in the scene with background is close or scene in when having that shadow changes faster, detecting device detects the prospect agglomerate regular meeting that obtains and takes place damaged or fracture, tracker then can occur following and lost with phenomenons such as mistakes this moment.This just requires tracker to follow the tracks of real goal exactly from the prospect agglomerate of damaged or fracture.

At present, the kind of tracking is a lot, and comparatively outstanding is Mean-Shift (average drifting) method.The Mean-Shift method is a common method of utilizing color distribution that moving target is detected and follows the tracks of, and it is to adopt density gradient to climb to find the nonparametric technique of probability distribution peak value, has stronger robustness and better practicability.Publication number is that the Chinese patent application of CN 101162525A discloses a kind of human body multi-joint characteristic tracking method based on Mean-Shift and artificial fish school intelligent optimizing; Publication number be US 6590999B1 U.S. Patent Application Publication a kind of non-rigid body object real-time tracking method based on Mean-Shift.But the method that above-mentioned patent provided can not directly solve from the prospect agglomerate of damaged or fracture the problem of tracking target exactly.

In sum, press for a kind of tracking means that can address the above problem of proposition at present.

Summary of the invention

In view of this, fundamental purpose of the present invention is to solve the problem of mating tracking target from the prospect agglomerate of damaged or fracture exactly, realizes that the correct coupling of target is followed the tracks of.

For achieving the above object, the invention provides a kind of tracking means based on multiple-target mean shift, described tracking means comprises:

At least one MSFG objective monomer tracker, be used to follow the tracks of a target and

The tracker pipeline manager is used for application, the destruction of described MSFG objective monomer tracker, and the importing of data, derivation;

Wherein: described tracker pipeline manager comprises:

The target requirement analysis module is used to judge whether target satisfies the process range of described tracking means, then carry out the data importing module if satisfy, otherwise described tracking means is abandoned the processing to target,

The data importing module is used to described MSFG objective monomer tracker to import the master data of target,

The tracker execution module is used for opening the described MSFG objective monomer tracker of execution,

Tracker fail-safe analysis module is used for the tracking results of described MSFG objective monomer tracker operation is carried out reliability assessment; If tracking target is reliably, then carry out data and derive module, if tracking results is insecure, then described tracking means to corresponding target abandon processing and

Data derive module, are used for the information of reliable tracking results correspondence is exported to external interface, read use to satisfy external module;

Described MSFG objective monomer tracker comprises:

Modelling/update module is used for the target of input is first set up model, to the target that has model carry out model modification judge with select whether this target to be upgraded and

The MS iteration module is used for iterative approach is carried out in the present frame position of target.

Preferably, judge in the described target requirement analysis module that the condition whether target satisfies the process range of described tracking means is:

(1) target is the stable trend target that has;

(2) piece mates the piece matching factor of foreground detection in the tracker less than first threshold T1;

(3) lifetime of target is greater than the second threshold value T2;

(4) exist described MSFG objective monomer tracker to be in idle condition;

Wherein, first threshold T1 ∈ [0.75,0.85], the second threshold value T2 ∈ [15,30] and be integer.

Preferably, the described stable trend target that has is defined as: after target generates in the piece coupling tracker, and Continuous Tracking multiframe and the target that moves a certain distance along a certain direction.

Preferably, the master data of described target comprises: the prospect agglomerate of the center of target, the framework of target, previous frame image, current frame image, previous frame, the prospect agglomerate of present frame, the foreground detection frame that Target id is corresponding with the target previous frame.

Preferably, described tracker fail-safe analysis module is judged Pasteur's coefficient B of the tracking target of described MSFG objective monomer tracker; If described Pasteur's coefficient B, thinks then that the tracking target of described MSFG objective monomer tracker is reliably greater than the 3rd threshold value T3, continue to carry out data and derive module; Otherwise think that the tracking target of described MSFG objective monomer tracker is insecure, described tracking means is done insecure tracking target and is abandoned handling, wherein, and the 3rd threshold value T3 ∈ [0.6,0.8].

Preferably, the information of tracking target correspondence comprises reliably: the area of the center of target, the framework of target, target and the histogram of target.

Preferably, described modelling/update module is by coming the target of input is first set up model as statistics with histogram to the foreground point in the previous frame target area, wherein, at first the gray-scale value to the foreground point deals with, and selects to calculate in advance good kernel function and is weighted; Suppose that HM is the object module histogram, be used for the standard of comparison as the MS iteration module, its computing formula is as follows:

$\mathrm{HM}\left(i\right)=\underset{x}{\mathrm{\Σ}}\underset{y}{\mathrm{\Σ}}\mathrm{KernelHist}\left(R^2\right)\·g(x,y)$

$\mathrm{KernelHist}\left(R^2\right)=\left\{\begin{array}{cc}1-R^2& 0\≤R^2\≤1\\ 0& \mathrm{other}\end{array}\right.$

$R^2={\left(\frac{{x-x}_0}{w}\right)}^2+{\left(\frac{{y-y}_0}{h}\right)}^2$

i＝GrayLevel(g(x，y))

Wherein, the value of the histogram i bar of HM (i) expression object module, KernelHist (R²) expression point (x, the y) coefficient value of correspondence in kernel function, R²(x is y) to the central point (x of target for the expression point₀, y₀) normalized square distance, w, h represent respectively target width and the height, g (x, y) expression point (x, y) gray-scale value, (g (x, y)) represents point (x GrayLevel, y) gray-scale value g (x, y) quantification of carrying out is to obtain this point (x, gray shade scale y), (x y) belongs to impact point to wherein said point.

Preferably, described modelling/update module is by the histogrammic total amount VM of object module of present frame_NewJudge with the ratio r of the histogrammic total amount VM of object module of previous frame whether the target that has model needs to carry out model modification, and the computing formula of this ratio r is as follows:

$r=\frac{{\mathrm{VM}}_{\mathrm{new}}}{\mathrm{VM}}$

${\mathrm{VM}}_{\mathrm{new}}=\underset{i}{\mathrm{\Σ}}{\mathrm{HM}}_{\mathrm{new}}\left(i\right)$

$\mathrm{VM}=\underset{i}{\mathrm{\Σ}}\mathrm{HM}\left(i\right)$

Wherein, HM_NewBe the object module histogram of present frame, VM_NewBe the histogrammic total amount of the object module of present frame, HM is the object module histogram of previous frame, and VM is the histogrammic total amount of the object module of previous frame; When this ratio r ∈ [the 4th threshold value T4, the 5th threshold value T5], think that this target allows execution model to upgrade, the object module histogram of previous frame is updated to the object module histogram of present frame, i.e. HM=HM_New, otherwise keep original object module histogram HM, and wherein, the 4th threshold value T4 ∈ [0.6,0.8], the 5th threshold value T5 ∈ [1.2,1.4].

Preferably, described MS iteration module adopts the Mean-Shift algorithm that iterative approach is carried out in the present frame position of target, this Mean-Shift algorithm calculates weight by pointwise in the target area, and the weight of whole points of target area averaged, thereby obtain the promotion vector of target, wherein, point in the target area (x, the computing formula of weight y) is as follows:

${\mathrm{\ω}}_{x,y}=\mathrm{KernelShift}\left(R^2\right)\·\sqrt{\frac{\mathrm{HM}\left(i\right)}{\mathrm{HC}\left(i\right)}}$

$\mathrm{KernelShift}\left(R^2\right)=\left\{\begin{array}{cc}1& 0\&le;{\mathrm{<figure-callout\; label="R"\; filenames="HSA00000031168600011.png"\; state="\{\{state\}\}">R</figure-callout>}}^2\&le;1\\ 0& \mathrm{other}\end{array}\right.$

Wherein, ω_{X, y}Be point (x, weight y), KernelShift (R²) expression point (x, the computing function of mean shift kernel function y), described computing function R²Calculate R²(x is y) to the central point (x of target for the expression point₀, y₀) normalized square distance, HC represents candidate's histogram, is used for being characterized in the described MS iteration module histogram distribution situation of candidate region, target place after the iteration each time, the value of HC (i) expression candidate histogram i bar.

J point (x in the hypothetical target zone_j, y_j) weights be ω_j, the promotion vector (V of target then_x, V_y) be calculated as follows:

$\left\{\begin{array}{c}{V}_x=\frac{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j\&CenterDot;x_j}{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j}-x_0\\ V_y=\frac{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j\&CenterDot;y_j}{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j}-y_0\end{array}\right.$

If the displacement V of target_xLess than 1 pixel and V_yLess than 1 pixel, or the Mean-Shift iterations of target then stops to carry out the Mean-Shift iteration greater than the 6th threshold value T6, wherein, and the 6th threshold value T6 ∈ [5,10] and be integer.

Preferably, described MS iteration module also comprises the position of target is adjusted; If B1＞B0, then target's center's point (x₀, y₀) the position become (x₀+ V_x, y₀+ V_y), otherwise, the point (x of target's center₀, y₀) the position become

Wherein, B0 represents the histogram of reposition target after the last iteration and Pasteur's coefficient of former object module histogram coupling, B1 represents the histogram of reposition target after the current iteration and Pasteur's coefficient of former object module histogram coupling, and the histogram of reposition target is as follows with the computing formula of Pasteur's coefficient B that former object module histogram mates:

$B=\frac{\underset{k}{\mathrm{\&Sigma;}}\sqrt{T_{\mathrm{new}}.\mathrm{hist}\left(k\right)*T_{\mathrm{mode}l}.\mathrm{hist}\left(k\right)}}{\sqrt{(\underset{k}{\mathrm{\&Sigma;}}{T}_{\mathrm{new}}.\mathrm{hist}\left(k\right))*(\underset{k}{\mathrm{\&Sigma;}}{T}_{\mathrm{mode}l}.\mathrm{hist}\left(k\right))}}\&Element;\left[\mathrm{0,1}\right]$

Wherein, T_New.hist be the histogram of reposition tracking target, T_Model.hist the histogram of representing former object module.

Preferably, described MSFG objective monomer tracker also comprises the FG correcting module, and the output result that described FG correcting module is used for the MS iteration module is a clue, and the output result of detecting device is data, the position and the size of the reliable target of correction prospect.

Preferably, described FG correcting module is a central point with the reposition of the target exported in the described MS iteration module at first, the original object zone is the limit, to around to expand the formed rectangle frame of certain proportion be search box, the search box center overlaps with former rectangle frame center, and the size calculation formula of described search box is as follows:

$\left\{\begin{array}{c}W=w*(1+2*\mathrm{\&alpha;})\\ H=h*(1+2*\mathrm{\&alpha;})\end{array}\right.$

Wherein, w, h represent the width and the height of the rectangle frame of target respectively, and W, H represent the width and the height of search box ,-0.25≤α≤0.5 respectively;

Calculate theratio R 1 of the overlapping area and the foreground detection region area in search box region of interest within and foreground detection zone then, and theratio R 2 of the overlapping area in search box region of interest within and foreground detection zone and target area area, if R1 greater than the 7th threshold value T7 and R2 greater than the 8th threshold value T8, think that then the prospect of this target correspondence is reliable, and the reposition of this target is carried out FG revise, otherwise jump out this module, the reposition to this target does not carry out the FG correction, wherein, the 7th threshold value T7 ∈ [0.55,0.65], the 8th threshold value T8 ∈ [0.45,0.55];

Present frame foreground detection in the search box is added up, calculate its zeroth order square, first moment and second moment, computing formula is as follows:

$\left\{\begin{array}{c}{m}_{00}=\mathrm{\&Sigma;\&Sigma;FG}(x,y)\\ m_{10}=\mathrm{\&Sigma;\&Sigma;x}\&CenterDot;\mathrm{FG}(x,y)\\ m_{01}=\mathrm{\&Sigma;\&Sigma;y}\&CenterDot;\mathrm{FG}(x,y)\\ m_{20}=\mathrm{\&Sigma;\&Sigma;}{x}^2\&CenterDot;\mathrm{FG}(x,y)\\ m_{02}=\mathrm{\&Sigma;\&Sigma;}{\mathrm{<figure-callout\; label="y"\; filenames="HSA00000031168600011.png"\; state="\{\{state\}\}">y</figure-callout>}}^2\&CenterDot;\mathrm{FG}(x,y)\end{array}\right.$

Wherein, FG (x, y) bianry image mid point (x, gray-scale value y), the m of expression foreground detection_IjThe square of the order that expression is corresponding by above-mentioned square, calculates the revised target reposition of FG, comprising: the central point of target (x '₀, y '₀), the width w ' of target area and height h ', its computing formula is as follows:

$\left\{\begin{array}{c}{x}_0^{\&prime;}=\frac{m_{10}}{m_{00}}\\ y_0^{\&prime;}=\frac{m_{01}}{m_{00}}\\ w^{\&prime;}=4\&CenterDot;\sqrt{\frac{m_{20}}{m_{00}}-x_0^{\&prime;2}}\\ h^{\&prime;}=4\&CenterDot;\sqrt{\frac{m_{02}}{m_{00}}-y_0^{\&prime;2}}\end{array}\right.$

Tracking means based on multiple-target mean shift provided by the present invention piece coupling tracker suitable and that use always is used, and has solved because factors such as illumination make piece mate the problem of losing with phenomenons such as mistakes of following of tracker appearance.

Description of drawings

Fig. 1 shows the overall framework figure according to the tracking means based on multiple-target mean shift of the present invention;

Fig. 2 shows the frame diagram according to the tracker pipeline manager of the tracking means based on multiple-target mean shift of the present invention;

Fig. 3 shows the frame diagram according to the MSFG objective monomer tracker of the tracking means based on multiple-target mean shift of the present invention;

Fig. 4 shows the frame diagram of a kind of preferred L SFG objective monomer tracker of the present invention.

Embodiment

For making your auditor can further understand structure of the present invention, feature and other purposes, now be described in detail as follows in conjunction with appended preferred embodiment, illustrated preferred embodiment only is used to technical scheme of the present invention is described, and non-limiting the present invention.

Tracking means based on multiple-target mean shift provided by the present invention is based on detecting device, piece coupling tracker (piece coupling tracker just is meant the general tracker that the contains object matching) basis, need carry out the processing of detecting device earlier during its concrete application, carry out the processing of piece coupling tracker again, and then carry out the processing of tracking means of the present invention.The present invention is the further optimization process to piece coupling tracker, be for solve in the piece coupling tracker object appearing with wrong with losing phenomenon.Because detecting device and piece coupling tracker are not inventive points, so the present invention no longer is specifically addressed it.Detecting device, piece coupling tracker can be realized by existing technology, for example can be 200910077433.6 and 200910077435.5 patented claim referring to application number.

As shown in Figure 1, Fig. 1 shows the overall framework figure according to the tracking means based on multiple-target mean shift of the present invention.As seen from Figure 1, tracking means based on multiple-target mean shift (Multi-objects Mean-Shift abbreviates MMS as) of the present invention comprisestracker pipeline manager 1 and at least one MSFG (MSFG is the abbreviation of " Mean-Shift withForeGround ") objective monomer tracker 2.Wherein,tracker pipeline manager 1 is used for application, the destruction to MSFGobjective monomer tracker 2, and the importing of data, derivation; Each described MSFG objective monomer tracker is used to follow the tracks of a target,tracker pipeline manager 1 is each other with each MSFGobjective monomer tracker 2 and communicates to connect, that is, be connected by general communication betweentracker pipeline manager 1 and each MSFGobjective monomer tracker 2 and transmit data and instruction.

As shown in Figure 2, Fig. 2 shows the frame diagram according to the tracker pipeline manager of the tracking means based on multiple-target mean shift of the present invention.As seen from Figure 2, comprise with lower module according totracker pipeline manager 1 of the present invention:

Targetrequirement analysis module 11 is used to judge whether target satisfies the process range of this tracking means, then carry outdata importing module 12 if satisfy, otherwise this tracking means is abandoned the processing to target;

Data importing module 12 is used to MSFGobjective monomer tracker 2 to import the master data of target;

Tracker execution module 13 is used for opening execution MSFGobjective monomer tracker 2;

Tracker fail-safe analysis module 14, be used for the tracking results of MSFGobjective monomer tracker 2 operations is carried out reliability assessment,, then carry out data and derivemodule 15 if tracking results is reliable, if tracking results is unreliable, then this tracking means is abandoned handling to corresponding target; With

Data derivemodule 15, are used for the information of reliable tracking results correspondence is exported to external interface, read use to satisfy external module.

Wherein, it is as follows to judge in the target requirement analysis module 11 whether target satisfies the condition of process range of this tracking means: (1) target is the stable trend target that has; (2) the piece matching factor of foreground detection (contains this piece matching factor among the result of piece coupling tracker output in the piece coupling tracker, for example application number is surveyed area in 200910077435.5 the patented claim and the matching factor of target) less than first threshold T1, first threshold T1 ∈ [0.75,0.85]; (3) (being the frame number of the target that Continuous Tracking arrives in the piece coupling tracker) is greater than the second threshold value T2 lifetime of target, the second threshold value T2 ∈ [15,30] and be integer; (4) exist MSFG objective monomer tracker 2 to be in idle condition.Wherein, the trend target that has stable in the condition (1) is defined as: after target generates in the piece coupling tracker, (this multiframe can be the 6-12 frame to the Continuous Tracking multiframe, for example can elect 8 frames as) and move a certain distance the target of (this certain distance can be a 8-12 pixel, for example can elect 10 pixels as) along a certain direction.

Judge through targetrequirement analysis module 11,, then carry outdata importing module 12 if target satisfies the process range of this tracking means.Thisdata importing module 12 is the master data that MSFGobjective monomer tracker 2 imports targets, and this master data comprises: the prospect agglomerate of the center of target, the framework of target, previous frame image, current frame image, previous frame, the prospect agglomerate of present frame, the foreground detection frame that Target id is corresponding with the target previous frame.

Aftertracker execution module 13 is opened execution MSFGobjective monomer tracker 2, Pasteur's coefficient B of the tracking target of 14 pairs of MSFG objective monomers of tracker fail-safeanalysis module tracker 2 is judged, if this Pasteur's coefficient B is greater than the 3rd threshold value T3, think that then the tracking target of MSFGobjective monomer tracker 2 is reliably, continue to carry out data and derive module 7; Otherwise think that the tracking target of MSFGobjective monomer tracker 2 is insecure, this tracking means is abandoned handling to corresponding target (being insecure tracking target).Wherein, the 3rd threshold value T3 ∈ [0.6,0.8].

The computing formula of Pasteur's coefficient B is as follows:

$B=\frac{\underset{k}{\mathrm{\&Sigma;}}\sqrt{T_{\mathrm{new}}.\mathrm{hist}\left(k\right)*T_{\mathrm{mode}l}.\mathrm{hist}\left(k\right)}}{\sqrt{(\underset{k}{\mathrm{\&Sigma;}}{T}_{\mathrm{new}}.\mathrm{hist}\left(k\right))*(\underset{k}{\mathrm{\&Sigma;}}{T}_{\mathrm{mode}l}.\mathrm{hist}\left(k\right))}}\&Element;\left[\mathrm{0,1}\right]$

Wherein, T_New.hist be the histogram of reposition tracking target, T_Model.hist the histogram of representing former object module.Histogrammic calculating belongs to prior art, normally statistic histogram.A certain physical quantity (zone or target) is done n duplicate measurements under the same conditions, obtain series of measured values, find out its maximal value and minimum value, determine an interval then, make it comprise whole measurement data, this interval is divided into some minizones, and statistical measurements appears at the frequency M of each minizone, is horizontal ordinate with the measurement data, with frequency M is ordinate, mark each minizone and corresponding frequency height thereof, then can obtain histogram, i.e. a statistic histogram.

For being judged as reliable tracking target in the tracker fail-safe analysis module 14, data derive the information thatmodule 15 derives this reliable tracking target correspondence, and this information comprises: the area of the center of target, the framework of target, target and the histogram of target.

As shown in Figure 3, Fig. 3 shows the frame diagram according to the MSFG objective monomer tracker of the tracking means based on multiple-target mean shift of the present invention.As seen from Figure 3, comprise with lower module according to MSFGobjective monomer tracker 2 of the present invention: modelling/update module 21, be used for the target of input is first set up model, the target that has model carried out model modification judge with selection whether this target is upgraded; WithMS iteration module 22, be used for iterative approach is carried out in the present frame position of target.

Wherein, modelling/update module 21 is set up model for the target of input first, and this sets up model is by the foreground point in the previous frame target area is carried out as statistics with histogram.When making statistics with histogram, need the gray-scale value of foreground point be dealt with, the kernel function that can select to calculate in advance is weighted.Suppose that HM is object module (Histogram ofModel), be used for the standard of comparison asMS iteration module 22, its computing formula is as follows:

$\mathrm{HM}\left(i\right)=\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}\mathrm{KernelHist}\left(R^2\right)\&CenterDot;g(x,y)$

$\mathrm{KernelHist}\left(R^2\right)=\left\{\begin{array}{cc}1-{\mathrm{<figure-callout\; label="R"\; filenames="HSA00000031168600011.png"\; state="\{\{state\}\}">R</figure-callout>}}^2& 0\&le;{\mathrm{<figure-callout\; label="R"\; filenames="HSA00000031168600011.png"\; state="\{\{state\}\}">R</figure-callout>}}^2\&le;1\\ 0& \mathrm{other}\end{array}\right.$

${\mathrm{<figure-callout\; label="R"\; filenames="HSA00000031168600011.png"\; state="\{\{state\}\}">R</figure-callout>}}^2={\left(\frac{{x-x}_0}{w}\right)}^2+{\left(\frac{{y-y}_0}{h}\right)}^2$

i＝GrayLevel(g(x，y))

Wherein, the value of the histogram i bar of HM (i) expression object module.KernelHist (R²) (x, y) coefficient value of correspondence in kernel function is used R here to the expression point²Come calculating K ernelHist (x, y); R²(x is y) to the central point (x of target for the expression point₀, y₀) normalized square distance (coordinate of the central point of target be detecting device output); W, h represent the width of target and height (width of target and highly also exported by detecting device) respectively; (x, y) (((x, ((which gray shade scale x y) belongs to gray-scale value g y) to GrayLevel to the expression point to g, can select 32 grades, i.e. i=0,1, K, 31 to obtain this point for x, the y) quantification of carrying out to point for g (x, y)) expression for x, gray-scale value y).Wherein, (x y) belongs to impact point to described point.

At first whether modelling/update module 21 needs to carry out the judgement of model modification for the target that has model, and its basis for estimation is the histogrammic total amount VM of the object module of present frame_NewThe ratio r of the histogrammic total amount VM of object module of (Volume of Model) and previous frame, this ratio r computing formula is as follows:

$r=\frac{{\mathrm{VM}}_{\mathrm{new}}}{\mathrm{VM}}$

${\mathrm{VM}}_{\mathrm{new}}=\underset{i}{\mathrm{\&Sigma;}}{\mathrm{HM}}_{\mathrm{new}}\left(i\right)$

$\mathrm{VM}=\underset{i}{\mathrm{\&Sigma;}}\mathrm{HM}\left(i\right)$

Wherein, HM_NewBe the object module histogram of present frame, VM_NewBe the histogrammic total amount of the object module of present frame, HM is the object module histogram of previous frame, and VM is the histogrammic total amount of the object module of previous frame, and the histogrammic computing method of object module as mentioned above.When this ratio r ∈ [the 4th threshold value, the 5th threshold value], think that this target allows execution model to upgrade, the object module histogram of previous frame is updated to the object module histogram of present frame, i.e. HM=HM_New, otherwise keep original object module histogram HM.Wherein, the 4th threshold value ∈ [0.6,0.8], the 5th threshold value ∈ [1.2,1.4].

MS iteration module 22 adopts the Mean-Shift algorithm that iterative approach is carried out in the present frame position of target.This Mean-Shift algorithm can be that classical Mean-Shift algorithm (sees also document " Dorin Comaniciu; Visvanathan Ramesh, Peter Meer:Real-TimeTracking of Non-Rigid Objects Using Mean-Shift.CVPR 2000 ".) this Mean-Shift algorithm calculates weight by pointwise in target area (being the circumscribed rectangular region of target), and the weight of whole points of target area is averaged, thereby obtain the promotion vector of target.Wherein, point in the target area (x, the computing formula of weight y) is as follows:

${\mathrm{\&omega;}}_{x,y}=\mathrm{KernelShift}\left(R^2\right)\&CenterDot;\sqrt{\frac{\mathrm{HM}\left(i\right)}{\mathrm{HC}\left(i\right)}}$

$\mathrm{KernelShift}\left(R^2\right)=\left\{\begin{array}{cc}1& 0\&le;{\mathrm{<figure-callout\; label="R"\; filenames="HSA00000031168600011.png"\; state="\{\{state\}\}">R</figure-callout>}}^2\&le;1\\ 0& \mathrm{other}\end{array}\right.$

Wherein, ω_{X, y}Be point (x, weight y), KernelShift (R²) expression point (x, the computing function of mean shift kernel function y), this computing function R²Calculate R²(x is y) to the central point (x of target for the expression point₀, y₀) normalized square distance.HC represents candidate's histogram, is used for being characterized in theMS iteration module 22 histogram distribution situation of candidate region, target place after the iteration each time, the value of HC (i) expression candidate histogram i bar.Wherein, the histogram calculation method of HC is identical with the histogram calculation method of HM.I, HM (i), GrayLevel (g (x, y)) and R²Implication or computing method in modelling/update module 21, describe, no longer be repeated in this description at this.

J point (x in the hypothetical target zone_j, y_j) weights be ω_j, the promotion vector (V of target then_x, V_y) be calculated as follows:

$\left\{\begin{array}{c}{V}_x=\frac{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j\&CenterDot;x_j}{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j}-x_0\\ V_y=\frac{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j\&CenterDot;y_j}{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j}-y_0\end{array}\right.$

If the displacement V of target_x＜1 pixel and V_y＜1 pixel, perhaps the Mean-Shift iterations of target then stops to carry out the Mean-Shift iteration greater than the 6th threshold value T6.Wherein, the 6th threshold value T6 ∈ [5,10] and be integer.

Because the calculating that is not Mean-Shift vector (being the promotion vector of above-mentioned target) each time all is correct, but principle according to the Mean-Shift climbing, Pasteur's coefficient of the histogram of reposition target and former object module histogram coupling should increase after the Mean-Shift iteration each time, have only the last time in the iteration, " crossing shuttle " phenomenon might occur, i.e. climbing process has surpassed the mountain top and has advanced along the direction of going down the hill again.ThereforeMS iteration module 22 also comprises the position of target is adjusted: if B1＞B0, then target's center's point (x₀, y₀) the position become (x₀+ V_x, y₀+ V_y); Otherwise, the point (x of target's center₀, y₀) the position become

Wherein, B0 represents the histogram of reposition target after the last iteration and Pasteur's coefficient of former object module histogram coupling, B1 represents the histogram of reposition target after the current iteration and Pasteur's coefficient of former object module histogram coupling, and the histogram of reposition target is as follows with the computing formula of Pasteur's coefficient B that former object module histogram mates:

$B=\frac{\underset{k}{\mathrm{\&Sigma;}}\sqrt{T_{\mathrm{new}}.\mathrm{hist}\left(k\right)*T_{\mathrm{mode}l}.\mathrm{hist}\left(k\right)}}{\sqrt{(\underset{k}{\mathrm{\&Sigma;}}{T}_{\mathrm{new}}.\mathrm{hist}\left(k\right))*(\underset{k}{\mathrm{\&Sigma;}}{T}_{\mathrm{mode}l}.\mathrm{hist}\left(k\right))}}\&Element;\left[\mathrm{0,1}\right]$

Wherein, T_New.hist be the histogram of reposition target, T_Model.hist the histogram of representing former object module.

Preferred embodiment a kind of as the present invention, MSFGobjective monomer tracker 2 also comprisesFG correcting module 23, the output result that FG correctingmodule 23 is used forMS iteration module 22 is a clue, and the output result of detecting device is data, the position and the size of the reliable target of correction prospect.

FG correcting module 23 is used for the position and the size of the reliable target of correction prospect, promptly just revises the reposition of this target of output in theMS iteration module 22 for the reliable target of prospect.FG correcting module 23 is a central point with the reposition of the target of output in theMS iteration module 22 at first, the original object zone is the limit, to around to expand the formed rectangle frame of certain proportion be search box, the search box center overlaps with former rectangle frame center, the size calculation formula of search box is as follows:

$\left\{\begin{array}{c}W=w*(1+2*\mathrm{\&alpha;})\\ H=h*(1+2*\mathrm{\&alpha;})\end{array}\right.$

Wherein, w, h represent the width and the height of the rectangle frame of target respectively, and W, H represent the width and the height of search box respectively.The span of α is [0.25,0.5], promptly-and 0.25≤α≤0.5.

Then calculate the ratio R 1 of the overlapping area and the foreground detection region area in search box region of interest within and foreground detection zone, calculate search box region of interest within and the overlapping area in foreground detection zone and the ratio R 2 of target area area.Wherein, calculate R1 by calculating the search box region of interest within and the overlapping area in foreground detection zone and the ratio of foreground detection region area, promptly

Overlap_area is the overlapping area in search box region of interest within and foreground detection zone, and fg_area is the foreground detection region area; Calculate R2 by calculating the search box region of interest within and the overlapping area in foreground detection zone and the ratio of target area area, promptly

Overlap_area is the overlapping area in search box region of interest within and foreground detection zone, and tgt_area is the target area area.If R1 greater than the 8th threshold value T8, thinks then that the prospect of this target correspondence is reliable greater than the 7th threshold value T7 and R2, and the reposition of this target is carried out FG revise, otherwise jump out this module, the reposition of this target is not carried out FG and revise.Wherein, the 7th threshold value T7 ∈ [0.55,0.65], the 8th threshold value T8 ∈ [0.45,0.55].Then the present frame foreground detection in the search box is added up, calculate its zeroth order square, first moment and second moment, computing formula is as follows:

$\left\{\begin{array}{c}{m}_{00}=\mathrm{\&Sigma;\&Sigma;FG}(x,y)\\ m_{10}=\mathrm{\&Sigma;\&Sigma;x}\&CenterDot;\mathrm{FG}(x,y)\\ m_{01}=\mathrm{\&Sigma;\&Sigma;y}\&CenterDot;\mathrm{FG}(x,y)\\ m_{20}=\mathrm{\&Sigma;\&Sigma;}{x}^2\&CenterDot;\mathrm{FG}(x,y)\\ m_{02}=\mathrm{\&Sigma;\&Sigma;}{\mathrm{<figure-callout\; label="y"\; filenames="HSA00000031168600011.png"\; state="\{\{state\}\}">y</figure-callout>}}^2\&CenterDot;\mathrm{FG}(x,y)\end{array}\right.$

Wherein, FG (x, y) bianry image mid point (x, gray-scale value y), the m of expression foreground detection_IjThe square of the order that expression is corresponding.By above-mentioned square, calculate the revised target reposition of FG, comprising: the central point of target (x '₀, y '₀), the width w ' of target area and height h ', it is calculated as follows:

$\left\{\begin{array}{c}{x}_0^{\&prime;}=\frac{m_{10}}{m_{00}}\\ y_0^{\&prime;}=\frac{m_{01}}{m_{00}}\\ w^{\&prime;}=4\&CenterDot;\sqrt{\frac{m_{20}}{m_{00}}-x_0^{\&prime;2}}\\ h^{\&prime;}=4\&CenterDot;\sqrt{\frac{m_{02}}{m_{00}}-y_0^{\&prime;2}}\end{array}\right.$

Be scene in one embodiment of the invention with outdoor, for example on the highway, on the path.Wherein, first threshold T1=0.8, the second threshold value T2=20, the 3rd threshold value T3=0.7, the 4th threshold value T4=0.7, the 5th threshold value T5=1.3, the 6th threshold value T6=8, the 7th threshold value T7=0.6, the 8th threshold value T8=0.5, α are 0.2.

Compared with prior art, solved according to the tracking means based on multiple-target mean shift of the present invention because piece coupling tracker appears with the problem of losing with phenomenons such as mistakes in factors such as illumination.

What need statement is that foregoing invention content and embodiment are intended to prove the practical application of technical scheme provided by the present invention, should not be construed as the qualification to protection domain of the present invention.Those skilled in the art are in spirit of the present invention and principle, when doing various modifications, being equal to and replacing or improve.Protection scope of the present invention is as the criterion with appended claims.

Claims (11)

1. the tracking means based on multiple-target mean shift is characterized in that, described tracking means comprises:

At least one MSFG objective monomer tracker, be used to follow the tracks of a target and

The tracker pipeline manager is used for application, the destruction of described MSFG objective monomer tracker, and the importing of data, derivation;

Wherein: described tracker pipeline manager comprises:

The target requirement analysis module is used to judge whether target satisfies the process range of described tracking means, then carry out the data importing module if satisfy, otherwise described tracking means is abandoned the processing to target,

The data importing module is used to described MSFG objective monomer tracker to import the master data of target,

The tracker execution module is used for opening the described MSFG objective monomer tracker of execution,

Tracker fail-safe analysis module is used for the tracking results of described MSFG objective monomer tracker operation is carried out reliability assessment; If tracking target is reliably, then carry out data and derive module, if tracking results is insecure, then described tracking means to corresponding target abandon processing and

Data derive module, are used for the information of reliable tracking results correspondence is exported to external interface, read use to satisfy external module;

Described MSFG objective monomer tracker comprises:

Modelling/update module is used for the target of input is first set up model, to the target that has model carry out model modification judge with select whether this target to be upgraded and

The MS iteration module is used for iterative approach is carried out in the present frame position of target.

2. tracking means according to claim 1 is characterized in that, judges in the described target requirement analysis module that the condition whether target satisfies the process range of described tracking means is:

(1) target is the stable trend target that has;

(2) piece mates the piece matching factor of foreground detection in the tracker less than first threshold T1;

(3) lifetime of target is greater than the second threshold value T2;

(4) exist described MSFG objective monomer tracker to be in idle condition;

Wherein, first threshold T1 ∈ [0.75,0.85], the second threshold value T2 ∈ [15,30] and be integer.

3. tracking means according to claim 1, it is characterized in that the master data of described target comprises: the prospect agglomerate of the center of target, the framework of target, previous frame image, current frame image, previous frame, the prospect agglomerate of present frame, the foreground detection frame that Target id is corresponding with the target previous frame.

4. tracking means according to claim 1 is characterized in that, described tracker fail-safe analysis module is judged Pasteur's coefficient B of the tracking target of described MSFG objective monomer tracker; If described Pasteur's coefficient B, thinks then that the tracking target of described MSFG objective monomer tracker is reliably greater than the 3rd threshold value T3, continue to carry out data and derive module; Otherwise think that the tracking target of described MSFG objective monomer tracker is insecure, described tracking means is done insecure tracking target and is abandoned handling, wherein, and the 3rd threshold value T3 ∈ [0.6,0.8].

5. tracking means according to claim 1 is characterized in that, the information of tracking target correspondence comprises reliably: the area of the center of target, the framework of target, target and the histogram of target.

6. tracking means according to claim 1, it is characterized in that, described modelling/update module is by coming the target of input is first set up model as statistics with histogram to the foreground point in the previous frame target area, wherein, at first the gray-scale value to the foreground point deals with, and selects to calculate in advance good kernel function and is weighted; Suppose that HM is the object module histogram, be used for the standard of comparison as the MS iteration module, its computing formula is as follows:

$\mathrm{HM}\left(i\right)=\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}\mathrm{KernelHist}\left(R^2\right)\&CenterDot;g(x,y)$

$\mathrm{KernelHist}\left(R^2\right)=\left\{\begin{array}{cc}1-R^2& 0\&le;R^2\&le;1\\ 0& \mathrm{other}\end{array}\right.$

$R^2={\left(\frac{x-x_0}{w}\right)}^2+{\left(\frac{y-y_0}{h}\right)}^2$

i＝GrayLevel(g(x，y))

Wherein, the value of the histogram i bar of HM (i) expression object module, KernelHist (R²) expression point (x, the y) coefficient value of correspondence in kernel function, R²(x is y) to the central point (x of target for the expression point₀, y₀) normalized square distance, w, h represent respectively target width and the height, g (x, y) expression point (x, y) gray-scale value, (g (x, y)) represents point (x GrayLevel, y) gray-scale value g (x, y) quantification of carrying out is to obtain this point (x, gray shade scale y), (x y) belongs to impact point to wherein said point.

7. tracking means according to claim 1 is characterized in that, described modelling/update module is by the histogrammic total amount VM of object module of present frame_NewJudge with the ratio r of the histogrammic total amount VM of object module of previous frame whether the target that has model needs to carry out model modification, and the computing formula of this ratio r is as follows:

$r=\frac{{\mathrm{VM}}_{\mathrm{new}}}{\mathrm{VM}}$

${\mathrm{VM}}_{\mathrm{new}}=\underset{i}{\mathrm{\&Sigma;}}{\mathrm{HM}}_{\mathrm{new}}\left(i\right)$

$\mathrm{VM}=\underset{i}{\mathrm{\&Sigma;}}\mathrm{HM}\left(i\right)$

Wherein, HM_NewBe the object module histogram of present frame, VM_NewBe the histogrammic total amount of the object module of present frame, HM is the object module histogram of previous frame, and VM is the histogrammic total amount of the object module of previous frame; When this ratio r ∈ [the 4th threshold value T4, the 5th threshold value T5], think that this target allows execution model to upgrade, the object module histogram of previous frame is updated to the object module histogram of present frame, i.e. HM=HM_New, otherwise keep original object module histogram HM, and wherein, the 4th threshold value T4 ∈ [0.6,0.8], the 5th threshold value T5 ∈ [1.2,1.4].

8. tracking means according to claim 1, it is characterized in that, described MS iteration module adopts the Mean-Shift algorithm that iterative approach is carried out in the present frame position of target, this Mean-Shift algorithm calculates weight by pointwise in the target area, and the weight of whole points of target area averaged, thereby obtain the promotion vector of target, wherein, point in the target area (x, the computing formula of weight y) is as follows:

${\mathrm{\&omega;}}_{x,y}=\mathrm{KernelShift}\left(R^2\right)\&CenterDot;\sqrt{\frac{\mathrm{HM}\left(i\right)}{\mathrm{HC}\left(i\right)}}$

$\mathrm{KernelShift}\left(R^2\right)=\left\{\begin{array}{cc}1& 0\&le;R^2\&le;1\\ 0& \mathrm{other}\end{array}\right.$

Wherein, ω_{X, y}Be point (x, weight y), KernelShift (R²) expression point (x, the computing function of mean shift kernel function y), described computing function R²Calculate R²(x is y) to the central point (x of target for the expression point₀, y₀) normalized square distance, HC represents candidate's histogram, is used for being characterized in the described MS iteration module histogram distribution situation of candidate region, target place after the iteration each time, the value of HC (i) expression candidate histogram i bar.

J point (x in the hypothetical target zone_j, y_j) weights be ω_j, the promotion vector (V of target then_x, V_y) be calculated as follows:

$\left\{\begin{array}{c}{V}_x=\frac{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j\&CenterDot;x_j}{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j}-x_0\\ V_y=\frac{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j\&CenterDot;y_j}{\underset{j}{\mathrm{\&Sigma;}}{\mathrm{\&omega;}}_j}-y_0\end{array}\right.$

If the displacement V of target_xLess than 1 pixel and V_yLess than 1 pixel, or the Mean-Shift iterations of target then stops to carry out the Mean-Shift iteration greater than the 6th threshold value T6, wherein, and the 6th threshold value T6 ∈ [5,10] and be integer.

9. tracking means according to claim 8 is characterized in that, described MS iteration module also comprises to be adjusted the position of target; If B1＞B0, then target's center's point (x₀, y₀) the position become (x₀+ V_x, y₀+ V_y), otherwise, the point (x of target's center₀, y₀) the position become

Wherein, B0 represents the histogram of reposition target after the last iteration and Pasteur's coefficient of former object module histogram coupling, B1 represents the histogram of reposition target after the current iteration and Pasteur's coefficient of former object module histogram coupling, and the histogram of reposition target is as follows with the computing formula of Pasteur's coefficient B that former object module histogram mates:

$B=\frac{\underset{k}{\mathrm{\&Sigma;}}\sqrt{T_{\mathrm{new}}.\mathrm{hist}\left(k\right)*T_{\mathrm{mode}l}.\mathrm{hist}\left(k\right)}}{\sqrt{(\underset{k}{\mathrm{\&Sigma;}}{T}_{\mathrm{new}}.\mathrm{hist}\left(k\right))*(\underset{k}{\mathrm{\&Sigma;}}{T}_{\mathrm{mode}l}.\mathrm{hisst}\left(k\right))}},\&Element;\left[\mathrm{0,1}\right]$

Wherein, T_New.hist be the histogram of reposition tracking target, T_Model.hist the histogram of representing former object module.

10. tracking means according to claim 1, it is characterized in that, described MSFG objective monomer tracker also comprises the FG correcting module, the output result that described FG correcting module is used for the MS iteration module is a clue, the output result of detecting device is data, the position and the size of the reliable target of correction prospect.

11. tracking means according to claim 10, it is characterized in that, described FG correcting module is a central point with the reposition of the target exported in the described MS iteration module at first, the original object zone is the limit, to around to expand the formed rectangle frame of certain proportion be search box, the search box center overlaps with former rectangle frame center, and the size calculation formula of described search box is as follows:

$\left\{\begin{array}{c}W=w*(1+2*\mathrm{\&alpha;})\\ H=h*(1+2*\mathrm{\&alpha;})\end{array}\right.$

Wherein, w, h represent the width and the height of the rectangle frame of target respectively, and W, H represent the width and the height of search box ,-0.25≤α≤0.5 respectively;

Calculate the ratio R 1 of the overlapping area and the foreground detection region area in search box region of interest within and foreground detection zone then, and the ratio R 2 of the overlapping area in search box region of interest within and foreground detection zone and target area area, if R1 greater than the 7th threshold value T7 and R2 greater than the 8th threshold value T8, think that then the prospect of this target correspondence is reliable, and the reposition of this target is carried out FG revise, otherwise jump out this module, the reposition to this target does not carry out the FG correction, wherein, the 7th threshold value T7 ∈ [0.55,0.65], the 8th threshold value T8 ∈ [0.45,0.55];

Present frame foreground detection in the search box is added up, calculate its zeroth order square, first moment and second moment, computing formula is as follows:

$\left\{\begin{array}{c}{m}_{00}=\mathrm{\&Sigma;\&Sigma;FG}(x,y)\\ m_{10}=\mathrm{\&Sigma;\&Sigma;x}\&CenterDot;\mathrm{FG}(x,y)\\ m_{01}=\mathrm{\&Sigma;\&Sigma;y}\&CenterDot;\mathrm{FG}(x,y)\\ m_{20}=\mathrm{\&Sigma;\&Sigma;}{x}^2\&CenterDot;\mathrm{FG}(x,y)\\ m_{02}=\mathrm{\&Sigma;\&Sigma;}{y}^2\&CenterDot;\mathrm{FG}(x,y)\end{array}\right.$

Wherein, FG (x, y) bianry image mid point (x, gray-scale value y), the m of expression foreground detection_IjThe square of the order that expression is corresponding by above-mentioned square, calculates the revised target reposition of FG, comprising: the central point of target (x '₀, y '₀), the width w ' of target area and height h ', its computing formula is as follows:

$\left\{\begin{array}{c}{x}_0^{\&prime;}=\frac{m_{10}}{m_{00}}\\ y_0^{\&prime;}=\frac{m_{01}}{m_{00}}\\ w^{\&prime;}=4\&CenterDot;\sqrt{\frac{m_{20}}{m_{00}}-x_0^{\&prime;2}}\\ h^{\&prime;}=4\&CenterDot;\sqrt{\frac{m_{02}}{m_{00}}-y_0^{\&prime;2}}\end{array}\right.$

Images