CN102034248B - Motion segmentation and three-dimensional (3D) expression method for single view image sequence - Google Patents

Abstract

Translated fromChinese

本发明公开了一种单目图像序列的运动分割和3D表达方法，本发明首先采集单目图像序列，利用梯度下降法估算主运动目标的运动信息和深度信息，利用水平集的方法演化分割曲线；验证估算得到的深度信息，并且修正不可靠的深度信息；利用修正后的深度信息和得到的分割曲线，再次进行能量函数的最小化，估算主运动目标的运动信息和深度信息。最后，固定分割曲线，依次对运功目标进行能量函数的最小化、深度信息可靠性的验证和修正、再次能量函数的最小化，得到每一个运动目标的深度信息和运动信息。本发明可以将运动分割和3D表达同时进行，本发明不需要提前知道图像序列中的目标数目，具有广泛的适用性。The invention discloses a motion segmentation and 3D expression method of a monocular image sequence. The invention first collects a monocular image sequence, uses the gradient descent method to estimate the motion information and depth information of the main moving target, and uses the level set method to evolve the segmentation curve ; Verify the estimated depth information, and correct unreliable depth information; use the corrected depth information and the obtained segmentation curve to minimize the energy function again, and estimate the motion information and depth information of the main moving target. Finally, the segmentation curve is fixed, the energy function is minimized, the reliability of depth information is verified and corrected, and the energy function is minimized again to obtain the depth information and motion information of each moving target. The invention can perform motion segmentation and 3D expression at the same time. The invention does not need to know the number of objects in the image sequence in advance, and has wide applicability.

Description

Translated fromChinese

单目图像序列的运动分割和3D表达方法Motion Segmentation and 3D Representation Method for Monocular Image Sequence

技术领域technical field

    本发明涉及一种单目图像序列的运动分割、3D运动信息和深度估计方法，尤其涉及一种采用主运动分割思想，利用区域竞争的水平集方法，基于包含有3D运动信息的时空域模型处理的单目图像序列的运动分割、3D运功信息和深度估计的方法。The present invention relates to a method for motion segmentation, 3D motion information and depth estimation of a monocular image sequence, in particular to a level set method that adopts the idea of main motion segmentation, utilizes regional competition, and processes based on a time-space domain model containing 3D motion information Methods for Motion Segmentation, 3D Motion Information, and Depth Estimation of Monocular Image Sequences.

背景技术Background technique

社会正在进入信息时代，计算机视觉愈来愈普遍地进入到各个领域之中。单目图像序列的运动检测、运动信息和深度估计是计算机视觉的重要分支，又是近年来理论和应用的研究热点。单目图像序列的运动检测、运动信息和深度估计的应用领域非常广泛，如机器视觉、公共设施、医疗诊断、车站和交通场景的监控以及宾馆、楼宇、商场的监控等。The society is entering the information age, and computer vision is more and more widely used in various fields. Motion detection, motion information, and depth estimation of monocular image sequences are important branches of computer vision, and they are also research hotspots in theory and application in recent years. Motion detection, motion information, and depth estimation of monocular image sequences are widely used, such as machine vision, public facilities, medical diagnosis, monitoring of stations and traffic scenes, and monitoring of hotels, buildings, and shopping malls.

单目图像序列的运动分割、运动信息和深度估计是指在一段单目图像序列上，利用三维空间中的真实运动特性来区分和识别不同运动特性的区域或目标，并且从图像序列中将运动目标的深度信息和运动参数恢复出来。Motion segmentation, motion information and depth estimation of a monocular image sequence refers to using the real motion characteristics in three-dimensional space to distinguish and identify regions or targets with different motion characteristics on a monocular image sequence, and extract the motion information from the image sequence The depth information and motion parameters of the target are recovered.

传统的运动分割方法有减背景阈值法、差分图像阈值法、光流法和基于水平集的方法。减背景法计算简单，但是适应性不好，容易出现分割错误。差分图像阈值法一般不能提取所有的相关像素点，运动物体容易出现空洞。光流法运算比较复杂，不能达到实时的应用效果。Traditional motion segmentation methods include background subtraction thresholding, differential image thresholding, optical flow and level set based methods. The background subtraction method is simple to calculate, but its adaptability is not good, and it is prone to segmentation errors. The differential image threshold method generally cannot extract all relevant pixels, and moving objects are prone to holes. The calculation of optical flow method is relatively complicated and cannot achieve real-time application effect.

Osher和Sethian首先提出依赖于时间的运动曲面的水平集（Level Set）描述，这种方法避免了对拓扑结构变化的处理、计算稳定。Caselles等人提出了基于边缘的水平集方法，但这类方法对曲线初始位置的选择非常敏感，不容易达到好的分割效果。Osher and Sethian first proposed the Level Set (Level Set) description of time-dependent motion surfaces. This method avoids the processing of topology changes and stabilizes calculations. Caselles et al. proposed an edge-based level set method, but this type of method is very sensitive to the selection of the initial position of the curve, and it is not easy to achieve a good segmentation effect.

运动目标的3D表达是从图像序列中将运动目标的3D结构和运动参数恢复出来。目前运动目标的3D表达可以分为稀疏的3D表达和稠密的3D表达。稀疏的3D表达是通过图像序列中的一个稀疏点集来计算的，3D表达受限于稀疏点集的选取，不能达到稳定的效果。稠密的3D表达是通过整个图像序列来计算的。但是，大量的基于单目图像序列的稠密3D表达是针对摄像系统运动而环境静止的情况。The 3D expression of the moving target is to restore the 3D structure and motion parameters of the moving target from the image sequence. At present, the 3D representation of moving objects can be divided into sparse 3D representation and dense 3D representation. The sparse 3D expression is calculated by a sparse point set in the image sequence, and the 3D expression is limited by the selection of the sparse point set, which cannot achieve a stable effect. A dense 3D representation is computed over the entire image sequence. However, a large number of dense 3D representations based on monocular image sequences are aimed at the situation where the camera system is moving and the environment is still.

2006年，Hicham第一次将运动分割和3D表达结合起来，然而Hicham的方法只能分割固定数目的运动目标。In 2006, Hicham combined motion segmentation and 3D expression for the first time, but Hicham's method can only segment a fixed number of moving objects.

发明内容Contents of the invention

本发明的目的在于解决针对现有技术的不足，提供一种单目图像序列的运动分割和3D表达方法。The purpose of the present invention is to solve the deficiencies in the prior art, and provide a method for motion segmentation and 3D expression of a monocular image sequence.

本发明的目的是通过以下技术方案来实现的：一种单目图像序列的运动分割和3D表达方法，包括如下步骤： The object of the present invention is achieved by the following technical solutions: a method for motion segmentation and 3D expression of a monocular image sequence, comprising the steps of:

（1）采集单目图像序列：利用摄像机拍摄包含运动目标的场景的图片，摄像机可以运动也可以不运动，运动目标的数目可以是一个也可以是多个；(1) Acquisition of monocular image sequences: Use the camera to take pictures of scenes containing moving objects. The camera can move or not, and the number of moving objects can be one or more;

（2）初始化，在采集得到的图像序列中初始化一个曲面，这样每一帧都会有一条曲线，将图片分为两部分；在整个图像序列区域内将深度                                               

初始化为一个常数；运动速度

也初始化为常数；(2) Initialization, initialize a surface in the acquired image sequence, so that each frame will have a curve, which divides the picture into two parts; in the entire image sequence area, the depth

Initialized as a constant; movement speed

is also initialized to a constant;

（3）通过梯度下降法求运动速度；(3) Calculate the motion speed by the gradient descent method;

（4）通过梯度下降法求深度；(4) Find the depth by the gradient descent method;

（5）通过水平集的方法进行曲面演化；(5) Surface evolution by level set method;

（6）判断收敛条件，如果不满足收敛条件，跳回第3步继续执行，如果满足收敛条件，则验证得到的深度信息是否可靠，修正不可靠的深度信息；(6) Judging the convergence conditions, if the convergence conditions are not met, jump back to step 3 to continue execution, if the convergence conditions are met, verify whether the obtained depth information is reliable, and correct the unreliable depth information;

（7）如果收敛，判断公式（11）已经得到的深度信息是否可靠，并且修正不可靠的深度信息；(7) If it converges, judge whether the depth information obtained by formula (11) is reliable, and correct the unreliable depth information;

（8）重新进行深度和运动信息的估算：用步骤7得到的深度信息和步骤3-5收敛后的运动信息和分割曲面执行步骤2的初始化，反复进行步骤3、4，直至收敛，得到最终的运动信息和深度信息；(8) Re-estimate the depth and motion information: use the depth information obtained in step 7, the motion information converged in steps 3-5, and the segmentation surface to perform the initialization of step 2, and repeat steps 3 and 4 until convergence to obtain the final motion information and depth information;

（9 ）依次将每一个运动的目标的分割曲线分别执行步骤（2）到（8），得到每一个运动目标的运动速度和深度信息。(9) Perform steps (2) to (8) on the segmentation curve of each moving target in turn to obtain the moving speed and depth information of each moving target.

本发明的有益效果是：The beneficial effects of the present invention are:

1、本发明可以将运动分割和3D表达同时进行。 1. The present invention can perform motion segmentation and 3D expression simultaneously. the

2、本发明不需要提前知道图像序列中的目标数目，具有广泛的适用性。2. The present invention does not need to know the number of objects in the image sequence in advance, and has wide applicability.

3、本发明在曲线的水平集运算中，采用了窄带方法，窄带是指距离水平集为-2，-

，-

，-1，1，

，

，2的点，这样可以保证运动分割的正确性，又可以极大地减小了计算量，提高运动分割和3D表达的效率。3. The present invention adopts the narrowband method in the level set operation of the curve, and the narrowband means that the distance level set is -2,-

,-

, -1, 1,

,

, 2 points, which can ensure the correctness of motion segmentation, and can greatly reduce the amount of calculation, improve the efficiency of motion segmentation and 3D expression.

附图说明Description of drawings

图1为本发明中空间三维信息投影至二维图像平面的投影关系所依据的坐标系；Fig. 1 is the coordinate system on which the projection relation of spatial three-dimensional information projected to two-dimensional image plane is based in the present invention;

图2为本发明中能量函数最小的方法流程图；Fig. 2 is the minimum method flowchart of energy function among the present invention;

图3为本发明中验证深度信息可靠性和修正不可靠深度的方法流程图。Fig. 3 is a flowchart of a method for verifying the reliability of depth information and correcting unreliable depth in the present invention.

具体实施方式Detailed ways

为此，本发明采用了一种时空域的处理模型，将3D运动分割和稠密3D表达问题作为一个整体来研究。模型采用了主运动分割的思想，利用了区域竞争的水平集的方法，通过摄像机针孔模型将3D运动信息映射至图像的二维运动中，直接运用3D信息进行运动分割和估计。由于采用了主运动的思想，在分割过程中，背景为主运动目标，这样一次可以分割出背景和所有的目标，并且可以估算出背景的运动信息和深度信息，然后逐个对目标进行运动估计和深度估计，不需要提前知道运动目标的数目。For this reason, the present invention adopts a time-space domain processing model, and studies 3D motion segmentation and dense 3D expression as a whole. The model adopts the idea of main motion segmentation, utilizes the level set method of regional competition, maps 3D motion information to the 2D motion of the image through the camera pinhole model, and directly uses 3D information for motion segmentation and estimation. Due to the adoption of the main motion idea, in the segmentation process, the background is the main moving object, so that the background and all objects can be segmented at one time, and the motion information and depth information of the background can be estimated, and then the motion estimation and For depth estimation, the number of moving objects does not need to be known in advance.

假设I是一个在时空区域D上定义的图像序列，

,

是

的开子集，T是时间序列，

分别是图像序列在横向，纵向和时间方向的灰度差分，设在图像（x,y）处的光流运动为（u,v），根据光流约束方程：Suppose I is an image sequence defined on a spatiotemporal regionD ,

,

yes

An open subset of ,T is a time series,

They are the grayscale difference of the image sequence in the horizontal, vertical and time directions, and the optical flow motion at the image(x, y) is(u, v) , according to the optical flow constraint equation:

                                              （1） (1)

假设

为一个运动刚体，同一刚体有着相同的平移和旋转运动，分别用

和

表示；假设空间中一点P（X,Y,Z）在摄像机平面投影在（x,y）点，根据投影关系，如图一所示，，

，

表示摄像机的焦距。suppose

is a moving rigid body, the same rigid body has the same translation and rotation motion, respectively

and

Representation; assuming that a pointP (X ,Y ,Z ) in the space is projected on the camera plane at point (x ,y ), according to the projection relationship, as shown in Figure 1, ,

,

Indicates the focal length of the camera.

所以摄像机平面上（x,y）点的运动表达为矢量的形式为

，表达为深度和六个运动变量的函数，有：So the motion of the point (x ,y ) on the camera plane is expressed as a vector in the form of

, expressed as a function of depth and six motion variables, has:

                    （2） (2)

                

    （3）

(3)

其中光流速度（u,v）和深度z是图像坐标和时间的函数。where optical flow velocity(u,v) and depthz are functions of image coordinates and time.

令：make:

                                               

                                   （4）                                                 

(4)

令：表示图像中背景区域，

是相对于

的目标区域，运动目标的分割和3D表达可以转化为下面能量函数的最小化问题。make: Indicates the background area in the image,

is relative to

The target area of , the segmentation and 3D expression of the moving target can be transformed into the minimization problem of the following energy function.

          

 （5）

(5)

其中，，

，

和

是用来调节能量函数中各项权重的实常数，in, ,

,

and

is a real constant used to adjust the weights in the energy function,

。

.

为满足在

上单调递减的函数，如

。

to satisfy in

A monotonically decreasing function, such as

.

第1、2项用来度量运动分割和背景的3D表达与实际情况的相似程度。第3项用来约束深度，保证深度的平滑性。第四项用来约束分割曲面，保证曲面的平滑性。Items 1 and 2 are used to measure the similarity between the 3D representation of the motion segmentation and the background and the actual situation. The third item is used to constrain the depth and ensure the smoothness of the depth. The fourth item is used to constrain the split surface to ensure the smoothness of the surface.

采用欧拉方程对能量函数进行最小化来求解分割曲面和背景的3D表达，运动分割、背景区域的深度估计和运动估计是同时进行的。利用能量函数，可分割出背景和所有的运动目标，得到背景的深度信息和运动信息；然后固定演化曲面，利用公式（1）、（2）和（3），可估算出每一个目标的深度信息与运动信息。 The Euler equation is used to minimize the energy function to solve the 3D representation of the segmented surface and background, and the motion segmentation, depth estimation and motion estimation of the background area are performed simultaneously. Using the energy function, the background and all moving objects can be segmented, and the depth information and motion information of the background can be obtained; then, the evolution surface can be fixed, and the depth of each object can be estimated by using formulas (1), (2) and (3). information and sports information. the

如图2所示，本方法单目图像序列的运动分割和3D表达方法，包括如下步骤： As shown in Figure 2, the motion segmentation and 3D expression method of the monocular image sequence of this method includes the following steps:

1、采集单目图像序列1. Acquisition of monocular image sequences

利用摄像机拍摄包含运动目标的场景的图片，摄像机可以运动也可以不运动，运动目标的数目可以是一个也可以是多个。A camera is used to take a picture of a scene containing a moving object, the camera may or may not move, and the number of moving objects may be one or more.

2、初始化，在采集得到的图像序列中初始化一个曲面，这样每一帧都会有一条曲线，将图片分为两部分（曲线可以包括全部目标，也可以包括一部分目标）。在整个图像序列区域内将深度

初始化为一个常数。运动速度

也初始化为常数。2. Initialization. Initialize a surface in the acquired image sequence, so that each frame will have a curve, which divides the picture into two parts (the curve can include all targets or a part of the target). Apply depth to the entire image sequence area

Initialized as a constant. Movement speed

Also initialized to a constant.

3、通过梯度下降法求运动速度3. Find the motion speed by gradient descent method

假设曲面和深度

不变，能量函数

对

和

求导，并且运用梯度下降法，得到：hypothetical surface and depth

Invariant, the energy function

right

and

Derivation, and using the gradient descent method, we get:

                         

              （6）

(6)

其中， 

和

分别表示平移和旋转速度,；in,

and

represent the translational and rotational speeds, respectively;

表示能量函数；

represents the energy function;

S代表分割曲面；S stands for split surface;

。

.

得到迭代公式：Get the iterative formula:

                         

              （7）

(7)

                        

           （8）。

(8).

4、通过梯度下降法求深度4. Find the depth by gradient descent method

假设运动参数和曲面不变，能量函数对深度求导，并且利用梯度下降法：Assuming that the motion parameters and the surface are constant, the energy function is derived from the depth, and the gradient descent method is used:

        

 （9）

(9)

其中，

in,

      

       

 

       

分别是图像序列在横向，纵向和时间方向的灰度差分 ；

Respectively, the grayscale difference of the image sequence in the horizontal, vertical and time directions;

        

为满足在

上单调递减的函数，如；

to satisfy in

A monotonically decreasing function, such as ;

  

；

;

表示焦距；

Indicates the focal length;

Z表示深度；Z means depth;

，

，

是用来调节能量函数中各项权重的实常数

,

,

is a real constant used to adjust the weights in the energy function

得到迭代公式：Get the iterative formula:

                            

              （10）。

(10).

5、通过水平集的方法进行曲面演化5. Surface evolution by level set method

假设运动参数和深度不变，能量函数对曲面求导：Assuming that the motion parameters and depth are constant, the energy function is derived for the surface:

                

    （11）

(11)

其中

，

，是用来调节能量函数中各项权重的实常数；in

,

, is a real constant used to adjust the weights in the energy function;

表示曲率；

Indicates the curvature;

；

;

为满足在上单调递减的函数，如

；

to satisfy in A monotonically decreasing function, such as

;

转化为水平集形式，得到曲面演化的偏微分方程：Transformed into a level set form, the partial differential equation of surface evolution is obtained:

                      

          （12）

(12)

其中

为符号距离函数，K_r为曲率。in

is the signed distance function, andK_r is the curvature.

得到曲面的迭代方程：Get the iterative equation for the surface:

                             

                 （13）

(13)

6、判断收敛条件，如果不满足收敛条件，跳回第3步继续执行，如果满足收敛条件，则验证得到的深度信息是否可靠，修正不可靠的深度信息。6. Judging the convergence conditions, if the convergence conditions are not met, jump back to step 3 to continue execution, if the convergence conditions are met, verify whether the obtained depth information is reliable, and correct the unreliable depth information.

如果步骤3连续三次处理得到的运动速度，差值均小于运动速度均值的10%，且步骤4连续三次处理得到的深度，差值均小于深度均值的10%，且步骤5连续三次处理得到的曲面，曲面位置差值均小于曲面位置均值的10%，则认为收敛条件满足，否则收敛条件不满足。If the motion speed obtained by step 3 is processed three times in a row, the difference is less than 10% of the average value of the motion speed, and the depth obtained by step 4 is processed three times in a row, the difference is less than 10% of the depth average, and step 5 is obtained by three consecutive processes surface, the surface position difference is less than 10% of the mean value of the surface position, then the convergence condition is considered to be satisfied, otherwise the convergence condition is not satisfied.

7、 如果收敛，判断公式（11）已经得到的深度信息是否可靠，并且修正不可靠的深度信息，如图3所示。7. If it converges, judge whether the depth information obtained by formula (11) is reliable, and correct the unreliable depth information, as shown in Figure 3.

对于公式（11）已经得到的深度信息，判断对应的能量函数是否大于阈值，标记大于阈值的深度点。For the depth information obtained by formula (11), judge whether the corresponding energy function is greater than the threshold, and mark the depth points greater than the threshold.

                                      （14） (14)

其中，是大于阈值的深度点的标记索引，in, is the label index of the depth point greater than the threshold,

代表点（i,j）对应的能量函数值， Represents the energy function value corresponding to the point(i, j) ,

为判断能量函数值正确与否的阈值。

It is the threshold for judging whether the value of the energy function is correct or not.

对于公式（11）已经得到的深度信息，判断深度点与其八邻域的深度差，如果有4个或者超过4个大于阈值，那么这个点将被认定为不可靠的。即，For the depth information obtained by formula (11), judge the depth difference between the depth point and its eight neighbors. If there are 4 or more than 4 points greater than the threshold, then this point will be considered as unreliable. Right now,

                                    （15） (15)

其中，

是不可靠深度点的标记索引，

是深度点（i,j）与其八邻域深度差大于阈值的数目。in,

is the labeled index of the unreliable depth point,

is the number of depth points(i, j) and its eight neighbors whose depth difference is greater than the threshold.

对于不可靠的深度点，利用邻域信息和对应的灰度信息进行修正。即修正深度，使得如下能量函数最小，For unreliable depth points, the neighborhood information and corresponding grayscale information are used to correct them. That is, the depth is corrected so that the following energy function is minimized,

                

    （16）

(16)

其中，

表示点（i,j）第t次迭代时的深度值，in,

Indicates the depth value of the point(i, j) at the tth iteration,

m,n为修正深度值时用到的邻域窗口大小。m,n is the size of the neighborhood window used when correcting the depth value.

、、

分别为三项权值，依次表示空间信息、灰度信息和能量信息对该点深度的影响。其定义如下：

, ,

They are three weights, which in turn represent the influence of spatial information, grayscale information and energy information on the depth of the point. It is defined as follows:

                      

          （17）

(17)

                    

        （18）

(18)

                    

        （19）

(19)

             其中，

为点（i,j）的灰度值；in,

is the gray value of point(i, j) ;

、

、

为系数

,

,

is the coefficient

是用来区分邻域可靠深度点和不可靠深度点的权重的，定义如下：

It is used to distinguish the weight of reliable depth points and unreliable depth points in the neighborhood, defined as follows:

                          （20） (20)

其中，

是公式（14）判断出的能量函数大于阈值的深度点或者公式（15）判断出的不可靠的深度点。in,

It is the depth point whose energy function judged by formula (14) is greater than the threshold or the unreliable depth point judged by formula (15).

8 、重新进行深度和运动信息的估算8. Re-estimate the depth and motion information

用7得到的深度信息和3-5步骤收敛后的运动信息和分割曲面执行第2步的初始化，反复进行3、4步骤，直至收敛，得到最终的运动信息和深度信息。Use the depth information obtained in 7, the motion information after convergence in steps 3-5, and the segmentation surface to perform the initialization of step 2, and repeat steps 3 and 4 until convergence to obtain the final motion information and depth information.

9 、依次将每一个运动的目标的分割曲线分别执行步骤（2）到（8），得到每一个运动目标的运动速度和深度信息。9. Carry out steps (2) to (8) for the segmentation curve of each moving target in turn to obtain the moving speed and depth information of each moving target.

Claims (5)

Translated fromChinese

1.一种单目图像序列的运动分割和3D表达方法，其特征在于，包括如下步骤：1. A motion segmentation and 3D expression method of a monocular image sequence, characterized in that, comprising the steps:(1)采集单目图像序列：利用摄像机拍摄包含运动目标的场景的图片，摄像机运动或不运动，运动目标的数目是一个或是多个；(1) Collection of monocular image sequences: use the camera to take pictures of scenes containing moving objects, whether the camera is moving or not, and the number of moving objects is one or more;(2)初始化，在采集得到的图像序列中初始化一个曲面，这样每一帧都会有一条曲线，将图片分为两部分；在整个图像序列区域内将深度Z初始化为一个常数；运动速度T，ω也初始化为常数；(2) Initialize, initialize a surface in the image sequence that is collected, so that each frame will have a curve that divides the picture into two parts; initialize the depth Z to a constant in the entire image sequence area; the motion speed T, ω is also initialized as a constant;(3)通过梯度下降法求运动速度；(3) Find the speed of motion by the gradient descent method;(4)通过梯度下降法求深度；(4) Depth is obtained by gradient descent method;(5)通过水平集的方法进行曲面演化；(5) Carry out surface evolution through the method of level set;(6)判断收敛条件，如果不满足收敛条件，跳回步骤(3)继续执行，如果满足收敛条件，则验证得到的深度信息是否可靠，修正不可靠的深度信息；(6) Judging the convergence condition, if the convergence condition is not satisfied, jump back to step (3) to continue execution, if the convergence condition is satisfied, verify whether the obtained depth information is reliable, and correct the unreliable depth information;(7)如果收敛，判断公式

已经得到的深度信息是否可靠，并且修正不可靠的深度信息；(7) If it converges, the judgment formula

Whether the obtained depth information is reliable, and correct the unreliable depth information;式中，a₁，a₂，a₄是用来调节能量函数中各项权重的实常数；k_r表示曲率；E₀＝(I_x·u+I_y·v+I_t)²；g(E₀)为满足在[0，+∞)上单调递减的函数；In the formula, a₁ , a₂ , a₄ are real constants used to adjust the weights in the energy function; k_r represents the curvature; E₀ =(I_x u+I_y v+I_t )² ; g (E₀ ) is a function that satisfies monotonically decreasing on [0, +∞);(8)重新进行深度和运动信息的估算：用步骤(7)得到的深度信息和步骤(3)-步骤(5)收敛后的运动信息和分割曲面执行步骤(2)的初始化，反复进行步骤(3)和步骤(4)，直至收敛，得到最终的运动信息和深度信息；(8) Re-evaluate the depth and motion information: use the depth information obtained in step (7) and the motion information after step (3)-step (5) convergence and the segmentation surface to perform the initialization of step (2), and repeat the steps (3) and step (4), until convergence, to obtain the final motion information and depth information;(9)依次将每一个运动的目标的分割曲线分别执行步骤(2)到(8)，得到每一个运动目标的运动速度和深度信息。(9) Steps (2) to (8) are performed on the segmentation curve of each moving target in turn to obtain the moving speed and depth information of each moving target.2.根据权利要求1所述单目图像序列的运动分割和3D表达方法，其特征在于，所述步骤(3)具体为：假设曲面s和深度Z不变，能量函数E(S，θ)对T和ω求导，并且运用梯度下降法，得到： 2. according to the motion segmentation of monocular image sequence described in claim 1 and 3D expression method, it is characterized in that, described step (3) is specifically: assume curved surface s and depth Z constant, energy function E (S, θ) Deriving T and ω, and using the gradient descent method, we get:

其中，T(t₁，t₂，t₃)和ω(ω₁，ω₂，ω₃)分别表示平移和旋转速度，Among them, T(t₁ , t₂ , t₃ ) and ω(ω₁ , ω₂ , ω₃ ) denote translational and rotational speeds, respectively,E(S，θ)表示能量函数，E(S, θ) represents the energy function,S代表分割曲面，S stands for split surface,θ(x，y)＝(T(x，y)，ω(x，y)，Z)。θ(x,y)=(T(x,y),ω(x,y),Z).3.根据权利要求1所述单目图像序列的运动分割和3D表达方法，其特征在于，所述步骤(4)具体为：假设运动参数和曲面不变，能量函数对深度求导，并且利用梯度下降法：3. according to the motion segmentation of monocular image sequence described in claim 1 and 3D expression method, it is characterized in that, described step (4) is specifically: assume that motion parameter and curved surface are constant, energy function derivates to depth, and utilizes Gradient descent method:

其中，

in,

E₁＝I_x·u+I_y·v+I_t，E₁ =I_x ·u+I_y ·v+I_t ,I_x，I_y，I_t分别是图像序列在横向，纵向和时间方向的灰度差分；I_x , I_y , It are_the gray level difference of the image sequence in the horizontal, vertical and time directions respectively;g(E₀)为满足在[0，+∞)上单调递减的函数；g(E₀ ) is a function that satisfies the monotonous decrease on [0, +∞);E₀＝(I_x·u+I_y·v+I_t)²；E₀ =(I_x u+I_y v+I_t )² ;f表示焦距；f represents the focal length;Z表示深度；Z means depth;a₁，a₂，a₃是用来调节能量函数中各项权重的实常数。a₁ , a₂ , and a₃ are real constants used to adjust the weights of each item in the energy function.4.根据权利要求1所述单目图像序列的运动分割和3D表达方法，其特征在于，所述步骤(5)具体为：假设运动参数和深度不变，能量函数对曲面求导：4. according to the motion segmentation of monocular image sequence described in claim 1 and 3D expression method, it is characterized in that, described step (5) is specially: assume that motion parameter and depth are constant, and energy function is derived to curved surface:

其中a₁，a₂，a₄是用来调节能量函数中各项权重的实常数； Among them, a₁ , a₂ , and a₄ are real constants used to adjust the weights in the energy function;k_r表示曲率；k_r represents the curvature;E₀＝(I_x·u+I_y·v+I_t)²；E₀ =(I_x u+I_y v+I_t )² ;g(E₀)为满足在[0，+∞)上单调递减的函数；g(E₀ ) is a function that satisfies the monotonous decrease on [0, +∞);转化为水平集形式，得到曲面演化的偏微分方程。Transformed into a level set form, the partial differential equation of surface evolution is obtained.5.根据权利要求1所述单目图像序列的运动分割和3D表达方法，其特征在于，所述步骤(6)中：如果步骤(3)连续三次处理得到的运动速度，差值均小于运动速度均值的10％，且步骤(4)连续三次处理得到的深度，差值均小于深度均值的10％，且步骤(5)连续三次处理得到的曲面，曲面位置差值均小于曲面位置均值的10％，则认为收敛条件满足，否则收敛条件不满足。 5. according to the motion segmentation of the described monocular image sequence of claim 1 and 3D expressing method, it is characterized in that, in described step (6): if step (3) processes the motion velocity that obtains for three consecutive times, difference is all less than motion speed 10% of the mean value of the speed, and the depth obtained by step (4) for three consecutive times, the difference is less than 10% of the mean value of the depth, and the surface obtained by step (5) for three consecutive times, the difference of the position of the surface is less than 10% of the mean value of the position of the surface 10%, it is considered that the convergence condition is satisfied, otherwise the convergence condition is not satisfied. the

Images