CN111307046B - Tree height measuring method based on hemispherical image - Google Patents

Abstract

The invention discloses a method for measuring tree height based on a hemispherical image. The method comprises the steps of adopting a measuring device consisting of an intelligent terminal, a fisheye lens matched with the intelligent terminal and a laser range finder, determining a distortion coefficient by utilizing a black-white checkerboard calibration plate and the measuring device, selecting the highest point and the lowest point of a tree to be measured on a screen of the intelligent terminal, and calculating through a measuring program to obtain the tree height. The method has wide measurement range, the single collected standing tree information amount is large, the tree height measurement of a row of trees can be realized in a hemispherical image, and the measurement workload is effectively reduced; the cost is low, the equipment is portable, the operation is simple, and the measurement can be finished by a single person.

Description

Tree height measuring method based on hemispherical image

Technical Field

The invention relates to the field of agriculture and forestry, in particular to a method for measuring the height of a tree based on a hemispherical image according to a fisheye camera imaging principle in national forest resource investigation.

Background

With the continuous promotion of accurate forestry requirement, it becomes one of the hot problem in the forestry research to measure the standing timber height accurately, conveniently, high-efficiently. In forest resource investigation, the measurement of the height of the standing tree is crucial, and the accumulation of the forest shares and the accuracy of forest biomass extraction are directly influenced. The currently widely adopted tree height measuring instruments include a Brucella height measuring instrument and an ultrasonic height measuring instrument, the commercialized equipment has the advantages of small volume, easiness in carrying and the like, but the measuring principle of the equipment has clear requirements on horizontal movement and vertical movement respectively, and operators cannot meet the requirements strictly, so that human errors are introduced. The method has the advantages that the three-dimensional point cloud data of the trees are obtained by scanning the unmanned aerial vehicle or the ground-based laser radar, the method is high in measurement accuracy, but the application universality is limited due to the factors of expensive equipment, large data processing capacity, long processing time, inconvenience in carrying of the equipment and the like. The total station and the electronic theodolite of the precision equipment have the problems of great weight, complex operation and the like, but the high-precision measurement result can be used as a tree height true value to verify the effectiveness of other measurement methods.

In order to meet the measurement requirements of low measurement cost, small measurement equipment and high measurement precision, in recent years, photogrammetry technology is applied to forestry research. The tree vision measuring system with data display mainly uses common camera to make monocular vision measurement or binocular vision measurement. The monocular vision measurement of a common camera has the advantages of simple camera calibration process, high operation speed and the like, but the angle between the projection of the camera and the optical axis and the vertical depression angle of the camera are required to be introduced due to small visual angle, and the angle information is difficult to measure. The binocular vision measurement can effectively solve the problem of visual angle, two cameras are used for extracting and matching feature points of two images shot by a measured standing tree, and three-dimensional reconstruction of pixel points is carried out. Therefore, a measuring method which is large in measuring range, small in data calculation amount, convenient to carry and capable of being intelligently realized is needed. The research of the method obtains the subsidy of the youth fund project of Heilongjiang province.

Disclosure of Invention

Aiming at the defects of the existing method, the invention provides the tree height measuring method which is large in measuring range, small in data calculation amount, convenient to carry and capable of being intelligently realized. The invention relates to a method for collecting a hemisphere image by using an intelligent terminal provided with a fisheye lens, establishing a tree height measurement model based on the calibration and correction principle of the fisheye lens and calculating the tree height.

1. Establishing fish-eye camera imaging model

The imaging system model of the fisheye camera is shown in fig. 1. The model establishes a world coordinate system (X)_W,Y_W,Z_W) -a fisheye lens coordinate system (X, Y, Z), a camera coordinate system (X ', Y ', Z ') and an image coordinate system (u, v). An imaging process is determined in which a target point in world coordinates is projected to an image point in image coordinates.

Let a given point P in space have world coordinates (x, y, z), OO ' is the optical axis imaged by the fisheye camera, r is the distance from point P to the optical axis, point P ' is the point P imaged, and r ' is the distance from point P ' to the image center O '. According to the theorem of equidistant projection, the method comprises the following steps of,

r'＝fω (4)

wherein f is the object focal length of the optical system, ω is the incident angle of the P point relative to the optical axis,

since the fisheye lens has barrel distortion and radial distortion, in order to ensure the uniformity of image plane illumination, a distortion coefficient lambda is introduced, and then the expression (4) is modified to

r′＝λfω (6)

The point P ' projected to the camera plane during imaging has coordinates (x ', y ') and the center point O ' of the camera plane has coordinates (x '₀,y′₀) The components of the distortion coefficient λ in the x 'direction and the y' direction are λ_xAnd λ_yThe distance from point P ' to point O ' is r ', then there is,

where θ is the azimuth of point P, and also the azimuth of point P' in the camera coordinate system (see fig. 1),

let the center point O' in the image coordinate system be the coordinate (u)₀,v₀) The coordinate of the P 'point which projects the P' point in the camera coordinate system to the image coordinate system at equal intervals is (u, v), the conversion formula from the camera coordinate to the image coordinate is as follows,

let k_x＝λ_xm_xf，k_y＝λ_ym_yf, combined vertical type (5), (6), (7) and (8) to obtain

Wherein k is_xAnd k_yThe distortion coefficient is called as a model distortion coefficient, an equidistant projection model of the distortion coefficient is introduced into the fisheye camera by the formula (9), and a transformation relation between world coordinates and image coordinates is established. The parameters to be determined by the model are the optical centers (u) of the images₀,v₀) Distortion coefficient k_xAnd k_yAnd the z value of point P in world coordinates, as can be seen from figure 1,

z＝h+l (10)

wherein l is the virtual imaging distance of the fisheye lens; h is the distance between the top tangent plane of the fisheye lens and the projection point of the point P in the z direction. After the model parameters are determined, the world coordinates can be obtained by knowing the coordinates of the image points.

According to the isometric projection theorem, when a distortion coefficient k is introduced_xAnd k_yThe component r 'of the distance r' from the projected point P 'to the central point O' of the image in the directions of the x-axis and the y-axis_xAnd r ″)_yAnd the incident angle omega of the point P in the calibration plate relative to the optical axis is satisfied

When neglecting l, the distance between the top tangent plane of the fish head of the glasses and the plane of the calibration plate is h, so that

Where r is the distance from point P to the optical axis.

Calibration projection model As shown in FIG. 3, the pixel coordinate of the center point O' of the fisheye image in the image plane is (u)₀,v₀) If the projected point P' of a point P in the calibration plate in the image plane has coordinates of (u, v)₀) Then r ″ ═ r ″_x＝|u-u₀If the projected point P' of a point P in the calibration plate in the image plane is in coordinates of (u)₀V), then r ″ ═ r ″_y＝|v-v₀The calculation formula for obtaining the distortion coefficient by the joint type (11) and (12) is

Wherein the pixel coordinate of the center point O' of the image is (u)₀,v₀) H is the distance from the intelligent terminal to the calibration board, the pixel coordinate of the angular point P is (u, v), and r is the distance from the point P to the optical axis.

2. Measuring method and procedure

The method comprises the steps of adopting a measuring device consisting of an intelligent terminal, a fisheye lens matched with the intelligent terminal and a laser range finder, determining a distortion coefficient by utilizing a black-white checkerboard calibration plate and the measuring device, selecting the highest point and the lowest point of a tree to be measured on a screen of the intelligent terminal, and calculating through a measuring program to obtain the tree height. Wherein: the fisheye lens and the laser range finder are fixed on the intelligent terminal through the clamp. The intelligent terminal is provided with a touch screen, a gyroscope and a Bluetooth module, a camera on the intelligent terminal is provided with a cross center reference line, and the intelligent terminal can be an intelligent tool such as a mobile phone or a tablet computer. Laser range finder passes through bluetooth communication with intelligent terminal, and laser range finder can be the laser range finder finished product that has bluetooth function, also can adopt industry laser rangefinder sensor module to install bluetooth module additional and realize the function. The black and white checkerboard is square, the side length is 1 meter, and the size of each grid is 50mm x 50 mm; the intelligent terminal solidifies the measuring operation program.

The measuring method comprises the following steps:

1) shooting a black and white chessboard pattern calibration board photo by using an intelligent terminal: the calibration plate is fixed perpendicular to the ground, shooting is guaranteed to be conducted by the aid of the intelligent terminal gyroscope, the camera is perpendicular to the ground, the distance between the camera and the calibration plate is preferably 1-2 m, the accurate distance value h is measured by the laser range finder, the center point of the cross center reference line of the camera is aligned to any angular point of any square of the calibration plate during shooting, and the point is the original point of the calibration plate.

2) Importing the photos to the MATLAB program of the computer system, centering the imagesEstablishing an image coordinate system by taking the point as an original point O' of the image coordinate system, respectively selecting 10 calibration plate small grid angular points along the directions of horizontal and vertical coordinate axes by taking a pixel as a unit, knowing the pixel coordinate, and determining a distortion coefficient k of a fisheye lens matched with an intelligent terminal camera through calculation_xAnd k_yDistortion coefficient k_xAnd k_yIs a function of the pixel coordinates P (u, v) of the corner points of the calibration plate, i.e.:

wherein the pixel coordinate of the center point O' of the image is (u)₀,v₀) H is the distance from the intelligent terminal to the calibration board and is measured by a laser range finder; the pixel coordinates of the angular point P are (u, v), r is the distance from the inner angular point of the calibration plate to the origin of the calibration plate, and the unit is millimeter; substituting 10 angular point data into the average value calculated by formula (1) to be used as k_xAnd k_yThe final value of (c). Will k_xAnd k_yAnd substituting the data into the solidified measuring program of the intelligent terminal.

3) Measuring the tree height: the method comprises the steps of starting a measurement program of the intelligent terminal to measure, enabling the intelligent terminal gyroscope to be perpendicular to the ground, transmitting the measured tree distance H to be measured to the intelligent terminal through Bluetooth by the laser range finder, selecting the highest point B and the lowest point A of the measured tree on a touch screen of the intelligent terminal, and calculating through the measurement program to obtain the tree height H.

And constructing a tree height measuring and calculating system model by taking a fisheye camera equidistant projection model with introduced distortion coefficients as a theoretical basis. The whole measuring system consists of an intelligent terminal with a fish-eye lens and a range finder. A schematic diagram of which is shown in fig. 2.

The coordinates of the hemispherical image point A 'and the hemispherical image point B' are known to be (u) respectively_A′,v_A′) And (u)_B′,v_B′) The unit is pixel, and the coordinates of the point A and the point B in the world coordinate system are determined.

According to the equation (9), the corresponding relationship between the image coordinate and the x coordinate in the world coordinate system is,

since in practical measurements typically h is greater than 1000mm and l is 8mm-16mm, then with l < < h, equation (7) is simplified to z ═ h. Combined vertical type (9), (11) are obtained

The coordinates of the point A and the point B in the world coordinate system are respectively (x)_A,y_AH) and (x)_B,y_BH), the calculation formula is:

wherein (u, v) are pixel coordinates of the point A and the point B in an image coordinate system, the step 3) is used for selecting on the display screen of the intelligent terminal, H is the distance of the tree to be measured by the laser range finder, the tree height H is obtained by calculation according to the distance between the two points in the space,

drawings

FIG. 1 is a diagram of a fish-eye camera imaging system

FIG. 2 is a schematic view of a tree height measurement system model

FIG. 3 is a diagram of a calibration projection model

Detailed Description

The specific implementation mode of the invention is illustrated by taking a group of data of field shooting experiments of experimental forest farms of northeast forestry university as an example.

The experimental equipment adopts an iPhone7 apple mobile phone with a fisheye lens, a black and white chessboard grid calibration board, a laser range finder (a Bluetooth module is additionally arranged on a laser range finding industrial sensor module), an ultrasonic range finder (Vertex III & Transponder T3) and a total station (southern NTS-362). The experimental scheme is as follows: the data measured by the method and the data measured by the ultrasonic distance meter are respectively compared with the true value by taking the total station instrument 5-time measurement mean value as the true value, so as to verify the accuracy of the method.

Shooting a black and white chessboard pattern calibration board photo by using an intelligent terminal: the size of each grid of the black and white checkerboard calibration board is 50mm x 50mm, the intelligent terminal gyroscope is utilized to enable the terminal to be perpendicular to the ground, the center of a cross reference line of the camera is aligned to any angular point of any grid of the calibration board during shooting, the angular point is used as an original point of calibration, a world coordinate system is established by taking an intersection point O of an optical axis of the fisheye camera and a bottom plane of the fisheye camera as a central point, and the laser range finder measures an accurate distance value h by taking millimeters as a unit. The image capturing distances were 541mm, 631mm, 766mm, 842mm, and 942mm, respectively.

Importing the photos into an MATLAB program of a computer system, establishing an image coordinate system, taking pixels as units, taking the central point position of the photos as an original point O' of the image coordinate system, respectively selecting pixel coordinates P of 10 small grid corner points of a calibration board along the horizontal and vertical directions of the original point, wherein the size of a hemispherical image is 3024 pixels by 3024 pixels, taking 12 image points for each of five images, obtaining 60 corner point coordinates, and determining a distortion coefficient k suitable for the intelligent terminal and a fisheye lens matched with the intelligent terminal through calculation_xAnd k_y，k_xAnd k_yRespectively take the average value of

And

will k_xAnd k_yAnd substituting the data into the solidified measuring program of the intelligent terminal.

Measuring the tree height: the method comprises the steps of starting a measurement program of an intelligent terminal to measure, enabling the terminal to be perpendicular to the ground by using an intelligent terminal gyroscope, selecting the highest point B and the lowest point A of a tree to be measured on a screen of the intelligent terminal, transmitting the distance H between a mobile phone and the tree to be measured to the intelligent terminal through Bluetooth by using a laser range finder, and calculating by the measurement program to obtain the tree height H.

The numbers of the collected hemisphere images are respectively 1, 2 and 3 in field measurement, the shooting distances h are respectively 8080mm, 8380mm and 8980mm, the numbers of trees to be calculated are respectively I-VI, the shooting distances are selected according to the characteristic that the total station and the ultrasonic distance meter have the highest precision when the height of a measured object is equal to the measuring distance, and the average value of 5 times of measurement of the total station is used as a true value. The measured tree heights of the ultrasonic range finder and the measured tree heights of the measuring tree proposed herein were compared under the same conditions as the shooting points and the measuring points, and the comparison data are shown in table 1.

TABLE 1 real ground stumpage estimation

Compared with the traditional ultrasonic height measurement method, the method for measuring the tree height has higher precision and more stable measurement result.

The invention combines the fisheye camera calibration technology, the close-range photogrammetry technology and the machine vision technology, and provides a stumpage measurement method based on fisheye images, the method has wide measurement range, the single collected stumpage information amount is large, the tree height measurement of a row of trees can be realized in a hemisphere image, and the measurement workload is effectively reduced; the cost is low, the equipment is portable, the operation is simple, and the measurement can be finished by a single person.

Claims (3)

Translated fromChinese

1.一种基于半球图像的树高测量方法，其特征在于：利用配有鱼眼镜头的智能终端采集半球图像，基于鱼眼镜头的标定和校正原理建立树高测量模型并计算树高的方法，采用智能终端、与智能终端适配的鱼眼镜头和激光测距仪组成的测量装置，利用黑白棋盘格标定板配合测量装置确定畸变系数，在智能终端屏幕上选取被测树木的最高点及最低点，经过测量程序计算得到树高；其中：鱼眼镜头与激光测距仪均通过夹具固定在智能终端上；智能终端具有触控屏、陀螺仪和蓝牙模块，智能终端上的相机具有十字中心参考线；激光测距仪与智能终端通过蓝牙通信；黑白棋盘格标定板为正方形，边长为1米，每格大小为50mm*50mm；智能终端固化了测量操作程序；1. a tree height measurement method based on hemispherical image, it is characterized in that: utilize the intelligent terminal that is equipped with fisheye lens to collect hemispherical image, set up tree height measurement model and calculate the method for tree height based on the calibration and correction principle of fisheye lens , using a measuring device composed of an intelligent terminal, a fisheye lens adapted to the intelligent terminal and a laser rangefinder, using a black and white checkerboard calibration board to cooperate with the measuring device to determine the distortion coefficient, and selecting the highest point and The lowest point, the tree height is calculated by the measurement program; among them: the fisheye lens and the laser range finder are fixed on the smart terminal by a fixture; the smart terminal has a touch screen, a gyroscope and a Bluetooth module, and the camera on the smart terminal has a cross Center reference line; the laser rangefinder communicates with the smart terminal through Bluetooth; the black and white checkerboard calibration board is a square, the side length is 1 meter, and the size of each grid is 50mm*50mm; the smart terminal solidifies the measurement operation procedure;测量方法包括如下步骤：The measurement method includes the following steps:1)使用智能终端拍摄黑白棋盘格标定板照片：将标定板垂直于地面固定，利用智能终端陀螺仪保证相机垂直于地面拍摄，相机与标定板距离1-2米之间，激光测距仪测量到标定板的准确距离数值h，拍摄时相机的十字中心参考线的中心点对准标定板任一方格的任一角点，该点即为标定板原点；1) Use the smart terminal to take a photo of the black-and-white checkerboard calibration board: fix the calibration board perpendicular to the ground, use the smart terminal gyroscope to ensure that the camera is perpendicular to the ground to shoot, the distance between the camera and the calibration board is 1-2 meters, and the laser rangefinder measures The exact distance value h to the calibration plate, the center point of the cross center reference line of the camera is aligned with any corner of any square of the calibration plate when shooting, and this point is the origin of the calibration plate;2)将照片导入计算机系统的MATLAB程序，将图像中心点作为图像坐标系原点O″建立图像坐标系，以像素为单位，沿水平及垂直坐标轴方向分别选取10个标定板小格角点，已知像素坐标，通过计算确定与智能终端相机适配的鱼眼镜头的畸变系数k_x和k_y，畸变系数k_x和k_y是关于标定板小格角点像素坐标(u,v)的函数，即：2) Import the photo into the MATLAB program of the computer system, use the image center point as the origin of the image coordinate system O" to establish the image coordinate system, take pixels as the unit, and select 10 small grid corner points of the calibration board respectively along the horizontal and vertical coordinate axis directions, Knowing the pixel coordinates, the distortion coefficients k_x and_{ky of the fisheye lens adapted to the smart terminal camera are determined by calculation. The distortion coefficients k x and kyarerelated} to the pixel coordinates (u, v) of the small grid corners of the calibration board. function, that is:

其中，图像的中心点O″的像素坐标为(u₀,v₀)，h为智能终端到标定板的距离，由激光测距仪测得；角点的像素坐标为(u,v)，r为标定板内角点到标定板原点的距离，以毫米为单位；将10个角点数据代入公式(1)计算得到的平均值作为k_x和k_y的最终值，将k_x和k_y代入智能终端已经固化的测量程序中；Among them, the pixel coordinates of the center point O″ of the image are (u₀ , v₀ ), h is the distance from the smart terminal to the calibration board, which is measured by the laser rangefinder; the pixel coordinates of the corner points are (u, v), r is the distance from the inner corner of the calibration plate to the origin of the calibration plate, in millimeters; the average value calculated by substituting the data of 10 corner points into formula (1) is taken as the final value of k_x and_ky , and k_x and_ky Substitute into the measurement program that has been solidified by the smart terminal;3)测量树高：打开智能终端的测量程序开始进行测量，利用智能终端陀螺仪使其垂直于地面，激光测距仪将测得的待测树木距离h通过蓝牙传输给智能终端，在智能终端触控屏上选取被测树木的最高点B及最低点A，世界坐标系中的A点和B点的坐标分别为(x_A,y_A,h)和(x_B,y_B,h)，计算公式为：3) Measure the tree height: Open the measurement program of the smart terminal to start the measurement, use the smart terminal gyroscope to make it perpendicular to the ground, and the laser range finder transmits the measured distance h of the tree to be measured to the smart terminal through Bluetooth. Select the highest point B and the lowest point A of the tested tree on the touch screen. The coordinates of point A and point B in the world coordinate system are (x_A , y_A , h) and (x_B , y_B , h) , the calculation formula is:

式中，(u,v)是A点和B点在图像坐标系中的像素坐标，h为激光测距仪测得的待测树木距离，根据空间内两点之间的距离计算得到树高H为：In the formula, (u, v) are the pixel coordinates of point A and point B in the image coordinate system, h is the distance of the tree to be measured measured by the laser rangefinder, and the tree height is calculated according to the distance between the two points in the space. H is:

2.根据权利要求1所述的基于半球图像的树高测量方法，所述智能终端为智能手机或平板电脑。2 . The method for measuring tree height based on a hemispherical image according to claim 1 , wherein the intelligent terminal is a smartphone or a tablet computer. 3 .3.根据权利要求1所述的基于半球图像的树高测量方法，所述激光测距仪为具有蓝牙功能的激光测距仪成品或工业激光测距传感器模块加装蓝牙模块。3 . The tree height measurement method based on a hemispherical image according to claim 1 , wherein the laser range finder is a finished laser range finder with bluetooth function or an industrial laser range finder sensor module with a bluetooth module added. 4 .

Images