CN112686877B - Construction and measurement method and system of 3D house damage model based on binocular camera - Google Patents
Construction and measurement method and system of 3D house damage model based on binocular cameraDownload PDFInfo
- Publication number
- CN112686877B CN112686877BCN202110009707.9ACN202110009707ACN112686877BCN 112686877 BCN112686877 BCN 112686877BCN 202110009707 ACN202110009707 ACN 202110009707ACN 112686877 BCN112686877 BCN 112686877B
- Authority
- CN
- China
- Prior art keywords
- damage
- house
- binocular camera
- algorithm
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及房屋测量技术领域。The invention relates to the technical field of house measurement.
背景技术Background technique
民用建筑在日常使用中，常因一些偶然或潜在的事件，如火灾、地震等，导致建筑出现裂缝、钢筋锈蚀、混凝土脱落等情况，这些情况会影响建筑的正常使用，甚至危及建筑结构安全。房屋质量监测是指由专业人士运用一定的技术手段和方法，对房屋结构质量进行日常检查测定，实施规律性的监测。房屋质量检测是指由专业人士对房屋出现的损伤进行检测和评估，并开具质量报告。In the daily use of civil buildings, some accidental or potential events, such as fires and earthquakes, often lead to cracks, corrosion of steel bars, and shedding of concrete in buildings, which will affect the normal use of buildings and even endanger the safety of building structures. Housing quality monitoring refers to the daily inspection and measurement of housing structure quality by professionals using certain technical means and methods, and the implementation of regular monitoring. Housing quality inspection refers to the detection and evaluation of damage to the house by professionals, and the issuance of a quality report.
当前的房屋质量监测和检测内容主要是结构沉降﹑倾斜﹑裂缝﹑材料强度检测﹑房屋抗震能力检测与鉴定等，其主要的技术设备包括测距仪﹑盒尺、游标卡尺﹑全站仪﹑钢筋探测仪﹑回弹仪等，这些检测设备和方法一方面是检测精度不高，速度慢，费用高，且无法实现长时间的监测，另一方面是整体检测程序较为复杂，检测结果常依靠经验判断，易受人为因素影响。因此，如何准确、快速的对房屋质量和损伤进行日常监测和检测，以及城市大规模建筑群灾后快速检测和安全评估成为一个亟待解决的问题。The current housing quality monitoring and testing content mainly includes structural settlement, inclination, cracks, material strength testing, building seismic capacity testing and appraisal, etc. The main technical equipment includes rangefinders, box rulers, vernier calipers, total stations, and steel bar detection. On the one hand, these detection equipment and methods are not high in detection accuracy, slow in speed, high in cost, and cannot realize long-term monitoring; on the other hand, the overall detection procedure is relatively complicated, and the detection results often rely on empirical judgment , susceptible to human factors. Therefore, how to accurately and quickly conduct daily monitoring and detection of housing quality and damage, as well as rapid post-disaster detection and safety assessment of large-scale urban buildings has become an urgent problem to be solved.
发明内容Contents of the invention
本发明的目的在于克服上述技术不足，提出一种基于双目相机的三维房屋损伤模型构建测量方法及系统，解决现有房屋检测精度低，效率慢且程序复杂，无法对大规模建筑群进行快速检测和安全评估的问题。The purpose of the present invention is to overcome the above-mentioned technical deficiencies, and propose a three-dimensional building damage model construction measurement method and system based on a binocular camera, which solves the problem of low detection accuracy, slow efficiency and complicated procedures of existing buildings, which cannot quickly conduct large-scale building groups. Issues of detection and safety assessment.
为达到上述技术目的，本发明的技术方案第一方面提供一种基于双目相机的三维房屋损伤模型构建测量方法，其包括如下步骤：In order to achieve the above technical objectives, the first aspect of the technical solution of the present invention provides a method for constructing a three-dimensional house damage model based on a binocular camera, which includes the following steps:
通过双目相机扫描得到房屋室内平面图像，并根据双目立体匹配算法计算得到深度图像；The indoor plane image of the house is scanned by the binocular camera, and the depth image is calculated according to the binocular stereo matching algorithm;
采用损伤识别分割算法将裂缝损伤自双目相机中左目相机的房屋室内平面图像中分割出来；The damage recognition and segmentation algorithm is used to segment the crack damage from the indoor plane image of the left camera of the binocular camera;
采用3D损伤精细化量化算法将分割得到的像素点投影到深度图上，即可获取到裂缝损伤中每个点的三维信息；Using the 3D damage refinement quantification algorithm to project the segmented pixel points onto the depth map, the 3D information of each point in the crack damage can be obtained;
利用3D Convex hull凸包算法对裂缝损伤进行三维体积计算。Using 3D Convex hull convex hull algorithm to calculate the three-dimensional volume of fracture damage.
本发明第二方面提供一种基于双目相机的三维房屋损伤模型构建测量系统，其包括如下功能模块：The second aspect of the present invention provides a three-dimensional building damage model construction measurement system based on a binocular camera, which includes the following functional modules:
图像采集计算模块，用于通过双目相机扫描得到房屋室内平面图像，并根据双目立体匹配算法计算得到深度图像；The image acquisition calculation module is used to obtain the indoor planar image of the house through binocular camera scanning, and calculate the depth image according to the binocular stereo matching algorithm;
裂缝损伤分割模块，用于采用损伤识别分割算法将裂缝损伤自双目相机中左目相机的房屋室内平面图像中分割出来；The crack damage segmentation module is used to segment the crack damage from the indoor plane image of the left eye camera in the binocular camera by using the damage recognition segmentation algorithm;
深度投影模块，用于采用3D损伤精细化量化算法将分割得到的像素点投影到深度图上，即可获取到裂缝损伤中每个点的三维信息；The depth projection module is used to project the segmented pixel points onto the depth map using the 3D damage refinement quantization algorithm, so as to obtain the three-dimensional information of each point in the crack damage;
损伤体积计算模块，用于利用3D Convex hull凸包算法对裂缝损伤进行三维体积计算。The damage volume calculation module is used to calculate the three-dimensional volume of fracture damage by using the 3D Convex hull algorithm.
本发明第三方面提供一种服务器，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述一种基于双目相机的三维房屋损伤模型构建测量方法的步骤。The third aspect of the present invention provides a server, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the above-mentioned one based on The steps of the measurement method for building a 3D house damage model with a binocular camera.
本发明第四方面提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现上述一种基于双目相机的三维房屋损伤模型构建测量方法的步骤。The fourth aspect of the present invention provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned three-dimensional building damage model construction measurement based on a binocular camera is realized method steps.
与现有技术相比，本发明利用双目相机成像视差原理，计算得到房屋的深度图，进而得到房屋结构的各个位置及各个构件的实际尺寸信息，实现对房屋结构进行非接触式高精度三维测量和无损检测，且检测效率高；同时将裂缝损伤自双目相机中左目相机的房屋室内平面图像中分割出来，并自2D视图投影到3D视图上，利用3D Convex hull凸包算法计算得到裂缝损伤的三维体积，即可直接获取房屋损伤的实际数据，便于城市大规模建筑群灾后快速检测和安全评估。Compared with the prior art, the present invention uses the imaging parallax principle of the binocular camera to calculate the depth map of the house, and then obtain the actual size information of each position of the house structure and each component, and realize the non-contact high-precision three-dimensional analysis of the house structure. Measurement and non-destructive testing with high detection efficiency; at the same time, the crack damage is segmented from the indoor plane image of the left camera in the binocular camera, and projected from the 2D view to the 3D view, and the crack is calculated using the 3D Convex hull algorithm The three-dimensional volume of damage can directly obtain the actual data of house damage, which is convenient for rapid detection and safety assessment of large-scale urban buildings after disasters.
附图说明Description of drawings
图1是本发明实施例所述的一种基于双目相机的三维房屋损伤模型构建测量方法的流程框图；Fig. 1 is a block flow diagram of a three-dimensional building damage model construction measurement method based on a binocular camera according to an embodiment of the present invention;
图2是本发明实施例所述的一种基于双目相机的三维房屋损伤模型构建测量系统的模块框图。Fig. 2 is a module block diagram of a three-dimensional building damage model building measurement system based on a binocular camera according to an embodiment of the present invention.
具体实施方式Detailed ways
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
如图1所示，本发明的实施例提供了一种基于双目相机的三维房屋损伤模型构建测量方法，其包括如下步骤：As shown in Figure 1, an embodiment of the present invention provides a method for constructing a three-dimensional house damage model based on a binocular camera, which includes the following steps:
S1、通过双目相机扫描得到房屋室内平面图像，并根据双目立体匹配算法计算得到深度图像。S1. Obtain the indoor plane image of the house through binocular camera scanning, and calculate the depth image according to the binocular stereo matching algorithm.
即基于房屋室内平面图像，利用三维建模模块建立室内三维重建模型，并利用点云实例分割算法将所有的承重构件及非承重构件在三维点云数据中进行实例分割，对不同构件类型进行分割与定位。That is, based on the indoor plane image of the house, use the 3D modeling module to establish an indoor 3D reconstruction model, and use the point cloud instance segmentation algorithm to segment all load-bearing components and non-load-bearing components in the 3D point cloud data, and segment different component types and positioning.
具体的，首先获取双目相机中相机内部参数信息，根据相机内部参数信息对相机获取图像进行畸变矫正；也可以通过IMU(惯性传感器)对相机坐标系进行矫正，通过对IMU与相机的初始位姿标定，获取IMU与相机的未知刚体变换，可以更好的利用IMU反馈数据进行实时稳像。在相机标定过程中主要根据三个坐标系进行标定，求解两两坐标系之间的关系。它们分别为图像坐标系、相机坐标系和世界坐标系。利用棋盘格标定的方法对相机进行标定，求解相机的内参和外参。将外参作为相机在世界坐标系下采集数据点的坐标值，将内参用于稳像过程中求解单应性矩阵，来完成IMU-相机的相对位姿标定。Specifically, first obtain the internal parameter information of the camera in the binocular camera, and correct the distortion of the image acquired by the camera according to the internal parameter information of the camera; the camera coordinate system can also be corrected through the IMU (inertial sensor), and the initial position of the IMU and the camera can be corrected. Attitude calibration, obtaining the unknown rigid body transformation between the IMU and the camera, can make better use of the IMU feedback data for real-time image stabilization. In the process of camera calibration, calibration is mainly carried out according to three coordinate systems, and the relationship between two coordinate systems is solved. They are image coordinate system, camera coordinate system and world coordinate system respectively. Use the checkerboard calibration method to calibrate the camera, and solve the internal and external parameters of the camera. The external parameters are used as the coordinate values of the data points collected by the camera in the world coordinate system, and the internal parameters are used to solve the homography matrix during the image stabilization process to complete the relative pose calibration of the IMU-camera.
然后通过双目相机进行室内场景的360度场景扫描，得到房屋室内平面图像以及点云数据，基于statistical outlier removal(SOR)filter对点云数据滤波之后，利用SDK库的RTAB-map机器人建图模型构建得到室内三维重建模型，通过室内三维重建模型可以实时测量室内结构构件的实际尺寸信息，用以替代现有房屋质量检测中利用接触式的测量方法。Then use the binocular camera to scan the 360-degree scene of the indoor scene to obtain the indoor plane image and point cloud data of the house. After filtering the point cloud data based on the statistical outlier removal (SOR) filter, use the RTAB-map robot of the SDK library to build a mapping model The indoor three-dimensional reconstruction model is constructed, and the actual size information of the indoor structural components can be measured in real time through the indoor three-dimensional reconstruction model, which is used to replace the contact measurement method used in the existing house quality inspection.
采用pointnet++训练网络对承重构件及非承重构件进行识别，并将不同构件分别通过不同的颜色在三维点云数据中进行分割显示，从而实现对不同构件类型进行分割与定位。The pointnet++ training network is used to identify the load-bearing and non-load-bearing components, and the different components are segmented and displayed in the 3D point cloud data in different colors, so as to realize the segmentation and positioning of different component types.
为了模拟人眼对立体场景的捕捉和对不同景物远近的识别能力，双目立体匹配算法(SGBM算法)要求采用两个摄像头代替人眼，通过获取两幅非常接近的图片以获取景深(视差：Disparity)，从而计算出不同景物与摄像头的距离,得到景深图。本发明采用双目相机代替人眼，即当两个同一水平线上的摄像头进行拍摄的时候，同一物体将在两个摄像机内被拍摄到，在两个摄像机内部，这个物体相对于摄像机中心点位置有不同的坐标，设Xleft是该物体在左摄像机内相对位置，Xright是该物体在右摄像机内相对位置。当将两幅图像重叠在一起的时候，左摄像机上P的投影和右摄像机上P的投影位置有一个距离|Xleft|+|Xright|，这个距离称为视差，基于视差原理，并以双目相机中左目相机的坐标为对齐坐标，根据相似三角形可以得到物体P距离摄像机的景深z，进而得到深度图。所述景深z的计算公式如下：In order to simulate the ability of the human eye to capture the stereo scene and recognize the distance of different scenes, the binocular stereo matching algorithm (SGBM algorithm) requires two cameras instead of the human eye, and obtains the depth of field by obtaining two very close pictures (parallax: Disparity), so as to calculate the distance between different scenes and the camera, and obtain the depth of field map. The present invention uses binocular cameras instead of human eyes, that is, when two cameras on the same horizontal line are shooting, the same object will be photographed in the two cameras. There are different coordinates, let Xleft be the relative position of the object in the left camera, and Xright be the relative position of the object in the right camera. When two images are overlapped together, there is a distance between the projection position of P on the left camera and the projection position of P on the right camera |Xleft|+|Xright|, this distance is called parallax, based on the principle of parallax, and binocular The coordinates of the left-eye camera in the camera are the alignment coordinates. According to the similar triangle, the depth z of the object P from the camera can be obtained, and then the depth map can be obtained. The calculation formula of the depth of field z is as follows:
z＝sf/dz=sf/d
其中，S为两个摄像机相距，f为焦距，d为视差Disparity(||Xleft|-|Xright||)。Among them, S is the distance between the two cameras, f is the focal length, and d is the parallax Disparity(||Xleft|-|Xright||).
S2、采用损伤识别分割算法将裂缝损伤自双目相机中左目相机的房屋室内平面图像中分割出来。S2. Segment the crack damage from the indoor planar image of the house in the left camera of the binocular camera by using a damage recognition segmentation algorithm.
即用深度网络Mask R-CNN自双目相机中左目相机的房屋室内平面图像中分割出来。That is, the deep network Mask R-CNN is used to segment the indoor planar image of the left camera in the binocular camera.
S3、采用3D损伤精细化量化算法将分割得到的像素点投影到深度图上，即可获取到裂缝损伤中每个点的三维信息。S3. Using the 3D damage refinement quantification algorithm to project the segmented pixel points onto the depth map, the three-dimensional information of each point in the crack damage can be obtained.
即通过3D损伤精细化量化算法强行将1080x720的三通道图像计算出一个1080x720的六通道点云，具体的，由于SGBM立体匹配的时候是将深度图和左目相机的坐标对齐，因此可以将每个坐标加上左目RGB的颜色，得到的点云的一个点的描述是[X,Y,Z,R,G,B]。That is, the 1080x720 three-channel image is forcibly calculated into a 1080x720 six-channel point cloud through the 3D damage refinement quantization algorithm. Specifically, since the coordinates of the depth map and the left-eye camera are aligned during SGBM stereo matching, each The coordinates are added to the RGB color of the left eye, and the description of a point of the obtained point cloud is [X, Y, Z, R, G, B].
其中，立体匹配不成功的区块的点无法计算出的点云坐标，则以NAN表示强行占据矩阵的位置，使2D图像坐标和3D点云坐标的位置对齐。比如严重反光,无限远之类的点云数据，这些数据实际上就是噪音。Among them, the point cloud coordinates of the points of the block whose stereo matching is unsuccessful can not be calculated, and the position of the matrix is forcibly occupied by NAN, so that the positions of the 2D image coordinates and the 3D point cloud coordinates are aligned. For example, point cloud data such as severe reflection and infinity, these data are actually noise.
所述2D图像坐标和3D点云坐标对齐的依据是通过选取每个像素点的视差，组成一个视差图，设置一个和视差图相关的全局能量函数，使这个能量函数最小化，以达到求解每个像素最优视差的目的，其函数表达如下：The basis for aligning the 2D image coordinates with the 3D point cloud coordinates is to form a disparity map by selecting the disparity of each pixel, and setting a global energy function related to the disparity map to minimize this energy function, so as to solve each The purpose of the optimal disparity of pixels, its function is expressed as follows:
其中，D代表视差图，E(D)代表视差图对应的能量函数；p,q代表图像中的某个像素；Np代表像素p的相邻像素点(一般为8连通)；C(p,Dp)指当前像素点视差为Dp时，该像素点的cost；p1为一个惩罚系数，它适用于像素p相邻像素中视差值与p的视差值相差1的那些像素；p2为一个惩罚系数，它适用于像素p相邻像素中视差值与p的视差值相差大于1的那些像素；I[...]表示函数返回1，如果函数中的参数为真，否则返回0。Among them, D represents the disparity map, E(D) represents the energy function corresponding to the disparity map; p, q represent a certain pixel in the image; Np represents the adjacent pixels of pixel p (generally 8-connected); C(p , Dp ) refers to the cost of the pixel when the disparity of the current pixel is Dp ; p1 is a penalty coefficient, which is applicable to those pixels whose disparity value differs from the disparity value of p in adjacent pixels of pixel p by 1; p2 is a penalty coefficient, which applies to those pixels whose disparity value differs from the disparity value of p in the adjacent pixels of pixel p by more than 1; I[…] means that the function returns 1, if the parameter in the function is true, Otherwise return 0.
投影之后再利用statistical outlier removal(SOR)filter进行点云数据滤波，就完成了在3D点云中的损伤识别与分割。After the projection, the point cloud data is filtered using the statistical outlier removal (SOR) filter to complete the damage identification and segmentation in the 3D point cloud.
另外，通过坐标对齐投影到3D点云，还可以精确获取多维度的尺寸信息，由于每个点云都带有实际的空间坐标(x,y,z)，因此只需要计算两个点云的距离即可以计算得到损伤的长宽高,比如点云A1(x1,y1,z1)和点云A2(x2,y2,z2)之间的距离d就可以用下式来计算得到：In addition, through coordinate alignment and projection to the 3D point cloud, multi-dimensional size information can also be accurately obtained. Since each point cloud has actual space coordinates (x, y, z), only two point clouds need to be calculated. The distance can be calculated to obtain the length, width and height of the damage. For example, the distance d between point cloud A1 (x1, y1, z1) and point cloud A2 (x2, y2, z2) can be calculated by the following formula:
S4、利用3D Convex hull凸包算法对裂缝损伤进行三维体积计算。S4. Using the 3D Convex hull algorithm to calculate the three-dimensional volume of the crack damage.
在利用3D Convex hull凸包算法对裂缝损伤进行三维体积计算之前，需要对提取出的点云进行插值添加点云密度，并进行下采样和体素化来模拟点云的几何形态。Before using the 3D Convex hull convex hull algorithm to calculate the three-dimensional volume of fracture damage, it is necessary to interpolate the extracted point cloud to add point cloud density, and perform downsampling and voxelization to simulate the geometry of the point cloud.
由于在双目扫描过程中，有些区域的点云没有扫描到或者被噪音污染，去噪后删除了一些噪点，缺少足够的点有可能造成后期3D convex hull计算体积的不准确，所以用插值法在相邻两个点之间进行插值添加点云；During the binocular scanning process, the point cloud in some areas is not scanned or is polluted by noise. After denoising, some noise points are deleted. The lack of enough points may cause inaccurate calculation of the volume of the 3D convex hull in the later stage, so the interpolation method is used. Interpolate between two adjacent points to add a point cloud;
相比于投影或体素分割的方法处理点云，直接对点云进行特征提取能较好地保留三维结构信息，但由于点云的无序性，直接处理的方式在对邻域进行搜索时需要较高的计算成本。因此本发明对点云进行下采样，将对全部点云的操作转换到下采样所得到的关键点上，降低计算量；具体是采用FPS(farthest point sampling)构造一个三维体素栅格，然后在每个体素内用体素内的所有点的重心近似显示体素中的其他点，如此，体素内所有点就用一个重心点来表示，从而达到滤波的效果，大大的减少了数据量。Compared with the method of projection or voxel segmentation to process point clouds, direct feature extraction of point clouds can better preserve the three-dimensional structure information, but due to the disorder of point clouds, the direct processing method is difficult to search for neighborhoods. High computational cost is required. Therefore, the present invention down-samples the point cloud, converts the operation of the entire point cloud to the key points obtained by down-sampling, and reduces the amount of calculation; specifically, a three-dimensional voxel grid is constructed using FPS (farthest point sampling), and then In each voxel, the center of gravity of all points in the voxel is used to approximate other points in the voxel. In this way, all points in the voxel are represented by a center of gravity, so as to achieve the effect of filtering and greatly reduce the amount of data .
因为下采样是通过构造一个三维体素栅格，所以点云被转换成了三维体素栅格，在三维空间中，点云数据中的每个点都存在一个体素与之对应，只需要为每个点附上体素标签就能够实现点云数据的体素化，本发明通过用空间均匀大小的体素网格(voxel grid)来模拟点云的几何形态。Because downsampling is by constructing a 3D voxel grid, the point cloud is converted into a 3D voxel grid. In 3D space, each point in the point cloud data has a voxel corresponding to it. You only need Attaching a voxel label to each point can realize the voxelization of point cloud data. The present invention simulates the geometry of point cloud by using a voxel grid with a uniform size in space.
利用3D Convex hull凸包算法对裂缝损伤的点云几何形态进行计算，从而得到裂缝损伤的三维体积，其计算公式如下：Using the 3D Convex hull convex hull algorithm to calculate the point cloud geometry of the crack damage, so as to obtain the three-dimensional volume of the crack damage, the calculation formula is as follows:
式中，P表示实向量空间中的点集；α表示凸边的边数；n代表点的个数。In the formula, P represents the point set in the real vector space; α represents the number of sides of the convex edge; n represents the number of points.
本发明所述基于双目相机的三维房屋损伤模型构建测量方法，其利用双目相机成像视差原理，计算得到房屋的深度图，进而得到房屋结构的各个位置及各个构件的实际尺寸信息，实现对房屋结构进行非接触式高精度三维测量和无损检测，且检测效率高；同时将裂缝损伤自双目相机中左目相机的房屋室内平面图像中分割出来，并自2D视图投影到3D视图上，利用3D Convex hull凸包算法计算得到裂缝损伤的三维体积，即可直接获取房屋损伤的实际数据，便于城市大规模建筑群灾后快速检测和安全评估。The binocular camera-based three-dimensional building damage model construction measurement method of the present invention uses the imaging parallax principle of the binocular camera to calculate the depth map of the house, and then obtains the actual size information of each position of the house structure and each component, so as to realize Non-contact high-precision three-dimensional measurement and non-destructive testing of the building structure with high detection efficiency; at the same time, the crack damage is segmented from the indoor planar image of the house in the left eye camera of the binocular camera, and projected from the 2D view to the 3D view. The 3D convex hull algorithm calculates the three-dimensional volume of the crack damage, and the actual data of the house damage can be obtained directly, which is convenient for rapid detection and safety assessment of large-scale urban buildings after disasters.
如图2所示，本发明实施例还公开了一种基于双目相机的三维房屋损伤模型构建测量系统，其包括如下功能模块：As shown in Figure 2, the embodiment of the present invention also discloses a three-dimensional building damage model construction measurement system based on a binocular camera, which includes the following functional modules:
图像采集计算模块10，用于通过双目相机扫描得到房屋室内平面图像，并根据双目立体匹配算法计算得到深度图像；The image acquisition calculation module 10 is used to obtain the indoor planar image of the house by scanning the binocular camera, and calculate the depth image according to the binocular stereo matching algorithm;
裂缝损伤分割模块20，用于采用损伤识别分割算法将裂缝损伤自双目相机中左目相机的房屋室内平面图像中分割出来；The crack damage segmentation module 20 is used to segment the crack damage from the indoor plane image of the left eye camera in the binocular camera by using the damage recognition segmentation algorithm;
深度投影模块30，用于采用3D损伤精细化量化算法将分割得到的像素点投影到深度图上，即可获取到裂缝损伤中每个点的三维信息；The depth projection module 30 is used to project the pixel points obtained by segmentation onto the depth map by using the 3D damage refinement quantization algorithm, so as to obtain the three-dimensional information of each point in the crack damage;
损伤体积计算模块40，用于利用3D Convex hull凸包算法对裂缝损伤进行三维体积计算。The damage volume calculation module 40 is used to calculate the three-dimensional volume of fracture damage by using the 3D Convex hull algorithm.
本实施例一种基于双目相机的三维房屋损伤模型构建测量系统的执行方式与上述基于双目相机的三维房屋损伤模型构建测量方法基本相同，故不作详细赘述。The execution method of the binocular camera-based three-dimensional building damage model construction measurement system in this embodiment is basically the same as the above-mentioned binocular camera-based three-dimensional house damage model construction measurement method, so details will not be described in detail.
本实施例服务器为提供计算服务的设备，通常指具有较高计算能力，通过网络提供给多个消费者使用的计算机。该实施例的服务器包括：存储器、处理器以及系统总线，所述存储器包括存储其上的可运行的程序，本领域技术人员可以理解，本实施例的终端设备结构并不构成对终端设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。The server in this embodiment is a device that provides computing services, and generally refers to a computer with relatively high computing capabilities that is provided to multiple consumers through the network. The server in this embodiment includes: a memory, a processor, and a system bus. The memory includes executable programs stored thereon. Those skilled in the art can understand that the structure of the terminal device in this embodiment does not constitute a limitation on the terminal device. , may include more or fewer components than shown, or combine certain components, or have different component arrangements.
存储器可用于存储软件程序以及模块，处理器通过运行存储在存储器的软件程序以及模块，从而执行终端的各种功能应用以及数据处理。存储器可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据终端的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。The memory can be used to store software programs and modules, and the processor executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playback function, an image playback function, etc.); The data created by the use (such as audio data, phone book, etc.) and so on. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.
在存储器上包含一种基于双目相机的三维房屋损伤模型构建测量方法的可运行程序，所述可运行程序可以被分割成一个或多个模块/单元，所述一个或多个模块/单元被存储在所述存储器中，并由处理器执行，以完成信息的获取及实现过程，所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序在所述服务器中的执行过程。例如，所述计算机程序可以被分割为图像采集计算模块、裂缝损伤分割模块、深度投影模块、损伤体积计算模块。An executable program of a three-dimensional building damage model construction measurement method based on a binocular camera is included in the memory, and the executable program can be divided into one or more modules/units, and the one or more modules/units are divided into Stored in the memory and executed by the processor to complete the information acquisition and implementation process, the one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used for An execution process of the computer program in the server is described. For example, the computer program can be divided into an image acquisition calculation module, a fracture damage segmentation module, a depth projection module, and a damage volume calculation module.
处理器是服务器的控制中心，利用各种接口和线路连接整个终端设备的各个部分，通过运行或执行存储在存储器内的软件程序和/或模块，以及调用存储在存储器内的数据，执行终端的各种功能和处理数据，从而对终端进行整体监控。可选的，处理器可包括一个或多个处理单元；优选的，处理器可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器中。The processor is the control center of the server. It uses various interfaces and lines to connect various parts of the entire terminal equipment. By running or executing software programs and/or modules stored in the memory, and calling data stored in the memory, the terminal executes Various functions and processing data, so as to carry out overall monitoring of the terminal. Optionally, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, application programs, etc., and the modem processing The device mainly handles wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor.
系统总线是用来连接计算机内部各功能部件，可以传送数据信息、地址信息、控制信息，其种类可以是例如PCI总线、ISA总线、VESA总线等。处理器的指令通过总线传递至存储器，存储器反馈数据给处理器，系统总线负责处理器与存储器之间的数据、指令交互。当然系统总线还可以接入其他设备，例如网络接口、显示设备等。The system bus is used to connect various functional components inside the computer, and can transmit data information, address information, and control information, and its type can be, for example, PCI bus, ISA bus, VESA bus, etc. The instructions of the processor are transmitted to the memory through the bus, and the memory feeds back data to the processor. The system bus is responsible for the data and instruction interaction between the processor and the memory. Of course, the system bus can also be connected to other devices, such as network interfaces, display devices, and the like.
所述服务器应至少包括CPU、芯片组、内存、磁盘系统等，其他构成部件在此不再赘述。The server should at least include a CPU, a chipset, a memory, a disk system, etc., and other components will not be repeated here.
在本发明实施例中，该终端所包括的处理器执行的可运行程序具体为：一种基于双目相机的三维房屋损伤模型构建测量方法，其包括如下步骤：In the embodiment of the present invention, the executable program executed by the processor included in the terminal is specifically: a method for constructing and measuring a three-dimensional house damage model based on a binocular camera, which includes the following steps:
通过双目相机扫描得到房屋室内平面图像，并根据双目立体匹配算法计算得到深度图像；The indoor plane image of the house is scanned by the binocular camera, and the depth image is calculated according to the binocular stereo matching algorithm;
采用损伤识别分割算法将裂缝损伤自双目相机中左目相机的房屋室内平面图像中分割出来；The damage recognition and segmentation algorithm is used to segment the crack damage from the indoor plane image of the left camera of the binocular camera;
采用3D损伤精细化量化算法将分割得到的像素点投影到深度图上，即可获取到裂缝损伤中每个点的三维信息；Using the 3D damage refinement quantification algorithm to project the segmented pixel points onto the depth map, the 3D information of each point in the crack damage can be obtained;
利用3D Convex hull凸包算法对裂缝损伤进行三维体积计算。Using 3D Convex hull convex hull algorithm to calculate the three-dimensional volume of fracture damage.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。In the above-mentioned embodiments, the descriptions of each embodiment have their own emphases, and for parts that are not detailed or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各实施例的模块、单元和/或方法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。Those skilled in the art can appreciate that the modules, units and/or method steps of the embodiments described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.
以上所述，以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still understand the foregoing The technical solutions recorded in each embodiment are modified, or some of the technical features are replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims (7)
Translated fromChinese
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110009707.9ACN112686877B (en) | 2021-01-05 | 2021-01-05 | Construction and measurement method and system of 3D house damage model based on binocular camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110009707.9ACN112686877B (en) | 2021-01-05 | 2021-01-05 | Construction and measurement method and system of 3D house damage model based on binocular camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112686877A CN112686877A (en) | 2021-04-20 |
| CN112686877Btrue CN112686877B (en) | 2022-11-11 |
Family
ID=75457398
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110009707.9AActiveCN112686877B (en) | 2021-01-05 | 2021-01-05 | Construction and measurement method and system of 3D house damage model based on binocular camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112686877B (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113469652A (en)* | 2021-06-30 | 2021-10-01 | 上海市建筑科学研究院有限公司 | Existing building damage recording system based on cloud platform |
| CN113870326B (en)* | 2021-09-01 | 2023-11-28 | 浙江大学 | Structural damage mapping, quantifying and visualizing method based on image and three-dimensional point cloud registration |
| CN114066834A (en)* | 2021-11-09 | 2022-02-18 | 上海建工四建集团有限公司 | Device and method for investigating damage of outer vertical surface of rural house |
| CN114140537A (en)* | 2021-12-01 | 2022-03-04 | 国网江苏省电力有限公司连云港供电分公司 | Vision-based pipeline detection method |
| CN114841923B (en)* | 2022-04-10 | 2025-03-21 | 同济大学 | A high-precision real-time crack detection method based on drones |
| CN114596313B (en)* | 2022-05-09 | 2022-10-04 | 深圳大学 | A method and related equipment for damage detection of building components based on indoor point cloud |
| CN115035184B (en)* | 2022-06-13 | 2024-05-28 | 浙江大学 | A method for estimating the volume of pomelo based on lateral multi-view reconstruction |
| CN115512055A (en)* | 2022-09-13 | 2022-12-23 | 武汉大学 | Method and device for performing indoor structure three-dimensional reconstruction based on two-dimensional video and computer equipment |
| CN116152697B (en)* | 2022-11-28 | 2025-09-09 | 湖南中登科技有限公司 | Three-dimensional model measuring method and related device for concrete structure cracks |
| CN115880243B (en)* | 2022-12-02 | 2023-07-07 | 广东机电职业技术学院 | Rail surface damage detection method, system and medium based on 3D point cloud segmentation |
| CN115597551B (en)* | 2022-12-14 | 2023-04-07 | 成都量芯集成科技有限公司 | Handheld laser-assisted binocular scanning device and method |
| CN116109954B (en)* | 2023-04-04 | 2023-07-04 | 深圳市城市公共安全技术研究院有限公司 | House potential safety hazard identification method, device and storage medium |
| CN118862506A (en)* | 2024-08-06 | 2024-10-29 | 兰州理工大学 | Multi-UAV collaborative post-earthquake building safety rapid assessment system and method |
| CN119131120B (en)* | 2024-11-08 | 2025-06-27 | 江南大学附属医院 | Burn area measurement method, burn area measurement system, electronic equipment and storage medium |
| CN119625435B (en)* | 2025-02-11 | 2025-05-13 | 安徽建筑大学 | Intelligent identification method and device for historical building diseases |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101147685A (en)* | 1999-11-18 | 2008-03-26 | 罗切斯特大学 | Apparatus and method for forming cone beam volume computed tomography breast images of a patient's breast |
| CN103983228A (en)* | 2014-06-04 | 2014-08-13 | 长江工程地球物理勘测武汉有限公司 | Method for measuring borehole fracture attitude by use of electronic rock core |
| CN104792804A (en)* | 2015-03-27 | 2015-07-22 | 江苏大学 | Method for diagnosing internal damage of cereal grains based on micro-CT (computed tomography) technology |
| CN106023226A (en)* | 2016-05-31 | 2016-10-12 | 彭博 | Crack automatic detection method based on three-dimensional virtual pavement |
| CN106791773A (en)* | 2016-12-30 | 2017-05-31 | 浙江工业大学 | A kind of novel view synthesis method based on depth image |
| CN107179322A (en)* | 2017-06-15 | 2017-09-19 | 长安大学 | A kind of bridge bottom crack detection method based on binocular vision |
| CN109242855A (en)* | 2018-07-19 | 2019-01-18 | 中国科学院自动化研究所 | Roof dividing method, system and equipment based on Three-dimensional Multi-resolution statistical information |
| CN110349251A (en)* | 2019-06-28 | 2019-10-18 | 深圳数位传媒科技有限公司 | A kind of three-dimensional rebuilding method and device based on binocular camera |
| CN111537610A (en)* | 2020-05-15 | 2020-08-14 | 华东理工大学 | A sensor array optimization method for damage localization of metal curved plates |
| CN112102491A (en)* | 2020-08-12 | 2020-12-18 | 西安交通大学 | Skin damage surface three-dimensional reconstruction method based on surface structured light |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016197006A1 (en)* | 2015-06-04 | 2016-12-08 | Eoplex Limited | Solid state battery and fabrication process therefor |
- 2021
- 2021-01-05CNCN202110009707.9Apatent/CN112686877B/enactiveActive
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101147685A (en)* | 1999-11-18 | 2008-03-26 | 罗切斯特大学 | Apparatus and method for forming cone beam volume computed tomography breast images of a patient's breast |
| CN103983228A (en)* | 2014-06-04 | 2014-08-13 | 长江工程地球物理勘测武汉有限公司 | Method for measuring borehole fracture attitude by use of electronic rock core |
| CN104792804A (en)* | 2015-03-27 | 2015-07-22 | 江苏大学 | Method for diagnosing internal damage of cereal grains based on micro-CT (computed tomography) technology |
| CN106023226A (en)* | 2016-05-31 | 2016-10-12 | 彭博 | Crack automatic detection method based on three-dimensional virtual pavement |
| CN106791773A (en)* | 2016-12-30 | 2017-05-31 | 浙江工业大学 | A kind of novel view synthesis method based on depth image |
| CN107179322A (en)* | 2017-06-15 | 2017-09-19 | 长安大学 | A kind of bridge bottom crack detection method based on binocular vision |
| CN109242855A (en)* | 2018-07-19 | 2019-01-18 | 中国科学院自动化研究所 | Roof dividing method, system and equipment based on Three-dimensional Multi-resolution statistical information |
| CN110349251A (en)* | 2019-06-28 | 2019-10-18 | 深圳数位传媒科技有限公司 | A kind of three-dimensional rebuilding method and device based on binocular camera |
| CN111537610A (en)* | 2020-05-15 | 2020-08-14 | 华东理工大学 | A sensor array optimization method for damage localization of metal curved plates |
| CN112102491A (en)* | 2020-08-12 | 2020-12-18 | 西安交通大学 | Skin damage surface three-dimensional reconstruction method based on surface structured light |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112686877A (en) | 2021-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112686877B (en) | Construction and measurement method and system of 3D house damage model based on binocular camera | |
| CN110568447B (en) | Visual positioning method, device and computer readable medium | |
| Golparvar-Fard et al. | Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques | |
| CN103607584B (en) | Real-time registration method for depth maps shot by kinect and video shot by color camera | |
| CN106033621B (en) | A kind of method and device of three-dimensional modeling | |
| KR101759798B1 (en) | Method, device and system for generating an indoor two dimensional plan view image | |
| CN113763478B (en) | Unmanned vehicle camera calibration method, device, equipment, storage medium and system | |
| US8155387B2 (en) | Method and system for position determination using image deformation | |
| CN103456038A (en) | Method for rebuilding three-dimensional scene of downhole environment | |
| CN102903101B (en) | Method for carrying out water-surface data acquisition and reconstruction by using multiple cameras | |
| CN102800127A (en) | Light stream optimization based three-dimensional reconstruction method and device | |
| WO2022088881A1 (en) | Method, apparatus and system for generating a three-dimensional model of a scene | |
| CN112880687A (en) | Indoor positioning method, device, equipment and computer readable storage medium | |
| WO2015179216A1 (en) | Orthogonal and collaborative disparity decomposition | |
| CN110766731A (en) | Method and device for automatically registering panoramic image and point cloud and storage medium | |
| CN116778094B (en) | A method and device for monitoring building deformation based on optimal viewing angle shooting | |
| CN112967345A (en) | External parameter calibration method, device and system of fisheye camera | |
| CN115147262B (en) | Image generation method and device | |
| CN113298708A (en) | Three-dimensional house type generation method, device and equipment | |
| CN113610702B (en) | Picture construction method and device, electronic equipment and storage medium | |
| CN113870365B (en) | Camera calibration method, device, equipment and storage medium | |
| CN113034347A (en) | Oblique photographic image processing method, device, processing equipment and storage medium | |
| WO2025039766A1 (en) | Real-time online cargo volume recognition method, system, and device, and medium | |
| CN116958218A (en) | A point cloud and image registration method and equipment based on calibration plate corner point alignment | |
| CN115830135A (en) | Image processing method, device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right | Effective date of registration:20250123 Address after:No.65 Chifeng Road, Yangpu District, Shanghai 200092 Patentee after:Tongji University Architectural Design and Research Institute (Group) Co.,Ltd. Country or region after:China Address before:200092 Siping Road 1239, Shanghai, Yangpu District Patentee before:TONGJI University Country or region before:China |