Intelligent patrol inspection management method based on ORB SLAMTechnical field
The present invention relates to a kind of patrol inspection management technology of FX, particularly develops one kind and is based on ORB SLAMThe patrol patrolling and checking management system of the vision positioning technology of algorithm.
Background technology
In the patrol patrol officer's administrative skill that there is currently, mainly by handhold GPS inspection device, enter by taskThe patrol of row fixed point.Portable equipment receives gps signal, while edits and releases patrol record according to application program, is sent out by GSM networkDeliver letters breath, host computer terminal receives the information that GPS inspection devices are sent, and is connected by cable with server, by inspection informationServer end is sent to, server end is connected by internet with computer again, and installation on computers shows software, will patrolAnti- routing information is shown.
Existing path orientation technology is mostly based on GPS device, and the dependence of signal and GSM network to GPS is bigger,Among actual environment, the high buildings and large mansions region that some gps signals are easily blocked, or some room areas are particularly, it is difficult toObtain accurately positioning so that very big deviation occurs in the routing information of patrol officer.
The content of the invention
The purpose of the present invention is to overcome above-mentioned the deficiencies in the prior art, designs a kind of intelligent patrol based on ORB SLAMInspection management method, some visual signature point information of current patrol inspection circuit are extracted by ORB SLAM algorithm, by baseHand can be passed through during patrol by being stored in information such as the point clouds of characteristic point of key frame into map file, patrol patrol officerThe image characteristic point and cartographic information of machine camera real-time image acquisition and utilization computer vision algorithms make extraction current environmentCharacteristic point is tracked matching to obtain the posture information of Current camera, so as to realize the positioning of current patrol inspection position.
The technical scheme is that provide a kind of intelligent patrol inspection management method based on ORB SLAM, methods describedIMAQ point using the image collecting device in mobile device terminal as monocular SLAM, patrol is established using ORB algorithms and patrolledExamine map；Patrol inspection is carried out using the image collecting device in mobile device terminal again, ultimately forms patrol inspection daily record.
Methods described specifically includes：
Step 1, using IMAQ point of the image collecting device in mobile device terminal as monocular SLAM, carry out mapInitialization, local map is formed, estimate video camera posture and calibrate initial three-dimensional point cloud；
Step 2, using motion model, key frame model or the reorientation initial map of model following, video camera posture is carried outOptimize and redefine three-dimensional point cloud；
Step 3, after tracking successfully, motion model is updated, updates the characteristic point between current key frame and other key framesWith relation, and new three-dimensional point cloud is generated using trigonometry；
Step 4, using optimization method in part and global adaptation pose, or using closed loop detection loop closure come excellentChange pose；
Step 5, then patrol inspection is carried out using the image collecting device in mobile device terminal, ultimately form patrol inspection dayWill；By the real-time positioning and the preservation and upload of trace information during patrol inspection.
Motion model, key frame model in the step 2 either reset bit model, key frame model or reorientationModel is specially：
Motion model is：The ORB features of present frame are extracted, Attitude estimation is carried out according to previous frame, or are carried out by overall situation weightPositioning initialization pose, the local map having built up is tracked, optimize pose, determine key frame；
Key frame model is：First, the bag of words BoW of present frame are calculated, and set initial pose as the position of previous frameAppearance；Secondly, characteristic matching is found according to pose and BoW dictionaries；Finally, the characteristic optimization pose of matching is utilized；
Verify the point map being newly generated and screened, then using local bundle collection adjustment Local BA, generate new mapPoint, finally the key frame of insertion is screened again, remove unnecessary key frame；
If motion model fails with key frame model following, the overall situation is relocated；Specific practice is：
First, the BoW vectors of present frame are calculated, secondly, some key frames are chosen alternately using BoW dictionaries；Again, findThere is the key frame of enough Feature Points Matchings；Finally, using Feature Points Matching iterative pose, if there is a certain keyFrame has enough interior points, then chooses the pose of the key frame optimization.
Described optimization method is：
The patrol inspection daily record is included comprising key frame, 3D point maps, altogether BoW vectors, view, Propagating Tree information.
The present invention compared with prior art, has following technique effect using above technical scheme：
The positioning precision that the intelligent patrol inspection management technology that the present invention realizes can be realized using computer vision algorithms make reachesBelow 1m, far above civilian GPS positioning precision, covering the weaker region of signal for GPS has good effect, while also onlyUsing common Android system mobile phone as terminal platform, the application in practical commercial scene is greatly reduced.
Brief description of the drawings
Fig. 1 is the implementing procedure figure of the intelligent patrol inspection management method of the invention based on ORB SLAM；
Fig. 2 is the intelligent patrol inspection management structure chart of the invention based on ORB SLAM.
Embodiment
The present invention provides a kind of intelligent patrol inspection management method based on ORB SLAM, to make the purpose of the present invention, skillArt scheme and effect are clearer, clear and definite, and referring to the drawings and give an actual example that the present invention is described in more detail.It should be understood thatSpecific implementation described herein is not intended to limit the present invention only to explain the present invention.
SLAM（Simultaneous Localization and Mapping）I.e. positioning simultaneously is with building figure.SLAM masterWork is to position and build figure.SLAM has many implementation methods at present, is broadly divided into laser radar and vision sensor twoDirection.Vision SLAM（Visual SLAM）The use depth of field including the use of monocular SLAM, binocular SLAM and using Kinect as representativeThe RGB-D SLAM of camera.
The ORB-SLAM that the present invention uses is a kind of three-dimensional localization based on ORB features and map structuring algorithm.ORB-SLAM is based on PTAM frameworks, adds map initialization and the function of closed loop detection, optimizes key frame extraction and map structuringMethod, all achieve good effect in processing speed, tracking effect and the accuracy of map, ORB-SLAM structures are a kind ofSparse map.
ORB-SLAM is the real-time monocular SLAM systems of a distinguished point based, in large-scale, small-scale, indoor roomOuter environment can be run.It is characterized in that all steps uniformly use the ORB features of image.ORB features are a kind of very fastThe feature extracting method of speed, has rotational invariance, and can go out scale invariability using pyramid construction.Use unificationORB features contribute to SLAM algorithms to have in feature extraction and the steps such as tracking, key frame extraction, three-dimensional reconstruction, closed loop detectionThe uniformity of interior life.
Mobile phone camera is utilized in the present invention, ORB SLAM key algorithm is realized in Android system, realizes oneThe vision positioning technology of the individual minimum system based on mobile phone, its main realization principle are made up of following components（See accompanying drawing 1）：
Data acquisition and map initialization：Use the forward sight camera of Android mobile phone, it is contemplated that at the requirement of algorithm and mobile phoneThe performance of device is managed, the resolution ratio for gathering image is fixed on 1280*720, sampling rate 10fps；Monocular SLAM maps are initialThe target of change is the initial three-dimensional point cloud of structure.Due to depth information can not be obtained from single frames, it is therefore desirable to from image sequenceImages more than two frames is chosen, video camera posture is estimated and reconstructs initial three-dimensional point cloud.
Tracking：This part of groundwork is that ORB features are extracted from the image of mobile phone camera collection, according to previous frameAttitude estimation is carried out, or relocated by the overall situation and initializes pose, reconstructed local map is then tracked, optimizesPose, new key frame is determined further according to some rules.
Build figure：This part is main to complete local map structure.Including the insertion to key frame, the ground being newly generated is verifiedFigure point is simultaneously screened, and then generates new point map, and collection adjustment is tied using local（Local BA）, finally again to the pass of insertionKey frame is screened, and removes unnecessary key frame.
Reorientation：When using constant motion model and when being tracked with key frame, match point is both less than the threshold value set,So explanation tracking failure is, it is necessary to which global reorientation could continue to track.
Closed loop detects：Among ORB SLAM algorithms, the pose for following the trail of to obtain all has error.With constantly prolonging for pathStretch, the error of previous frame can pass up to goes below, and causing error of the pose of last frame in world coordinate system to have canCan be very big.Except that in part and global adaptation pose, winding can also be utilized to detect using optimization method（loopclosure）To optimize pose.
Patrol inspection route positioning record：Had built up by the ORB SLAM algorithms of 1-5 steps and treat patrol inspection areaThe map of the circuit in domain, map file is stored among the storage card of mobile phone, when patrol patrol officer starts patrol inspection pipeAfter managing application software, forward sight camera brings into operation, and patrol patrol officer holds mobile phone along default patrol inspection route rowEnter, ORB SLAM algorithms bring into operation, and detect the image of collection front region in real time by forward sight camera, calculate current phaseThe posture information of machine and being recorded according to timestamp is put among file day, after a patrol inspection is completed, by journal fileUpload to patrol inspection management server software（It can be uploaded in real time by wifi or 4G networks in the case that network is good）,Complete the tracing process of a patrol inspection.
It is proposed by the present invention whole as scene using general Android system mobile phone using intelligent patrol inspection management technologyEnd, the image of patrol inspection route is gathered using the forward sight camera of mobile phone, patrol is built up by ORB SLAM algorithms in advance and patrolledThe three-dimensional point cloud map of inspection circuit is stored among mobile phone, the ORB features of extract real-time current image frame during patrol inspectionPoint goes out the positional information when preceding camera with map progress matching primitives（That is the position of patrol patrol officer）And upload to patrolAmong inspection management server.The technology can complete the positioning of inspection route independent of GPS, to the adaptability of environmentBy force, commercialized cost is very low.
Embodiment
The solution of the present invention, which includes, mainly includes two stages：First stage is advance according to specific patrol region of patrolling and examiningBuild up the ORB SLAM maps of patrol inspection circuit；Second stage is the real-time positioning and trace information during patrol inspectionPreserve and upload.The step of realizing in the two stages and embodiment is described in detail below：
1. patrol inspection route determines and hsrdware requirements
The patrol inspection management Technical comparing of the present invention is suitable for FX（Such as cell, campus, plant area, tubular service）Deng,Patrol inspection route is relatively fixed（Need to re-establish map if any change）.Choose the mobile phone of Android system（Four cores or more,2G internal memories, 16G space stored above, containing forward sight camera more than high definition）As IMAQ and the platform of algorithm.
2. that realizes ORB SLAM builds figure and positioning：
Because the input information of ORB SLAM algorithms is the image of mobile phone camera collection, it is ensured that build the appearance of camera during figureState is smaller with actually using central scene differentiation, and the height of handheld camera and angle are in less error model when ensureing to build figureIn enclosing.
Whole figure of building can be gone to implement with the process positioned according to following three aspects（Such as figure）：
Tracking：
SlAM map is initialized first, can not obtain three-dimensional information from single-frame images, it is necessary to choose image sequences more than two framesRow, estimate the posture of camera and rebuild initial three-dimensional information.
ORB-SLAM algorithm proposes a kind of model selection method based on statistics, and the first step extracts feature from imagePut and match, second step, to each model, normalize all characteristic points first.Then, in every step iteration, 1. according to spySign point is to calculating homography or fundamental matrix.Homography computational methods are normalizedDLT, fundamental matrix computational methods are the points of normalized 8.2. calculate each point pairSymmetric transfer errors, and the respective value of chi square distribution compare, and thus judge whether the point is interior point.It is accumulativeThe total score of interior point.3. comparing this score and historical scores, take best result and record relevant parameter.3rd step is according to certainCriterion preference pattern.Homography score is represented with SH, SF represents fundamental matrix score.If SH/ (SH + SF) >0.4, then selection homography, otherwise selection fundamental matrix.4th step is according to choosingThe model selected calculates pose.5th step Full bundle adjustment.Because requirements of the ORB-SLAM to initialization is higher,The scene of a feature rich can be selected when therefore initializing, mobile camera provides it with enough parallaxes.Further, sinceCoordinate system can be attached to the position of that successful two field picture of initialization, therefore initialization cannot be guaranteed in same position every time.
Tracing process mainly uses following three kinds of models：
Motion model（Tracking with motion model）：Assuming that camera is in uniform motion, then can use oneThe pose of frame and speed estimate the pose of present frame.The speed of previous frame can be calculated by the pose of above several frames.This model is applied to movement velocity and direction ratio is more consistent, in the case of rotation greatly,
Key frame（Tracking with reference key frame）：If motion model have failed, then firstIt can first attempt to do matching with a nearest key frame.The distance of present frame and a upper key frame is not also far after all.Bag of words can be make use of（BoW）To accelerate to match.First, the BoW of present frame is calculated, and sets initial pose to be upperThe pose of one frame；Secondly, characteristic matching is found according to pose and BoW dictionaries；Finally, the characteristic optimization pose of matching is utilized.
Reorientation（Relocalization）：If the matching of present frame and arest neighbors key frame also have failed, it is meant thatNow present frame has been lost, and can not determine its actual position.Now, only go and all key frames match, can see findSuitable position.First, the Bow vectors of present frame are calculated.Secondly, some key frames are chosen alternately using BoW dictionaries；AgainIt is secondary, find the key frame for there are enough Feature Points Matchings；Finally, Feature Points Matching iterative pose is utilized（RANSAC framesUnder frame, because relative pose may be bigger, point not in the know can be relatively more）.If key frame has enough interior points, then choosingTake the pose of the key frame optimization.
The main thought of pose refinement part be in present frame and（It is local）Corresponding pass as much as possible is found between mapSystem, to optimize the pose of present frame.
, it is necessary to update motion model after following the trail of successfully, and judge whether present frame is new key frame.IfIt is to be added into and update local map（local map）, the annexation of current key frame and other key frames is established, moreThe newly Feature Points Matching relation between current key frame and other key frames, and new three-dimensional point is generated using trigonometry, finallyDo a local optimum（Local BA, including adjacent key frame and three-dimensional point corresponding to them.
Because camera calibration and the precision of tracking are inadequate.The error of camera calibration can embody in the reconstruction（Such as threeWhen horn cupping is rebuild）, and the error followed the trail of can be then embodied in the pose between different key frames.The continuous accumulation of error can causeThe pose of subsequent frames is more and more remote from attained pose, eventually limits the precision of system positioning.If have between two frames enoughCorresponding points, then can both directly obtain the pose between two frames（As in initialization）, can also be excellent by solving oneChange problem obtains（Such as solvePnP）.Due to the uncertainty of monocular mesoscale, the error of yardstick can be also introduced.Due toThe always relative pose that tracking is obtained, before a certain frame error can pass up to below go, cause tracking to arriveLast position and attitude error is possible to very big., can be with 1. in part and global optimization pose in order to improve tracking precision；2. detected using closed loop（loop closure）To optimize pose.The details specifically optimized is detailed not in the range of this patent hereinState.
The preservation of patrol inspection map and the upload of information：
The cartographic information of patrol region of patrolling and examining is generated by the process of step 2, it is vectorial, common comprising key frame, 3D point maps, BoWThe information such as view, Propagating Tree.After patrol patrol officer is starting patrol inspection management software on mobile phone, forward sight camera is certainlyIt is dynamic to open the process for starting tracing and positioning, it should be noted that patrol patrol officer needs during along route patrol inspectionWill as far as possible keep mobile phone height and angle with building figure when posture otherness it is not too big, can so ensure position precisionWith stability in good controllable scope etc.The positional information of patrol patrol officer carries out real-time with current mapMatch somebody with somebody, and preserve current position and temporal information among mobile phone, it is real when thering are wifi signals to cover or 4G networks are goodWhen upload among patrol inspection management server system, current patrol can be shown in real time among terminal monitoring roomPatrolled and examined track map.