Background technology
Robot universalness needs to solve " how to get to " and " which is gone to " two large problems.In factory, professional and technical personnel has beenThrough can be very good to command industrial robot " how to get to " and " which is gone to ", therefore industrial robot is in factory's popularization；ButIt is for the civil areas such as household consumption, office particularly multicell house type service environment, due to target ground can not be looked at straightDomain therefore environment complexity is considerably beyond aerial and outdoor road occasion, and ordinary consumer does not possess professional machines people yetTechnology, therefore ordinary people and robot will be allowed freely to carry out map datum man-machine interaction in civilian service robot field is partHighly difficult thing.The patent document such as CN201510846232, CN201510690749, CN201510169963 illustrates machine at presentDevice people relies on the laser radar of itself equipment, the first-class equipment of depth camera, and to surrounding environment, autonomous structure point cloud and outline line are digitalUp-to-date robot automation's traveling technology such as map (numerical map described herein refers both to such) and Camera calibration, make " whyWalk " problem solved；But these prior arts also cannot solve " which is gone to " this problem, because " which is gone to " showsSo it is intended to be determined by people and robot is inputed to by people, but all robots now recording in above-mentioned up-to-date patent literary compositionThe link of mankind's input position data is not occurred, so above-mentioned prior art substantially cannot solve " to go in workflowWhich " such man-machine interaction problem is it is impossible to meet the need that the free robot command of ordinary people is engaged in service in multicell house type environmentWill.At present, the complex environment position data interaction technique of Human-to-Machine people belongs to the professional skill that professional and technical personnel could graspArt, also cannot spread in colony of domestic consumer, this is to sit because amateur domestic consumer is difficult to understand those abstract mapsMark data, outline line numerical map, point cloud numerical map etc., are also ignorant of how map datum being inputed to machine by reference formatThe such professional knowledge of device people, let alone allow domestic consumer oneself edit the numerical map of oneself house type.Therefore existing skillArt cannot solve the problems, such as to make each ordinary people convenient and robot carries out map datum man-machine interaction, so people are noMethod goes service implementation according to certain specified place in needs free robot command to certain room of oneself, also cannot be directly viewedRobot in which particular location in which room, this just seriously limit the indoor application of service robot and market popularize intoJourney.
In order to solve the above problems, present invention proposition is a kind of to carry out map datum man-machine interaction with robot multi-view imageMethod, make layman also can directly to robot Freely input map datum and understand its position be located, thus allowing generalLogical people easily can carry out map datum with robot and interact, and meet the need using robot in multicell house type complex environmentWill.
The present invention describes a kind of method carrying out map datum man-machine interaction with robot multi-view image, and concrete steps are such asUnder：
Step S1, robot obtain fundamental digital map；
Step S2, robot shoot current region real scene image by the image capturing device itself installed, and synchronization is adoptedThe zone digit map of collection current region, the model of the zone digit ground map combining of the scope that this real scene image covers and synchronous acquisitionEnclose least partially overlapped；
Step S3, intercepted this zone digit map portion overlapping with both real scene images visual field of sync pulse jamming by computerPoint, and make to be mutually matched both this；
Step S4, by computer, the zone digit map matching and real scene image are synthesized with map location dataOutdoor scene positioning image being stored in database；
Step S5, by database outdoor scene positioning image be shown on screen, user click directly on outdoor scene positioning imageOn certain point, machine solution separates out the map datum of this point, can also show robot location and mobile trajectory data in the same mannerShow that the correspondence position point that outdoor scene positions on image is checked for user, thus realizing the man-machine interaction of map datum.
The invention has the beneficial effects as follows it is such：Due to prior art robot use itself equipment laser radar,The ground mapping sensor such as depth camera, ultrasonic radar can have been realized environment mapping, automatically build point cloud and profileThe abilities such as line fundamental digital map, autonomous positioning independent navigation, therefore proposed by the present invention carry out ground with robot multi-view imageThe method of diagram data man-machine interaction can also be shared these existing resources, can directly run in these existing hardware environmentsThis method the fundamental digital map of these existing sensor collections of direct access.Robot is allowed to utilize above-mentioned prior art firstObtain the fundamental digital map in house, and shoot current region real scene image synchronous acquisition using the image capturing device of itselfThen the zone digit collecting map and real scene image are synthesized by the zone digit map of current regional area in a computerThe outdoor scene having map datum for a subband positions image, and by this, each room is all carried out with outdoor scene positioning image acquisition and processing.Due to including the fundamental digital map in multiple rooms originally by many secondary areas of the sensors such as above-mentioned existing laser radar mappingDomain numerical map split builds, and any one location point therefore in above-mentioned zone numerical map can be with fundamental digital groundThe relevant position point of in figure forms one-to-one mapping relation that is to say, that can be accurately positioned in fundamental digital map, instituteAny point in image is positioned with the outdoor scene in each room and can be mapped to fundamental digital map according to itself map datumCorresponding position.So when user is when clicking on outdoor scene and positioning any one point on image in screen, just it is equivalent to taking outThe fundamental digital map of elephant carries out position data input, but at this moment seen by person be oneself the real scene shooting figure being easily understood mostAs naturally not having any difficulty.For example, as long as clicking on a particular location in the outdoor scene positioning image in certain room for the peoplePoint, computer system just can parse this concrete map datum in fundamental digital map, thus realizing to robotMap datum input, robot just can according to this map datum sail for designated room this specified place implement takeBusiness；In the same manner, position data in moving process for the robot can also be shown to the corresponding position that outdoor scene positions image, makes oneDirectly can see particular location in outdoor scene ambient image for the robot, so whole interaction be not required to user and possessThe professional ability of interpretation fundamental digital map.It can be seen that, the present invention solve first ordinary people fail to understand abstract numerical map,It is ignorant of the difficult problems such as coordinate knowledge, even if ordinary people includes children under multicell house type complex environment, old man can easily and machineDevice people carries out map datum interaction；And method described in the invention can the installations and facilities outside completely good robot be propped upThe prior art parts such as image capturing device, ground mapping sensor and the embedded computer hold, only being carried using robot, so not only reduce cost of goods manufactured, Ye Shi robot buys use of can directly starting shooting back home, be in without userDispose any matching running environment, be extremely suitable for average family application.
Specific embodiment
Understandable for enabling the above-mentioned purpose of the present invention, feature, advantage to become apparent from, below in conjunction with the accompanying drawings to the present invention'sSpecific embodiment elaborates.A lot of details will be illustrated in the following description it is therefore intended that helping this area skillArt personnel are more fully understood by the present invention, but it is clear that those skilled in the art can be without prejudice on the premise of intension of the present inventionTo be changed with more alternate manners different from the present embodiment specific descriptions and to implement, therefore the present invention is not subject to following public toolThe restriction of body embodiment.
Accompanying drawing 1 is a kind of stream of method carrying out map datum man-machine interaction with robot multi-view image of the present inventionCheng Tu.
The present embodiment methods described can allow layman directly conveniently enter destination locations data to robot,The current outdoor scene position of robot can also intuitively be shown for layman, thus realizing layman and robotMap datum man-machine interaction, specifically includes following steps：
Step S1, robot obtain fundamental digital map.
This step mainly improves data acquisition quality for step S2 and efficiency lays the foundation.At present, robot obtains baseThe method of plinth numerical map is a lot, including the ground mapping sensor installed using itself independently build this fundamental digital map,Can also be issued by host computer, the mode such as resource-sharing obtains this fundamental digital map.Wherein, because prior art has hadMultiple commercially available ground mapping sensors supply robot to select, such as laser radar, binocular depth camera, structure light mapping systemSystem, ultrasonic radar etc., therefore existing robot has been provided with surveying and drawing, to house internal autonomy, the energy building fundamental digital mapPower, so for the robot being mounted with these existing ground mapping sensors, as long as just permissible after the start of this robotStart automatically scanning surrounding enviroment and build fundamental digital map.If robot can move traveling, can be made it in the whole seriesTravel one time in room, the ground mapping sensor of its own just can construct the fundamental digital map in a whole set of room automatically.Certainly, robot can also be issued by control centre or other host computer and wait channel to obtain this fundamental digital map or by mutualNetworking and the resource-sharing of group of robot obtain, and these prior arts much will not enumerate volume.Obtain this fundamental digital groundAfter figure, robot can plan optimal data collection travel route and optimal data collection direction etc., be next step expeditiouslyObtain optimum data to lay the foundation.This step S1 can once complete it is also possible to random device people is arrived it in usePlace gradually improves until covering a whole set of room.If being used in the method takes on thing manipulator, what this step S1 obtained is exactlyThe fundamental digital map of this robot work environment, reason is identical.
Step S2, robot shoot current region real scene image by the image capturing device itself installed, and synchronization is adoptedThe zone digit map of collection current region, the model of the zone digit ground map combining of the scope that this real scene image covers and synchronous acquisitionEnclose least partially overlapped.
This step is to gather the real scene image of region within the jurisdiction and corresponding zone digit map.This step S2 was both permissibleWith the substep execution of step S1, can also mixed running in Embedded step S1.The method that wherein step S2 is executed with step S1 substepIt is：After the completion of step S1, again region within the jurisdiction is carried out once again according to the fundamental digital map of step S1 by robotData acquisition travels, and utilizes the image capturing device itself installed to shoot real scene image, simultaneously profit at the node of driving pathWith the current zone digit map of the ground mapping sensor synchronous scanning of itself installation, the area of this real scene image and synchronous acquisitionBoth domain numerical maps overlay area is least partially overlapped.Certainly this image capturing device can be common camera, shootingMachine etc. or 360 degree of panorama image shooting apparatus, when robot uses 360 degree of panorama image shooting apparatus and 360 degreeDuring the integrated mode gathered data of ground mapping sensor, the coverage of the real scene image of acquisition is with zone digit map360 degree of panoramas have also just been automatically obtained completely overlapped certainly, and the data processing that this will significantly reduce the subsequent steps such as step S3 is difficultSpend, significantly improve man-machine interaction experience.
Step S2 Embedded step S1 run method be：Appoint in the fundamental digital map structuring of robot execution step S1During business, gather operation in the real scene image of each path node Embedded step S2 of step S1 and zone digit map,The fundamental digital map that step S1 need not be waited is fully completed and starts step S2 again, and the advantage of so embedded execution is that of avoiding secondaryTravel, shortcoming is due to not having ready-made fundamental digital map to lead to not the collection path of Optimization Steps S2, such as robotA lot of duplicate paths can be produced being repeated to many room environments during exploration travels, thus lead to embedded executionStep S2 can produce the little zone digit map of more value and real scene image data, can increase at embedded computer dataThe difficulty of reason also increases hardware and software cost, and the later stage needs manually to delete those useless node datas, to useFamily also increases additional workload, if not manual delete, can affect user experience；Review step S1 to divide with step S2The mode of step execution, if robot has completed step S1, step S2 just can be planned in fundamental digital map in advanceOptimal data gathers driving path, does not naturally also just have duplicate paths to lead to the too many problem of hash, makes to collectThe quality of data of zone digit map and corresponding real scene image and availability is significantly high, data volume is greatly reduced, no longerUser is needed to do manual deletion work.Thus it is clear that the data acquisition quality implemented step by step of step S1 and step S2 is best but stepIn rapid S2 Embedded step S1, mixed running can make workflow simpler, and therefore those skilled in the art can be actual according to productWorking environment independently selects.If this method is used in taking in thing manipulator product, this step S2 is taking the shifting of thing manipulatorAlso work in the same manner when dynamic.
The method that the robot itself being fitted without ground mapping sensor for those realizes step 2 is such：ThisClass robot issues acquisition fundamental digital map or passes through other resource-sharing sides generally by the host computer of control centreFormula obtains, and obtains the location data of current location by location technologies such as UWB radio position finding radio directional bearing, vision positioning, bluetooth positioning.Thus by this current position determination data, host computer can be issued or the fundamental digital map of resource-sharing carries out local sectionIt is taken as needing the zone digit map of collection for step S2, and the real scene image of sync pulse jamming current location, equally can realizeThe purpose of step S2, also belongs to the description scope of step S2.
Step S3, intercepted this zone digit map portion overlapping with both real scene images visual field of sync pulse jamming by computerPoint, and make to be mutually matched both this.
This step is to generate outdoor scene by next step to position the pretreatment that image is done.Wherein, zone digit map and reality are madeScape images match is related to multiple projects and includes：Visual angle coupling, visual field coupling and lens distortion coupling etc. are a lot.Visual field is matedTypically exist because the ground mapping sensor of the types such as the laser radar that general robot adopts all has larger horizontal field of view180-360 degree, but the image capturing device that robot adopts only has 90-120 degree because cost limits mean level visual field, becauseThis needs to carry out intercepting just to make visual field covering area overlapping both this partly mutually identical, is typically all filled with image takingThe field range put is to intercept benchmark naturally it is also possible to flexibly grasp according to specific needs.The computer that this step S3 is related to cutsIt is all the very common prior art of field of Computer Graphics it is only necessary to according to robot itself figure that figure data such as processes at the technologyAs the lens parameters of filming apparatus and the visual field parameter of ground mapping sensor intercept, those skilled in the art all can implementHere repeat no more.Further need exist for carrying out visual angle coupling, because ground mapping sensor is generally mounted to lower position, imageThe general installation site of filming apparatus is higher, and therefore the two visual angle may differ greatly, the precision of impact position data interaction and makingWith effect, so it may also be desirable to for zone digit map to carry out view transformation, it is allowed to the real scene image visual angle with sync pulse jammingConsistent or meet position of related features design requirement.Computer view transformation algorithm belongs in computer graphics three-dimension varyingBasic rudimentary knowledge, belongs to known technology, will not be described here.The image capturing device that some robots itself are installed is due to adoptingIt is the equipment such as special machine vision, panoramic vision, such as Microsoft Kinect binocular camera sensor, IntelRealSense 3D camera sensing device, bumblebee2 binocular vision system, structure light mapping system, 360 degree of panoramic picturesFilming apparatus mate assembled scheme of 360 degree of ground mapping sensors etc., and they can be directly automatically complete inside equipment of itselfBecome real scene image shootings, zone digit ground mapping, visual angle visual field coupling etc. a series of activities, therefore at this moment fully equivalent in oneselfMove and perform step S2 and step S3.In addition, in actual product design, according to the difference of taking lens parameter, also may be usedCan need to carry out lens distortion coupling, this is because optical lens all can have more or less optical distortion, lead to real scene imageMultiple geometric distortions can be produced it is impossible to zone digit map accurate match, if therefore robot product to position interaction essenceDegree require higher it is necessary to real scene image being carried out with lens distortion reparation or adaptability tune being carried out to the data of zone digit mapWhole meet design accuracy and require.In a word, because the variance factor of different hardware equipment is too many, real scene image and zone digitMap match may also relate to much above-mentioned do not address project, need specifically to be mated in light of the circumstances in actual product designAdjust, make up to design accuracy and require.
Step S4, by computer, the zone digit map matching and real scene image are synthesized with map location dataOutdoor scene positioning image being stored in database.
The purpose of this step is to generate outdoor scene positioning image and storage.When robot ride or take thing manipulator moving processThe data of middle collection processes through step S3 and generates after the real scene image that is mutually matched and zone digit map it is necessary to mutualThe real scene image joined and zone digit map carry out computer picture synthesis process, make each location point in real scene image withThe map datum of zone digit map sets up man-to-man mapping relations, thus generates using real scene image as scene and carriesThe outdoor scene having map datum positions image；Certainly the map datum that this outdoor scene positions in image is usually sightless, but also may be usedWith with being mounted in scene image in the form of through data visualizations such as parallel, grid lines.This outdoor scene positions the processing procedure of imageCan also carry out on Internet Server on the local computer.The program making this outdoor scene positioning image belongs to meterCalculation machine technical field of geographic information very common figure layer process knowledge, those of ordinary skill in the art all can skillfully grasp no longer superfluousState.The all outdoor scene positioning images generating through computer disposal are required for being saved in database in case calling at any time.This numberCan be placed on locally it is also possible to be placed on Internet Server it is clear that the database on Internet Server can be more square according to storehouseJust realize resource-sharing with other users.
If obtaining the complete outdoor scene positioning image data base in a whole set of room.Robot needs complete in all roomsTravel one time, here travel during robot can continuous repeat step S1 to step S4, completely count after entirely covering institute roomJust put up according to storehouse.Certainly this process can once complete to implement step by step in use, be gradually completing, and also may be usedDirectly bring use so that existing ready-made outdoor scene is positioned by view data resource by resource-sharing, these are all considered as having been carried outThis step.
Step S5, by database outdoor scene positioning image be shown on screen, user click directly on outdoor scene positioning imageOn certain point, machine solution separates out the map datum of this point, can also show robot location and mobile trajectory data in the same mannerShow that the correspondence position point that outdoor scene positions on image is checked for user, thus realizing the man-machine interaction of map datum.
The purpose of this step is to carry out map datum man-machine interaction with robot visual angle.When user will be to robot input bitConfidence breath when, in database each room outdoor scene positioning image can uniformly be shown on screen it is also possible to by itself groundReason information position data is mounted to the corresponding position of the fundamental digital map in a whole set of room respectively, certainly can also be according to otherAligned transfer shows.User can directly find that the secondary outdoor scene comprising desired location on screen and position image, and clicks on thisSpecific location point in outdoor scene positioning image, computer system just can be parsed using the map datum that this outdoor scene positions imageThe map parameter of click location is simultaneously transferred to robot, and such ordinary people just can easily input in any room to robotThe concrete map datum of a bit.Robot can sail for after receiving this destination locations data designated room refer specifically to positionPut, the current location data of robot can also be with the common data visualization shape such as icon, track described point in the process of movingFormula is shown in that secondary outdoor scene positioning image of corresponding map datum, is so achieved that the outdoor scene of robot current location is anti-Feedback, this feedback system equally employs the real scene image form that ordinary people is easy to understand, and common people can understand.Display shouldThe screen that outdoor scene positions image can be the screen of robot itself or be connected with information networks such as internet, LANsThe mobile device screen connecing and server end screen, pcs terminal equipment screen etc..This is carried out with robot multi-view imageThe method of map datum man-machine interaction is a full imaging, and be independent of exterior technology equipment support can implement map numberAccording to man-machine interaction, it is the map datum interactive mode that the mankind are most easily understood by and use, be very suitable for not having robot and numeralThe ordinary people of map professional knowledge carries out the robot location's field of information interaction under many rooms complex environment it is also possible to be used for generalLogical people commander takes the occasions such as thing manipulator.
These are only presently preferred embodiments of the present invention and institute's application technology principle, its description is more concrete and detailed, but simultaneouslyCan not be therefore understands that being the restriction to the scope of the claims of the present invention it is clear that this method is applicable not only to many rooms Indoor Robot and takesThing manipulator etc., is also applied for the field that other has similar map data man-machine interaction demand.Therefore, general for this areaFor logical technical staff, without departing from the inventive concept of the premise, more shape changeable and improvement etc. can also be made, these all belong toIn protection scope of the present invention.