Handling system and its control method, floor tile paving systemTechnical field
The present invention relates to robot fields, in particular to a kind of handling system and its control method, floor tile paving systemSystem.
Background technique
Patch block has wide application in building field, and the work of patch block at present is usually by manually carrying out, artificial surveyAmount is usually inaccurate, and construction speed is slower, therefore in this field of patch block, has been gradually introduced patch block robot.
Current patch block robot is during paving floor tile, from fortune brick AGV (Automated GuidedVehicle, active guide transport lorry) crawl floor tile, it is transported to paving position, carries out alignment with reference to progress floor tile paving after brick.ButSince AGV positioning accuracy is limited (error 20mm), and the paving of floor tile is to precision requirement with higher, therefore which is difficult toMeet requirement of the paving to precision of floor tile.
Aiming at the problem that robot for being used for patch block in the prior art is unable to satisfy the required precision of patch block, not yet mention at presentEffective solution scheme out.
Summary of the invention
The embodiment of the invention provides a kind of handling system and its control methods, floor tile paving system, existing at least to solveThere is the technical issues of robot in technology for patch block is unable to satisfy the required precision of patch block.
According to an aspect of an embodiment of the present invention, a kind of control method of handling system is provided, handling system includes:First image collecting device, the second image collecting device, third image collecting device and robot, the control method of handling systemIt include: the first image prediction crawl position that robot is acquired according to the first image collecting device in capture area, and according to firstImage collecting device places position in the second image prediction that the first placement region acquires；Robot controls the second image collecting deviceThe third image of the second placement region is acquired, and controls the 4th image of third image acquisition device third placement region,Wherein, the second placement region and third placement region include two different feature locations of placing objects；Robot obtainsAccording in third image fisrt feature position and the 4th image in the first offset for determining of second feature position, wherein theOne offset is used to indicate the deviation of the placement position of practical placement position and prediction；Prediction is put according to the first offset in robotSet is adjusted.
Further, the first image and the first image collector that the first image collecting device is acquired in capture area are obtainedSet the second image in the acquisition of the first placement region；Obtain according in the first image third feature position and preset crawl spyIt levies the second offset that position determines, and according to the fourth feature position in the second image and preset places the of feature locationsThree offsets；The crawl position according to crawl feature locations and the prediction of the second offset is obtained, and is obtained according to placement base positionWith the placement position of third offset prediction, wherein place base position and obtained according to feature locations are placed.
Further, it is grabbed according to the first image collecting device in the first image prediction that capture area acquires in robotPosition, and according to the first image collecting device after the second image prediction that the first placement region acquires places position, robot controlThe crawl position that mechanical arm processed is moved to prediction grabs object to be handled；Robot control mechanical arm carries object to be handled and is moved toThe placement position of prediction.
Further, the first offset is compared with preset value for robot；If the first offset is greater than preset value,Then increase by the first offset on the basis of third offset and obtains the 4th offset；According to putting for the 4th offset adjustment predictionSet.
Further, robot control mechanical arm is moved to capture area, and sends first to the first image collecting deviceImage capture instruction, wherein the first image collecting device acquires the first image according to the first image capture instruction；Robot controlMechanical arm is moved to the first placement region, and sends the second image capture instruction to the first image collecting device, wherein the first figureAs acquisition device acquires the second image according to the second image capture instruction.
Further, the first image and the first Image Acquisition that the first image collecting device is acquired in capture area are being obtainedDevice obtains crawl feature locations and places feature locations, wherein obtain before the second image that the first placement region acquiresCrawl feature locations and the step of placing feature locations include: control manipulator motion to preset initial capture area, and are led toCross initial first image of the first image acquisition device；The fifth feature position in initial first image is extracted, is grabbedFeature locations；Manipulator motion is controlled to preset initial placement region, and passes through the first image acquisition device initial theTwo images；The sixth feature position in preliminary second image is extracted, obtains placing feature locations.
Further, object to be handled is to paving floor tile, and fisrt feature position is in the first image to paving floor tilePosition where brick angle, second feature position are the position where the brick angle of paving floor tile in the second image.
According to an aspect of an embodiment of the present invention, a kind of handling system is provided, comprising: the first image collecting device,For acquiring the first image in capture area, the second image is acquired in the first placement region, and grab according to the first image predictionPosition is placed according to the second image prediction in position；Second image collecting device, for acquiring the third image of the second placement region, andDetermine the fisrt feature position in third image；Third image collecting device, for acquiring the 4th image of third placement region,And determine the second feature position in the 4th image, wherein the second placement region and third placement region include placing objectsTwo different feature locations；Robot, for obtaining determine according to fisrt feature position and second feature position firstOffset, and position is placed to prediction according to the first offset and is adjusted.
Further, the first image collecting device is also used to be grabbed according to the third feature position in the first image with presetIt takes feature locations to determine the second offset, is determined according to the fourth feature position in the second image with preset placement feature locationsThird offset, and according to crawl feature locations and the second offset prediction crawl position, it is inclined according to placement base position and thirdPosition is placed in the prediction of shifting amount, wherein is placed base position and is obtained according to feature locations are placed.
Further, robot further includes mechanical arm, and robot is moved to prediction crawl position crawl for controlling mechanical armObject to be handled, and control mechanical arm and be moved to prediction placement position according to object to be handled is carried.
Further, robot further includes mechanical arm, and the first image collecting device is set to the end of mechanical arm.
Further, robot includes strut, and the second image collecting device and third image collecting device are set to strutEnd.
Further, object to be handled is to paving floor tile, and fisrt feature position is in the first image to paving floor tilePosition where brick angle, second feature position are the position where the brick angle of paving floor tile in the second image.
Further, robot is also used to for the first offset being compared with preset value, if the first offset is greater thanPreset value then increases by the first offset on the basis of third offset and obtains the 4th offset, and according to the 4th offset tuneThe placement position of whole prediction.
According to an aspect of an embodiment of the present invention, a kind of floor tile paving system is provided, including above-mentioned handling system.
According to an aspect of an embodiment of the present invention, a kind of control method of handling system is provided, comprising: according to crawlFirst image prediction of region acquisition grabs position, and is predicted to place position according to the first placement region；Obtain the second placement region4th image of third image and third placement region, wherein the second placement region and third placement region include having placed pairTwo different feature locations of elephant；According in third image fisrt feature position and the 4th image in second feature position it is trueThe first fixed offset, wherein the first offset is used to indicate the deviation of the placement position of practical placement position and prediction；According to firstOffset is adjusted the placement position of prediction.
In embodiments of the present invention, it is grabbed according to the first image collecting device in the first image prediction that capture area acquiresPosition, and position is placed in the second image prediction that the first placement region acquires according to the first image collecting device；Control the second imageAcquisition device acquires the third image of the second placement region, and controls the of third image acquisition device third placement regionFour images, wherein the second placement region and third placement region include two different feature locations of placing objects；It obtainsAccording in third image fisrt feature position and the 4th image in the first offset for determining of second feature position, wherein theOne offset is used to indicate the deviation of the placement position of practical placement position and prediction；And according to the first offset to the placement position of predictionIt is adjusted.Above scheme is based on the first image collecting device prediction crawl position and places position, and is based on the second image collectorSet with third image collecting device, rectify a deviation to the placement position of prediction, thus achieved the purpose that promoted paving precision.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present invention, constitutes part of this application, this hairBright illustrative embodiments and their description are used to explain the present invention, and are not constituted improper limitations of the present invention.In the accompanying drawings:
Fig. 1 is the flow chart according to the control method of the handling system of the embodiment of the present application；
Fig. 2 is the schematic diagram according to one kind second placement region and third placement region of the embodiment of the present application；
Fig. 3 is the schematic diagram to be worked according to a kind of handling system of the embodiment of the present application；
Fig. 4 is the control flow chart according to a kind of handling system of the embodiment of the present application；
Fig. 5 is the schematic diagram according to a kind of handling system of the embodiment of the present application；
Fig. 6 is according to the schematic diagram demarcated in the embodiment of the present application to image collecting device；
Fig. 7 is the framework map according to a kind of handling system of the embodiment of the present application；
Fig. 8 is the schematic diagram according to the data communication module of image collecting device in the embodiment of the present application；And
Fig. 9 is the flow chart according to the control method of another handling system of the embodiment of the present application.
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present inventionAttached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is onlyThe embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill peopleThe model that the present invention protects all should belong in member's every other embodiment obtained without making creative workIt encloses.
It should be noted that description and claims of this specification and term " first " in above-mentioned attached drawing, "Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this wayData be interchangeable under appropriate circumstances, so as to the embodiment of the present invention described herein can in addition to illustrating herein orSequence other than those of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that coverCover it is non-exclusive include, for example, the process, method, system, product or equipment for containing a series of steps or units are not necessarily limited toStep or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, productOr other step or units that equipment is intrinsic.
Embodiment 1
According to embodiments of the present invention, a kind of embodiment of the control method of handling system is provided, it should be noted thatThe step of process of attached drawing illustrates can execute in a computer system such as a set of computer executable instructions, also,It, in some cases, can be to be different from shown in sequence execution herein although logical order is shown in flow chartsThe step of out or describing.
Fig. 1 is according to the flow chart of the control method of the handling system of the embodiment of the present application, and handling system includes: the first figureAs acquisition device, the second image collecting device, third image collecting device and robot, as shown in Figure 1, this method includes as followsStep:
Step S102, robot are grabbed according to the first image collecting device in the first image prediction that capture area acquiresPosition, and position is placed in the second image prediction that the first placement region acquires according to the first image collecting device.
Specifically, above-mentioned image collecting device all can be smart camera, wherein the first image collecting device can be to setSet the smart camera in mechanical arm tail end.Robot may include strut, the second image collecting device and third image collectorIt installs and is placed in strut end.
In an alternative embodiment, by taking handling system carries out the paving of floor tile as an example, robot can control machineryArm is moved to placement region, first the second image of image acquisition device of mechanical arm tail end, and is calculated according to the second imageThe characteristic point for the reference brick placed in the second image out and the offset for placing feature locations, are transmitted to control for the offsetDevice, controller can be predicted according to the offset places position.Robot controls mechanical arm again and is moved to capture area, mechanical armFirst the first image of image acquisition device of end, and floor tile to be handled in the first image is calculated according to the first imageThe offset of characteristic point and crawl feature locations, is transmitted to controller for the offset, controller can be true according to the offsetMake crawl position.
Step S104, robot control the third image of second the second placement region of image acquisition device, and control4th image of third image acquisition device third placement region, wherein the second placement region and third placement region packetInclude two different feature locations of placing objects.
Specifically, strut has can be set in robot, above-mentioned second image collecting device and third image collecting device canTo be arranged in strut end, for acquiring the image of different placement regions, i.e., above-mentioned third image and the 4th image.It is above-mentioned to have putThe coordinate for setting at least two characteristic points of the two different feature locations of object for indicating placing objects, for example, withFor placing objects is the floor tiles of paving, two different feature locations can be two brick angles of the floor tile of paving.
It should be noted that above-mentioned second placement region is different from third placement region, but the first placement region can be withSecond placement region is identical, or identical as third placement region.
Fig. 2 is according to the schematic diagram of one kind second placement region and third placement region of the embodiment of the present application, in conjunction with Fig. 2It is shown, it is the floor tile of paving with reference to brick.By the first image and the second image prediction of the first image acquisition deviceIt obtains after placing position, mechanical arm will be moved to the surface of the placement position of prediction to paving floor tile, and (Fig. 2 is top view, to pavingGround floor tile and with reference to brick not in same plane), at this point, being taken pictures (the second rest area of area 2 by the second image acquisition deviceDomain) image, taken pictures the image of area 3 (third placement region) by third image acquisition device, to realize to placementThe effect of region progress double vision open country observation.
It should also be noted that, there may be the visuals field for the first image collecting device in the larger-size situation of floor tileInfull problem, and may not be caused entirely due to the visual field accuracy decline (such as: a brick angle alignment of floor tile, but other brick anglesAnd unjustified), therefore there is putting for different characteristic position using the second image collecting device and the measurement of third image collecting deviceRegion is set, so that two on the adjacent edge of floor tile brick angle can be aligned, and then improves the precision of floor tile paving.
Step S106, robot obtain according in third image fisrt feature position and the 4th image in second feature positionSet the first determining offset, wherein the first offset is used to indicate the deviation of the placement position of practical placement position and prediction.
Specifically, above-mentioned first offset can be the offset of the feature locations in fisrt feature position and the second imageThe offset of feature locations in amount and/or second feature position and the second image, so as to indicate practical placements position in advanceThe deviation of survey placed between position.
It in an alternative embodiment, include fourth feature position in the second image, robot is according to fourth feature positionSet and the deviation of benchmark placement location come predict place position.It is got in the second image collecting device and third image collecting deviceAfter third image and the 4th image, fisrt feature position and second feature position that will be extracted from third image and the 4th imageIt is sent to controller, controller can be according to fourth feature position included in the second image and fisrt feature position and theTwo feature locations, operation obtain above-mentioned first offset.
More specifically the first offset is illustrated.It is illustrated by taking Fig. 2 as an example, is to take pictures in the first placement regionIn the case where area 2, the fourth feature position in the second image is the coordinate P1_ at brick angle of the paving floor tile in area 2 of taking picturesRef (1) can be obtained in the second image according to the size of the coordinate combination floor tile, operation with reference to brick angle of the brick in area 3 of taking picturesCoordinate P2_ref (1).
Above-mentioned third image can be third image collecting device in the image of the shooting of area 2 of taking pictures, and fisrt feature position isFor the brick angular coordinate P1_ref (2) for referring to brick in area 2 of taking pictures；4th image can be the 4th image collecting device in area 3 of taking picturesThe image of shooting, second feature position are the brick angular coordinate P2_ref (2) for taking pictures and referring to brick in area 3.Then host computer can be withAbove-mentioned first offset is obtained according to P1_ref (1), P2_ref (1), P2_ref (1) and P2_ref (2) operation, for example, theOne offset P=(A, B), wherein A=P1_ref (2)-P1_ref (1), B=P2_ref (2)-P2_ref (1).
Step S108, robot are adjusted according to placement position of first offset to prediction.
Since the first offset is used to indicate the deviation of the placement position of practical placement position and prediction, robot can rootAccording to the first offset, the placement position of prediction is adjusted, to obtain final placement position.Specific adjustment mode can be withIt is on the basis of the placement position of prediction, to increase above-mentioned first offset.
Fig. 3 is the schematic diagram to be worked according to a kind of handling system of the embodiment of the present application, as shown in connection with fig. 3, the first imageAcquisition device acquires the first image in area 2 of taking pictures, and taking pictures, area 1 acquires the second image, and according to the first image and the second imagePrediction obtains crawl position and places position.Robot is moved to raw material storage according to crawl position, after grabbing paving brick by sucker, will spreadPatch block is moved to the placement position predicted to paving area.Second image collecting device determines after area 2 of taking pictures acquires third imageThe fisrt feature position of brick is referred in third image, third image collecting device determines the after area 3 of taking pictures acquires the 4th imageThe second feature position of brick is referred in four images.Second image collecting device and third image collecting device are respectively by fisrt featurePosition and second feature position are uploaded to host computer, and host computer calculates the offset of fisrt feature position and second feature position(i.e. above-mentioned first offset), to be adjusted to position is placed.It should be noted that with reference to brick and to paving area adjacent bricksGap width should meet 2 ± 0.5mm.
It should be noted that since the first image collecting device only acquires a characteristic point of placement region, it is pre-There may be deviations for the placement position of survey, and object to be handled is grabbed in mechanical arm, and object to be handled is moved to putting for predictionDuring this of set, since the movements such as the sucker crawl of mechanical arm can also introduce certain error, therefore, it is difficult to floor tileCarry out accurate paving.Above-mentioned steps using the second image collecting device and third image acquisition device third image and4th image rectifies a deviation to the placement position of prediction, to eliminate the introduced error of a variety of causes, promotes paving precision.
It follows that the first image that the above embodiments of the present application are acquired according to the first image collecting device in capture areaPrediction crawl position, and position is placed in the second image prediction that the first placement region acquires according to the first image collecting device；ControlThe third image of second the second placement region of image acquisition device, and control the placement of third image acquisition device third4th image in region, wherein the second placement region and third placement region include two different features of placing objectsPosition；Obtain according in third image fisrt feature position and the 4th image in second feature position determine first offsetAmount, wherein the first offset is used to indicate the deviation of the placement position of practical placement position and prediction；And according to the first offset to pre-The placement position of survey is adjusted.Above scheme is based on the first image collecting device prediction crawl position and places position, and is based on secondImage collecting device and third image collecting device rectify a deviation to the placement position of prediction, to reach promotion paving precisionPurpose.
As a kind of optional embodiment, the first figure that robot is acquired according to the first image collecting device in capture areaPosition, packet are placed in the second image prediction that the first placement region acquires as prediction crawl position, and according to the first image collecting deviceIt includes: obtaining the first image and the first image collecting device that the first image collecting device is acquired in capture area in the first rest areaSecond image of domain acquisition；Obtain according in the first image third feature position and preset crawl feature locations determine theTwo offsets, and the third offset according to fourth feature position and preset placement feature locations in the second image；It obtainsAccording to the crawl position of crawl feature locations and the prediction of the second offset, and obtain pre- according to placement base position and third offsetThe placement position of survey, wherein place base position and obtained according to feature locations are placed.
Specifically, above-mentioned placement feature locations and crawl feature locations can be the position determined during teaching,It is also possible to the position artificially specified.By taking teaching as an example, during teaching, the movement of manual control robot and mechanical armMovement, so as to complete the handling process of an object, and then determined during this and to have placed feature locations and crawlFeature locations.Above-mentioned placement base position, which can be, to be aligned on the basis of placing feature locations and has distance to a declared goalThe position of (such as 2mm).
Above-mentioned third feature position can be the position of the characteristic point of the object to be handled in the first image, and this feature point canTo be the central point of object to be handled in the first image, the position at the angle of the object to be handled in the first image can also be.TheFour feature locations are identical as the setting of fisrt feature position, and details are not described herein again.
Above-mentioned second offset can be the difference of third feature position and preset crawl feature locations, third offsetIt can be the difference of fourth feature position and placement feature locations.
As shown in Fig. 2, above-mentioned object to be handled can be floor tile, in figure, taking brick area of taking pictures is current capture area, patchBrick area of taking pictures is current placement region.
In an alternative embodiment, third offset and crawl feature locations can be adjusted to the same seat firstIn mark system, then increase third offset on the basis of grabbing feature locations, crawl position can be obtained.It similarly, can be firstBy the 4th offset and to place benchmark be to adjust into the same coordinate system, then increases by the on the basis of placing base positionFour offsets can be obtained and place position.
As a kind of optional embodiment, first acquired according to the first image collecting device in capture area in robotImage prediction grabs position, and the second image prediction placement position acquire according to the first image collecting device in the first placement region itAfterwards, method further include: the crawl position that robot control mechanical arm is moved to prediction grabs object to be handled；Robot control is mechanicalArm carries the placement position that object to be handled is moved to prediction.
In above-mentioned steps, prediction obtain crawl position and place position after, can control mechanical arm be moved to crawl position grabAfter taking object to be handled, control mechanical arm, which is moved to, places position, places position so that object to be handled is moved to.
It should be noted that in above-mentioned steps, mechanical arm is carried after object to be handled is moved to the placement position of prediction, andDo not loosen sucker and treat moving object and placed, but needs according to the second image collecting device and third image collecting deviceIt is adjusted to position is placed.
As a kind of optional embodiment, robot is adjusted according to placement position of first offset to prediction, comprising:First offset is compared with preset value for robot；If the first offset is greater than preset value, in third offsetOn the basis of increase by the first offset and obtain the 4th offset；According to the placement position of the 4th offset adjustment prediction.
Specifically, above-mentioned preset value can be 0.5mm, and in above-mentioned steps, if the first offset is greater than preset value,Illustrate that deviation is larger to need to adjust, therefore increases by the first offset on the basis of third offset and obtain the 4th offset, rootAccording to the 4th offset and base position is placed, position is placed in prediction again.And if the first offset is less than or equal to preset value,Illustrate that deviation is smaller without adjustment.
After obtaining updated placement position according to adjustment prediction, the second image collecting device and third figure can also be controlledIt as acquisition device continues to acquire the image of the second placement region and third placement region, and repeats the above steps, until according to theThe offset that the image that two image collecting devices and third image collecting device continue acquisition determines is less than preset value.
Since above-mentioned adjustment process can execute repeatedly, subsequent process (such as process of vibrating) can also be overcome to be drawnThe error of the floor tile offset entered.
Fig. 4 is according to a kind of control flow chart of handling system of the embodiment of the present application, in conjunction with shown in Fig. 3 and Fig. 4, programWhen beginning, it is the world that user coordinates, which is first arranged, and tool coordinates are flange.In this embodiment, the first placement region and second is putSetting region is area 2 of taking pictures, and third placement region is area 2 of taking pictures, and the first image collector is set to the hand that mechanical arm tail end is arranged inEye camera, the second image collecting device and third image collector are set to fixed camera.
It is initialized first, the value for some registers that program is used is set to 0, such as: R [1], R [2], R [3], PR[1], [4] PR, PR [5], PR [6], PR [7] etc., wherein
R [1] is used to indicate to take pictures the image collecting device taken pictures in area 1 with the presence or absence of needs；
R [2] is used to indicate to take pictures the image collecting device taken pictures in area 2 with the presence or absence of needs；
R [3] is for indicating whether floor tile is to be moved to place the corresponding alignment position in position.
PR [1] includes translational movement and angle in the offset information for recording the offset information that each operation obtainsValue；
PR [4] is used to record in PR [1] according to the translational movement in take pictures area 2 and the determining offset in area 3 of taking pictures；
PR [5] is used to record in PR [1] according to the angle value in take pictures area 2 and the determining offset in area 3 of taking pictures；
PR [6] is used to record in PR [1] according to the translational movement for the determining offset in area 1 of taking pictures；
PR [7] is used to record in PR [1] according to the angle value for the determining offset in area 1 of taking pictures.
Mechanical arm is moved to the first placement region (area 2 of taking pictures), sets R [2]=1.PC control trick camera is taken pictures and is calculatedAfter offset, the value of R [2] is set to 0, at this moment can take from PR [1] and offset information.Due to translation be in user partiallyIt moves, rotation is to deviate in the tool, therefore the translation of PR [1] and spin data are respectively put into this 2 positions PR [4] and PR [5]In register.Mechanical arm is moved to capture area (area 1 of taking pictures), sets R [1]=1, and control trick camera is taken pictures.Etc. taking pictures and partiallyShifting amount calculate after the completion of, R [1] is set to 0, similarly, by PR [1] displacement and spin data be stored in PR [6] and PR [7] respectivelyIn.
Mechanical arm is moved to crawl position crawl floor tile, specifically, adding PR according to the crawl feature locations determined when teaching[6], crawl position can be obtained in the offset in PR [7].After movement in place, the air pressure of sucker is opened, draws floor tile.
Sucker grasps the alignment position (surface for placing position) that floor tile is moved to paving, specifically, determining according to when teachingPlacement base position add PR [4], the offset in PR [5] can be obtained and place position.Mobile [3]=1 postposition R in place, lead toIt crosses the part PC and sends photographing instruction to fixed double camera.Two fixed camera acquisition images, calculate offset completion at location featureAfterwards, R [3] is put back into 0.
Judge the offset value in PR [1], if each offset value is 0, represent deviation be less than setting value (translation <0.3mm, angle < 0.1 degree), paving alignment procedure terminates.Otherwise, the value in PR [1] is added up into PR [4], in PR [5].It moves againUpdated paving alignment position is moved, i.e., adds updated PR [4] according to the placement base position determined when teaching, PR [5]In offset.R [3]=1 are set, send photographing instruction to fixed double camera again, this adjustment process are repeated, until PR [1]In offset be all 0.When the alignment procedure on two sides of adjacent floor tile is completed, mobile mechanical arm to patch block position, position is being alignedThe underface of position.Sucker is closed, puts down floor tile, paving process terminates.
As a kind of optional embodiment, the first image that the first image collecting device is acquired in capture area and the are obtainedThe second image that one image collecting device is acquired in the first placement region, comprising: robot control mechanical arm is moved to catching blockDomain, and the first image capture instruction is sent to the first image collecting device, wherein the first image collecting device is according to the first imageAcquisition instructions acquire the first image；Robot control mechanical arm is moved to the first placement region, and to the first image collecting deviceSend the second image capture instruction, wherein the first image collecting device acquires the second image according to the second image capture instruction.
Fig. 2 is the schematic diagram to be worked according to a kind of handling system of the embodiment of the present application, as shown in connection with fig. 2, the first imageAcquisition device can be respectively moved to capture area and placement region under the drive of the mechanical arm of robot, and grab being moved toAfter taking region or placement region, specified signal is sent (for example, specific bit in the register of controller can be modified to controllerNumerical value), controller sends the instruction of Image Acquisition, the first image to the first image collecting device after receiving the specified signalAcquisition device acquires the first image or the second image after receiving the instruction of Image Acquisition.
As a kind of optional embodiment, the first image for obtaining the first image collecting device and acquire in capture area withFirst image collecting device is before the second image that the first placement region acquires, method further include: obtains crawl feature locationsWith placement feature locations, wherein the step of obtaining crawl feature locations and placement feature locations includes: to control manipulator motion extremelyPreset initial capture area, and pass through initial first image of the first image acquisition device；It extracts in initial first imageFifth feature position, obtain crawl feature locations；Manipulator motion is controlled to preset initial placement region, and passes through firstImage acquisition device preliminary second image；The sixth feature position in preliminary second image is extracted, obtains placing Q-characterIt sets.
When the first image of the current placement region of image acquisition device and the second image of current capture area, i.e.,Start to treat moving object and be carried, before this, controller also needs to obtain crawl feature locations and places feature locations,With when treating moving object and being carried, according to crawl feature locations and it feature locations can be placed determine current crawl positionWith placement position.
In an alternative embodiment, can by way of artificial teaching, complete above-mentioned control mechanical arm it is mobile withAnd the step of initial first image of image acquisition device and preliminary second image, so that the process based on artificial teaching obtainsIt grabs feature locations and places benchmark position.
Still by taking paving floor tile as an example, above-mentioned acquisition crawl feature locations and place feature locations can paving floor tile indoorsInitial stage carry out, i.e., when starting to an indoor carry out floor tile paving, can be obtained grab by way of teaching firstIt takes feature locations to be and place feature locations, then carries out the paving of floor tile again.
Specifically, above-mentioned initial placement region and initial capture area can be preassigned region, for example, in pavingFirst piece of region (for example, indoor corner) for needing to place to paving floor tile is initial placement region when floor tile, crawlFirst piece is initial capture area to the region where paving floor tile.
As a kind of optional embodiment, object to be handled is to paving floor tile, and fisrt feature position is in third imageTo the position where the brick angle of paving floor tile, second feature position is the position where the brick angle of paving floor tile in the 4th imageIt sets.
Specifically, in the above scheme, can be square to paving floor tile, rectangle, equilateral hexagon etc. have angleShape.The brick angle of paving floor tile can be specified different two brick angle in above-mentioned third image and the 4th image.
Embodiment 2
The embodiment of the present application also provides a kind of handling system, which can be to execute arbitrary steps in embodiment 1System.Fig. 5 is according to a kind of schematic diagram of handling system of the embodiment of the present application, and as shown in connection with fig. 5, which includes:
First image collecting device 10, for acquiring the first image in capture area, the of the acquisition of the first placement regionTwo images, and position is grabbed according to the first image prediction, position is placed according to the second image prediction.
Specifically, above-mentioned image collecting device all can be smart camera, wherein the first image collecting device can be to setSet the smart camera in mechanical arm tail end.Robot may include strut, the second image collecting device and third image collectorIt installs and is placed in strut end.
In an alternative embodiment, by taking handling system carries out the paving of floor tile as an example, robot can control machineryArm is moved to placement region, first the second image of image acquisition device of mechanical arm tail end, and is calculated according to the second imageThe characteristic point for the reference brick placed in the second image out and the offset for placing feature locations, are transmitted to control for the offsetDevice, controller can be predicted according to the offset places position.Robot controls mechanical arm again and is moved to capture area, mechanical armFirst the first image of image acquisition device of end, and floor tile to be handled in the first image is calculated according to the first imageThe offset of characteristic point and crawl feature locations, is transmitted to controller for the offset, controller can be true according to the offsetMake crawl position.
Second image collecting device 20 for acquiring the third image of the second placement region, and determines in third imageFisrt feature position.
Third image collecting device 30 for acquiring the 4th image of third placement region, and determines in the 4th imageSecond feature position, wherein the second placement region and third placement region include two different Q-characters of placing objectsIt sets.
Specifically, strut has can be set in robot, above-mentioned second image collecting device and third image collecting device canTo be arranged in strut end, for acquiring the image of different placement regions, i.e., above-mentioned third image and the 4th image.It is above-mentioned to have putThe coordinate for setting at least two characteristic points of the two different feature locations of object for indicating placing objects, for example, withFor placing objects is the floor tiles of paving, two different feature locations can be two brick angles of the floor tile of paving.
It should be noted that above-mentioned second placement region is different from third placement region, but the first placement region can be withSecond placement region is identical, or identical as third placement region.
Fig. 2 is according to the schematic diagram of one kind second placement region and third placement region of the embodiment of the present application, in conjunction with Fig. 2It is shown, it is the floor tile of paving with reference to brick.By the first image and the second image prediction of the first image acquisition deviceIt obtains after placing position, mechanical arm will be moved to the surface of the placement position of prediction to paving floor tile, and (Fig. 2 is top view, to pavingGround floor tile and with reference to brick not in same plane), at this point, being taken pictures (the second rest area of area 2 by the second image acquisition deviceDomain) image, taken pictures the image of area 3 (third placement region) by third image acquisition device, to realize to placementThe effect of region progress double vision open country observation.
It is also to be noted that image collecting device is demarcated before use, Fig. 6 is according to the embodiment of the present applicationIn schematic diagram that image collecting device is demarcated, as shown in connection with fig. 6, by taking the first image collecting device as an example, for firstImage collecting device selectes the feature in shot region, and mobile camera to different location (can be 9), record feature is being schemedPixel coordinate and corresponding robot coordinate as in, to establish the relationship of image coordinate and robot world's coordinate, in turnObtain calibration matrix.For the second image collecting device and third image collecting device, the feature in shot region is selected, it is mobileCamera is to different location (can be 12), the pixel coordinate and corresponding robot coordinate of record feature in the picture, thusThe relationship of image coordinate and robot world's coordinate is established, and then obtains calibration matrix.
Robot 40, for obtaining the first offset determined according to fisrt feature position and second feature position, and rootIt is adjusted according to placement position of first offset to prediction.
Specifically, above-mentioned first offset can be the offset of the feature locations in fisrt feature position and the second imageThe offset of feature locations in amount and/or second feature position and the second image, so as to indicate practical placements position in advanceThe deviation of survey placed between position.
It in an alternative embodiment, include fourth feature position in the second image, robot is according to fourth feature positionSet and the deviation of benchmark placement location come predict place position.It is got in the second image collecting device and third image collecting deviceAfter third image and the 4th image, fisrt feature position and second feature position that will be extracted from third image and the 4th imageIt is sent to controller, controller can be according to fourth feature position included in the second image and fisrt feature position and theTwo feature locations, operation obtain above-mentioned first offset.
More specifically the first offset is illustrated.It is illustrated by taking Fig. 2 as an example, is to take pictures in the first placement regionIn the case where area 2, the fourth feature position in the second image is the coordinate P1_ at brick angle of the paving floor tile in area 2 of taking picturesRef (1) can be obtained in the second image according to the size of the coordinate combination floor tile, operation with reference to brick angle of the brick in area 3 of taking picturesCoordinate P2_ref (1).
Above-mentioned third image can be third image collecting device in the image of the shooting of area 2 of taking pictures, and fisrt feature position isFor the brick angular coordinate P1_ref (2) for referring to brick in area 2 of taking pictures；4th image can be the 4th image collecting device in area 3 of taking picturesThe image of shooting, second feature position are the brick angular coordinate P2_ref (2) for taking pictures and referring to brick in area 3.Then host computer can be withAbove-mentioned first offset is obtained according to P1_ref (1), P2_ref (1), P2_ref (1) and P2_ref (2) operation, for example, theOne offset P=(A, B), wherein A=P1_ref (2)-P1_ref (1), B=P2_ref (2)-P2_ref (1).
Since the first offset is used to indicate the deviation of the placement position of practical placement position and prediction, robot can rootAccording to the first offset, the placement position of prediction is adjusted, to obtain final placement position.Specific adjustment mode can be withIt is on the basis of the placement position of prediction, to increase above-mentioned first offset.
It follows that the first image that the above embodiments of the present application are acquired according to the first image collecting device in capture areaPrediction crawl position, and position is placed in the second image prediction that the first placement region acquires according to the first image collecting device；ControlThe third image of second the second placement region of image acquisition device, and control the placement of third image acquisition device third4th image in region, wherein the second placement region and third placement region include two different features of placing objectsPosition；Obtain according in third image fisrt feature position and the 4th image in second feature position determine first offsetAmount, wherein the first offset is used to indicate the deviation of the placement position of practical placement position and prediction；And according to the first offset to pre-The placement position of survey is adjusted.Above scheme is based on the first image collecting device prediction crawl position and places position, and is based on secondImage collecting device and third image collecting device rectify a deviation to the placement position of prediction, to reach promotion paving precisionPurpose.
Fig. 7 is according to a kind of framework map of handling system of the embodiment of the present application, below with reference to shown in Fig. 7, to above-mentioned systemEach device of system is specifically described:
First image collecting device, the second image collecting device and third image collecting device are smart camera, intelligenceEnergy camera mainly includes Image Acquisition, camera calibration, feature location, these modules of data communication, wherein trick camera, i.e.,First image collecting device further includes offset computing module.Host computer PC part is calculated (fixed double camera, i.e., the by offsetTwo image collecting devices and third image collecting device) and two module compositions of data communication, robot part is with machinePeople's control module.
Robot control module is responsible for the floor tile crawl and paving movement of mechanical arm, and sends in position of taking pictures accordingly to PCPhotographing instruction (by the value for modifying corresponding registers).After the communication module of the part PC receives photographing instruction, analyze the instruction, andPhotographing instruction is sent to corresponding camera according to the difference of instruction (by ICP/IP protocol, to instruct as " T1 ", " T2 " etc.), trigger itAcquire image.The communication module of smart camera analyzes the instruction, and triggers image capture module acquisition image.Trick camera is in featureIn locating module, the feature (angle of such as floor tile) in image is positioned by template matching, obtains its pixel coordinate, recycles cameraDemarcating module obtains corresponding robot coordinate, further obtains the mobile coordinate shift of mechanical arm by offset computing moduleInformation is measured, is sent to PC machine after handling using communication module.Fixed double camera directly leads to the robot coordinate of feature in image(character string for being converted into specific format) is sent to PC after crossing communication module processing.
The part PC receives the data from smart camera with communication module, extracts seat from the information that trick camera is sentOffset information is marked, extracts coordinate from the coordinate of fixed double camera.Offset computing module in PC passes through special in imageCorresponding coordinate shift amount information is calculated in the coordinate information of sign.Robot control is written in this offset data by communication moduleIn the location register of device processed, and modify the value of respective value register, inform controller take pictures and offset calculating processThrough completing.Robot control module learn take pictures process after the completion of, read corresponding position register in coordinate shift amount letterBreath, control mechanical arm are moved, and realize the floor tile crawl and paving of vision guide.
Fig. 8 is according to the schematic diagram of the data communication module of image collecting device in the embodiment of the present application, in conjunction with Fig. 8 instituteShow, sends two parts about the data communication module in image collecting device, including data receiver and data.Data reception portionDivide and photographing instruction is received from the part PC by ICP/IP protocol, read command information, triggers camera and acquire image.Data sending partDivide and the offset data of coordinate or coordinate is subjected to form collator, the part PC is sent to by ICP/IP protocol.
Specifically, input data include the photographing instruction from the part PC and the coordinate from modules such as feature locations or itsOffset information.Photographing instruction includes " T1 " (taking pictures capture area), " T2 " (placement region is taken pictures) and " T3 " (fixed double cameraIt takes pictures).
Output data module receives photographing instruction first and exports trigger signal to image capture module.Obtain coordinate or itsAfter offset information, they are sent to the part PC by ICP/IP protocol.After the part PC obtains these coordinate informations, phase is calculatedThe offset answered, and send the crawl and paving movement that floor tile is completed to robot.
For the data communication module of host computer PC, the communication module of the part PC includes the communication and and intelligence with robotThe communication of energy camera.The part PC and the communication of robot are carried out by the included communication software of robot.
For the offset computing module in host computer PC, since the size of floor tile is bigger (side length 800mm), a phaseThe visual field (about 250mm) of machine can not cover entire floor tile, therefore 2 angles of floor tile are observed using double camera.By acquiring image,The angle for positioning floor tile obtains take pictures area 2 and take pictures 3 middle berth patch block of area and vertex P1_put, P2_put, P1_ref with reference to brick,The robot coordinate of P2_ref.Paving brick is determined by vertex and with reference to the side of brick, then by allowing this 2 sides keeping parallelism (angleSpend deviation < 0.05 degree), and the distance of the 2mm that keeps at a distance (deviation < 0.3mm) come realize the gap widths of this two pieces of floor tiles for 2 ±0.5mm。
As a kind of optional embodiment, the first image collecting device is also used to according to the third feature position in the first imageThe second offset determined with preset crawl feature locations is set, is put according to the fourth feature position in the second image with presetThe third offset of feature locations is set, and according to crawl feature locations and the second offset prediction crawl position, according to placement benchmarkPosition is placed in position and the prediction of third offset, wherein is placed base position and is obtained according to feature locations are placed.
Specifically, above-mentioned placement feature locations and crawl feature locations can be the position determined during teaching,It is also possible to the position artificially specified.By taking teaching as an example, during teaching, the movement of manual control robot and mechanical armMovement, so as to complete the handling process of an object, and then determined during this and to have placed feature locations and crawlFeature locations.
Above-mentioned third feature position can be the position of the characteristic point of the object to be handled in the first image, and this feature point canTo be the central point of object to be handled in the first image, the position at the angle of the object to be handled in the first image can also be.TheFour feature locations are identical as the setting of fisrt feature position, and details are not described herein again.
Above-mentioned second offset can be the difference of third feature position and preset crawl feature locations, third offsetIt can be the difference of fourth feature position and placement feature locations.
As shown in Fig. 2, above-mentioned object to be handled can be floor tile, in figure, taking brick area of taking pictures is current capture area, patchBrick area of taking pictures is current placement region.
In an alternative embodiment, third offset and crawl feature locations can be adjusted to the same seat firstIn mark system, then increase third offset on the basis of grabbing feature locations, crawl position can be obtained.It similarly, can be firstBy the 4th offset and to place benchmark be to adjust into the same coordinate system, then increases by the on the basis of placing base positionFour offsets can be obtained and place position.
As a kind of optional embodiment, robot further includes mechanical arm, robot for control mechanical arm be moved to it is pre-The crawl position of survey grabs object to be handled, and controls the placement position that mechanical arm is moved to prediction according to object to be handled is carried.
In the above scheme, prediction obtain crawl position and place position after, can control mechanical arm be moved to crawl position grabAfter taking object to be handled, control mechanical arm, which is moved to, places position, places position so that object to be handled is moved to.
It should be noted that in the above scheme, mechanical arm is carried after object to be handled is moved to the placement position of prediction, andDo not loosen sucker and treat moving object and placed, but needs according to the second image collecting device and third image collecting deviceIt is adjusted to position is placed.
As a kind of optional embodiment, robot further includes mechanical arm, and the first image collecting device is set to mechanical armEnd.
As a kind of optional embodiment, robot includes strut, the second image collecting device and third image collectorIt installs and is placed in strut end.
As a kind of optional embodiment, object to be handled is to paving floor tile, and fisrt feature position is in third imageTo the position where the brick angle of paving floor tile, second feature position is the position where the brick angle of paving floor tile in the 4th imageIt sets.
Specifically, in the above scheme, can be square to paving floor tile, rectangle, equilateral hexagon etc. have angleShape.The brick angle of paving floor tile can be specified different two brick angle in above-mentioned third image and the 4th image.
As a kind of optional embodiment, robot is also used to for the first offset being compared with preset value, if theOne offset is greater than preset value, then increases by the first offset on the basis of third offset and obtain the 4th offset, and according toThe placement position of 4th offset adjustment prediction.
Specifically, above-mentioned preset value can be 0.5mm, and in above-mentioned steps, if the first offset is greater than preset value,Illustrate that deviation is larger to need to adjust, therefore increases by the first offset on the basis of third offset and obtain the 4th offset, rootAccording to the 4th offset and base position is placed, position is placed in prediction again.And if the first offset is less than or equal to preset value,Illustrate that deviation is smaller without adjustment.
After obtaining updated placement position according to adjustment prediction, the second image collecting device and third figure can also be controlledIt as acquisition device continues to acquire the image of the second placement region and third placement region, and repeats the above steps, until according to theThe offset that the image that two image collecting devices and third image collecting device continue acquisition determines is less than preset value.
Since above-mentioned adjustment process can execute repeatedly, subsequent process (such as process of vibrating) can also be overcome to be drawnThe error of the floor tile offset entered.
Embodiment 3
According to embodiments of the present invention, a kind of embodiment of the control method of handling system is provided, Fig. 9 is according to the applicationThe flow chart of the control method of another handling system of embodiment, as shown in connection with fig. 9, this method comprises:
Step S902 grabs position according to the first image prediction of capture area acquisition, and is predicted according to the first placement regionPlace position.
Step S904 obtains the third image of the second placement region and the 4th image of third placement region, wherein secondPlacement region and third placement region include two different feature locations of placing objects.
Step S906, according in third image fisrt feature position and the 4th image in second feature position determine theOne offset, wherein the first offset is used to indicate the deviation of the placement position of practical placement position and prediction.
Step S908 is adjusted according to placement position of first offset to prediction.
It should be noted that above-mentioned first image and the second image can be obtained by the same image collecting device, thirdImage and the 4th image can be obtained by different acquisition devices respectively.But the present embodiment is defined not to this.
It should also be noted that, above-mentioned steps can be executed by PC control robot, and side provided in this embodimentCase can also include other steps in embodiment 1, and the visible embodiment 1 of specific embodiment, details are not described herein again.
Embodiment 4
According to embodiments of the present invention, a kind of floor tile paving system, including handling system as described in example 2 are provided.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
In the above embodiment of the invention, it all emphasizes particularly on different fields to the description of each embodiment, does not have in some embodimentThe part of detailed description, reference can be made to the related descriptions of other embodiments.
In several embodiments provided herein, it should be understood that disclosed technology contents can pass through othersMode is realized.Wherein, the apparatus embodiments described above are merely exemplary, such as the division of the unit, Ke YiweiA kind of logical function partition, there may be another division manner in actual implementation, for example, multiple units or components can combine orPerson is desirably integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutualBetween coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or communication link of unit or moduleIt connects, can be electrical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unitThe component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multipleOn unit.It can some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unitIt is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated listMember both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent productWhen, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantiallyThe all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other wordsIt embodies, which is stored in a storage medium, including some instructions are used so that a computerEquipment (can for personal computer, server or network equipment etc.) execute each embodiment the method for the present invention whole orPart steps.And storage medium above-mentioned includes: that USB flash disk, read-only memory (ROM, Read-Only Memory), arbitrary access are depositedReservoir (RAM, Random Access Memory), mobile hard disk, magnetic or disk etc. be various to can store program codeMedium.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the artFor member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answeredIt is considered as protection scope of the present invention.