Summary of the invention
The purpose of the present invention is to solve the problems of the prior art, a kind of the man-machine of flexible mechanical arm remote operating is proposedInteractive terminal and method.
In order to solve the above technical problems, the human-computer interaction terminal of flexible mechanical arm remote operating proposed by the present invention includes man-machineInterface hardware, 3D software operation interface, data calculation module, message processing module；The human-computer interaction hardware includes that VR is wornDisplay and operation handle；The 3D software operation interface is used to show the centre visual angle and data of VR head-mounted display；It is describedData calculation module is for being recognized and being optimized to operational order；The message processing module is for realizing interactive terminal and softProperty mechanical arm communication and data storage, be responsible for carrying out the control instruction of 3D Software Create into safety detection, and according to interfaceIt is required that being converted into regulation instruction format is sent to flexible mechanical arm；To which the operation of people can be intended to convert by the human-computer interaction terminalFor flexible mechanical arm control instruction, real-time to generate, send master-slave operation command sequence, control flexible mechanical arm is completed to specify distantOperation task.
In some embodiments, further include following technical characteristic:
The flexible mechanical arm is used based on rope driving and super Redundancy Design, and flexible mechanical arm joint designs use twoA freedom degree design, adjacent segment are mutually perpendicular to, and connect and compose whole flexible mechanical arm by each module, this conformational flexibilityMechanical arm has the three-dimensional space motion ability of super redundancy.
The data calculation module is also used under kinematic constraint condition, and the kinematics for completing flexible mechanical arm resolves, and is obtainedTo at least one of flexible mechanical arm rope length, end pose, configuration angle three, restrict so that the human-computer interaction terminal exportsAt least one of long message, configuration angle, end pose three are further completed to the control from end robot.
The present invention also proposes a kind of man-machine interaction method of flexible mechanical arm remote operating, it is characterised in that including walking as followsIt is rapid: S1, the 3 D stereo Visual Scene that flexible mechanical arm is generated by VR head-mounted display；S2, it is controlled by operation handleFlexible mechanical arm terminal angle, so that flexible mechanical arm arm type is controlled, to achieve the effect that operate flexible mechanical arm in person.
In some embodiments, further include following technical characteristic:
Further include following steps: S3, operational order being recognized and optimized by data calculation module, and is being movedUnder constraint condition, the kinematics for completing flexible mechanical arm is resolved, and obtains flexible mechanical arm rope length, end pose, configuration angle threeAt least one of person.
Further include following steps: message processing module realizes that the communication of interactive terminal and flexible mechanical arm and data are depositedStorage is responsible for carrying out the control instruction of generation into safety detection, and according to interface requirement be converted into regulation instruction format be sent to it is softProperty mechanical arm.
Flexible arm, which moves Three-dimensional Display, to be controlled by joint angles, turns the letter of rope length by joint angle in systemNumber calculates rope length, and rope length data are sent to ground control system by UDP, and ground control system is by rope length data conversionFor motor speed, motor rotation is thus controlled, so that flexible mechanical arm be driven to move.
Flexible mechanical arm ambient enviroment is observed by VR head-mounted display.
Further include: receive the instruction of remote operating subsystem, judge current subsystem and terminal whether failure exception, by then intoIn next step, not by then stopping remote operating task, task terminates row.
The invention further relates to a kind of storage mediums, which is characterized in that is stored thereon with computer program, the computer journeySequence can be performed to realize above-mentioned method.
Compared with prior art, it the beneficial effects of the present invention are as follows: the present invention is a kind of contactless interaction technique, is based onThe interaction technique of machine vision.Due to be it is contactless, whole operation mode is non-intrusion type, is done to operatorDisturbing can greatly reduce.In this kind of interaction technique, operator can go to control with more intuitive and natural command mode, hasEffect avoids interference brought by contact equipment.
Specific embodiment
In order to which technical problem to be solved of the embodiment of the present invention, technical solution and beneficial effect is more clearly understood,The present invention is further described in detail below with reference to the accompanying drawings and embodiments.It should be appreciated that specific implementation described hereinExample is not intended to limit the present invention only to explain the present invention.
It should be noted that it can be directly another when element is referred to as " being fixed on " or " being set to " another elementOn one element or indirectly on another element.When an element is known as " being connected to " another element, it canTo be directly to another element or be indirectly connected on another element.In addition, connection can be for fixingEffect is also possible to act on for circuit communication.
It is to be appreciated that term " length ", " width ", "upper", "lower", "front", "rear", "left", "right", "vertical",The orientation or positional relationship of the instructions such as "horizontal", "top", "bottom" "inner", "outside" is that orientation based on the figure or position are closedSystem is merely for convenience of the description embodiment of the present invention and simplifies description, rather than the device or element of indication or suggestion meaning mustThere must be specific orientation, be constructed and operated in a specific orientation, therefore be not considered as limiting the invention.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importanceOr implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed orImplicitly include one or more this feature.In the description of the embodiment of the present invention, the meaning of " plurality " is two or twoMore than, unless otherwise specifically defined.
The following embodiments of the present invention devise dedicated remote operating people for rope driving flexible mechanical arm (as shown in Figure 1A)The operation of people can be intended to be converted into flexible mechanical arm control by machine interactive terminal (as shown in Figure 1B) by human-computer interaction terminalSystem instruction, can export rope length information, configuration angle, end pose, complete to the control from end robot.Major function has:
(1) the 3 D stereo Visual Scene for producing flexible mechanical arm can make operator reach operation in person flexibleThe effect of mechanical arm, more accurately judges to make；
(2) it by human-computer interaction interface, realizes the control visual angle of main side operation and supervises aobvious visual angle coordinate synchronization from end state,Increase operating flexibility；
(3) by the VR helmet and VR handle, the operation information of operator is obtained in real time；
(4) operation information of operator is converted to the control instruction of flexible mechanical arm；
(5) master-slave operation order, the end pose six degree of freedom control of achievable flexible mechanical arm are sent in real time；
Flexible mechanical arm interactive terminal major function is to convert control instruction for the operation intention of people, is generated in real time, hairMaster-slave operation command sequence is sent, control flexible mechanical arm completes specified remote operating task.
Using the flexible mechanical arm end in VR handle operation virtual scene, control instruction control is generated from end flexible mechanicalArm.Flexible mechanical arm described herein is used based on rope driving and super Redundancy Design, and flexible arm joint designs use twoA freedom degree design, adjacent segment are mutually perpendicular to, and by the whole flexible arm that connects and composes of each module, thus this configuration is mechanicalArm has the three-dimensional space motion ability of super redundancy, it can be achieved that small space obstacle passes through.As shown in Figure 1A, flexible mechanical arm by4 sections of compositions, every section of two freedom degrees, totally 8 freedom degrees, every section of two freedom degrees are known as configuration angle, and each section by 3 rope lengthsControl the configuration angle of this section.
As shown in Figure 1B, flexible mechanical arm human-computer interaction terminal is grasped by human-computer interaction hardware, Unity3D (a kind of 3D software)Make interface, data calculation module, message processing module composition.
Human-computer interaction hardware is mainly made of VR head-mounted display with operation handle.The signal of human-computer interaction hardware effort is as schemedShown in 2, human-computer interaction interface composition is as shown in Figure 3.The 3 D stereo that VR head-mounted display generates flexible mechanical arm visualizes fieldScape, operation handle can control flexible mechanical arm terminal angle, so that flexible mechanical arm arm type is controlled, it is flexible to reach operation in personThe effect of mechanical arm；Unity3D operation interface can show the centre visual angle and data of VR head-mounted display, do convenient for operatorData point reuse and instruction modification；Data calculation module is recognized and is optimized to operational order, and under kinematic constraint condition,The kinematics for completing flexible mechanical arm resolves, and obtains flexible mechanical arm rope length, end pose, configuration angle；Message processing moduleRealize interactive terminal and flexible mechanical arm communication (message processing module is communicated by UDP with ground control system) andData storage is responsible for the control instruction for generating Unity and carries out safety detection, and is converted into regulation instruction lattice according to interface requirementFormula is sent to flexible mechanical arm.
Wherein, it includes: to obtain from the operation handle of operator that data calculation module, which recognizes operational order and optimizes,The data that take are passed to data calculation module, are instructed by the format formation operation of setting, by overrun testing, (current data subtractsPrevious bag data is gone, is compared divided by after the time with maximum speed, less than then passing through) after, it completes kinematics and resolves.
Obtain flexible mechanical arm rope length, end pose, configuration angle method include: obtain terminal position posture, pass throughJacobian matrix can calculate flexible mechanical arm joint angle；It can be by joint angle by D-H (Denavit-Hartenberg) matrixCalculate end pose；Configuration angle is obtained, rope length can be calculated by homogeneous transform matrix；It can will be restricted by Numerical Iteration MethodLength is converted to joint angle.
It is above-mentioned overrun testing that the control instruction that Unity is generated, which carries out safety detection, calculated rope length data, is usedCurrent data subtracts previous bag data, is compared divided by after the time with rope length maximum speed, less than then passing through, is greater than then to thisBag data is handled: by increment (this bag data subtracts a upper bag data) divided by 2, being added in a bag data, obtained numberAccording to re-starting overrun testing.
Human-computer interaction hardware
The effect of human-computer interaction hardware is to convert computer instruction for the intention of people, can obtain flexible mechanical in such a systemArm end posture information or configuration angle information or rope length information.The available Oculus Rift control handle of UnityPosture information, the pose that this information can map to flexible mechanical arm end in Unity (can only directly acquire end position in this engineeringAppearance can be converted to arm type angle and rope length by function).
Detailed process is as shown in Figure 4.
(1) VR head-mounted display passes through the unity system software connection communication on cable and computer；
(2) data (operation handle including sensor positioning capturing provided in unity software by VR head-mounted displayData), with function library (in library include end pose turns joint angle, joint angle turns rope length, joint angle turns the letters such as end poseNumber) flexible mechanical arm position and attitude information current in virtual scene is calculated.
(3) information data in unity software is mapped to flexible mechanical arm end pose, obtains flexible mechanical arm endThe instruction of pose master & slave control --- the flexible arm movement Three-dimensional Display in unity software is controlled by joint angles,Rope length is calculated by the function that joint angle turns rope length in system, and rope length data are sent to ground control system by UDP,Rope length data are converted to motor speed by ground control system, motor rotation are thus controlled, so that flexible mechanical arm be driven to move.
Human-computer interaction interface
Human-computer interaction interface mainly includes the initialization of flexible mechanical arm interactive terminal and management, control model setting, interfaceThe functions such as display setting.It is mainly realized by the selection of software interface button, the operation of entire terminal is managed.
1) Unity operating platform
The function of Unity operating platform specifically includes that
(1) starting of virtual flexible mechanical arm scene；
(2) the TCP/IP network communication connection and disconnection of platform and data transmission interface；
(3) transmission and stopping of flexible mechanical arm pose data.
2) VR head-mounted display and VR operation handle
The function of VR head-mounted display and VR operation handle specifically includes that the 3 D stereo for making operator in flexible mechanical armVirtual flexible mechanical arm is manipulated in Visual Scene.
3) (the forms program that Visual Studio writes is mainly used for showing data, data safety data transmission interfaceDetection and transmission)
Data transmission interface content specifically includes that
(1) sending cycle is instructed, wherein the instruction cycle includes 100ms, 200ms, 1000ms.
(2) flexible mechanical arm rope drive control, end Pose Control, configuration angle control the selection of three kinds of control models；
(3) the control data that flexible mechanical arm currently issues, with the telemetry received；
(4) control instruction type selects；
(5) Tcp/Ip is connected to the network；
(6) Udp is communicated to connect.
Flexible mechanical arm operational process is as shown in figure 4, mainly comprise the steps that
Remote operating task starts, and opens human-computer interaction hardware and computer；
(1) VR head-mounted display is connected, runs head-mounted display software, and start control handle
(2) Unity3D software is run；
(3) network connection button is clicked, the TCP/IP network communication for completing Unity operating platform and data transmission interface connectsIt connects；
(4) start to execute remote operating task, click flexible mechanical arm data send button；
(5) VR head-mounted display and control handle generate flexible mechanical arm relative pose data；
(6) in VR head-mounted display, operator can naked eyes intuitive judgment arm type it is whether feasible, meanwhile, it is right in UnityData command carries out safety detection, is then carried out in next step by detection, not by needing to regenerate master & slave control instruction；
(7) connection remote operating subsystem button is clicked in data transmission interface, the data received from Unity platform is turnedIt is changed to master & slave control instruction；
(8) receive the instruction of remote operating subsystem, judge current subsystem and terminal whether failure exception, by then carrying out downOne step, not by then stopping remote operating task, task terminates；
(9) data transmission interface sends remote operating by Tcp or Udp and instructs to subsystem；
(10) flexible mechanical arm runs to designated position, and remote operating task terminates, and saves data, closes terminal.
3.3 the above embodiment of the present invention technical solution bring beneficial effects have:
(1) have friendly human-computer interaction function, realize the control visual angle of main side operation and supervise aobvious visual angle from end state and assistIt adjusts and synchronizes, operator is facilitated to complete the interactive operation to flexible mechanical arm.
(2) control to flexible mechanical arm flexibly can be naturally realized by VR equipment；
(3) the observation flexible mechanical arm ambient enviroment that can be more clear, to make more accurate judgement.
(4) under abnormal conditions occurs in operation in flexible mechanical arm interactive terminal, manual intervention can be carried out rapidly.
(5) it realizes to flexible mechanical arm end Pose Control；
5, key problem in technology of the invention point includes:
1, flexible mechanical arm interactive terminal is by VR head-mounted display and operation handle, Unity operating platform, data calculation mouldBlock, message processing module composition.
2, the 3 D stereo Visual Scene of flexible mechanical arm is produced by VR head-mounted display and operation handle, it can be withOperator is set to achieve the effect that operate flexible mechanical arm in person.
3, it can generate in real time, send principal and subordinate's remote operating control instruction, have the ability for sending different time intervals instruction, it is distantOperational order form includes rope length instruction, the instruction of end pose, configuration angle command.
Data transmission interface can receive the telemetering rope length data that flexible arm motor is passed back, turn joint angle function by rope length,Joint angular data is generated, in transferring back to Unity.Establish another flexible mechanical arm in unity system, and with the pass receivedSave angular data driving.Operator can intuitively observe two arms arm type difference in Unity Three-dimensional Display, adjust transmitting terminal with thisThe data of generation control arm type.
Data generation is the end pose data by the pose and state of sensor capture operation handle to generate, and is led toJoint angular data and rope length data can be acquired by crossing end pose data, be instructed to generate according to instruction format.
4, flexible mechanical arm interactive terminal should carry out form safety detection to the operational order of generation, including continuousProperty transfinites.The method how to examine: the difference of the rope length data and original rope length data that newly receive is divided by between data twiceTime interval, to calculate rope length speed.By it compared with rope length maximum speed, if exceeding, this packet incremental data is halved,It recalculates and is judged, issued again until meeting condition.
5, flexible mechanical arm interactive terminal by network with from end flexible mechanical arm connection, to distal flexible mechanical arm intoRow control.
The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be said thatSpecific implementation of the invention is only limited to these instructions.For those skilled in the art to which the present invention belongs, it is not taking offUnder the premise of from present inventive concept, several equivalent substitute or obvious modifications can also be made, and performance or use is identical, all answeredWhen being considered as belonging to protection scope of the present invention.