Summary of the invention
In view of the above problems, it proposes on the present invention overcomes the above problem or at least be partially solved in order to provide one kindState a kind of assembling process management method and system of problem.
According to an aspect of the invention, there is provided a kind of AR glasses, are used for assembling process management, including collector, placeManage device and display, in which: the collector, for acquire the image for assembling scene, and by the figure of the assembly scene of acquisitionAs being sent to processor；The image of the processor, the assembly scene for acquiring to the collector identifies, according to knowledgeOther result obtains the assembly parameter of component to be assembled, and the assembly parameter of the component to be assembled is sent to the display；The display, for showing the assembly parameter of the component to be assembled, to indicate assembly manipulation.
Optionally, it includes described in identification that the processor, which carries out identification to the image for the assembly scene that the collector acquires,The attribute information of component to be assembled in scene is assembled, and the attribute information according to the component to be assembled obtains component to be assembledAssembly parameter.
Optionally, the collector is also used to acquire the encoded information of assembly tool, and the processor is according to the acquisitionThe encoded information of device acquisition obtains the mount attribute information of the assembly tool, according to the mount attribute information of the assembly toolThe assembly instruction of the assembly tool is obtained, to indicate that the assembly tool carries out assembly work.
Optionally, the AR glasses are connected with server, wherein the assembly yard that the processor acquires the collectorAfter the image of scape is identified, recognition result is sent to server so that server determined according to the recognition result it is to be installedAssembly parameter with component, and obtain from server the assembly parameter of component to be assembled.
According to another aspect of the present invention, a kind of assembling process management system, including AR glasses and server are provided,In: the image of the AR glasses acquisition assembly scene, and the image of the assembly scene of acquisition is sent to server；The serviceDevice identifies the image for the assembly scene that the collector acquires, and is joined according to the assembly that recognition result obtains component to be assembledNumber, and the assembly parameter of the component to be assembled is sent to the AR glasses；The AR glasses show the component to be assembledAssembly parameter, to indicate assembly manipulation.
In accordance with a further aspect of the present invention, a kind of assembling process management method is provided, comprising: the figure of acquisition assembly scenePicture；The image of the assembly scene of collector acquisition is identified, the assembly of component to be assembled is obtained according to recognition resultParameter；The assembly parameter of the component to be assembled is shown, to indicate assembly manipulation.
Optionally, it includes identifying the assembly yard that the image of the assembly scene to collector acquisition, which carries out identification,The attribute information of component to be assembled in scape, and the attribute information according to the component to be assembled obtains the assembly ginseng of component to be assembledNumber.
Optionally, the method also includes: acquire the encoded information of assembly tool, according to the collector acquire codingThe mount attribute information of assembly tool described in acquisition of information is assembled according to the mount attribute acquisition of information of the assembly toolThe assembly of tool indicates, to indicate that the assembly tool carries out assembly work.
Optionally, the method also includes: after being identified to the image of the assembly scene of the acquisition, according to the knowledgeOther result determines the assembly parameter of component to be assembled, and obtains the assembly parameter of component to be assembled.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage canIt is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawingExemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth hereIt is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosureCompletely it is communicated to those skilled in the art.
Currently, in the assembling process management of the industries such as automobile, household electrical appliances, by some process controling elements and radio frequencyIt identifies the technologies such as (RFID), barcode scanning, realizes digitlization, the flexibility of assembly.In maintenance industry, also there is new technologyScheme: job content, software are selected by the good process flow of software plan, then by manual operation computer or direct barcode scanningThe technique torque value of job content is transmitted to Intelligent spanner by system, and Intelligent spanner tightens bolt, and tightening result is anti-It feeds software systems.But there are still a biggish defects for the technical solution: i.e. it cannot be guaranteed that the bolt of worker's practical operation withIt is consistent specified in process flow, and cannot be guaranteed that worker is according to sequentially being tightened as defined in process flow.For example,The bolt being circumferentially distributed in the plane, it is desirable that operated according to the method diagonally tightened, but be difficult to ensure that worker is in practiceIt is operated according to requiring.
Fig. 1 is the structural block diagram of AR glasses first embodiment of the present invention.As shown in Figure 1, AR glasses 10 include collector101, processor 102 and display 103.
AR glasses 10 are used for assembling process management.
Augmented reality (Augmented Reality, AR) is a kind of interactive experience based on true environment.AR equipment canVirtual sensory information is generated, as vision, the sense of hearing, tactile, body-sensing, smell, so that user is obtained one kind and expandsOr the impression of real world that enhances.AR glasses, also referred to as intelligent glasses, wearing mode, can be in true scapes such as glassesFor information about as upper Overlapping display.
In embodiments of the present invention, assembly scene is a part of locomotive to be repaired, is assembled multiple to be installed in sceneThe component matched is bolt to be tightened.In some embodiments, component to be assembled can also be stud, screw, nut, padCircle, pin etc..Need to be tightened bolt to be bolt of the same race, bolt of the same race is the bolt of same model under same category, from shapeOn be difficult to differentiate between.In some embodiments, bolt to be tightened is similar bolt, and similar bolt is different model under same categoryBolt is closer in shape.In further embodiments, bolt to be tightened is not of the same race or similar bolt, in shape difference compared withGreatly.Each bolt has unique number.
The image of the acquisition assembly scene of collector 101, and the image of the assembly scene of acquisition is sent to processor 102.
After AR glasses 10 are worn above, towards assembly scene, the photo of shooting assembly scene is simultaneously sent to processor 102.
Processor 102 identifies the image for the assembly scene that collector 101 acquires, and is obtained according to recognition result to be installedAssembly parameter with component, and the assembly parameter of component to be assembled is sent to display 103.Assembly parameter includes assembly torqueAnd rigging position etc..
Display 103 shows the assembly parameter of the component to be assembled, to indicate assembly manipulation.Assembly parameter can all be shownShow, can also partially show.In embodiments of the present invention, it only shows rigging position, assembles torque without showing.In some embodimentsIn, it both shows rigging position, also shows assembly torque.
In embodiments of the present invention, AR glasses 10 indicate bolt to be assembled.The mode of instruction is nextThe arrow that display is flashed above bolt to be assembled, arrow is directed toward the bolt, and shows a virtual spanner in bolt location.AR glasses 10 show the information such as bolt assembly parameter, instruct picture and text to be superimposed upon in assembly scene assembling work, to assist, refer toIt leads and tightens operation.
In embodiments of the present invention, assembly scene image is acquired using AR glasses, the bolt in scene is identified, it is rightBolt to be assembled is indicated, it is ensured that is assembled by defined assembling process, helps to reduce in assembling processIt slips and avoids the problem that incorrect assembling process causes.
In some embodiments, processor 102 identifies the image for the assembly scene that collector 101 acquires, includingThe attribute information of component to be assembled in identification assembly scene, and the attribute information according to component to be assembled obtains component to be assembledAssembly parameter.The attribute information of component to be assembled includes the location information of component to be assembled, identity information etc., may also include otherFor being equipped with the information of indicative significance.
In some embodiments, collector 101 acquires the encoded information of assembly tool, and processor 102 is according to collector 101The encoded information of acquisition obtains the mount attribute information of assembly tool, according to the mount attribute acquisition of information assembler of assembly toolThe assembly of tool indicates, to indicate that the assembly tool carries out assembly work.The mount attribute information of assembly tool includes assemblerThe identity information of tool, the identity information can be corresponding with specific assembler.
In some embodiments, AR glasses 10 are connected with server, wherein the dress that processor 102 acquires collector 101After image with scene is identified, recognition result is sent to server, so that server is determined according to the recognition resultThe assembly parameter of component to be assembled, and obtain from server the assembly parameter of component to be assembled.
In some embodiments, assembly tool is Intelligent spanner.There are motor and controller, the output of motor in Intelligent spannerEnd one strain-type torque sensor of concatenation, the sensor can feed back real-time torque value to controller.Intelligent spanner can receiveThe target torque value that server is sent, and start motor.When motor rotates, the constantly more real-time torque value of controller and target are turned roundSquare value, and leave the motor off or slow down in due course according to control logic.After the completion, tightening result data are sent to clothes by Intelligent spannerBusiness device achieves.
There is the bar code for indicating its identity on Intelligent spanner.When Intelligent spanner is enclosed in bolt to be tightened indicated by AR glassesWhen upper, AR glasses, which scan the bar code of the Intelligent spanner, knows its number to identify the Intelligent spanner, and sends clothes for numberBusiness device.In some embodiments, there is its identity of two-dimentional code mark on Intelligent spanner.In further embodiments, on Intelligent spannerThere is its number, AR glasses shoot the photo of its number, and are sent to server and are identified.In embodiments of the present invention, it servicesDevice according to the preparatory locomotive assembling work process data for receiving process engineer's input, check the Intelligent spanner whether with the boltMatching, i.e., whether the Intelligent spanner is the tool tightening the bolt and should using.If matching, by the predetermined torque value of the boltIt is sent to the Intelligent spanner, it otherwise, i.e., will not be by the predetermined torque value of the bolt when Intelligent spanner is placed on other boltsIt is sent to the Intelligent spanner.After Intelligent spanner receives, the bolt is tightened by predetermined torque value automatically.If the spiral shell of operationBolt is not indicated by AR glasses, and AR glasses will issue warning note, while send server for faulty operation record and depositingShelves.After the completion of tightening operation, Intelligent spanner numbers the bolt, the artificial board number of operative employee, tightens deadline, actual torqueThe work datas such as value are uploaded to server archive.Server sends messages to AR glasses informing and is completed to working as front bolt after receivingTighten.AR glasses continue to indicate next bolt to be tightened.And so on, until completing to twist whole boltsTightly.
In some embodiments, assembly tool is the spanner with display screen, and display screen can show screw-down torque value.ARMirror shoots the spanner display screen photo and is sent to server.Server identifies the reading in display screen photo, and judges to identifyTorque value whether meet when front bolt predetermined torque value, the work data of this assembly manipulation is achieved if not meeting, andAR glasses are made to issue prompt, i.e., current torque value is bigger than normal or less than normal, tightens with prompt amendment to the bolt.
Fig. 2 is the structural block diagram of assembling process management system first embodiment of the present invention.As shown in Fig. 2, assembling process pipeReason system 20 includes AR glasses 201 and server 202.
The image of the acquisition assembly scene of AR glasses 201, and the image of the assembly scene of acquisition is sent to server 202.
Server 202 identifies the image for the assembly scene that the AR glasses 201 acquire, and is obtained according to recognition resultThe assembly parameter of component to be assembled, and the assembly parameter of component to be assembled is sent to AR glasses 201.
AR glasses 201 show the assembly parameter of component to be assembled, to indicate assembly manipulation.
In embodiments of the present invention, AR glasses 201 and server 202 pass through interconnected with wireless network.Assembling scene is wait tie upThe a part for the locomotive repaired, assembling multiple components to be assembled in scene is bolt of the same race to be tightened.AR glasses 201 acquireThe image for assembling scene uploads to clothes with the asynchronous mode of thread after carrying out Base64 coding to the assembly scene image of acquisitionBusiness device 202.The photo of the scene is compared server 202 with pre-generated model of place library, to identify thisScape is to get the number for knowing the scene.Model of place library includes some identified scene picture datas.Server 202 can be withIt identifies bolt to be assembled in scene, learns the number and location information of each bolt.Server 202 sends recognition resultTo AR glasses 201.The identity information of multiple components to be assembled is sent server 202 by AR glasses 201, and from server202 obtain the sequence assembled to multiple bolts to be assembled.AR glasses 201 are according to the assembly obtained from server 202Sequence provides the assembly instruction of multiple components to be assembled.Assembly tool according to sequence shown by AR glasses 201 to it is multiple toThe bolt of assembly is assembled.
In embodiments of the present invention, AR glasses acquisition assembly scene image, server identify the bolt in scene,AR glasses indicate bolt to be assembled by preset sequence, it is ensured that are filled by defined assembling processMatch, facilitate the careless omission reduced in assembling process and avoids the problem that incorrect assembling process causes.
In some embodiments, it includes knowing that server 202, which carries out identification to the image for the assembly scene that AR glasses 201 acquire,Not Zhuan Pei in scene component to be assembled attribute information, and the attribute information according to component to be assembled obtains the dress of component to be assembledWith parameter.
In some embodiments, AR glasses 201 are also used to acquire the encoded information of assembly tool, and server 202 is according to ARThe encoded information that glasses 201 acquire obtains the mount attribute information of assembly tool, is obtained according to the mount attribute information of assembly toolThe assembly of assembly tool is taken to indicate, to indicate that assembly tool carries out assembly work.
In some embodiments, server 202 includes image recognition server 2021 and technological data bank server 2022.
Image recognition server 2021 can assembly scene image collected to AR glasses, instrument and meter display screen, withAnd bar code or image in 2 D code are identified.Preferably, the GPU performance in image recognition server 2021 is not less than GTX 1080The performance of video card, and image recognition is carried out using deep learning algorithm.
Image recognition server 2021 in advance learns scene.The method learnt to a scene is as follows.It is firstThe archives of the scene are first added in technological data bank server 2022.Then the image of the scene is acquired with AR glasses in workshop,It is acquired as far as possible from multiple angles, under different illumination conditions.After carrying out Base64 coding to the image of acquisition, thread is usedAsynchronous mode uploads to technological data bank server 2022, is added in the archives of the scene.Image recognition server 2021Receive being labeled to the scene image for process engineer's input: to each bolt in scene image, from given inventoryMiddle selection sets a unique number.Image recognition server 2021 is with deep learning algorithm in the scene archivesImage is learnt.After study, the scene image for providing acquisition verifies learning outcome.It is required that the response time is less thanIt 1 second, identifies that the accuracy of scene is greater than 98%, identifies that the accuracy of any bolt in scene is greater than 99%, give number instructionThe accuracy of bolt location is greater than 99%, otherwise continues acquisition scene image and is learnt.
The algorithm that scene is identified include: scene characteristic point extract, scene positioning, with sample compare, if comparison atFunction, then export scene title, number and confidence level, otherwise resurveys scene image and is identified.
The algorithm identified to the bolt in scene includes: that bolt positioning, image segmentation and sample compare, if comparisonSuccess, then calculate bolt location and export confidence level, otherwise resurvey image and identified.During practical set,AR glasses potential range bolt is close, and bolt is larger, and image captured by AR glasses may be bolt itself or part.IfMultiple bolts to be assembled be it is of the same race or similar, then be difficult to distinguish.The algorithm of bolt is identified in the present invention specifically:Into assembly scene when, that is, the background for starting joint bolt position is identified, and continuous collecting assembly scene image, withAnd the variation of tracking image, if variation is continuously, to inherit the result that an image is identified.
The algorithm of optical character identification (Optical Character Recognition, OCR) is carried out to display screen photoIt include: display screen area location algorithm, character locating and partitioning algorithm and character recognition algorithm.Position, the direction of spanner canTo be arbitrary.It is required that the response time, less than 1 second, accuracy rate is greater than 99%.
The algorithm identified to bar code includes: shooting bar code image, image binaryzation, positioning, image segmentation；IfBar code positioning success then carries out rotational correction to bar code and then extracts barcode data, otherwise re-shoots bar code image and knownNot.It is required that the response time, less than 50 milliseconds, discrimination is greater than 99.5%.Bar code requires printing clear, and length is wide greater than 3 centimetresDegree is greater than 2 centimetres, and white edge and black surround width are all larger than 2 millimeters in bar code.
Technological data bank server 2022 stores the master data of vehicle, scene, bolt, tool etc., including vehicle nameClaim, number, manufacturer and scene, the number of bolt etc..The data of vehicle, scene, bolt, tool etc. can be managedMaintenance, including data increased newly, are deleted, are modified, inquires etc. and operated.Technological data bank server 2022 receives technique in advanceThe assembling work task of engineer's typing, the requirement of assembling work technique, predetermined torque value and multiple spiral shells including each boltThe assemble sequence of bolt.
The process for carrying out the business game of locomotive maintenance using assembling process management system 20 is as follows.Between bogie car,Assembler is gradually completing assembling work according to the instruction of AR glasses.Workshop should have good illumination condition.After worker's working,The tools such as spanner are got to teams and groups' strong point, and recruitment board swipes the card and obtains a secondary AR glasses.Server is on backstage by the work of the workerBoard and the AR glasses are bound, and AR glasses is made to enter the working condition for identifying rack bar code.In worker's wearing after AR glasses, according toThe prompt of display, before going to rack to be worked, scanning gantry bar code is simultaneously uploaded onto the server.Worker should face bar code as far as possible,Left and right visual angle is no more than 20 degree.AR glasses should in a certain range at a distance from bar code, and such as larger than 10 centimetres and less than 40 lisRice.Server checks whether the bar code matches with the number to operation platform, if matching, server make AR glasses enter identificationThe mode of scene, otherwise AR glasses show miscue and still in the states of identification bar code.After worker enters assembly scene, ARGlasses identify locating scene.Worker should face scene as far as possible, and left and right visual angle is no more than 40 degree.If locating scene be toScene is assembled, AR glasses will indicate that worker executes the task of current scene.After the completion of all tasks in current scene, AR glassesInstruction guides worker to next scene.After the completion of working as all tasks of front platform, AR glasses instruction guidance worker is to nextA rack.Worker can just go to next rack and carry out operation only after the completing a rack of the task, cannot be in two platformsOperation back and forth between frame.Worker can power cut-off at any time, when lower task, the AR glasses of wearing can be with the difference of last timeSystem can be automatically since the task that worker's last time does not complete.
Fig. 3 is the flow chart of assembling process management method first embodiment of the present invention.As shown in figure 3, the assembling process pipeReason method the following steps are included:
Step 301: the image of acquisition assembly scene.
In embodiments of the present invention, AR glasses and server pass through interconnected with wireless network.Assembling scene is machine to be repairedA part of vehicle, assembling multiple components to be assembled in scene is bolt of the same race to be tightened.AR glasses acquisition assembly sceneImage uploaded onto the server after carrying out Base64 coding to the assembly scene image of acquisition with the asynchronous mode of thread.
Step 302: the image of the assembly scene of collector acquisition being identified, is obtained according to recognition result to be installedAssembly parameter with component.
In embodiments of the present invention, the photo of the scene is compared server with pre-generated model of place library,To identify the scene to get the number of the scene is known.Model of place library includes some identified scene picture datas.ClothesBusiness device may recognize that bolt to be assembled in scene, learn the number and location information of each bolt.Server will identifyAs a result AR glasses are sent to.The identity information of multiple components to be assembled is sent server by AR glasses, and obtains from serverTake the sequence assembled to multiple bolts to be assembled.
Step 303: the assembly parameter of the display component to be assembled, to indicate assembly manipulation.
In embodiments of the present invention, AR glasses provide multiple portions to be assembled according to the sequence of the assembly obtained from serverThe assembly of part indicates.Assembly tool assembles multiple bolts to be assembled according to sequence shown by AR glasses.
In embodiments of the present invention, AR glasses acquisition assembly scene image, server identify the bolt in scene,AR glasses indicate bolt to be assembled by preset sequence, it is ensured that are filled by defined assembling processMatch, facilitate the careless omission reduced in assembling process and avoids the problem that incorrect assembling process causes.
In some embodiments, identification is carried out including to be assembled in identification assembly scene to the image of the assembly scene of acquisitionThe attribute information of component, and the attribute information according to component to be assembled obtains the assembly parameter of component to be assembled.
In some embodiments, method further include: the encoded information for acquiring assembly tool is obtained according to the encoded information of acquisitionThe mount attribute information for taking assembly tool, according to the assembly of the mount attribute acquisition of information assembly tool of assembly tool indicate, withIndicate that assembly tool carries out assembly work.
AR glasses, server, Intelligent spanner in conjunction with people, are constituted increasing by assembling process management method and system of the present inventionStrong manual work system.AR glasses enhance the vision of worker --- and worker, which wears AR glasses, can see briefing letterBreath.Server enhances the brain of worker --- and all process flows of server storing memory and job specification simultaneously issue instruction.Intelligent spanner enhances the both hands of worker --- and Intelligent spanner, which receives the torque value that server is sent and is automatically performed, tightens operation.Assembling process management system of the present invention improves traditional fastener installation hand with technologies such as AR, image recognition, Internet of ThingsSection, from tool, setting torsion to intelligent select tool, setting torsion is selected manually, from traditional drawing, Guide Book to ARMirror display reminding realizes assembly scene Recognition, assembling work instruction, assembling work control, assembling work feedback, assembling workThe closed-loop control of driving facilitates the burden for promoting the quality of assembly work, mitigating assembler, promotes assembling work efficiency,It can be used for the assembling process management such as " the secondary assembly " of locomotive inspection and repair.