Disclosure of Invention
In view of the above, the invention provides an article instruction book display system and a display method based on AR technology, which can display a boring and odorless instruction book to a user in an animation mode, thereby solving the problems of poor readability and insufficient interestingness of the existing paper instruction book.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
in one aspect, the invention provides an article specification display system based on AR technology, which comprises a requester terminal and a cloud server, wherein the requester terminal is in communication connection with the cloud server;
the cloud server is used for receiving the picture information of the target specification sent by the request terminal, making corresponding virtual use animation according to the picture information of the target specification, sending the virtual use animation to the request terminal, and downloading and displaying the virtual use animation on the target specification by the request terminal.
Further, the cloud server includes:
the receiving module is used for receiving the picture data of the target instruction book uploaded by the requester terminal;
the feature extraction module is used for extracting key feature information in the picture of the target specification;
the database is used for storing AR animation data of various specifications;
the AR construction module is used for analyzing the key feature information extracted by the feature extraction module and calling corresponding AR animation data in the database to manufacture virtual use animation; and
and the issuing module is used for issuing the virtual usage animation manufactured by the AR construction module to the requester terminal.
Further, the cloud server further comprises an auditing module, the auditing module is used for conducting form auditing on the picture data received by the receiving module, judging whether the received picture is a specification picture, if the auditing result is a non-specification picture, feeding back the auditing result to the request terminal, and if the auditing result is the specification picture, sending an instruction to the feature extraction module.
Further, the AR construction module includes:
the classifying unit is used for classifying the target specifications according to the key feature information extracted by the feature extracting module;
the picture detection unit is used for detecting whether the key characteristic information contains picture data or not;
the data matching unit is used for directly matching AR animation data corresponding to the target specification according to other key characteristic information of the target specification picture from the specification AR animation data of the corresponding class in the database when the picture data is not detected;
the picture extraction unit is used for extracting picture information after the picture data are detected;
the model construction unit is used for constructing a 3D model corresponding to the target object according to the picture information extracted by the picture extraction unit; and
and the animation production unit is used for producing virtual use animation according to the AR animation data corresponding to the target instruction matched by the data matching unit, or producing virtual use animation according to the 3D model constructed by the model construction unit and calling related AR animation data in the database.
Further, the AR construction module further includes a customization unit, where the customization unit is used for a user to adjust the display content and/or the display progress of the virtual usage animation.
The setting of the custom unit is mainly used for conveniently and simply adjusting the playing sequence and the display progress of the animation according to the actual requirements of the user, so that the ideal experience effect of the user can be achieved more easily.
In another aspect, the present invention also provides an article instruction book display method based on AR technology, including:
shooting a picture of the target specification, and uploading picture information of the shot target specification;
making corresponding virtual use animation according to the picture information of the target instruction;
the virtual usage animation is downloaded and presented on the target specification.
Further, the method for making the corresponding virtual use animation according to the picture information of the target instruction specifically comprises the following steps:
receiving data: receiving the image data of the target specification uploaded by a requester;
extracting characteristics: extracting key characteristic information in the picture data of the target specification;
animation is made: analyzing the extracted key characteristic information, and calling corresponding AR animation data in the AR animation data of various specifications stored in advance to manufacture virtual use animation;
the following starting picture: and issuing the manufactured virtual usage animation to the requesting party.
Further, before extracting the key feature information in the picture data of the target instruction book, the method further comprises:
form audit is carried out on the received picture data, and whether the received picture is an instruction picture or not is judged: if the auditing result is a non-instruction picture, feeding back the auditing result to the requesting party, and if the auditing result is an instruction picture, entering a step of extracting the characteristics.
In the invention, the setting of the form audit function of the picture data is mainly used for rejecting a user to upload non-specification pictures as specification display requests, so that the reliability of animation display is prevented from being influenced by error information.
Further, the step of producing the animation specifically includes:
classifying the target specifications according to the extracted key characteristic information;
detecting whether the key feature information contains picture data or not;
when the picture data is not detected, matching AR animation data corresponding to the target specification according to other key characteristic information of the target specification picture directly from the pre-stored specification AR animation data of the corresponding class;
after the picture data is detected, extracting picture information from the key feature information;
constructing a 3D model corresponding to the target object according to the extracted picture information;
and creating virtual use animation according to the AR animation data corresponding to the matched target specification, or creating virtual use animation according to the constructed 3D model and calling the related AR animation data stored in advance.
Further, the key feature information includes a specification name, a picture, each subtitle name, a trademark, a brand name, and a product model.
For structural objects, the specifications generally relate to picture information such as the overall appearance, the internal structure, the component composition and the like of the product, and the key information is extracted to obtain a 3D model of the product more accurately, so that the three-dimensional form and the disassembly and assembly description of the object are simulated more truly by using an AR technology; for example, in the case of pharmaceutical products, text information in the specifications is relatively large, and by extracting information such as a pharmaceutical name, a brand, and the like, it is possible to match animation data having a relatively high similarity from among a large number of pre-stored specification AR animation data, and thereby display text images.
Compared with the prior art, the invention discloses the object instruction book display system and the object instruction book display method based on the AR technology, wherein the virtual use animation of the object instruction book is generated by taking the picture of the object instruction book and taking the picture as the basis and combining the pre-stored AR animation data of a large number of instruction books, and a user can more intuitively know the use information of the object through the virtual use animation, so that the object instruction book display system and the object instruction book display method are convenient for the user to learn the object use instruction scientifically and efficiently.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
On the one hand, referring to fig. 1, the embodiment of the invention discloses an article specification display system based on an AR technology, which comprises a requester terminal 2 and a cloud server 1, wherein the requester terminal 2 is in communication connection with the cloud server 1;
the requester terminal 2 is configured to take a picture of the target specification and upload the taken picture to the cloud server 1, the cloud server 1 is configured to receive the picture information of the target specification sent by the requester terminal 2, make a corresponding virtual usage animation according to the picture information of the target specification, send the virtual usage animation to the requester terminal 2, and the requester terminal 2 downloads and displays the virtual usage animation on the target specification.
In an embodiment, the requester terminal 2 may be an electronic product having a scanning function such as a smart phone or a tablet computer. In order to ensure the animation display quality, the animation display device can be used for displaying by matching with related AR products, such as AR glasses, and a user wears the AR glasses to watch virtual use animation, so that a better visual experience effect is achieved.
In a specific embodiment, referring to fig. 2, the cloud server 1 includes:
a receiving module 11, configured to receive the picture data of the target specification uploaded by the requester terminal 2;
the feature extraction module 12 is used for extracting key feature information in the picture of the target specification;
a database 13 for storing AR animation data of various specifications;
the AR construction module 14 is configured to analyze the key feature information extracted by the feature extraction module 12, and call corresponding AR animation data in the database 13 to make a virtual use animation; and
and the issuing module 15 is used for issuing the virtual usage animation manufactured by the AR construction module 14 to the requester terminal 2.
In a specific embodiment, the cloud server further includes an auditing module 16, where the auditing module 16 is configured to perform form audit on the picture data received by the receiving module 11, determine whether the received picture is a description picture, and if the auditing result is a non-description picture, feed back the auditing result to the requester terminal, and if the auditing result is a description picture, send an instruction to the feature extraction module 12.
In a specific embodiment, referring to fig. 3, ar building block 14 includes:
a classification unit 141, configured to classify the target specification according to the key feature information extracted by the feature extraction module 12;
a picture detection unit 142, configured to detect whether the key feature information includes picture data;
a data matching unit 143, configured to directly match, when no picture data is detected, AR animation data corresponding to the target specification according to other key feature information of the target specification picture from the specification AR animation data of the corresponding class in the database 13;
a picture extraction unit 144 for extracting picture information after detecting the picture data;
a model construction unit 145, configured to construct a 3D model corresponding to the target object according to the picture information extracted by the picture extraction unit 144; and
and an animation producing unit 146 for producing a virtual usage animation based on the AR animation data corresponding to the target specification matched by the data matching unit 143, or producing a virtual usage animation based on the 3D model constructed by the model constructing unit 145 and calling the related AR animation data in the database 13.
In the present embodiment, the model construction unit 145 may be completed by 3D modeling software, such as Max3D, maya, C4D, ZBrush, and the like.
In order to ensure the animation display effect, the embodiment does not adopt the traditional mode of directly going to the instruction book AR animation data with higher similarity in the database, but reasonably selects an animation production scheme according to specific instruction book types and picture conditions in the content, wherein the instruction book types refer to electronic equipment types, mechanical equipment types, daily necessities, medical drugs and the like, and a plurality of small types extend out of the specific types, so that the animation production scheme is more accurately matched with the data of the corresponding types in the database.
Meanwhile, in order to ensure the real-time performance of the data in the database, in some embodiments, the database and the third-party website can be established with a data link relationship, so that when the corresponding product cannot be matched in the database, the data with higher similarity can be more easily searched by searching in the form of keywords through the key information in the target instruction.
In the present embodiment, the specification AR animation data stored in the database contains AR resource data such as 3D model, action, and special effects.
In a specific embodiment, the AR construction module 14 further includes a customization unit 147, and the customization unit 147 allows the user to adjust the presentation content and/or the presentation progress of the virtual use animation.
The custom unit 147 is mainly used for simply adjusting the playing sequence and the displaying progress of the animation according to the actual requirement of the user, so that the ideal experience effect of the user can be achieved more easily.
In another aspect, referring to fig. 4, the present invention further provides an article instruction manual display method based on AR technology, where the method includes:
s1: shooting a picture of the target specification, and uploading picture information of the shot target specification;
s2: making corresponding virtual use animation according to the picture information of the target instruction;
s3: the virtual usage animation is downloaded and presented on the target specification.
In a specific embodiment, referring to fig. 5, in step S2, a corresponding virtual usage animation is generated according to the picture information of the target specification, which specifically includes:
s21: receiving data: receiving the image data of the target specification uploaded by a requester;
s23: extracting characteristics: extracting key characteristic information in the picture data of the target specification;
s24: animation is made: analyzing the extracted key characteristic information, and calling corresponding AR animation data in the AR animation data of various specifications stored in advance to manufacture virtual use animation;
s25: the following starting picture: and issuing the manufactured virtual usage animation to the requesting party.
In a specific embodiment, before extracting the key feature information in the picture data of the target specification, the method further includes:
s22: form audit is carried out on the received picture data, and whether the received picture is an instruction picture or not is judged: if the auditing result is a non-instruction picture, feeding back the auditing result to the requesting party, and if the auditing result is an instruction picture, entering a step of extracting the characteristics.
In the invention, the setting of the form audit function of the picture data is mainly used for rejecting a user to upload non-specification pictures as specification display requests, so that the reliability of animation display is prevented from being influenced by error information.
In a specific embodiment, referring to fig. 6, the step S24 of animating specifically includes:
s241: classifying the target specifications according to the extracted key characteristic information;
s242: detecting whether the key feature information contains picture data or not;
s243: when the picture data is not detected, matching AR animation data corresponding to the target specification according to other key characteristic information of the target specification picture directly from the pre-stored specification AR animation data of the corresponding class;
s244: after the picture data is detected, extracting picture information from the key feature information;
s245: constructing a 3D model corresponding to the target object according to the extracted picture information;
s246: and creating virtual use animation according to the AR animation data corresponding to the matched target specification, or creating virtual use animation according to the constructed 3D model and calling the related AR animation data stored in advance.
In a specific embodiment, the key feature information includes a specification name, a picture, each subtitle name, a trademark, a brand name, a product model number, and the like, and may also include two-dimensional code information, for example.
For structural objects, the specifications generally relate to picture information such as the overall appearance, the internal structure, the component composition and the like of the product, and the key information is extracted to obtain a 3D model of the product more accurately, so that the three-dimensional form and the disassembly and assembly description of the object are simulated more truly by using an AR technology; for example, in the case of pharmaceutical products, text information in the specifications is relatively large, and by extracting information such as a pharmaceutical name, a brand, and the like, it is possible to match animation data having a relatively high similarity from among a large number of pre-stored specification AR animation data, and thereby display text images.
In this embodiment, the virtual instruction animation refers to a virtual animation for showing an instruction for use of a product, and taking a product as an example of a medicine, the virtual instruction animation may be a virtual animation showing a medicine taking amount and a medicine taking time; taking the product as a building block toy as an example, the virtual usage instruction animation can be a virtual animation showing the assembly process of the building blocks.
The AR (Augmented Reality ) mentioned in this embodiment is a technique of integrating real world information and virtual world information "seamlessly" by calculating the position and angle of camera images in real time and adding corresponding virtual images.
In a specific embodiment, after determining the virtual usage instruction animation corresponding to the target instruction, the requesting party issues a download path of the virtual usage instruction animation to the requesting party, and the requesting party downloads the virtual usage instruction animation according to the received download path, determines the position and angle of the target instruction through the AR technology, and displays the downloaded virtual usage instruction animation on the target instruction.
In some embodiments, the target instruction may correspond to more than one virtual instruction animation, for example, taking an instruction of the target instruction is an instruction of the electric cooker as an example, and the corresponding virtual instruction animation may include a virtual instruction animation of a use process such as preparation before use, cooking, heating with cold rice, reservation, soup cooking, and the like. Specifically, the virtual usage instruction animation displayed can be selected according to the user operation, so that the display content of the virtual usage instruction animation can be adjusted, and the display progress of the virtual usage instruction animation can be adjusted according to the user operation. Taking the use instruction of the electric cooker as an example, after the user completes the preparation work through the virtual use instruction animation of the previous preparation process, the user can watch the virtual use instruction animation of the cooking process by clicking the next step. That is, the present embodiment can also adjust the presentation content and/or the presentation progress of the animation using the virtual usage instruction according to the user's demand.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.