Device and method for realizing augmented reality interactionTechnical Field
The present invention relates to the field of Augmented Reality (AR), and in particular, to an apparatus and method for implementing Augmented Reality interaction.
Background
Virtual Reality (VR) is a computer simulation system that can create and experience a Virtual world, and it uses a computer to create a Virtual environment, providing a user with an imagination space, and through the user's simulation interaction with it, giving the user a realistic experience to immerse it.
As VR technology moves from concept to reality in recent years, VR technology becomes a hotspot of the technology industry; especially a large number of VR devices (mainly the advent of VR headsets). However, the VR technology is not suitable for the application, and the effect of the VR technology in terms of immersion and the like is greatly reduced. For example, VR devices often need to fully cover the user's field of view to achieve better immersion (especially VR headset devices), but virtual world scenes still cannot achieve the real world visual effect for reasons such as better than technology. In addition, because the field of view is obstructed, users of VR devices are often fixed within certain safe areas to use and experience VR technology for personal safety reasons. This greatly restricts the application and popularization of VR technology.
Considering that some users seek experience which cannot be realized in the real world and real visual effect of the real world brought by the virtual world, and require that the users are not limited to regions and experience in the real world arbitrarily when in use, under the condition that the VR technology cannot achieve high reduction of the real world and the requirement is in logical contradiction, a certain technical means is needed to realize the effect so as to realize seamless integration of the real world and the virtual world.
Augmented Reality (AR for short) is a technology for seamlessly integrating real world information and virtual world information, which simulates and simulates contents which are difficult to experience in the real world originally through a computer and other technologies, and applies and combines the obtained virtual contents to the real world to enable the virtual contents to be perceived by a user, thereby achieving the experience beyond Reality. Although, augmented reality technology has begun to implement applications such as some augmented reality functions in google glasses, the recent popular augmented reality game Pokemon Go; however, because the method is limited by the prior art, in the application process of the enhancement implementation, the effect of the virtual content is rigid and stiff, and the user experience cannot be satisfied, or the generated virtual content cannot be well interfaced with the real world (including the sense of incongruity caused by the integration of the mobile phone frame in the real world and the virtual world). In addition, due to errors of calculation, display and the like of the equipment, the AR equipment also has influence on the visual field of the user, and still has great potential safety hazard.
Disclosure of Invention
In order to solve the above problems, the present invention provides a simple AR device, which enables the virtual content to be integrated with the real world well, and simultaneously utilizes the periscope principle to realize the accurate transmission of real world images, thereby affecting the line of sight of the user by the customer service equipment, and meanwhile, the semi-transparent and semi-reflective lens is provided to have no influence on the loading of the virtual content, thereby realizing the integration of real world information and virtual world information on the premise of not splitting the real world display.
The invention provides a device for realizing augmented reality interaction, which comprises a real image transmission unit and a virtual image loading unit, wherein the real image transmission unit consists of a shell and two groups of periscopes arranged on the shell, real world images are relayed and transmitted to the sight of a user through the two groups of periscopes, the two groups of periscopes are arranged in a concave shape, and the concave position of the periscopes is provided with a slot for carrying the virtual image loading unit; the central areas of a first lens and a last lens in the periscope and the central area of the display area of the virtual image loading unit are all on the horizontal extension line of the sight line of a user, and the display area of the virtual image loading unit falls in the visual field area of the first lens and the last lens; the last periscope lens is a semi-transparent semi-reflective lens, the virtual image loading unit is used for generating and displaying a virtual content image, the virtual content image is transmitted to the sight of a user through the semi-transparent semi-reflective lens, and the virtual content excludes a scene part; through the processing, the real world image and the virtual content image are naturally superposed, so that the integration of the real world information and the virtual world information is realized on the premise of not splitting the real world display, and meanwhile, the influence of the equipment on the sight of a user is overcome. The invention also provides a method for realizing the augmented reality interaction.
The embodiment of the invention provides a device for realizing augmented reality interaction, which comprises a real image transmission unit and a virtual image loading unit; wherein,
the real image transmission unit consists of a shell and two groups of periscopes arranged on the shell;
the two groups of periscopes are arranged in a concave shape; a slot for carrying the virtual image loading unit is arranged at a sunken position between the two groups of periscopes of the shell;
the two groups of periscopes can acquire real world image information and relay the real world image information to the sight of a user;
the central area of the first lens (namely the first lens when the real world image enters) of the periscope for transmitting the real world image information to the user, the central area of the last lens (namely the last lens when the real world image is transmitted to the sight of the user, the last lens is arranged in front of the virtual image loading unit) for transmitting the real world image information to the user and the central area of the display area of the virtual image loading unit are both on the horizontal extension line of the sight of the user;
the display area of the virtual image loading unit falls in the visual field area of the first lens and the last lens;
preferably, the visual field areas of the first lens and the last lens coincide with the display area of the virtual image loading unit;
the last periscope lens is a semi-transparent and semi-reflective lens;
the virtual image loading unit is used for generating and displaying a virtual content image, and transmitting the virtual content image to the sight of a user through the semi-transparent and semi-reflective lens; the virtual content excludes a scene part to avoid the inconsistency with a real scene; the virtual content generally includes virtual characters, virtual special effects, virtual articles and the like.
Further, in the above-mentioned apparatus,
the shell is composed of a shell body for carrying the periscope and the virtual image loading unit and a shell fixing part capable of fixing the shell on the head of a user.
Further, in the above-mentioned apparatus,
the virtual image loading unit comprises a viewing module, a motion track acquisition module, a virtual scene module, an operation execution module and a display module; the virtual image loading unit can be an intelligent mobile device with functions of photographing and motion trail acquisition, such as a mobile phone, a tablet and the like; wherein,
the view finding module is used for finding a view of a real scene to obtain real scene information;
the motion track acquisition module is used for capturing the motion track of the device and generating motion track information;
the virtual scene module is used for creating a virtual scene, and determining an initial virtual position of a user (namely, an initial position of a virtual character of the user in the virtual scene);
the operation execution module acquires the motion track information, the virtual scene and the initial virtual position of the user, and dynamically loads the virtual position of the user and the virtual scene around the user according to the motion track information and the initial virtual position of the user; meanwhile, according to the virtual position of the user, the virtual scene and a preset rule, the generation (such as generation time, position, trigger condition and the like) of the virtual content is controlled, and the dynamic loading of the virtual content is controlled according to the motion track of the virtual content in the virtual scene;
the display module can receive and display the virtual content, the virtual position of the user and the virtual scene around the user;
still further, in the above-mentioned patent document,
before the device is used, the virtual image loading unit is taken out and adjusted to enter a correction mode; in this mode, the display module is only used for receiving and displaying the virtual scene and the virtual position of the user;
then, framing the real scene to be used, creating a virtual scene according to the framing, and determining an initial virtual position of a user;
furthermore, the user moves a plurality of distances (which are large enough to be collected by the motion track collection module and display a motion track in a virtual scene), and the virtual image loading unit dynamically loads the virtual scene and the virtual position of the user; if the movement track of the user in the virtual scene is not consistent with the movement track of the user in the real scene, adjusting the virtual scene through a visual interface arranged on the display module (mainly by adjusting the direction and a scale of the movement distance of the virtual scene/the movement distance of the real world) to be consistent with the virtual movement track in the real scene;
after the correction is finished, the virtual image loading unit is adjusted to be in a use mode and is placed back to the slot; in this mode, the display module is only used for receiving and displaying the virtual content;
still further, in the above-mentioned patent document,
the view finding module is a shooting device of the virtual image loading unit; disposed on a back side opposite the display module;
the motion track acquisition module is realized by a sensor fixed on the virtual image loading unit;
the virtual scene module is used for constructing a three-dimensional virtual scene according to real scene information and a user framing position based on a computer graphics principle and determining an initial virtual position of a user;
still further, in the above-mentioned patent document,
in the operation execution module, the motion track of the virtual content in the virtual scene is controlled by a preset program of the operation execution module, and the preset program is generally compiled based on a behavior tree.
In addition, on the basis of the device, corresponding adjustment is carried out to realize real-time correction, specifically, the first lens is replaced by a semi-transparent and semi-reflective lens, meanwhile, a view finding module is arranged on the back surface of the virtual image loading unit to enable the virtual image loading unit to view in real time, the view finding information of the virtual image loading unit is ensured to be consistent with the real scene transmitted to the sight of the user by the real image transmission unit, the virtual position of the user is continuously corrected through the real-time view finding information, and the virtual content image is corrected according to the virtual position, so that the virtual content image is consistent with the real world image information.
The embodiment of the invention also provides a method for realizing augmented reality interaction, which is characterized by comprising the following steps:
a user acquires real world image information through relay transmission of two groups of periscopes; meanwhile, the two groups of periscopes are arranged in a concave shape;
a virtual image loading unit for generating and displaying virtual content images is arranged at the concave position between the two groups of periscopes and can be taken down; the virtual image loading unit may be a virtual image loading unit of any one of the above devices;
the central area of the first lens for transmitting real world image information to a user, the central area of the last lens for transmitting the real world image information to the user and the central area of the display area of the virtual image loading unit in the periscope are all on the horizontal extension line of the sight line of the user; the display area of the virtual image loading unit falls in the visual field area of the first lens and the last lens;
the last periscope lens is a semi-transparent and semi-reflective lens so as to transmit the virtual content image to the sight of a user through the last periscope lens; the virtual content excludes scene portions.
Further, on the basis of the method, the first lens is replaced by a semi-transparent semi-reflective lens, a viewing module is arranged on the back surface of the virtual image loading unit, so that the virtual image loading unit views in real time through the semi-transparent semi-reflective lens, the viewing information of the virtual image loading unit is ensured to be consistent with the real scene transmitted to the sight of the user by the real image transmission unit, the virtual position of the user is continuously corrected through the real-time viewing information, and the virtual content image is corrected according to the virtual position so as to be consistent with the real world image information.
The invention provides a device for realizing augmented reality interaction, which comprises a real image transmission unit and a virtual image loading unit, wherein the real image transmission unit transmits real world images to the sight of a user in a relay way through two groups of periscopes, the two groups of periscopes are arranged in a concave shape, and the concave position of the periscopes is provided with the virtual image loading unit; the last periscope lens in the periscope is a semi-transparent semi-reflective lens, the virtual image loading unit is used for generating and displaying a virtual content image, the virtual content image is transmitted to the sight of a user through the semi-transparent semi-reflective lens, and the virtual content excludes a scene part. The invention also provides a method for realizing the augmented reality interaction. By using the device or the method, the real world image and the virtual content image are naturally superposed, so that the integration of the real world information and the virtual world information is realized on the premise of not splitting the real world display, and meanwhile, the influence of the equipment on the sight of the user is overcome.
Drawings
Fig. 1 is a schematic structural diagram of an organization of an apparatus for implementing augmented reality interaction according to embodiment 1 of the present invention;
fig. 2 is a schematic view of an organization structure of a virtual image loading unit in an apparatus for implementing augmented reality interaction according to embodiment 2 of the present invention;
note: in fig. 1, a general arrow indicates a transmission path of a real image; the arrow indicates a transmission path of the virtual content image.
Detailed Description
In order to make the purpose, technical scheme and beneficial effects obtained by the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention is clearly and completely described below with reference to the accompanying drawings. It is to be understood that the embodiments described herein are merely illustrative and explanatory of the invention and are not restrictive thereof. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the device for implementing augmented reality interaction according to embodiment 1 of the present invention includes a real image transmission unit 101 and a virtual image loading unit 102; wherein,
the real image transmission unit consists of a shell 1011 and two groups of periscopes 1012 arranged on the shell;
the two groups of periscopes are arranged in a concave shape; a slot 1011a for carrying the virtual image loading unit is arranged at the recessed position of the shell between the two groups of periscopes;
the two groups of periscopes can acquire real world image information and relay the real world image information to the sight of a user (see the indication path of a common arrow in the figure 1);
the central area of the first lens (namely the first lens when the real world image enters) of the periscope for transmitting the real world image information to the user, the central area of the last lens (namely the last lens when the real world image is transmitted to the sight of the user, the last lens is arranged in front of the virtual image loading unit) for transmitting the real world image information to the user and the central area of the display area of the virtual image loading unit are both on the horizontal extension line of the sight of the user;
the display area of the virtual image loading unit falls in the visual field area of the first lens and the last lens;
preferably, the visual field areas of the first lens and the last lens coincide with the display area of the virtual image loading unit;
the last periscope lens is a semi-transparent and semi-reflective lens;
the virtual image loading unit is used for generating and displaying a virtual content image, and transmitting the virtual content image to the sight of a user through the semitransparent mirror (see the path indicated by the arrow in fig. 1); the virtual content excludes a scene part to avoid the inconsistency with a real scene; the virtual content generally includes virtual characters, virtual special effects, virtual articles and the like.
In the device for implementing augmented reality interaction according to embodiment 2 of the present invention, based on embodiment 1, the virtual image loading unit, as shown in fig. 2, includes a view finding module 201, a motion trajectory acquisition module 202, a virtual scene module 203, an operation execution module 204, and a display module 205; the virtual image loading unit can be an intelligent mobile device with functions of photographing and motion trail acquisition, such as a mobile phone, a tablet and the like; wherein,
the view finding module is used for finding a view of a real scene to obtain real scene information;
the motion track acquisition module is used for capturing the motion track of the device and generating motion track information;
the virtual scene module is used for creating a virtual scene, and determining an initial virtual position of a user (namely, an initial position of a virtual character of the user in the virtual scene);
the operation execution module acquires the motion track information, the virtual scene and the initial virtual position of the user, and dynamically loads the virtual position of the user and the virtual scene around the user according to the motion track information and the initial virtual position of the user; meanwhile, according to the virtual position of the user, the virtual scene and a preset rule, the generation (such as generation time, position, trigger condition and the like) of the virtual content is controlled, and the dynamic loading of the virtual content is controlled according to the motion track of the virtual content in the virtual scene;
the display module can receive and display the virtual content, the virtual position of the user and the virtual scene around the user.
The device for realizing augmented reality interaction provided by the embodiment 3 of the invention has the advantages that on the basis of the embodiment 2,
before the device is used, the virtual image loading unit is taken out and adjusted to enter a correction mode; in this mode, the display module is only used for receiving and displaying the virtual scene and the virtual position of the user;
then, framing the real scene to be used, creating a virtual scene according to the framing, and determining an initial virtual position of a user;
furthermore, the user moves a plurality of distances (which are large enough to be collected by the motion track collection module and display a motion track in a virtual scene), and the virtual image loading unit dynamically loads the virtual scene and the virtual position of the user; if the movement track of the user in the virtual scene is not consistent with the movement track of the user in the real scene, adjusting the virtual scene through a visual interface arranged on the display module (mainly by adjusting the direction and a scale of the movement distance of the virtual scene/the movement distance of the real world) to be consistent with the virtual movement track in the real scene;
after the correction is finished, the virtual image loading unit is adjusted to be in a use mode and is placed back to the slot; in this mode, the display module is only used to receive and display the virtual content.
The device for realizing augmented reality interaction provided by the embodiment 4 of the invention has the advantages that on the basis of the embodiment 2,
the view finding module is a shooting device of the virtual image loading unit; disposed on a back side opposite the display module;
the motion track acquisition module is realized by a sensor fixed on the virtual image loading unit;
the virtual scene module constructs a three-dimensional virtual scene according to the real scene information and the framing position of the user based on the computer graphics principle, and determines the initial virtual position of the user.
The device for realizing augmented reality interaction provided by the embodiment 5 of the invention has the advantages that on the basis of the embodiment 2,
in the operation execution module, the motion track of the virtual content in the virtual scene is controlled by a preset program of the operation execution module, and the preset program is generally compiled based on a behavior tree.
The device for realizing augmented reality interaction provided by the embodiment 6 of the invention is characterized in that on the basis of the embodiment 1,
the shell is composed of a shell body for carrying the periscope and the virtual image loading unit and a shell fixing part capable of fixing the shell on the head of a user.
The device for realizing augmented reality interaction provided by the embodiment 7 of the invention is based on any one of the embodiments 1 to 6,
and performing corresponding adjustment to realize real-time correction, specifically, replacing the first lens with a semi-transparent and semi-reflective lens, arranging a view finding module on the back of the virtual image loading unit to realize real-time view finding, ensuring that view finding information of the virtual image loading unit is consistent with a real scene transmitted to the sight of a user by the real image transmission unit, continuously correcting the virtual position of the user through the real-time view finding information, and correcting the virtual content image according to the virtual position to make the virtual content image consistent with the real world image information.
An embodiment 8 of the present invention provides a method for implementing augmented reality interaction, where the method includes:
a user acquires real world image information through relay transmission of two groups of periscopes; meanwhile, the two groups of periscopes are arranged in a concave shape;
a virtual image loading unit for generating and displaying virtual content images is arranged at the concave position between the two groups of periscopes and can be taken down; the virtual image loading unit may be a virtual image loading unit of any one of the apparatuses in embodiments 1 to 7;
the central area of the first lens for transmitting real world image information to a user, the central area of the last lens for transmitting the real world image information to the user and the central area of the display area of the virtual image loading unit in the periscope are all on the horizontal extension line of the sight line of the user; the display area of the virtual image loading unit falls in the visual field area of the first lens and the last lens;
the last periscope lens is a semi-transparent and semi-reflective lens so as to transmit the virtual content image to the sight of a user through the last periscope lens; the virtual content excludes scene portions.
Embodiment 9 of the present invention provides a method for implementing augmented reality interaction, which comprises, based on embodiment 8,
and replacing the first lens with a semi-transparent semi-reflective lens, arranging a viewing module on the back of the virtual image loading unit to enable the viewing module to view in real time through the semi-transparent semi-reflective lens, ensuring that viewing information of the viewing module is consistent with a real scene transmitted to the sight of a user by the real image transmission unit, continuously correcting the virtual position of the user through the real-time viewing information, and correcting the virtual content image according to the virtual position to enable the virtual content image to be consistent with the real world image information. The method can correct in real time, so that the virtual content image always keeps reasonable position and movement in a real scene, avoids the dislocation of the virtual content image and the discomfort of a user, and further always keeps seamless integration of real world information and virtual world information.
The device and the method for realizing the augmented reality interaction provided by the embodiment can enable the real world image and the virtual content image to be naturally superposed, thereby realizing the integration of the real world information and the virtual world information on the premise of not splitting the real world display and simultaneously overcoming the influence of the equipment on the sight of a user.
Finally, it should be noted that: the above examples are only for illustrating the technical solutions of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.