CN111429519B - Three-dimensional scene display method and device, readable storage medium and electronic equipment - Google Patents
Three-dimensional scene display method and device, readable storage medium and electronic equipmentDownload PDFInfo
- Publication number
- CN111429519B CN111429519BCN202010228728.5ACN202010228728ACN111429519BCN 111429519 BCN111429519 BCN 111429519BCN 202010228728 ACN202010228728 ACN 202010228728ACN 111429519 BCN111429519 BCN 111429519B
- Authority
- CN
- China
- Prior art keywords
- virtual
- face
- dimensional scene
- pose
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a three-dimensional scene display method and apparatus, a computer-readable storage medium, and an electronic device.
Background
Currently, the three-dimensional model scene display mainly changes the position in the displayed three-dimensional model scene by manually controlling the change of the viewing angle by a user, for example, the user uses a handheld mobile device (a mobile phone, a tablet computer, etc.) to complete the adjustment of the viewing angle by touching a screen with a finger, or by adjusting the holding posture (reading a gyroscope sensor in the device) to control the change of the viewing angle.
Disclosure of Invention
The embodiment of the disclosure provides a three-dimensional scene display method and device, a computer-readable storage medium and electronic equipment.
The embodiment of the disclosure provides a three-dimensional scene display method, which includes: acquiring a target face image; determining face pose information of a target face indicated by the target face image relative to the target window; based on the face pose information, adjusting the pose of a virtual viewfinder corresponding to the target window in the virtual three-dimensional scene; and displaying an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window.
In some embodiments, the method further comprises: adjusting the pose of the virtual viewfinder based on the operation in response to the operation of manually adjusting the pose of the virtual viewfinder; and displaying an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window.
In some embodiments, adjusting the pose of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose information includes: and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the first display mode, adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose angle information and the information representing the movement of the target face along the plane parallel to the target window, wherein the face pose angle information is included in the face position information.
In some embodiments, adjusting the pose of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose information includes: and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the second display mode, adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face posture angle information and the distance information which are included by the face position information, wherein the distance information is used for representing the distance between the target face and the target window.
In some embodiments, the method further comprises: in response to determining that the distance between the target face and the target window is greater than or equal to a preset distance threshold, ceasing to adjust the pose of the virtual viewfinder based on the face pose information.
According to another aspect of the embodiments of the present disclosure, there is provided a three-dimensional scene display apparatus including: the acquisition module is used for acquiring a target face image; the determining module is used for determining the face pose information of the target face indicated by the target face image relative to the target window; the first adjusting module is used for adjusting the pose of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose information; and the display module is used for displaying the image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window.
In some embodiments, the apparatus further comprises: a second adjustment module for adjusting the pose of the virtual viewfinder based on the operation in response to the operation of manually adjusting the pose of the virtual viewfinder; the display module is also used for displaying the image which is acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window.
In some embodiments, the first adjustment module is further to: and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the first display mode, adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose angle information and the information representing the movement of the target face along the plane parallel to the target window, wherein the face pose angle information is included in the face position information.
In some embodiments, the first adjustment module is further to: and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the second display mode, adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face posture angle information and the distance information which are included by the face position information, wherein the distance information is used for representing the distance between the target face and the target window.
In some embodiments, the apparatus further comprises: a stopping module for stopping adjusting the pose of the virtual viewfinder based on the face pose information in response to determining that the distance between the target face and the target window is greater than or equal to a preset distance threshold.
According to another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the above-described three-dimensional scene display method.
According to another aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing processor-executable instructions; and the processor is used for reading the executable instructions from the memory and executing the instructions to realize the three-dimensional scene display method.
Based on the three-dimensional scene display method, the three-dimensional scene display device, the computer readable storage medium and the electronic equipment provided by the embodiments of the disclosure, face pose information of a target face is obtained by recognizing the face image, then the pose of the virtual viewfinder is adjusted based on the face pose information, and finally the image currently acquired by the virtual viewfinder in the virtual three-dimensional scene is displayed in a target window, so that the view angle of viewing the virtual three-dimensional scene by changing the face pose control of a user is realized, the view angle does not need to be adjusted by holding, the flexibility and the convenience of viewing the virtual three-dimensional scene are improved, and the interaction mode of the user and the virtual three-dimensional scene is enriched.
The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 is a system diagram to which the present disclosure is applicable.
Fig. 2 is a flowchart illustrating a three-dimensional scene display method according to an exemplary embodiment of the disclosure.
Fig. 3 is a schematic diagram of an application scenario of the three-dimensional scene display method according to the embodiment of the present disclosure.
Fig. 4 is a schematic structural diagram of a three-dimensional scene display device according to an exemplary embodiment of the present disclosure.
Fig. 5 is a schematic structural diagram of a three-dimensional scene display apparatus according to another exemplary embodiment of the present disclosure.
Fig. 6 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.
Detailed Description
Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.
It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.
It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.
It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.
In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.
It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set top boxes, programmable consumer electronics, network pcs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems, and the like.
Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.
Summary of the application
The mode that present virtual three-dimensional scene was looked over mainly through manual touch screen, changes handheld device's gesture, uses the mode of equipment such as mouse, handle, and the flexibility is relatively poor, can't release user's both hands, and the operation is inconvenient.
Exemplary System
Fig. 1 illustrates anexemplary system architecture 100 of a three-dimensional scene display method or a three-dimensional scene display apparatus to which embodiments of the present disclosure may be applied.
As shown in fig. 1,system architecture 100 may includeterminal device 101,network 102, andserver 103. Network 102 is the medium used to provide communication links betweenterminal devices 101 andserver 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may useterminal device 101 to interact withserver 103 overnetwork 102 to receive or send messages and the like. Various communication client applications, such as a three-dimensional model application, a map application, a shooting application, and the like, may be installed on theterminal device 101.
Theterminal device 101 may be various electronic devices including a camera, including, but not limited to, mobile terminals such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a car terminal (e.g., car navigation terminal), and the like, and fixed terminals such as a digital TV, a desktop computer, and the like.
Theserver 103 may be a server that provides various services, such as a background image processing server that recognizes a face image uploaded by theterminal device 101. The background image processing server may process the received target face image to obtain a processing result (e.g., face pose information, pose of the virtual viewfinder, etc.).
It should be noted that the three-dimensional scene display method provided by the embodiment of the present disclosure may be executed by theserver 103 or theterminal device 101, and accordingly, the three-dimensional scene display apparatus may be disposed in theserver 103 or theterminal device 101.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. In the case that the face image does not need to be acquired from a remote place, the system architecture may not include a network, and only include a server or a terminal device.
Exemplary method
Fig. 2 is a flowchart illustrating a three-dimensional scene display method according to an exemplary embodiment of the disclosure. The embodiment can be applied to an electronic device (such as theterminal device 101 or theserver 103 shown in fig. 1), and as shown in fig. 2, the method includes the following steps:
In this embodiment, the electronic device may acquire the target face image locally or remotely. The target face image is an image obtained by shooting a target face by a camera on the electronic device, and the target face may be a face currently in a shooting range of the camera. In general, the target face image is a face image captured by a front camera of the electronic apparatus.
In this embodiment, the electronic device may determine face pose information of the target face indicated by the target face image relative to the target window. The target window is a window for displaying a virtual three-dimensional scene, and usually, the target window is displayed on a screen of the electronic device. The face pose information is used for representing the pose angle of the target face and the position of the target face. As shown in fig. 3, the attitude angle includes a pitch angle (pitch), a yaw angle (yaw), and a roll angle (roll).
Specifically, the electronic device may determine the face pose information of the target face using an existing face pose estimation method. As an example, the target face image may be input into a pre-trained deep learning model to obtain face pose information. And 3D Face detection technology (3D Face Tracking) can be used for determining a 4 x 4 pose matrix of the currently shot Face image, and the pose angle of the target Face and the position of the target Face under a camera coordinate system can be extracted from the pose matrix.
And step 203, adjusting the pose of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose information.
In this embodiment, the electronic device may adjust the pose of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose information. The virtual viewfinder is a virtual device for acquiring corresponding scene images in a virtual three-dimensional scene. In general, a virtual viewfinder corresponds to a target window, and corresponds to a virtual camera in a virtual three-dimensional scene, and an image captured by the virtual camera is displayed on the target window. It should be noted that the principle and implementation method of the virtual viewfinder are well-known technologies that are widely used and studied at present, and are not described in detail here.
Generally, the pose of the virtual viewfinder can be obtained from face pose information. For example, the pose of the virtual viewfinder can be the same as the face pose, i.e., the true face pose of the model in the virtual three-dimensional scene, thereby realizing the display of the scene seen by the simulated human eyes on the target window. The pose of the virtual viewfinder may also be mapped to the pose of the face in other ways, for example, a zoom factor may be set with which the pose of the face is transformed to the pose of the corresponding virtual viewfinder.
And step 204, displaying an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in a target window.
In this embodiment, the electronic device may display, in real time, an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window. Therefore, the face posture is corresponding to the virtual viewfinder, and the scene seen by the simulated actual visual angle is displayed to the user in real time. As shown in fig. 3, when the face of the targethuman face 301 rotates, a scene corresponding to the virtual three-dimensional scene 303 is displayed on thetarget window 302.
In some optional implementations, the electronic device may further perform the following steps:
first, in response to an operation of manually adjusting the pose of the virtual viewfinder, the pose of the virtual viewfinder is adjusted based on the operation. The manual operation includes adjusting the pose of the virtual viewfinder, such as touching a screen, clicking a mouse, using a remote controller, using a handle, and the like.
Then, an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene is displayed in the target window.
The priority of the operation of manually adjusting the pose of the virtual viewfinder in this implementation is higher than that of the mode of adjusting the pose of the virtual viewfinder based on the face pose. Generally, the range of adjusting the pose of the virtual viewfinder based on the face pose is small, and small-range changes in the viewing angle can be realized. The operation of manually adjusting the pose of the virtual viewfinder can realize operations of largely adjusting the angle of view such as moving, zooming, rotating, and the like. By combining the manual adjustment mode with the adjustment mode based on the face pose, the display of the virtual three-dimensional scene can be controlled more flexibly, and the flexibility and the convenience for viewing the virtual three-dimensional scene are improved.
In some alternative implementations,step 203 may include the steps of:
in response to determining that the display mode of the virtual three-dimensional scene is the first display mode, adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose angle information and information representing the movement of the target face along a plane parallel to the target window (for example, a plane on which an axis corresponding to Yaw and an axis corresponding to Pitch are located as shown in fig. 3). In the first display mode, when the pose of the virtual viewfinder is adjusted based on the face pose, the viewfinder pose adjustment is performed without using information representing the distance between the target face and the target window (for example, the distance between thetarget face 301 and thetarget window 302 shown in fig. 3). As an example, the first display mode may be a display mode of a god viewing angle, that is, when a distance between a target face and a target window changes, the virtual viewfinder does not move in the distance direction in the virtual three-dimensional scene. For another example, the first display mode may also be a display mode of a 360 ° panorama, and when the distance between the target face and the target window changes, the display scale of the panorama is not changed.
According to the implementation mode, when the virtual three-dimensional scene is displayed in the first display mode, the virtual viewfinder does not move along the distance direction, the quantity of hardware processing information can be reduced, and the efficiency of viewing the virtual three-dimensional model is improved.
In some alternative implementations,step 203 may include the steps of:
and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the second display mode, adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face posture angle information and the distance information which are included by the face position information. The distance information is used to characterize the distance between the target face and the target window (e.g., the distance between thetarget face 301 and thetarget window 302 shown in fig. 3). The second display mode does not limit the position of the virtual viewfinder in the virtual three-dimensional model. As an example, the second display mode may be used to view various objects within a building, such as an interior. In the second display mode, when the position and the attitude angle of the target face relative to the target window are changed, the position of the virtual viewfinder is changed along with the change.
The realization mode can randomly adjust the position of the virtual viewfinder in the virtual three-dimensional space, and improves the precision of checking the virtual three-dimensional space.
In some optional implementations, the electronic device may further perform the following steps:
in response to determining that the distance between the target face and the target window is greater than or equal to a preset distance threshold, ceasing to adjust the pose of the virtual viewfinder based on the face pose information. When the distance between the target face and the target window is too large, the accuracy of face pose information determination is reduced, and further the position of the virtual viewfinder is adjusted wrongly, so that the user can view the virtual three-dimensional scene. By setting the threshold value, the face pose information can be prevented from being determined wrongly, and the possibility of blocking when the user views the virtual three-dimensional scene is reduced.
According to the method provided by the embodiment of the disclosure, the face pose information of the target face is obtained by identifying the face image, the pose of the virtual viewfinder is adjusted based on the face pose information, and finally the image currently acquired by the virtual viewfinder in the virtual three-dimensional scene is displayed in the target window, so that the view angle for viewing the virtual three-dimensional scene by changing the face pose control of a user is realized, the view angle does not need to be adjusted by holding, the flexibility and the convenience for viewing the virtual three-dimensional scene are improved, and the interaction mode of the user and the virtual three-dimensional scene is enriched.
Exemplary devices
Fig. 4 is a schematic structural diagram of a three-dimensional scene display device according to an exemplary embodiment of the present disclosure. The present embodiment can be applied to an electronic device, and as shown in fig. 4, the three-dimensional scene display apparatus includes: an obtainingmodule 401, configured to obtain a target face image; a determiningmodule 402, configured to determine face pose information of a target face indicated by the target face image with respect to the target window; afirst adjusting module 403, configured to adjust a pose of a virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose information; and adisplay module 404, configured to display an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window.
In this embodiment, theacquisition module 401 may acquire the target face image locally or remotely. The target face image is an image obtained by shooting a target face by a camera on the three-dimensional scene display device, and the target face can be a face currently in a shooting range of the camera. In general, the target face image is a face image photographed by a front camera of the above-described three-dimensional scene display apparatus.
In this embodiment, thedetermination module 402 may determine face pose information of a target face indicated by the target face image relative to the target window. The target window is a window for displaying a virtual three-dimensional scene, and generally, the target window is displayed on a screen of the three-dimensional scene display device. The face pose information is used for representing the pose angle of the target face and the position of the target face. As shown in fig. 3, the attitude angle includes a pitch angle (pitch), a yaw angle (yaw), and a roll angle (roll).
Specifically, the determiningmodule 402 may determine the face pose information of the target face using an existing face pose estimation method. As an example, the target face image may be input into a pre-trained deep learning model to obtain face pose information. And 3D Face detection technology (3D Face Tracking) can be used for determining a 4 x 4 pose matrix of the currently shot Face image, and the pose angle of the target Face and the position of the target Face under a camera coordinate system can be extracted from the pose matrix.
In this embodiment, thefirst adjusting module 403 may adjust the pose of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose information. The virtual viewfinder is a virtual device for acquiring corresponding scene images in a virtual three-dimensional scene. In general, a virtual viewfinder corresponds to a target window, and corresponds to a virtual camera in a virtual three-dimensional scene, and an image captured by the virtual camera is displayed on the target window. It should be noted that the principle and implementation method of the virtual viewfinder are well-known technologies that are widely used and studied at present, and are not described in detail here.
Generally, the pose of the virtual viewfinder can be obtained from face pose information. For example, the pose of the virtual viewfinder can be the same as the face pose, i.e., the true face pose of the model in the virtual three-dimensional scene, thereby realizing the display of the scene seen by the simulated human eyes on the target window. The pose of the virtual viewfinder may also be mapped to the pose of the face in other ways, for example, a zoom factor may be set with which the pose of the face is transformed to the pose of the corresponding virtual viewfinder.
In this embodiment, thedisplay module 404 may display an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window in real time. Therefore, the face posture is corresponding to the virtual viewfinder, and the scene seen by the simulated actual visual angle is displayed to the user in real time.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a three-dimensional scene display apparatus according to another exemplary embodiment of the present disclosure.
In some optional implementations, the three-dimensional scene display apparatus may further include: asecond adjustment module 405 for adjusting the pose of the virtual viewfinder based on the operation in response to the operation of manually adjusting the pose of the virtual viewfinder; thedisplay module 404 is further configured to display an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window.
In some optional implementations, thefirst adjusting module 403 may be further configured to: and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the first display mode, adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose angle information and the information representing the movement of the target face along the plane parallel to the target window, wherein the face pose angle information is included in the face position information.
In some optional implementations, thefirst adjusting module 403 may be further configured to: and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the second display mode, adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face posture angle information and the distance information which are included by the face position information, wherein the distance information is used for representing the distance between the target face and the target window.
In some optional implementations, the three-dimensional scene display apparatus may further include: a stopping module 406 for stopping adjusting the pose of the virtual viewfinder based on the face pose information in response to determining that the distance between the target face and the target window is greater than or equal to a preset distance threshold.
The three-dimensional scene display device provided by the embodiment of the disclosure obtains the face pose information of the target face by recognizing the face image, adjusts the pose of the virtual viewfinder based on the face pose information, and finally displays the image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window, so that the viewing angle for viewing the virtual three-dimensional scene by changing the face pose control of a user is realized, the viewing angle does not need to be adjusted by holding, the flexibility and the convenience for viewing the virtual three-dimensional scene are improved, and the interaction mode between the user and the virtual three-dimensional scene is enriched.
Exemplary electronic device
Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 6. The electronic device may be either or both of theterminal device 101 and theserver 103 as shown in fig. 1, or a stand-alone device separate from them, which may communicate with theterminal device 101 and theserver 103 to receive the collected input signals therefrom.
FIG. 6 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.
As shown in fig. 6, the electronic device 600 includes one ormore processors 601 andmemory 602.
Theprocessor 601 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 600 to perform desired functions.
Thememory 602 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, Random Access Memory (RAM), cache memory (or the like). The non-volatile memory may include, for example, Read Only Memory (ROM), a hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer readable storage medium and executed by theprocessor 601 to implement the three-dimensional scene display methods of the various embodiments of the present disclosure above and/or other desired functions. Various contents such as a face image, a virtual three-dimensional scene, etc. may also be stored in the computer-readable storage medium.
In one example, the electronic device 600 may further include: aninput device 603 and anoutput device 604, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
For example, when the electronic device is theterminal device 101 or theserver 103, theinput device 603 may be a camera or the like, and is used for inputting a face image. When the electronic device is a stand-alone device, theinput device 603 may be a communication network connector for receiving the input face image from theterminal device 101 and theserver 103.
Theoutput device 604 may output various information to the outside, including the determined face pose information, the image of the scene displayed on the target window, and the like. Theoutput devices 604 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.
Of course, for simplicity, only some of the components of the electronic device 600 relevant to the present disclosure are shown in fig. 6, omitting components such as buses, input/output interfaces, and the like. In addition, electronic device 600 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the three-dimensional scene display method according to various embodiments of the present disclosure described in the "exemplary methods" section of this specification above.
The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the three-dimensional scene display method according to various embodiments of the present disclosure described in the "exemplary methods" section above in this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.
In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (8)
1. A three-dimensional scene display method, comprising:
acquiring a target face image;
determining face pose information of a target face indicated by the target face image relative to a target window;
based on the face pose information, adjusting the pose of a virtual viewfinder corresponding to the target window in a virtual three-dimensional scene;
displaying an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window;
based on the face pose information, adjusting the pose of the virtual viewfinder corresponding to the target window in a virtual three-dimensional scene, including:
in response to the fact that the display mode of the virtual three-dimensional scene is determined to be a first display mode, fixing the position of the virtual viewfinder in the depth direction in the virtual three-dimensional scene, and adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on face pose angle information and information representing that the target face moves along a plane parallel to the target window, wherein the face pose angle information comprises face pose angle information, and the face pose information represents that the target face moves along the plane parallel to the target window;
the method further comprises the following steps:
in response to determining that the distance between the target face and the target window is greater than or equal to a preset distance threshold, ceasing to adjust the pose of the virtual viewfinder based on the face pose information.
2. The method of claim 1, wherein the method further comprises:
in response to an operation of manually adjusting the pose of the virtual viewfinder, adjusting the pose of the virtual viewfinder based on the operation;
and displaying an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window.
3. The method of claim 1, wherein the adjusting the pose of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on the face pose information comprises:
and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the second display mode, adjusting the pose of a virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on face pose angle information and distance information included by the face pose information, wherein the distance information is used for representing the distance between the target face and the target window.
4. A three-dimensional scene display apparatus comprising:
the acquisition module is used for acquiring a target face image;
the determining module is used for determining the face pose information of the target face indicated by the target face image relative to a target window;
the first adjusting module is used for adjusting the pose of the virtual viewfinder corresponding to the target window in a virtual three-dimensional scene based on the face pose information;
the display module is used for displaying an image which is acquired by the virtual viewfinder under the virtual three-dimensional scene at present in the target window;
the first adjustment module is further to:
in response to the fact that the display mode of the virtual three-dimensional scene is determined to be a first display mode, fixing the position of the virtual viewfinder in the depth direction in the virtual three-dimensional scene, and adjusting the position of the virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on face pose angle information and information representing that the target face moves along a plane parallel to the target window, wherein the face pose angle information comprises face pose angle information, and the face pose information represents that the target face moves along the plane parallel to the target window;
the device further comprises:
a stopping module for stopping adjusting the pose of the virtual viewfinder based on the face pose information in response to determining that the distance between the target face and the target window is greater than or equal to a preset distance threshold.
5. The apparatus of claim 4, wherein the apparatus further comprises:
a second adjustment module configured to adjust, in response to an operation of manually adjusting a pose of the virtual viewfinder, the pose of the virtual viewfinder based on the operation;
the display module is further used for displaying an image currently acquired by the virtual viewfinder in the virtual three-dimensional scene in the target window.
6. The apparatus of claim 4, wherein the first adjustment module is further to:
and in response to the fact that the display mode of the virtual three-dimensional scene is determined to be the second display mode, adjusting the pose of a virtual viewfinder corresponding to the target window in the virtual three-dimensional scene based on face pose angle information and distance information included by the face pose information, wherein the distance information is used for representing the distance between the target face and the target window.
7. A computer-readable storage medium, the storage medium storing a computer program for performing the method of any of the preceding claims 1-3.
8. An electronic device, the electronic device comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010228728.5ACN111429519B (en) | 2020-03-27 | 2020-03-27 | Three-dimensional scene display method and device, readable storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010228728.5ACN111429519B (en) | 2020-03-27 | 2020-03-27 | Three-dimensional scene display method and device, readable storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111429519A CN111429519A (en) | 2020-07-17 |
| CN111429519Btrue CN111429519B (en) | 2021-07-16 |
Family
ID=71548916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010228728.5AActiveCN111429519B (en) | 2020-03-27 | 2020-03-27 | Three-dimensional scene display method and device, readable storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111429519B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113572967B (en)* | 2021-09-24 | 2021-12-31 | 北京天图万境科技有限公司 | Viewfinder of virtual scene and viewfinder system |
| CN116095356A (en)* | 2023-01-18 | 2023-05-09 | 北京有竹居网络技术有限公司 | Method, device, device and storage medium for presenting virtual scene |
| CN119006603A (en)* | 2023-05-18 | 2024-11-22 | 华为技术有限公司 | Display control method and device and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102681654A (en)* | 2011-03-18 | 2012-09-19 | 深圳富泰宏精密工业有限公司 | System and method for automatic adjustment of three-dimensional visual angles |
| CN102917232A (en)* | 2012-10-23 | 2013-02-06 | 深圳创维－Rgb电子有限公司 | Face recognition based 3D (three dimension) display self-adaptive adjusting method and face recognition based 3D display self-adaptive adjusting device |
| CN104407694A (en)* | 2014-10-29 | 2015-03-11 | 山东大学 | Man-machine interaction method and device combining human face and gesture control |
| CN107643828A (en)* | 2011-08-11 | 2018-01-30 | 视力移动技术有限公司 | The method and system that user behavior in vehicle is identified and responded |
| CN108170282A (en)* | 2018-01-19 | 2018-06-15 | 百度在线网络技术（北京）有限公司 | For controlling the method and apparatus of three-dimensional scenic |
| US10146301B1 (en)* | 2015-03-26 | 2018-12-04 | Amazon Technologies, Inc. | Rendering rich media content based on head position information |
| CN110035270A (en)* | 2019-02-28 | 2019-07-19 | 努比亚技术有限公司 | A kind of 3D rendering display methods, terminal and computer readable storage medium |
| CN110728749A (en)* | 2019-10-10 | 2020-01-24 | 青岛大学附属医院 | Virtual three-dimensional image display system and method |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8885882B1 (en)* | 2011-07-14 | 2014-11-11 | The Research Foundation For The State University Of New York | Real time eye tracking for human computer interaction |
| CN104503092B (en)* | 2014-11-28 | 2018-04-10 | 深圳市魔眼科技有限公司 | Different angle and apart from adaptive 3 D displaying method and equipment |
| CN104601981A (en)* | 2014-12-30 | 2015-05-06 | 深圳市亿思达科技集团有限公司 | Method for adjusting viewing angles based on human eyes tracking and holographic display device |
| CN105988578B (en)* | 2015-03-04 | 2019-06-21 | 华为技术有限公司 | A method, device and system for interactive video display |
| CN108153422B (en)* | 2018-01-08 | 2023-02-17 | 维沃移动通信有限公司 | A display object control method and mobile terminal |
| CN109558008B (en)* | 2018-11-30 | 2020-10-27 | 腾讯科技（深圳）有限公司 | Control method, control device, computer equipment and storage medium |
- 2020
- 2020-03-27CNCN202010228728.5Apatent/CN111429519B/enactiveActive
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102681654A (en)* | 2011-03-18 | 2012-09-19 | 深圳富泰宏精密工业有限公司 | System and method for automatic adjustment of three-dimensional visual angles |
| CN107643828A (en)* | 2011-08-11 | 2018-01-30 | 视力移动技术有限公司 | The method and system that user behavior in vehicle is identified and responded |
| CN102917232A (en)* | 2012-10-23 | 2013-02-06 | 深圳创维－Rgb电子有限公司 | Face recognition based 3D (three dimension) display self-adaptive adjusting method and face recognition based 3D display self-adaptive adjusting device |
| CN104407694A (en)* | 2014-10-29 | 2015-03-11 | 山东大学 | Man-machine interaction method and device combining human face and gesture control |
| US10146301B1 (en)* | 2015-03-26 | 2018-12-04 | Amazon Technologies, Inc. | Rendering rich media content based on head position information |
| CN108170282A (en)* | 2018-01-19 | 2018-06-15 | 百度在线网络技术（北京）有限公司 | For controlling the method and apparatus of three-dimensional scenic |
| CN110035270A (en)* | 2019-02-28 | 2019-07-19 | 努比亚技术有限公司 | A kind of 3D rendering display methods, terminal and computer readable storage medium |
| CN110728749A (en)* | 2019-10-10 | 2020-01-24 | 青岛大学附属医院 | Virtual three-dimensional image display system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111429519A (en) | 2020-07-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4198694A1 (en) | Positioning and tracking method and platform, head-mounted display system, and computer-readable storage medium | |
| CN112835484B (en) | Dynamic display method and device based on operation body, storage medium and electronic equipment | |
| EP3465620B1 (en) | Shared experience with contextual augmentation | |
| CN111178191B (en) | Information playing method and device, computer readable storage medium and electronic equipment | |
| JP6072237B2 (en) | Fingertip location for gesture input | |
| CN111429519B (en) | Three-dimensional scene display method and device, readable storage medium and electronic equipment | |
| CN111432119B (en) | Image shooting method and device, computer readable storage medium and electronic equipment | |
| CN112399188A (en) | Image frame splicing method and device, readable storage medium and electronic equipment | |
| US10452953B2 (en) | Image processing device, image processing method, program, and information recording medium | |
| US20140320404A1 (en) | Image processing device, image processing method, and program | |
| CN112509047A (en) | Image-based pose determination method and device, storage medium and electronic equipment | |
| WO2017209979A1 (en) | Video pinning | |
| CN107329671B (en) | Model display method and device | |
| CN112907755B (en) | Model display method and device in three-dimensional house model | |
| EP4525438A1 (en) | Method and apparatus for displaying picture on head-mounted display device, and electronic device | |
| CN115131528B (en) | Virtual reality scene determination method, device and system | |
| CN111612842A (en) | Method and device for generating pose estimation model | |
| KR102337209B1 (en) | Method for notifying environmental context information, electronic apparatus and storage medium | |
| CN111415386A (en) | Shooting equipment position prompting method and device, storage medium and electronic equipment | |
| CN117237532A (en) | Panorama display method and device for points outside model, equipment and medium | |
| CN113112613B (en) | Model display method and device, electronic equipment and storage medium | |
| CN112328150B (en) | Automatic screenshot method, device and equipment, and storage medium | |
| CN113989470A (en) | Picture display method and device, storage medium and electronic equipment | |
| CN108874141B (en) | Somatosensory browsing method and device | |
| CN108920598B (en) | Panorama browsing method and device, terminal equipment, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right | Effective date of registration:20200921 Address after:100085 Floor 102-1, Building No. 35, West Second Banner Road, Haidian District, Beijing Applicant after:Seashell Housing (Beijing) Technology Co.,Ltd. Address before:300 457 days Unit 5, Room 1, 112, Room 1, Office Building C, Nangang Industrial Zone, Binhai New Area Economic and Technological Development Zone, Tianjin Applicant before:BEIKE TECHNOLOGY Co.,Ltd. | |
| GR01 | Patent grant | ||
| GR01 | Patent grant |