The content of the invention
It is an object of the present invention to provide a kind of new solution that can solve the problem that one of above mentioned problem.
According to the first aspect of the invention there is provided a kind of synchronous method for head-mounted display, including：
Obtain virtual reality image and original sound signal corresponding with the virtual reality image；
Determine source of sound corresponding with each original sound signal in the virtual reality image；
Detect that the posture and the head-mounted display of the head-mounted display are believed relative to the position of each source of soundBreath；
The parameter of each original sound signal is adjusted according to the posture and all positional informationsTo present sound signals；
Sound-producing device is controlled to play the present sound signals.
Optionally, the parameter includes at least one of intensity, frequency, tone and tone color.
Optionally, the posture and the head-mounted display of the detection head-mounted display are described relative to eachAlso include after the positional information of source of sound：
Processing is moved to the virtual reality image according to the posture and the positional information, current virtual is obtainedReal world images；
Control display device shows the current virtual real world images.
Optionally, it is described to determine source of sound corresponding with each original sound signal in the virtual reality imageIncluding：
The original sound signal at least including two sound channels is decomposed, the original channel letter of correspondence number of channels is obtainedNumber；
It is determined that with each one-to-one source of sound of original channel signal.
Optionally, it is described according to the ginseng of the posture and all positional informations to each original sound signalNumber, which is adjusted, to be obtained present sound signals and includes：
The parameter of each original channel signal is adjusted respectively according to the posture and each positional informationIt is whole to obtain corresponding current channel signal；
Processing is merged to all current channel signals, the present sound signals are obtained.
According to the second aspect of the invention there is provided a kind of sychronisation for head-mounted display, including：
Acquisition module, for obtaining virtual reality image and original sound signal corresponding with the virtual reality image；
Source of sound determining module, it is corresponding with each original sound signal in the virtual reality image for determiningSource of sound；
Detection module, for detecting that the posture and the head-mounted display of the head-mounted display are described relative to eachThe positional information of source of sound；
Adjusting module, for according to the ginseng of the posture and all positional informations to each original sound signalNumber, which is adjusted, obtains present sound signals；
Sounding control module, for controlling sound-producing device to play the present sound signals.
Optionally, the parameter includes at least one of intensity, frequency, tone and tone color.
Optionally, described device also includes：
Mobile module, for moving place to the virtual reality image according to the posture and the positional informationReason, obtains current virtual real world images；
Display control module, for controlling display device to show the current virtual real world images.
Optionally, the source of sound determining module includes：
Resolving cell, for being decomposed to the original sound signal at least including two sound channels, obtains correspondence channel numberThe original channel signal of amount；
Determining unit, for determining and each one-to-one source of sound of original channel signal.
Optionally, the adjusting module includes：
Adjustment unit, for according to the ginseng of the posture and each positional information to each original channel signalNumber is adjusted respectively obtains corresponding current channel signal；
Combining unit, for merging processing to all current channel signals, obtains the present sound signals.
According to the third aspect of the invention we there is provided a kind of head-mounted display, including it is described according to a second aspect of the present inventionSychronisation.
According to the fourth aspect of the invention there is provided a kind of head-mounted display, including memory and processor, the storageDevice is used for store instruction, and the instruction is described according to a first aspect of the present invention to perform for controlling the processor to be operatedSynchronous method.
The present invention a beneficial effect be, can be by true audio in virtual reality by embodiments of the inventionReduced in environment, and original sound signal can be dynamically adjusted according to information such as position, the postures of user, realized virtual existingSound in real system is synchronous with actual environment, so as to reach the purpose of simulation true environment audio, lifts Consumer's Experience.
By referring to the drawings to the detailed description of the exemplary embodiment of the present invention, further feature of the invention and itsAdvantage will be made apparent from.
Embodiment
The various exemplary embodiments of the present invention are described in detail now with reference to accompanying drawing.It should be noted that：Unless had in additionBody illustrates that the part and the positioned opposite of step, numerical expression and numerical value otherwise illustrated in these embodiments does not limit thisThe scope of invention.
The description only actually at least one exemplary embodiment is illustrative below, never as to the present inventionAnd its any limitation applied or used.
It may be not discussed in detail for technology, method and apparatus known to person of ordinary skill in the relevant, but suitableIn the case of, the technology, method and apparatus should be considered as a part for specification.
In shown here and discussion all examples, any occurrence should be construed as merely exemplary, withoutIt is as limitation.Therefore, other examples of exemplary embodiment can have different values.
It should be noted that：Similar label and letter represents similar terms in following accompanying drawing, therefore, once a certain Xiang YiIt is defined, then it need not be further discussed in subsequent accompanying drawing in individual accompanying drawing.
In order to solve present in prior art when the position of head-mounted display or posture change, user can notThere is provided a kind of synchronization side for head-mounted display to the problem of this change for the sound experience played by head-mounted displayMethod.Wherein, the head-mounted display can be all-in-one or split air conditioner.
Fig. 1 is the flow chart for a kind of implementation of the synchronous method of head-mounted display according to the present invention.
According to Fig. 1, the synchronous method comprises the following steps：
Step S110, obtains virtual reality image and original sound signal corresponding with the virtual reality image.
Virtual reality image can be head-mounted display camera gather in advance or head-mounted display from netDownloaded on network；Original sound signal can be head-mounted display microphone gather in advance or above and below networkCarry, can also be what is gathered in real time by microphone.And have between virtual reality image and original sound signal one a pairThe relation answered, any virtual reality image at least has a corresponding original sound signal.
Order or acquisition original sound are believed in no particular order for the acquisition of virtual reality image and original sound signalNumber and virtual reality image corresponding with the original sound signal.For example can be to be previously stored with reflection virtual reality imageThe table of comparisons of corresponding relation between original sound signal.By the virtual reality image of acquisition, search the table of comparisons with determine withThe corresponding original sound signal of the virtual reality image；Can also be the original sound signal by acquisition, search the table of comparisons withIt is determined that virtual reality image corresponding with the original sound signal.
Further, after execution of step S110, display device can also be controlled to show the virtual reality image.WithReality environment is provided a user, Consumer's Experience is lifted.
Step S120, it is determined that with source of sound corresponding with each original sound signal in virtual reality image.
Specifically, can be the original sound signal for prestoring each source of sound in corresponding virtual reality image, and willCorresponding original sound signal and source of sound are marked, in order to carry out original sound signal and source of sound when performing step S120Matching.
If original sound signal is monophonic signal, then, it is corresponding with the original sound signal in virtual reality imageSource of sound then be one.So, a source of sound corresponding with the original sound signal of the monophonic can be simulated in space,This source of sound simulated is corresponding with the position of the source of sound in virtual reality image.
If original sound signal, which is multi-channel signal, i.e. original sound signal, includes at least two sound channels, original with thisThe corresponding source of sound quantity of voice signal is then identical with the number of channels of the original sound signal, in the case, step S120It may comprise steps of S121-S122：
Step S121, is decomposed to the original sound signal including at least two sound channels, obtains correspondence number of channelsOriginal channel signal.
Further, for including the original sound signal of two-channel or multichannel, according to the quantity of sound channel and definition,Original sound signal can be decomposed into multiple original channel signals, and the quantity and the quantity phase of original channel signal of sound channelDeng.
Step S122, it is determined that with each one-to-one source of sound of original channel signal.
For example each original channel signal can in space be simulated a source of sound, these sounds simulatedSource is corresponded with the sound source position in virtual reality image.
Step S130, detects the positional information of posture and head-mounted display relative to each source of sound of head-mounted display.
The positional information includes：Head-mounted display is relative to the distance of each source of sound, angle, side in virtual reality imageTo etc..
Specifically, in an embodiment of the present invention, for detecting that the attitude transducer of head-mounted display posture is included but notIt is limited to：Inertial Measurement Unit, wearable position capture sensor either head identification sensor etc., wherein inertia measurement listMember at least includes acceleration transducer and gyroscope, can also include geomagnetic sensor.For detect head-mounted display relative toThe airborne sensor of the positional information of each source of sound includes but is not limited to Inertial Measurement Unit or imaged first-class.
Step S140, is adjusted according to the posture and all positional informations to the parameter of each original sound signalTo present sound signals.
The parameter includes at least one of intensity, frequency, tone and tone color.
Specifically, step S140 includes receiving original sound signal, the posture of head-mounted display and head-mounted display phasePositional information for all sources of sound and the parameter of original sound signal is adjusted and present sound signals etc. are exportedProcess, these processes can be completed in the audio synthesizer of head-mounted display system.
Further, can also be relative according to the self-defined posture and head-mounted display for receiving head-mounted display of user's requestIn the frequency of the positional information of all sources of sound, for example can with but be not limited to be to receive once for each second, so, as long as userPosition or posture change, and these data will change, and finally wear the sound that the user of the head-mounted display hearsSound audio will also change therewith, it is possible thereby to improve the real-time of audio change.
In the case that original sound signal is multi-channel signal in correspondence above-described embodiment, step S140 includes following stepRapid S141 and step S142.
Step S141, is adjusted to the parameter of each original channel signal according to posture and all positional informations and obtainsCurrent channel signal.
All current channel signals are merged processing, obtain present sound signals by step S142.
In one particular embodiment of the present invention, the sound-producing device for playing voice signal can be loudspeaker,It can be earphone.In the case where sound-producing device is loudspeaker, all current channel signals can be merged into a current soundMessage number.In the case where sound-producing device is earphone, processing can be merged to all current channels, obtain working as L channelThe present sound signals of preceding voice signal and R channel, and the present sound signals of L channel are exported to left ear earphone, by the right sideThe present sound signals of sound channel are exported to auris dextra earphone.So, during the sound that user's left and right ear is heard is exactly whole environmentThe mixing of multiple sound, and the stereo sound with surrounding sound effect.
According to the change of positional information between the positional information of user, each source of sound, the intensity of part original channel signalIt can weaken, the intensity of part original channel signal can be strengthened, part original channel signal can be due to other original channel signalsDisturb and produce change.This may result in the present sound signals intensity increase of L channel, and the current sound of R channel is believedNumber intensity reduces.
On this basis, embodiments of the invention can also according to the environmental form of virtual reality image (such as cubicle, orPerson's spaciousness scene) sensation for echoing, propagating of same sound is simulated, adjusted automatically by source of sound and the position relationship of sense of hearing sideThe effect of whole sound, by dynamically adjusting some sound (adjustment of such as voice) in some environment (such as cinema).The present inventionEmbodiment can also according to sound dispersion, successively decrease, there occurs object collision (such as during the characteristic, and sound transmission such as echoWall), merge (such as several sound can interact) characteristic voice signal is handled.Embodiments of the invention can be withDifferent weighted gain or decay are carried out to voice signal by distinguishing left and right acoustic channels, so that user's sense of reality is arrived and source of soundBetween position situation of change.
It is trueer to reach if meeting pre-defined original sound signal, it might even be possible to define more attributes to itReal simulation effect.For example have the dialogue of two people in one sound, can to two dialogues in addition to sound attribute in itself,The attribute (such as mood) of definition is assigned, sound can be handled for the attribute of oneself definition as needed afterwards.For exampleThe sound of voice frequency band can also individually be handled, be obscured when cryying, component frequency is drawn high in indignationEtc. reaching more life-like effect.
Step S150, control sound-producing device plays the present sound signals.
The sound-producing device such as can be, but not limited to be the loudspeaker of head-mounted display, earphone either sound equipment electronics setIt is standby.
So, by embodiments of the invention, true audio can be reduced in reality environment, and can rootOriginal sound signal is dynamically adjusted according to information such as the position of user, postures, so that the purpose of simulation true environment audio is reached,Lift Consumer's Experience.
Further, embodiments of the invention can also freely transmit different sound according to the difference of use environment, withAnd from the incoming desired parameter of distinct device, you can the effect for wanting to obtain is realized, so, embodiments of the invention can be extendedUse scope.
Fig. 2 is the flow chart for the another embodiment of the synchronous method of head-mounted display according to the present invention.
The synchronous method of the present invention is after execution of step S130, in addition to the step according to Fig. 2：
Step S210, processing is moved to virtual reality image according to posture and positional information, is obtained current virtual and is showedReal image.
For example, when user is moved to the left, the processing that can be moved right simultaneously to virtual reality image, to realizeCurrent virtual actual environment is synchronous with actual environment.
Step S220, control display device shows current virtual real world images.
So, it is achieved that in reality environment that sound is synchronous with image, gives user more real effect, enter oneStep improves Consumer's Experience.
Corresponding with the above method, present invention also offers a kind of sychronisation for head-mounted display.Fig. 3 is rootFrame principle figure according to the present invention for a kind of implementation structure of the sychronisation of head-mounted display.
According to Fig. 3, the sychronisation 300 include acquisition module 310, source of sound determining module 320, detection module 330,Adjusting module 340 and sounding control module 350.
Above-mentioned acquisition module 310 is used to obtain virtual reality image and original sound corresponding with virtual reality image letterNumber.
Above-mentioned source of sound determining module 320 is used to determine sound corresponding with each original sound signal in virtual reality imageSource.
Above-mentioned detection module 330 is used for the posture and head-mounted display that detect head-mounted display relative to each source of soundPositional information.
Above-mentioned adjusting module 340 is used to carry out the parameter of each original sound signal according to posture and all positional informationsAdjustment obtains present sound signals.Wherein the parameter includes at least one of intensity, frequency, tone and tone color.
Above-mentioned sounding control module 350 is used to control sound-producing device to play present sound signals.
Fig. 4 is the frame principle figure for another implementation structure of the sychronisation of head-mounted display according to the present invention.
According to Fig. 4, the device 300 also includes mobile module 410 and display control module 420.The mobile module 410For moving processing to virtual reality image according to posture and positional information, current virtual real world images are obtained；The displayControl module 420 is used to control display device to show current virtual real world images.
Further, the source of sound determining module 320 includes resolving cell 321 and determining unit 322, the resolving cell 321For being decomposed to the original sound signal at least including two sound channels, the original channel signal of correspondence number of channels is obtained；The determining unit 322 is used to determine and each one-to-one source of sound of original channel signal.
On this basis, the adjusting module 340 includes adjustment unit 341 and combining unit 342, and the adjustment unit 341 is usedCorresponding current sound is obtained in being adjusted respectively to the parameter of each original channel signal according to posture and each position informationRoad signal；The combining unit 342 is used to merge processing to all current channel signals, obtains present sound signals.
Present invention also offers a kind of head-mounted display, on the one hand, the head-mounted display includes being previously described for wearingThe sychronisation 300 of display.
Fig. 5 is the frame principle figure of the implementation structure of the head-mounted display according to another aspect of the present invention.
According to Fig. 5, the head-mounted display 500 includes memory 501 and processor 502, and the memory 501 is used to depositStorage instruction, the instruction is operated to perform the above-mentioned synchronous method for head-mounted display for control processor 502.
The processor 502 is such as can be central processor CPU, Micro-processor MCV.The memory 501 for example includingROM (read-only storage), RAM (random access memory), nonvolatile memory of hard disk etc..
In addition, according to Fig. 5, the head-mounted display 500 also includes interface arrangement 503, input unit 504, shownShowing device 505, communicator 506, loudspeaker 507, microphone 508 etc..Although figure 5 illustrates multiple devices,Head-mounted display of the present invention can only relate to partial devices therein, for example, processor 501, memory 502, loudspeaker 507Deng.
Above-mentioned communicator 506 can for example carry out wired or wireless communication.
Above-mentioned interface arrangement 503 is such as including earphone jack, USB interface.
Above-mentioned input unit 504 is such as can include touch-screen, button.
Above-mentioned display device 505 is, for example, LCDs, touch display screen etc..
The various embodiments described above primary focus describes the difference with other embodiment, but those skilled in the art should be clearChu, the various embodiments described above can be used alone or be combined with each other as needed.
Each embodiment in this specification is described by the way of progressive, identical similar portion between each embodimentPoint cross-reference, what each embodiment was stressed is the difference with other embodiment, but people in the artMember is it should be understood that the various embodiments described above can be used alone or be combined with each other as needed.In addition, for deviceFor embodiment, because it is corresponding with embodiment of the method, so describing fairly simple, related part is implemented referring to methodThe explanation of the corresponding part of example.System embodiment described above is only schematical, wherein being used as separating componentThe module of explanation can be or may not be physically separate.
The present invention can be device, method and/or computer program product.Computer program product can include computerReadable storage medium storing program for executing, containing for making processor realize the computer-readable program instructions of various aspects of the invention.
Computer-readable recording medium can keep and store to perform the tangible of the instruction that equipment is used by instructionEquipment.Computer-readable recording medium for example can be-- but be not limited to-- storage device electric, magnetic storage apparatus, optical storageEquipment, electromagnetism storage device, semiconductor memory apparatus or above-mentioned any appropriate combination.Computer-readable recording mediumMore specifically example (non exhaustive list) includes：Portable computer diskette, hard disk, random access memory (RAM), read-only depositIt is reservoir (ROM), erasable programmable read only memory (EPROM or flash memory), static RAM (SRAM), portableCompact disk read-only storage (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical coding equipment, for example thereonBe stored with instruction punch card or groove internal projection structure and above-mentioned any appropriate combination.It is used herein above to calculateMachine readable storage medium storing program for executing is not construed as instantaneous signal in itself, the electromagnetic wave of such as radio wave or other Free propagations, logicalCross the electromagnetic wave (for example, the light pulse for passing through fiber optic cables) of waveguide or the propagation of other transmission mediums or transmitted by electric wireElectric signal.
Computer-readable program instructions as described herein can be downloaded to from computer-readable recording medium each calculate/Processing equipment, or outer computer is downloaded to or outer by network, such as internet, LAN, wide area network and/or wireless networkPortion's storage device.Network can be transmitted, be wirelessly transferred including copper transmission cable, optical fiber, router, fire wall, interchanger, gatewayComputer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be countedCalculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipmentIn calculation machine readable storage medium storing program for executing.
For perform the computer program instructions that operate of the present invention can be assembly instruction, instruction set architecture (ISA) instruction,Machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programming languagesSource code or object code that any combination is write, programming language of the programming language including object-oriented-such asSmalltalk, C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.ComputerReadable program instructions can perform fully on the user computer, partly perform on the user computer, as one solelyVertical software kit is performed, part is performed or completely in remote computer on the remote computer on the user computer for partOr performed on server.In the situation of remote computer is related to, remote computer can be by network-bag of any kindLAN (LAN) or wide area network (WAN)-be connected to subscriber computer are included, or, it may be connected to outer computer is (such as sharpWith ISP come by Internet connection).In certain embodiments, by using computer-readable program instructionsStatus information carry out personalized customization electronic circuit, such as PLD, field programmable gate array (FPGA) or canProgrammed logic array (PLA) (PLA), the electronic circuit can perform computer-readable program instructions, so as to realize each side of the present inventionFace.
Referring herein to method according to embodiments of the present invention, device (system) and computer program product flow chart and/Or block diagram describes various aspects of the invention.It should be appreciated that each square frame and flow chart of flow chart and/or block diagram and/Or in block diagram each square frame combination, can be realized by computer-readable program instructions.
These computer-readable program instructions can be supplied to all-purpose computer, special-purpose computer or other programmable datasThe processor of processing unit, so as to produce a kind of machine so that these instructions are passing through computer or other programmable datasDuring the computing device of processing unit, work(specified in one or more of implementation process figure and/or block diagram square frame is generatedThe device of energy/action.Can also be the storage of these computer-readable program instructions in a computer-readable storage medium, these refer toOrder causes computer, programmable data processing unit and/or other equipment to work in a specific way, so that, be stored with instructionComputer-readable medium then includes a manufacture, and it is included in one or more of implementation process figure and/or block diagram square frameThe instruction of the various aspects of defined function/action.
Computer-readable program instructions can also be loaded into computer, other programmable data processing units or otherIn equipment so that perform series of operation steps on computer, other programmable data processing units or miscellaneous equipment, to produceRaw computer implemented process, so that performed on computer, other programmable data processing units or miscellaneous equipmentInstruct function/action specified in one or more of implementation process figure and/or block diagram square frame.
Flow chart and block diagram in accompanying drawing show system, method and the computer journey of multiple embodiments according to the present inventionArchitectural framework in the cards, function and the operation of sequence product.At this point, each square frame in flow chart or block diagram can generationOne module of table, program segment or a part for instruction, the module, program segment or a part for instruction are used comprising one or moreIn the executable instruction for realizing defined logic function.In some realizations as replacement, the function of being marked in square frameCan be with different from the order marked in accompanying drawing generation.For example, two continuous square frames can essentially be held substantially in parallelOK, they can also be performed in the opposite order sometimes, and this is depending on involved function.It is also noted that block diagram and/orThe combination of each square frame in flow chart and the square frame in block diagram and/or flow chart, can use function as defined in execution or dynamicThe special hardware based system made is realized, or can be realized with the combination of specialized hardware and computer instruction.It is rightFor those skilled in the art it is well known that, realized by hardware mode, realized by software mode and by software andIt is all of equal value that the mode of combination of hardware, which is realized,.
It is described above various embodiments of the present invention, described above is exemplary, and non-exclusive, andIt is not limited to disclosed each embodiment.In the case of without departing from the scope and spirit of illustrated each embodiment, for this skillMany modifications and changes will be apparent from for the those of ordinary skill in art field.The selection of term used herein, purportThe principle, practical application or the technological improvement to the technology in market of each embodiment are best being explained, or is leading this technologyOther those of ordinary skill in domain are understood that each embodiment disclosed herein.The scope of the present invention is limited by appended claimsIt is fixed.