A kind of adaptive VR radio transmitting method based on viewing point predictionTechnical field
It is related to technical field of information transmission, especially a kind of adaptive virtual reality radio transmitting method.
Background technique
VR technical application is in the ascendant at present, and in order to make user obtain good experience, it is necessary that high-definition image, which is shown,Condition.VR head-mounted display apparatus generally needs to be carried out data transmission with cable.This greatly influences user experience, restricts technologyFurther development.Therefore, the demand how solved at present to VR wireless transmission is very urgent.
Traditional VR transmission is based on equivalent rectangular projection algorithm or cube (hexahedron) algorithm.Virtual realityHead-mounted display apparatus (head is aobvious) receives all video informations, for 4K60 frame, the wireless transmission of 8K60 frame or even 4K120 frameFor data volume it is too big.Therefore in order to solve this contradiction, there are two types of paths: taking the compression of more height ratio, or reduces instituteThe transmission quantity needed.
Since the visual field of people is limited, image only has in the certain angle of front and can be viewed by a user, the visual fieldThe content of edge and behind can not be actually noted (as shown in Figure 1).The information that can choose low resolution is passedIt is defeated to save data traffic.This resolving ideas is as follows, the effective field of view of user generally with FoV (Field of View) come intoRow description.Image within the scope of FoV uses high level rate respectively, and edge or even behind are added with the resolution ratio of several low levelsWith transmission.Here it is adaptive VR transmission algorithms.
Consistency between the piecemeal image quality in the visual field for adaptively referring to user and selection transmission here.It utilizesOverall picture block transmission mode can choose the image quality of disparate modules, Adaptive Transmission be realized, such as Fig. 2 and Fig. 3 instituteShow.
The streaming media transmission protocol that this different pictures quality obtains is MPEG-DASH or HLS, and coding mode can beH.264 or H.265.By Adaptive Transmission algorithm, under stationary conditions, preferable laser propagation effect can achieve.
But this mode still remains many problems, most obvious one is exactly, the head rotation of user to bandwidth stillHigh demand is proposed, for example the scheme based on user's form and viewing point piecemeal is respectively necessary for the bandwidth of original 5 times, 3 timesDemand, as shown in Figure 4.Its reason is that adaptive algorithm can all abandon buffered content, and again from serverThe content of New Century Planned Textbook is requested and caches, this will lead to needs and repeats caching partial content, and different degrees of view angle switch occursDelay.This results in the unstability of transmission of video to be increased sharply, and influences picture real-time.
Summary of the invention
It is asked to solve the instable technology that adaptive wireless transmission algorithm is generated when user's head shakes violentTopic, promotes visual effect, and the present invention proposes the adaptive VR based on viewing point prediction aiming at the problems existing in the prior artRadio transmitting method, specific technical solution are a kind of adaptive VR radio transmitting method based on viewing point prediction, featureIt is, comprising:
Initialization system obtains head and shows sensing data content；
After aobvious motion state revert to true value, viewing point prediction is carried out；
Viewing point drop point according to prediction loads the image data near viewing point in advance, and real-time measurement head shows viewing pointMobile speed and direction.
Further, the initialization system includes that read head shows sensor content, obtains initial viewing point and to appearanceThe angular speed assignment at state angle.
Further, the attitude angle includes pitch angle, yaw angle and roll angle.
Further, the viewing point prediction includes by utilizing the aobvious viewing point coordinate position combination Kalman provided of headFiltering algorithm predicts 0.5~1 second later viewing point position.
Further, it is attached to load prediction viewing point position with higher resolution ratio in advance for the viewing point drop point according to predictionClose image data.
Further, further includes:
Aobvious viewing point combines measured value when being moved to next viewing point position, by the image data near viewing point withThe load of highest picture quality.
Further, further includes:
Judge that head shows velocity of rotation in conjunction with measured value, if it is more than limit value that head, which shows velocity of rotation, system is with lower resolutionLoaded and displayed picture.
Compared with prior art, the core content of the method for the present invention is combined using the aobvious viewing point coordinate position provided of headKalman filtering algorithm predicts 0.5~1 second later viewing point position, thus in advance, in a planned way buffered video content.FromAnd the present invention solves caused change dramatically bandwidth demand and adaptive wireless transmission when the shaking of traditional adaptive algorithm headUnstability that algorithm is generated when user's head shakes violent promotes visual effect, realizes and saves bandwidth, high stabilityVR transmission of video demand.Compared to other course control method for use, the present invention is reliable and stable using Kalman filter, and operation is succinctly fastSpeed is able to satisfy real-time requirement, is particularly suitable for VR and transmits this feature of long time continuous working.
Detailed description of the invention
Fig. 1 is VR user visual field schematic diagram；
Fig. 2 is VR aobvious picture partitioned mode schematic diagram；
Fig. 3 is VR aobvious image quality selection mode schematic diagram；
Fig. 4 is that the aobvious head rotation of head leads to bandwidth demand data profile jumpy；
Fig. 5 is that head shows attitude angle schematic diagram；
Fig. 6 is video cache logical schematic of the present invention；
Fig. 7 is that viewing point of the present invention predicts VR Adaptive Transmission flow diagram.
Specific embodiment
With reference to the accompanying drawings and examples, specific embodiments of the present invention will be described in further detail.Implement belowExample is not intended to limit the scope of the invention for illustrating the present invention.
Embodiment 1:
As shown in figure 5, commercial head is aobvious generally to provide one group of attitude angle to describe viewing point position/orientation, it is pitching respectivelyAngle, yaw angle and roll angle (pitch/yaw/roll).Viewing point neighboring area needs to put into most resources to load.InitiallyMoment, the aobvious initial value for providing attitude angle of head,And provide attitude angular velocity and angular acceleration initial value with initializing(in fact can be set as 0 with initializing, can soon revert to true value in engineering).
For tiMoment, it is assumed that knowThe Kalman filtering algorithm of expansion can be passed through(Extended Kalman Filter) is to ti+1The viewing point state at moment carries out one-step prediction.Obtain
The method of discretization can be set into time interval 200ms and carry out once-through operation, and with the prediction knot at this momentFruit loads the surrounding pictures of target direction in advance, plays the role of reinforcement reliability/reduction bandwidth demand.According to reports, 1s is pre-Survey can reduce bandwidth until 80%.
It is knownAccording to EKF, state variable selection is a certain attitude angle plus the aobvious drift value of headThe bivector constituted, such as
Such as
The state equation (measurement and noise) of system is as follows:
∑(ti)=H (tt)X(ti)+V(ti)
The state equation of discretization is as follows:
Xi+1=KI+1, iXi+ΓiUi
Zi=HkXk
It enables
Ui=(dotangle 0)
In formula: dt is the sampling interval, and dotangle is the angle value that sensor exports within the dt time.
As shown in fig. 6, caching can be targetedly controlled after effectively predicting viewing point, Fig. 6 descriptionCache policy based on predicted value.In tiMoment carries out one-step prediction, obtains subsequent time (ti+1) viewing point position.TogetherWhen image near new viewing point is loaded with higher resolution ratio.Measured value is combined when subsequent time really arrives, and will be regardedThe picture of near focal point is with the load of highest picture quality.
Meanwhile it is known that user's head motion state, due to people in head quick rotation eyes can not quickly it is rightCoke, i.e., the scenery during can not seeing clearly rapidly.Multiple operating conditions can be set accordingly, be more than such as certain value in head rotation speedAfterwards, the picture of real-time loading is lower resolution.This will also be effectively relieved head and rotates the inadequate problem of Time Bandwidth rapidly.
Embodiment 2:
Fig. 7 is the description of overall transfer method, in the present embodiment, is first initialized to system at 0 moment, it is therefore an objective toRead head shows sensor content, obtains initial viewing point and to several angular speed assignment.By the operation of a bit of timeAfterwards, head shows motion state and revert to true value.At this moment it can be carried out accurately viewing point prediction, and assist adaptive passIt is defeated.In tiMoment predicts ti+1The viewing point drop point at moment, and buffered in advance this partial content.And if it is judged that at this timeThat carves head moves past certain speed (thinking that user can not correctly focus at this time), it is possible to reduce supplements the content of transmission, savesBandwidth-saving.Until ti+1When moment really arrives, the part do not downloaded of caching is supplemented again with measured value.This algorithm operates in nothingOn line VR head-mounted display apparatus, the VR adaptive wireless transmission of high stability can be effectively realized.