技术领域technical field
本发明涉及虚拟现实技术领域，尤其涉及一种虚拟现实视频的多人分享系统及虚拟现实设备。The invention relates to the technical field of virtual reality, in particular to a virtual reality video multi-person sharing system and virtual reality equipment.
背景技术Background technique
虚拟现实(Virtual Reality，VR)技术是一种可以创建和体验虚拟世界的计算机仿真技术，其利用计算机技术模拟生成沉浸式的三维模拟环境，用户通过特制的显示设备(例如头戴式VR眼镜等)感知该三维模拟环境，并可以在该环境中进行一系列交互操作。Virtual reality (Virtual Reality, VR) technology is a computer simulation technology that can create and experience a virtual world. It uses computer technology to simulate and generate an immersive three-dimensional simulation environment. ) perceive the three-dimensional simulation environment, and can perform a series of interactive operations in the environment.
传统的VR视频播放技术是针对单用户的，即用户通常只进行单人的VR视频播放，并不会与其他用户进行VR音视频上的互动，即现有技术中无法实现多个用户同时在VR应用中观看同一视频的功能，从而影响了多用户互动的体验。The traditional VR video playback technology is aimed at a single user, that is, a user usually only plays a single VR video, and does not interact with other users on VR audio and video, that is, it is impossible for multiple users to play in the same time in the existing technology. The function of watching the same video in VR applications affects the experience of multi-user interaction.
发明内容Contents of the invention
根据现有技术中存在的上述问题，现提供一种虚拟现实视频的多人分享系统及虚拟现实设备的技术方案，旨在满足多用户同步观看虚拟现实视频资源的需求，扩展了虚拟现实技术的适用范围，从而提升用户体验。According to the above-mentioned problems existing in the prior art, a technical solution of a virtual reality video multi-person sharing system and a virtual reality device is now provided, aiming at meeting the needs of multiple users to watch virtual reality video resources synchronously, expanding the scope of virtual reality technology Scope of application, thereby improving user experience.
上述技术方案具体包括：The above-mentioned technical solutions specifically include:
一种虚拟现实视频的多人分享系统，其中，包括多个虚拟现实设备，每个所述虚拟现实设备作为服务端或者接收端，所述服务端与至少一个所述接收端远程连接，并向所连接的所述接收端分享虚拟现实视频；A multi-person sharing system for virtual reality video, including a plurality of virtual reality devices, each of the virtual reality devices is used as a server or a receiving end, and the server is remotely connected to at least one of the receiving ends, and sends The connected receiving end shares the virtual reality video;
所述服务端包括：The server includes:
第一连接单元，用于向所述接收端提供连接信息，以供所述接收端远程连接所述服务端；a first connection unit, configured to provide connection information to the receiving end, so that the receiving end can remotely connect to the server;
第一发送单元，所述接收端连接所述服务端后，向所述服务端发送视频分享请求，所述第一发送单元向发送所述视频分享请求的所有所述接收端统一发送所述虚拟现实视频的连续多段视频缓冲流；The first sending unit, after the receiving end is connected to the server, sends a video sharing request to the server, and the first sending unit uniformly sends the virtual video to all receiving ends that send the video sharing request. Continuous multi-segment video buffer stream of real video;
维护单元，连接所述第一发送单元，于所述维护单元中设置一用户列表，于所述用户列表中，针对每个发送所述视频分享请求的所述接收端分别设置一个接收端状态，每个所述接收端状态的初始状态为待命状态；A maintenance unit, connected to the first sending unit, setting a user list in the maintenance unit, in the user list, setting a receiving end status for each receiving end sending the video sharing request, The initial state of each receiving end state is a standby state;
第一接收单元，连接所述维护单元，用于接收所述接收端的用于表示已经接收到所述视频缓冲流的反馈信息；A first receiving unit, connected to the maintenance unit, for receiving feedback information from the receiving end indicating that the video buffer stream has been received;
第一编辑单元，分别连接所述第一发送单元、所述第一接收单元和所述维护单元；a first editing unit, respectively connected to the first sending unit, the first receiving unit and the maintenance unit;
当所述第一发送单元向一个所述接收端发送一段所述视频缓冲流后，所述第一编辑单元将所述用户列表中对应的所述接收端状态更改为待反馈状态；以及After the first sending unit sends a section of the buffered video stream to one of the receiving terminals, the first editing unit changes the status of the corresponding receiving terminal in the user list to the status to be fed back; and
当所述第一接收单元接收到一个所述接收端发送的所述反馈信息后，所述编辑单元将所述用户列表中对应的所述接收端状态更改为所述待命状态；After the first receiving unit receives the feedback information sent by the receiving end, the editing unit changes the state of the corresponding receiving end in the user list to the standby state;
在发送完一段所述视频缓冲流后，所述第一发送单元对所述用户列表中的所述接收端状态进行判断，只有当所有所述接收端状态均为所述待命状态时，所述第一发送单元才向发送所述视频分享请求的所有所述接收端统一发送下一段所述视频缓冲流。After sending a segment of the video buffer stream, the first sending unit judges the state of the receiving end in the user list, and only when all the receiving end states are in the standby state, the The first sending unit uniformly sends the next segment of the video buffer stream to all the receiving ends that send the video sharing request.
优选的，该多人分享系统，其中，所述连接信息包括所述服务端的网络连接地址和网络端口号。Preferably, in the multi-person sharing system, the connection information includes the network connection address and network port number of the server.
优选的，该多人分享系统，其中，所述第一发送单元通过UDP协议或者RTP协议将所述视频缓冲流发送至所述接收端。Preferably, in the multi-person sharing system, the first sending unit sends the buffered video stream to the receiving end through UDP protocol or RTP protocol.
优选的，该多人分享系统，其中，所述第一发送单元具体包括：Preferably, the multi-person sharing system, wherein the first sending unit specifically includes:
接收模块，用于接收所述接收端发送的所述视频分享请求；a receiving module, configured to receive the video sharing request sent by the receiving end;
发送模块，连接所述接收模块，用于向发送所述视频分享请求的所述接收端发送所述虚拟现实视频的连续多段的所述视频缓冲流；The sending module is connected to the receiving module, and is used to send the video buffer stream of multiple consecutive segments of the virtual reality video to the receiving end that sends the video sharing request;
判断模块，连接所述发送模块，在所述发送模块发送完一段所述视频缓冲流后，所述判断模块判断所述用户列表中的所有所述接收端状态是否均处于所述待命状态，并输出一第一判断结果；The judging module is connected to the sending module, and after the sending module sends a section of the video buffer stream, the judging module judges whether all the receiving end states in the user list are in the standby state, and Outputting a first judgment result;
所述发送模块根据所述第一判断结果，在存在有所述接收端状态未处于所述待命状态时向发送所述视频分享请求的所有所述接收端统一发送一段空白的所述视频缓冲流，以及在所有所述接收端状态均处于所述待命状态时向发送所述视频分享请求的所有所述接收端统一发送关联于所述虚拟现实视频的下一段所述视频缓冲流。According to the first judgment result, the sending module uniformly sends a blank video buffer stream to all the receiving ends that send the video sharing request when the state of the receiving end is not in the standby state , and uniformly send the next video buffer stream associated with the virtual reality video to all the receiving ends that send the video sharing request when all the receiving ends are in the standby state.
优选的，该多人分享系统，所述接收端包括：Preferably, in the multi-person sharing system, the receiving end includes:
第二连接单元，用于根据接收到的所述连接信息与所述服务端建立远程连接；a second connection unit, configured to establish a remote connection with the server according to the received connection information;
第二发送单元，用于向所述服务端发送所述视频分享请求；a second sending unit, configured to send the video sharing request to the server;
第二接收单元，用于接收所述服务端发送的连续多段的所述视频缓冲流；The second receiving unit is configured to receive the video buffer stream of multiple consecutive segments sent by the server;
第一判断单元，分别连接所述第二接收单元和所述第二发送单元，在所述第二接收单元接收到一段所述视频缓冲流后，所述第一判断单元对所述视频缓冲流的性质进行判断，并输出一相应的第二判断结果；The first judging unit is respectively connected to the second receiving unit and the second sending unit, and after the second receiving unit receives a segment of the video buffer stream, the first judging unit performs a check on the video buffer stream The nature of the judgment is made, and a corresponding second judgment result is output;
在所述第二判断结果表示所述视频缓冲流为空白的视频缓冲流时，所述第二发送单元在一预设的第一时间间隔后再次向所述服务端发送所述视频分享请求；以及When the second judgment result indicates that the video buffer stream is a blank video buffer stream, the second sending unit sends the video sharing request to the server again after a preset first time interval; as well as
在所述第二判断结果表示所述视频缓冲流为关联于所述虚拟现实视频的视频缓冲流时，所述第二发送单元向所述服务端发送所述反馈信息。When the second determination result indicates that the buffered video stream is a buffered video stream associated with the virtual reality video, the second sending unit sends the feedback information to the server.
优选的，该多人分享系统，其中，所述接收端还包括：Preferably, the multi-person sharing system, wherein the receiving end further includes:
显示单元，分别连接所述第一判断单元和所述第二接收单元，用于根据所述第二判断结果，显示所述第二接收单元接收到的关联于所述虚拟现实视频的视频缓冲流。a display unit, respectively connected to the first judging unit and the second receiving unit, for displaying the video buffer stream associated with the virtual reality video received by the second receiving unit according to the second judging result .
优选的，该多人分享系统，其中，所述服务端还包括：Preferably, the multi-person sharing system, wherein the server also includes:
第二判断单元，分别连接所述所述维护单元和所述第一接收单元，用于针对所述用户列表中的每个所述接收端，判断所述接收端是否已经掉线，并输出相应的第三判断结果；The second judging unit is connected to the maintenance unit and the first receiving unit respectively, and is used for judging whether the receiving end has been disconnected for each receiving end in the user list, and outputs a corresponding The third judgment result of
第二编辑单元，分别连接所述维护单元和所述第二判断单元，用于根据所述第三判断结果，将已经掉线的所述接收端从所述用户列表中剔除。The second editing unit is respectively connected to the maintenance unit and the second judging unit, and is configured to remove the offline receiving end from the user list according to the third judging result.
一种虚拟现实设备，其中，应用在上述的虚拟现实视频的多人分享系统中。A virtual reality device, which is applied in the above-mentioned multi-person sharing system of virtual reality video.
上述技术方案的有益效果是：提供一种虚拟现实视频的多人分享系统，能够满足多用户同步观看虚拟现实视频资源的需求，扩展了虚拟现实技术的适用范围，从而提升用户体验。The beneficial effect of the above technical solution is to provide a virtual reality video multi-person sharing system, which can meet the needs of multiple users to watch virtual reality video resources synchronously, expand the scope of application of virtual reality technology, and improve user experience.
附图说明Description of drawings
图1是本发明的较佳的实施例中，一种虚拟现实视频的多人分享系统的总体结构示意图；Fig. 1 is a schematic diagram of the overall structure of a virtual reality video multi-person sharing system in a preferred embodiment of the present invention;
图2是本发明的较佳的实施例中，于图1的基础上，第一发送单元的具体结构示意图；FIG. 2 is a schematic diagram of the specific structure of the first sending unit based on FIG. 1 in a preferred embodiment of the present invention;
图3是本发明的较佳的实施例中，应用在多人分享系统中的虚拟现实设备的结构示意图。Fig. 3 is a schematic structural diagram of a virtual reality device applied in a multi-person sharing system in a preferred embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
需要说明的是，在不冲突的情况下，本发明中的实施例及实施例中的特征可以相互组合。It should be noted that, in the case of no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.
下面结合附图和具体实施例对本发明作进一步说明，但不作为本发明的限定。The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but not as a limitation of the present invention.
根据现有技术中存在的上述问题，现提供一种虚拟现实视频的多人分享系统，该多人分享系统中具体包括多个虚拟现实设备，每个虚拟现实设备均可以作为服务端或者接收端。图1中示出了该多人分享系统的具体构成。According to the above-mentioned problems in the prior art, a multi-person sharing system for virtual reality video is now provided. The multi-person sharing system specifically includes a plurality of virtual reality devices, and each virtual reality device can be used as a server or a receiving end . FIG. 1 shows the specific composition of the multi-person sharing system.
上述服务端A与至少一个接收端B远程连接(图1中仅示出一个接收端B)，上述服务端A中包括：The above-mentioned server A is remotely connected to at least one receiver B (only one receiver B is shown in Figure 1), and the above-mentioned server A includes:
第一连接单元A1，用于向接收端B提供连接信息，以供接收端B远程连接服务端A；The first connection unit A1 is configured to provide connection information to the receiving end B, so that the receiving end B can remotely connect to the server A;
第一发送单元A2，接收端B连接服务端A后，向服务端A发送视频分享请求，第一发送单元A2向发送视频分享请求的所有接收端B统一发送虚拟现实视频的连续多段视频缓冲流；The first sending unit A2, after the receiving end B is connected to the server A, sends a video sharing request to the server A, and the first sending unit A2 uniformly sends a continuous multi-segment video buffer stream of the virtual reality video to all receiving ends B that send the video sharing request ;
维护单元A3，连接第一发送单元A2，于维护单元A3中设置一用户列表，于用户列表中，针对每个发送视频分享请求的接收端B分别设置一个接收端状态，每个接收端状态的初始状态为待命状态；The maintenance unit A3 is connected to the first sending unit A2, and a user list is set in the maintenance unit A3. In the user list, a receiving end state is respectively set for each receiving end B that sends a video sharing request, and the state of each receiving end is The initial state is standby state;
第一接收单元A4，连接维护单元A3，用于接收接收端B的用于表示已经接收到视频缓冲流的反馈信息；The first receiving unit A4 is connected to the maintenance unit A3, and is used to receive feedback information from the receiving end B indicating that the video buffer stream has been received;
第一编辑单元A5，分别连接第一发送单元A2、第一接收单元A4和维护单元A3；The first editing unit A5 is respectively connected to the first sending unit A2, the first receiving unit A4 and the maintenance unit A3;
当第一发送单元A2向一个接收端B发送一段视频缓冲流后，第一编辑单元A5将用户列表中对应的接收端状态更改为待反馈状态；以及After the first sending unit A2 sends a section of video buffer stream to a receiving terminal B, the first editing unit A5 changes the status of the corresponding receiving terminal in the user list to the status to be fed back; and
当第一接收单元A4接收到一个接收端B发送的反馈信息后，第一编辑单元A5将用户列表中对应的接收端状态更改为待命状态；After the first receiving unit A4 receives the feedback information sent by a receiving terminal B, the first editing unit A5 changes the state of the corresponding receiving terminal in the user list to the standby state;
在发送完一段视频缓冲流后，第一发送单元A2对用户列表中的接收端状态进行判断，只有当所有接收端状态均为待命状态时，第一发送单元A2才向发送视频分享请求的所有接收端B统一发送下一段视频缓冲流。After sending a section of video buffer stream, the first sending unit A2 judges the state of the receiving end in the user list, and only when all receiving end states are in standby state, the first sending unit A2 sends a video sharing request to all Receiver B sends the next video buffer stream uniformly.
具体地，本实施例中，作为需要分享VR视频的VR设备，其首先会在本地建立一个服务器，该服务器即作为本技术方案中的服务端A，上述第一连接单元A1用于向各个接收端B提供服务端A的连接信息，以供有需要获取分享的VR视频的接收端远程连接服务端A。上述连接信息可以为例如服务端A的网络连接地址以及网络端口号，连接信息可以由服务端A直接发给各接收端B，也可以由服务端A进行广播，由接收端B接收广播信息并解析得到，还可以通过第三方数据传输途径将连接信息发送至各接收端B(例如通过扫描二维码或者输入特定网址获取等)，本技术方案中并不限制接收端B获取连接信息的方式，下文中也不再赘述。Specifically, in this embodiment, as a VR device that needs to share VR video, it will first establish a server locally, which is the server A in this technical solution, and the above-mentioned first connection unit A1 is used to send Terminal B provides the connection information of server A, so that the receiving terminal that needs to obtain the shared VR video can remotely connect to server A. The above connection information can be, for example, the network connection address and network port number of the server A, the connection information can be directly sent by the server A to each receiver B, or can be broadcast by the server A, and the receiver B can receive the broadcast information and send After analysis, the connection information can also be sent to each receiving terminal B through a third-party data transmission channel (for example, by scanning a QR code or entering a specific URL, etc.). This technical solution does not limit the way that receiving terminal B obtains connection information , which will not be described further below.
本实施例中，接收端B获取连接信息后，根据该连接信息与服务端A之间建立远程连接。随后有视频分享需求的接收端B就会向服务端A发送视频分享请求，以请求服务端A向其发送可以分享的VR视频。服务端A中的维护单元A3建立并维护一张用户列表，该用户列表中记载了所有向服务端A发送视频分享请求的接收端B(即所有想要分享视频的接收端B)。进一步地，在用户列表中记载了每个接收端B当前的接收端状态，该接收端状态用于表示对应的接收端B当前是否成功接收到被分享的VR视频。In this embodiment, after receiving the connection information, the receiving end B establishes a remote connection with the server A according to the connection information. Then the receiving end B that has a video sharing requirement will send a video sharing request to the server A, so as to request the server A to send it a VR video that can be shared. The maintenance unit A3 in the server A establishes and maintains a user list, which records all receivers B who send video sharing requests to the server A (that is, all receivers B who want to share videos). Further, the current receiving end status of each receiving end B is recorded in the user list, and the receiving end status is used to indicate whether the corresponding receiving end B has successfully received the shared VR video.
具体地，用户列表中的每个接收端B初始的接收端状态均为待命状态。所谓待命状态，是指接收端B已经准备好接收VR视频。接收端状态还包括一个待反馈状态，是指服务端A已将需要共享的VR视频发送至接收端B，等待接收端B的反馈，在待反馈状态下，服务端A并不确定接收端B是否成功接收到VR视频。Specifically, the initial state of each receiving end B in the user list is the standby state. The so-called standby state means that the receiver B is ready to receive VR video. The status of the receiving end also includes a state of waiting for feedback, which means that server A has sent the VR video to be shared to receiving end B, and is waiting for the feedback of receiving end B. In the state of waiting for feedback, server A is not sure about receiving end B. Whether the VR video is successfully received.
本实施例中，上述第一发送单元A2用于向各接收端B发送需要分享的VR视频。具体地，处于带宽和传输时效性等因素的考虑，一个VR视频可能由多段连续的视频缓冲流，因此第一发送单元A2在实际发送时，需要连续发送VR视频的多段视频缓冲流。In this embodiment, the above-mentioned first sending unit A2 is used to send the VR video to be shared to each receiving end B. Specifically, considering factors such as bandwidth and transmission timeliness, a VR video may consist of multiple consecutive video buffer streams, so the first sending unit A2 needs to continuously send multiple video buffer streams of the VR video during actual transmission.
由于用户列表中每个接收端B的接收端状态初始为待命状态，因此第一发送单元A2直接向各接收端B发送需要分享的VR视频的第一段视频缓冲流。Since the receiving end state of each receiving end B in the user list is initially on standby, the first sending unit A2 directly sends the first video buffer stream of the VR video to be shared to each receiving end B.
当第一发送单元A2发送完第一段视频缓冲流后，第一编辑单元将用户列表中的所有接收端B的接收端状态均修改为待反馈状态。随后，服务端A通过第一接收单元A4接收各接收端B的反馈信息，第一编辑单元A5根据第一接收单元A4接收到的反馈信息，将接收到反馈信息的对应的接收端B的接收端状态更改为待命状态。After the first sending unit A2 finishes sending the first segment of video buffer stream, the first editing unit modifies the receiving end status of all receiving end B in the user list to the state to be fed back. Subsequently, the server A receives the feedback information of each receiving terminal B through the first receiving unit A4, and the first editing unit A5 converts the corresponding receiving terminal B receiving the feedback information according to the feedback information received by the first receiving unit A4. The terminal state changes to standby state.
则第一发送单元A2在发送下一段视频缓冲流之前先读取用户列表并进行判断：只有当用户列表中的所有接收端状态均为待命状态(即表示所有接收端B均已接收到上一段已发的视频缓冲流)，第一发送单元A2才会向所有接收端B发送下一段视频缓冲流。只要用户列表中存在至少一个接收端状态为待反馈状态，第一发送单元A2就不发送下一段视频缓冲流。Then the first sending unit A2 reads the user list and judges before sending the next section of video buffer stream: only when all receiving end states in the user list are standby (that is, it means that all receiving end B have received the previous section sent video buffer stream), the first sending unit A2 will send the next video buffer stream to all receivers B. As long as there is at least one receiver in the user list whose status is waiting for feedback, the first sending unit A2 will not send the next video buffer stream.
上述视频分享的方法能够保证每个用户具有相同的观看进度，即具有同时延迟和同时卡顿，因此能够在多用户同时观看VR视频时能够分享相同的观看体验。The above video sharing method can ensure that each user has the same viewing progress, that is, has simultaneous delay and simultaneous freezing, so it can share the same viewing experience when multiple users watch VR videos at the same time.
本发明的较佳的实施例中，上述第一发送单元A2通过用户数据报协议(UserDatagram Protocol，UDP)或者实时传输协议(Real-time Transport Protocol，RTP)将视频缓冲流发送至各接收端B。当然，在本技术方案中，上述第一发送的那元A2还可以采用其他适宜的数据传输协议来传输VR视频的视频缓冲流，在此不再赘述。In a preferred embodiment of the present invention, the above-mentioned first sending unit A2 sends the video buffer stream to each receiving end B through User Datagram Protocol (UserDatagram Protocol, UDP) or Real-time Transport Protocol (Real-time Transport Protocol, RTP) . Of course, in this technical solution, the first sending element A2 may also use other suitable data transmission protocols to transmit the video buffer stream of the VR video, which will not be repeated here.
本发明的较佳的实施例中，如图2中所示，上述第一发送单元A2具体包括：In a preferred embodiment of the present invention, as shown in FIG. 2, the above-mentioned first sending unit A2 specifically includes:
接收模块A21，用于接收接收端发送的视频分享请求；The receiving module A21 is used to receive the video sharing request sent by the receiving end;
发送模块A22，连接接收模块A21，用于向发送视频分享请求的接收端B发送虚拟现实视频的连续多段的视频缓冲流；The sending module A22 is connected to the receiving module A21, and is used to send a continuous multi-segment video buffer stream of the virtual reality video to the receiving end B that sends the video sharing request;
判断模块A23，连接发送模块A22，在发送模块发送完一段视频缓冲流后，判断模块A23判断用户列表中的所有接收端状态是否均处于待命状态，并输出一第一判断结果；Judging module A23 is connected to sending module A22. After the sending module has sent a section of video buffer stream, judging module A23 judges whether all receiving end states in the user list are in standby state, and outputs a first judgment result;
发送模块A22根据第一判断结果，在存在有接收端状态未处于待命状态时向发送视频分享请求的所有接收端B统一发送一段空白的视频缓冲流，以及在所有接收端状态均处于待命状态时向发送视频分享请求的所有接收端B统一发送关联于虚拟现实视频的下一段视频缓冲流。According to the first judgment result, the sending module A22 sends a blank video buffer stream to all receivers B sending the video sharing request when there are receivers that are not in the standby state, and when all the receivers are in the standby state. The next video buffer stream associated with the virtual reality video is uniformly sent to all receivers B that send the video sharing request.
具体地，虽然如上文中所述，第一发送单元A2只有在所有接收端状态均为待命状态时才向各接收端B发送下一段视频缓冲流，但是在实际进行视频分享的过程中，第一发送单元A2是在不断发送视频缓冲流的，可以看作每隔一预设的发送间隔就发送下一段缓冲流。因此为了实现上述同时延迟同时卡顿的目的，本技术方案设计如下：Specifically, although as mentioned above, the first sending unit A2 sends the next video buffer stream to each receiving end B only when all receiving end states are in the standby state, but in the actual process of video sharing, the first The sending unit A2 is constantly sending the buffered video stream, which can be regarded as sending the next buffered stream every preset sending interval. Therefore, in order to achieve the above-mentioned purpose of simultaneous delay and simultaneous freeze, this technical solution is designed as follows:
当用户列表中存在有接收端状态未处于待命状态(即处理待反馈状态)时，发送模块A22依然会向所有接收端B统一发送一段视频缓冲流，但是这段视频缓冲流是空白的，并没有任何画面，也不关联于正在分享的VR视频，纯粹是用于填充因为上述判断导致的视频延迟时刻。When there is a receiving end state in the user list that is not in the standby state (that is, the processing pending feedback state), the sending module A22 will still uniformly send a section of video buffer stream to all receiving end B, but this video buffer stream is blank, and There is no picture, and it is not related to the VR video being shared. It is purely used to fill in the video delay caused by the above judgment.
当用户列表中的所有接收端状态均处于待命状态时，发送模块A22才会统一向各接收端B发送下一段视频缓冲流，以继续发送正在分享的VR视频。When all the receivers in the user list are in the standby state, the sending module A22 will uniformly send the next video buffer stream to each receiver B to continue sending the VR video being shared.
进一步地，本发明的较佳的实施例中，仍然如图1中所示，上述接收端B具体包括：Further, in a preferred embodiment of the present invention, still as shown in FIG. 1, the above-mentioned receiving end B specifically includes:
第二连接单元B1，用于根据接收到的连接信息与服务端建立远程连接；The second connection unit B1 is configured to establish a remote connection with the server according to the received connection information;
第二发送单元B2，用于向服务端发送视频分享请求；The second sending unit B2 is used to send a video sharing request to the server;
第二接收单元B3，用于接收服务端发送的连续多段的视频缓冲流；The second receiving unit B3 is used to receive the continuous multi-segment video buffer stream sent by the server;
第一判断单元B4，分别连接第二接收单元B3和第二发送单元B2，在第二接收单元B3接收到一段视频缓冲流后，第一判断单元B4对视频缓冲流的性质进行判断，并输出一相应的第二判断结果；The first judging unit B4 is connected to the second receiving unit B3 and the second sending unit B2 respectively. After the second receiving unit B3 receives a video buffer stream, the first judging unit B4 judges the nature of the video buffer stream and outputs a corresponding second judgment result;
在第二判断结果表示视频缓冲流为空白的视频缓冲流时，第二发送单元B2在一预设的第一时间间隔后再次向服务端A发送视频分享请求；以及When the second judgment result indicates that the video buffer stream is a blank video buffer stream, the second sending unit B2 sends the video sharing request to the server A again after a preset first time interval; and
在第二判断结果表示视频缓冲流为关联于虚拟现实视频的视频缓冲流时，第二发送单元B2向服务端A发送反馈信息。When the second judgment result indicates that the buffered video stream is a buffered video stream associated with virtual reality video, the second sending unit B2 sends feedback information to the server A.
上述服务端A下发的空白的视频缓冲流实际为服务端A下发的中断信号，用于告知接收端B此时尚有其他接收端B未成功进行响应。上述空白的视频缓冲流可由服务端A依据实际情况(是否存在有接收端B未发送反馈信号)直接进行发送。The above-mentioned blank video buffer stream sent by server A is actually an interrupt signal sent by server A, which is used to inform receiver B that there are other receivers B that have failed to respond at this time. The above-mentioned blank video buffer stream can be directly sent by the server A according to the actual situation (whether there is a feedback signal not sent by the receiving end B).
上述预设的第一时间间隔可由实际应用场景和网络状态自动或者手动进行调整，其优选的数值可以为例如100ms，也可以为其他适宜的数值。下文中不再赘述。The above preset first time interval may be adjusted automatically or manually according to actual application scenarios and network status, and its preferred value may be, for example, 100 ms, or other appropriate values. No further details will be given below.
具体地，本实施例中，接收端B根据服务端A发送的视频缓冲流确定是否需要向服务端A发送反馈信息：Specifically, in this embodiment, the receiver B determines whether to send feedback information to the server A according to the video buffer stream sent by the server A:
若接收端B接收到的视频缓冲流为空白的视频缓冲流，此时接收端B判断尚有其他接收端B未成功接收到上一段视频缓冲流。此时接收端B并不会向服务端A发送反馈信息，而是隔一预设的第一时间间隔后再向服务端A重新提出视频分享请求，以请求服务端A再次下发视频缓冲流。If the video buffer stream received by the receiving end B is a blank video buffering stream, at this time, the receiving end B judges that there are other receiving end Bs that have not successfully received the previous segment of the video buffering stream. At this time, the receiver B will not send feedback information to the server A, but will resubmit the video sharing request to the server A after a preset first time interval, so as to request the server A to send the video buffer stream again .
若接收端B接收到的视频缓冲流为关联于当前分享的VR视频的视频缓冲流(即不是空白的视频缓冲流)，此时接收端B会向服务端A发送一个反馈信息，用以表示接收端B已经成功接收到一段有内容的视频缓冲流。If the video buffer stream received by receiver B is the video buffer stream associated with the currently shared VR video (that is, not a blank video buffer stream), then receiver B will send a feedback message to server A to indicate Receiver B has successfully received a buffered video stream with content.
因此，依据上文中所述的，接收端B的运行原理为：接收端持续“接收空白的视频缓冲流→等待一预设的第一时间间隔→向服务端A重新发送视频分享请求→接收空白的视频缓冲流”这一循环过程多次，直到接收端B接收到关联于VR视频的视频缓冲流为止。Therefore, according to the above, the operating principle of the receiving end B is: the receiving end continues to "receive blank video buffer stream → wait for a preset first time interval → resend video sharing request to server A → receive blank The looping process of the "video buffer stream" is repeated multiple times until the receiver B receives the video buffer stream associated with the VR video.
进一步地，本发明的较佳的实施例中，仍然如图1中所示，接收端B还包括：Further, in a preferred embodiment of the present invention, still as shown in Figure 1, the receiving end B also includes:
显示单元B5，分别连接第一判断单元B4和第二接收单元B3，用于根据第二判断结果，显示第二接收单元B3接收到的关联于虚拟现实视频的视频缓冲流。The display unit B5 is connected to the first judging unit B4 and the second receiving unit B3 respectively, and is used to display the video buffer stream associated with the virtual reality video received by the second receiving unit B3 according to the second judging result.
具体地，如上文中所述，接收端B有可能会接收到服务端A发送的两种视频缓冲流：空白的视频缓冲流以及有内容的并关联于VR视频的视频缓冲流。当接收端B接收到空白的视频缓冲流时，其不通过显示单元B5播放该视频缓冲流，因此用户在观感上就是VR视频发生了延迟或者卡顿的现象。只有在接收端B接收到关联于VR视频的视频缓冲流时才会通过显示单元B5播放该视频缓冲流，以提供给用户观看分享的VR视频。Specifically, as mentioned above, the receiver B may receive two kinds of video buffer streams sent by the server A: a blank video buffer stream and a content-related video buffer stream associated with the VR video. When the receiver B receives a blank video buffer stream, it does not play the video buffer stream through the display unit B5, so the user perceives that the VR video is delayed or stuck. Only when the receiving terminal B receives the video buffer stream associated with the VR video, the video buffer stream will be played through the display unit B5, so as to provide the user with viewing the shared VR video.
本发明的较佳的实施例中，仍然如图1中所示，上述服务端A还包括：In a preferred embodiment of the present invention, still as shown in Figure 1, the above-mentioned server A also includes:
第二判断单元A6，分别连接维护单元A3和第一接收单元A4，用于针对用户列表中的每个接收端B，判断接收端B是否已经掉线，并输出相应的第三判断结果；The second judging unit A6 is connected to the maintenance unit A3 and the first receiving unit A4 respectively, and is used for judging whether the receiving end B has been disconnected for each receiving end B in the user list, and outputs a corresponding third judgment result;
第二编辑单元A7，分别连接维护单元A3和第二判断单元A6，用于根据第三判断结果，将已经掉线的接收端从用户列表中剔除。The second editing unit A7 is respectively connected to the maintenance unit A3 and the second judging unit A6, and is used for removing the disconnected receiver from the user list according to the third judging result.
具体地，本实施例中，在视频分享的过程中，可能存在一些接收端B因故掉线，而基于上文中所述的工作原理，若不执行某种掉线相关的策略，那么在用户列表中的已经掉线的接收端B所对应的接收端状态永远是待反馈状态，导致下一段视频缓冲流永远无法进行发送，从而导致视频分享失败。Specifically, in this embodiment, during the video sharing process, some receiving end B may be disconnected for some reason, but based on the working principle described above, if a strategy related to disconnection is not implemented, then the user The state of the receiving end corresponding to the offline receiving end B in the list is always in the status of waiting for feedback, resulting in the failure of sending the next video buffer stream forever, resulting in the failure of video sharing.
基于上述问题，现在本技术方案中增加一个掉线删除的机制。具体地，在服务端A中维护一个有关于接收端B的优先级队列，并且针对每个接收端B记录其相关的最近一次接收到的反馈信息的接收时刻，换言之，上述优先级队列中包括接收端以及该接收端最近一次成功响应的时刻。在维护该优先级队列的过程中，服务端A每次成功接收到一个接收端B的反馈信息，就将该接收端B移至上述优先级队列的队尾，同时更新该接收端B最近一次的响应时刻。Based on the above problems, a mechanism for disconnection deletion is now added to the technical solution. Specifically, a priority queue related to receiving terminal B is maintained in server A, and for each receiving terminal B, the receiving time of the latest received feedback information is recorded. In other words, the above priority queue includes The receiving end and the moment when the receiving end successfully responded last time. In the process of maintaining the priority queue, each time the server A successfully receives a feedback information from the receiving end B, it will move the receiving end B to the end of the above priority queue, and update the last time of the receiving end B at the same time. response time.
同时，在第二判断单元A6内预设一个判断周期(例如1s)，在每个判断周期到时，第二判断单元A6对上述优先级队列的队尾的接收端B进行判断，具体判断原理是判断该接收端B最近一次的成功响应时刻距离当前时刻是否已经超过上述判断周期，并将超过判断周期的接收端B确认为已经掉线。相应地，当判断有接收端B掉线时，上述第二判断单元A6输出相应的第三判断结果。Simultaneously, a judgment cycle (such as 1s) is preset in the second judging unit A6, and when each judging cycle arrives, the second judging unit A6 judges the receiving end B of the queue tail of the above-mentioned priority queue, and the specific judging principle It is to judge whether the last successful response time of the receiver B has exceeded the above judgment period from the current time, and confirm that the receiver B that has exceeded the judgment period is offline. Correspondingly, when it is judged that the receiving terminal B is disconnected, the above-mentioned second judging unit A6 outputs a corresponding third judging result.
本发明的较佳的实施例中，本发明的较佳的实施例中，基于上文中所述的虚拟现实视频的多人分享系统，还提供一种虚拟现实设备，其应用在上述多人分享系统中。上文中所述的“服务端”和“接收端”均为该虚拟现实设备的两种不同工作状态。当虚拟现实设备想要作为分享VR视频的源头，则该虚拟现实设备会建立一个服务端来进行视频分享和下发。当虚拟现实设备想要接收其他虚拟现实设备发送来的VR视频，该VR设备会作为接收端来接收被分享的VR视频。因此，本实施例中，如图3中所示，在一个虚拟现实设备中应当同时包括服务端A和接收端B两种工作模式，这两种工作模式的不同的功能模块均可以由虚拟现实设备的硬件处理芯片和存储单元来实现，在此不再赘述。In a preferred embodiment of the present invention, in a preferred embodiment of the present invention, based on the multi-person sharing system of virtual reality video described above, a virtual reality device is also provided, which is applied to the above-mentioned multi-person sharing system. The "server" and "receiver" mentioned above are two different working states of the virtual reality device. When a virtual reality device wants to be used as a source for sharing VR videos, the virtual reality device will establish a server for video sharing and distribution. When a virtual reality device wants to receive VR videos sent by other virtual reality devices, the VR device will act as a receiver to receive the shared VR video. Therefore, in this embodiment, as shown in FIG. 3, two working modes of server A and receiving terminal B should be included in a virtual reality device at the same time, and the different functional modules of these two working modes can be controlled by virtual reality. It is realized by the hardware processing chip and the storage unit of the device, which will not be repeated here.
以上所述仅为本发明较佳的实施例，并非因此限制本发明的实施方式及保护范围，对于本领域技术人员而言，应当能够意识到凡运用本发明说明书及图示内容所作出的等同替换和显而易见的变化所得到的方案，均应当包含在本发明的保护范围内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the implementation and protection scope of the present invention. For those skilled in the art, they should be able to realize that all equivalents made by using the description and illustrations of the present invention The solutions obtained by replacement and obvious changes shall all be included in the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810242195.9ACN108667795B (en) | 2018-03-22 | 2018-03-22 | Virtual reality video multi-person sharing system and virtual reality equipment |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810242195.9ACN108667795B (en) | 2018-03-22 | 2018-03-22 | Virtual reality video multi-person sharing system and virtual reality equipment |
| Publication Number | Publication Date |
|---|---|
| CN108667795Atrue CN108667795A (en) | 2018-10-16 |
| CN108667795B CN108667795B (en) | 2022-11-22 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810242195.9AExpired - Fee RelatedCN108667795B (en) | 2018-03-22 | 2018-03-22 | Virtual reality video multi-person sharing system and virtual reality equipment |
| Country | Link |
|---|---|
| CN (1) | CN108667795B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116501208A (en)* | 2023-04-11 | 2023-07-28 | 怀化学院 | An immersive landscape design method and system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110119392A1 (en)* | 2009-11-19 | 2011-05-19 | Shamilian John H | Method And Apparatus For Providing Collaborative Interactive Video Streaming |
| CN102638443A (en)* | 2011-02-15 | 2012-08-15 | 中兴通讯股份有限公司 | Method and system for sharing streaming media resources, and device with DLNA (Digital Living Network Alliance) function |
| US20120233644A1 (en)* | 2007-06-05 | 2012-09-13 | Bindu Rama Rao | Mobile device capable of substantially synchronized sharing of streaming media with other devices |
| CN103905483A (en)* | 2012-12-26 | 2014-07-02 | 腾讯科技（深圳）有限公司 | Audio and video sharing method, equipment and system |
| CN103974135A (en)* | 2014-05-07 | 2014-08-06 | 广州骏视信息科技有限公司 | Video sharing method and system |
| CN104125476A (en)* | 2013-04-28 | 2014-10-29 | 腾讯科技（深圳）有限公司 | Video playing method and device |
| CN105406999A (en)* | 2015-11-20 | 2016-03-16 | 福建星网锐捷安防科技有限公司 | Multi-data distribution method and distribution equipment |
| CN106358065A (en)* | 2016-08-26 | 2017-01-25 | 乐视控股（北京）有限公司 | Multimedia information processing method and system |
| CN206451132U (en)* | 2017-02-22 | 2017-08-29 | 亿航智能设备（广州）有限公司 | A kind of virtual reality device |
| CN107333146A (en)* | 2017-07-12 | 2017-11-07 | 易视腾科技股份有限公司 | video group viewing control method and system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120233644A1 (en)* | 2007-06-05 | 2012-09-13 | Bindu Rama Rao | Mobile device capable of substantially synchronized sharing of streaming media with other devices |
| US20110119392A1 (en)* | 2009-11-19 | 2011-05-19 | Shamilian John H | Method And Apparatus For Providing Collaborative Interactive Video Streaming |
| CN102638443A (en)* | 2011-02-15 | 2012-08-15 | 中兴通讯股份有限公司 | Method and system for sharing streaming media resources, and device with DLNA (Digital Living Network Alliance) function |
| CN103905483A (en)* | 2012-12-26 | 2014-07-02 | 腾讯科技（深圳）有限公司 | Audio and video sharing method, equipment and system |
| CN104125476A (en)* | 2013-04-28 | 2014-10-29 | 腾讯科技（深圳）有限公司 | Video playing method and device |
| CN103974135A (en)* | 2014-05-07 | 2014-08-06 | 广州骏视信息科技有限公司 | Video sharing method and system |
| CN105406999A (en)* | 2015-11-20 | 2016-03-16 | 福建星网锐捷安防科技有限公司 | Multi-data distribution method and distribution equipment |
| CN106358065A (en)* | 2016-08-26 | 2017-01-25 | 乐视控股（北京）有限公司 | Multimedia information processing method and system |
| CN206451132U (en)* | 2017-02-22 | 2017-08-29 | 亿航智能设备（广州）有限公司 | A kind of virtual reality device |
| CN107333146A (en)* | 2017-07-12 | 2017-11-07 | 易视腾科技股份有限公司 | video group viewing control method and system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116501208A (en)* | 2023-04-11 | 2023-07-28 | 怀化学院 | An immersive landscape design method and system |
| CN116501208B (en)* | 2023-04-11 | 2024-03-08 | 怀化学院 | Immersive landscape design method and system |
| Publication number | Publication date |
|---|---|
| CN108667795B (en) | 2022-11-22 |
| Publication | Publication Date | Title |
|---|---|---|
| CN103237191B (en) | The method of synchronized push audio frequency and video in video conference | |
| CN107846633B (en) | Live broadcast method and system | |
| CN106488265A (en) | A kind of method and apparatus sending Media Stream | |
| CN102802048A (en) | Multi-screen interaction system and method | |
| US12238363B2 (en) | Methods, systems, and media for synchronized media content playback on multiple devices | |
| CN114546308B (en) | Method, device, equipment and storage medium for screen projection of application interface | |
| EP3329670A1 (en) | Virtual video driver bridge system for multi-source collaboration within a web conferencing system | |
| CN110933470B (en) | Video data sharing method | |
| CN113225577A (en) | Live stream processing method, device and system, electronic equipment and storage medium | |
| CN114979692B (en) | Audio and video streaming mode switching method, device and system and storage medium | |
| CN112911325B (en) | Method and device for recovering inter-live-broadcast room connection | |
| CN112291496A (en) | Instant messaging method and system based on content | |
| CN102195955A (en) | Method and corresponding equipment for switching between live broadcast service and time shifting service | |
| CN103856809A (en) | Method, system and terminal equipment for multipoint at the same screen | |
| CN114554277B (en) | Multimedia processing method, device, server and computer readable storage medium | |
| US8379150B2 (en) | Data transmission method and audio/video system capable of splitting and synchronizing audio/video data | |
| CN108667795B (en) | Virtual reality video multi-person sharing system and virtual reality equipment | |
| CN106412646B (en) | Method and device for realizing synchronous playing | |
| TWI697236B (en) | Video conference audio and video sharing method | |
| CN114125576A (en) | Synchronization method and device for multimedia resources, storage medium and electronic device | |
| TW201929551A (en) | Streaming system with backup mechanism and backup method thereof | |
| EP3089459B1 (en) | Apparatus and method for implementing video-on-demand quick switching among multiple screens | |
| JP2009044287A (en) | Video distribution device, distribution video switching method, distribution video switching program, and distribution video switching program recording medium | |
| CN113726534A (en) | Conference control method, conference control device, electronic equipment and storage medium | |
| CN113612728B (en) | Streaming media playing method, transmission equipment and system |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right | Effective date of registration:20221107 Address after:201109 floor 1, building 5, No. 951, Jianchuan Road, Minhang District, Shanghai Applicant after:Shanghai Nayi Technology Co.,Ltd. Address before:200240 Rooms 315 and 317, Building 5, No. 951, Jianchuan Road, Minhang District, Shanghai Applicant before:SHANGHAI PYGMAL INFORMATION TECHNOLOGY CO.,LTD. | |
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20221122 |