技术领域technical field
本发明实施例涉及通信技术领域，尤其涉及一种多画面组合的呈现方法及装置。Embodiments of the present invention relate to the field of communication technologies, and in particular, to a method and device for presenting a multi-picture combination.
背景技术Background technique
随着近几年视频会议的发展，多点会议的应用也越来越广泛，但经常会出现会议中的某一个用户会场可能接收到的视频画面的数目多于自身的显示屏幕，尤其是对于远程呈现会议来说，其用户会场中会有多路摄像机、多路活动视频以及多个显示器，这使得接收视频画面数目与显示屏幕数目不匹配的现象更加突出，而为了能达到更为完整的会议呈现，需要在一个显示屏幕上同时观看到多个视频画面，即需要利用多画面组合技术进行多画面组合；具体的，多点会议系统中的多点控制单元（Multipoint Control Unit，以下简称MCU）通过图像拼接和缩放技术，把多路视频图像画面组合在一个画面中，或者，该多画面组合技术也适用于点对点会议，但需要点对点会议中的会场是有多个画面生成以完成多画面组合的会场，则多画面组合由系统中的画面控制设备来完成。一般的，将发送组合画面的一方称为媒体提供方，接收组合画面的一方称为媒体接收方。With the development of video conferencing in recent years, the application of multipoint conferences has become more and more extensive. However, it often happens that a certain user site may receive more video images than its own display screen, especially for For telepresence conferences, there will be multiple cameras, multiple active videos, and multiple monitors in the user site, which makes the mismatch between the number of received video images and the number of display screens more prominent, and in order to achieve a more complete For conference presentation, multiple video images need to be watched simultaneously on one display screen, that is, multi-screen combination technology needs to be used for multi-screen combination; specifically, the multipoint control unit (Multipoint Control Unit, hereinafter referred to as MCU) in the multipoint conference system ) through image splicing and zooming technology to combine multiple video images into one screen, or this multi-screen combination technology is also applicable to point-to-point conferences, but it needs multiple screens to be generated at the site in the point-to-point conference to complete the multi-screen In the combined venue, the multi-screen combination is completed by the screen control device in the system. Generally, the party that sends the combined screen is called the media provider, and the party that receives the combined screen is called the media receiver.
现有技术中，由媒体提供方将设定的多画面组合的视频媒体画面发送给媒体接收方，由媒体接收方直接进行呈现。但是这种多画面组合的视频媒体画面呈现方法缺乏灵活性，影响接收方用户的会议体验度。In the prior art, the media provider sends the set multi-screen combined video media screens to the media receiver, and the media receiver directly presents them. However, this multi-picture combined video media screen presentation method lacks flexibility, which affects the conference experience of the receiving user.
发明内容Contents of the invention
本发明实施例提供一种多画面组合的呈现方法及装置，解决了现有技术中媒体接收方被动接收媒体提供方发送的组合画面导致媒体接收方用户的会议体验度不高的问题。Embodiments of the present invention provide a method and device for presenting multi-screen combination, which solves the problem in the prior art that the media receiver passively receives the combined screen sent by the media provider, resulting in poor meeting experience for the user of the media receiver.
第一方面，本发明实施例提供一种多画面组合的呈现方法，包括：In a first aspect, an embodiment of the present invention provides a method for presenting a multi-picture combination, including:
发送媒体画面组合能力指示信息给媒体接收方；Send media picture combination capability indication information to the media receiver;
接收所述媒体接收方发送的根据所述媒体画面组合能力指示信息确定的媒体画面组成信息；receiving the media picture composition information determined according to the media picture combination capability indication information sent by the media receiver;
根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Perform corresponding operations to the media receiver according to the support capability of the media screen composition information.
结合第一方面，在第一方面的第一种可能的实现方式中，所述媒体画面组成信息包括：至少两路媒体画面的组合信息，或，至少两路媒体画面的组合信息和至少两路媒体画面的布局信息。With reference to the first aspect, in a first possible implementation manner of the first aspect, the media screen composition information includes: combination information of at least two media screens, or combination information of at least two media screens and at least two Layout information of the media screen.
结合第一方面，在第一方面的第二种可能的实现方式中，所述根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作，包括：With reference to the first aspect, in a second possible implementation of the first aspect, performing corresponding operations to the media receiver according to the support capability of the media screen composition information includes:
若支持所述媒体画面组成信息，则发送组合画面的第一媒体参数给所述媒体接收方；接收所述媒体接收方发送的所述组合画面的第二媒体参数；将根据所述媒体画面组成信息、所述第一媒体参数和所述第二媒体参数确定的所述组合画面发送给所述媒体接收方；或者，If the media picture composition information is supported, send the first media parameter of the combined picture to the media receiver; receive the second media parameter of the combined picture sent by the media receiver; information, the combined picture determined by the first media parameter and the second media parameter is sent to the media receiver; or,
若不支持所述媒体画面组成信息，则发送否定应答消息给所述媒体接收方。If the media screen composition information is not supported, a negative response message is sent to the media receiver.
结合第一方面的第二种可能的实施方式，在第一方面的第三种可能的实施方式中，发送媒体画面组合能力指示信息给媒体接收方，包括：With reference to the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, sending the media screen combination capability indication information to the media receiver includes:
发送第一远程呈现的多流控制CLUE媒体通告消息给所述媒体接收方，所述第一CLUE媒体通告消息中携带所述媒体画面组合能力指示信息；Sending a first telepresence multi-stream control CLUE media notification message to the media receiver, where the first CLUE media notification message carries the media screen combination capability indication information;
接收所述媒体接收方发送的根据所述媒体画面组合能力指示信息确定的媒体画面组成信息，包括：Receiving the media picture composition information determined according to the media picture combination capability indication information sent by the media receiver includes:
接收所述媒体接收方发送的第一CLUE媒体配置消息，所述第一CLUE媒体配置消息中携带所述媒体画面组成信息。Receive a first CLUE media configuration message sent by the media receiver, where the first CLUE media configuration message carries the media screen composition information.
结合第一方面的第二种可能的实施方式，在第一方面的第四种可能的实施方式中，发送媒体画面组合能力指示信息给媒体接收方，包括：With reference to the second possible implementation manner of the first aspect, in the fourth possible implementation manner of the first aspect, sending the media screen combination capability indication information to the media receiver includes:
发送第一会话描述协议SDP请求消息给所述媒体接收方，所述第一SDP请求消息中携带所述媒体画面组合能力指示信息；则Sending a first Session Description Protocol SDP request message to the media receiver, where the first SDP request message carries the media screen combination capability indication information; then
接收所述媒体接收方发送的根据所述媒体画面组合能力指示信息确定的媒体画面组成信息，包括：Receiving the media picture composition information determined according to the media picture combination capability indication information sent by the media receiver includes:
接收所述媒体接收方发送的第一SDP响应消息，所述第一SDP响应消息中携带所述媒体画面组成信息。Receive a first SDP response message sent by the media receiver, where the first SDP response message carries the media screen composition information.
结合第一方面的第二种可能的实施方式，在第一方面的第五种可能的实施方式中，发送媒体画面组合能力指示信息给媒体接收方，包括：With reference to the second possible implementation manner of the first aspect, in the fifth possible implementation manner of the first aspect, sending the media screen combination capability indication information to the media receiver includes:
发送媒体画面组合能力指示信息以及媒体画面的缩略图给门户网站，以使所述媒体接收方从所述门户网站获取所述媒体画面组合能力指示信息和所述媒体画面的缩略图；则Sending the indication information of the media screen combination capability and the thumbnail of the media screen to the portal website, so that the media receiver obtains the indication information of the media screen combination capability and the thumbnail of the media screen from the portal website;
接收所述媒体接收方发送的根据所述媒体画面组合能力指示信息确定的媒体画面组成信息，包括：Receiving the media picture composition information determined according to the media picture combination capability indication information sent by the media receiver includes:
从所述门户网站获取所述媒体接收方发送的根据所述媒体画面组合能力指示信息和所述媒体画面的缩略图确定的所述媒体画面组成信息。Obtaining the media screen composition information sent by the media receiver and determined according to the media screen combination capability indication information and the thumbnail of the media screen from the portal website.
结合第一方面的第三种可能的实施方式，在第一方面的第六种可能的实施方式中，发送远程呈现的多流控制CLUE媒体通告消息给所述媒体接收方之前，还包括：With reference to the third possible implementation manner of the first aspect, in the sixth possible implementation manner of the first aspect, before sending the telepresence multi-stream control CLUE media notification message to the media receiver, the method further includes:
发送第二SDP请求消息给所述媒体接收方，所述第二SDP请求消息携带支持CLUE协议指示；Sending a second SDP request message to the media receiver, where the second SDP request message carries an indication of supporting the CLUE protocol;
接收所述媒体接收方发送的第二SDP响应消息，所述第二SDP响应消息携带支持所述CLUE协议指示。Receive a second SDP response message sent by the media receiver, where the second SDP response message carries an indication of supporting the CLUE protocol.
结合第一方面的第三种可能的实施方式或第一方面的第四种可能的实施方式，在第一方面的第七种可能的实施方式中，发送组合画面的第一媒体参数给所述媒体接收方，包括：With reference to the third possible implementation manner of the first aspect or the fourth possible implementation manner of the first aspect, in a seventh possible implementation manner of the first aspect, sending the first media parameter of the combined picture to the Media recipients, including:
发送第三SDP请求消息给所述媒体接收方，所述第三SDP请求消息中携带所述组合画面的第一媒体参数；则sending a third SDP request message to the media receiver, where the third SDP request message carries the first media parameter of the combined picture; then
接收所述媒体接收方发送的所述组合画面的第二媒体参数，包括：Receiving the second media parameter of the combined picture sent by the media receiver includes:
接收所述媒体接收方发送的第三SDP响应消息，所述第三SDP响应消息中携带所述组合画面的第二媒体参数。Receive a third SDP response message sent by the media receiver, where the third SDP response message carries the second media parameter of the combined screen.
结合第一方面的第五种可能的实施方式，在第一方面的第八种可能的实施方式中，发送组合画面的第一媒体参数给所述媒体接收方，包括：With reference to the fifth possible implementation manner of the first aspect, in the eighth possible implementation manner of the first aspect, sending the first media parameter of the combined screen to the media receiver includes:
发送所述组合画面的第一媒体参数给门户网站，以使所述媒体接收方从所述门户网站获取所述组合画面的所述第一媒体参数；则sending the first media parameter of the combined picture to the portal website, so that the media receiver can obtain the first media parameter of the combined picture from the portal website; then
接收所述媒体接收方发送的所述组合画面的第二媒体参数，包括：Receiving the second media parameter of the combined picture sent by the media receiver includes:
从门户网站获取所述媒体接收方发送的所述组合画面的第二媒体参数。Obtaining the second media parameter of the combined screen sent by the media receiver from the portal website.
结合第一方面，在第一方面的第九种可能的实施方式中，根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，还包括：With reference to the first aspect, in the ninth possible implementation manner of the first aspect, after performing corresponding operations to the media receiver according to the support capability of the media screen composition information, the method further includes:
发送媒体画面变化信息给所述媒体接收方；Send media screen change information to the media receiver;
接收所述媒体接收方根据所述媒体画面变化信息生成的更新后的第一媒体画面组成信息，所述第一媒体画面组成信息中包括：所述至少两路媒体画面的第一更新组合信息和所述至少两路媒体画面的第一更新布局信息，或，所述至少两路媒体画面的第一更新组合信息；receiving the updated first media picture composition information generated by the media receiver according to the media picture change information, the first media picture composition information including: the first update combination information and The first updated layout information of the at least two media screens, or the first updated combination information of the at least two media screens;
根据对所述第一媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Perform corresponding operations to the media receiver according to the support capability of the composition information of the first media picture.
结合第一方面，在第一方面的第十种可能的实施方式中，根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，还包括：With reference to the first aspect, in the tenth possible implementation manner of the first aspect, after performing corresponding operations to the media receiver according to the support capability of the media screen composition information, the method further includes:
接收所述媒体接收方更新后的第二媒体画面组成信息，所述第二媒体画面组成信息中包括：所述至少两路媒体画面的第二更新组合信息和所述至少两路媒体画面的第二更新布局信息，或，所述至少两路媒体画面的第二更新组合信息；receiving the updated second media picture composition information from the media receiver, the second media picture composition information including: the second updated combination information of the at least two media pictures and the first update combination information of the at least two media pictures 2. Update the layout information, or, the second update combination information of the at least two media screens;
根据对所述第二媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Perform corresponding operations to the media receiver according to the support capability of the composition information of the second media picture.
结合第一方面，在第一方面的第十一种可能的实施方式中，发送媒体画面组合能力指示信息给媒体接收方，还包括：With reference to the first aspect, in the eleventh possible implementation manner of the first aspect, sending the media screen combination capability indication information to the media receiver further includes:
发送至少一种媒体画面组合选择信息给所述媒体接收方，以使所述媒体接收方根据所述至少一种媒体画面组合选择信息确定媒体画面组成信息。Sending at least one media screen combination selection information to the media receiver, so that the media receiver determines media screen composition information according to the at least one media screen combination selection information.
本发明实施例第二方面提供一种多画面组合的呈现方法，包括：The second aspect of the embodiment of the present invention provides a method for presenting a multi-picture combination, including:
接收媒体提供方发送的媒体画面组合能力指示信息；Receive the media screen combination capability indication information sent by the media provider;
根据所述媒体画面组合能力指示信息确定媒体画面组成信息；determining media picture composition information according to the media picture combination capability indication information;
将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Sending the media screen composition information to the media provider, so that the media provider performs corresponding operations on the media receiver according to the support capability of the media screen composition information.
结合第二方面，在第二方面的第一种可能的实施方式中，所述媒体画面组成信息包括：至少两路媒体画面的组合信息，或，至少两路媒体画面的组合信息和至少两路媒体画面的布局信息。With reference to the second aspect, in the first possible implementation manner of the second aspect, the media picture composition information includes: combination information of at least two media pictures, or combination information of at least two media pictures and at least two Layout information of the media screen.
结合第二方面，在第二方面的第二种可能的实施方式中，将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作，包括：With reference to the second aspect, in a second possible implementation manner of the second aspect, the media screen composition information is sent to the media provider, so that the media provider can Supporting capabilities, performing corresponding operations to the media receiver, including:
若所述媒体提供方支持所述媒体画面组成信息，则接收所述媒体提供方发送的组合画面的第一媒体参数；并发送所述组合画面的第二媒体参数给所述媒体提供方；并接收所述媒体提供方根据所述媒体画面组成信息、所述第一媒体参数和所述第二媒体参数确定的所述组合画面；或者，If the media provider supports the media picture composition information, then receive the first media parameter of the combined picture sent by the media provider; and send the second media parameter of the combined picture to the media provider; and receiving the combined picture determined by the media provider according to the media picture composition information, the first media parameter, and the second media parameter; or,
若所述媒体提供方不支持所述媒体画面组成信息，则接收所述媒体提供方发送的否定应答消息。If the media provider does not support the media screen composition information, receive a negative response message sent by the media provider.
结合第二方面的第二种可能的实施方式，在第二方面的第三种可能的实施方式中，接收媒体提供方发送的媒体画面组合能力指示信息，包括：With reference to the second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, receiving the media screen combination capability indication information sent by the media provider includes:
接收所述媒体提供方发送的第一远程呈现的多流控制CLUE媒体通告消息，所述第一CLUE媒体通告消息中携带所述媒体画面组合能力指示信息；receiving the first telepresence multi-stream control CLUE media notification message sent by the media provider, where the first CLUE media notification message carries the media screen combination capability indication information;
则将所述媒体画面组成信息发送给所述媒体提供方，包括：Then send the composition information of the media screen to the media provider, including:
发送第一CLUE媒体配置消息给所述媒体提供方，所述第一CLUE媒体配置消息中携带所述媒体画面组成信息。Sending a first CLUE media configuration message to the media provider, where the first CLUE media configuration message carries the media screen composition information.
结合第二方面的第二种可能的实施方式，在第二方面的第四种可能的实施方式中，接收媒体提供方发送的媒体画面组合能力指示信息，包括：With reference to the second possible implementation manner of the second aspect, in a fourth possible implementation manner of the second aspect, receiving the media screen combination capability indication information sent by the media provider includes:
接收所述媒体提供方发送的第一会话描述协议SDP请求消息，所述第一SDP请求消息中携带所述媒体画面组合能力指示信息；则receiving a first Session Description Protocol SDP request message sent by the media provider, where the first SDP request message carries the media screen combination capability indication information; then
将所述媒体画面组成信息发送给所述媒体提供方，包括：Sending the composition information of the media screen to the media provider includes:
发送第一SDP响应消息给所述媒体提供方，所述第一SDP响应消息中携带所述媒体画面组成信息。Sending a first SDP response message to the media provider, where the first SDP response message carries the media screen composition information.
结合第二方面的第二种可能的实施方式，在第二方面的第五种可能的实施方式中，接收媒体提供方发送的媒体画面组合能力指示信息，包括：With reference to the second possible implementation manner of the second aspect, in the fifth possible implementation manner of the second aspect, receiving the media screen combination capability indication information sent by the media provider includes:
从门户网站获取所述媒体提供方发送的所述媒体画面组合能力指示信息以及媒体画面的缩略图；则Acquiring the media screen combination capability indication information and the thumbnail of the media screen sent by the media provider from the portal website; then
将所述媒体画面组成信息发送给所述媒体提供方，包括：Sending the composition information of the media screen to the media provider includes:
根据所述媒体画面组合能力指示信息和所述媒体画面的缩略图确定所述媒体画面组成信息，并将所述媒体画面组成信息发送给所述门户网站，以使所述媒体提供方从所述门户网站获取所述媒体画面组成信息。Determine the media screen composition information according to the media screen combination capability indication information and the thumbnail of the media screen, and send the media screen composition information to the portal website, so that the media provider can obtain from the The portal website obtains the composition information of the media screen.
结合第二方面的第三种可能的实施方式，在第二方面的第六种可能的实施方式中，在接收所述媒体提供方发送的第一远程呈现的多流控制CLUE媒体通告消息之前，还包括：With reference to the third possible implementation manner of the second aspect, in a sixth possible implementation manner of the second aspect, before receiving the first telepresence multi-stream control CLUE media announcement message sent by the media provider, Also includes:
接收所述媒体提供方发送的第二SDP请求消息，所述第二SDP请求消息携带支持CLUE协议指示；receiving a second SDP request message sent by the media provider, where the second SDP request message carries an indication of supporting the CLUE protocol;
发送第二SDP响应消息给所述媒体提供方，所述第二SDP响应消息中携带支持所述CLUE协议指示。Sending a second SDP response message to the media provider, where the second SDP response message carries an indication of supporting the CLUE protocol.
结合第二方面的第三种可能的实施方式或第二方面的第四种可能的实施方式，在第二方面的第七种可能的实施方式中，接收所述媒体提供方发送的组合画面的第一媒体参数，包括：With reference to the third possible implementation manner of the second aspect or the fourth possible implementation manner of the second aspect, in a seventh possible implementation manner of the second aspect, receiving the combination picture sent by the media provider The first media parameter, including:
接收所述媒体提供方发送的第三SDP请求消息，所述第三SDP请求消息中携带所述组合画面的第一媒体参数；则receiving a third SDP request message sent by the media provider, where the third SDP request message carries the first media parameter of the combined picture; then
发送所述组合画面的第二媒体参数给所述媒体提供方，包括：Sending the second media parameter of the combined picture to the media provider includes:
发送第三SDP响应消息给所述媒体提供方，所述第三SDP响应消息中携带所述组合画面的第二媒体参数。Sending a third SDP response message to the media provider, where the third SDP response message carries the second media parameter of the combined screen.
结合第二方面的第五种可能的实施方式，在第二方面的第八种可能的实施方式中，接收所述媒体提供方发送的组合画面的第一媒体参数，包括：With reference to the fifth possible implementation manner of the second aspect, in the eighth possible implementation manner of the second aspect, receiving the first media parameter of the combined picture sent by the media provider includes:
从门户网站获取所述媒体提供方发送的所述组合画面的第一媒体参数；则Obtaining the first media parameter of the combined picture sent by the media provider from the portal website; then
发送所述组合画面的第二媒体参数给所述媒体提供方，包括：Sending the second media parameter of the combined picture to the media provider includes:
将所述组合画面的第二媒体参数发送至所述门户网站，以使所述媒体提供方从所述门户网站获取所述组合画面的第二媒体参数。sending the second media parameter of the combined picture to the portal website, so that the media provider acquires the second media parameter of the combined picture from the portal website.
结合第二方面，在第二方面的第九种可能的实施方式中，在将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，还包括：With reference to the second aspect, in a ninth possible implementation manner of the second aspect, after sending the media screen composition information to the media provider, the media provider makes the media screen composition information Supporting capabilities, after performing corresponding operations to the media receiver, it also includes:
接收所述媒体提供方发送的媒体画面变化信息；receiving media screen change information sent by the media provider;
根据所述媒体画面变化信息生成更新后的第一媒体画面组成信息，所述第一媒体画面组成信息中包括：所述至少两路媒体画面的第一更新组合信息和所述至少两路媒体画面的第一更新布局信息，或，所述至少两路媒体画面的第一更新组合信息；Generate updated first media screen composition information according to the media screen change information, the first media screen composition information includes: the first updated combination information of the at least two media screens and the at least two media screens The first updated layout information of the first updated layout information, or the first updated combined information of the at least two media screens;
将所述第一媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述第一媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Sending the first media picture composition information to the media provider, so that the media provider performs corresponding operations to the media receiver according to the support capability of the first media picture composition information.
结合第二方面，在第二方面的第十种可能的实施方式中，在将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，还包括：With reference to the second aspect, in a tenth possible implementation manner of the second aspect, the media screen composition information is sent to the media provider, so that the media provider can Supporting capabilities, after performing corresponding operations to the media receiver, it also includes:
发送更新后的第二媒体画面组成信息给所述媒体提供方，所述第二媒体画面组成信息中包括：所述至少两路媒体画面的第二更新组合信息和所述至少两路媒体画面的第二更新布局信息，或，所述至少两路媒体画面的第二更新组合信息，以使所述媒体提供方根据对所述第二媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Send the updated second media picture composition information to the media provider, the second media picture composition information includes: the second updated combination information of the at least two media pictures and the at least two media pictures The second updated layout information, or the second updated combination information of the at least two media screens, so that the media provider can execute the corresponding operation.
结合第二方面，在第二方面的第十一种可能的实施方式中，所述接收媒体提供方发送的媒体画面组合能力指示信息，还包括：With reference to the second aspect, in the eleventh possible implementation manner of the second aspect, the receiving the media screen combination capability indication information sent by the media provider further includes:
接收所述媒体提供方发送的至少一种媒体画面组合选择信息；receiving at least one media screen combination selection information sent by the media provider;
则根据所述媒体画面组合能力指示信息确定媒体画面组成信息，包括：Then determine the media picture composition information according to the media picture combination capability indication information, including:
根据所述至少一种媒体画面组合选择信息确定媒体画面组成信息。Determine media picture composition information according to the at least one media picture combination selection information.
本发明第三方面提供一种多画面组合的装置，包括：The third aspect of the present invention provides a multi-picture combination device, including:
第一发送模块，用于发送媒体画面组合能力指示信息给媒体接收方；The first sending module is used to send media picture combination capability indication information to the media receiver;
第一接收模块，用于接收所述媒体接收方发送的根据所述媒体画面组合能力指示信息确定的媒体画面组成信息；The first receiving module is configured to receive the media picture composition information determined according to the media picture combination capability indication information sent by the media receiver;
处理模块，用于根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。A processing module, configured to perform corresponding operations to the media receiver according to the support capability of the media picture composition information.
结合第三方面，在第三方面的第一种可能的实施方式中，所述媒体画面组成信息包括：至少两路媒体画面的组合信息，或，至少两路媒体画面的组合信息和至少两路媒体画面的布局信息。With reference to the third aspect, in a first possible implementation manner of the third aspect, the media picture composition information includes: combination information of at least two media pictures, or combination information of at least two media pictures and at least two Layout information of the media screen.
结合第三方面，在第三方面的第二种可能的实施方式中，所述处理模块，包括：With reference to the third aspect, in a second possible implementation manner of the third aspect, the processing module includes:
第一处理单元，用于若支持所述媒体画面组成信息，则发送组合画面的第一媒体参数给所述媒体接收方；并接收所述媒体接收方发送的所述组合画面的第二媒体参数；并将根据所述媒体画面组成信息、所述第一媒体参数和所述第二媒体参数确定的所述组合画面发送给所述媒体接收方；或者，The first processing unit is configured to send the first media parameter of the combined picture to the media receiver if the media picture composition information is supported; and receive the second media parameter of the combined picture sent by the media receiver. ; and sending the combined picture determined according to the media picture composition information, the first media parameter and the second media parameter to the media receiver; or,
第二处理单元，用于若不支持所述媒体画面组成信息，则发送否定应答消息给所述媒体接收方。The second processing unit is configured to send a negative response message to the media receiver if the media screen composition information is not supported.
结合第三方面的第二种可能的实施方式，在第三方面的第三种可能的实施方式中，所述第一发送模块，包括：With reference to the second possible implementation manner of the third aspect, in a third possible implementation manner of the third aspect, the first sending module includes:
第一发送单元，用于发送第一远程呈现的多流控制CLUE媒体通告消息给所述媒体接收方，所述第一CLUE媒体通告消息中携带所述媒体画面组合能力指示信息；A first sending unit, configured to send a first telepresence multi-stream control CLUE media notification message to the media receiver, where the first CLUE media notification message carries the media screen combination capability indication information;
则所述第一接收模块包括：Then the first receiving module includes:
第一接收单元，用于接收所述媒体接收方发送的第一CLUE媒体配置消息，所述第一CLUE媒体配置消息中携带所述媒体画面组成信息。The first receiving unit is configured to receive a first CLUE media configuration message sent by the media receiver, where the first CLUE media configuration message carries the media screen composition information.
结合第三方面的第二种可能的实施方式，在第三方面的第四种可能的实施方式中，所述第一发送模块，包括：With reference to the second possible implementation manner of the third aspect, in a fourth possible implementation manner of the third aspect, the first sending module includes:
第二发送单元，用于发送第一会话描述协议SDP请求消息给所述媒体接收方，所述第一SDP请求消息中携带所述媒体画面组合能力指示信息；A second sending unit, configured to send a first Session Description Protocol SDP request message to the media receiver, where the first SDP request message carries the media screen combination capability indication information;
则所述第一接收模块，包括：Then the first receiving module includes:
第二接收单元，用于接收所述媒体接收方发送的第一SDP响应消息，所述第一SDP响应消息中携带所述媒体画面组成信息。The second receiving unit is configured to receive the first SDP response message sent by the media receiver, where the first SDP response message carries the media picture composition information.
结合第三方面的第二种可能的实施方式，在第三方面的第五种可能的实施方式中，所述第一发送模块，包括：With reference to the second possible implementation manner of the third aspect, in a fifth possible implementation manner of the third aspect, the first sending module includes:
第三发送单元，用于发送媒体画面组合能力指示信息以及媒体画面的缩略图给门户网站，以使所述媒体接收方从所述门户网站获取所述媒体画面组合能力指示信息和所述媒体画面的缩略图；The third sending unit is used to send the indication information of the media picture combination capability and the thumbnail of the media picture to the portal website, so that the media receiver can obtain the indication information of the media picture combination capability and the media picture from the portal website thumbnail of the
则所述第一接收模块，包括：Then the first receiving module includes:
第三接收单元，用于从所述门户网站获取所述媒体接收方发送的根据所述媒体画面组合能力指示信息和所述媒体画面的缩略图确定的所述媒体画面组成信息。The third receiving unit is configured to obtain, from the portal website, the media screen composition information sent by the media receiver and determined according to the media screen combination capability indication information and the thumbnail of the media screen.
结合第三方面的第三种可能的实施方式，在第三方面的第六种可能的实施方式中，所述装置还包括：With reference to the third possible implementation manner of the third aspect, in a sixth possible implementation manner of the third aspect, the device further includes:
第二发送模块，用于在发送远程呈现的多流控制CLUE媒体通告消息给所述媒体接收方之前，发送第二SDP请求消息给所述媒体接收方，所述第二SDP请求消息携带支持CLUE协议指示；The second sending module is configured to send a second SDP request message to the media receiver before sending the telepresence multi-stream control CLUE media announcement message to the media receiver, the second SDP request message carrying CLUE support protocol instructions;
第二接收模块，用于接收所述媒体接收方发送的第二SDP响应消息，所述第二SDP响应消息携带支持所述CLUE协议指示。The second receiving module is configured to receive a second SDP response message sent by the media receiver, where the second SDP response message carries an indication of supporting the CLUE protocol.
结合第三方面的第三种可能的实施方式或第三方面的第四种可能的实施方式，在第三方面的第七种可能的实施方式中，所述第一处理单元，包括：With reference to the third possible implementation manner of the third aspect or the fourth possible implementation manner of the third aspect, in a seventh possible implementation manner of the third aspect, the first processing unit includes:
第一发送子单元，用于发送第三SDP请求消息给所述媒体接收方，所述第三SDP请求消息中携带所述组合画面的第一媒体参数；The first sending subunit is configured to send a third SDP request message to the media receiver, where the third SDP request message carries the first media parameter of the combined picture;
第一接收子单元，用于接收所述媒体接收方发送的第三SDP响应消息，所述第三SDP响应消息中携带所述组合画面的第二媒体参数。The first receiving subunit is configured to receive a third SDP response message sent by the media receiver, where the third SDP response message carries the second media parameter of the combined screen.
结合第三方面的第五种可能的实施方式，在第三方面的第八种可能的实施方式中，所述第一处理单元，包括：With reference to the fifth possible implementation manner of the third aspect, in an eighth possible implementation manner of the third aspect, the first processing unit includes:
第二发送子单元，用于发送所述组合画面的第一媒体参数给门户网站，以使所述媒体接收方从所述门户网站获取所述组合画面的所述第一媒体参数；The second sending subunit is configured to send the first media parameter of the combined picture to the portal website, so that the media receiver can obtain the first media parameter of the combined picture from the portal website;
第二接收子单元，用于从门户网站获取所述媒体接收方发送的所述组合画面的第二媒体参数。The second receiving subunit is configured to acquire the second media parameter of the combined screen sent by the media receiver from the portal website.
结合第三方面，在第三方面的第九种可能的实施方式中，所述装置还包括：With reference to the third aspect, in a ninth possible implementation manner of the third aspect, the device further includes:
第三发送模块，用于在根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，发送媒体画面变化信息给所述媒体接收方；A third sending module, configured to send media screen change information to the media receiver after performing corresponding operations to the media receiver according to the support capability of the media screen composition information;
第三接收模块，用于接收所述媒体接收方根据所述媒体画面变化信息生成的更新后的第一媒体画面组成信息，所述第一媒体画面组成信息中包括：所述至少两路媒体画面的第一更新组合信息和所述至少两路媒体画面的第一更新布局信息，或，所述至少两路媒体画面的第一更新组合信息；The third receiving module is configured to receive the updated first media screen composition information generated by the media receiver according to the media screen change information, the first media screen composition information includes: the at least two channels of media screens The first updated combination information of the at least two media pictures and the first updated layout information of the at least two media pictures, or the first updated combination information of the at least two media pictures;
则所述处理模块还用于根据对所述第一媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Then the processing module is further configured to perform corresponding operations to the media receiver according to the support capability of the composition information of the first media picture.
结合第三方面，在第三方面的第十种可能的实施方式中，所述装置还包括：With reference to the third aspect, in a tenth possible implementation manner of the third aspect, the device further includes:
第四接收模块，用于在根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，接收所述媒体接收方更新后的第二媒体画面组成信息，所述第二媒体画面组成信息中包括：所述至少两路媒体画面的第二更新组合信息和所述至少两路媒体画面的第二更新布局信息，或，所述至少两路媒体画面的第二更新组合信息；The fourth receiving module is configured to receive the updated second media picture composition information of the media receiver after performing corresponding operations on the media receiver according to the support capability of the media picture composition information, the The second media screen composition information includes: the second update combination information of the at least two media screens and the second updated layout information of the at least two media screens, or the second update of the at least two media screens combination information;
则所述处理模块还用于根据对所述第二媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。The processing module is further configured to perform corresponding operations to the media receiver according to the support capability of the composition information of the second media picture.
结合第三方面，在第三方面的第十一种可能的实施方式中，第一发送模块，还用于发送至少一种媒体画面组合选择信息给所述媒体接收方，以使所述媒体接收方根据所述至少一种媒体画面组合选择信息确定媒体画面组成信息。With reference to the third aspect, in an eleventh possible implementation manner of the third aspect, the first sending module is further configured to send at least one media screen combination selection information to the media receiver, so that the media receiver The party determines media picture composition information according to the at least one media picture combination selection information.
本发明实施例第四方面提供一种多画面组合的装置，包括：The fourth aspect of the embodiment of the present invention provides a multi-picture combination device, including:
第一接收模块，用于接收媒体提供方发送的媒体画面组合能力指示信息；The first receiving module is used to receive the media screen combination capability indication information sent by the media provider;
确定模块，用于根据所述媒体画面组合能力指示信息确定媒体画面组成信息；A determining module, configured to determine media picture composition information according to the media picture combination capability indication information;
第一发送模块，用于将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。The first sending module is configured to send the media screen composition information to the media provider, so that the media provider can perform corresponding actions to the media receiver according to the support capability of the media screen composition information operate.
结合第四方面，在第四方面的第一种可能的实施方式中，所述媒体画面组成信息包括：至少两路媒体画面的组合信息，或，至少两路媒体画面的组合信息和至少两路媒体画面的布局信息。With reference to the fourth aspect, in the first possible implementation manner of the fourth aspect, the media picture composition information includes: combination information of at least two media pictures, or combination information of at least two media pictures and at least two Layout information of the media screen.
结合第四方面，在第四方面的第二种可能的实施方式中，所述第一发送模块，包括：With reference to the fourth aspect, in a second possible implementation manner of the fourth aspect, the first sending module includes:
第一处理单元，用于若所述媒体提供方支持所述媒体画面组成信息，则接收所述媒体提供方发送的组合画面的第一媒体参数；并发送所述组合画面的第二媒体参数给所述媒体提供方；并接收所述媒体提供方根据所述媒体画面组成信息、所述第一媒体参数和所述第二媒体参数确定的所述组合画面；或者，The first processing unit is configured to receive the first media parameter of the combined picture sent by the media provider if the media provider supports the composition information of the media picture; and send the second media parameter of the combined picture to The media provider; and receiving the combined screen determined by the media provider according to the media screen composition information, the first media parameter, and the second media parameter; or,
第二处理单元，用于若所述媒体提供方不支持所述媒体画面组成信息，则接收所述媒体提供方发送的否定应答消息。The second processing unit is configured to receive a negative response message sent by the media provider if the media provider does not support the media screen composition information.
结合第四方面的第二种可能的实施方式，在第四方面的第三种可能的实施方式中，所述第一接收模块，包括：With reference to the second possible implementation manner of the fourth aspect, in a third possible implementation manner of the fourth aspect, the first receiving module includes:
第一接收单元，用于接收所述媒体提供方发送的第一远程呈现的多流控制CLUE媒体通告消息，所述第一CLUE媒体通告消息中携带所述媒体画面组合能力指示信息；The first receiving unit is configured to receive the first telepresence multi-stream control CLUE media notification message sent by the media provider, where the first CLUE media notification message carries the media screen combination capability indication information;
则所述第一发送模块，包括：Then the first sending module includes:
第一发送单元，用于发送第一CLUE媒体配置消息给所述媒体提供方，所述第一CLUE媒体配置消息中携带所述媒体画面组成信息。The first sending unit is configured to send a first CLUE media configuration message to the media provider, where the first CLUE media configuration message carries the media screen composition information.
结合第四方面的第二种可能的实施方式，在第四方面的第四种可能的实施方式中，所述第一接收模块，包括：With reference to the second possible implementation manner of the fourth aspect, in a fourth possible implementation manner of the fourth aspect, the first receiving module includes:
第二接收单元，用于接收所述媒体提供方发送的第一会话描述协议SDP请求消息，所述第一SDP请求消息中携带所述媒体画面组合能力指示信息；The second receiving unit is configured to receive a first Session Description Protocol SDP request message sent by the media provider, where the first SDP request message carries the media screen combination capability indication information;
则所述第一发送模块，包括：Then the first sending module includes:
第二发送单元，用于发送第一SDP响应消息给所述媒体提供方，所述第一SDP响应消息中携带所述媒体画面组成信息。The second sending unit is configured to send a first SDP response message to the media provider, where the first SDP response message carries the media screen composition information.
结合第四方面的第二种可能的实施方式，在第四方面的第五种可能的实施方式中，所述第一接收模块，包括：With reference to the second possible implementation manner of the fourth aspect, in a fifth possible implementation manner of the fourth aspect, the first receiving module includes:
第三接收单元，用于从门户网站获取所述媒体提供方发送的所述媒体画面组合能力指示信息以及媒体画面的缩略图；A third receiving unit, configured to acquire the indication information of the media screen combination capability and the thumbnail of the media screen sent by the media provider from the portal website;
则所述第一发送模块，包括：Then the first sending module includes:
第三发送单元，用于根据所述媒体画面组合能力指示信息和所述媒体画面的缩略图确定所述媒体画面组成信息，并将所述媒体画面组成信息发送给所述门户网站，以使所述媒体提供方从所述门户网站获取所述媒体画面组成信息。A third sending unit, configured to determine the media screen composition information according to the media screen combination capability indication information and the thumbnail of the media screen, and send the media screen composition information to the portal website, so that all The media provider obtains the composition information of the media screen from the portal website.
结合第四方面的第三种可能的实施方式，在第四方面的第六种可能的实施方式中，所述装置还包括：With reference to the third possible implementation manner of the fourth aspect, in a sixth possible implementation manner of the fourth aspect, the device further includes:
第二接收模块，用于在接收所述媒体提供方发送的第一远程呈现的多流控制CLUE媒体通告消息之前，接收所述媒体提供方发送的第二SDP请求消息，所述第二SDP请求消息携带支持CLUE协议指示；The second receiving module is configured to receive the second SDP request message sent by the media provider before receiving the first telepresence multi-stream control CLUE media announcement message sent by the media provider, the second SDP request The message carries an indication of support for the CLUE protocol;
第二发送模块，用于发送第二SDP响应消息给所述媒体提供方，所述第二SDP响应消息中携带支持所述CLUE协议指示。The second sending module is configured to send a second SDP response message to the media provider, where the second SDP response message carries an indication of supporting the CLUE protocol.
结合第四方面的第三种可能的实施方式或第四方面的第四种可能的实施方式，在第四方面的第七种可能的实施方式中，所述第一处理单元，包括：With reference to the third possible implementation manner of the fourth aspect or the fourth possible implementation manner of the fourth aspect, in a seventh possible implementation manner of the fourth aspect, the first processing unit includes:
第一接收子单元，用于接收所述媒体提供方发送的第三SDP请求消息，所述第三SDP请求消息中携带所述组合画面的第一媒体参数；The first receiving subunit is configured to receive a third SDP request message sent by the media provider, where the third SDP request message carries the first media parameter of the combined picture;
第一发送子单元，用于发送第三SDP响应消息给所述媒体提供方，所述第三SDP响应消息中携带所述组合画面的第二媒体参数。The first sending subunit is configured to send a third SDP response message to the media provider, where the third SDP response message carries the second media parameter of the combined picture.
结合第四方面的第五种可能的实施方式，在第四方面的第八种可能的实施方式中，所述第一处理单元，包括：With reference to the fifth possible implementation manner of the fourth aspect, in an eighth possible implementation manner of the fourth aspect, the first processing unit includes:
第二接收子单元，用于从门户网站获取所述媒体提供方发送的所述组合画面的第一媒体参数；The second receiving subunit is configured to acquire the first media parameter of the combined screen sent by the media provider from the portal website;
第二发送子单元，用于将所述组合画面的第二媒体参数发送至所述门户网站，以使所述媒体提供方从所述门户网站获取所述组合画面的第二媒体参数。The second sending subunit is configured to send the second media parameter of the combined picture to the portal website, so that the media provider obtains the second media parameter of the combined picture from the portal website.
结合第四方面，在第四方面的第九种可能的实施方式中，所述装置还包括：With reference to the fourth aspect, in a ninth possible implementation manner of the fourth aspect, the device further includes:
第三接收模块，用于在将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，接收所述媒体提供方发送的媒体画面变化信息；The third receiving module is configured to send the media screen composition information to the media provider, so that the media provider can perform corresponding actions to the media receiver according to the support capability of the media screen composition information. After the operation, receiving the media screen change information sent by the media provider;
生成模块，用于根据所述媒体画面变化信息生成更新后的第一媒体画面组成信息，所述第一媒体画面组成信息中包括：所述至少两路媒体画面的第一更新组合信息和所述至少两路媒体画面的第一更新布局信息，或，所述至少两路媒体画面的第一更新组合信息；A generating module, configured to generate updated first media screen composition information according to the media screen change information, where the first media screen composition information includes: the first updated combination information of the at least two media screens and the The first updated layout information of at least two media screens, or the first updated combination information of the at least two media screens;
第三发送模块，用于将所述第一媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述第一媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。The third sending module is configured to send the composition information of the first media screen to the media provider, so that the media provider can receive information from the media according to the support capability of the composition information of the first media screen side to perform the corresponding operation.
结合第四方面，在第四方面的第十种可能的实施方式中，所述装置还包括：With reference to the fourth aspect, in a tenth possible implementation manner of the fourth aspect, the device further includes:
第四发送模块，用于在将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，发送更新后的第二媒体画面组成信息给所述媒体提供方，所述第二媒体画面组成信息中包括：所述至少两路媒体画面的第二更新组合信息和所述至少两路媒体画面的第二更新布局信息，或，所述至少两路媒体画面的第二更新组合信息，以使所述媒体提供方根据对所述第二的支持能力，向所述媒体接收方执行相应的操作。The fourth sending module is configured to send the media screen composition information to the media provider, so that the media provider can perform corresponding operations to the media receiver according to the support capability of the media screen composition information. After the operation, send the updated second media screen composition information to the media provider, the second media screen composition information includes: the second updated combination information of the at least two media screens and the at least two The second updated layout information of one channel of media pictures, or the second updated combination information of the at least two channels of media pictures, so that the media provider can implement to the media receiver according to the support capability of the second channel corresponding operation.
结合第四方面，在第四方面的第十一种可能的实施方式中，所述第一接收模块还用于接收所述媒体提供方发送的至少一种媒体画面组合选择信息；With reference to the fourth aspect, in an eleventh possible implementation manner of the fourth aspect, the first receiving module is further configured to receive at least one media screen combination selection information sent by the media provider;
则所述确定模块具体用于根据所述至少一种媒体画面组合选择信息确定媒体画面组成信息。Then the determining module is specifically configured to determine media picture composition information according to the at least one media picture combination selection information.
本发明实施例提供的方法，通过媒体提供方向媒体接收方发送媒体画面组合能力指示信息，媒体接收方根据所接收到的媒体画面能力指示信息、媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，即确定所要呈现的媒体画面的组合信息以及布局信息，媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以根据自身能力呈现所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the media screen combination capability indication information to the media receiver, and the media receiver sends the media screen capability indication information according to the received media screen capability indication information, the media receiver's terminal display capability, and the media receiver's local The strategy determines the composition information of the media screen, that is, determines the combination information and layout information of the media screen to be presented, and the media provider performs corresponding operations according to the media screen composition information determined by the media receiver, so that the media combination received by the media receiver The picture is a combined media picture determined by the media provider according to the composition information of the media picture sent by the media receiver, that is, the media receiver can present the combined picture to be presented according to its own capabilities, thereby improving the flexibility of presenting the combined media picture and improving the quality of the media. Conference experience of the receiving user.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.
图1为本发明提供的多画面组合的呈现方法实施例一的流程示意图；FIG. 1 is a schematic flow chart of Embodiment 1 of a multi-picture combination presentation method provided by the present invention;
图2为本发明提供的多画面组合的呈现方法实施例二的流程示意图；FIG. 2 is a schematic flowchart of Embodiment 2 of the presenting method of multi-picture combination provided by the present invention;
图3为本发明提供的多画面组合的呈现方法实施例三的流程示意图；FIG. 3 is a schematic flowchart of Embodiment 3 of the presenting method of multi-picture combination provided by the present invention;
图4为本发明提供的多画面组合的呈现方法实施例四的流程示意图；FIG. 4 is a schematic flowchart of Embodiment 4 of the presenting method for multi-picture combination provided by the present invention;
图5为本发明提供的多画面组合的呈现方法实施例五的流程示意图；FIG. 5 is a schematic flowchart of Embodiment 5 of the presenting method of multi-picture combination provided by the present invention;
图6为本发明提供的多画面组合的呈现方法实施例六的流程示意图；FIG. 6 is a schematic flowchart of Embodiment 6 of the presenting method of multi-picture combination provided by the present invention;
图7为本发明提供的多画面组合的呈现方法实施例七的流程示意图；FIG. 7 is a schematic flowchart of Embodiment 7 of the presenting method of multi-picture combination provided by the present invention;
图8为本发明提供的多画面组合的呈现方法实施例八的流程示意图；FIG. 8 is a schematic flowchart of Embodiment 8 of the presenting method of multi-picture combination provided by the present invention;
图9为本发明提供的多画面组合的呈现方法实施例九的流程示意图；FIG. 9 is a schematic flowchart of Embodiment 9 of the presenting method of multi-picture combination provided by the present invention;
图10为本发明提供的多画面组合的呈现方法实施例十的流程示意图；FIG. 10 is a schematic flow chart of Embodiment 10 of the multi-picture combination presentation method provided by the present invention;
图11为本发明提供的多画面组合的呈现方法实施例十一的流程示意图；FIG. 11 is a schematic flowchart of Embodiment 11 of the presenting method of multi-picture combination provided by the present invention;
图12为本发明提供的多画面组合的呈现方法实施例十二的流程示意图；FIG. 12 is a schematic flow chart of Embodiment 12 of the presentation method of multi-picture combination provided by the present invention;
图13为本发明提供的多画面组合的呈现方法实施例十三的流程示意图；FIG. 13 is a schematic flow chart of Embodiment 13 of the multi-picture combination presentation method provided by the present invention;
图14为本发明提供的多画面组合的呈现方法实施例十四的流程示意图；FIG. 14 is a schematic flowchart of Embodiment 14 of the presenting method of multi-picture combination provided by the present invention;
图15为本发明提供的多画面组合的呈现方法实施例十五的信令流程图；FIG. 15 is a signaling flowchart of Embodiment 15 of the presenting method for multi-picture combination provided by the present invention;
图16为本发明提供的多画面组合的呈现方法实施例十六的信令流程图；FIG. 16 is a signaling flowchart of Embodiment 16 of the presenting method for multi-picture combination provided by the present invention;
图17为本发明提供的多画面组合的呈现方法实施例十七的信令流程图；FIG. 17 is a signaling flow chart of Embodiment 17 of the presenting method for multi-picture combination provided by the present invention;
图18为本发明提供的多画面组合的呈现方法实施例十八的信令流程图；FIG. 18 is a signaling flowchart of Embodiment 18 of the presenting method for multi-picture combination provided by the present invention;
图19为本发明提供的多画面组合的呈现方法实施例十九的信令流程图；FIG. 19 is a signaling flow chart of Embodiment 19 of the presenting method for multi-picture combination provided by the present invention;
图20为本发明提供的多画面组合的呈现方法实施例二十的信令流程图；FIG. 20 is a signaling flowchart of Embodiment 20 of the presenting method of multi-picture combination provided by the present invention;
图21为本发明提供的多画面组合的装置实施例一的结构示意图；Fig. 21 is a schematic structural diagram of Embodiment 1 of a device for multi-picture combination provided by the present invention;
图22为本发明提供的多画面组合的装置实施例二的结构示意图；Fig. 22 is a schematic structural diagram of Embodiment 2 of the multi-picture combination device provided by the present invention;
图23为本发明提供的多画面组合的装置实施例三的结构示意图；Fig. 23 is a schematic structural diagram of Embodiment 3 of the multi-picture combination device provided by the present invention;
图24为本发明提供的多画面组合的装置实施例四的结构示意图；Fig. 24 is a schematic structural diagram of Embodiment 4 of the multi-picture combination device provided by the present invention;
图25为本发明提供的多画面组合的装置实施例五的结构示意图；Fig. 25 is a schematic structural diagram of Embodiment 5 of the multi-picture combination device provided by the present invention;
图26为本发明提供的多画面组合的装置实施例六的结构示意图。Fig. 26 is a schematic structural diagram of Embodiment 6 of the device for multi-picture combination provided by the present invention.
具体实施方式detailed description
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
图1为本发明提供的多画面组合的呈现方法实施例一的流程示意图，本发明实施例涉及的执行主体是媒体提供方，这里的媒体提供方可以是发送组合画面的会议终端，媒体接收方是接收该组合画面的会议终端。如图1所示，该方法包括：Fig. 1 is a schematic flow chart of Embodiment 1 of the presentation method of multi-picture combination provided by the present invention. The execution subject involved in the embodiment of the present invention is a media provider, where the media provider may be a conference terminal sending a combined screen, and a media receiver is the conference terminal that receives the composite screen. As shown in Figure 1, the method includes:
S101：发送媒体画面组合能力指示信息给媒体接收方。S101: Send media picture combination capability indication information to a media receiver.
该媒体画面组合能力指示信息用于向媒体接收方告知该媒体提供方具有组合媒体画面的能力，媒体画面组合能力指示信息可以有多种不同的实现方式：例如简单的仅仅说明媒体发送方可以组合；或者媒体发送方给出一些多路媒体画面的组合、布局信息的可选项等等。The media picture combination capability indication information is used to inform the media receiver that the media provider has the ability to combine media pictures. ; or the media sender provides some combinations of multi-channel media screens, options for layout information, and so on.
S102：接收媒体接收方发送的根据上述媒体画面组合能力指示信息确定的媒体画面组成信息。S102: Receive the media screen composition information determined according to the media screen composition capability indication information sent by the media receiver.
具体的，媒体接收方在收到媒体提供方发送的媒体画面组合能力指示信息之后，根据该媒体画面组合能力指示信息、媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面组成信息，并且这里的媒体画面组成信息可以包括至少两路媒体画面的组合信息和布局信息的场景，或者，仅有至少两路媒体画面组合信息的场景；媒体接收方将该媒体画面组成信息发送给媒体提供方。Specifically, after receiving the media screen combination capability indication information sent by the media provider, the media receiver determines the media screen composition information according to the media screen combination capability indication information, the media receiver's terminal display capability, and the media receiver's local policy. , and the media picture composition information here may include at least two channels of media picture combination information and a scene of layout information, or a scene with only at least two channels of media picture combination information; the media receiver sends the media picture composition information to the media provider.
S103：根据对上述媒体画面组成信息的支持能力，向媒体接收方执行相应的操作。S103: Perform corresponding operations to the media receiver according to the support capability of the composition information of the above media screen.
具体的，媒体提供方接收媒体接收方发送的媒体画面组成信息，并根据对该媒体画面组成信息中的任一场景的的支持能力，向媒体接收方执行相应的操作，例如：若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，则媒体提供方按照媒体组成信息以及相关的媒体参数向媒体接收方发送组合画面；若媒体提供方不支持至少两路媒体画面的组合和至少两路媒体画面的布局，则媒体提供方向媒体接收方发送否定应答消息，以使媒体接收方重新生成新的媒体画面组成信息。Specifically, the media provider receives the media screen composition information sent by the media receiver, and performs corresponding operations to the media receiver according to the support capability of any scene in the media screen composition information, for example: if the media provider To support the combination of at least two media screens and the layout of at least two media screens, the media provider sends the combined screen to the media receiver according to the media composition information and related media parameters; if the media provider does not support at least two media screens combination and layout of at least two media screens, the media provider sends a negative response message to the media receiver so that the media receiver can regenerate new media screen composition information.
本发明实施例提供的方法，通过媒体提供方向媒体接收方发送媒体画面组合能力指示信息，媒体接收方根据所接收到的媒体画面能力指示信息、媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，即确定所要呈现的媒体画面的组合信息以及布局信息，媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以根据自身能力呈现所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the media screen combination capability indication information to the media receiver, and the media receiver sends the media screen capability indication information according to the received media screen capability indication information, the media receiver's terminal display capability, and the media receiver's local The strategy determines the composition information of the media screen, that is, determines the combination information and layout information of the media screen to be presented, and the media provider performs corresponding operations according to the media screen composition information determined by the media receiver, so that the media combination received by the media receiver The picture is a combined media picture determined by the media provider according to the composition information of the media picture sent by the media receiver, that is, the media receiver can present the combined picture to be presented according to its own capabilities, thereby improving the flexibility of presenting the combined media picture and improving the quality of the media. Conference experience of the receiving user.
在上述图1所示实施例的基础上，本实施例涉及的方法是媒体提供方根据对媒体接收方发送的媒体组成信息的支持能力执行的具体操作，即上述S103包括：若支持上述媒体画面组成信息，则发送组合画面的第一媒体参数给媒体接收方；接收媒体接收方发送的上述组合画面的第二媒体参数；将根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定的所述组合画面发送给媒体接收方；或者，若不支持上述媒体画面组成信息，则发送否定应答消息给媒体接收方。On the basis of the above-mentioned embodiment shown in Figure 1, the method involved in this embodiment is a specific operation performed by the media provider according to the support capability of the media composition information sent by the media receiver, that is, the above-mentioned S103 includes: if the above-mentioned media screen is supported Composition information, then send the first media parameter of the combined picture to the media receiver; receive the second media parameter of the above-mentioned combined picture sent by the media receiver; determine according to the above-mentioned media picture composition information, first media parameter and second media parameter Send the combined picture of the media receiver to the media receiver; or, if the above media picture composition information is not supported, send a negative response message to the media receiver.
具体的，媒体接收方将媒体画面组成信息发送给媒体提供方，该媒体画面组成信息包括：至少两路媒体画面的组合信息，或，至少两路媒体画面的组合信息和至少两路媒体画面的布局信息；媒体提供方对媒体接收方发送的媒体画面组成信息进行是否支持的判断，分为两种实现方式：Specifically, the media receiver sends the media screen composition information to the media provider. The media screen composition information includes: the combination information of at least two media screens, or the combination information of at least two media screens and the at least two media screen composition information. Layout information; the media provider judges whether to support the media screen composition information sent by the media receiver, and there are two implementation methods:
第一种：若媒体提供方支持上述媒体画面组成信息中所包含的至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持至少两路媒体画面的组合两种场景中的任一个场景，则媒体提供方发送组合画面的第一媒体参数给媒体接收方，并且该组合画面是媒体提供方根据媒体画面组成信息所确定的；媒体接收方接收到媒体提供方发送的第一媒体参数之后，向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商；之后，媒体提供方根据上述至少两路媒体画面的组合信息、至少两路媒体画面的布局信息、第一媒体参数和第二媒体参数确定组合画面，并将组合画面发送给媒体接收方予以呈现。The first one: if the media provider supports the combination of at least two media screens and the layout of at least two media screens contained in the above media screen composition information, or supports the combination of at least two media screens in any of the two scenarios In one scenario, the media provider sends the first media parameter of the combined picture to the media receiver, and the combined picture is determined by the media provider according to the composition information of the media picture; the media receiver receives the first media parameter sent by the media provider. After the parameters, send the second media parameter to the media provider, where the first media parameter and the second media parameter are the negotiation between the media provider and the media receiver on parameters such as the codec method and the transmission bandwidth of the combined picture; after that, the media The provider determines the combined picture according to the combination information of the at least two channels of media pictures, the layout information of the at least two channels of media pictures, the first media parameter and the second media parameter, and sends the combined picture to the media receiver for presentation.
第二种：若媒体提供方不支持上述媒体画面组成信息中的任一场景，则发送否定应答消息给媒体接收方，向媒体接收方表示不接受该媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息再次发送给媒体提供方，媒体提供方接收更新后的媒体画面组成信息，依然对其再次进行上述的判断过程。The second type: If the media provider does not support any of the above media screen composition information, it will send a negative response message to the media receiver, expressing to the media receiver that it does not accept the media screen composition information. After the negative response message, regenerate new media screen composition information, that is, update the combination information of at least two media screens and the layout information of at least two media screens, and send the updated media screen composition information to the media provider again, The media provider receives the updated media screen composition information, and still performs the above judgment process on it again.
图2为本发明提供的多画面组合的呈现方法实施例二的流程示意图，本发明实施例的执行主体是媒体提供方，在上述实施例的基础上，如图2所示，该方法包括：Fig. 2 is a schematic flow chart of Embodiment 2 of the multi-picture combination presentation method provided by the present invention. The execution subject of the embodiment of the present invention is the media provider. On the basis of the above embodiments, as shown in Fig. 2 , the method includes:
S201：发送第一远程呈现的多流控制（Controlling multiple streams forTelepresence，以下简称CLUE）媒体通告消息给媒体接收方，该第一CLUE媒体通告消息中携带媒体画面组合能力指示信息。S201: Send a first Controlling multiple streams for Telepresence (CLUE for short) media notification message to a media receiver, where the first CLUE media notification message carries media screen combination capability indication information.
具体的，媒体提供方发送第一CLUE媒体通告消息给媒体接收方，该第一CLUE媒体通告消息携带媒体画面组合能力指示信息，用以使得媒体接收方根据第一CLUE媒体通告消息获知媒体提供方具有组合画面的能力、媒体提供方提供的多路媒体画面以及多路媒体画面相关的位置信息；在此以一个例子来说明：媒体提供方可以提供3路单独媒体画面(VC0,VC1,VC2)，媒体提供方在第一CLUE媒体通告消息中的媒体组合画面能力相关描述如下：Specifically, the media provider sends the first CLUE media notification message to the media receiver, and the first CLUE media notification message carries media screen combination capability indication information, so that the media receiver can inform the media provider according to the first CLUE media notification message It has the ability to combine screens, multiple media screens provided by the media provider, and location information related to multiple media screens; here is an example to illustrate: the media provider can provide 3 separate media screens (VC0, VC1, VC2) , the media provider's media composition screen capabilities in the first CLUE media notification message are described as follows:
Advertisement:Advertisement:
Capture Scene1:Capture Scene1:
Capture Scene Entry1:VC0,VC1,VC2Capture Scene Entry1: VC0, VC1, VC2
Capture Scene Entry2:VC4Capture Scene Entry2: VC4
VC0:encoding group=EG0,content=view:table.left;VC0:encoding group=EG0,content=view:table.left;
VC1:encoding group=EG0,content=view:table.center;VC1:encoding group=EG0,content=view:table.center;
VC2:encoding group=EG0,content=view:table.right;VC2:encoding group=EG0,content=view:table.right;
VC4:encoding group=EG1,content=view:room,composed=true；VC4:encoding group=EG1,content=view:room,composed=true;
上述例子中，对于媒体画面组合能力描述的定义包含在composed中，携带该composed参数，即隐含表征媒体提供方具有媒体画面组合的能力。In the above example, the definition of the media screen composition capability description is included in composed, and the composed parameter is carried, which implicitly indicates that the media provider has the media screen composition capability.
S202：接收媒体接收方发送的第一CLUE媒体配置消息，该第一CLUE媒体配置消息中携带媒体画面组成信息。S202: Receive a first CLUE media configuration message sent by a media receiver, where the first CLUE media configuration message carries media screen composition information.
具体的，媒体接收方接收到媒体提供方发送的第一CLUE媒体通告消息后，结合媒体接收方会议室的终端显示能力以及媒体接收方的本地策略，确定媒体画面组成信息，例如选择由哪几个媒体画面来进行组合、该组合画面如何布局等，并向媒体提供方发送第一CLUE配置消息，该第一CLUE媒体配置消息中携带媒体画面组成信息；此处，同样以一个例子来说明：假设媒体接收方只有一个显示屏，只能接收1路媒体画面，最终媒体接收方选择采用其中2路媒体画面(VC0,VC1)来组合成一路，二路呈现左右平分画面；媒体接收方在第一CLUE配置消息中的媒体组合选择相关描述如下：Specifically, after receiving the first CLUE media notification message sent by the media provider, the media receiver determines the composition information of the media screen in combination with the terminal display capability of the media receiver's meeting room and the local strategy of the media receiver, such as which How to combine a media screen, how to layout the combined screen, etc., and send the first CLUE configuration message to the media provider, the first CLUE media configuration message carries the composition information of the media screen; here, an example is also used to illustrate: Assume that the media receiver has only one display screen and can only receive one channel of media images. Finally, the media receiver chooses to use two channels of media images (VC0, VC1) to combine into one channel, and the second channel presents a left and right split screen; The relevant description of media combination selection in a CLUE configuration message is as follows:
Configuration:Configuration:
Capture Scene1:Capture Scene1:
Capture Scene Entry2:VC4Capture Scene Entry2: VC4
VC4：encoding group=EG1，content=view:room.partial，composed=true，composed layout={VC0（0,0,0）(512,0,0)(0,768,0)(512,768,0),VC1(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}VC4: encoding group=EG1, content=view:room.partial, composed=true, composed layout={VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0),VC1 (512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}
上述例子中，媒体画面组合结果包含在composed layout参数中，表示该组合画面由哪些单独媒体画面组成，而且画面的布局中每个单独画面采用三维立体坐标来表示媒体画面的范围。In the above example, the composition result of the media screen is included in the composed layout parameter, indicating which individual media screens are composed of the combined screen, and each individual screen in the layout of the screen uses three-dimensional coordinates to represent the range of the media screen.
S203：根据对上述媒体画面组成信息的支持能力，向媒体接收方执行相应的操作。S203: Perform corresponding operations to the media receiver according to the support capability of the composition information of the above media screen.
具体的，媒体提供方接收媒体接收方发送的第一CLUE媒体配置消息后，根据对上述媒体画面组成信息的支持能力，向媒体接收方执行相应的操作，即S203具体可以分为两种实现方式：Specifically, after receiving the first CLUE media configuration message sent by the media receiver, the media provider performs corresponding operations to the media receiver according to the support capability of the above-mentioned media screen composition information, that is, S203 can be specifically divided into two implementations :
第一种：若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持上述至少两路媒体画面的组合这两种场景中的任一场景，则上述S203具体包括：The first one: if the media provider supports the combination of the at least two media screens and the layout of the at least two media screens, or supports any of the two scenarios of the combination of the at least two media screens above, then the above S203 Specifically include:
S2031：发送组合画面的第一媒体参数给媒体接收方。S2031: Send the first media parameter of the combined screen to the media receiver.
S2032：接收媒体接收方发送的组合画面的第二媒体参数。S2032: Receive the second media parameter of the combined screen sent by the media receiver.
S2033：将根据至少两路媒体画面的组合信息、至少两路媒体画面的布局信息、第一媒体参数和第二媒体参数确定的组合画面发送给媒体接收方。S2033: Send the combined picture determined according to the combination information of the at least two channels of media pictures, the layout information of the at least two channels of media pictures, the first media parameter, and the second media parameter to the media receiver.
具体的，媒体提供方发送组合画面的第一媒体参数给媒体接收方，并且上述组合画面是媒体提供方根据媒体画面组成信息所确定的；媒体接收方接收到媒体提供方发送的第一媒体参数之后，向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，媒体接收方和媒体提供方根据媒体画面参数的协商结果，建立双向多流媒体传输通道，媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定组合画面，并将组合画面发送给媒体接收方予以呈现。Specifically, the media provider sends the first media parameter of the combined picture to the media receiver, and the combined picture is determined by the media provider according to the composition information of the media picture; the media receiver receives the first media parameter sent by the media provider Afterwards, send the second media parameter to the media provider, where the first media parameter and the second media parameter are the negotiation between the media provider and the media receiver on parameters such as codec mode and transmission bandwidth of the combined picture, the media receiver and the media receiver The media provider establishes a two-way multi-stream media transmission channel according to the negotiation result of the media screen parameters, and the media provider determines the composite screen according to the above media screen composition information, the first media parameter and the second media parameter, and sends the composite screen to the media receiver party to be presented.
第二种：若媒体提供方不支持上述媒体画面组成信息中的任一场景，则上述S203具体为：发送否定应答消息给媒体接收方。The second type: if the media provider does not support any scene in the above media screen composition information, the above S203 is specifically: sending a negative response message to the media receiver.
具体的，媒体提供方发送否定应答消息给媒体接收方，向媒体接收方表示不接受该媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息携带在第一CLUE媒体配置消息中再次发送给媒体提供方，媒体提供方接收更新后的第一CLUE媒体配置消息，依然对其再次进行上述的判断过程。Specifically, the media provider sends a negative response message to the media receiver, expressing to the media receiver that it does not accept the media picture composition information, and the media receiver regenerates new media picture composition information after receiving the negative response message, namely Update the composition information of at least two channels of media pictures and the layout information of at least two channels of media pictures, carry the updated media picture composition information in the first CLUE media configuration message and send it to the media provider again, and the media provider receives the updated For the first CLUE media configuration message, the above judgment process is still performed on it again.
本发明实施例提供的方法，通过媒体提供方向媒体接收方发送携带媒体画面组合能力指示信息的第一CLUE媒体通告消息，媒体接收方根据所接收到的第一CLUE媒体通告消息、媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，并将该媒体画面的组成信息携带在第一CLUE媒体配置消息中，媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the first CLUE media announcement message carrying the indication information of the media screen combination capability to the media receiver, and the media receiver sends the first CLUE media announcement message according to the received first CLUE media announcement message, the media receiver's The display capability of the terminal and the local policy of the media receiver determine the composition information of the media screen, and carry the composition information of the media screen in the first CLUE media configuration message, and the media provider executes according to the media screen composition information determined by the media receiver. Corresponding operations make the media combination picture received by the media receiver be the media combination picture determined by the media provider according to the media picture composition information sent by the media receiver, that is, the media receiver can customize the combination picture it wants to present, thereby improving the The flexibility of combined media screen presentation improves the conference experience of media receiver users.
在上述图2所示实施例的基础上，进一步地，在S201之前，该方法还包括：On the basis of the above embodiment shown in FIG. 2 , further, before S201, the method further includes:
S301：发送第二SDP请求消息给媒体接收方，该第二SDP请求消息携带支持CLUE协议指示。S301: Send a second SDP request message to the media receiver, where the second SDP request message carries an indication of supporting the CLUE protocol.
S302：接收媒体接收方发送的第二SDP响应消息，该第二SDP响应消息携带支持所述CLUE协议指示。S302: Receive a second SDP response message sent by the media receiver, where the second SDP response message carries an indication of supporting the CLUE protocol.
具体的，在图2所示的采用CLUE协议方式实现多画面组合的实施例中，在媒体提供方发送第一CLUE媒体通告消息给媒体接收方之前，要进行CLUE通道的建立，即媒体提供方会将支持CLUE协议指示的信息携带在第二SDP请求消息中发送给媒体接收方，相应的，媒体接收方在获知媒体提供方支持CLUE协议后，会向媒体提供方发送第二SDP响应消息，该第二SDP响应消息中携带支持CLUE协议指示，CLUE通道建立成功。Specifically, in the embodiment shown in Fig. 2 that adopts the CLUE protocol mode to realize multi-picture combination, before the media provider sends the first CLUE media announcement message to the media receiver, the establishment of the CLUE channel will be carried out, that is, the media provider The information supporting the CLUE protocol indication will be carried in the second SDP request message and sent to the media receiver. Correspondingly, the media receiver will send the second SDP response message to the media provider after learning that the media provider supports the CLUE protocol. The second SDP response message carries an indication of supporting the CLUE protocol, and the CLUE channel is established successfully.
图3为本发明提供的多画面组合的呈现方法实施例三的流程示意图，在上述实施例的基础上，如图3所示，该方法的执行主体为媒体提供方，该方法包括：Fig. 3 is a schematic flowchart of Embodiment 3 of the multi-picture combination presentation method provided by the present invention. On the basis of the above embodiment, as shown in Fig. 3, the execution subject of the method is the media provider, and the method includes:
S401：发送第一会话描述协议（Session Description Protocol，以下简称SDP）请求消息给媒体接收方，该第一SDP请求消息中携带媒体画面组合能力指示信息。S401: Send a first Session Description Protocol (Session Description Protocol, SDP for short) request message to a media receiver, where the first SDP request message carries media screen combination capability indication information.
具体的，媒体提供方向媒体接收方发送第一SDP请求消息，该第一SDP请求消息携带媒体画面组合能力指示信息，并且还携带了多路媒体画面信息，并且该第一SDP请求消息可以携带在会话初始化协议（Session Initiation Protocol，以下简称SIP）消息中传输，此处以一个例子来说明：媒体提供方可以提供3路单独媒体视频画面(VC0,VC1,VC2)，媒体提供方在第一SDP请求消息中的媒体画面信息相关描述如下：Specifically, the media provider sends a first SDP request message to the media receiver, the first SDP request message carries media picture combination capability indication information, and also carries multi-channel media picture information, and the first SDP request message can be carried in the It is transmitted in the Session Initiation Protocol (SIP) message, here is an example to illustrate: the media provider can provide 3 separate media video images (VC0, VC1, VC2), and the media provider requests in the first SDP The relevant description of the media screen information in the message is as follows:
a=video-compose-capa，指示具有画面组合能力；a=video-compose-capa, indicating that it has the ability to compose images;
m=video 10000 RTP/AVP 98，指示第一路画面信息VC0；m=video 10000 RTP/AVP 98, indicating the first video information VC0;
a=label:VC0a=label:VC0
a=rtpmap:98 H264/90000a=rtpmap:98 H264/90000
m=video 10002 RTP/AVP 98，指示第二路画面信息VC1；m=video 10002 RTP/AVP 98, indicating the second video information VC1;
a=label:VC1a=label:VC1
a=rtpmap:98 H264/9000a=rtpmap:98 H264/9000
m=video 10004RTP/AVP 31，指示第三路画面信息VC2；m=video 10004RTP/AVP 31, indicating the third video information VC2;
a=label:VC2a=label:VC2
a=rtpmap:31 H261/9000。a=rtpmap:31 H261/9000.
S402：接收媒体接收方发送的第一SDP响应消息，所述第一SDP响应消息中携带所述媒体画面组成信息，该媒体画面组成信息包括：至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，或，至少两路媒体画面的组合信息。S402: Receive the first SDP response message sent by the media receiver, the first SDP response message carries the media screen composition information, the media screen composition information includes: combination information of at least two media screens and at least two media screens Screen layout information, or combination information of at least two media screens.
具体的，媒体接收方接收媒体提供方发送的第一SDP请求消息，结合媒体接收方会议室的终端显示能力以及媒体接收方的本地策略，确定媒体画面组成信息，例如选择由哪几个媒体画面来进行组合、该组合画面如何布局等，并向媒体提供方发送第一SDP响应消息，该第一SDP响应消息中携带媒体画面组成信息；此处，同样以一个例子来说明：假设媒体接收方只有一个显示屏，只能接收1路视频画面，最终媒体接收方选择采用其中2路视频画面(VC0,VC1)来组合成一路，二路呈现左右平分画面；媒体接收方在第一SDP响应消息中的媒体画面组合的相关描述如下：Specifically, the media receiver receives the first SDP request message sent by the media provider, and combines the terminal display capability of the media receiver's conference room and the local strategy of the media receiver to determine the composition information of the media screen, such as which media screens to select to perform combination, how to layout the combined screen, etc., and send the first SDP response message to the media provider, the first SDP response message carries the composition information of the media screen; here, an example is also used to illustrate: suppose the media receiver There is only one display screen and can only receive 1 channel of video images. In the end, the media receiver chooses to use 2 channels of video images (VC0, VC1) to combine into one channel, and the second channel presents a left and right split image; the media receiver responds to the first SDP message The relevant descriptions of the media screen combinations in are as follows:
a=composed layout：VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0)，VC1(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)，表示媒体接收方希望将VC0,VC1组合，按照左右布局，各占50%；a=composed layout: VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0), VC1(512,0,0)(1024,0,0)(512,768,0) )(1024,760,0), indicating that the media receiver wants to combine VC0 and VC1, according to the left and right layout, each accounting for 50%;
m=video 20000 RTP/AVP 98，指示第一路画面信息VC0；m=video 20000 RTP/AVP 98, indicating the first video information VC0;
a=label：VC0a=label:VC0
a=rtpmap：98 H264/90000a=rtpmap:98 H264/90000
m=video 0 RTP/AVP 31，指示第二路画面信息VC1；m=video 0 RTP/AVP 31, indicating the second video information VC1;
a=label：VC1a=label:VC1
a=rtpmap：31 H261/9000a=rtpmap:31 H261/9000
m=video 0 RTP/AVP 31，指示第三路画面信息VC2；m=video 0 RTP/AVP 31, indicating the third video information VC2;
a=label：VC2a=label:VC2
a=rtpmap：31 H261/9000。a=rtpmap:31 H261/9000.
S403：根据对上述媒体画面组成信息的支持能力，向媒体接收方执行相应的操作。S403: Perform corresponding operations to the media receiver according to the support capability of the above media screen composition information.
具体的，媒体提供方接收媒体接收方发送的第一SDP响应消息后，根据对上述媒体画面组成信息的支持能力，向媒体接收方执行相应的操作，即S303具体可以分为两种实现方式：Specifically, after receiving the first SDP response message sent by the media receiver, the media provider performs corresponding operations to the media receiver according to the support capability of the above-mentioned media screen composition information, that is, S303 can be specifically divided into two implementations:
第一种：若媒体提供方支持至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持上述至少两路媒体画面的组合两种场景中的任一场景，则上述S403具体包括：The first one: if the media provider supports the combination of at least two media screens and the layout of at least two media screens, or supports any of the above two scenarios of the combination of at least two media screens, then the above S403 specifically includes :
S4031：发送组合画面的第一媒体参数给媒体接收方。S4031: Send the first media parameter of the combined picture to the media receiver.
S4032：接收媒体接收方发送的组合画面的第二媒体参数。S4032: Receive the second media parameter of the combined picture sent by the media receiver.
S4033：将根据至少两路媒体画面的组合信息、至少两路媒体画面的布局信息、第一媒体参数和第二媒体参数确定的组合画面发送给媒体接收方。S4033: Send the combined picture determined according to the combination information of the at least two channels of media pictures, the layout information of the at least two channels of media pictures, the first media parameter, and the second media parameter to the media receiver.
具体的，媒体提供方发送组合画面的第一媒体参数给媒体接收方，并且上述组合画面是媒体提供方根据媒体画面组成信息所确定的；媒体接收方接收到媒体提供方发送的第一媒体参数之后，向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，媒体接收方和媒体提供方根据媒体画面参数的协商结果，建立双向多流媒体传输通道，媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定组合画面，并将组合画面发送给媒体接收方予以呈现。Specifically, the media provider sends the first media parameter of the combined picture to the media receiver, and the combined picture is determined by the media provider according to the composition information of the media picture; the media receiver receives the first media parameter sent by the media provider Afterwards, send the second media parameter to the media provider, where the first media parameter and the second media parameter are the negotiation between the media provider and the media receiver on parameters such as codec mode and transmission bandwidth of the combined picture, the media receiver and the media receiver The media provider establishes a two-way multi-stream media transmission channel according to the negotiation result of the media screen parameters, and the media provider determines the composite screen according to the above media screen composition information, the first media parameter and the second media parameter, and sends the composite screen to the media receiver party to be presented.
第二种：若媒体提供方不支持媒体画面组成信息中的任一场景，则上述S403具体为：发送否定应答消息给媒体接收方。The second type: if the media provider does not support any scene in the media picture composition information, the above S403 is specifically: sending a negative response message to the media receiver.
具体的，媒体提供方发送否定应答消息给媒体接收方，向媒体接收方表示不接受该媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息携带在第一SDP响应消息中再次发送给媒体提供方，媒体提供方接收更新后的第一SDP响应消息，依然对其再次进行上述的判断过程。Specifically, the media provider sends a negative response message to the media receiver, expressing to the media receiver that it does not accept the media picture composition information, and the media receiver regenerates new media picture composition information after receiving the negative response message, namely Updating the composition information of at least two channels of media pictures and the layout information of at least two channels of media pictures, carrying the updated media picture composition information in the first SDP response message and sending it to the media provider again, and the media provider receives the updated first SDP response message An SDP response message, the above judgment process is still performed on it again.
本发明实施例提供的方法，通过媒体提供方向媒体接收方发送携带媒体画面组合能力指示信息的第一SDP请求消息，媒体接收方根据所接收到的第一SDP请求消息、媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，并将该媒体画面的组成信息携带在第一SDP响应消息中，媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the first SDP request message carrying the indication information of the media picture combination capability to the media receiver, and the media receiver displays the The capability and the local policy of the media receiver determine the composition information of the media screen, and carry the composition information of the media screen in the first SDP response message, and the media provider performs corresponding operations according to the media screen composition information determined by the media receiver , so that the media combination picture received by the media receiver is the media combination picture determined by the media provider according to the composition information of the media picture sent by the media receiver, that is, the media receiver can customize the combination picture it wants to present, thereby improving the combined media picture The flexibility of presentation improves the conference experience of media receiver users.
图4为本发明提供的多画面组合的呈现方法实施例四的流程示意图，本发明实施例的执行主体为媒体提供方，该方法包括：Fig. 4 is a schematic flowchart of Embodiment 4 of the presenting method of multi-picture combination provided by the present invention. The execution body of the embodiment of the present invention is the media provider, and the method includes:
S501：发送媒体画面组合能力指示信息以及媒体画面的缩略图给门户网站，以使媒体接收方从门户网站获取媒体画面组合能力指示信息和媒体画面的缩略图。S501: Send the indication information of the media picture combination capability and the thumbnail of the media picture to the portal website, so that the media receiver acquires the indication information of the media picture combination capability and the thumbnail of the media picture from the portal website.
具体的，本发明实施例中媒体提供方和媒体接收放都已经成功登陆到门户网站上；媒体提供方通过门户网站的接口将媒体画面组合能力指示信息和媒体画面的缩略图发送给门户网站，并且呈现在门户网站的界面上，以使媒体接收方从门户网站获取媒体画面组合能力指示信息和媒体画面的缩略图。Concretely, in the embodiment of the present invention, both the media provider and the media receiver have successfully logged on to the portal website; the media provider sends the media screen combination capability indication information and the thumbnail of the media screen to the portal website through the interface of the portal website, And it is presented on the interface of the portal website, so that the media receiver can obtain the indication information of the media picture combination capability and the thumbnail of the media picture from the portal website.
S502：从上述门户网站获取媒体接收方发送的根据媒体画面组合能力指示信息和媒体画面的缩略图确定的媒体画面组成信息，该媒体画面组成信息包括：至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，或，至少两路媒体画面的组合信息。S502: Obtain from the above-mentioned portal website the media screen composition information determined according to the media screen combination capability indication information and the thumbnail of the media screen sent by the media receiver. The media screen composition information includes: combination information of at least two media screens and at least two The layout information of one channel of media pictures, or the combination information of at least two channels of media pictures.
具体的，媒体接收方通过门户网站接口查看到媒体提供方提供的媒体画面缩略图，从门户网站获取媒体画面组合能力指示信息和媒体画面的缩略图，并结合媒体接收方的会议室的终端显示能力以及媒体接收方本地策略，确定媒体画面组成信息，并将该媒体画面组成信息中包含至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，或，至少两路媒体画面的组合信息，通过门户网站接口呈现在门户网站界面上；媒体提供方从门户网站上获取该媒体画面组成信息。Specifically, the media receiver views the thumbnail image of the media screen provided by the media provider through the interface of the portal website, obtains the indication information of the media screen composition capability and the thumbnail image of the media screen from the portal website, and combines the information displayed on the terminal in the conference room of the media receiver capability and the local strategy of the media receiver, determine the composition information of the media screen, and include the combination information of at least two media screens and the layout information of at least two media screens in the media screen composition information, or the combination of at least two media screens The information is presented on the portal website interface through the portal website interface; the media provider obtains the composition information of the media screen from the portal website.
S503：根据对上述媒体画面组成信息的支持能力，向媒体接收方执行相应的操作。S503: Perform corresponding operations to the media receiver according to the support capability of the composition information of the above media screen.
具体的，媒体提供方从门户网站上获取该媒体画面组成信息之后，对该媒体画面组成信息中所包含的内容进行是否支持的判断，分为两种实现方式：Specifically, after the media provider obtains the composition information of the media screen from the portal website, it judges whether the content contained in the composition information of the media screen supports or not. There are two implementation methods:
第一种：若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持上述至少两路媒体画面的组合两种场景中的任一场景，则媒体提供方发送组合画面的第一媒体参数给门户网站，以使媒体接收方从门户网站获取第一媒体参数；媒体接收方从门户网站获取到媒体提供方发送的第一媒体参数之后，向门户网站发送第二媒体参数，以使媒体提供方可以从门户网站获取第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商；之后，媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定组合画面，并将组合画面发送给媒体接收方予以呈现。Type 1: If the media provider supports the above-mentioned combination of at least two media screens and the layout of at least two media screens, or supports any of the two scenarios of the above-mentioned combination of at least two media screens, the media provider Send the first media parameter of the combined picture to the portal website, so that the media receiver obtains the first media parameter from the portal website; after the media receiver obtains the first media parameter sent by the media provider from the portal website, it sends the first media parameter to the portal website Two media parameters, so that the media provider can obtain the second media parameter from the portal website, where the first media parameter and the second media parameter are parameters such as the codec method and transmission bandwidth of the combined screen by the media provider and the media receiver Afterwards, the media provider determines the combined picture according to the above media picture composition information, the first media parameter and the second media parameter, and sends the combined picture to the media receiver for presentation.
第二种：若媒体提供方不支持上述媒体画面组成信息中的任一场景，则发送否定应答消息给门户网站，以使媒体接收方从门户网站获知媒体提供方不接受该媒体画面组成信息，则媒体接收方会重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息再次发送给门户网站，媒体提供方从门户网站获取更新后的媒体画面组成信息，依然对其再次进行上述的判断过程。The second method: if the media provider does not support any scenario in the above media screen composition information, then send a negative response message to the portal website, so that the media receiver knows from the portal website that the media provider does not accept the media screen composition information, Then the media receiver will regenerate new media screen composition information, that is, update the combination information of at least two media screens and the layout information of at least two media screens, and send the updated media screen composition information to the portal website again, and the media provides The party obtains the updated media screen composition information from the portal website, and still performs the above-mentioned judgment process on it again.
本发明实施例提供的方法，通过媒体提供方向门户网站发送携带媒体画面组合能力指示信息和媒体画面缩略图，媒体接收方从门户网站获取该媒体画面组合能力指示信息和媒体画面缩略图后，结合媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，并将该媒体画面的组成信息发送给门户网站；媒体提供方根据从门户网站获取的媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the media screen combination capability indication information and the media screen thumbnail to the portal website, and the media receiver acquires the media screen combination capability indication information and the media screen thumbnail from the portal website, and combines the The terminal display capability of the media receiver and the local strategy of the media receiver determine the composition information of the media screen, and send the composition information of the media screen to the portal website; The screen composition information performs corresponding operations, so that the media combination screen received by the media receiver is the media combination screen determined by the media provider according to the media screen composition information sent by the media receiver, that is, the media receiver can customize the combination screen it wants to present , so as to improve the flexibility of presenting combined media screens and improve the conference experience of the media receiver user.
在图2和图3所示的实施例的基础上，进一步地，S2031和S4031具体为：发送第三SDP请求消息给媒体接收方，该第三SDP请求消息中携带组合画面的第一媒体参数；则S2032和S4032具体为：接收所述媒体接收方发送的第三SDP响应消息，所述第三SDP响应消息中携带所述组合画面的第二媒体参数。On the basis of the embodiment shown in Figure 2 and Figure 3, further, S2031 and S4031 are specifically: sending a third SDP request message to the media receiver, the third SDP request message carrying the first media parameter of the combined screen ; Then S2032 and S4032 are specifically: receiving a third SDP response message sent by the media receiver, where the third SDP response message carries the second media parameter of the combined screen.
图5为本发明提供的多画面组合的呈现方法实施例五的流程示意图，本实施例主要涉及的是在视频会议召开之后，由于某种需要，可以是网络情况变化（比如拥塞，或者组合画面中某一源会场离会，或者新的会场入会等），组合画面需要做出调整，即媒体组成信息需要进行变更的处理方法；本发明实施例的执行主体是媒体提供方，在图1所示实施例的基础上，在S103之后，该方法还包括：Fig. 5 is a schematic flowchart of Embodiment 5 of the presenting method of multi-screen combination provided by the present invention. This embodiment mainly involves that after a video conference is held, due to certain needs, it may be a change in network conditions (such as congestion, or combined screen One of the source conference sites leaves the conference, or a new conference site joins the conference, etc.), the combined screen needs to be adjusted, that is, the processing method that the media composition information needs to be changed; the execution subject of the embodiment of the present invention is the media provider, as shown in Figure 1 On the basis of the illustrated embodiment, after S103, the method further includes:
S601：发送媒体画面变化信息给媒体接收方。S601: Send media screen change information to a media receiver.
具体的，在视频会议召开之后，即在媒体提供方发送组合画面给媒体接收方之后，媒体提供方检测到网络情况发生变化，比如拥塞或者组合画面中某一源会场离会或者新的会场入会等，则媒体提供方会向媒体接收方发送媒体画面变化信息，其中，该媒体画面变化信息可以携带在CLUE媒体通告消息中，也可以携带在SDP请求消息中，此处以该媒体画面变化信息携带在CLUE媒体通告消息中为例来说明，假设媒体提供方原来提供3路单独视频画面(VC0,VC1,VC2)，媒体接收方选择VC0，VC1进行组合，这时VC0画面消失，媒体提供方在CLUE媒体通告消息中通知组合画面变更，具体描述如下：Specifically, after the video conference is held, that is, after the media provider sends the combined picture to the media receiver, the media provider detects that the network situation changes, such as congestion or a source site in the combined picture leaves the conference or a new site joins the conference etc., then the media provider will send the media screen change information to the media receiver, wherein the media screen change information can be carried in the CLUE media notification message, or can be carried in the SDP request message, where the media screen change information is carried Take the CLUE media announcement message as an example. Suppose the media provider originally provided 3 separate video images (VC0, VC1, VC2), and the media receiver selects VC0 and VC1 for combination. The CLUE media notification message notifies the composition screen change, and the specific description is as follows:
Advertisement：Advertisements:
Capture Scene1：Capture Scene1:
Capture Scene Entry1：VC1,VC2Capture Scene Entry1: VC1, VC2
Capture Scene Entry2：VC4Capture Scene Entry2: VC4
VC1：encoding group=EG0，content=view:table.center；VC1: encoding group=EG0, content=view:table.center;
VC2：encoding group=EG0，content=view:table.right；VC2: encoding group=EG0, content=view:table.right;
VC4：encoding group=EG1，content=view:room,composed=true,composedupdate={VC0：off，VC1}；VC4: encoding group=EG1, content=view:room, composed=true, composedupdate={VC0:off, VC1};
上述例子中，媒体画面组合结果包含在composed update中，表示组合画面变更，VC0关闭。In the above example, the composition result of the media screen is included in the composed update, indicating that the composite screen has changed, and VC0 is closed.
S602：接收媒体接收方根据上述媒体画面变化信息生成的更新后的第一媒体画面组成信息，该第一媒体画面组成信息中包括：至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，或，至少两路媒体画面的第一更新组合信息。S602: Receive the updated first media screen composition information generated by the media recipient according to the above media screen change information, the first media screen composition information includes: the first updated combination information of at least two media screens and at least two media screens The first updated layout information of the screen, or the first updated combination information of at least two media screens.
具体的，媒体接收方在接收到媒体提供方发送的媒体画面变化信息之后，获知该视频会议的网络情况发生变化，则媒体接收方根据该媒体画面变化信息生成更新后的第一媒体画面组成信息，该第一媒体画面组成信息中包括：至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，或，至少两路媒体画面的第一更新组合信息；并将该第一媒体画面组成信息发送给媒体提供方，其中，该第一媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP请求响应消息中，与S601中所举例子对应，这里同样以第一媒体画面组成信息携带在CLUE媒体配置消息中为例来说明：媒体接收方选择采用2路视频画面(VC1,VC2)来组合成一路，二路呈现左右平分画面，媒体接收方在CLUE媒体配置消息中的新的媒体画面组合选择相关描述如下：Specifically, after receiving the media screen change information sent by the media provider, the media receiver learns that the network conditions of the video conference have changed, and the media receiver generates updated first media screen composition information according to the media screen change information , the first media picture composition information includes: first updated combination information of at least two media pictures and first updated layout information of at least two media pictures, or, first updated combination information of at least two media pictures; and Send the first media screen composition information to the media provider, where the first media screen composition information can be carried in the CLUE media configuration message, or can be carried in the SDP request response message, corresponding to the example in S601, Here also take the first media picture composition information carried in the CLUE media configuration message as an example to illustrate: the media receiver chooses to use 2 channels of video pictures (VC1, VC2) to combine into one channel, and the second channel presents a left and right split screen. The relevant description of the new media picture combination selection in the CLUE media configuration message is as follows:
Configuration：Configuration:
Capture Scene1：Capture Scene1:
Capture Scene Entry2：VC4Capture Scene Entry2: VC4
VC4:encoding group=EG1content=view:room.partial,composed=true,VC4:encoding group=EG1content=view:room.partial,composed=true,
composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
S603：根据对上述第一媒体画面组成信息的支持能力，向媒体接收方执行相应的操作。S603: Perform corresponding operations to the media receiver according to the support capability of the composition information of the first media screen.
具体的，媒体提供方接收媒体接收方发送的第一媒体画面组成信息，并对第一媒体组成信息中的内容执行判断操作，即判断是否支持至少两路媒体画面的第一更新组合和至少两路媒体画面的第一更新布局，或，是否支持至少两路媒体画面的第一更新组合两种场景中的任一场景，可以分为两种实现方式：Specifically, the media provider receives the first media screen composition information sent by the media receiver, and performs a judgment operation on the content in the first media composition information, that is, judges whether the first update combination of at least two media screens and at least two media screens are supported. Whether to support the first update layout of at least two media screens, or whether to support the first update of at least two media screens to combine any of the two scenarios, can be divided into two implementation methods:
第一种：若媒体提供方支持上述第一媒体画面组成信息中的任一场景，可选的，媒体提供方可以发送组合画面的第一媒体参数给媒体接收方；媒体接收方接收到媒体提供方发送的第一媒体参数之后，会向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，需要注意的，本实施例中也可以不进行组合画面的第一媒体参数和组合画面的第二媒体参数的交互，因为在视频会议召开之前媒体发送方和媒体接收方针对组合画面的第一媒体参数和第二媒体参数已经协商过了；之后，媒体提供方根据上述第一媒体画面组成信息、第一媒体参数和第二媒体参数确定更新后的组合画面，并将更新后组合画面发送给媒体接收方予以呈现。Type 1: If the media provider supports any scene in the composition information of the first media picture above, optionally, the media provider can send the first media parameters of the combined picture to the media receiver; the media receiver receives the media provided After the first media parameter sent by the media provider, the second media parameter will be sent to the media provider. Here, the first media parameter and the second media parameter are the codec method and transmission bandwidth of the combined screen by the media provider and the media receiver. For the negotiation of parameters, it should be noted that in this embodiment, the interaction between the first media parameter of the combined picture and the second media parameter of the combined picture may not be performed, because before the video conference is held, the media sender and media receiver will The first media parameter and the second media parameter have been negotiated; after that, the media provider determines the updated combined picture according to the composition information of the first media picture, the first media parameter and the second media parameter, and combines the updated The picture is sent to the media receiver for presentation.
第二种：若媒体提供方不支持上述第一媒体画面组成信息中的任一场景，则发送否定应答消息给媒体接收方，向媒体接收方表示不接受该第一媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，将更新后的媒体画面组成信息再次发送给媒体提供方，媒体提供方接收更新后的媒体画面组成信息，依然对其再次进行上述的判断过程。The second type: if the media provider does not support any scene in the above-mentioned first media picture composition information, then send a negative response message to the media receiver, expressing to the media receiver that it does not accept the first media picture composition information, and the media receiver After receiving the negative response message, the party regenerates new media picture composition information, that is, updates the first updated combination information of at least two channels of media pictures and the first updated layout information of at least two channels of media pictures, and updates the updated media picture The screen composition information is sent to the media provider again, and the media provider receives the updated media screen composition information, and still performs the above judgment process on it again.
在图5所示实施例的基础上，进一步地，上述的媒体画面变化信息可以携带在CLUE媒体通告消息中，第一媒体画面组成信息可以携带在CLUE媒体配置消息中，即图5所示的实施例可以与图2所示的实施例进行结合（在S203之后执行S601、S602、S603）；或者，媒体画面变化信息可以携带在SDP请求消息中，第一媒体画面组成信息可以携带在SDP响应消息中，即图5所示的实施例可以与图3所示的实施例进行结合（在S403之后，执行S601、S602、S603）；或者，媒体提供方可以将媒体画面变化信息发送给门户网站，由媒体接收方从门户网站获取该媒体画面变化信息，从而发送第一媒体组成信息给门户网站，以使媒体提供方从门户网站获取第一媒体组成信息，即图5所示的实施例可以与图4所示的实施例进行结合（在S503之后，执行S601、S602、S603）。On the basis of the embodiment shown in FIG. 5 , further, the above-mentioned media screen change information may be carried in the CLUE media notification message, and the first media screen composition information may be carried in the CLUE media configuration message, that is, as shown in FIG. 5 The embodiment may be combined with the embodiment shown in FIG. 2 (S601, S602, and S603 are executed after S203); or, the media screen change information may be carried in the SDP request message, and the first media screen composition information may be carried in the SDP response In the message, that is, the embodiment shown in Figure 5 can be combined with the embodiment shown in Figure 3 (after S403, perform S601, S602, S603); or, the media provider can send the media screen change information to the portal website , the media receiver obtains the media screen change information from the portal website, thereby sending the first media composition information to the portal website, so that the media provider obtains the first media composition information from the portal website, that is, the embodiment shown in FIG. 5 can Combining with the embodiment shown in FIG. 4 (after S503, execute S601, S602, S603).
本发明实施例提供的方法，在视频会议召开之后，网络发生变化的情况下，通过媒体提供方向媒体接收方发送媒体画面变化信息，以使媒体接收方根据媒体画面变化信息生成更新后的第一媒体画面组成信息，媒体提供方根据第一媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的第一媒体画面组成信息确定的媒体组合画面，同时还使得媒体提供方和媒体接收方可以实时根据网络变化情况对所所组合的媒体画面做出调整，提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度和适用性。In the method provided by the embodiment of the present invention, when the network changes after the video conference is held, the media provider sends the media screen change information to the media receiver, so that the media receiver can generate an updated first video according to the media screen change information. Media screen composition information, the media provider performs corresponding operations according to the first media screen composition information, so that the media combination screen received by the media receiver is the media combination determined by the media provider according to the first media screen composition information sent by the media receiver At the same time, it also enables the media provider and the media receiver to make real-time adjustments to the combined media screen according to network changes, which improves the flexibility of the combined media screen presentation and improves the conference experience and experience of the media receiver user. applicability.
图6为本发明提供的多画面组合的呈现方法实施例六的流程示意图，本发明实施例涉及的方法主要是媒体接收方自行发起组合画面调整的指示，该方法的执行主体是媒体提供方，在上述图1所示实施例的基础上，在S103之后，该方法还包括：Fig. 6 is a schematic flowchart of Embodiment 6 of the presenting method of multi-picture combination provided by the present invention. The method involved in the embodiment of the present invention is mainly that the media receiver initiates an instruction to adjust the combined screen by itself, and the execution subject of the method is the media provider. On the basis of the above-mentioned embodiment shown in FIG. 1, after S103, the method further includes:
S701：接收所述媒体接收方更新后的第二媒体画面组成信息，该第二媒体画面组成信息中包括：至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，或，至少两路媒体画面的第二更新组合信息。S701: Receive the updated second media screen composition information from the media receiver, the second media screen composition information includes: the second updated combination information of at least two media screens and the second updated layout of at least two media screens information, or the second update combination information of at least two channels of media images.
S702：根据对上述第二媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。S702: Perform a corresponding operation to the media receiver according to the support capability of the composition information of the second media picture.
具体的，媒体接收方自行想更新所呈现的组合画面，则媒体接收方根据自身的能力生成更新后的第二媒体画面组成信息，并将该第二媒体画面组成信息发送给媒体提供方，该第二媒体画面组成信息中包括：至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，或，至少两路媒体画面的第二更新组合信息，其中，该第二媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP响应消息中，这里以第二媒体画面组成信息携带在CLUE媒体配置消息中为例：媒体接收方选择采用2路视频画面(VC1,VC2)来组合成一路，二路呈现左右平分画面，媒体接收方在CLUE媒体配置消息中的新媒体画面组合选择相关描述如下：Specifically, if the media receiver wants to update the presented combined screen, the media receiver generates updated second media screen composition information according to its own capabilities, and sends the second media screen composition information to the media provider. The second media screen composition information includes: the second updated combination information of at least two media screens and the second updated layout information of at least two media screens, or the second updated combination information of at least two media screens, wherein the The composition information of the second media screen can be carried in the CLUE media configuration message, and can also be carried in the SDP response message. Here, the composition information of the second media screen is carried in the CLUE media configuration message as an example: the media receiver chooses to use 2-channel video Screens (VC1, VC2) are combined into one channel, and the second channel presents a left and right split screen. The media receiver’s new media screen combination selection in the CLUE media configuration message is described as follows:
Configuration:Configuration:
Capture Scene1：Capture Scene1:
Capture Scene Entry2:VC4Capture Scene Entry2: VC4
VC4：encoding group=EG1,content=view:room.partial,composed=true,VC4: encoding group=EG1, content=view:room.partial, composed=true,
composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
媒体提供方接收媒体接收方发送的第二媒体画面组成信息，根据对上述第二媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作，即判断是否支持至少两路媒体画面的第二更新组合和至少两路媒体画面的第二更新布局，或，是否支持至少两路媒体画面的第二更新组合这两种场景中的任一场景，可以分为两种实现方式：The media provider receives the second media screen composition information sent by the media receiver, and performs corresponding operations to the media receiver according to the support capability of the above-mentioned second media screen composition information, that is, judges whether to support at least two media screens. The second update combination and the second update layout of at least two media images, or whether to support the second update combination of at least two media images, can be divided into two implementations:
第一种：若媒体提供方支持上述第二媒体画面组成信息中的任一场景，可选的，媒体提供方可以发送组合画面的第一媒体参数给媒体接收方；媒体接收方接收到媒体提供方发送的第一媒体参数之后，向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，需要注意的，本实施例中也可以不进行组合画面的第一媒体参数和组合画面的第二媒体参数的交互，因为在视频会议召开之前媒体发送方和媒体接收方针对组合画面的第一媒体参数和第二媒体参数已经协商过了；之后，媒体提供方根据上述第二媒体画面组成信息、第一媒体参数和第二媒体参数确定更新后的组合画面，并将更新后组合画面发送给媒体接收方予以呈现。The first one: if the media provider supports any scene in the composition information of the second media picture above, optionally, the media provider can send the first media parameters of the combined picture to the media receiver; the media receiver receives the media provided After the first media parameter sent by the party, the second media parameter is sent to the media provider, where the first media parameter and the second media parameter are parameters such as the codec method and the transmission bandwidth of the combined screen by the media provider and the media receiver It should be noted that in this embodiment, the interaction between the first media parameter of the combined picture and the second media parameter of the combined picture may not be performed, because before the video conference is held, the media sender and the media receiver will The first media parameter and the second media parameter have been negotiated; afterward, the media provider determines the updated combined picture according to the above-mentioned second media picture composition information, the first media parameter and the second media parameter, and updates the combined picture Sent to the media receiver for rendering.
第二种：若媒体提供方不支持上述第二媒体画面组成信息中的任一场景，则发送否定应答消息给媒体接收方，向媒体接收方表示不接受该第二媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，将更新后的媒体画面组成信息再次发送给媒体提供方，媒体提供方接收更新后的媒体画面组成信息，依然对其再次进行上述的判断过程。The second type: if the media provider does not support any scene in the above-mentioned second media picture composition information, then send a negative response message to the media receiver, expressing to the media receiver that it does not accept the second media picture composition information, and the media receiver After receiving the negative response message, the party regenerates new media picture composition information, that is, updates the second update combination information of at least two channels of media pictures and the second update layout information of at least two channels of media pictures, and updates the updated media picture The screen composition information is sent to the media provider again, and the media provider receives the updated media screen composition information, and still performs the above judgment process on it again.
在图6所示实施例的基础上，进一步地，上述第二媒体画面组成信息可以携带在CLUE媒体配置消息中，即图6所示的实施例可以与图2所示的实施例进行结合（在S203之后执行S701、S702）；或者，第二媒体画面组成信息可以携带在SDP响应消息中，即图6所示的实施例可以与图3所示的实施例进行结合（在S403之后，执行S701、S702）；或者，媒体接收方发送第二媒体组成信息给门户网站，以使媒体提供方从门户网站获取第二媒体组成信息，即图5所示的实施例可以与图4所示的实施例进行结合（在S503之后，执行S701、S702）。On the basis of the embodiment shown in FIG. 6, further, the above-mentioned second media screen composition information may be carried in the CLUE media configuration message, that is, the embodiment shown in FIG. 6 may be combined with the embodiment shown in FIG. 2 ( Execute S701 and S702 after S203); or, the second media picture composition information may be carried in the SDP response message, that is, the embodiment shown in FIG. 6 may be combined with the embodiment shown in FIG. 3 (after S403, execute S701, S702); or, the media receiver sends the second media composition information to the portal website, so that the media provider obtains the second media composition information from the portal website, that is, the embodiment shown in FIG. 5 can be compared with that shown in FIG. 4 The embodiment is combined (after S503, execute S701 and S702).
本发明实施例提供的方法，在视频会议召开之后，媒体提供方自行更新媒体画面，即媒体接收方自行生成更新后的第二媒体画面组成信息，媒体提供方根据第二媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的第二媒体画面组成信息确定的媒体组合画面，提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度和适用性。In the method provided by the embodiment of the present invention, after the video conference is held, the media provider updates the media screen by itself, that is, the media receiver generates the updated second media screen composition information by itself, and the media provider executes the corresponding operation according to the second media screen composition information. The operation makes the media combination picture received by the media receiver be the media combination picture determined by the media provider according to the composition information of the second media picture sent by the media receiver, which improves the flexibility of combining media picture presentation and improves the media receiver’s User's meeting experience and applicability.
图7为本发明提供的多画面组合的呈现方法实施例七的流程示意图，本发明实施例涉及的方法是媒体提供方提供多种默认的组合画面给媒体接收方，以使媒体接收方根据自身的终端显示能力和本地策略进行选择，本发明实施例的执行主体是媒体提供方，该方法包括如下步骤：Fig. 7 is a schematic flow chart of Embodiment 7 of the multi-picture combination presentation method provided by the present invention. The method involved in the embodiment of the present invention is that the media provider provides a variety of default combined screens to the media receiver, so that the media receiver can use its own The terminal display capability and the local strategy are selected. The execution subject of the embodiment of the present invention is the media provider. The method includes the following steps:
S801：发送至少一种媒体画面组合选择信息给媒体接收方。S801: Send at least one media screen combination selection information to a media receiver.
S802：接收媒体接收方根据上述至少一种媒体画面组合选择信息确定媒体画面组成信息。S802: receiving the media The receiver determines media screen composition information according to the at least one media screen combination selection information.
具体的，媒体提供方向媒体接收方发送至少一种媒体画面组合选择信息，该媒体画面组合选择信息为媒体提供方自行组合的媒体组合画面，其中，该媒体画面组合选择信息可以携带在CLUE媒体通告消息中，也可以携带在SDP请求消息中，还可以是媒体提供方将至少一种的媒体画面组合选择信息发送给门户网站，由媒体接收方从门户网站获取该至少一种媒体画面组合选择信息，此处以上述媒体画面组合选择信息携带在CLUE媒体配置消息中为例：媒体提供方原来提供3路单独视频画面(VC0,VC1,VC2)，默认的组合方式为：三个单独画面的组合{VC0,VC1,VC2}，两个单独画面的组合{VC0，VC1}，媒体提供方在CLUE媒体通告消息中通知默认的媒体组合画面，具体描述如下：Specifically, the media provider sends at least one media picture combination selection information to the media receiver, and the media picture combination selection information is a media combination picture assembled by the media provider itself, wherein the media picture combination selection information can be carried in the CLUE media announcement In the message, it may also be carried in the SDP request message. It may also be that the media provider sends at least one media screen combination selection information to the portal website, and the media receiver obtains the at least one media screen combination selection information from the portal website. , here is an example where the above media screen combination selection information is carried in the CLUE media configuration message: the media provider originally provided 3 separate video screens (VC0, VC1, VC2), and the default combination method is: a combination of three separate screens{ VC0, VC1, VC2}, the combination of two separate screens {VC0, VC1}, the media provider notifies the default media combination screen in the CLUE media announcement message, the specific description is as follows:
Advertisement：Advertisements:
Capture Scene1：Capture Scene1:
Capture Scene Entry1：VC0，VC1，VC2Capture Scene Entry1: VC0, VC1, VC2
Capture Scene Entry2：VC4Capture Scene Entry2: VC4
VC0：encoding group=EG0，content=view:table.left；VC0: encoding group=EG0, content=view:table.left;
VC1：encoding group=EG0，content=view:table.center；VC1: encoding group=EG0, content=view:table.center;
VC2：encoding group=EG0，content=view:table.right；VC2: encoding group=EG0, content=view:table.right;
VC4：encoding group=EG1，content=view:room，composed=true，composedoption={composed layout1，composed layout2}，VC4: encoding group=EG1, content=view:room, composed=true, composedoption={composed layout1, composed layout2},
composed layout1={VC0(0,0,0)(341,0,0)(0,768,0)(512,768,0)，composed layout1={VC0(0,0,0)(341,0,0)(0,768,0)(512,768,0),
VC1(512,384,0)(1024,384,0)(512,768,0)(1024,768,0)，VC1(512,384,0)(1024,384,0)(512,768,0)(1024,768,0),
VC2(512,0,0)(1024,0,0)(512,384,0)(1024,384,0)}，VC2(512,0,0)(1024,0,0)(512,384,0)(1024,384,0)},
Composed layout2={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0)，Composed layout2={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
媒体接收方接收媒体提供方发送的至少一种媒体画面组合选择信息，并结合媒体接收方的终端显示能力、媒体接收方的本地策略确定媒体画面组成信息，该媒体画面组成信息中包括了媒体接收方的组合画面的选择结果，该媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP响应消息中，还可以是媒体接收方将媒体画面组成信息发送给门户网站，由媒体提供方从门户网站获取媒体接收方的组合画面的选择结果，此处以媒体画面组成信息携带在CLUE媒体配置消息中为例，媒体接收方选择采用3路视频画面(VC0,VC1,VC2)的组合，媒体接收方在CLUE媒体配置消息中的媒体画面组合选择相关描述如下：The media receiver receives at least one media screen combination selection information sent by the media provider, and determines the media screen composition information in combination with the media receiver's terminal display capability and the media receiver's local policy. The media screen composition information includes the media reception The media screen composition information can be carried in the CLUE media configuration message or in the SDP response message. It can also be that the media receiver sends the media screen composition information to the portal website, which is provided by the media. The party obtains the selection result of the combined picture of the media receiver from the portal website. Here, taking the media picture composition information carried in the CLUE media configuration message as an example, the media receiver chooses to use a combination of 3 video pictures (VC0, VC1, VC2). The media receiver's media screen combination selection in the CLUE media configuration message is described as follows:
Configuration:Configuration:
Capture Scene1:Capture Scene1:
Capture Scene Entry2:VC4Capture Scene Entry2: VC4
VC4:encoding group=EG1,content=view:room.full,composed=true,composedoption={composed layout1}。VC4:encoding group=EG1,content=view:room.full,composed=true,composed option={composed layout1}.
媒体提供方获知媒体接收方的组合画面选择结果之后，向媒体接收方发送组合画面的第一媒体参数，该第一媒体参数可以携带在第三SDP响应消息中，还可以是媒体提供方将该组合画面的第一媒体参数发送给门户网站，由媒体接收方从门户网站获取；媒体接收方接收到组合画面的第一媒体参数之后，向媒体提供方发送组合画面的第二媒体参数，该第二媒体参数可以携带在第三SDP响应消息中，还可以是媒体接收方将组合画面的第二媒体参数发送给门户网站，由媒体提供方从门户网站获取该组合画面的第二媒体参数，需要注意的是，媒体提供方和媒体接收方之间进行组合画面的第一媒体参数和第二媒体参数的交互主要是进行组合画面的编解码方式和带宽等参数的协商；之后，媒体提供方将媒体接收方选择的组合画面发送给媒体接收方予以呈现。After the media provider learns the selection result of the combined picture of the media receiver, it sends the first media parameter of the combined picture to the media receiver. The first media parameter can be carried in the third SDP response message, or the media provider can send the combined picture The first media parameter of the combined picture is sent to the portal website, and the media receiver obtains it from the portal website; after the media receiver receives the first media parameter of the combined picture, it sends the second media parameter of the combined picture to the media provider. The second media parameter can be carried in the third SDP response message, and the media receiver can also send the second media parameter of the combined picture to the portal website, and the media provider can obtain the second media parameter of the combined picture from the portal website. It should be noted that the interaction between the first media parameter and the second media parameter of the combined picture between the media provider and the media receiver is mainly to negotiate parameters such as codec mode and bandwidth of the combined picture; after that, the media provider will The combined picture selected by the media receiver is sent to the media receiver for presentation.
本实施例提供的方法中，通过媒体提供方将至少一种媒体画面组合选择信息发送给媒体接收方，媒体接收方根据其终端的显示能力以及本地策略媒体画面组合选择信息进行选择，并将选择结果携带在媒体画面组成信息中发送给媒体提供方，以使媒体提供方将对应的组合画面发送给媒体接收方，即媒体接收方是按照自身的意愿或者能力对组合画面进行选择和呈现，因此提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided in this embodiment, the media provider sends at least one media screen combination selection information to the media receiver, and the media receiver selects according to the display capability of its terminal and the local strategy media screen combination selection information, and the selected The result is carried in the media screen composition information and sent to the media provider, so that the media provider can send the corresponding combined screen to the media receiver, that is, the media receiver selects and presents the combined screen according to its own wishes or capabilities, so Improves the flexibility of combining media screen presentation, and improves the conference experience of media receiver users.
图8为本发明提供的多画面组合的呈现方法实施例八的流程示意图，本实施例涉及的方法的执行主体为媒体接收方，该方法包括：Fig. 8 is a schematic flowchart of Embodiment 8 of the presenting method of multi-picture combination provided by the present invention. The execution subject of the method involved in this embodiment is the media receiver, and the method includes:
S901：接收媒体提供方发送的媒体画面组合能力指示信息。S901: Receive media screen combination capability indication information sent by a media provider.
S902：根据媒体画面组合能力指示信息确定媒体画面组成信息。S902: Determine media picture composition information according to the media picture combination capability indication information.
S903：将媒体画面组成信息发送给媒体提供方，以使媒体提供方根据对上述媒体画面组成信息的支持能力，向媒体接收方执行相应的操作。S903: Send the media screen composition information to the media provider, so that the media provider performs corresponding operations to the media receiver according to the support capability of the above media screen composition information.
具体的，媒体提供方发送媒体画面组合能力指示信息给媒体接收方，该媒体画面组合能力指示信息用于向媒体接收方告知该媒体提供方具有组合媒体画面的能力，媒体画面组合能力指示信息可以有多种不同的实现方式：例如简单的仅仅说明媒体发送方可以组合；或者媒体发送方给出一些多路媒体画面的组合、布局信息的可选项等等；媒体接收方在收到媒体提供方发送的媒体画面组合能力指示信息之后，结合该媒体画面组合能力指示信息、媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面组成信息，并且这里的媒体画面组成信息可以包括至少两路媒体画面的组合信息和布局信息的场景，或者，包括仅有至少两路媒体画面组合信息的场景；。Specifically, the media provider sends media picture combination capability indication information to the media receiver, and the media picture combination capability indication information is used to inform the media receiver that the media provider has the ability to combine media pictures, and the media picture combination capability indication information can be There are many different implementation methods: for example, it simply means that the media sender can combine; or the media sender provides some combinations of multi-channel media screens, optional options for layout information, etc.; the media receiver receives the media provider After sending the media picture composition capability indication information, combine the media picture composition capability indication information, the terminal display capability of the media receiver, and the local policy of the media receiver to determine the media picture composition information, and the media picture composition information here may include at least two A scene with combination information and layout information of one channel of media pictures, or a scene with combination information of at least two channels of media pictures;
媒体接收方将上述媒体画面组成信息发送给媒体提供方，媒体提供方根据对上述媒体画面组成信息中任一场景的的支持能力，向媒体接收方执行相应的操作，例如：若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，则媒体接收方接收媒体提供方按照媒体组成信息以及相关的媒体参数发送的组合画面；若媒体提供方不支持至少两路媒体画面的组合和至少两路媒体画面的布局，则媒体接收方接收媒体提供方发送的否定应答消息。The media receiver sends the above media screen composition information to the media provider, and the media provider performs corresponding operations to the media receiver according to its ability to support any scene in the above media screen composition information, for example: if the media provider supports The combination of the above-mentioned at least two media screens and the layout of at least two media screens, the media receiver receives the combined screen sent by the media provider according to the media composition information and related media parameters; if the media provider does not support at least two media screens combination and the layout of at least two media screens, the media receiver receives the negative acknowledgment message sent by the media provider.
本发明实施例提供的方法，通过媒体接收方接收媒体提供方发送的媒体画面组合能力指示信息，并结合媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，即确定所要呈现的媒体画面的组合信息以及布局信息，使得媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media receiver receives the indication information of the media screen combination capability sent by the media provider, and combines the media receiver's terminal display capability and the media receiver's local policy to determine the composition information of the media screen, that is, to determine The combination information and layout information of the media screen to be presented enables the media provider to perform corresponding operations according to the composition information of the media screen determined by the media receiver, so that the media receiver can customize the combined screen to be presented by itself, thereby improving the combined media The flexibility of screen presentation improves the conference experience of media receiver users.
在图8所示实施例的基础上，进一步地，上述S903包括：若媒体提供方支持媒体画面组成信息，则接收媒体提供方发送的组合画面的第一媒体参数；并发送组合画面的第二媒体参数给媒体提供方；并接收媒体提供方根据所述媒体画面组成信息、第一媒体参数和第二媒体参数确定的组合画面；或者，若媒体提供方不支持媒体画面组成信息，则接收媒体提供方发送的否定应答消息。On the basis of the embodiment shown in FIG. 8 , further, the above S903 includes: if the media provider supports media screen composition information, then receiving the first media parameter of the combined screen sent by the media provider; and sending the second media parameter of the combined screen. The media parameter is given to the media provider; and the combined picture determined by the media provider according to the composition information of the media screen, the first media parameter and the second media parameter is received; or, if the media provider does not support the composition information of the media screen, the media is received A negative acknowledgment message sent by the provider.
具体的，媒体接收方将媒体画面组成信息发送给媒体提供方，该媒体画面组成信息包括：至少两路媒体画面的组合信息，或，至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，使得媒体提供方对媒体接收方发送的媒体画面组成信息中所包含的内容进行是否支持的判断，媒体接收方根据媒体提供方的判断做出相应的操作，分为两种实现方式：Specifically, the media receiver sends the media screen composition information to the media provider. The media screen composition information includes: the combination information of at least two media screens, or the combination information of at least two media screens and the at least two media screen composition information. The layout information enables the media provider to judge whether the content contained in the media screen composition information sent by the media receiver is supported, and the media receiver makes corresponding operations according to the judgment of the media provider. There are two implementation methods:
第一种：若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持至少两路媒体画面的组合这两种场景中的任一场景，则媒体接收方接收媒体提供方发送的组合画面的第一媒体参数（该组合画面是媒体提供方根据媒体画面组成信息所确定的），并向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商；之后，媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定组合画面，并将组合画面发送给媒体接收方予以呈现。Type 1: If the media provider supports the above-mentioned combination of at least two media screens and the layout of at least two media screens, or supports any of the two scenarios of the combination of at least two media screens, the media receiver Receive the first media parameter of the combined screen sent by the media provider (the combined screen is determined by the media provider based on the composition information of the media screen), and send the second media parameter to the media provider, where the first media parameter and the second media parameter The second media parameter is the negotiation between the media provider and the media receiver on parameters such as the codec method and transmission bandwidth of the combined picture; after that, the media provider determines the combined picture according to the composition information of the above media picture, the first media parameter and the second media parameter , and send the combined screen to the media receiver for presentation.
第二种：若媒体提供方不支持上述媒体画面组成信息中的任一场景，则媒体接收方接收媒体提供方发送的否定应答消息，该否定应答消息表示媒体提供方不接受该媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息再次发送给媒体提供方，媒体提供方接收更新后的媒体画面组成信息，依然对其再次进行上述的判断过程。The second type: if the media provider does not support any of the scenarios in the above media screen composition information, the media receiver receives the negative response message sent by the media provider, and the negative response message indicates that the media provider does not accept the media screen composition information , after receiving the negative response message, the media receiver regenerates new media screen composition information, that is, updates the combination information of at least two media screens and the layout information of at least two media screens, and updates the media screen composition information Send it to the media provider again, and the media provider receives the updated media screen composition information, and still performs the above judgment process on it again.
图9为本发明提供的多画面组合的呈现方法实施例九的流程示意图，本发明实施例涉及的方法的执行主体是媒体接收方，该方法包括：Fig. 9 is a schematic flow chart of Embodiment 9 of the presenting method of multi-picture combination provided by the present invention. The execution subject of the method involved in the embodiment of the present invention is the media receiver, and the method includes:
S1001：接收媒体提供方发送的第一CLUE媒体通告消息，该第一CLUE媒体通告消息中携带媒体画面组合能力指示信息。S1001: Receive a first CLUE media notification message sent by a media provider, where the first CLUE media notification message carries media picture combination capability indication information.
具体的，媒体接收方接收媒体发送方发送的第一CLUE媒体通告消息，根据该第一CLUE媒体通告消息中的媒体画面组合能力指示信息获知媒体提供方具有组合画面的能力、媒体提供方提供的多路媒体画面以及多路媒体画面相关的位置信息，媒体提供方在第一CLUE媒体通告消息中的媒体组合画面能力相关描述如下：Specifically, the media receiver receives the first CLUE media announcement message sent by the media sender, and learns that the media provider has the ability to combine images according to the media picture combination capability indication information in the first CLUE media announcement message, and the media provider provides the information provided by the media provider. The multi-channel media screen and the location information related to the multi-channel media screen, and the related description of the media combination screen capability of the media provider in the first CLUE media announcement message are as follows:
Advertisement:Advertisement:
Capture Scene1:Capture Scene1:
Capture Scene Entry1:VC0,VC1,VC2Capture Scene Entry1: VC0, VC1, VC2
Capture Scene Entry2:VC4Capture Scene Entry2: VC4
VC0:encoding group=EG0,content=view:table.left;VC0:encoding group=EG0,content=view:table.left;
VC1:encoding group=EG0,content=view:table.center;VC1:encoding group=EG0,content=view:table.center;
VC2:encoding group=EG0,content=view:table.right;VC2:encoding group=EG0,content=view:table.right;
VC4:encoding group=EG1,content=view:room,composed=true；VC4:encoding group=EG1,content=view:room,composed=true;
上述例子中，对于媒体画面组合能力描述的定义包含在composed中，携带该composed参数，即隐含表征媒体提供方具有媒体画面组合的能力。In the above example, the definition of the media screen composition capability description is included in composed, and the composed parameter is carried, which implicitly indicates that the media provider has the media screen composition capability.
S1002：根据媒体画面组合能力指示信息确定媒体画面组成信息。S1002: Determine media picture composition information according to the media picture combination capability indication information.
S1003：发送第一CLUE媒体配置消息给媒体提供方，该第一CLUE媒体配置消息中携带媒体画面组成信息。S1003: Send a first CLUE media configuration message to the media provider, where the first CLUE media configuration message carries media screen composition information.
具体的，媒体接收方在接收到媒体提供方发送的第一CLUE媒体通告消息之后，结合媒体接收方会议室的终端显示能力以及媒体接收方的本地策略，确定媒体画面组成信息，例如选择由哪几个媒体画面来进行组合、该组合画面如何布局等，并向媒体提供方发送第一CLUE配置消息，该第一CLUE媒体配置消息中携带媒体画面组成信息；此处，同样以一个例子来说明：假设媒体接收方只有一个显示屏，只能接收1路媒体画面，最终媒体接收方选择采用其中2路媒体画面(VC0,VC1)来组合成一路，二路呈现左右平分画面；媒体接收方在第一CLUE配置消息中的媒体组合选择相关描述如下：Specifically, after receiving the first CLUE media announcement message sent by the media provider, the media receiver determines the composition information of the media screen in combination with the terminal display capabilities of the media receiver's meeting room and the local strategy of the media receiver, such as selecting which Combine several media screens, how to layout the combined screen, etc., and send the first CLUE configuration message to the media provider. The first CLUE media configuration message carries media screen composition information; here, an example is also used to illustrate : Assume that the media receiver has only one display screen and can only receive one channel of media images. Finally, the media receiver chooses to use two channels of media images (VC0, VC1) to combine into one channel, and the two channels present a left and right split screen; The relevant description of the media combination selection in the first CLUE configuration message is as follows:
Configuration:Configuration:
Capture Scene1:Capture Scene1:
Capture Scene Entry2:VC4Capture Scene Entry2: VC4
VC4：encoding group=EG1，content=view:room.partial，composed=true，composed layout={VC0（0,0,0）(512,0,0)(0,768,0)(512,768,0),VC1(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}VC4: encoding group=EG1, content=view:room.partial, composed=true, composed layout={VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0),VC1 (512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}
上述例子中，媒体画面组合结果包含在composed layout参数中，表示该组合画面由哪些单独媒体画面组成，而且画面的布局中每个单独画面采用三维立体坐标来表示媒体画面的范围。In the above example, the composition result of the media screen is included in the composed layout parameter, indicating which individual media screens are composed of the combined screen, and each individual screen in the layout of the screen uses three-dimensional coordinates to represent the range of the media screen.
媒体接收方将第一CLUE媒体配置消息发送给媒体提供方之后，媒体提供方根据第一CLUE媒体配置消息中的媒体画面组成信息对媒体接收方进行相应的操作，即媒体提供方根据对上述媒体画面组成信息中的任一场景的支持能力，向媒体接收方执行相应的操作，具体有两种实现方式：After the media receiver sends the first CLUE media configuration message to the media provider, the media provider performs corresponding operations on the media receiver according to the media screen composition information in the first CLUE media configuration message. The support capability of any scene in the screen composition information performs corresponding operations to the media receiver. There are two specific implementation methods:
第一种：若媒体提供方支持上述媒体画面组成信息中的任一场景，则媒体接收方接收媒体提供方发送的组合画面的第一媒体参数，并向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，媒体接收方和媒体提供方根据媒体画面参数的协商结果，建立双向多流媒体传输通道，之后，媒体接收方接收媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定的组合画面，予以呈现。The first type: if the media provider supports any scene in the composition information of the above media screen, the media receiver receives the first media parameter of the combined screen sent by the media provider, and sends the second media parameter to the media provider, where The first media parameter and the second media parameter are the negotiation between the media provider and the media receiver on parameters such as the codec method and transmission bandwidth of the combined picture. The media receiver and the media provider establish a two-way Afterwards, the media receiver receives the combined screen determined by the media provider according to the above media screen composition information, the first media parameter and the second media parameter, and presents it.
第二种：若媒体提供方不支持上述媒体画面组成信息中的任一场景，则媒体接收方接收媒体提供方发送的否定应答消息，获知媒体提供方不接受该媒体画面组成信息，则媒体接收方重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息携带在第一CLUE媒体配置消息中再次发送给媒体提供方，媒体提供方接收更新后的第一CLUE媒体配置消息，依然对其再次进行上述的判断过程。The second type: if the media provider does not support any of the scenarios in the above media screen composition information, the media receiver receives the negative response message sent by the media provider and learns that the media provider does not accept the media screen composition information, then the media receiver The party regenerates new media picture composition information, promptly updates the composition information of at least two paths of media pictures and the layout information of at least two paths of media pictures, carries the updated media picture composition information in the first CLUE media configuration message and sends it to The media provider receives the updated first CLUE media configuration message, and still performs the above judgment process on it again.
本发明实施例提供的方法，通过媒体接收方接收媒体提供方发送的携带媒体画面组合能力指示信息的第一CLUE媒体通告消息，并结合媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，将该媒体画面的组成信息携带在第一CLUE媒体配置消息中，使得媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。According to the method provided by the embodiment of the present invention, the media receiver receives the first CLUE media notification message carrying the indication information of the media picture combination capability sent by the media provider, and combines the terminal display capability of the media receiver and the local policy of the media receiver to determine The composition information of the media screen, the composition information of the media screen is carried in the first CLUE media configuration message, so that the media provider performs corresponding operations according to the media screen composition information determined by the media receiver, that is, the media receiver can customize itself The combined screen to be presented improves the flexibility of presenting the combined media screen and improves the conference experience of the media receiver user.
在上述图9所示实施例的基础上，进一步地，在S1001之前，该方法还包括：On the basis of the above embodiment shown in FIG. 9 , further, before S1001, the method further includes:
S2001：接收媒体提供方发送的第二SDP请求消息，该第二SDP请求消息携带支持CLUE协议指示。S2001: Receive a second SDP request message sent by a media provider, where the second SDP request message carries an indication of supporting the CLUE protocol.
S2002：发送第二SDP响应消息给媒体提供方，该第二SDP响应消息中携带支持CLUE协议指示。S2002: Send a second SDP response message to the media provider, where the second SDP response message carries an indication of supporting the CLUE protocol.
具体的，在图9所示的采用CLUE协议方式实现多画面组合的实施例中，在媒体接收方接收媒体提供方发送的第一CLUE媒体通告消息之前，要进行CLUE通道的建立，即媒体提供方会将支持CLUE协议指示的信息携带在第二SDP请求消息中发送给媒体接收方，相应的，媒体接收方在获知媒体提供方支持CLUE协议后，会向媒体提供方发送第二SDP响应消息，该第二SDP响应消息中携带支持CLUE协议指示，CLUE通道建立成功。Specifically, in the embodiment shown in FIG. 9 that uses the CLUE protocol to realize multi-picture combination, before the media receiver receives the first CLUE media notification message sent by the media provider, the CLUE channel needs to be established, that is, the media provider The party will carry the information indicating that the CLUE protocol is supported in the second SDP request message and send it to the media receiver. Correspondingly, after the media receiver knows that the media provider supports the CLUE protocol, it will send the second SDP response message to the media provider. , the second SDP response message carries an indication of supporting the CLUE protocol, and the CLUE channel is established successfully.
图10为本发明提供的多画面组合的呈现方法实施例十的流程示意图，在上述实施例的基础上，如图10所示，该方法的执行主体为媒体接收方，该方法包括如下步骤：Fig. 10 is a schematic flow chart of Embodiment 10 of the multi-picture combination presentation method provided by the present invention. On the basis of the above embodiments, as shown in Fig. 10, the execution subject of the method is the media receiver, and the method includes the following steps:
S3001：接收媒体提供方发送的第一SDP请求消息，该第一SDP请求消息中携带所述媒体画面组合能力指示信息。S3001: Receive a first SDP request message sent by a media provider, where the first SDP request message carries the media picture combination capability indication information.
具体的，媒体接收方接收媒体提供方发送的第一SDP请求消息，该第一SDP请求消息携带媒体画面组合能力指示信息，并且还携带了多路媒体画面信息，并且该第一SDP请求消息可以携带在SIP消息中传输，此处以一个例子来说明：媒体提供方可以提供3路单独媒体视频画面(VC0,VC1,VC2)，媒体提供方在第一SDP请求消息中的媒体画面信息相关描述如下：Specifically, the media receiver receives the first SDP request message sent by the media provider, the first SDP request message carries media picture combination capability indication information, and also carries multi-channel media picture information, and the first SDP request message can Carried in SIP messages for transmission, here is an example to illustrate: the media provider can provide 3 separate media video images (VC0, VC1, VC2), and the media provider’s media image information in the first SDP request message is described as follows :
a=video-compose-capa，指示具有画面组合能力；a=video-compose-capa, indicating that it has the ability to compose images;
m=video 10000 RTP/AVP 98，指示第一路画面信息VC0；m=video 10000 RTP/AVP 98, indicating the first video information VC0;
a=label:VC0a=label:VC0
a=rtpmap:98 H264/90000a=rtpmap:98 H264/90000
m=video 10002 RTP/AVP 98，指示第二路画面信息VC1；m=video 10002 RTP/AVP 98, indicating the second video information VC1;
a=label:VC1a=label:VC1
a=rtpmap:98 H264/9000a=rtpmap:98 H264/9000
m=video 10004 RTP/AVP 31，指示第三路画面信息VC2；m=video 10004 RTP/AVP 31, indicating the third video information VC2;
a=label:VC2a=label:VC2
a=rtpmap:31 H261/9000。a=rtpmap:31 H261/9000.
S3002：根据媒体画面组合能力指示信息确定媒体画面组成信息。S3002: Determine media picture composition information according to the media picture combination capability indication information.
S3003：发送第一SDP响应消息给媒体提供方，该第一SDP响应消息中携带所述媒体画面组成信息。S3003: Send a first SDP response message to the media provider, where the first SDP response message carries the media picture composition information.
具体的，媒体接收方接收媒体提供方发送的第一SDP请求消息，结合媒体接收方会议室的终端显示能力以及媒体接收方的本地策略，确定媒体画面组成信息，例如选择由哪几个媒体画面来进行组合、该组合画面如何布局等，并向媒体提供方发送第一SDP响应消息，该第一SDP响应消息中携带媒体画面组成信息；此处，同样以一个例子来说明：假设媒体接收方只有一个显示屏，只能接收1路视频画面，最终媒体接收方选择采用其中2路视频画面(VC0,VC1)来组合成一路，二路呈现左右平分画面；媒体接收方在第一SDP响应消息中的媒体画面组合的相关描述如下：Specifically, the media receiver receives the first SDP request message sent by the media provider, and combines the terminal display capability of the media receiver's conference room and the local strategy of the media receiver to determine the composition information of the media screen, such as which media screens to select to perform combination, how to layout the combined screen, etc., and send the first SDP response message to the media provider, the first SDP response message carries the composition information of the media screen; here, an example is also used to illustrate: suppose the media receiver There is only one display screen and can only receive 1 channel of video images. In the end, the media receiver chooses to use 2 channels of video images (VC0, VC1) to combine into one channel, and the second channel presents a left and right split image; the media receiver responds to the first SDP message The relevant description of the combination of media screens in is as follows:
a=composed layout：VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0)，VC1(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)，表示媒体接收方希望将VC0,VC1组合，按照左右布局，各占50%；a=composed layout: VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0), VC1(512,0,0)(1024,0,0)(512,768,0) )(1024,760,0), indicating that the media receiver wants to combine VC0 and VC1, according to the left and right layout, each accounting for 50%;
m=video 20000 RTP/AVP 98，指示第一路画面信息VC0；m=video 20000 RTP/AVP 98, indicating the first video information VC0;
a=label：VC0a=label:VC0
a=rtpmap：98 H264/90000a=rtpmap:98 H264/90000
m=video 0 RTP/AVP 31，指示第二路画面信息VC1；m=video 0 RTP/AVP 31, indicating the second video information VC1;
a=label：VC1a=label:VC1
a=rtpmap：31 H261/9000a=rtpmap:31 H261/9000
m=video 0 RTP/AVP 31，指示第三路画面信息VC2；m=video 0 RTP/AVP 31, indicating the third video information VC2;
a=label：VC2a=label:VC2
a=rtpmap：31 H261/9000。a=rtpmap:31 H261/9000.
媒体提供方接收媒体接收方发送的第一SDP响应消息后，根据对上述媒体画面组成信息中的内容的支持能力，向媒体接收方执行相应的操作，实际上也就是媒体接收方根据媒体提供方对媒体画面组成信息进行是否支持的判断结果执行相应的操作，具体可以分为两种实现方式：After the media provider receives the first SDP response message sent by the media receiver, it performs corresponding operations to the media receiver according to its ability to support the content in the composition information of the media screen above. Perform corresponding operations on the result of judging whether the composition information of the media screen is supported, which can be divided into two specific implementation methods:
第一种：若媒体提供方支持至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持至少两路媒体画面的组合这两种场景中的任一场景，媒体接收方接收媒体提供方发送的组合画面的第一媒体参数，之后向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，媒体接收方和媒体提供方根据媒体画面参数的协商结果，建立双向多流媒体传输通道，媒体接收方接收媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定的组合画面，予以呈现。Type 1: If the media provider supports the combination of at least two media screens and the layout of at least two media screens, or supports any of the two scenarios of the combination of at least two media screens, the media receiver receives the media The provider sends the first media parameter of the combined picture, and then sends the second media parameter to the media provider, where the first media parameter and the second media parameter are the encoding and decoding methods of the combined picture by the media provider and the media receiver. Negotiation of parameters such as transmission bandwidth, the media receiver and the media provider establish a two-way multi-stream media transmission channel according to the negotiation results of the media screen parameters, and the media receiver receives the media provider’s information based on the above media screen composition information, the first media parameter and the second The combined picture determined by the two media parameters is presented.
第二种：若媒体提供方不支持上述媒体画面组成信息中的任一场景，则媒体提供方发送否定应答消息给媒体接收方，向媒体接收方表示不接受该媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息携带在第一SDP响应消息中再次发送给媒体提供方，媒体提供方接收更新后的第一SDP响应消息，依然对其再次进行上述的判断过程。The second type: if the media provider does not support any of the scenarios in the above media screen composition information, the media provider sends a negative response message to the media receiver, expressing to the media receiver that it does not accept the media screen composition information, and the media receiver After receiving the negative acknowledgment message, regenerate new media picture composition information, that is, update the combination information of at least two channels of media pictures and the layout information of at least two channels of media pictures, and carry the updated media picture composition information in the first The SDP response message is sent to the media provider again, and the media provider receives the updated first SDP response message, and still performs the above judgment process on it again.
本发明实施例提供的方法，通过媒体接收方接收媒体提供方发送的携带媒体画面组合能力指示信息的第一SDP请求消息，并结合媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，将该媒体画面的组成信息携带在第一SDP响应消息中，媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media receiver receives the first SDP request message carrying the indication information of the media screen combination capability sent by the media provider, and determines the media in combination with the terminal display capability of the media receiver and the local policy of the media receiver. The composition information of the picture, the composition information of the media picture is carried in the first SDP response message, the media provider performs corresponding operations according to the media picture composition information determined by the media receiver, so that the media combination picture received by the media receiver It is a media combination screen determined by the media provider according to the media screen composition information sent by the media receiver, that is, the media receiver can customize the combination screen it wants to present, thereby improving the flexibility of the presentation of the combined media screen and improving the user experience of the media receiver. meeting experience.
图11为本发明提供的多画面组合的呈现方法实施例十一的流程示意图，本发明实施例的执行主体为媒体接收方，如图11所示，该方法包括：Fig. 11 is a schematic flow chart of Embodiment 11 of the method for presenting a multi-picture combination provided by the present invention. The execution subject of the embodiment of the present invention is the media receiver. As shown in Fig. 11 , the method includes:
S4001：从门户网站获取媒体提供方发送的所述媒体画面组合能力指示信息以及媒体画面的缩略图。S4001: Obtain the indication information of the media picture combination capability and the thumbnail of the media picture sent by the media provider from the portal website.
S4002：根据媒体画面组合能力指示信息和媒体画面的缩略图确定媒体画面组成信息，并将该媒体画面组成信息发送给门户网站，以使媒体提供方从门户网站获取媒体画面组成信息。S4002: Determine the media screen composition information according to the media screen combination capability indication information and the thumbnail of the media screen, and send the media screen composition information to the portal website, so that the media provider can obtain the media screen composition information from the portal website.
具体的，本发明实施例中媒体提供方和媒体接收放都已经成功登陆到门户网站上；媒体提供方通过门户网站的接口将媒体画面组合能力指示信息和媒体画面的缩略图发送给门户网站，并且呈现在门户网站的界面上；媒体接收方从门户网站获取媒体画面组合能力指示信息和媒体画面的缩略图，并结合媒体接收方的会议室的终端显示能力以及媒体接收方本地策略，确定媒体画面组成信息，并将该媒体画面组成信息中包含至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，或，至少两路媒体画面的组合信息，通过门户网站接口呈现在门户网站界面上；媒体提供方从门户网站上获取该媒体画面组成信息。Concretely, in the embodiment of the present invention, both the media provider and the media receiver have successfully logged on to the portal website; the media provider sends the media screen combination capability indication information and the thumbnail of the media screen to the portal website through the interface of the portal website, And it is presented on the interface of the portal website; the media receiver obtains the indication information of the media picture combination capability and the thumbnail of the media picture from the portal website, and combines the terminal display capability of the media receiver’s conference room and the local strategy of the media receiver to determine the media Screen composition information, and the media screen composition information includes the combination information of at least two media screens and the layout information of at least two media screens, or, the combination information of at least two media screens is presented on the portal website through the portal website interface On the interface; the media provider obtains the composition information of the media screen from the portal website.
媒体提供方从门户网站上获取该媒体画面组成信息之后，判断是否支持该媒体画面组成信息中所包含的内容，媒体接收方根据媒体提供方的判断结果进行相应的操作，具体分为两种实现方式：After the media provider obtains the composition information of the media screen from the portal website, it judges whether it supports the content contained in the composition information of the media screen, and the media receiver performs corresponding operations according to the judgment result of the media provider, which is specifically divided into two implementations Way:
第一种：若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持上述至少两路媒体画面的组合这两种场景中的任一场景，则媒体提供方发送组合画面的第一媒体参数给门户网站，媒体接收方从门户网站获取第一媒体参数，并向门户网站发送第二媒体参数，以使媒体提供方可以从门户网站获取第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商；之后，媒体接收方接收媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定的组合画面，予以呈现。Type 1: If the media provider supports the above-mentioned combination of at least two media screens and the layout of at least two media screens, or supports any of the above two scenarios of the above-mentioned combination of at least two media screens, the media provider The party sends the first media parameter of the combined screen to the portal website, the media receiver obtains the first media parameter from the portal website, and sends the second media parameter to the portal website, so that the media provider can obtain the second media parameter from the portal website, Here, the first media parameter and the second media parameter are the negotiation between the media provider and the media receiver on parameters such as the codec method and the transmission bandwidth of the combined picture; after that, the media receiver receives the media provider's composition information according to the above media picture, The combined screen determined by the first media parameter and the second media parameter is presented.
第二种：若媒体提供方不支持上述媒体画面组成信息中的任一场景，，则发送否定应答消息给门户网站，媒体接收方从门户网站获知媒体提供方不接受该媒体画面组成信息，则媒体接收方会重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息再次发送给门户网站，媒体提供方从门户网站获取更新后的媒体画面组成信息，依然对其再次进行上述的判断过程。The second method: if the media provider does not support any of the above media screen composition information, then send a negative response message to the portal, and the media receiver learns from the portal that the media provider does not accept the media screen composition information, then The media receiver will regenerate new media screen composition information, that is, update the combination information of at least two media screens and the layout information of at least two media screens, and send the updated media screen composition information to the portal website again, and the media provider The updated media screen composition information is obtained from the portal website, and the above-mentioned judgment process is still performed on it again.
本发明实施例提供的方法，通过媒体提供方向门户网站发送携带媒体画面组合能力指示信息和媒体画面缩略图，媒体接收方从门户网站获取该媒体画面组合能力指示信息和媒体画面缩略图后，结合媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，并将该媒体画面的组成信息发送给门户网站；媒体提供方根据从门户网站获取的媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the media screen combination capability indication information and the media screen thumbnail to the portal website, and the media receiver acquires the media screen combination capability indication information and the media screen thumbnail from the portal website, and combines the The terminal display capability of the media receiver and the local strategy of the media receiver determine the composition information of the media screen, and send the composition information of the media screen to the portal website; The screen composition information performs corresponding operations, so that the media combination screen received by the media receiver is the media combination screen determined by the media provider according to the media screen composition information sent by the media receiver, that is, the media receiver can customize the combination screen it wants to present , so as to improve the flexibility of presenting combined media screens and improve the conference experience of the media receiver user.
在图9和图10所示实施例的基础上，进一步地，接收媒体提供方发送的组合画面的第一媒体参数，具体可以为：接收媒体提供方发送的第三SDP请求消息，该第三SDP请求消息中携带组合画面的第一媒体参数；则发送组合画面的第二媒体参数给所述媒体提供方，具体可以为发送第三SDP响应消息给所述媒体提供方，该第三SDP响应消息中携带所述组合画面的第二媒体参数。On the basis of the embodiments shown in Fig. 9 and Fig. 10, further, receiving the first media parameter of the combined screen sent by the media provider may specifically be: receiving the third SDP request message sent by the media provider, the third Carry the first media parameter of combined picture in the SDP request message; Then send the second media parameter of combined picture to described media provider, specifically can be to send the 3rd SDP response message to described media provider, and this 3rd SDP responds The message carries the second media parameter of the combined picture.
图12为本发明提供的多画面组合的呈现方法实施例十二的流程示意图，本实施例主要涉及的是在视频会议召开之后，由于某种需要，可以是网络情况变化（比如拥塞，或者组合画面中某一源会场离会，或者新的会场入会等），组合画面需要做出调整，即媒体组成信息需要进行变更的处理方法；本发明实施例的执行主体是媒体接收方，在图8所示实施例的基础上，在S903之后，该方法还包括：Fig. 12 is a schematic flowchart of Embodiment 12 of the presentation method of multi-screen combination provided by the present invention. A certain source site in the screen leaves the conference, or a new conference site joins the conference, etc.), the combined screen needs to be adjusted, that is, the processing method that the media composition information needs to be changed; the execution subject of the embodiment of the present invention is the media receiver, as shown in Figure 8 On the basis of the illustrated embodiment, after S903, the method further includes:
S5001：接收媒体提供方发送的媒体画面变化信息。S5001: Receive media screen change information sent by a media provider.
具体的，在视频会议召开之后，即在媒体提供方发送组合画面给媒体接收方之后，媒体提供方检测到网络情况发生变化，比如拥塞或者组合画面中某一源会场离会或者新的会场入会等，则媒体提供方会向媒体接收方发送媒体画面变化信息，其中，该媒体画面变化信息可以携带在CLUE媒体通告消息中，也可以携带在SDP请求消息中，媒体接收方接收媒体提供方发送的媒体画面变化信息，此处以该媒体画面变化信息携带在CLUE媒体通告消息中为例来说明，假设媒体提供方原来提供3路单独视频画面(VC0,VC1,VC2)，媒体接收方选择VC0，VC1进行组合，这时VC0画面消失，媒体提供方在CLUE媒体通告消息中通知组合画面变更，具体描述如下：Specifically, after the video conference is held, that is, after the media provider sends the combined picture to the media receiver, the media provider detects that the network situation changes, such as congestion or a source site in the combined picture leaves the conference or a new site joins the conference etc., then the media provider will send the media screen change information to the media receiver, where the media screen change information can be carried in the CLUE media notification message or in the SDP request message, and the media receiver receives the media provider’s sent The change information of the media screen, here is an example of the media screen change information carried in the CLUE media announcement message, assuming that the media provider originally provided 3 separate video screens (VC0, VC1, VC2), and the media receiver selects VC0, When VC1 is combined, the screen of VC0 disappears, and the media provider notifies the change of the combined screen in the CLUE media notification message, the specific description is as follows:
Advertisement：Advertisements:
Capture Scene1：Capture Scene1:
Capture Scene Entry1：VC1,VC2Capture Scene Entry1: VC1, VC2
Capture Scene Entry2：VC4Capture Scene Entry2: VC4
VC1：encoding group=EG0，content=view:table.center；VC1: encoding group=EG0, content=view:table.center;
VC2：encoding group=EG0，content=view:table.right；VC2: encoding group=EG0, content=view:table.right;
VC4：encoding group=EG1，content=view:room,composed=true,composedupdate={VC0：off，VC1}；VC4: encoding group=EG1, content=view:room, composed=true, composedupdate={VC0:off, VC1};
上述例子中，媒体画面组合结果包含在composed update中，表示组合画面变更，VC0关闭。In the above example, the composition result of the media screen is included in the composed update, indicating that the composite screen has changed, and VC0 is closed.
S5002：根据媒体画面变化信息生成更新后的第一媒体画面组成信息，该第一媒体画面组成信息中包括：至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，或，至少两路媒体画面的第一更新组合信息。S5002: Generate updated first media screen composition information according to media screen change information, the first media screen composition information includes: first updated combination information of at least two media screens and first updated layout of at least two media screens information, or the first update combination information of at least two channels of media images.
S5003：将第一媒体画面组成信息发送给媒体提供方，以使媒体提供方根据对上述第一媒体画面组成信息的支持能力，向媒体接收方执行相应的操作。S5003: Send the first media screen composition information to the media provider, so that the media provider performs corresponding operations to the media receiver according to the support capability of the first media screen composition information.
具体的，媒体接收方在接收到媒体提供方发送的媒体画面变化信息之后，获知该视频会议的网络情况发生变化，则媒体接收方根据该媒体画面变化信息生成更新后的第一媒体画面组成信息，该第一媒体画面组成信息中包括：至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，或，至少两路媒体画面的第一更新组合信息，并将该第一媒体画面组成信息发送给媒体提供方，其中，该第一媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP请求响应消息中，与S4001中所举例子对应，这里同样以第一媒体画面组成信息携带在CLUE媒体配置消息中为例来说明：媒体接收方选择采用2路视频画面(VC1,VC2)来组合成一路，二路呈现左右平分画面，媒体接收方在CLUE媒体配置消息中的新的媒体画面组合选择相关描述如下：Specifically, after receiving the media screen change information sent by the media provider, the media receiver learns that the network conditions of the video conference have changed, and the media receiver generates updated first media screen composition information according to the media screen change information , the first media picture composition information includes: the first updated combination information of at least two media pictures and the first updated layout information of at least two media pictures, or the first updated combination information of at least two media pictures, and Send the first media screen composition information to the media provider, where the first media screen composition information can be carried in the CLUE media configuration message, or can be carried in the SDP request response message, corresponding to the example in S4001, Here also take the first media picture composition information carried in the CLUE media configuration message as an example to illustrate: the media receiver chooses to use 2 channels of video pictures (VC1, VC2) to combine into one channel, and the second channel presents a left and right split screen. The relevant description of the new media picture combination selection in the CLUE media configuration message is as follows:
Configuration：Configuration:
Capture Scene1：Capture Scene1:
Capture Scene Entry2：VC4Capture Scene Entry2: VC4
VC4:encoding group=EG1content=view:room.partial,composed=true,VC4:encoding group=EG1content=view:room.partial,composed=true,
composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
媒体提供方接收媒体接收方发送的第一媒体画面组成信息，对第一媒体组成信息中的内容执行判断操作，即判断是否支持至少两路媒体画面的第一更新组合和至少两路媒体画面的第一更新布局，或，是否支持至少两路媒体画面的第一更新组合这两种场景中的任一场景，媒体接收方根据媒体提供方的判断结果作出相应的操作，具体可以分为两种实现方式：The media provider receives the first media screen composition information sent by the media receiver, and performs a judgment operation on the content in the first media composition information, that is, judges whether to support the first update combination of at least two media screens and at least two media screens. The first update layout, or whether to support the first update combination of at least two media screens, any of these two scenarios, the media receiver makes corresponding operations according to the judgment result of the media provider, which can be divided into two types Method to realize:
第一种：若媒体提供方支持上述第一媒体画面组成信息中的任一场景，可选的，媒体提供方可以发送组合画面的第一媒体参数给媒体接收方；媒体接收方接收到媒体提供方发送的第一媒体参数之后，会向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，需要注意的，本实施例中也可以不进行组合画面的第一媒体参数和组合画面的第二媒体参数的交互，因为在视频会议召开之前媒体发送方和媒体接收方针对组合画面的第一媒体参数和第二媒体参数已经协商过了；之后，媒体接收方接收媒体提供方根据第一媒体画面组成信息、第一媒体参数和第二媒体参数确定的更新后的组合画面，并将更新后组合画面予以呈现。Type 1: If the media provider supports any scene in the composition information of the first media picture above, optionally, the media provider can send the first media parameters of the combined picture to the media receiver; the media receiver receives the media provided After the first media parameter sent by the media provider, the second media parameter will be sent to the media provider. Here, the first media parameter and the second media parameter are the codec method and transmission bandwidth of the combined screen by the media provider and the media receiver. For the negotiation of parameters, it should be noted that in this embodiment, the interaction between the first media parameter of the combined picture and the second media parameter of the combined picture may not be performed, because before the video conference is held, the media sender and media receiver will The first media parameter and the second media parameter have been negotiated; afterward, the media receiver receives the updated combined picture determined by the media provider according to the composition information of the first media picture, the first media parameter and the second media parameter, and The updated combined screen will be presented.
第二种：若媒体提供方不支持上述第一媒体画面组成信息中的任一场景，则发送否定应答消息给媒体接收方，向媒体接收方表示不接受该第一媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，将更新后的媒体画面组成信息再次发送给媒体提供方，媒体提供方接收更新后的媒体画面组成信息，依然对其再次进行上述的判断过程。The second type: if the media provider does not support any scene in the above-mentioned first media picture composition information, then send a negative response message to the media receiver, expressing to the media receiver that it does not accept the first media picture composition information, and the media receiver After receiving the negative response message, the party regenerates new media picture composition information, that is, updates the first updated combination information of at least two channels of media pictures and the first updated layout information of at least two channels of media pictures, and updates the updated media picture The screen composition information is sent to the media provider again, and the media provider receives the updated media screen composition information, and still performs the above judgment process on it again.
在图12所示实施例的基础上，进一步地，上述的媒体画面变化信息可以携带在CLUE媒体通告消息中，第一媒体画面组成信息可以携带在CLUE媒体配置消息中，即图12所示的实施例可以与图9所示的实施例进行结合（在S1003之后执行S5001、S5002、S5003）；或者，媒体画面变化信息可以携带在SDP请求消息中，第一媒体画面组成信息可以携带在SDP响应消息中，即图12所示的实施例可以与图10所示的实施例进行结合（在S3003之后，执行S5001、S5002、S5003）；或者，媒体提供方可以将媒体画面变化信息发送给门户网站，由媒体接收方从门户网站获取该媒体画面变化信息，从而发送第一媒体组成信息给门户网站，以使媒体提供方从门户网站获取第一媒体组成信息，即图12所示的实施例可以与图11所示的实施例进行结合（在S4002之后，执行S5001、S5002、S5003）。On the basis of the embodiment shown in FIG. 12 , further, the above-mentioned media screen change information may be carried in the CLUE media notification message, and the first media screen composition information may be carried in the CLUE media configuration message, that is, as shown in FIG. 12 The embodiment can be combined with the embodiment shown in FIG. 9 (S5001, S5002, and S5003 are executed after S1003); or, the media screen change information can be carried in the SDP request message, and the first media screen composition information can be carried in the SDP response In the message, that is, the embodiment shown in Figure 12 can be combined with the embodiment shown in Figure 10 (after S3003, execute S5001, S5002, S5003); or, the media provider can send the media screen change information to the portal website , the media receiver obtains the media screen change information from the portal website, thereby sending the first media composition information to the portal website, so that the media provider obtains the first media composition information from the portal website, that is, the embodiment shown in FIG. 12 can Combining with the embodiment shown in FIG. 11 (after S4002, execute S5001, S5002, and S5003).
本发明实施例提供的方法，在视频会议召开之后，网络发生变化的情况下，通过媒体提供方向媒体接收方发送媒体画面变化信息，媒体接收方根据媒体画面变化信息生成更新后的第一媒体画面组成信息，即生成更新的媒体画面组合和布局，媒体提供方根据第一媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的第一媒体画面组成信息确定的媒体组合画面，同时还使得媒体提供方和媒体接收方可以实时根据网络变化情况对所所组合的媒体画面做出调整，提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度和适用性。In the method provided by the embodiment of the present invention, when the network changes after the video conference is held, the media provider sends the media screen change information to the media receiver, and the media receiver generates an updated first media screen according to the media screen change information Composition information, that is, to generate an updated media picture combination and layout, and the media provider performs corresponding operations according to the composition information of the first media picture, so that the media combination picture received by the media receiver is the media provider according to the first media picture sent by the media receiver. The media combination picture determined by the composition information of the media picture also enables the media provider and the media receiver to make real-time adjustments to the combined media picture according to network changes, which improves the flexibility of the combination media picture presentation and improves the media quality. The meeting experience and applicability of the recipient user.
图13为本发明提供的多画面组合的呈现方法实施例十三的流程示意图，本发明实施例涉及的方法主要是媒体接收方自行发起组合画面调整的指示，该方法的执行主体是媒体接收方，在上述图8所示实施例的基础上，在S903之后，该方法还包括：Fig. 13 is a schematic flowchart of Embodiment 13 of the presentation method of multi-picture combination provided by the present invention. The method involved in the embodiment of the present invention is mainly that the media receiver initiates an instruction to adjust the combined screen by itself, and the execution subject of the method is the media receiver. , on the basis of the above embodiment shown in FIG. 8, after S903, the method further includes:
S6001：发送更新后的第二媒体画面组成信息给媒体提供方，该第二媒体画面组成信息中包括：至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，或，至少两路媒体画面的第二更新组合信息，以使媒体提供方根据对第二媒体画面组成信息的支持能力，向媒体接收方执行相应的操作。S6001: Send the updated second media screen composition information to the media provider, the second media screen composition information includes: the second updated combination information of at least two media screens and the second updated layout information of at least two media screens , or, the second update combination information of at least two channels of media pictures, so that the media provider performs corresponding operations to the media receiver according to the support capability of the second media picture composition information.
具体的，媒体接收方自行想更新所呈现的组合画面，则媒体接收方根据自身的能力生成更新后的第二媒体画面组成信息，并将该第二媒体画面组成信息发送给媒体提供方，该第二媒体画面组成信息中包括：至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，或，至少两路媒体画面的第二更新组合信息，其中，该第二媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP响应消息中，这里第二媒体画面组成信息携带在CLUE媒体配置消息中为例：媒体接收方选择采用2路视频画面(VC1,VC2)来组合成一路，二路呈现左右平分画面，媒体接收方在CLUE媒体配置消息中的新媒体画面组合选择相关描述如下：Specifically, if the media receiver wants to update the presented combined screen, the media receiver generates updated second media screen composition information according to its own capabilities, and sends the second media screen composition information to the media provider. The second media screen composition information includes: the second updated combination information of at least two media screens and the second updated layout information of at least two media screens, or the second updated combination information of at least two media screens, wherein the The composition information of the second media screen can be carried in the CLUE media configuration message, and can also be carried in the SDP response message. Here, the composition information of the second media screen is carried in the CLUE media configuration message. For example: the media receiver chooses to use 2 video screens (VC1, VC2) to form one channel, and the second channel presents a left and right split screen. The media receiver’s new media screen combination selection in the CLUE media configuration message is described as follows:
Configuration:Configuration:
Capture Scene1：Capture Scene1:
Capture Scene Entry 2:VC4Capture Scene Entry 2: VC4
VC4：encoding group=EG1,content=view:room.partial,composed=true,VC4: encoding group=EG1, content=view:room.partial, composed=true,
composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
媒体接收方向媒体提供方发送第二媒体画面组成信息，媒体提供方根据对第二媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作，即判断是否支持至少两路媒体画面的第二更新组合和至少两路媒体画面的第二更新布局，或，是否支持至少两路媒体画面的第二更新组合这两种场景中的任一场景，媒体接收方根据媒体提供方的判断结果进行相应的操作，具体可以分为两种实现方式：The media receiving party sends the second media screen composition information to the media provider, and the media provider performs corresponding operations to the media receiver according to the support capability of the second media screen composition information, that is, judges whether to support at least two media screens The second update combination and the second update layout of at least two media screens, or, whether to support the second update combination of at least two media screens in any of these two scenarios, the media receiver can use the judgment result of the media provider The corresponding operation can be divided into two implementation methods:
第一种：若媒体提供方支持上述第二媒体画面组成信息中的任一场景，可选的，媒体提供方可以发送组合画面的第一媒体参数给媒体接收方；媒体接收方接收到媒体提供方发送的第一媒体参数之后，向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，需要注意的，本实施例中也可以不进行组合画面的第一媒体参数和组合画面的第二媒体参数的交互，因为在视频会议召开之前媒体发送方和媒体接收方针对组合画面的第一媒体参数和第二媒体参数已经协商过了；之后，媒体接收方接收媒体提供方根据第二媒体画面组成信息、第一媒体参数和第二媒体参数确定的更新后的组合画面，并将更新后组合画面发送予以呈现。The first one: if the media provider supports any scene in the composition information of the second media picture above, optionally, the media provider can send the first media parameters of the combined picture to the media receiver; the media receiver receives the media provided After the first media parameter sent by the party, the second media parameter is sent to the media provider, where the first media parameter and the second media parameter are parameters such as the codec method and the transmission bandwidth of the combined screen by the media provider and the media receiver It should be noted that in this embodiment, the interaction between the first media parameter of the combined picture and the second media parameter of the combined picture may not be performed, because before the video conference is held, the media sender and the media receiver will The first media parameter and the second media parameter have been negotiated; afterward, the media receiver receives the updated combined picture determined by the media provider according to the composition information of the second media picture, the first media parameter and the second media parameter, and After the update the composition screen is sent to render.
第二种：若媒体提供方不支持上述至少两路媒体画面的第二更新组合和至少两路媒体画面的第二更新布局，则发送否定应答消息给媒体接收方，向媒体接收方表示不接受该第二媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，将更新后的媒体画面组成信息再次发送给媒体提供方，媒体提供方接收更新后的媒体画面组成信息，依然对其再次进行上述的判断过程。The second type: if the media provider does not support the above-mentioned second update combination of at least two channels of media images and the second update layout of at least two channels of media images, then send a negative response message to the media receiver, indicating that it does not accept the media receiver For the second media picture composition information, after receiving the negative response message, the media receiver regenerates new media picture composition information, that is, updates the second update combination information of at least two media pictures and the first update combination information of at least two media pictures. 2. Update the layout information, send the updated media screen composition information to the media provider again, and the media provider receives the updated media screen composition information, and still perform the above judgment process on it again.
在图13所示实施例的基础上，进一步地，上述第二媒体画面组成信息可以携带在CLUE媒体配置消息中，即图13所示的实施例可以与图9所示的实施例进行结合（在S1003之后执行S6001）；或者，第二媒体画面组成信息可以携带在SDP响应消息中，即图13所示的实施例可以与图10所示的实施例进行结合（在S3003之后，执行S6001）；或者，媒体接收方发送第二媒体组成信息给门户网站，以使媒体提供方从门户网站获取第二媒体组成信息，即图5所示的实施例可以与图11所示的实施例进行结合（在S4002之后，执行S6001）。On the basis of the embodiment shown in FIG. 13 , further, the above-mentioned second media screen composition information may be carried in the CLUE media configuration message, that is, the embodiment shown in FIG. 13 may be combined with the embodiment shown in FIG. 9 ( Execute S6001 after S1003); or, the second media screen composition information may be carried in the SDP response message, that is, the embodiment shown in FIG. 13 may be combined with the embodiment shown in FIG. 10 (after S3003, execute S6001) Or, the media receiver sends the second media composition information to the portal website, so that the media provider obtains the second media composition information from the portal website, that is, the embodiment shown in Figure 5 can be combined with the embodiment shown in Figure 11 (After S4002, execute S6001).
本发明实施例提供的方法，在视频会议召开之后，媒体提供方自行更新媒体画面，即媒体接收方自行生成更新后的第二媒体画面组成信息，媒体提供方根据第二媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的第二媒体画面组成信息确定的媒体组合画面，提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度和适用性。In the method provided by the embodiment of the present invention, after the video conference is held, the media provider updates the media screen by itself, that is, the media receiver generates the updated second media screen composition information by itself, and the media provider executes the corresponding operation according to the second media screen composition information. The operation makes the media combination picture received by the media receiver be the media combination picture determined by the media provider according to the composition information of the second media picture sent by the media receiver, which improves the flexibility of combining media picture presentation and improves the media receiver’s User's meeting experience and applicability.
图14为本发明提供的多画面组合的呈现方法实施例十四的流程示意图，本发明实施例涉及的方法是媒体提供方提供多种默认的组合画面给媒体接收方，媒体接收方根据自身的终端显示能力和本地策略进行选择，本发明实施例的执行主体是媒体接收方，该方法包括如下步骤：Fig. 14 is a schematic flow chart of Embodiment 14 of the multi-picture combination presentation method provided by the present invention. The method involved in the embodiment of the present invention is that the media provider provides multiple default combined screens to the media receiver, and the media receiver according to its own Terminal display capability and local policy are selected, and the executive subject of the embodiment of the present invention is the media receiver, and the method includes the following steps:
S7001：接收媒体提供方发送的至少一种媒体画面组合选择信息。S7001: Receive at least one media screen combination selection information sent by a media provider.
S7002：根据上述至少一种媒体画面组合选择信息确定媒体画面组成信息。S7002: Determine media picture composition information according to the at least one kind of media picture combination selection information.
具体的，媒体接收方接收媒体提供方发送的至少一种媒体画面组合选择信息，该媒体画面组合选择信息为媒体提供方自行组合的媒体组合画面，其中，该媒体画面组合选择信息可以携带在CLUE媒体通告消息中，也可以携带在SDP请求消息中，还可以是媒体提供方将至少一种的媒体画面组合选择信息发送给门户网站，由媒体接收方从门户网站获取该至少一种媒体画面组合选择信息，此处以上述媒体画面组合选择信息携带在CLUE媒体配置消息中为例：媒体提供方原来提供3路单独视频画面(VC0,VC1,VC2)，默认的组合方式为：三个单独画面的组合{VC0,VC1,VC2}，两个单独画面的组合{VC0，VC1}，媒体提供方在CLUE媒体通告消息中通知默认的媒体组合画面，具体描述如下：Specifically, the media receiver receives at least one media screen combination selection information sent by the media provider, and the media screen combination selection information is a media combination screen assembled by the media provider itself, wherein the media screen combination selection information can be carried in the CLUE In the media announcement message, it can also be carried in the SDP request message, and the media provider can also send at least one media picture combination selection information to the portal website, and the media receiver can obtain the at least one media picture combination from the portal website Selection information. Here, the above-mentioned media screen combination selection information is carried in the CLUE media configuration message as an example: the media provider originally provided 3 separate video screens (VC0, VC1, VC2), and the default combination method is: three separate screens Combination {VC0, VC1, VC2}, the combination of two separate screens {VC0, VC1}, the media provider notifies the default media combination screen in the CLUE media announcement message, the specific description is as follows:
Advertisement：Advertisements:
Capture Scene1：Capture Scene1:
Capture Scene Entry1：VC0，VC1，VC2Capture Scene Entry1: VC0, VC1, VC2
Capture Scene Entry2：VC4Capture Scene Entry2: VC4
VC0：encoding group=EG0，content=view:table.left；VC0: encoding group=EG0, content=view:table.left;
VC1：encoding group=EG0，content=view:table.center；VC1: encoding group=EG0, content=view:table.center;
VC2：encoding group=EG0，content=view:table.right；VC2: encoding group=EG0, content=view:table.right;
VC4：encoding group=EG1，content=view:room，composed=true，composedoption={composed layout1，composed layout2}，VC4: encoding group=EG1, content=view:room, composed=true, composedoption={composed layout1, composed layout2},
composed layout1={VC0(0,0,0)(341,0,0)(0,768,0)(512,768,0)，composed layout1={VC0(0,0,0)(341,0,0)(0,768,0)(512,768,0),
VC1(512,384,0)(1024,384,0)(512,768,0)(1024,768,0)，VC1(512,384,0)(1024,384,0)(512,768,0)(1024,768,0),
VC2(512,0,0)(1024,0,0)(512,384,0)(1024,384,0)}，VC2(512,0,0)(1024,0,0)(512,384,0)(1024,384,0)},
Composed layout2={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0)，Composed layout2={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
媒体接收方接收媒体提供方发送的至少一种媒体画面组合选择信息，并结合媒体接收方的终端显示能力、媒体接收方的本地策略确定媒体画面组成信息，该媒体画面组成信息中包括了媒体接收方的组合画面的选择结果，该媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP响应消息中，还可以是媒体接收方将媒体画面组成信息发送给门户网站，由媒体提供方从门户网站获取媒体接收方的组合画面的选择结果，此处以媒体画面组成信息携带在CLUE媒体配置消息中为例，媒体接收方选择采用3路视频画面(VC0,VC1,VC2)的组合，媒体接收方在CLUE媒体配置消息中的媒体画面组合选择相关描述如下：The media receiver receives at least one media screen combination selection information sent by the media provider, and determines the media screen composition information in combination with the media receiver's terminal display capability and the media receiver's local policy. The media screen composition information includes the media reception The media screen composition information can be carried in the CLUE media configuration message or in the SDP response message. It can also be that the media receiver sends the media screen composition information to the portal website, which is provided by the media. The party obtains the selection result of the combined picture of the media receiver from the portal website. Here, taking the media picture composition information carried in the CLUE media configuration message as an example, the media receiver chooses to use a combination of 3 video pictures (VC0, VC1, VC2). The media receiver's media screen combination selection in the CLUE media configuration message is described as follows:
Configuration:Configuration:
Capture Scene1:Capture Scene1:
Capture Scene Entry 2:VC4Capture Scene Entry 2: VC4
VC4:encoding group=EG1,content=view:room.full,composed=true,composedoption={composed layout1}。VC4:encoding group=EG1,content=view:room.full,composed=true,composed option={composed layout1}.
媒体提供方获知媒体接收方的组合画面选择结果之后，向媒体接收方发送组合画面的第一媒体参数，该第一媒体参数可以携带在第三SDP响应消息中，还可以是媒体提供方将该组合画面的第一媒体参数发送给门户网站，由媒体接收方从门户网站获取；媒体接收方接收到组合画面的第一媒体参数之后，向媒体提供方发送组合画面的第二媒体参数，该第二媒体参数可以携带在第三SDP响应消息中，还可以是媒体接收方将组合画面的第二媒体参数发送给门户网站，由媒体提供方从门户网站获取该组合画面的第二媒体参数，需要注意的是，媒体提供方和媒体接收方之间进行组合画面的第一媒体参数和第二媒体参数的交互主要是进行组合画面的编解码方式和带宽等参数的协商；之后，媒体提供方将媒体接收方选择的组合画面发送给媒体接收方予以呈现。After the media provider learns the selection result of the combined picture of the media receiver, it sends the first media parameter of the combined picture to the media receiver. The first media parameter can be carried in the third SDP response message, or the media provider can send the combined picture The first media parameter of the combined picture is sent to the portal website, and the media receiver obtains it from the portal website; after the media receiver receives the first media parameter of the combined picture, it sends the second media parameter of the combined picture to the media provider. The second media parameter can be carried in the third SDP response message, and the media receiver can also send the second media parameter of the combined picture to the portal website, and the media provider can obtain the second media parameter of the combined picture from the portal website. It should be noted that the interaction between the first media parameter and the second media parameter of the combined picture between the media provider and the media receiver is mainly to negotiate parameters such as codec mode and bandwidth of the combined picture; after that, the media provider will The combined picture selected by the media receiver is sent to the media receiver for presentation.
本实施例提供的方法中，通过媒体提供方将至少一种媒体画面组合选择信息发送给媒体接收方，媒体接收方根据其终端的显示能力以及本地策略媒体画面组合选择信息进行选择，并将选择结果携带在媒体画面组成信息中发送给媒体提供方，以使媒体提供方将对应的组合画面发送给媒体接收方，即媒体接收方是按照自身的意愿或者能力对组合画面进行选择和呈现，因此提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided in this embodiment, the media provider sends at least one media screen combination selection information to the media receiver, and the media receiver selects according to the display capability of its terminal and the local strategy media screen combination selection information, and the selected The result is carried in the media screen composition information and sent to the media provider, so that the media provider can send the corresponding combined screen to the media receiver, that is, the media receiver selects and presents the combined screen according to its own wishes or capabilities, so Improves the flexibility of combining media screen presentation, and improves the conference experience of media receiver users.
图15为本发明提供的多画面组合的呈现方法实施例十五的信令流程图，本实施例涉及的方法是通过CULE方式来实现多画面组合呈现的，如图15所示，该方法包括：FIG. 15 is a signaling flow chart of Embodiment 15 of the multi-picture combination presentation method provided by the present invention. The method involved in this embodiment implements multi-picture combination presentation through CULE. As shown in FIG. 15 , the method includes :
S8001：媒体提供方发送第二SDP请求消息给媒体接收方，该第二SDP请求消息携带支持CLUE协议指示。S8001: The media provider sends a second SDP request message to the media receiver, where the second SDP request message carries an indication of supporting the CLUE protocol.
S8002：媒体接收方发送第二SDP响应消息给媒体提供方，该第二SDP响应消息携带支持CLUE协议指示。S8002: The media receiver sends a second SDP response message to the media provider, where the second SDP response message carries an indication of supporting the CLUE protocol.
具体的，在媒体提供方发送第一CLUE媒体通告消息给媒体接收方之前，要进行CLUE通道的建立，即媒体提供方将支持CLUE协议指示的信息携带在第二SDP请求消息中发送给媒体接收方，相应的，媒体接收方在获知媒体提供方支持CLUE协议后，会向媒体提供方发送第二SDP响应消息，该第二SDP响应消息中携带支持CLUE协议指示，CLUE通道建立成功。Specifically, before the media provider sends the first CLUE media announcement message to the media receiver, the establishment of the CLUE channel must be performed, that is, the media provider carries the information indicated by the CLUE protocol in the second SDP request message and sends it to the media receiver. Correspondingly, after learning that the media provider supports the CLUE protocol, the media receiver will send a second SDP response message to the media provider. The second SDP response message carries an indication of supporting the CLUE protocol, and the CLUE channel is established successfully.
S8003：媒体提供方发送第一CLUE媒体通告消息给媒体接收方，该第一CLUE媒体通告消息中携带媒体画面组合能力指示信息。S8003: The media provider sends a first CLUE media notification message to the media receiver, where the first CLUE media notification message carries media screen combination capability indication information.
S8004：媒体接收方发送第一CLUE媒体配置消息给媒体提供方，第一CLUE媒体配置消息中携带媒体画面组成信息。S8004: The media receiver sends a first CLUE media configuration message to the media provider, where the first CLUE media configuration message carries media screen composition information.
具体的，媒体提供方发送第一CLUE媒体通告消息给媒体接收方，该第一CLUE媒体通告消息携带媒体画面组合能力指示信息，用以使得媒体接收方根据第一CLUE媒体通告消息获知媒体提供方具有组合画面的能力、媒体提供方提供的多路媒体画面以及多路媒体画面相关的位置信息；在此以一个例子来说明：媒体提供方可以提供3路单独媒体画面(VC0,VC1,VC2)，媒体提供方在第一CLUE媒体通告消息中的媒体组合画面能力相关描述如下：Specifically, the media provider sends the first CLUE media notification message to the media receiver, and the first CLUE media notification message carries media screen combination capability indication information, so that the media receiver can inform the media provider according to the first CLUE media notification message It has the ability to combine screens, multiple media screens provided by the media provider, and location information related to multiple media screens; here is an example to illustrate: the media provider can provide 3 separate media screens (VC0, VC1, VC2) , the media provider's media composition screen capabilities in the first CLUE media notification message are described as follows:
Advertisement:Advertisement:
Capture Scene 1:Capture Scene 1:
Capture Scene Entry 1:VC0,VC1,VC2Capture Scene Entry 1:VC0,VC1,VC2
Capture Scene Entry 2:VC4Capture Scene Entry 2: VC4
VC0:encoding group=EG0,content=view:table.left;VC0:encoding group=EG0,content=view:table.left;
VC1:encoding group=EG0,content=view:table.center;VC1:encoding group=EG0,content=view:table.center;
VC2:encoding group=EG0,content=view:table.right;VC2:encoding group=EG0,content=view:table.right;
VC4:encoding group=EG1,content=view:room,composed=true；VC4:encoding group=EG1,content=view:room,composed=true;
上述例子中，对于媒体画面组合能力描述的定义包含在composed中，携带该composed参数，即隐含表征媒体提供方具有媒体画面组合的能力。In the above example, the definition of the media screen composition capability description is included in composed, and the composed parameter is carried, which implicitly indicates that the media provider has the media screen composition capability.
具体的，媒体接收方接收到媒体提供方发送的第一CLUE媒体通告消息后，结合媒体接收方会议室的终端显示能力以及媒体接收方的本地策略，确定媒体画面组成信息，例如选择由哪几个媒体画面来进行组合、该组合画面如何布局等，并向媒体提供方发送第一CLUE配置消息，该第一CLUE媒体配置消息中携带媒体画面组成信息；此处，同样以一个例子来说明：假设媒体接收方只有一个显示屏，只能接收1路媒体画面，最终媒体接收方选择采用其中2路媒体画面(VC0,VC1)来组合成一路，二路呈现左右平分画面；媒体接收方在第一CLUE配置消息中的媒体组合选择相关描述如下：Specifically, after receiving the first CLUE media notification message sent by the media provider, the media receiver determines the composition information of the media screen in combination with the terminal display capability of the media receiver's meeting room and the local strategy of the media receiver, such as which How to combine a media screen, how to layout the combined screen, etc., and send the first CLUE configuration message to the media provider, the first CLUE media configuration message carries the composition information of the media screen; here, an example is also used to illustrate: Assume that the media receiver has only one display screen and can only receive one channel of media images. Finally, the media receiver chooses to use two channels of media images (VC0, VC1) to combine into one channel, and the second channel presents a left and right split screen; The relevant description of media combination selection in a CLUE configuration message is as follows:
Configuration:Configuration:
Capture Scene1:Capture Scene1:
Capture Scene Entry2:VC4Capture Scene Entry2: VC4
VC4：encoding group=EG1，content=view:room.partial，composed=true，composed layout={VC0（0,0,0）(512,0,0)(0,768,0)(512,768,0),VC1(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}VC4: encoding group=EG1, content=view:room.partial, composed=true, composed layout={VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0),VC1 (512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}
上述例子中，媒体画面组合结果包含在composed layout参数中，表示该组合画面由哪些单独媒体画面组成，而且画面的布局中每个单独画面采用三维立体坐标来表示媒体画面的范围。In the above example, the composition result of the media screen is included in the composed layout parameter, indicating which individual media screens are composed of the combined screen, and each individual screen in the layout of the screen uses three-dimensional coordinates to represent the range of the media screen.
S8005：媒体提供方判断是否支持上述媒体画面组成信息；若支持，执行步骤S8006，若不支持，则执行步骤S8009。S8005: The media provider judges whether the above media screen composition information is supported; if yes, execute step S8006; if not, execute step S8009.
S8006：媒体提供方发送第三SDP请求消息给媒体接收方，该第三SDP请求消息中携带组合画面的第一媒体参数。S8006: The media provider sends a third SDP request message to the media receiver, where the third SDP request message carries the first media parameter of the combined picture.
S8007：媒体接收方发送第三SDP响应消息给媒体提供方，该第三SDP响应消息中携带组合画面的第二媒体参数。S8007: The media receiver sends a third SDP response message to the media provider, where the third SDP response message carries the second media parameter of the combined picture.
S8008：媒体提供方发送组合画面给媒体接收方。S8008: The media provider sends the combined picture to the media receiver.
具体的，媒体提供方接收媒体接收方发送的第一CLUE媒体配置消息后，该第一CLUE媒体配置消息携带媒体画面组成信息，并且该媒体画面组成信息包括至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，或，至少两路媒体画面的组合信息，则媒体提供方根据对上述媒体画面组成信息中的支持能力，进行判断：若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持上述至少两路媒体画面的这两种场景中的任一场景，则媒体提供方发送第三SDP请求消息给媒体接收方，该第三SDP请求消息中携带组合画面的第一媒体参数，并且上述组合画面是媒体提供方根据媒体画面组成信息所确定的；媒体接收方接收到媒体提供方发送的第三SDP请求消息之后，发送第三SDP响应消息给媒体提供方，该第三SDP响应消息中携带组合画面的第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，媒体接收方和媒体提供方根据媒体画面参数的协商结果，建立双向多流媒体传输通道，媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定组合画面，并将组合画面发送给媒体接收方予以呈现。Specifically, after the media provider receives the first CLUE media configuration message sent by the media receiver, the first CLUE media configuration message carries media screen composition information, and the media screen composition information includes combination information of at least two media screens and at least The layout information of two channels of media screens, or the combination information of at least two channels of media screens, the media provider shall judge according to the support capability of the composition information of the above media screens: if the media provider supports the at least two channels of media screens Combination and layout of at least two media screens, or, to support any of the above two scenarios of at least two media screens, the media provider sends a third SDP request message to the media receiver, and the third SDP request The first media parameter of the combined picture is carried in the message, and the combined picture is determined by the media provider according to the composition information of the media picture; after the media receiver receives the third SDP request message sent by the media provider, it sends the third SDP response The message is sent to the media provider, and the third SDP response message carries the second media parameter of the combined picture, where the first media parameter and the second media parameter are the encoding and decoding methods and transmission of the combined picture by the media provider and the media receiver Negotiation of parameters such as bandwidth, the media receiver and the media provider establish a two-way multi-stream media transmission channel according to the negotiation results of the media screen parameters, and the media provider determines the combination according to the above media screen composition information, the first media parameter and the second media parameter screen, and send the combined screen to the media receiver for presentation.
S8009：媒体提供方发送否定应答消息给媒体接收方。S8009: The media provider sends a negative response message to the media receiver.
S8100：媒体接收方根据否定应答消息生成更新后的媒体画面组成信息并携带在更新后的第一CLUE媒体配置消息中，发送给媒体提供方。S8100: The media receiver generates updated media screen composition information according to the negative response message, carries it in the updated first CLUE media configuration message, and sends it to the media provider.
S8101：媒体提供方对第一CLUE媒体配置消息进行S8005的判断。S8101: The media provider performs S8005 judgment on the first CLUE media configuration message.
具体的，若媒体提供方不支持上述媒体画面组成信息，，则媒体提供方发送否定应答消息给媒体接收方，向媒体接收方表示不接受该媒体画面组成信息，该否定应答消息可以是CLUE否定应答消息；媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息携带在更新后的第一CLUE媒体配置消息中再次发送给媒体提供方，媒体提供方接收更新后的第一CLUE媒体配置消息，依然对其再次进行上述S8005的判断过程，在此不再赘述。Specifically, if the media provider does not support the above-mentioned media picture composition information, then the media provider sends a negative response message to the media receiver, expressing to the media receiver that it does not accept the media picture composition information, and the negative response message can be CLUE negative Response message; after receiving the negative response message, the media receiver regenerates new media screen composition information, that is, updates the combination information of at least two media screens and the layout information of at least two media screens, and updates the media screen The composition information is carried in the updated first CLUE media configuration message and sent to the media provider again, and the media provider receives the updated first CLUE media configuration message, and still performs the above-mentioned judgment process of S8005 on it again. repeat.
本发明实施例提供的方法，通过媒体提供方向媒体接收方发送携带媒体画面组合能力指示信息的第一CLUE媒体通告消息，媒体接收方根据所接收到的第一CLUE媒体通告消息、媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，并将该媒体画面的组成信息携带在第一CLUE媒体配置消息中，媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the first CLUE media announcement message carrying the indication information of the media screen combination capability to the media receiver, and the media receiver sends the first CLUE media announcement message according to the received first CLUE media announcement message, the media receiver's The display capability of the terminal and the local policy of the media receiver determine the composition information of the media screen, and carry the composition information of the media screen in the first CLUE media configuration message, and the media provider executes according to the media screen composition information determined by the media receiver. Corresponding operations make the media combination picture received by the media receiver be the media combination picture determined by the media provider according to the media picture composition information sent by the media receiver, that is, the media receiver can customize the combination picture it wants to present, thereby improving the The flexibility of combined media screen presentation improves the conference experience of media receiver users.
图16为本发明提供的多画面组合的呈现方法实施例十六的信令流程图，本实施例涉及的方法是通过SDP协议的方式实现多画面组合的呈现的，如图16所示，该方法包括：FIG. 16 is a signaling flow chart of Embodiment 16 of the method for presenting a multi-picture combination provided by the present invention. The method involved in this embodiment realizes the presentation of a multi-picture combination through the SDP protocol. As shown in FIG. 16 , this Methods include:
S9001：媒体提供方发送第一SDP请求消息给媒体接收方，该第一SDP请求消息中携带媒体画面组合能力指示信息。S9001: The media provider sends a first SDP request message to the media receiver, where the first SDP request message carries media picture combination capability indication information.
S9002：媒体接收方发送第一SDP响应消息给媒体提供方，该第一SDP响应消息中携带所述媒体画面组成信息。S9002: The media receiver sends a first SDP response message to the media provider, where the first SDP response message carries the media picture composition information.
具体的，媒体提供方向媒体接收方发送第一SDP请求消息，该第一SDP请求消息携带媒体画面组合能力指示信息，并且还携带了多路媒体画面信息，并且该第一SDP请求消息可以携带在SIP消息中传输；此处以一个例子来说明：媒体提供方可以提供3路单独媒体视频画面(VC0,VC1,VC2)，媒体提供方在第一SDP请求消息中的媒体画面信息相关描述如下：Specifically, the media provider sends a first SDP request message to the media receiver, the first SDP request message carries media picture combination capability indication information, and also carries multi-channel media picture information, and the first SDP request message can be carried in the Transmission in a SIP message; an example is used here to illustrate: the media provider can provide 3 separate media video images (VC0, VC1, VC2), and the relevant description of the media image information in the first SDP request message of the media provider is as follows:
a=video-compose-capa，指示具有画面组合能力；a=video-compose-capa, indicating that it has the ability to compose images;
m=video 10000 RTP/AVP 98，指示第一路画面信息VC0；m=video 10000 RTP/AVP 98, indicating the first video information VC0;
a=label:VC0a=label:VC0
a=rtpmap:98 H264/90000a=rtpmap:98 H264/90000
m=video 10002 RTP/AVP 98，指示第二路画面信息VC1；m=video 10002 RTP/AVP 98, indicating the second video information VC1;
a=label:VC1a=label:VC1
a=rtpmap:98 H264/9000a=rtpmap:98 H264/9000
m=video 10004 RTP/AVP 31，指示第三路画面信息VC2；m=video 10004 RTP/AVP 31, indicating the third video information VC2;
a=label:VC2a=label:VC2
a=rtpmap:31 H261/9000。a=rtpmap:31 H261/9000.
媒体接收方接收媒体提供方发送的第一SDP请求消息后，结合媒体接收方会议室的终端显示能力以及媒体接收方的本地策略，确定媒体画面组成信息，例如选择由哪几个媒体画面来进行组合、该组合画面如何布局等，并向媒体提供方发送第一SDP响应消息，该第一SDP响应消息中携带媒体画面组成信息；此处，同样以一个例子来说明：假设媒体接收方只有一个显示屏，只能接收1路视频画面，最终媒体接收方选择采用其中2路视频画面(VC0,VC1)来组合成一路，二路呈现左右平分画面；媒体接收方在第一SDP响应消息中的媒体画面组合的相关描述如下：After receiving the first SDP request message sent by the media provider, the media receiver determines the composition information of the media screen, for example, which media screens to select based on the terminal display capability of the media receiver's conference room and the local strategy of the media receiver. Combination, how to layout the combined screen, etc., and send the first SDP response message to the media provider, the first SDP response message carries the composition information of the media screen; here, an example is also used to illustrate: assume that the media receiver has only one The display screen can only receive 1 channel of video images. Finally, the media receiver chooses to use 2 channels of video images (VC0, VC1) to combine into one channel. The relevant description of the combination of media screens is as follows:
a=composed layout：VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0)，VC1(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)，表示媒体接收方希望将VC0,VC1组合，按照左右布局，各占50%；a=composed layout: VC0(0,0,0)(512,0,0)(0,768,0)(512,768,0), VC1(512,0,0)(1024,0,0)(512,768,0) )(1024,760,0), indicating that the media receiver wants to combine VC0 and VC1, according to the left and right layout, each accounting for 50%;
m=video 20000 RTP/AVP 98，指示第一路画面信息VC0；m=video 20000 RTP/AVP 98, indicating the first video information VC0;
a=label：VC0a=label:VC0
a=rtpmap：98 H264/90000a=rtpmap:98 H264/90000
m=video 0 RTP/AVP 31，指示第二路画面信息VC1；m=video 0 RTP/AVP 31, indicating the second video information VC1;
a=label：VC1a=label:VC1
a=rtpmap：31 H261/9000a=rtpmap:31 H261/9000
m=video 0 RTP/AVP 31，指示第三路画面信息VC2；m=video 0 RTP/AVP 31, indicating the third video information VC2;
a=label：VC2a=label:VC2
a=rtpmap：31 H261/9000。a=rtpmap:31 H261/9000.
S9003：媒体提供方判断是否支持上述媒体画面组成信息；若支持，执行步骤S9004，若不支持，则执行步骤S9007。S9003: The media provider judges whether the above media screen composition information is supported; if yes, execute step S9004; if not, execute step S9007.
S9004：媒体提供方发送第三SDP请求消息给媒体接收方，该第三SDP请求消息中携带组合画面的第一媒体参数。S9004: The media provider sends a third SDP request message to the media receiver, where the third SDP request message carries the first media parameter of the combined picture.
S9005：媒体接收方发送第三SDP响应消息给媒体提供方，该第三SDP响应消息中携带组合画面的第二媒体参数。S9005: The media receiver sends a third SDP response message to the media provider, where the third SDP response message carries the second media parameter of the combined picture.
S9006：媒体提供方发送组合画面给媒体接收方。S9006: The media provider sends the combined picture to the media receiver.
具体的，媒体提供方接收媒体接收方发送的第一SDP响应消息，该第一SDP响应消息中携带媒体画面组成信息，并且该媒体画面组成信息中包括至少两路媒体画面的组合信息以及至少两路媒体画面的布局信息，或，至少两路媒体画面的组合信息，媒体提供方根据对上述至少两路媒体画面的组合和至少两路媒体画面的布局的支持能力，进行判断；若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，则若媒体提供方支持上述至少两路媒体画面的组合和至少两路媒体画面的布局，或，支持上述至少两路媒体画面的组合这两种场景中的任一场景，则媒体提供方发送第三SDP请求消息给媒体接收方，该第三SDP请求消息中携带组合画面的第一媒体参数，并且上述组合画面是媒体提供方根据媒体画面组成信息所确定的；媒体接收方接收到媒体提供方发送的第三SDP请求消息之后，发送第三SDP响应消息给媒体提供方，该第三SDP响应消息中携带组合画面的第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，媒体接收方和媒体提供方根据媒体画面参数的协商结果，建立双向多流媒体传输通道，媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定组合画面，并将组合画面发送给媒体接收方予以呈现。Specifically, the media provider receives the first SDP response message sent by the media receiver, the first SDP response message carries media screen composition information, and the media screen composition information includes combination information of at least two media screens and at least two The layout information of media screens of at least two channels, or the combination information of at least two media screens, the media provider shall make a judgment based on the above-mentioned combination of at least two media screens and the ability to support the layout of at least two media screens; if the media provider Support the combination of the above-mentioned at least two media screens and the layout of at least two media screens, if the media provider supports the above-mentioned combination of at least two media screens and the layout of at least two media screens, or, supports the above-mentioned at least two media screens combination of any of these two scenarios, the media provider sends a third SDP request message to the media receiver, the third SDP request message carries the first media parameter of the combined picture, and the combined picture is the media provider After receiving the third SDP request message sent by the media provider, the media receiver sends the third SDP response message to the media provider, and the third SDP response message carries the third SDP response message of the combined picture. Two media parameters, where the first media parameter and the second media parameter are the negotiation between the media provider and the media receiver on parameters such as the codec method and the transmission bandwidth of the combined picture, and the media receiver and the media provider are based on the parameters of the media picture. As a result of the negotiation, a two-way multi-stream media transmission channel is established, and the media provider determines the combined picture according to the above media picture composition information, the first media parameter and the second media parameter, and sends the combined picture to the media receiver for presentation.
S9007：媒体提供方发送否定应答消息给媒体接收方。S9007: The media provider sends a negative response message to the media receiver.
S9008：媒体接收方根据否定应答消息生成更新后的媒体画面组成信息并携带在第一SDP响应消息中，发送给媒体提供方。S9008: The media receiver generates updated media screen composition information according to the negative response message, carries it in the first SDP response message, and sends it to the media provider.
S9009：媒体提供方对第一SDP响应消息执行S9003的判断。S9009: The media provider performs the judgment of S9003 on the first SDP response message.
具体的，若媒体提供方不支持上述媒体画面组成信息中的任一场景，则媒体提供方发送否定应答消息给媒体接收方，向媒体接收方表示不接受该媒体画面组成信息，该否定应答消息可以是SIP否定应答消息；媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息携带在第一SDP响应消息中再次发送给媒体提供方，媒体提供方接收更新后的第一SDP响应消息，依然对其再次进行上述S9003的判断过程，在此不再赘述。Specifically, if the media provider does not support any scenario in the above-mentioned media picture composition information, the media provider sends a negative response message to the media receiver, expressing to the media receiver that it does not accept the media picture composition information, and the negative response message It can be a SIP negative response message; after receiving the negative response message, the media receiver regenerates new media picture composition information, that is, updates the composition information of at least two channels of media pictures and the layout information of at least two channels of media pictures, and will update The updated media screen composition information is carried in the first SDP response message and sent to the media provider again, and the media provider receives the updated first SDP response message, and still performs the above-mentioned judgment process of S9003 on it again, and will not go into details here .
本发明实施例提供的方法，通过媒体提供方向媒体接收方发送携带媒体画面组合能力指示信息的第一SDP请求消息，媒体接收方根据所接收到的第一SDP请求消息、媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，并将该媒体画面的组成信息携带在第一SDP响应消息中，媒体提供方根据媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the first SDP request message carrying the indication information of the media picture combination capability to the media receiver, and the media receiver displays the The capability and the local policy of the media receiver determine the composition information of the media screen, and carry the composition information of the media screen in the first SDP response message, and the media provider performs corresponding operations according to the media screen composition information determined by the media receiver , so that the media combination picture received by the media receiver is the media combination picture determined by the media provider according to the composition information of the media picture sent by the media receiver, that is, the media receiver can customize the combination picture it wants to present, thereby improving the combined media picture The flexibility of presentation improves the conference experience of media receiver users.
图17为本发明提供的多画面组合的呈现方法实施例十七的信令流程图，本实施例涉及的方法是通过门户网站的方式实现多画面组合的呈现的，如图17所示，该方法包括：Fig. 17 is a signaling flow chart of the seventeenth embodiment of the method for presenting a multi-picture combination provided by the present invention. The method involved in this embodiment realizes the presentation of a multi-picture combination through a portal website, as shown in Fig. 17 . Methods include:
S1100：媒体提供方发送媒体画面组合能力指示信息以及媒体画面的缩略图给门户网站。S1100: The media provider sends media screen combination capability indication information and a thumbnail of the media screen to the portal website.
S1101：媒体接收方从门户网站获取媒体画面组合能力指示信息和媒体画面的缩略图，并根据上述媒体画面组合能力指示信息和媒体画面的缩略图确定媒体画面组成信息。S1101: The media receiver obtains media screen combination capability indication information and media screen thumbnails from the portal website, and determines media screen composition information according to the media screen combination capability indication information and media screen thumbnails.
S1102：媒体接收方发送媒体画面组成信息给门户网站。S1102: The media receiver sends the composition information of the media screen to the portal website.
S1103：媒体提供方从门户网站获取媒体接收方发送的媒体画面组成信息。S1103: The media provider obtains the composition information of the media screen sent by the media receiver from the portal website.
具体的，本发明实施例中媒体提供方和媒体接收放都已经成功登陆到门户网站上；媒体提供方通过门户网站的接口将媒体画面组合能力指示信息和媒体画面的缩略图发送给门户网站，并且呈现在门户网站的界面上，以使媒体接收方从门户网站获取媒体画面组合能力指示信息和媒体画面的缩略图；媒体接收方通过门户网站接口查看到媒体提供方提供的媒体画面缩略图，从门户网站获取媒体画面组合能力指示信息和媒体画面的缩略图，并结合媒体接收方的会议室的终端显示能力以及媒体接收方本地策略，确定媒体画面组成信息，并将该媒体画面组成信息中包含至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，或，至少两路媒体画面的组合信息，通过门户网站接口呈现在门户网站界面上；媒体提供方从门户网站上获取该媒体画面组成信息。Concretely, in the embodiment of the present invention, both the media provider and the media receiver have successfully logged on to the portal website; the media provider sends the media screen combination capability indication information and the thumbnail of the media screen to the portal website through the interface of the portal website, And present on the interface of the portal website, so that the media receiver obtains the thumbnail of the media picture combination capability indication information and the media picture from the portal website; the media receiver checks the media picture thumbnail provided by the media provider through the portal website interface, Obtain the indication information of the media screen combination capability and the thumbnail of the media screen from the portal website, and combine the terminal display capability of the media receiver's conference room and the local strategy of the media receiver to determine the media screen composition information, and include the media screen composition information in the media screen composition information Contains combination information of at least two channels of media images and layout information of at least two channels of media images, or, combination information of at least two channels of media images is presented on the interface of the portal website through the interface of the portal website; the media provider obtains the information from the portal website Media picture composition information.
S1104：媒体提供方判断是否支持上述媒体画面组成信息；若支持，执行步骤S1105，若不支持，则执行步骤S1108。S1104: The media provider judges whether the above media screen composition information is supported; if yes, execute step S1105; if not, execute step S1108.
S1105：媒体提供方发送组合画面的第一媒体参数给门户网站。S1105: The media provider sends the first media parameter of the combined screen to the portal website.
S1106：媒体接收方从门户网站获取组合画面的第一媒体参数，并发送组合画面的第二媒体参数给门户网站。S1106: The media receiver obtains the first media parameter of the combined screen from the portal website, and sends the second media parameter of the combined screen to the portal website.
S1107：媒体提供方接收上述第二媒体参数，并发送组合画面给媒体接收方。S1107: The media provider receives the above-mentioned second media parameters, and sends the combined picture to the media receiver.
具体的，媒体提供方从门户网站上获取该媒体画面组成信息，并且该媒体画面组成信息包括至少两路媒体画面的组合信息及至少两路媒体画面的布局信息，或，至少两路媒体画面的组合信息；媒体提供方对该媒体画面组成信息中所包含的内容进行是否支持的判断，具体为：若媒体提供方支持上述媒体画面组成信息，则媒体提供方发送组合画面的第一媒体参数给门户网站，以使媒体接收方从门户网站获取第一媒体参数；媒体接收方从门户网站获取到媒体提供方发送的第一媒体参数之后，向门户网站发送第二媒体参数，以使媒体提供方可以从门户网站获取第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商；之后，媒体提供方根据上述媒体画面组成信息、第一媒体参数和第二媒体参数确定组合画面，并将组合画面发送给媒体接收方予以呈现。Specifically, the media provider obtains the media screen composition information from the portal website, and the media screen composition information includes combination information of at least two media screens and layout information of at least two media screens, or, at least two media screen layout information Combination information; the media provider judges whether the content contained in the media screen composition information is supported, specifically: if the media provider supports the above media screen composition information, the media provider sends the first media parameter of the composite screen to The portal website, so that the media receiver obtains the first media parameter from the portal website; after the media receiver obtains the first media parameter sent by the media provider from the portal website, it sends the second media parameter to the portal website, so that the media provider The second media parameter can be obtained from the portal website, where the first media parameter and the second media parameter are the negotiation between the media provider and the media receiver on parameters such as codec mode and transmission bandwidth of the combined picture; The composition information of the media screen, the first media parameter and the second media parameter determine the combined screen, and send the combined screen to the media receiver for presentation.
S1108：媒体提供方发送否定应答消息给门户网站。S1108: The media provider sends a negative response message to the portal website.
S1109：媒体接收方从门户网站获取否定应答消息，并根据否定应答消息生成更新后的媒体画面组成信息，发送给门户网站。S1109: The media receiver acquires a negative response message from the portal website, generates updated media screen composition information according to the negative response message, and sends it to the portal website.
S1110：媒体提供方从门户网站获取更新后的媒体画面组成信息，并对更新后的媒体画面组成信息执行S1104的判断。S1110: The media provider obtains the updated media screen composition information from the portal website, and performs the judgment of S1104 on the updated media screen composition information.
具体的，若媒体提供方不支持上述媒体画面组成信息，则发送否定应答消息给门户网站，以使媒体接收方从门户网站获知媒体提供方不接受该媒体画面组成信息，则媒体接收方会重新生成新的媒体画面组成信息，即更新至少两路媒体画面的组合信息和至少两路媒体画面的布局信息，将更新后的媒体画面组成信息再次发送给门户网站，媒体提供方从门户网站获取更新后的媒体画面组成信息，依然对其再次进行S1104的判断过程，在此不再赘述。Concretely, if the media provider does not support the above-mentioned media screen composition information, then send a negative response message to the portal website, so that the media receiver knows from the portal website that the media provider does not accept the media screen composition information, and the media receiver will re- Generate new media screen composition information, that is, update the combination information of at least two channels of media screens and the layout information of at least two channels of media screens, and send the updated media screen composition information to the portal website again, and the media provider obtains updates from the portal website The subsequent media screen composition information is still subjected to the judgment process of S1104 again, which will not be repeated here.
本发明实施例提供的方法，通过媒体提供方向门户网站发送携带媒体画面组合能力指示信息和媒体画面缩略图，媒体接收方从门户网站获取该媒体画面组合能力指示信息和媒体画面缩略图后，结合媒体接收方的终端显示能力以及媒体接收方的本地策略确定媒体画面的组成信息，并将该媒体画面的组成信息发送给门户网站；媒体提供方根据从门户网站获取的媒体接收方所确定的媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的媒体画面组成信息确定的媒体组合画面，即媒体接收方可以定制自身所要呈现的组合画面，从而提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided by the embodiment of the present invention, the media provider sends the media screen combination capability indication information and the media screen thumbnail to the portal website, and the media receiver acquires the media screen combination capability indication information and the media screen thumbnail from the portal website, and combines the The terminal display capability of the media receiver and the local strategy of the media receiver determine the composition information of the media screen, and send the composition information of the media screen to the portal website; The screen composition information performs corresponding operations, so that the media combination screen received by the media receiver is the media combination screen determined by the media provider according to the media screen composition information sent by the media receiver, that is, the media receiver can customize the combination screen it wants to present , so as to improve the flexibility of presenting combined media screens and improve the conference experience of the media receiver user.
图18为本发明提供的多画面组合的呈现方法实施例十八的信令流程图，本实施例涉及的方法针对的是会议召开之后，网络情况发生变化，组合画面发生变更的处理方法，如图18所示，该方法包括：Fig. 18 is a signaling flow chart of Embodiment 18 of the presenting method of multi-picture combination provided by the present invention. The method involved in this embodiment is aimed at the processing method of changing the combined screen after the meeting is held and the network situation changes, such as As shown in Figure 18, the method includes:
S2100：媒体提供方发送媒体画面变化信息给媒体接收方。S2100: The media provider sends media screen change information to the media receiver.
S2101：媒体接收方接收该媒体画面变化信息，并根据该媒体画面变化信息生成的更新后的第一媒体画面组成信息，该第一媒体画面组成信息中包括：至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，或，至少两路媒体画面的第一更新组合信息。S2101: The media receiver receives the media screen change information, and generates updated first media screen composition information according to the media screen change information, the first media screen composition information includes: the first updates of at least two media screens Combination information and first updated layout information of at least two channels of media images, or first updated combination information of at least two channels of media images.
S2102：媒体接收方发送更新后的第一媒体画面组成信息。S2102: The media receiver sends updated composition information of the first media screen.
具体的，在视频会议召开之后，即在媒体提供方发送组合画面给媒体接收方之后，媒体提供方检测到网络情况发生变化，比如拥塞或者组合画面中某一源会场离会或者新的会场入会等，则媒体提供方会向媒体接收方发送媒体画面变化信息，其中，该媒体画面变化信息可以携带在CLUE媒体通告消息中，也可以携带在SDP请求消息中，还可以是媒体提供方将媒体画面变化信息发送给门户网站，由媒体接收方从门户网站获取该媒体画面变化信息；此处以该媒体画面变化信息携带在CLUE媒体通告消息中为例来说明，假设媒体提供方原来提供3路单独视频画面(VC0,VC1,VC2)，媒体接收方选择VC0，VC1进行组合，这时VC0画面消失，媒体提供方在CLUE媒体通告消息中通知组合画面变更，具体描述如下：Specifically, after the video conference is held, that is, after the media provider sends the combined picture to the media receiver, the media provider detects that the network situation changes, such as congestion or a source site in the combined picture leaves the conference or a new site joins the conference etc., then the media provider will send the media screen change information to the media receiver, where the media screen change information can be carried in the CLUE media notification message, or can be carried in the SDP request message, or the media provider can send the media The screen change information is sent to the portal website, and the media receiver obtains the media screen change information from the portal website; here, the media screen change information is carried in the CLUE media notification message as an example, assuming that the media provider originally provided 3 separate For video images (VC0, VC1, VC2), the media receiver selects VC0 and VC1 for combination, then the VC0 image disappears, and the media provider notifies the combined image change in the CLUE media notification message, the specific description is as follows:
Advertisement：Advertisements:
Capture Scene 1：Capture Scene 1:
Capture Scene Entry 1：VC1,VC2Capture Scene Entry 1: VC1, VC2
Capture Scene Entry 2：VC4Capture Scene Entry 2: VC4
VC1：encoding group=EG0，content=view:table.center；VC1: encoding group=EG0, content=view:table.center;
VC2：encoding group=EG0，content=view:table.right；VC2: encoding group=EG0, content=view:table.right;
VC4：encoding group=EG1，content=view:room,composed=true,composedupdate={VC0：off，VC1}；VC4: encoding group=EG1, content=view:room, composed=true, composedupdate={VC0:off, VC1};
上述例子中，媒体画面组合结果包含在composed update中，表示组合画面变更，VC0关闭。In the above example, the composition result of the media screen is included in the composed update, indicating that the composite screen has changed, and VC0 is closed.
媒体接收方在接收到媒体提供方发送的媒体画面变化信息之后，获知该视频会议的网络情况发生变化，则媒体接收方根据该媒体画面变化信息生成更新后的第一媒体画面组成信息，该第一媒体画面组成信息中包括：至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，或，至少两路媒体画面的第一更新组合信息，并将该第一媒体画面组成信息发送给媒体提供方，其中，该第一媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP请求响应消息中，还可以是媒体接收方将更新后的第一媒体画面组成信息发送给门户网站，由媒体提供方从门户网站获取更新后的第一媒体画面组成信息；这里同样以第一媒体画面组成信息携带在CLUE媒体配置消息中为例来说明：媒体接收方选择采用2路视频画面(VC1,VC2)来组合成一路，二路呈现左右平分画面，媒体接收方在CLUE媒体配置消息中的新的媒体画面组合选择相关描述如下：After receiving the media screen change information sent by the media provider, the media receiver learns that the network conditions of the video conference have changed, and the media receiver generates updated first media screen composition information based on the media screen change information. A piece of media picture composition information includes: first updated combination information of at least two channels of media pictures and first updated layout information of at least two channels of media pictures, or, first updated combination information of at least two channels of media pictures, and the first updated combination information of at least two channels of media pictures A piece of media picture composition information is sent to the media provider, wherein the first media picture composition information can be carried in the CLUE media configuration message, can also be carried in the SDP request response message, and can also be the updated first media picture composition information by the media receiver. A media screen composition information is sent to the portal website, and the media provider obtains the updated first media screen composition information from the portal website; here, the first media screen composition information is carried in the CLUE media configuration message as an example to illustrate: media The receiver chooses to use 2 channels of video screens (VC1, VC2) to combine into one channel, and the two channels present a left and right split screen. The media receiver's new media screen combination selection in the CLUE media configuration message is described as follows:
Configuration：Configuration:
Capture Scene 1：Capture Scene 1:
Capture Scene Entry 2：VC4Capture Scene Entry 2: VC4
VC4:encoding group=EG1content=view:room.partial,composed=true,VC4:encoding group=EG1content=view:room.partial,composed=true,
composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
S2103：媒体提供方判断是否支持上述第一媒体画面组成信息，若支持，则执行S2104，若不支持，则执行S2107。S2103: The media provider determines whether the above-mentioned first media screen composition information is supported, and if yes, executes S2104, and if not, executes S2107.
S2104：媒体提供方发送第三SDP请求消息给媒体接收方，该第三SDP请求消息中携带组合画面的第一媒体参数。S2104: The media provider sends a third SDP request message to the media receiver, where the third SDP request message carries the first media parameter of the combined picture.
S2105：媒体接收方发送第三SDP响应消息给媒体提供方，该第三SDP响应消息中携带组合画面的第二媒体参数。S2105: The media receiver sends a third SDP response message to the media provider, where the third SDP response message carries the second media parameter of the combined picture.
S2106：媒体提供方发送组合画面给媒体接收方。S2106: The media provider sends the combined picture to the media receiver.
具体的，媒体提供方接收媒体接收方发送的第一媒体画面组成信息，并对第一媒体组成信息中的内容执行判断操作，即判断是否支持至少两路媒体画面的第一更新组合和至少两路媒体画面的第一更新布局，或，至少两路媒体画面的第一更新组合这两种场景中的任一场景，具体为：若媒体提供方支持上述第一媒体画面组成信息，可选的，媒体提供方可以发送组合画面的第一媒体参数给媒体接收方；媒体接收方接收到媒体提供方发送的第一媒体参数之后，会向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，需要注意的，本实施例中也可以不进行组合画面的第一媒体参数和组合画面的第二媒体参数的交互，因为在视频会议召开之前媒体发送方和媒体接收方针对组合画面的第一媒体参数和第二媒体参数已经协商过了；之后，媒体提供方根据上述第一媒体画面组成信息、第一媒体参数和第二媒体参数确定更新后的组合画面，并将更新后组合画面发送给媒体接收方予以呈现。Specifically, the media provider receives the first media screen composition information sent by the media receiver, and performs a judgment operation on the content in the first media composition information, that is, judges whether the first update combination of at least two media screens and at least two media screens are supported. The first update layout of the first media screen, or, the first update of at least two media screens combines any of these two scenarios, specifically: if the media provider supports the above-mentioned first media screen composition information, the optional , the media provider can send the first media parameter of the combined screen to the media receiver; after the media receiver receives the first media parameter sent by the media provider, it will send the second media parameter to the media provider, where the first media The parameter and the second media parameter are the negotiation between the media provider and the media receiver on parameters such as codec mode and transmission bandwidth of the combined screen. It should be noted that the first media parameter and combination of the combined screen may not be performed in this embodiment The interaction of the second media parameter of the picture, because the media sender and the media receiver have negotiated the first media parameter and the second media parameter of the combined picture before the video conference is held; The screen composition information, the first media parameter and the second media parameter determine the updated combined screen, and send the updated combined screen to the media receiver for presentation.
S2107：媒体提供方发送否定应答消息给媒体接收方。S2107: The media provider sends a negative response message to the media receiver.
S2108：媒体接收方根据否定应答消息生成更新后的媒体画面组成信息，发送给媒体提供方。S2108: The media receiver generates updated media screen composition information according to the negative response message, and sends it to the media provider.
S2109：媒体提供方对更新后的媒体画面组成信息执行S2103的判断。S2109: The media provider performs the judgment of S2103 on the updated media screen composition information.
具体的，媒体提供方不支持上述第一媒体画面组成信息，则发送否定应答消息给媒体接收方，向媒体接收方表示不接受该第一媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的第一更新组合信息和至少两路媒体画面的第一更新布局信息，将更新后的媒体画面组成信息再次发送给媒体提供方，媒体提供方接收更新后的媒体画面组成信息，依然对其再次进行上述S2103的判断过程，在此不再赘述。Specifically, if the media provider does not support the composition information of the first media picture, it sends a negative response message to the media receiver, expressing to the media receiver that it does not accept the composition information of the first media picture, and the media receiver receives the negative response After the message, regenerate the new media screen composition information, that is, update the first updated combination information of at least two media screens and the first updated layout information of at least two media screens, and send the updated media screen composition information to the media again The provider, the media provider receives the updated media screen composition information, and still performs the above-mentioned judgment process of S2103 on it again, which will not be repeated here.
本发明实施例提供的方法，在视频会议召开之后，网络发生变化的情况下，通过媒体提供方向媒体接收方发送媒体画面变化信息，以使媒体接收方根据媒体画面变化信息生成更新后的第一媒体画面组成信息，媒体提供方根据第一媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的第一媒体画面组成信息确定的媒体组合画面，同时还使得媒体提供方和媒体接收方可以实时根据网络变化情况对所所组合的媒体画面做出调整，提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度和适用性。In the method provided by the embodiment of the present invention, when the network changes after the video conference is held, the media provider sends the media screen change information to the media receiver, so that the media receiver can generate an updated first video according to the media screen change information. Media screen composition information, the media provider performs corresponding operations according to the first media screen composition information, so that the media combination screen received by the media receiver is the media combination determined by the media provider according to the first media screen composition information sent by the media receiver At the same time, it also enables the media provider and the media receiver to make real-time adjustments to the combined media screen according to network changes, which improves the flexibility of the combined media screen presentation and improves the conference experience and experience of the media receiver user. applicability.
图19为本发明提供的多画面组合的呈现方法实施例十九的信令流程图，本实施例涉及的方法是在会议召开之后，由媒体接收方主动发起组合画面调整的指示，如图19所示，该方法包括：Figure 19 is a signaling flow chart of the nineteenth embodiment of the multi-screen combination presentation method provided by the present invention. The method involved in this embodiment is that after the meeting is held, the media receiver actively initiates an instruction to adjust the combined screen, as shown in Figure 19 As shown, the method includes:
S3100：媒体接收方发送更新后的第二媒体画面组成信息，该第二媒体画面组成信息中包括：至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，或，至少两路媒体画面的第二更新组合信息。S3100: The media receiver sends updated second media screen composition information, the second media screen composition information includes: second updated combination information of at least two media screens and second updated layout information of at least two media screens, Or, the second update combination information of at least two channels of media images.
具体的，媒体接收方自行想更新所呈现的组合画面，则媒体接收方根据自身的能力生成更新后的第二媒体画面组成信息，并将该第二媒体画面组成信息发送给媒体提供方，该第二媒体画面组成信息中包括：至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，或，至少两路媒体画面的第二更新组合信息，其中，该第二媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP响应消息中，还可以是媒体接收方将更新后的第二媒体画面组成信息发送给门户网站，由媒体提供方从门户网站获取该更新后的第二媒体画面组成信息；这里以第二媒体画面组成信息携带在CLUE媒体配置消息中为例：媒体接收方选择采用2路视频画面(VC1,VC2)来组合成一路，二路呈现左右平分画面，媒体接收方在CLUE媒体配置消息中的新媒体画面组合选择相关描述如下：Specifically, if the media receiver wants to update the presented combined screen, the media receiver generates updated second media screen composition information according to its own capabilities, and sends the second media screen composition information to the media provider. The second media screen composition information includes: the second updated combination information of at least two media screens and the second updated layout information of at least two media screens, or the second updated combination information of at least two media screens, wherein the The composition information of the second media picture can be carried in the CLUE media configuration message, and can also be carried in the SDP response message. It can also be that the media receiver sends the updated second media picture composition information to the portal website, and the media provider sends the information from the portal website. The portal website obtains the updated second media screen composition information; here, the second media screen composition information is carried in the CLUE media configuration message as an example: the media receiver chooses to use 2 video screens (VC1, VC2) to combine into one , the second channel presents a left and right split screen, and the media receiver's selection of new media screen combination in the CLUE media configuration message is described as follows:
Configuration:Configuration:
Capture Scene 1：Capture Scene 1:
Capture Scene Entry 2:VC4Capture Scene Entry 2: VC4
VC4：encoding group=EG1,content=view:room.partial,composed=true,VC4: encoding group=EG1, content=view:room.partial, composed=true,
composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),composed layout={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
S3101：媒体提供方接收更新后的第二媒体画面组成信息，判断是否支持上述第二媒体画面组成信息；若支持，执行S3102，若不支持，则执行S3105。S3101: The media provider receives the updated second media screen composition information, and judges whether the above second media screen composition information is supported; if yes, execute S3102; if not, execute S3105.
S3102：媒体提供方发送第三SDP请求消息给媒体接收方，该第三SDP请求消息中携带组合画面的第一媒体参数。S3102: The media provider sends a third SDP request message to the media receiver, where the third SDP request message carries the first media parameter of the combined picture.
S3103：媒体接收方发送第三SDP响应消息给媒体提供方，该第三SDP响应消息中携带组合画面的第二媒体参数。S3103: The media receiver sends a third SDP response message to the media provider, where the third SDP response message carries the second media parameter of the combined picture.
S3104：媒体提供方发送组合画面给媒体接收方。S3104: The media provider sends the combined picture to the media receiver.
具体的，媒体提供方接收媒体接收方发送的第二媒体画面组成信息，判断是否支持上述更新后的第二媒体画面组成信息，具体为：若媒体提供方支持上述至少两路媒体画面的第二更新组合和至少两路媒体画面的第二更新布局，或，支持上述至少两路媒体画面的第二更新组合这两种场景中的任一场景，可选的，媒体提供方可以发送组合画面的第一媒体参数给媒体接收方；媒体接收方接收到媒体提供方发送的第一媒体参数之后，向媒体提供方发送第二媒体参数，这里的第一媒体参数和第二媒体参数是媒体提供方和媒体接收方对组合画面的编解码方式以及传输带宽等参数的协商，需要注意的，本实施例中也可以不进行组合画面的第一媒体参数和组合画面的第二媒体参数的交互，因为在视频会议召开之前媒体发送方和媒体接收方针对组合画面的第一媒体参数和第二媒体参数已经协商过了；之后，媒体提供方根据上述第二媒体画面组成信息、第一媒体参数和第二媒体参数确定更新后的组合画面，并将更新后组合画面发送给媒体接收方予以呈现。Specifically, the media provider receives the second media screen composition information sent by the media receiver, and judges whether to support the above-mentioned updated second media screen composition information, specifically: if the media provider supports the above-mentioned at least two channels of media screen composition information The update combination and the second update layout of at least two media images, or support any of the above two scenarios of the second update combination of at least two media images. Optionally, the media provider may send the combined image The first media parameter is sent to the media receiver; after the media receiver receives the first media parameter sent by the media provider, it sends the second media parameter to the media provider, where the first media parameter and the second media parameter are the media provider Negotiation with the media receiver on parameters such as the codec method and transmission bandwidth of the combined picture, it should be noted that in this embodiment, the interaction between the first media parameter of the combined picture and the second media parameter of the combined picture may not be performed, because Before the video conference is held, the media sender and the media receiver have negotiated for the first media parameter and the second media parameter of the combined picture; after that, the media provider, according to the composition information of the second media picture, the first media parameter and the second media parameter, The second media parameter determines the updated combined picture, and sends the updated combined picture to the media receiver for presentation.
S3105：媒体提供方发送否定应答消息给媒体接收方。S3105: The media provider sends a negative response message to the media receiver.
S3106：媒体接收方根据否定应答消息生成更新后的媒体画面组成信息，发送给媒体提供方。S3106: The media receiver generates updated media screen composition information according to the negative response message, and sends it to the media provider.
S3107：媒体提供方接收更新后的媒体画面组成信息执行S3101的判断。S3107: The media provider receives the updated media screen composition information and executes the judgment of S3101.
具体的，若媒体提供方不支持上述第二媒体画面组成信息，则发送否定应答消息给媒体接收方，向媒体接收方表示不接受该第二媒体画面组成信息，媒体接收方在接收到该否定应答消息之后，重新生成新的媒体画面组成信息，即更新至少两路媒体画面的第二更新组合信息和至少两路媒体画面的第二更新布局信息，将更新后的媒体画面组成信息再次发送给媒体提供方，媒体提供方接收更新后的媒体画面组成信息，依然对其再次进行上述S3101的判断过程，在此不再赘述。Specifically, if the media provider does not support the above-mentioned second media picture composition information, then send a negative response message to the media receiver, expressing to the media receiver that it does not accept the second media picture composition information, and the media receiver receives the negative response message. After answering the message, regenerate the new media picture composition information, promptly update the second updated combination information of at least two paths of media pictures and the second update layout information of at least two paths of media pictures, and send the updated media picture composition information to The media provider receives the updated media screen composition information, and still performs the above-mentioned judgment process of S3101 again, which will not be repeated here.
本发明实施例提供的方法，在视频会议召开之后，媒体提供方自行更新媒体画面，即媒体接收方自行生成更新后的第二媒体画面组成信息，媒体提供方根据第二媒体画面组成信息执行相应的操作，使得媒体接收方接收到的媒体组合画面为媒体提供方根据媒体接收方发送的第二媒体画面组成信息确定的媒体组合画面，提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度和适用性。In the method provided by the embodiment of the present invention, after the video conference is held, the media provider updates the media screen by itself, that is, the media receiver generates the updated second media screen composition information by itself, and the media provider executes the corresponding operation according to the second media screen composition information. The operation makes the media combination picture received by the media receiver be the media combination picture determined by the media provider according to the composition information of the second media picture sent by the media receiver, which improves the flexibility of combining media picture presentation and improves the media receiver’s User's meeting experience and applicability.
图20为本发明提供的多画面组合的呈现方法实施例二十的信令流程图，本实施例涉及的方法是针对多点视频会议中，媒体提供方发送多个默认组合的媒体画面给媒体接收方，以供媒体接收方根据自身的能力选择合适的组合画面，如图20所示，该方法包括：Fig. 20 is a signaling flow chart of Embodiment 20 of the presentation method of multi-picture combination provided by the present invention. The method involved in this embodiment is aimed at sending multiple default combined media screens to the media in a multi-point video conference. The receiving party, for the media receiving party to select a suitable combined picture according to its own ability, as shown in Figure 20, the method includes:
S4100：媒体提供方发送至少一种媒体画面组合选择信息给媒体接收方。S4100: The media provider sends at least one media screen combination selection information to the media receiver.
S4101：媒体接收方根据上述至少一种媒体画面组合选择信息确定媒体画面组成信息。S4101: The media receiver determines media screen composition information according to at least one media screen combination selection information.
具体的，媒体提供方向媒体接收方发送至少一种媒体画面组合选择信息，该媒体画面组合选择信息为媒体提供方自行组合的媒体组合画面，其中，该媒体画面组合选择信息可以携带在CLUE媒体通告消息中，也可以携带在SDP请求消息中，还可以是媒体提供方将至少一种的媒体画面组合选择信息发送给门户网站，由媒体接收方从门户网站获取该至少一种媒体画面组合选择信息，此处以上述媒体画面组合选择信息携带在CLUE媒体配置消息中为例：媒体提供方原来提供3路单独视频画面(VC0,VC1,VC2)，默认的组合方式为：三个单独画面的组合{VC0,VC1,VC2}，两个单独画面的组合{VC0，VC1}，媒体提供方在CLUE媒体通告消息中通知默认的媒体组合画面，具体描述如下：Specifically, the media provider sends at least one media picture combination selection information to the media receiver, and the media picture combination selection information is a media combination picture assembled by the media provider itself, wherein the media picture combination selection information can be carried in the CLUE media announcement In the message, it may also be carried in the SDP request message. It may also be that the media provider sends at least one media screen combination selection information to the portal website, and the media receiver obtains the at least one media screen combination selection information from the portal website. , here is an example where the above media screen combination selection information is carried in the CLUE media configuration message: the media provider originally provided 3 separate video screens (VC0, VC1, VC2), and the default combination method is: a combination of three separate screens{ VC0, VC1, VC2}, the combination of two separate screens {VC0, VC1}, the media provider notifies the default media combination screen in the CLUE media announcement message, the specific description is as follows:
Advertisement：Advertisements:
Capture Scene 1：Capture Scene 1:
Capture Scene Entry1：VC0，VC1，VC2Capture Scene Entry1: VC0, VC1, VC2
Capture Scene Entry2：VC4Capture Scene Entry2: VC4
VC0：encoding group=EG0，content=view:table.left；VC0: encoding group=EG0, content=view:table.left;
VC1：encoding group=EG0，content=view:table.center；VC1: encoding group=EG0, content=view:table.center;
VC2：encoding group=EG0，content=view:table.right；VC2: encoding group=EG0, content=view:table.right;
VC4：encoding group=EG1，content=view:room，composed=true，composedoption={composed layout1，composed layout2}，VC4: encoding group=EG1, content=view:room, composed=true, composedoption={composed layout1, composed layout2},
composed layout1={VC0(0,0,0)(341,0,0)(0,768,0)(512,768,0)，composed layout1={VC0(0,0,0)(341,0,0)(0,768,0)(512,768,0),
VC1(512,384,0)(1024,384,0)(512,768,0)(1024,768,0)，VC1(512,384,0)(1024,384,0)(512,768,0)(1024,768,0),
VC2(512,0,0)(1024,0,0)(512,384,0)(1024,384,0)}，VC2(512,0,0)(1024,0,0)(512,384,0)(1024,384,0)},
Composed layout2={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0)，Composed layout2={VC1(0,0,0)(512,0,0)(0,768,0)(512,768,0),
VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}。VC2(512,0,0)(1024,0,0)(512,768,0)(1024,760,0)}.
媒体接收方接收媒体提供方发送的至少一种媒体画面组合选择信息，并结合媒体接收方的终端显示能力、媒体接收方的本地策略确定媒体画面组成信息，该媒体画面组成信息中包括了媒体接收方的组合画面的选择结果，该媒体画面组成信息可以携带在CLUE媒体配置消息中，也可以携带在SDP响应消息中，还可以是媒体接收方将媒体画面组成信息发送给门户网站，由媒体提供方从门户网站获取媒体接收方的组合画面的选择结果，此处以媒体画面组成信息携带在CLUE媒体配置消息中为例，媒体接收方选择采用3路视频画面(VC0,VC1,VC2)的组合，媒体接收方在CLUE媒体配置消息中的媒体画面组合选择相关描述如下：The media receiver receives at least one media screen combination selection information sent by the media provider, and determines the media screen composition information in combination with the media receiver's terminal display capability and the media receiver's local policy. The media screen composition information includes the media reception The media screen composition information can be carried in the CLUE media configuration message or in the SDP response message. It can also be that the media receiver sends the media screen composition information to the portal website, which is provided by the media. The party obtains the selection result of the combined picture of the media receiver from the portal website. Here, taking the media picture composition information carried in the CLUE media configuration message as an example, the media receiver chooses to use a combination of 3 video pictures (VC0, VC1, VC2). The media receiver's media screen combination selection in the CLUE media configuration message is described as follows:
Configuration:Configuration:
Capture Scene 1:Capture Scene 1:
Capture Scene Entry 2:VC4Capture Scene Entry 2: VC4
VC4:encoding group=EG1,content=view:room.full,composed=true,composedoption={composed layout1}。VC4:encoding group=EG1,content=view:room.full,composed=true,composed option={composed layout1}.
媒体提供方获知媒体接收方的组合画面选择结果之后，向媒体接收方发送组合画面的第一媒体参数，该第一媒体参数可以携带在第三SDP响应消息中，还可以是媒体提供方将该组合画面的第一媒体参数发送给门户网站，由媒体接收方从门户网站获取；媒体接收方接收到组合画面的第一媒体参数之后，向媒体提供方发送组合画面的第二媒体参数，该第二媒体参数可以携带在第三SDP响应消息中，还可以是媒体接收方将组合画面的第二媒体参数发送给门户网站，由媒体提供方从门户网站获取该组合画面的第二媒体参数，需要注意的是，媒体提供方和媒体接收方之间进行组合画面的第一媒体参数和第二媒体参数的交互主要是进行组合画面的编解码方式和带宽等参数的协商；之后，媒体提供方将媒体接收方选择的组合画面发送给媒体接收方予以呈现。After the media provider learns the selection result of the combined picture of the media receiver, it sends the first media parameter of the combined picture to the media receiver. The first media parameter can be carried in the third SDP response message, or the media provider can send the combined picture The first media parameter of the combined picture is sent to the portal website, and the media receiver obtains it from the portal website; after the media receiver receives the first media parameter of the combined picture, it sends the second media parameter of the combined picture to the media provider. The second media parameter can be carried in the third SDP response message, and the media receiver can also send the second media parameter of the combined picture to the portal website, and the media provider can obtain the second media parameter of the combined picture from the portal website. It should be noted that the interaction between the first media parameter and the second media parameter of the combined picture between the media provider and the media receiver is mainly to negotiate parameters such as codec mode and bandwidth of the combined picture; after that, the media provider will The combined picture selected by the media receiver is sent to the media receiver for presentation.
本实施例提供的方法中，通过媒体提供方将至少一种媒体画面组合选择信息发送给媒体接收方，媒体接收方根据其终端的显示能力以及本地策略媒体画面组合选择信息进行选择，并将选择结果携带在媒体画面组成信息中发送给媒体提供方，以使媒体提供方将对应的组合画面发送给媒体接收方，即媒体接收方是按照自身的意愿或者能力对组合画面进行选择和呈现，因此提高了组合媒体画面呈现的灵活性，提升了媒体接收方用户的会议体验度。In the method provided in this embodiment, the media provider sends at least one media screen combination selection information to the media receiver, and the media receiver selects according to the display capability of its terminal and the local policy media screen combination selection information, and the selected The result is carried in the media screen composition information and sent to the media provider, so that the media provider can send the corresponding combined screen to the media receiver, that is, the media receiver selects and presents the combined screen according to its own wishes or capabilities, so It improves the flexibility of combining media screen presentation, and improves the conference experience of media receiver users.
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.
图21为本发明提供的多画面组合的装置实施例一的结构示意图，如图21所示，该装置包括：第一发送模块10，用于发送媒体画面组合能力指示信息给媒体接收方；第一接收模块11，用于接收所述媒体接收方发送的根据所述媒体画面组合能力指示信息确定的媒体画面组成信息；处理模块12，用于根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Fig. 21 is a schematic structural diagram of Embodiment 1 of a multi-picture combination device provided by the present invention. As shown in Fig. 21 , the device includes: a first sending module 10, configured to send media picture combination capability indication information to a media receiver; A receiving module 11, used to receive the media picture composition information determined according to the media picture combination capability indication information sent by the media receiver; processing module 12, used to send to the media picture according to the support capability of the media picture composition information The media receiver performs corresponding operations.
本实施例的多画面组合的装置可以执行图1所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 1 , and the implementation principles are similar, so details are not repeated here.
图22为本发明提供的多画面组合的装置实施例二的结构示意图，在上述图21所示实施例的基础上，上述处理模块12，包括：第一处理单元120，用于若支持所述媒体画面组成信息，则发送组合画面的第一媒体参数给所述媒体接收方；并接收所述媒体接收方发送的所述组合画面的第二媒体参数；并将根据所述媒体画面组成信息、所述第一媒体参数和所述第二媒体参数确定的所述组合画面发送给所述媒体接收方；或者，第二处理单元121，用于若不支持所述媒体画面组成信息，则发送否定应答消息给所述媒体接收方。Fig. 22 is a schematic structural diagram of Embodiment 2 of the multi-screen combination device provided by the present invention. On the basis of the embodiment shown in Fig. 21 above, the processing module 12 includes: a first processing unit 120 configured to support the media picture composition information, then send the first media parameter of the combined picture to the media receiver; and receive the second media parameter of the combined picture sent by the media receiver; and according to the media picture composition information, The combined picture determined by the first media parameter and the second media parameter is sent to the media receiver; or, the second processing unit 121 is configured to send a negative if the media picture composition information is not supported Reply message to the media receiver.
在上述图22所示实施例的基础上，进一步地，上述第一发送模块10，包括：第一发送单元101，用于发送第一远程呈现的多流控制CLUE媒体通告消息给所述媒体接收方，所述第一CLUE媒体通告消息中携带所述媒体画面组合能力指示信息；则上述第一接收模块11包括：第一接收单元111，用于接收所述媒体接收方发送的第一CLUE媒体配置消息，所述第一CLUE媒体配置消息中携带所述媒体画面组成信息。On the basis of the above-mentioned embodiment shown in FIG. 22 , further, the above-mentioned first sending module 10 includes: a first sending unit 101, configured to send a first telepresence multi-stream control CLUE media notification message to the media receiver party, the first CLUE media notification message carries the media screen combination capability indication information; then the above-mentioned first receiving module 11 includes: a first receiving unit 111, configured to receive the first CLUE media sent by the media receiver A configuration message, where the first CLUE media configuration message carries the media screen composition information.
本实施例的多画面组合的装置可以执行图2所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 2 , and the implementation principles are similar, so details are not repeated here.
继续参照图22，进一步地，在上述图22所示实施例的基础上，上述第一发送模块10包括：第二发送单元102，用于发送第一会话描述协议SDP请求消息给所述媒体接收方，所述第一SDP请求消息中携带所述媒体画面组合能力指示信息；则上述第一接收模块11，包括：第二接收单元112，用于接收所述媒体接收方发送的第一SDP响应消息，所述第一SDP响应消息中携带所述媒体画面组成信息。Continuing to refer to FIG. 22 , further, on the basis of the above-mentioned embodiment shown in FIG. 22 , the above-mentioned first sending module 10 includes: a second sending unit 102, configured to send a first session description protocol SDP request message to the media receiver party, the first SDP request message carries the media picture combination capability indication information; then the above-mentioned first receiving module 11 includes: a second receiving unit 112, configured to receive the first SDP response sent by the media receiver message, the first SDP response message carries the media picture composition information.
本实施例的多画面组合的装置可以执行图3所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 3 , and the implementation principles are similar, so details are not repeated here.
继续参照图22，进一步地，在上述图22所示实施例的基础上，上述第一发送模块10包括：第三发送单元103，用于发送媒体画面组合能力指示信息以及媒体画面的缩略图给门户网站，以使所述媒体接收方从所述门户网站获取所述媒体画面组合能力指示信息和所述媒体画面的缩略图；则所述第一接收模块11，包括：第三接收单元113，用于从所述门户网站获取所述媒体接收方发送的根据所述媒体画面组合能力指示信息和所述媒体画面的缩略图确定的所述媒体画面组成信息。Continuing to refer to FIG. 22 , further, on the basis of the above-mentioned embodiment shown in FIG. 22 , the above-mentioned first sending module 10 includes: a third sending unit 103, configured to send media screen combination capability indication information and media screen thumbnails to A portal website, so that the media receiver obtains the media screen combination capability indication information and the thumbnail of the media screen from the portal website; then the first receiving module 11 includes: a third receiving unit 113, The method is used to obtain, from the portal website, the media screen composition information sent by the media receiver and determined according to the media screen combination capability indication information and the thumbnail of the media screen.
本实施例的多画面组合的装置可以执行图4所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 4 , and the implementation principles are similar, so details are not repeated here.
图23为本发明提供的多画面组合的装置实施例三的结构示意图，在上述图22所示实施例的基础上，该装置还包括：第二发送模块13，用于在发送远程呈现的多流控制CLUE媒体通告消息给所述媒体接收方之前，发送第二SDP请求消息给所述媒体接收方，所述第二SDP请求消息携带支持CLUE协议指示；第二接收模块14，用于接收所述媒体接收方发送的第二SDP响应消息，所述第二SDP响应消息携带支持所述CLUE协议指示。Fig. 23 is a schematic structural diagram of the third embodiment of the multi-picture combination device provided by the present invention. On the basis of the above-mentioned embodiment shown in Fig. 22, the device further includes: a second sending module 13, which is used to send the telepresence multi-screen Before the flow control CLUE media announcement message is sent to the media receiver, a second SDP request message is sent to the media receiver, and the second SDP request message carries an indication of supporting the CLUE protocol; the second receiving module 14 is configured to receive the A second SDP response message sent by the media receiver, where the second SDP response message carries an indication of supporting the CLUE protocol.
本实施例的多画面组合的装置可以执行图2所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 2 , and the implementation principles are similar, so details are not repeated here.
继续参照图23，在图23所示实施例的基础上，上述第一处理单元120包括：第一发送子单元1201，用于发送第三SDP请求消息给所述媒体接收方，所述第三SDP请求消息中携带所述组合画面的第一媒体参数；第一接收子单元1202，用于接收所述媒体接收方发送的第三SDP响应消息，所述第三SDP响应消息中携带所述组合画面的第二媒体参数。Continuing to refer to FIG. 23 , on the basis of the embodiment shown in FIG. 23 , the first processing unit 120 includes: a first sending subunit 1201 configured to send a third SDP request message to the media receiver, and the third The SDP request message carries the first media parameter of the combined picture; the first receiving subunit 1202 is configured to receive the third SDP response message sent by the media receiver, and the third SDP response message carries the combination The second media parameter of the screen.
本实施例的多画面组合的装置可以执行上述方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the above-mentioned method embodiment, and the implementation principles are similar, so details are not repeated here.
继续参照图23，在图23所示实施例的基础上，上述第一处理单元120还包括：第二发送子单元1203，用于发送所述组合画面的第一媒体参数给门户网站，以使所述媒体接收方从所述门户网站获取所述组合画面的所述第一媒体参数；第二接收子单元1204，用于从门户网站获取所述媒体接收方发送的所述组合画面的第二媒体参数。Continuing to refer to FIG. 23 , on the basis of the embodiment shown in FIG. 23 , the above-mentioned first processing unit 120 further includes: a second sending subunit 1203, configured to send the first media parameters of the combined screen to the portal website, so that The media receiver acquires the first media parameter of the combined picture from the portal website; the second receiving subunit 1204 is configured to acquire the second parameter of the combined picture sent by the media receiver from the portal website. media parameters.
本实施例的多画面组合的装置可以执行上述方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the above-mentioned method embodiment, and the implementation principles are similar, so details are not repeated here.
继续参照图23，在图23所示实施例的基础上，该装置还包括：第三发送模块15，用于在根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，发送媒体画面变化信息给所述媒体接收方；第三接收模块16，用于接收所述媒体接收方根据所述媒体画面变化信息生成的更新后的第一媒体画面组成信息，所述第一媒体画面组成信息中包括：所述至少两路媒体画面的第一更新组合信息和所述至少两路媒体画面的第一更新布局信息，或，所述至少两路媒体画面的第一更新组合信息；则所述处理模块12还用于根据对所述第一媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Continuing to refer to FIG. 23 , on the basis of the embodiment shown in FIG. 23 , the device further includes: a third sending module 15 , configured to send a corresponding message to the media receiver according to the support capability of the media picture composition information. After the operation, send the media screen change information to the media receiver; the third receiving module 16 is used to receive the updated first media screen composition information generated by the media receiver according to the media screen change information, so The composition information of the first media screen includes: the first updated combination information of the at least two media screens and the first updated layout information of the at least two media screens, or, the first updated layout information of the at least two media screens update the combination information; then the processing module 12 is further configured to perform corresponding operations to the media receiver according to the support capability of the composition information of the first media picture.
本实施例的多画面组合的装置可以执行图5所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 5 , and the implementation principles are similar, so details are not repeated here.
继续参照图23，在图23所示实施例的基础上，该装置还包括：第四接收模块17，用于在根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，接收所述媒体接收方更新后的第二媒体画面组成信息，所述第二媒体画面组成信息中包括：所述至少两路媒体画面的第二更新组合信息和所述至少两路媒体画面的第二更新布局信息，或，所述至少两路媒体画面的第二更新组合信息；则所述处理模块12还用于根据对所述第二媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Continuing to refer to FIG. 23, on the basis of the embodiment shown in FIG. 23, the device further includes: a fourth receiving module 17, configured to perform corresponding After the operation, the second media screen composition information updated by the media receiver is received, and the second media screen composition information includes: the second updated combination information of the at least two media screens and the at least two media screens The second update layout information of the media picture, or, the second update combination information of the at least two media pictures; then the processing module 12 is also configured to send to all The above media receiver performs corresponding operations.
本实施例的多画面组合的装置可以执行图6所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 6 , and the implementation principles are similar, so details are not repeated here.
进一步地，在图23所示实施例的基础上，上述第一发送模块10，还用于发送至少一种媒体画面组合选择信息给所述媒体接收方，以使所述媒体接收方根据所述至少一种媒体画面组合选择信息确定媒体画面组成信息。Further, on the basis of the embodiment shown in FIG. 23 , the above-mentioned first sending module 10 is also configured to send at least one media screen combination selection information to the media recipient, so that the media recipient At least one media picture combination selection information determines media picture composition information.
本实施例的多画面组合的装置可以执行图7所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 7 , and the implementation principles are similar, so details are not repeated here.
图24为本发明提供的多画面组合的装置实施例四的结构示意图，如图24所示，该装置包括：第一接收模块20，用于接收媒体提供方发送的媒体画面组合能力指示信息；确定模块21，用于根据所述媒体画面组合能力指示信息确定媒体画面组成信息；第一发送模块22，用于将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。FIG. 24 is a schematic structural diagram of Embodiment 4 of a multi-picture combination device provided by the present invention. As shown in FIG. 24 , the device includes: a first receiving module 20, configured to receive media picture combination capability indication information sent by a media provider; The determining module 21 is used to determine the media picture composition information according to the media picture combination capability indication information; the first sending module 22 is used to send the media picture composition information to the media provider, so that the media provider The party performs corresponding operations to the media receiver according to the support capability of the media screen composition information.
本实施例的多画面组合的装置可以执行图8所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 8 , and the implementation principles are similar, so details are not repeated here.
图25为本发明提供的多画面组合的装置实施例五的结构示意图，在上述图24所示实施例的基础上，上述第一发送模块22，包括：第一处理单元220，用于若所述媒体提供方支持所述媒体画面组成信息，则接收所述媒体提供方发送的组合画面的第一媒体参数；并发送所述组合画面的第二媒体参数给所述媒体提供方；并接收所述媒体提供方根据所述媒体画面组成信息、所述第一媒体参数和所述第二媒体参数确定的所述组合画面；或者，第二处理单元221，用于若所述媒体提供方不支持所述媒体画面组成信息，则接收所述媒体提供方发送的否定应答消息。Fig. 25 is a schematic structural diagram of Embodiment 5 of the multi-picture combination device provided by the present invention. On the basis of the embodiment shown in Fig. 24 above, the above-mentioned first sending module 22 includes: a first processing unit 220 configured to If the media provider supports the composition information of the media screen, then receive the first media parameter of the combined screen sent by the media provider; and send the second media parameter of the combined screen to the media provider; and receive the The combined screen determined by the media provider according to the media screen composition information, the first media parameter, and the second media parameter; or, the second processing unit 221 is configured to, if the media provider does not support The media picture composition information receives a negative response message sent by the media provider.
在上述图25所示实施例的基础上，进一步地，上述第一接收模块20，包括：第一接收单元201，用于接收所述媒体提供方发送的第一远程呈现的多流控制CLUE媒体通告消息，所述第一CLUE媒体通告消息中携带所述媒体画面组合能力指示信息；则所述第一发送模块22，包括：第一发送单元222，用于发送第一CLUE媒体配置消息给所述媒体提供方，所述第一CLUE媒体配置消息中携带所述媒体画面组成信息。On the basis of the above-mentioned embodiment shown in FIG. 25 , further, the above-mentioned first receiving module 20 includes: a first receiving unit 201, configured to receive the first telepresence multi-stream control CLUE media sent by the media provider Notification message, the first CLUE media notification message carries the media picture combination capability indication information; then the first sending module 22 includes: a first sending unit 222, configured to send the first CLUE media configuration message to the For the media provider, the first CLUE media configuration message carries the media screen composition information.
本实施例的多画面组合的装置可以执行图9所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 9 , and the implementation principles are similar, so details are not repeated here.
继续参照图25，在图25所示实施例的基础上，进一步地，上述第一接收模块20，包括：第二接收单元202，用于接收所述媒体提供方发送的第一会话描述协议SDP请求消息，所述第一SDP请求消息中携带所述媒体画面组合能力指示信息；则所述第一发送模块22，包括：第二发送单元223，用于发送第一SDP响应消息给所述媒体提供方，所述第一SDP响应消息中携带所述媒体画面组成信息。Continuing to refer to FIG. 25 , on the basis of the embodiment shown in FIG. 25 , further, the above-mentioned first receiving module 20 includes: a second receiving unit 202 configured to receive the first session description protocol SDP sent by the media provider request message, the first SDP request message carries the media picture combination capability indication information; then the first sending module 22 includes: a second sending unit 223, configured to send the first SDP response message to the media For the provider, the first SDP response message carries the media picture composition information.
本实施例的多画面组合的装置可以执行图10所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 10 , and the implementation principles are similar, so details are not repeated here.
继续参照图25，在图25所示实施例的基础上，进一步地，上述第一接收模块20，包括：第三接收单元203，用于从门户网站获取所述媒体提供方发送的所述媒体画面组合能力指示信息以及媒体画面的缩略图；则所述第一发送模块22，包括：第三发送单元224，用于根据所述媒体画面组合能力指示信息和所述媒体画面的缩略图确定所述媒体画面组成信息，并将所述媒体画面组成信息发送给所述门户网站，以使所述媒体提供方从所述门户网站获取所述媒体画面组成信息。Continuing to refer to FIG. 25 , on the basis of the embodiment shown in FIG. 25 , further, the above-mentioned first receiving module 20 includes: a third receiving unit 203, configured to acquire the media sent by the media provider from a portal website Screen combination capability indication information and a thumbnail of a media picture; then the first sending module 22 includes: a third sending unit 224, configured to determine the media picture combination capability indication information and the thumbnail of the media picture according to the the media screen composition information, and send the media screen composition information to the portal website, so that the media provider obtains the media screen composition information from the portal website.
本实施例的多画面组合的装置可以执行图11所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 11 , and the implementation principles are similar, so details are not repeated here.
图26为本发明提供的多画面组合的装置实施例六的结构示意图，在上述图25所示实施例的基础上，该装置还包括：第二接收模块23，用于在接收所述媒体提供方发送的第一远程呈现的多流控制CLUE媒体通告消息之前，接收所述媒体提供方发送的第二SDP请求消息，所述第二SDP请求消息携带支持CLUE协议指示；第二发送模块24，用于发送第二SDP响应消息给所述媒体提供方，所述第二SDP响应消息中携带支持所述CLUE协议指示。Fig. 26 is a schematic structural diagram of Embodiment 6 of the multi-picture combination device provided by the present invention. On the basis of the embodiment shown in Fig. 25 above, the device further includes: a second receiving module 23 for receiving the media provided Before the multi-stream control CLUE media notification message of the first telepresence sent by the party, the second SDP request message sent by the media provider is received, and the second SDP request message carries an indication of supporting the CLUE protocol; the second sending module 24, It is used to send a second SDP response message to the media provider, where the second SDP response message carries an indication of supporting the CLUE protocol.
本实施例的多画面组合的装置可以执行图9所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 9 , and the implementation principles are similar, so details are not repeated here.
继续参照图26，在图26所示实施例的基础上，进一步地，上述第一处理单元220，包括：第一接收子单元2201，用于接收所述媒体提供方发送的第三SDP请求消息，所述第三SDP请求消息中携带所述组合画面的第一媒体参数；第一发送子单元2202，用于发送第三SDP响应消息给所述媒体提供方，所述第三SDP响应消息中携带所述组合画面的第二媒体参数。Continuing to refer to FIG. 26 , on the basis of the embodiment shown in FIG. 26 , further, the above-mentioned first processing unit 220 includes: a first receiving subunit 2201 configured to receive the third SDP request message sent by the media provider , the third SDP request message carries the first media parameter of the combined picture; the first sending subunit 2202 is configured to send a third SDP response message to the media provider, and the third SDP response message includes Carry the second media parameter of the combined picture.
本实施例的多画面组合的装置可以执行上述方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the above-mentioned method embodiment, and the implementation principles are similar, so details are not repeated here.
继续参照图26，在图26所示实施例的基础上，进一步地，上述第一处理单元220，还包括：第二接收子单元2203，用于从门户网站获取所述媒体提供方发送的所述组合画面的第一媒体参数；第二发送子单元2204，用于将所述组合画面的第二媒体参数发送至所述门户网站，以使所述媒体提供方从所述门户网站获取所述组合画面的第二媒体参数。Continuing to refer to FIG. 26 , on the basis of the embodiment shown in FIG. 26 , further, the above-mentioned first processing unit 220 further includes: a second receiving subunit 2203 , configured to obtain from the portal website the information sent by the media provider. The first media parameter of the combined screen; the second sending subunit 2204, configured to send the second media parameter of the combined screen to the portal website, so that the media provider can obtain the The second media parameter of the composite screen.
本实施例的多画面组合的装置可以执行上述方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the above-mentioned method embodiment, and the implementation principles are similar, so details are not repeated here.
继续参照图26，在图26所示实施例的基础上，进一步地，该装置还包括：第三接收模块25，用于在将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，接收所述媒体提供方发送的媒体画面变化信息；生成模块26，用于根据所述媒体画面变化信息生成更新后的第一媒体画面组成信息，所述第一媒体画面组成信息中包括：所述至少两路媒体画面的第一更新组合信息和所述至少两路媒体画面的第一更新布局信息，或，所述至少两路媒体画面的第一更新组合信息；第三发送模块27，用于将所述第一媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述第一媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Continuing to refer to FIG. 26, on the basis of the embodiment shown in FIG. 26, the device further includes: a third receiving module 25, configured to send the media screen composition information to the media provider, so that The media provider performs corresponding operations to the media receiver according to the support capability of the media screen composition information, and then receives the media screen change information sent by the media provider; the generating module 26 is configured to The media screen change information generates updated first media screen composition information, and the first media screen composition information includes: the first update combination information of the at least two media screens and the first updated combination information of the at least two media screens An updated layout information, or, the first updated combination information of the at least two media screens; the third sending module 27 is configured to send the composition information of the first media screen to the media provider, so that the The media provider performs corresponding operations to the media receiver according to the support capability of the composition information of the first media screen.
本实施例的多画面组合的装置可以执行图12所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 12 , and the implementation principles are similar, so details are not repeated here.
继续参照图26，在图26所示实施例的基础上，进一步地，该装置还包括：第四发送模块28，用于在将所述媒体画面组成信息发送给所述媒体提供方，以使所述媒体提供方根据对所述媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作之后，发送更新后的第二媒体画面组成信息给所述媒体提供方，所述第二媒体画面组成信息中包括：所述至少两路媒体画面的第二更新组合信息和所述至少两路媒体画面的第二更新布局信息，或，所述至少两路媒体画面的第二更新组合信息，以使所述媒体提供方根据对所述第二媒体画面组成信息的支持能力，向所述媒体接收方执行相应的操作。Continuing to refer to FIG. 26, on the basis of the embodiment shown in FIG. 26, further, the device further includes: a fourth sending module 28, configured to send the media screen composition information to the media provider, so that The media provider sends the updated second media screen composition information to the media provider after performing corresponding operations to the media receiver according to the support capability of the media screen composition information, and the second The media picture composition information includes: the second updated combination information of the at least two media pictures and the second updated layout information of the at least two media pictures, or, the second updated combination information of the at least two media pictures , so that the media provider performs corresponding operations on the media receiver according to the support capability of the composition information of the second media screen.
本实施例的多画面组合的装置可以执行图13所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 13 , and the implementation principles are similar, so details are not repeated here.
继续参照图26，在图26所示实施例的基础上，进一步地，上述第一接收模块20还用于接收所述媒体提供方发送的至少一种媒体画面组合选择信息；则所述确定模块21具体用于根据所述至少一种媒体画面组合选择信息确定媒体画面组成信息。Continuing to refer to FIG. 26 , on the basis of the embodiment shown in FIG. 26 , further, the above-mentioned first receiving module 20 is further configured to receive at least one media screen combination selection information sent by the media provider; then the determining module 21 is specifically configured to determine media picture composition information according to the at least one media picture combination selection information.
本实施例的多画面组合的装置可以执行图14所示的方法实施例，实现原理相类似，在此不再赘述。The device for multi-picture combination in this embodiment can execute the method embodiment shown in FIG. 14 , and the implementation principles are similar, so details are not repeated here.
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310279136.6ACN104253969B (en) | 2013-06-28 | 2013-06-28 | The rendering method and device of more picture compositions |
| PCT/CN2014/077796WO2014206155A1 (en) | 2013-06-28 | 2014-05-19 | Method and apparatus for presenting multi-picture combination |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310279136.6ACN104253969B (en) | 2013-06-28 | 2013-06-28 | The rendering method and device of more picture compositions |
| Publication Number | Publication Date |
|---|---|
| CN104253969A CN104253969A (en) | 2014-12-31 |
| CN104253969Btrue CN104253969B (en) | 2017-12-05 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310279136.6AExpired - Fee RelatedCN104253969B (en) | 2013-06-28 | 2013-06-28 | The rendering method and device of more picture compositions |
| Country | Link |
|---|---|
| CN (1) | CN104253969B (en) |
| WO (1) | WO2014206155A1 (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019070998A (en)* | 2017-10-11 | 2019-05-09 | アズビル株式会社 | Field equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1571508A (en)* | 2003-07-19 | 2005-01-26 | 华为技术有限公司 | A method for implementing multi-frame |
| CN1620133A (en)* | 2003-11-21 | 2005-05-25 | 华为技术有限公司 | The Method of Realizing Single-screen and Multi-screen Switching in Video Conference System |
| CN101031065A (en)* | 2007-04-27 | 2007-09-05 | 华为技术有限公司 | Method, apparatus and system for switching pictures in video service |
| JP2007272620A (en)* | 2006-03-31 | 2007-10-18 | Fujitsu Ltd | File evaluation apparatus, file evaluation method, and file evaluation program |
| CN102082944A (en)* | 2009-11-30 | 2011-06-01 | 华为终端有限公司 | Conference control method, device and system containing tele-presence conference hall |
| CN102221959A (en)* | 2011-03-30 | 2011-10-19 | 合一网络技术(北京)有限公司 | Multi-picture display and control method and system for embedded media player, and application |
| CN102843542A (en)* | 2012-09-07 | 2012-12-26 | 华为技术有限公司 | Method, equipment and system of media negotiation of multithread meeting |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101262587B (en)* | 2008-03-31 | 2011-04-20 | 杭州华三通信技术有限公司 | A method and multi-point control units for realizing multi-image video conference |
| CN102118602B (en)* | 2011-03-15 | 2013-08-21 | 深圳市捷视飞通科技有限公司 | Method and system for displaying auxiliary streaming video in multiple pictures |
| CN102724433A (en)* | 2012-06-18 | 2012-10-10 | 深圳锐取信息技术股份有限公司 | Method and device for realizing multi-video signal image composition |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1571508A (en)* | 2003-07-19 | 2005-01-26 | 华为技术有限公司 | A method for implementing multi-frame |
| CN1620133A (en)* | 2003-11-21 | 2005-05-25 | 华为技术有限公司 | The Method of Realizing Single-screen and Multi-screen Switching in Video Conference System |
| JP2007272620A (en)* | 2006-03-31 | 2007-10-18 | Fujitsu Ltd | File evaluation apparatus, file evaluation method, and file evaluation program |
| CN101031065A (en)* | 2007-04-27 | 2007-09-05 | 华为技术有限公司 | Method, apparatus and system for switching pictures in video service |
| CN102082944A (en)* | 2009-11-30 | 2011-06-01 | 华为终端有限公司 | Conference control method, device and system containing tele-presence conference hall |
| CN102221959A (en)* | 2011-03-30 | 2011-10-19 | 合一网络技术(北京)有限公司 | Multi-picture display and control method and system for embedded media player, and application |
| CN102843542A (en)* | 2012-09-07 | 2012-12-26 | 华为技术有限公司 | Method, equipment and system of media negotiation of multithread meeting |
| Publication number | Publication date |
|---|---|
| CN104253969A (en) | 2014-12-31 |
| WO2014206155A1 (en) | 2014-12-31 |
| Publication | Publication Date | Title |
|---|---|---|
| AU2016213760B2 (en) | Media negotiation method, device, and system for multi-stream conference | |
| US10015440B2 (en) | Multiple channel communication using multiple cameras | |
| US9154737B2 (en) | User-defined content magnification and multi-point video conference system, method and logic | |
| AU2011258272B2 (en) | Systems and methods for scalable video communication using multiple cameras and multiple monitors | |
| US8379075B2 (en) | Method, device, and computer-readable medium for processing images during video conferencing | |
| CN103686219B (en) | A kind of method, equipment and the system of video conference recorded broadcast | |
| CN103051865B (en) | The method that picture controls and terminal, video conference device | |
| EP2437491A1 (en) | Method and system for video conference control, network equipment and meeting places for video conference | |
| US9497390B2 (en) | Video processing method, apparatus, and system | |
| CN105532008A (en) | User Adaptive Video Telephony | |
| WO2011140812A1 (en) | Multi-picture synthesis method and system, and media processing device | |
| WO2011063763A1 (en) | Method, device and system for conference control including remote display conference places | |
| WO2012075937A1 (en) | Video call method and videophone | |
| WO2015003532A1 (en) | Multimedia conferencing establishment method, device and system | |
| CN103281509B (en) | A kind of video conference optimizes display packing and device | |
| CN104253969B (en) | The rendering method and device of more picture compositions | |
| TW201220847A (en) | Video call system, main-side terminal, and subordinate-side terminals | |
| TW201332359A (en) | Video-conference system and its associated method | |
| KR20130132154A (en) | System and method for telepresence using multiple devices |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20171205 |