技术领域Technical Field
本申请涉及计算机技术领域，特别是涉及一种投屏方法及装置。The present application relates to the field of computer technology, and in particular to a screen projection method and device.
背景技术Background Art
在投屏场景中，用户可以将共享设备的屏幕上显示的内容投屏至投屏设备的屏幕上显示。其中，将共享设备的屏幕上显示的内容投屏至投屏设备的屏幕上显示时，可以将共享设备的屏幕上显示的内容作为视频流传输给投屏设备，由投屏设备渲染接收到的视频流。In the screen projection scenario, the user can project the content displayed on the screen of the shared device to the screen of the projection device. When the content displayed on the screen of the shared device is projected to the screen of the projection device, the content displayed on the screen of the shared device can be transmitted to the projection device as a video stream, and the projection device renders the received video stream.
目前，在投屏场景中，投屏设备的屏幕上显示的鼠标存在动作不流畅的问题，例如，出现卡顿等现象。Currently, in screen projection scenarios, the mouse displayed on the screen of the projection device has problems with unsmooth movements, for example, freezing and other phenomena.
因此，急需一种方案，能够提升所述投屏设备的屏幕上显示的鼠标的动作流畅性。Therefore, there is an urgent need for a solution that can improve the smoothness of the mouse movements displayed on the screen of the projection device.
发明内容Summary of the invention
本申请所要解决的技术问题是：如何提升所述投屏设备的屏幕上显示的鼠标的动作流畅性，提供一种投屏方法及装置。The technical problem to be solved by the present application is: how to improve the smoothness of the mouse movements displayed on the screen of the screen projection device, and to provide a screen projection method and device.
第一方面，本申请实施例提供了一种投屏方法，应用于共享设备，所述方法包括：In a first aspect, an embodiment of the present application provides a screen projection method, which is applied to a shared device, and the method includes:
以第一帧率采集屏幕流数据；Collecting screen stream data at a first frame rate;
以第二帧率采集鼠标数据；collecting mouse data at a second frame rate;
分别将所述屏幕流数据和所述鼠标数据发送至投屏设备，以使所述投屏设备基于所述屏幕流数据和所述鼠标数据渲染投屏界面。The screen stream data and the mouse data are respectively sent to the screen projection device, so that the screen projection device renders the screen projection interface based on the screen stream data and the mouse data.
可选的，所述方法还包括：Optionally, the method further includes:
响应于用户触发的投屏操作，向所述投屏设备发送投屏请求；In response to a screen projection operation triggered by a user, sending a screen projection request to the screen projection device;
接收所述投屏设备针对所述投屏请求发送的指示信息，所述指示信息用于指示所述投屏设备支持单独接收鼠标数据。Receive indication information sent by the screen projection device in response to the screen projection request, where the indication information is used to indicate that the screen projection device supports separate reception of mouse data.
可选的，所述共享设备的屏幕上显示的内容投屏至多个投屏设备上显示，所述多个投屏设备均支持单独接收鼠标数据。Optionally, the content displayed on the screen of the shared device is projected onto multiple projection devices for display, and the multiple projection devices all support separate reception of mouse data.
可选的，所述采集鼠标数据，包括：Optionally, collecting mouse data includes:
响应于检测到所述共享设备的屏幕上的鼠标的特征发生改变，获取所述鼠标数据，所述鼠标的特征，包括：鼠标位置和/或鼠标样式。In response to detecting that a feature of a mouse on the screen of the shared device changes, the mouse data is acquired, wherein the feature of the mouse includes: a mouse position and/or a mouse style.
可选的，在所述鼠标的位置发生改变、且所述鼠标的样式未发生改变的情况下，所述鼠标的特征包括所述鼠标位置且不包括所述鼠标样式。Optionally, when the position of the mouse changes and the style of the mouse does not change, the characteristics of the mouse include the mouse position but do not include the mouse style.
可选的，在所述鼠标的位置发生改变、且所述鼠标的样式发生改变的情况下，所述鼠标的特征包括所述鼠标位置和所述鼠标样式。Optionally, when the position of the mouse changes and the style of the mouse changes, the characteristics of the mouse include the mouse position and the mouse style.
可选的，在所述鼠标的位置未发生改变、且所述鼠标的样式发生改变的情况下，所述鼠标的特征包括所述鼠标样式且不包括所述鼠标位置。Optionally, when the position of the mouse does not change and the style of the mouse changes, the characteristics of the mouse include the mouse style but do not include the mouse position.
可选的，所述方法还包括：Optionally, the method further includes:
接收所述投屏设备针对接收到的所述鼠标数据发送的第一应答；Receiving a first response sent by the screen projection device in response to the received mouse data;
基于发送信息和应答信息，确定所述第二帧率，所述发送信息和所述接收信息用于指示所述共享设备向所述投屏设备发送鼠标数据的时延，或者，所述发送信息和所述接收信息用于指示共享设备向所述投屏设备发送鼠标数据的丢包情况。Based on the sending information and the response information, the second frame rate is determined, and the sending information and the receiving information are used to indicate the delay of the shared device sending mouse data to the screen projection device, or the sending information and the receiving information are used to indicate the packet loss situation of the shared device sending mouse data to the screen projection device.
可选的，Optional,
所述发送信息包括：所述共享设备向所述投屏设备发送所述鼠标数据的第一时间戳；The sending information includes: the sharing device sending a first timestamp of the mouse data to the screen projection device;
所述应答信息包括：所述共享设备接收到所述第一应答的第二时间戳。The response information includes: a second timestamp when the sharing device receives the first response.
可选的，Optional,
所述发送信息包括：所述共享设备向所述投屏设备发送的鼠标数据的数量；The sending information includes: the amount of mouse data sent by the sharing device to the screen projection device;
所述应答信息包括：所述共享设备接收到所述投屏设备发送的应答的数量。The response information includes: the number of responses sent by the screen projection device and received by the sharing device.
可选的，所述共享设备支持单独发送鼠标数据。Optionally, the shared device supports sending mouse data separately.
第二方面，本申请实施例提供了一种基于投屏的鼠标数据处理方法，应用于投屏设备，所述方法包括：In a second aspect, an embodiment of the present application provides a method for processing mouse data based on screen projection, which is applied to a screen projection device, and the method includes:
接收共享设备发送的鼠标数据，所述鼠标数据为共享设备的屏幕上显示的鼠标的鼠标数据，所述共享设备的屏幕上显示的内容投屏至所述投屏设备的屏幕上显示；Receiving mouse data sent by a sharing device, wherein the mouse data is mouse data of a mouse displayed on a screen of the sharing device, and content displayed on the screen of the sharing device is projected onto a screen of the projection device for display;
缓存所述鼠标数据；caching the mouse data;
基于从缓存中读取的所述鼠标数据，在所述投屏设备的屏幕上渲染鼠标。Based on the mouse data read from the cache, the mouse is rendered on the screen of the screen projection device.
可选的，所述鼠标数据包括：鼠标位置和/或鼠标样式。Optionally, the mouse data includes: mouse position and/or mouse style.
可选的，在所述鼠标数据不包括所述鼠标样式的情况下，所述方法还包括：Optionally, when the mouse data does not include the mouse style, the method further includes:
获取缓存的鼠标样式；Get the cached mouse style;
所述基于从缓存中读取的所述鼠标数据，在所述投屏设备的屏幕上渲染鼠标，包括：The rendering of the mouse on the screen of the screen projection device based on the mouse data read from the cache includes:
基于从缓存中读取的所述鼠标数据和所述缓存的鼠标样式，在所述投屏设备的屏幕上渲染鼠标。Based on the mouse data read from the cache and the cached mouse style, the mouse is rendered on the screen of the screen projection device.
可选的，在所述鼠标数据不包括所述鼠标位置的情况下，所述方法还包括：Optionally, when the mouse data does not include the mouse position, the method further includes:
获取缓存的鼠标位置；Get the cached mouse position;
所述基于从缓存中读取的所述鼠标数据，在所述投屏设备的屏幕上渲染鼠标，包括：The rendering of the mouse on the screen of the screen projection device based on the mouse data read from the cache includes:
基于从缓存中读取的所述鼠标数据和所述缓存的鼠标位置，在所述投屏设备的屏幕上渲染鼠标。Based on the mouse data read from the cache and the cached mouse position, the mouse is rendered on the screen of the screen projection device.
可选的，所述方法还包括：Optionally, the method further includes:
接收所述共享设备发送的投屏请求；Receiving a screen projection request sent by the sharing device;
基于所述投屏请求向所述共享设备发送指示信息，所述指示信息用于指示所述投屏设备支持单独接收鼠标数据。Based on the screen projection request, indication information is sent to the sharing device, where the indication information is used to indicate that the screen projection device supports separate reception of mouse data.
第三方面，本申请实施例提供了一种投屏装置，应用于共享设备，所述装置包括：In a third aspect, an embodiment of the present application provides a screen projection device, which is applied to a shared device, and the device includes:
第一采集单元，用于以第一帧率采集屏幕流数据；A first acquisition unit, configured to acquire screen stream data at a first frame rate;
第二采集单元，用于以第二帧率采集鼠标数据；A second acquisition unit, used for acquiring mouse data at a second frame rate;
第一发送单元，用于分别将所述屏幕流数据和所述鼠标数据发送至投屏设备，以使所述投屏设备基于所述屏幕流数据和所述鼠标数据渲染投屏界面。The first sending unit is used to send the screen flow data and the mouse data to the screen projection device respectively, so that the screen projection device renders the screen projection interface based on the screen flow data and the mouse data.
可选的，所述装置还包括：Optionally, the device further comprises:
第二发送单元，用于响应于用户触发的投屏操作，向所述投屏设备发送投屏请求；A second sending unit, configured to send a screen projection request to the screen projection device in response to a screen projection operation triggered by a user;
第一接收单元，用于接收所述投屏设备针对所述投屏请求发送的指示信息，所述指示信息用于指示所述投屏设备支持单独接收鼠标数据。The first receiving unit is used to receive indication information sent by the screen projection device in response to the screen projection request, where the indication information is used to indicate that the screen projection device supports separate reception of mouse data.
可选的，所述共享设备的屏幕上显示的内容投屏至多个投屏设备上显示，所述多个投屏设备均支持单独接收鼠标数据。Optionally, the content displayed on the screen of the shared device is projected onto multiple projection devices for display, and the multiple projection devices all support separate reception of mouse data.
可选的，所述采集鼠标数据，包括：Optionally, collecting mouse data includes:
响应于检测到所述共享设备的屏幕上的鼠标的特征发生改变，获取所述鼠标数据，所述鼠标的特征，包括：鼠标位置和/或鼠标样式。In response to detecting that a feature of a mouse on the screen of the shared device changes, the mouse data is acquired, wherein the feature of the mouse includes: a mouse position and/or a mouse style.
可选的，在所述鼠标的位置发生改变、且所述鼠标的样式未发生改变的情况下，所述鼠标的特征包括所述鼠标位置且不包括所述鼠标样式。Optionally, when the position of the mouse changes and the style of the mouse does not change, the characteristics of the mouse include the mouse position but do not include the mouse style.
可选的，在所述鼠标的位置发生改变、且所述鼠标的样式发生改变的情况下，所述鼠标的特征包括所述鼠标位置和所述鼠标样式。Optionally, when the position of the mouse changes and the style of the mouse changes, the characteristics of the mouse include the mouse position and the mouse style.
可选的，在所述鼠标的位置未发生改变、且所述鼠标的样式发生改变的情况下，所述鼠标的特征包括所述鼠标样式且不包括所述鼠标位置。Optionally, when the position of the mouse does not change and the style of the mouse changes, the characteristics of the mouse include the mouse style but do not include the mouse position.
可选的，所述装置还包括：Optionally, the device further comprises:
第二接收单元，用于接收所述投屏设备针对接收到的所述鼠标数据发送的第一应答；A second receiving unit, configured to receive a first response sent by the screen projection device in response to the received mouse data;
确定单元，用于基于发送信息和应答信息，确定所述第二帧率，所述发送信息和所述接收信息用于指示所述共享设备向所述投屏设备发送鼠标数据的时延，或者，所述发送信息和所述接收信息用于指示共享设备向所述投屏设备发送鼠标数据的丢包情况。A determination unit is used to determine the second frame rate based on the sending information and the response information, wherein the sending information and the receiving information are used to indicate the delay of the shared device sending mouse data to the screen projection device, or the sending information and the receiving information are used to indicate the packet loss situation of the shared device sending mouse data to the screen projection device.
可选的，Optional,
所述发送信息包括：所述共享设备向所述投屏设备发送所述鼠标数据的第一时间戳；The sending information includes: the sharing device sending a first timestamp of the mouse data to the screen projection device;
所述应答信息包括：所述共享设备接收到所述第一应答的第二时间戳。The response information includes: a second timestamp when the sharing device receives the first response.
可选的，Optional,
所述发送信息包括：所述共享设备向所述投屏设备发送的鼠标数据的数量；The sending information includes: the amount of mouse data sent by the sharing device to the screen projection device;
所述应答信息包括：所述共享设备接收到所述投屏设备发送的应答的数量。The response information includes: the number of responses sent by the screen projection device and received by the sharing device.
可选的，所述共享设备支持单独发送鼠标数据。Optionally, the shared device supports sending mouse data separately.
第四方面，本申请实施例提供了一种基于投屏的鼠标数据处理装置，应用于投屏设备，所述装置包括：In a fourth aspect, an embodiment of the present application provides a mouse data processing device based on screen projection, which is applied to a screen projection device, and the device includes:
第一接收单元，用于接收共享设备发送的鼠标数据，所述鼠标数据为共享设备的屏幕上显示的鼠标的鼠标数据，所述共享设备的屏幕上显示的内容投屏至所述投屏设备的屏幕上显示；A first receiving unit is used to receive mouse data sent by a shared device, where the mouse data is mouse data of a mouse displayed on a screen of the shared device, and content displayed on the screen of the shared device is projected onto a screen of the projection device for display;
缓存单元，用于缓存所述鼠标数据；A cache unit, used for caching the mouse data;
渲染单元，用于基于从缓存中读取的所述鼠标数据，在所述投屏设备的屏幕上渲染鼠标。A rendering unit is used to render the mouse on the screen of the projection device based on the mouse data read from the cache.
可选的，所述鼠标数据包括：鼠标位置和/或鼠标样式。Optionally, the mouse data includes: mouse position and/or mouse style.
可选的，在所述鼠标数据不包括所述鼠标样式的情况下，所述装置还包括：Optionally, when the mouse data does not include the mouse style, the device further includes:
第一获取单元，用于获取缓存的鼠标样式；A first acquisition unit, used for acquiring a cached mouse style;
所述渲染单元，用于：The rendering unit is used for:
基于从缓存中读取的所述鼠标数据和所述缓存的鼠标样式，在所述投屏设备的屏幕上渲染鼠标。Based on the mouse data read from the cache and the cached mouse style, the mouse is rendered on the screen of the screen projection device.
可选的，在所述鼠标数据不包括所述鼠标位置的情况下，所述装置还包括：Optionally, when the mouse data does not include the mouse position, the device further includes:
第二获取单元，用于获取缓存的鼠标位置；A second acquisition unit, used to acquire the cached mouse position;
所述渲染单元，用于：The rendering unit is used for:
基于从缓存中读取的所述鼠标数据和所述缓存的鼠标位置，在所述投屏设备的屏幕上渲染鼠标。Based on the mouse data read from the cache and the cached mouse position, the mouse is rendered on the screen of the screen projection device.
可选的，所述装置还包括：Optionally, the device further comprises:
第二接收单元，用于接收所述共享设备发送的投屏请求；A second receiving unit, configured to receive a screen projection request sent by the sharing device;
发送单元，用于基于所述投屏请求向所述共享设备发送指示信息，所述指示信息用于指示所述投屏设备支持单独接收鼠标数据。A sending unit is used to send indication information to the sharing device based on the screen projection request, and the indication information is used to indicate that the screen projection device supports separate reception of mouse data.
第五方面，本申请实施例提供了一种设备，所述设备包括处理器和存储器；In a fifth aspect, an embodiment of the present application provides a device, the device comprising a processor and a memory;
所述处理器用于执行所述存储器中存储的指令，以使得所述设备执行以上第一方面任一项所述的方法，或者，以使得所述设备执行以上第二方面任一项所述的方法。The processor is used to execute instructions stored in the memory so that the device executes the method described in any one of the first aspects above, or so that the device executes the method described in any one of the second aspects above.
第六方面，本申请实施例提供了一种计算机可读存储介质，包括指令，所述指令指示设备执行以上第一方面任一项所述的方法，或者，所述指令指示设备执行以上第二方面任一项所述的方法。In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, wherein the instructions instruct a device to execute any method described in the first aspect above, or the instructions instruct a device to execute any method described in the second aspect above.
第七方面，本申请实施例提供了一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得计算机执行以上第一方面任一项所述的方法，或者，使得计算机执行以上第二方面任一项所述的方法。In a seventh aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above, or enables the computer to execute any of the methods described in the second aspect above.
与现有技术相比，本申请实施例具有以下优点：Compared with the prior art, the embodiments of the present application have the following advantages:
本申请实施例提供了一种投屏方法，所述方法可以应用于共享设备。在一个示例中，共享设备可以以第一帧率采集屏幕流数据、并以第二帧率采集鼠标数据，而后，分别将所述屏幕流数据和所述鼠标数据发送给投屏设备，以使所述投屏设备基于所述屏幕流数据和所述鼠标数据渲染投屏界面。由于共享设备是分别将所述屏幕流数据和所述鼠标数据发送至投屏设备的，因此，鼠标数据的传输受带宽的影响较小，因此，所述第二帧率可以比较高，相应的，投屏设备基于所述鼠标数据渲染投屏界面时，鼠标的动作流畅性也会比较高。An embodiment of the present application provides a screen projection method, which can be applied to a shared device. In one example, the shared device can collect screen stream data at a first frame rate and collect mouse data at a second frame rate, and then send the screen stream data and the mouse data to the screen projection device respectively, so that the screen projection device renders the projection interface based on the screen stream data and the mouse data. Since the shared device sends the screen stream data and the mouse data to the screen projection device respectively, the transmission of the mouse data is less affected by the bandwidth. Therefore, the second frame rate can be relatively high. Correspondingly, when the screen projection device renders the projection interface based on the mouse data, the smoothness of the mouse movement will also be relatively high.
本申请实施例提供了一种投屏方法，所述方法可以应用于投屏设备。在一个示例中，投屏设备可以接收共享设备发送的鼠标数据，并缓存接收到的鼠标数据。而后，从缓存中读取鼠标数据并基于所述鼠标数据渲染鼠标。采用这种方式，在投屏设备同时接收到共享设备发送的多个鼠标数据时，可以首选缓存该多个鼠标数据，而不是基于该多个鼠标数据中的其中一个鼠标数据渲染鼠标并丢弃该多个鼠标数据中的其它鼠标数据，采用本方案，能够避免丢弃鼠标数据而导致的投屏设备的屏幕上显示的鼠标动作不流畅的问题。An embodiment of the present application provides a screen projection method, which can be applied to a screen projection device. In one example, the screen projection device can receive mouse data sent by a shared device and cache the received mouse data. Then, the mouse data is read from the cache and the mouse is rendered based on the mouse data. In this way, when the screen projection device simultaneously receives multiple mouse data sent by a shared device, the multiple mouse data can be cached first, rather than rendering the mouse based on one of the multiple mouse data and discarding the other mouse data in the multiple mouse data. By adopting this solution, the problem of unsmooth mouse movements displayed on the screen of the screen projection device due to discarding mouse data can be avoided.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
图1为本申请实施例提供的一种投屏方法的信令交互图；FIG1 is a signaling interaction diagram of a screen projection method provided in an embodiment of the present application;
图2为本申请实施例提供的一种渲染鼠标的方法的流程示意图；FIG2 is a schematic flow chart of a method for rendering a mouse provided in an embodiment of the present application;
图3为本申请实施例提供的一种投屏装置的结构示意图；FIG3 is a schematic structural diagram of a screen projection device provided in an embodiment of the present application;
图4为本申请实施例提供的一种投屏装置的结构示意图。FIG. 4 is a schematic diagram of the structure of a screen projection device provided in an embodiment of the present application.
具体实施方式DETAILED DESCRIPTION
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
本申请的发明人经过研究发现，传统技术中，将共享设备的屏幕上显示的内容投屏至投屏设备的屏幕上显示时，可以按照特定帧率将共享设备的屏幕上显示的内容作为视频流传输给投屏设备，由投屏设备渲染接收到的视频流。其中，视频流包括屏幕流和鼠标数据。而共享设备将视频流发送给投屏设备时，受网络带宽的影响，所述特定帧率一般不会太高，例如，所述特定帧率为15帧每秒。正是因为所述特定帧率不是太高，因此，当共享设备侧鼠标变化频率较高时，投屏设备侧会出现鼠标动作不流畅的问题。After research, the inventor of the present application discovered that in the traditional technology, when the content displayed on the screen of a shared device is projected onto the screen of a projection device for display, the content displayed on the screen of the shared device can be transmitted to the projection device as a video stream at a specific frame rate, and the projection device renders the received video stream. The video stream includes screen stream and mouse data. When the shared device sends the video stream to the projection device, affected by the network bandwidth, the specific frame rate is generally not too high. For example, the specific frame rate is 15 frames per second. It is precisely because the specific frame rate is not too high that when the mouse changes frequently on the shared device side, the mouse movement will be jerky on the projection device side.
本申请实施例的发明人还发现，在一些场景中，共享设备的屏幕流数据的变化很小。例如，在幻灯片(PowerPoint，PPT)讲解的场景中，共享设备的屏幕流数据可能在一段时间内都停留在PPT的某一页。对于这种情况，若以相同的帧率采集屏幕流数据和鼠标数据，并将包括屏幕流数据和鼠标数据的视频流同步发送给投屏设备，由于屏幕流传输受带宽的影响较大，因此，采集屏幕流的帧率不会很高，而由于采集鼠标数据的帧率与采集屏幕流数据的帧率相同，则采集鼠标数据的帧率也不会很高，因此，若鼠标处于移动过程中，则会导致投屏设备侧会出现鼠标动作不流畅的问题。The inventors of the embodiments of the present application also found that in some scenarios, the screen stream data of the shared device changes very little. For example, in the scenario of slide presentation (PowerPoint, PPT), the screen stream data of the shared device may stay on a certain page of the PPT for a period of time. In this case, if the screen stream data and mouse data are collected at the same frame rate, and the video stream including the screen stream data and the mouse data is synchronously sent to the projection device, since the screen stream transmission is greatly affected by the bandwidth, the frame rate of the screen stream will not be very high, and since the frame rate of the mouse data is the same as the frame rate of the screen stream data, the frame rate of the mouse data will not be very high. Therefore, if the mouse is in the process of moving, it will cause the mouse movement to be not smooth on the projection device side.
另外，由于在该场景下，一段时间内屏幕流数据是相同的，采用向投屏设备发送包括屏幕流和鼠标数据的视频流的方式，会导致向投屏设备重复发送相同的屏幕流数据，导致不必要的带宽浪费。In addition, since the screen stream data is the same for a period of time in this scenario, sending a video stream including screen stream and mouse data to the projection device will cause the same screen stream data to be repeatedly sent to the projection device, resulting in unnecessary waste of bandwidth.
鉴于此，本申请实施例提供了一种投屏方法及装置。In view of this, an embodiment of the present application provides a screen projection method and device.
下面结合附图，详细说明本申请的各种非限制性实施方式。Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.
示例性方法Exemplary Methods
参见图1，该图为本申请实施例提供的一种投屏方法的信令交互图。在本实施例中，所述方法例如可以包括以下步骤S101-S105实现。See Figure 1, which is a signaling interaction diagram of a screen projection method provided in an embodiment of the present application. In this embodiment, the method may include, for example, the following steps S101-S105.
关于图1所示的共享设备和投屏设备，需要说明的是，所述共享设备的屏幕上显示的内容，投屏至投屏设备的屏幕上显示。本申请实施例不具体限定所述共享设备和投屏设备。在一个示例中，所述共享设备和投屏设备可以均为终端设备。在一个示例中，所述共享设备的屏幕上显示的内容可以投屏至一个投屏设备上显示。在又一个示例中，例如，在会议场景中，共享设备(主讲者对应的设备)可以的屏幕上显示的内容可以投屏至多个投屏设备(多个其它会议参与者对应的设备)上显示。本申请实施例不做具体限定。Regarding the sharing device and screen projection device shown in Figure 1, it should be noted that the content displayed on the screen of the sharing device is projected onto the screen of the screen projection device for display. The embodiments of the present application do not specifically limit the sharing device and the screen projection device. In one example, the sharing device and the screen projection device may both be terminal devices. In one example, the content displayed on the screen of the sharing device can be projected onto a screen projection device for display. In another example, for example, in a conference scenario, the content displayed on the screen of the sharing device (the device corresponding to the presenter) can be projected onto multiple screen projection devices (devices corresponding to multiple other conference participants) for display. The embodiments of the present application do not make specific limitations.
S101：共享设备以第一帧率采集屏幕流数据。S101: A sharing device collects screen stream data at a first frame rate.
S102：共享设备以第二帧率采集鼠标数据。S102: The shared device collects mouse data at a second frame rate.
在本申请实施例中，共享设备可以以第一帧率采集屏幕流数据，本申请实施例不具体限定所述第一帧率。此处提及的屏幕流数据，例如可以是所述共享设备的屏幕流数据。在一个示例中，共享设备可以按照一定的采样间隔判断屏幕流数据是否发生改变，若确定所述屏幕流数据发生改变，则采集所述屏幕流数据。采用这种方式，可以避免在屏幕流数据未发生改变时，也采集并向投屏设备发送屏幕流数据，从而向投屏设备发送重复的屏幕流数据，以节省带宽。In an embodiment of the present application, the sharing device can collect screen flow data at a first frame rate, and the embodiment of the present application does not specifically limit the first frame rate. The screen flow data mentioned here can be, for example, the screen flow data of the shared device. In one example, the sharing device can determine whether the screen flow data has changed according to a certain sampling interval. If it is determined that the screen flow data has changed, the screen flow data is collected. In this way, it is possible to avoid collecting and sending screen flow data to the projection device when the screen flow data has not changed, thereby sending repeated screen flow data to the projection device to save bandwidth.
在本申请实施例中，共享设备以第二帧率采集鼠标数据，本申请实施例不具体限定所述第二帧率。此处提及的鼠标数据，指的是所述共享设备的鼠标数据。In the embodiment of the present application, the shared device collects mouse data at a second frame rate, and the embodiment of the present application does not specifically limit the second frame rate. The mouse data mentioned here refers to the mouse data of the shared device.
本申请实施例不具体限定所述第一帧率和所述第二帧率，所述第一帧率和所述第二帧率可以相同，也可以不同。可以理解的是，在不同的应用场景中，第一帧率和/或第二帧率可以根据实际场景的需要人为设置或者根据设定的帧率调节方式自适应地调节。在一个示例中，例如，在前述PPT讲解的场景中，所述第二帧率大于所述第一帧率。The embodiment of the present application does not specifically limit the first frame rate and the second frame rate, and the first frame rate and the second frame rate may be the same or different. It is understandable that in different application scenarios, the first frame rate and/or the second frame rate may be manually set according to the needs of the actual scenario or adaptively adjusted according to the set frame rate adjustment method. In one example, for example, in the scenario explained in the aforementioned PPT, the second frame rate is greater than the first frame rate.
在一个示例中，S102中的“采集鼠标数据”在具体实现时，可以在所述共享设备的屏幕上的鼠标的特征发生改变的情况下，获取所述鼠标数据。采用这种方式，可以避免在鼠标特征未发生改变的情况下，采集并向投屏设备发送重复的鼠标数据，从而节省带宽。In one example, when the “collecting mouse data” in S102 is specifically implemented, the mouse data can be obtained when the characteristics of the mouse on the screen of the shared device change. In this way, it is possible to avoid collecting and sending repeated mouse data to the projection device when the mouse characteristics have not changed, thereby saving bandwidth.
本申请实施例中，鼠标的特征可以包括鼠标位置和/或鼠标样式。在一个示例中，在所述鼠标的位置发生改变、且所述鼠标的样式未发生改变的情况下，所述鼠标的特征可以包括所述鼠标位置且不包括所述鼠标样式。在又一个示例中，在所述鼠标的位置发生改变、且所述鼠标的样式发生改变的情况下，所述鼠标的特征可以包括所述鼠标位置和所述鼠标样式。在另一个示例中，在所述鼠标的位置未发生改变、且所述鼠标的样式发生改变的情况下，所述鼠标的特征包括所述鼠标样式且不包括所述鼠标位置。In an embodiment of the present application, the characteristics of the mouse may include the mouse position and/or the mouse style. In one example, when the position of the mouse changes and the style of the mouse does not change, the characteristics of the mouse may include the mouse position but not the mouse style. In another example, when the position of the mouse changes and the style of the mouse changes, the characteristics of the mouse may include the mouse position and the mouse style. In another example, when the position of the mouse does not change and the style of the mouse changes, the characteristics of the mouse include the mouse style but not the mouse position.
关于所述鼠标数据，需要说明的是，所述鼠标数据指的是共享设备的屏幕上显示的鼠标相关的数据。所述鼠标数据包括鼠标样式和/或鼠标位置。在一个示例中，客户端可以利用其操作系统的应用程序编程接口(Application Programming Interface，API)获得鼠标样式以及鼠标位置。Regarding the mouse data, it should be noted that the mouse data refers to the mouse-related data displayed on the screen of the shared device. The mouse data includes a mouse style and/or a mouse position. In one example, the client can obtain the mouse style and the mouse position using the application programming interface (API) of its operating system.
在一个示例中，所述鼠标数据与鼠标的特征有关。具体地：In one example, the mouse data is related to the characteristics of the mouse. Specifically:
在一个示例中，若所述鼠标的特征包括所述鼠标位置且不包括所述鼠标样式，则所述鼠标数据可以包括所述鼠标位置且不包括所述鼠标样式。In one example, if the characteristics of the mouse include the mouse position but do not include the mouse style, the mouse data may include the mouse position but do not include the mouse style.
在又一个示例中，若所述鼠标的特征包括所述鼠标位置和所述鼠标样式。则所述鼠标数据可以包括所述鼠标位置和所述鼠标样式。其中，所述鼠标样式例如可以是图片格式。In another example, if the characteristics of the mouse include the mouse position and the mouse style, then the mouse data may include the mouse position and the mouse style, wherein the mouse style may be, for example, a picture format.
在另一个示例中，若所述鼠标的特征包括所述鼠标样式且不包括所述鼠标位置，则所述鼠标数据可以包括所述鼠标样式且不包括所述鼠标位置。In another example, if the characteristics of the mouse include the mouse style but not the mouse position, the mouse data may include the mouse style but not the mouse position.
S103：共享设备分别将所述屏幕流数据和所述鼠标数据发送至投屏设备。S103: The sharing device sends the screen stream data and the mouse data to the screen projection device respectively.
在本申请实施例中，所述共享设备具备单独发送鼠标数据的能力，即：共享设备支持单独发送鼠标数据。换言之，共享设备不必一定要按照相同的帧率采集屏幕流数据和鼠标数据、并将包括屏幕流数据和鼠标数据的视频流发送给投屏设备，而可以分别以不同的帧率采集屏幕流数据和鼠标数据，并按照不同的帧率将采集的屏幕流数据和鼠标数据发送给投屏设备。正是因为所述共享设备具备单独发送鼠标数据的能力，因此，共享设备可以分别将所述屏幕流数据和所述鼠标数据发送至投屏设备。In an embodiment of the present application, the shared device has the ability to send mouse data separately, that is, the shared device supports sending mouse data separately. In other words, the shared device does not have to collect screen stream data and mouse data at the same frame rate and send the video stream including screen stream data and mouse data to the screen projection device, but can collect screen stream data and mouse data at different frame rates respectively, and send the collected screen stream data and mouse data to the screen projection device at different frame rates. Precisely because the shared device has the ability to send mouse data separately, the shared device can send the screen stream data and the mouse data to the screen projection device separately.
在一个示例中，执行S103的另一个条件是：所述投屏设备具备单独接收鼠标数据的能力，即：所述投屏设备支持单独接收鼠标数据。In one example, another condition for executing S103 is that the screen projection device has the ability to receive mouse data independently, that is, the screen projection device supports receiving mouse data independently.
在一个示例中，在执行S103之前，所述共享设备可以确定所述投屏设备是否支持单独接收鼠标数据，若确定投屏设备支持单独接收鼠标数据，则共享设备可以执行S103。在一个示例中，在用户触发投屏操作之后，所述共享设备可以响应于用户触发的投屏操作，向所述投屏设备发送投屏请求。作为一个示例，若所述投屏设备支持单独接收鼠标数据，则所述投屏设备接收到所述投屏请求之后，会向所述共享设备发送指示信息，所述指示信息用于指示所述投屏设备支持单独接收鼠标数据。共享设备接收到该指示信息之后，即可确定所述投屏设备支持单独接收鼠标数据。此处提及的投屏设备支持单独接收鼠标数据，指的是投屏设备支持单独绘制鼠标。作为一个示例，投屏设备中可以包括单独绘制鼠标的功能模块。该单独绘制鼠标的功能模块的在接收到鼠标数据之后即可执行，其不受屏幕流数据的影响。In one example, before executing S103, the shared device may determine whether the screen projection device supports receiving mouse data separately. If it is determined that the screen projection device supports receiving mouse data separately, the shared device may execute S103. In one example, after the user triggers the screen projection operation, the shared device may send a screen projection request to the screen projection device in response to the screen projection operation triggered by the user. As an example, if the screen projection device supports receiving mouse data separately, the screen projection device will send an indication message to the shared device after receiving the screen projection request, and the indication message is used to indicate that the screen projection device supports receiving mouse data separately. After the shared device receives the indication message, it can be determined that the screen projection device supports receiving mouse data separately. The screen projection device mentioned here supports receiving mouse data separately, which means that the screen projection device supports drawing mouse separately. As an example, the screen projection device may include a functional module for drawing mouse separately. The functional module for drawing mouse separately can be executed after receiving mouse data, and it is not affected by screen flow data.
关于用户触发的投屏操作，本申请实施例不做具体限定，在一个示例中，所述用户可以利用所述共享设备上安装的用于提供投屏服务的应用程序(Application，APP)来触发投屏操作。Regarding the screen projection operation triggered by the user, the embodiments of the present application do not make any specific limitations. In one example, the user can trigger the screen projection operation by using an application (Application, APP) installed on the shared device for providing screen projection services.
如前所述，所述共享设备的屏幕上显示的内容可以投屏至一个或者多个投屏设备上显示。当所述共享设备的屏幕上显示的内容投屏至多个投屏设备上显示时，所述多个投屏设备均支持单独接收鼠标数据。对于这种情况，所述共享设备可以分别向各个投屏设备发送前述投屏请求，并接收所述各个投屏设备发送的前述指示信息。As mentioned above, the content displayed on the screen of the shared device can be projected onto one or more projection devices for display. When the content displayed on the screen of the shared device is projected onto multiple projection devices for display, the multiple projection devices all support separate reception of mouse data. In this case, the shared device can send the aforementioned projection request to each projection device respectively, and receive the aforementioned indication information sent by each projection device.
S104：投屏设备接收共享设备发送的屏幕流数据和鼠标数据。S104: The screen projection device receives the screen stream data and mouse data sent by the sharing device.
S105：投屏设备基于所述屏幕流数据和所述鼠标数据渲染投屏界面。S105: The screen projection device renders a screen projection interface based on the screen stream data and the mouse data.
S105在具体实现时，可以基于屏幕流数据渲染投屏界面中的页面内容，并基于所述鼠标数据在所述投屏设备的屏幕上渲染鼠标。In a specific implementation, S105 can render the page content in the projection interface based on the screen stream data, and render the mouse on the screen of the projection device based on the mouse data.
在一个示例中，投屏设备接收到鼠标数据之后，可以直接基于所述鼠标数据在所述投屏设备的屏幕上渲染鼠标。In one example, after receiving the mouse data, the screen projection device can directly render the mouse on the screen of the screen projection device based on the mouse data.
通过以上描述可知，利用本申请实施例提供的方案，共享设备能够分别向投屏设备发送屏幕流数据和鼠标数据。这样一来，在上述PPT讲解的场景中，按照不同的帧率分别采集屏幕流数据和鼠标数据，并且，由于屏幕流数据变化很小，因此可以降低采集屏幕流数据的帧率，并适当提升采集鼠标数据的帧率。由于降低了采集屏幕流数据的帧率，从而可以减少向投屏设备发送相同的屏幕流数据的次数，节省网络带宽。而且，由于提升了采集鼠标数据的帧率，因此，投屏设备侧接收所述鼠标数据的帧率也相应得到了提升，从而可以提升投屏设备侧鼠标的动作流畅性。It can be seen from the above description that, by utilizing the solution provided in the embodiment of the present application, the sharing device can send screen stream data and mouse data to the projection device respectively. In this way, in the scenario explained in the above PPT, screen stream data and mouse data are collected respectively at different frame rates, and since the screen stream data changes very little, the frame rate of collecting screen stream data can be reduced, and the frame rate of collecting mouse data can be appropriately increased. Since the frame rate of collecting screen stream data is reduced, the number of times the same screen stream data is sent to the projection device can be reduced, saving network bandwidth. Moreover, since the frame rate of collecting mouse data is increased, the frame rate of receiving the mouse data on the projection device side is also correspondingly improved, thereby improving the smoothness of the mouse movement on the projection device side.
在一个示例中，如图2所示，投屏设备可以基于如下的步骤S201～S203来渲染鼠标数据In one example, as shown in FIG. 2 , the screen projection device can render the mouse data based on the following steps S201 to S203:
S201：接收共享设备发送的鼠标数据，所述鼠标数据为共享设备的屏幕上显示的鼠标的鼠标数据，所述共享设备的屏幕上显示的内容投屏至所述投屏设备的屏幕上显示。S201: Receive mouse data sent by a sharing device, where the mouse data is mouse data of a mouse displayed on a screen of the sharing device, and content displayed on the screen of the sharing device is projected onto a screen of the projection device for display.
在一个示例中，共享设备可以按照第二帧率采集鼠标数据。共享设备采集鼠标数据之后，可以向投屏设备发送鼠标数据，相应的，投屏设备可以接收共享设备发送的鼠标数据。In one example, the sharing device may collect mouse data at the second frame rate. After collecting the mouse data, the sharing device may send the mouse data to the screen projection device, and correspondingly, the screen projection device may receive the mouse data sent by the sharing device.
关于共享设备按照第二帧率采集鼠标数据的具体实现，可以参考上文的相关描述部分，此处不再重复描述。For the specific implementation of the shared device collecting mouse data according to the second frame rate, please refer to the relevant description part above, which will not be repeated here.
S202：缓存所述鼠标数据。S202: Cache the mouse data.
S203：基于从缓存中读取的所述鼠标数据，在所述投屏设备的屏幕上渲染鼠标。S203: Rendering a mouse on the screen of the screen projection device based on the mouse data read from the cache.
考虑到在一些场景中，若共享设备和投屏设备之间的网络出现抖动，则投屏设备可能会出现同时接收到多个鼠标数据的情况。即：共享设备多次发送的鼠标数据同时被投屏设备接收到。对于这种情况，若投屏设备基于该多个鼠标数据中的其中一个鼠标数据渲染鼠标、并丢弃该多个鼠标数据中的其它鼠标数据，则会导致投屏设备的屏幕上的鼠标动作不流畅。对于这种情况，为了使得投屏设备的屏幕上的鼠标动作较为自然流畅。在一个示例中，所述投屏设备可以缓存接收到的多个鼠标数据。例如，投屏设备可以将所述多个鼠标数据缓存在第一缓存区。又如，投屏设备可以将所述多个鼠标数据缓存在某一缓存队列中。在渲染鼠标时，可以依次从缓存中读取缓存的鼠标数据，并基于读取的鼠标数据渲染鼠标。例如，所述投屏设备可以按照一定的帧率从所述缓存中读取鼠标数据，并基于读取的鼠标数据渲染鼠标。Considering that in some scenarios, if the network between the sharing device and the screen projection device jitters, the screen projection device may receive multiple mouse data at the same time. That is, the mouse data sent multiple times by the sharing device is received by the screen projection device at the same time. In this case, if the screen projection device renders the mouse based on one of the multiple mouse data and discards the other mouse data in the multiple mouse data, the mouse movement on the screen of the screen projection device will be unsmooth. In this case, in order to make the mouse movement on the screen of the screen projection device more natural and smooth. In one example, the screen projection device can cache the multiple mouse data received. For example, the screen projection device can cache the multiple mouse data in a first buffer area. For another example, the screen projection device can cache the multiple mouse data in a certain cache queue. When rendering the mouse, the cached mouse data can be read from the cache in sequence, and the mouse can be rendered based on the read mouse data. For example, the screen projection device can read the mouse data from the cache at a certain frame rate, and render the mouse based on the read mouse data.
在一个示例中，基于从缓存中读取的鼠标数据在投屏设备的屏幕上渲染鼠标时，若所述鼠标数据包括鼠标位置但不包括鼠标样式，则所述投屏设备可以获取缓存的鼠标样式，进一步地，基于从缓存中读取的所述鼠标数据(即鼠标位置)和所述缓存的鼠标样式，在所述投屏设备的屏幕上渲染鼠标。此处提及的缓存的鼠标样式，指的是投屏设备上一次渲染鼠标所使用的鼠标样式。换言之，投屏设备上一次渲染鼠标时，可以将渲染鼠标所使用的鼠标样式缓存在缓存中(例如缓存在第二缓存区)。In one example, when rendering a mouse on the screen of a projection device based on mouse data read from a cache, if the mouse data includes a mouse position but does not include a mouse style, the projection device can obtain the cached mouse style, and further render the mouse on the screen of the projection device based on the mouse data (i.e., the mouse position) read from the cache and the cached mouse style. The cached mouse style mentioned here refers to the mouse style used by the projection device to render the mouse last time. In other words, the mouse style used to render the mouse can be cached in the cache (for example, cached in the second cache area) when the projection device renders the mouse last time.
在一个示例中，基于从缓存的鼠标数据在投屏设备的屏幕上渲染鼠标时，若所述鼠标数据包括鼠标样式但不包括鼠标位置，则所述投屏设备可以获取缓存的鼠标位置，进一步地，基于从缓存中读取的所述鼠标数据(即鼠标样式)和所述缓存的鼠标位置，在所述投屏设备的屏幕上渲染鼠标。此处提及的缓存的鼠标位置，指的是投屏设备上一次渲染鼠标所使用的鼠标位置。换言之，投屏设备上一次渲染鼠标时，可以将渲染鼠标所使用的鼠标位置缓存在缓存中(例如缓存在第三缓存区)。In one example, when rendering a mouse on the screen of a projection device based on cached mouse data, if the mouse data includes a mouse style but does not include a mouse position, the projection device can obtain the cached mouse position, and further render the mouse on the screen of the projection device based on the mouse data (i.e., mouse style) read from the cache and the cached mouse position. The cached mouse position mentioned here refers to the mouse position used by the projection device to render the mouse last time. In other words, the last time the projection device rendered the mouse, the mouse position used to render the mouse can be cached in the cache (for example, cached in a third cache area).
在一个示例中，若投屏设备当前渲染鼠标所使用的鼠标位置是从缓存(例如第一缓存区)中获取的，则说明与上一次渲染鼠标相比，鼠标的位置发生了改变，对于这种情况，所述投屏设备可以将第三缓存区中存储的鼠标位置替换为当前渲染鼠标所使用的鼠标位置，以便于下一次渲染鼠标时，若鼠标位置未发生改变、但鼠标样式发生改变，则可以基于第三缓存区中的鼠标位置渲染鼠标。In one example, if the mouse position currently used by the projection device to render the mouse is obtained from a cache (for example, the first cache area), it means that the mouse position has changed compared to the last time the mouse was rendered. In this case, the projection device can replace the mouse position stored in the third cache area with the mouse position currently used to render the mouse, so that the next time the mouse is rendered, if the mouse position has not changed but the mouse style has changed, the mouse can be rendered based on the mouse position in the third cache area.
另外，在一个示例中，若投屏设备当前渲染鼠标所使用的鼠标样式是从缓存(例如第一缓存区)中获取的，则说明与上一次渲染鼠标相比，鼠标的样式发生了改变，对于这种情况，所述投屏设备可以将第二缓存区中存储的鼠标样式替换为当前渲染鼠标所使用的鼠标样式，以便于下一次渲染鼠标时，若鼠标样式未发生改变、但鼠标位置发生改变，则可以基于第二缓存区中的鼠标样式渲染鼠标。In addition, in one example, if the mouse style currently used by the projection device to render the mouse is obtained from a cache (for example, the first cache area), it means that the mouse style has changed compared to the last time the mouse was rendered. In this case, the projection device can replace the mouse style stored in the second cache area with the mouse style currently used to render the mouse, so that the next time the mouse is rendered, if the mouse style has not changed but the mouse position has changed, the mouse can be rendered based on the mouse style in the second cache area.
在一个示例中，共享设备将鼠标数据发送给投屏设备之后，投屏设备接收到鼠标数据，可以向共享设备发送针对所述鼠标数据的应答。进一步地，共享设备可以基于发送信息和应答信息，确定前述第二帧率。其中：In one example, after the sharing device sends the mouse data to the screen projection device, the screen projection device receives the mouse data and can send a response to the mouse data to the sharing device. Further, the sharing device can determine the aforementioned second frame rate based on the sent information and the response information.
所述发送信息和所述接收信息用于指示所述共享设备向所述投屏设备发送鼠标数据的时延，或者，所述发送信息和所述接收信息用于指示共享设备向所述投屏设备发送鼠标数据的丢包情况。The sending information and the receiving information are used to indicate the delay of the shared device sending mouse data to the screen projection device, or the sending information and the receiving information are used to indicate the packet loss situation of the shared device sending mouse data to the screen projection device.
在一个示例中，当所述发送信息和所述接收信息用于指示所述共享设备向所述投屏设备发送鼠标数据的时延时，所述发送信息包括：所述共享设备向所述投屏设备发送所述鼠标数据的第一时间戳。所述应答信息包括：所述共享设备接收到所述第一应答的第二时间戳。In one example, when the sending information and the receiving information are used to indicate the delay of the shared device sending mouse data to the screen projection device, the sending information includes: a first timestamp when the shared device sends the mouse data to the screen projection device. The response information includes: a second timestamp when the shared device receives the first response.
作为一个示例，所述共享设备在向投屏设备发送鼠标数据时，可以记录所述第一时间戳。相应的，所述共享设备在接收到所述投屏设备发送的第一应答时，可以记录第二时间戳。基于第二时间戳和第一时间戳，即可计算出所述共享设备向所述投屏设备发送鼠标数据的时延。例如，所述时延为0.5×(第二时间戳-第一时间戳)。计算出所述时延之后，所述共享设备可以基于所述时延来确定前述第二帧率。例如，第二帧率可以是所述时延的倒数，或者，第二帧率可以略小于所述时延的倒数。As an example, the shared device may record the first timestamp when sending mouse data to the screen projection device. Correspondingly, the shared device may record the second timestamp when receiving the first response sent by the screen projection device. Based on the second timestamp and the first timestamp, the delay for the shared device to send mouse data to the screen projection device can be calculated. For example, the delay is 0.5×(second timestamp-first timestamp). After calculating the delay, the shared device may determine the aforementioned second frame rate based on the delay. For example, the second frame rate may be the inverse of the delay, or the second frame rate may be slightly less than the inverse of the delay.
在一个示例中，当所述发送信息和所述接收信息用于指示共享设备向所述投屏设备发送鼠标数据的丢包情况时，所述发送信息包括：所述共享设备向所述投屏设备发送的鼠标数据的数量；所述应答信息包括：所述共享设备接收到所述投屏设备发送的应答的数量。In one example, when the sending information and the receiving information are used to indicate the packet loss situation of mouse data sent by the sharing device to the screen projection device, the sending information includes: the amount of mouse data sent by the sharing device to the screen projection device; the response information includes: the number of responses sent by the screen projection device received by the sharing device.
在一个示例中，所述共享设备可以在与投屏设备交互的过程中对第二帧率进行调整，其中，第二帧率的初始值例如可以是一个默认值。具体地：In one example, the sharing device may adjust the second frame rate during the interaction with the screen projection device, wherein the initial value of the second frame rate may be, for example, a default value. Specifically:
共享设备可以以第二帧率向所述投屏设备发送鼠标数据，而后，基于所述共享设备向所述投屏设备发送的鼠标数据的数量、以及所述共享设备接收到所述投屏设备发送的应答的数量，对所述第二帧率进行调整。例如，所述共享设备向所述投屏设备发送的鼠标数据的数量大于所述共享设备接收到所述投屏设备发送的应答的数量，则说明所述共享设备向所述投屏设备发送的鼠标数据发生了丢包，则所述共享设备可以减小所述第二帧率。又如，所述共享设备向所述投屏设备发送的鼠标数据的数量等于所述共享设备接收到所述投屏设备发送的应答的数量，则说明所述共享设备向所述投屏设备发送的鼠标数据并未发生丢包，则所述共享设备可以增大所述第二帧率或者保持第二帧率不变。The sharing device may send mouse data to the screen projection device at a second frame rate, and then adjust the second frame rate based on the amount of mouse data sent by the sharing device to the screen projection device and the amount of responses sent by the screen projection device received by the sharing device. For example, if the amount of mouse data sent by the sharing device to the screen projection device is greater than the amount of responses sent by the screen projection device received by the sharing device, it means that packet loss has occurred in the mouse data sent by the sharing device to the screen projection device, and the sharing device may reduce the second frame rate. For another example, if the amount of mouse data sent by the sharing device to the screen projection device is equal to the amount of responses sent by the screen projection device received by the sharing device, it means that packet loss has not occurred in the mouse data sent by the sharing device to the screen projection device, and the sharing device may increase the second frame rate or keep the second frame rate unchanged.
示例性设备Exemplary Devices
基于以上实施例提供的方法，本申请实施例还提供了一种装置，以下结合附图介绍该装置。Based on the method provided in the above embodiment, the embodiment of the present application further provides a device, which is described below in conjunction with the accompanying drawings.
参见图3，该图为本申请实施例提供的一种投屏装置的结构示意图。所述装置300可以应用于共享设备，所述装置300例如可以具体包括：第一采集单元301、第二采集单元302和第一发送单元303。Referring to Fig. 3, which is a schematic diagram of the structure of a screen projection device provided in an embodiment of the present application, the device 300 can be applied to a shared device, and the device 300 can specifically include: a first acquisition unit 301, a second acquisition unit 302, and a first sending unit 303.
第一采集单元301，用于以第一帧率采集屏幕流数据；A first acquisition unit 301 is used to acquire screen stream data at a first frame rate;
第二采集单元302，用于以第二帧率采集鼠标数据；A second acquisition unit 302, configured to acquire mouse data at a second frame rate;
第一发送单元303，用于分别将所述屏幕流数据和所述鼠标数据发送至投屏设备，以使所述投屏设备基于所述屏幕流数据和所述鼠标数据渲染投屏界面。The first sending unit 303 is used to send the screen flow data and the mouse data to the screen projection device respectively, so that the screen projection device renders the screen projection interface based on the screen flow data and the mouse data.
可选的，所述装置还包括：Optionally, the device further comprises:
第二发送单元，用于响应于用户触发的投屏操作，向所述投屏设备发送投屏请求；A second sending unit, configured to send a screen projection request to the screen projection device in response to a screen projection operation triggered by a user;
第一接收单元，用于接收所述投屏设备针对所述投屏请求发送的指示信息，所述指示信息用于指示所述投屏设备支持单独接收鼠标数据。The first receiving unit is used to receive indication information sent by the screen projection device in response to the screen projection request, where the indication information is used to indicate that the screen projection device supports separate reception of mouse data.
可选的，所述共享设备的屏幕上显示的内容投屏至多个投屏设备上显示，所述多个投屏设备均支持单独接收鼠标数据。Optionally, the content displayed on the screen of the shared device is projected onto multiple projection devices for display, and the multiple projection devices all support separate reception of mouse data.
可选的，所述采集鼠标数据，包括：Optionally, collecting mouse data includes:
响应于检测到所述共享设备的屏幕上的鼠标的特征发生改变，获取所述鼠标数据，所述鼠标的特征，包括：鼠标位置和/或鼠标样式。In response to detecting that a feature of a mouse on the screen of the shared device changes, the mouse data is acquired, wherein the feature of the mouse includes: a mouse position and/or a mouse style.
可选的，在所述鼠标的位置发生改变、且所述鼠标的样式未发生改变的情况下，所述鼠标的特征包括所述鼠标位置且不包括所述鼠标样式。Optionally, when the position of the mouse changes and the style of the mouse does not change, the characteristics of the mouse include the mouse position but do not include the mouse style.
可选的，在所述鼠标的位置发生改变、且所述鼠标的样式发生改变的情况下，所述鼠标的特征包括所述鼠标位置和所述鼠标样式。Optionally, when the position of the mouse changes and the style of the mouse changes, the characteristics of the mouse include the mouse position and the mouse style.
可选的，在所述鼠标的位置未发生改变、且所述鼠标的样式发生改变的情况下，所述鼠标的特征包括所述鼠标样式且不包括所述鼠标位置。Optionally, when the position of the mouse does not change and the style of the mouse changes, the characteristics of the mouse include the mouse style but do not include the mouse position.
可选的，所述装置还包括：Optionally, the device further comprises:
第二接收单元，用于接收所述投屏设备针对接收到的所述鼠标数据发送的第一应答；A second receiving unit, configured to receive a first response sent by the screen projection device in response to the received mouse data;
确定单元，用于基于发送信息和应答信息，确定所述第二帧率，所述发送信息和所述接收信息用于指示所述共享设备向所述投屏设备发送鼠标数据的时延，或者，所述发送信息和所述接收信息用于指示共享设备向所述投屏设备发送鼠标数据的丢包情况。A determination unit is used to determine the second frame rate based on the sending information and the response information, wherein the sending information and the receiving information are used to indicate the delay of the shared device sending mouse data to the screen projection device, or the sending information and the receiving information are used to indicate the packet loss situation of the shared device sending mouse data to the screen projection device.
可选的，Optional,
所述发送信息包括：所述共享设备向所述投屏设备发送所述鼠标数据的第一时间戳；The sending information includes: the sharing device sending a first timestamp of the mouse data to the screen projection device;
所述应答信息包括：所述共享设备接收到所述第一应答的第二时间戳。The response information includes: a second timestamp when the sharing device receives the first response.
可选的，Optional,
所述发送信息包括：所述共享设备向所述投屏设备发送的鼠标数据的数量；The sending information includes: the amount of mouse data sent by the sharing device to the screen projection device;
所述应答信息包括：所述共享设备接收到所述投屏设备发送的应答的数量。The response information includes: the number of responses sent by the screen projection device and received by the sharing device.
可选的，所述共享设备支持单独发送鼠标数据。Optionally, the shared device supports sending mouse data separately.
由于所述装置300是与以上方法实施例提供的方法对应的装置，所述装置300的各个单元的具体实现，均与以上方法实施例为同一构思，因此，关于所述装置300的各个单元的具体实现，可以参考以上方法实施例对于共享设备执行的步骤的描述部分，此处不再赘述。Since the device 300 is a device corresponding to the method provided in the above method embodiment, the specific implementation of each unit of the device 300 is based on the same concept as the above method embodiment. Therefore, regarding the specific implementation of each unit of the device 300, you can refer to the description part of the steps performed by the above method embodiment for the shared device, which will not be repeated here.
参见图4，该图为本申请实施例提供的一种投屏装置的结构示意图。所述装置400可以应用于投屏设备，所述装置400例如可以具体包括：第一接收单元401、缓存单元402和渲染单元403。4 is a schematic diagram of a screen projection device according to an embodiment of the present application. The device 400 may be applied to a screen projection device, and the device 400 may specifically include: a first receiving unit 401 , a cache unit 402 , and a rendering unit 403 .
第一接收单元401，用于接收共享设备发送的鼠标数据，所述鼠标数据为共享设备的屏幕上显示的鼠标的鼠标数据，所述共享设备的屏幕上显示的内容投屏至所述投屏设备的屏幕上显示；The first receiving unit 401 is used to receive mouse data sent by a shared device, where the mouse data is mouse data of a mouse displayed on a screen of the shared device, and content displayed on the screen of the shared device is projected onto a screen of the projection device for display;
缓存单元402，用于缓存所述鼠标数据；A cache unit 402, used for caching the mouse data;
渲染单元403，用于基于从缓存中读取的所述鼠标数据，在所述投屏设备的屏幕上渲染鼠标。The rendering unit 403 is used to render the mouse on the screen of the projection device based on the mouse data read from the cache.
可选的，所述鼠标数据包括：鼠标位置和/或鼠标样式。Optionally, the mouse data includes: mouse position and/or mouse style.
可选的，在所述鼠标数据不包括所述鼠标样式的情况下，所述装置还包括：Optionally, when the mouse data does not include the mouse style, the device further includes:
第一获取单元，用于获取缓存的鼠标样式；A first acquisition unit, used for acquiring a cached mouse style;
所述渲染单元403，用于：The rendering unit 403 is used to:
基于从缓存中读取的所述鼠标数据和所述缓存的鼠标样式，在所述投屏设备的屏幕上渲染鼠标。Based on the mouse data read from the cache and the cached mouse style, the mouse is rendered on the screen of the screen projection device.
可选的，在所述鼠标数据不包括所述鼠标位置的情况下，所述装置还包括：Optionally, when the mouse data does not include the mouse position, the device further includes:
第二获取单元，用于获取缓存的鼠标位置；A second acquisition unit, used to acquire the cached mouse position;
所述渲染单元403，用于：The rendering unit 403 is used to:
基于从缓存中读取的所述鼠标数据和所述缓存的鼠标位置，在所述投屏设备的屏幕上渲染鼠标。Based on the mouse data read from the cache and the cached mouse position, the mouse is rendered on the screen of the screen projection device.
可选的，所述装置还包括：Optionally, the device further comprises:
第二接收单元，用于接收所述共享设备发送的投屏请求；A second receiving unit, configured to receive a screen projection request sent by the sharing device;
发送单元，用于基于所述投屏请求向所述共享设备发送指示信息，所述指示信息用于指示所述投屏设备支持单独接收鼠标数据。A sending unit is used to send indication information to the sharing device based on the screen projection request, and the indication information is used to indicate that the screen projection device supports separate reception of mouse data.
由于所述装置400是与以上方法实施例提供的方法对应的装置，所述装置400的各个单元的具体实现，均与以上方法实施例为同一构思，因此，关于所述装置400的各个单元的具体实现，可以参考以上方法实施例对于投屏设备执行的步骤的描述部分，此处不再赘述。Since the device 400 is a device corresponding to the method provided in the above method embodiment, the specific implementation of each unit of the device 400 is the same concept as the above method embodiment. Therefore, regarding the specific implementation of each unit of the device 400, you can refer to the description part of the steps performed by the screen projection device in the above method embodiment, which will not be repeated here.
本申请实施例提供了一种设备，所述设备包括处理器和存储器；An embodiment of the present application provides a device, the device comprising a processor and a memory;
所述处理器用于执行所述存储器中存储的指令，以使得所述设备执行以上方法实施例任一项所述的由共享设备或者投屏设备执行的方法。The processor is used to execute instructions stored in the memory so that the device executes the method performed by the sharing device or the screen projection device as described in any of the above method embodiments.
本申请实施例提供了一种计算机可读存储介质，包括指令，所述指令指示设备执行以上方法实施例任一项由共享设备或者投屏设备执行的方法。An embodiment of the present application provides a computer-readable storage medium, including instructions, wherein the instructions instruct a device to execute any one of the above method embodiments performed by a sharing device or a screen projection device.
本申请实施例提供了一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得计算机执行以上方法实施例任一项由共享设备或者投屏设备执行的方法。An embodiment of the present application provides a computer program product. When the computer program product is run on a computer, the computer executes any one of the above method embodiments performed by a sharing device or a screen projection device.
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
以上所述仅为本申请的较佳实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210143178.6ACN114510191B (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210143178.6ACN114510191B (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
| Publication Number | Publication Date |
|---|---|
| CN114510191A CN114510191A (en) | 2022-05-17 |
| CN114510191Btrue CN114510191B (en) | 2024-10-18 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210143178.6AActiveCN114510191B (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
| Country | Link |
|---|---|
| CN (1) | CN114510191B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115052186B (en)* | 2022-07-12 | 2023-09-15 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115037978B (en)* | 2022-07-13 | 2023-08-25 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115209117B (en)* | 2022-07-20 | 2024-06-18 | 北京字跳网络技术有限公司 | Screen projection method and device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155606A (en)* | 2015-04-07 | 2016-11-23 | 中国移动通信集团公司 | A kind of multi-screen interaction method and device |
| CN106375841A (en)* | 2015-07-23 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Wireless screen projection data processing method and device, video data display method and device, and electronic device |
| CN107317655A (en)* | 2017-06-06 | 2017-11-03 | 努比亚技术有限公司 | Transfer control method, system and the readable storage medium storing program for executing of screen prjection |
| CN110519632A (en)* | 2019-07-30 | 2019-11-29 | 华为技术有限公司 | Throw screen method and apparatus |
| CN111541919A (en)* | 2020-05-13 | 2020-08-14 | 北京百度网讯科技有限公司 | Video frame transmission method and device, electronic equipment and readable storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112399215A (en)* | 2020-10-26 | 2021-02-23 | 努比亚技术有限公司 | Screen projection parameter regulation and control method and device and computer readable storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155606A (en)* | 2015-04-07 | 2016-11-23 | 中国移动通信集团公司 | A kind of multi-screen interaction method and device |
| CN106375841A (en)* | 2015-07-23 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Wireless screen projection data processing method and device, video data display method and device, and electronic device |
| CN107317655A (en)* | 2017-06-06 | 2017-11-03 | 努比亚技术有限公司 | Transfer control method, system and the readable storage medium storing program for executing of screen prjection |
| CN110519632A (en)* | 2019-07-30 | 2019-11-29 | 华为技术有限公司 | Throw screen method and apparatus |
| CN111541919A (en)* | 2020-05-13 | 2020-08-14 | 北京百度网讯科技有限公司 | Video frame transmission method and device, electronic equipment and readable storage medium |
| Publication number | Publication date |
|---|---|
| CN114510191A (en) | 2022-05-17 |
| Publication | Publication Date | Title |
|---|---|---|
| CN114510191B (en) | Screen projection method and device | |
| US11778006B2 (en) | Data transmission method and apparatus | |
| CN110268720B (en) | Method, system, and apparatus for synchronizing streaming application environments with remote device input | |
| CN106686438B (en) | method, device and system for synchronously playing audio images across equipment | |
| CN108810636A (en) | Video playing method, device and system | |
| WO2019154221A1 (en) | Method for sending streaming data and data sending device | |
| EP1566038A1 (en) | Methods and systems for transferring events including multimedia data | |
| CN103974057B (en) | A kind of video quality user experience value assessment method, equipment and system | |
| CN109936769A (en) | A video freeze detection method, detection system, mobile terminal and storage device | |
| WO2019100475A1 (en) | Data transmission method, apparatus and device, and storage medium | |
| CN114666015A (en) | Method and apparatus for facilitating data transmission | |
| CN115834918B (en) | Video live broadcast method and device, electronic equipment and readable storage medium | |
| CN119790641A (en) | Sender-based adaptive bitrate control | |
| WO2017005098A1 (en) | Method and device for realizing the fast-forward or fast-backward of video stream | |
| CN110192181A (en) | Streaming application environment with recovery of lost or delayed input events | |
| CN115883854A (en) | Cloud desktop display method, terminal, cloud desktop system, device and readable medium | |
| CN118764673A (en) | A video transmission method and transmission system for outdoor large screens | |
| CN116916095A (en) | Smooth display method, device and equipment of cloud video and storage medium | |
| CN110113641B (en) | Video data transmission method, device, edge service node and medium | |
| CN114760522A (en) | Data processing method, device and equipment | |
| CN112866745B (en) | Streaming media video data processing method, device, computer equipment and storage medium | |
| CN115942000A (en) | Video stream transcoding method, device, equipment and medium based on H.264 format | |
| CN110798700B (en) | Video processing method, video processing device, storage medium and electronic equipment | |
| KR20220078509A (en) | Concurrent downloading of video | |
| JP2024513707A (en) | Method, device, and system for improving quality of experience in cloud gaming |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |