



技术领域technical field
本发明涉及播控技术领域,更具体地说,它涉及一种可扩展的基于chromium内核的多媒体播控系统。The invention relates to the technical field of broadcast control, more specifically, it relates to an expandable multimedia broadcast control system based on the chromium kernel.
背景技术Background technique
多媒体渲染系统实现方案一般采用c/s模型,这种实现方式性能比较高效,但是随着人机交互方式越来越复杂,软件的扩展、维护成本越来越高。B/S模型可以解决这些痛点,但是,目前基于chromium的浏览器仅仅能支持有限的信号源类型:如本地音视频、WEBRTC、PDF等一些传统的信号源,然而,如RTSP、RTMP、SIP、自定义的流媒体协议等信号源在chromium内核层面上没有实现,信号源甚至包括word、ppt、excel、投屏等。The implementation scheme of the multimedia rendering system generally adopts the c/s model, which is more efficient in performance, but as the human-computer interaction becomes more and more complex, the software expansion and maintenance costs are getting higher and higher. The B/S model can solve these pain points. However, the current chromium-based browsers can only support limited signal source types: such as local audio and video, WEBRTC, PDF and other traditional signal sources. However, such as RTSP, RTMP, SIP, Signal sources such as custom streaming media protocols are not implemented at the chromium kernel level, and signal sources even include word, ppt, excel, screen projection, etc.
其中WEBRTC为网页即时通信(英语:Web Real-Time Communication)的缩写,是一个支持网页浏览器进行实时语音对话或视频对话的API。RTSP(Real Time StreamingProtocol),RFC2326,实时流传输协议,是TCP/IP协议体系中的一个应用层协议,由哥伦比亚大学、网景和RealNetworks公司提交的IETF RFC标准。RTMP是Real Time MessagingProtocol(实时消息传输协议)的首字母缩写。RTMP是一种设计用来进行实时数据通信的网络协议,主要用来在Flash/AIR平台和支持RTMP协议的流媒体/交互服务器之间进行音视频和数据通信。WEBRTC is the abbreviation of Web Real-Time Communication (English: Web Real-Time Communication), which is an API that supports web browsers to conduct real-time voice conversations or video conversations. RTSP (Real Time Streaming Protocol), RFC2326, real-time streaming protocol, is an application layer protocol in the TCP/IP protocol system, an IETF RFC standard submitted by Columbia University, Netscape and RealNetworks. RTMP is the acronym for Real Time Messaging Protocol (Real Time Messaging Protocol). RTMP is a network protocol designed for real-time data communication. It is mainly used for audio, video and data communication between the Flash/AIR platform and streaming media/interactive servers that support the RTMP protocol.
发明内容Contents of the invention
本发明要解决的技术问题是针对现有技术的上述不足,本发明的目的是提供一种可以支持不同信号源类型的可扩展的基于chromium内核的多媒体播控系统。The technical problem to be solved by the present invention is aimed at the above-mentioned deficiencies of the prior art, and the purpose of the present invention is to provide an expandable multimedia broadcast control system based on the chromium kernel that can support different signal source types.
本发明的技术方案是:一种可扩展的基于chromium内核的多媒体播控系统,包括:The technical solution of the present invention is: a kind of scalable multimedia broadcast control system based on chromium kernel, comprising:
Browser进程,所述Browser进程是使用chromium提供的content层进行定制的,用于对多媒体数据源管理进程用户进行授权处理;Browser process, the Browser process is customized using the content layer provided by chromium, and is used to authorize the user of the multimedia data source management process;
多媒体渲染进程,所述多媒体渲染进程为基于chromium的PPAPI框架实现,用于对多媒体信号进行解码、渲染;A multimedia rendering process, the multimedia rendering process is implemented based on the chromium-based PPAPI framework, and is used to decode and render multimedia signals;
多媒体数据源管理进程,所述多媒体数据源管理进程为基于chromium的PPAPI框架实现,用于管理信号源插件进程;The multimedia data source management process, the multimedia data source management process is realized based on the PPAPI framework of chromium, and is used to manage the signal source plug-in process;
信号源插件进程,所述信号源插件进程用于采集信号源数据,并发送给所述多媒体渲染进程;不同信号源类型对应于不同的信号源插件进程;A signal source plug-in process, the signal source plug-in process is used to collect signal source data and send it to the multimedia rendering process; different signal source types correspond to different signal source plug-in processes;
所述多媒体渲染进程与所述多媒体数据源管理进程、信号源插件进程建立IPC连接。The multimedia rendering process establishes an IPC connection with the multimedia data source management process and the signal source plug-in process.
进一步地,所述多媒体渲染进程包括:Further, the multimedia rendering process includes:
进程间通信模块,用于负责建立与所述多媒体数据源管理进程的IPC连接、负责建立与所述信号源插件进程的IPC连接;An inter-process communication module, configured to be responsible for establishing an IPC connection with the multimedia data source management process, and for establishing an IPC connection with the signal source plug-in process;
事件接收与转发模块,用于负责接收来自鼠标、键盘、触摸、JS脚本的消息,并根据所述消息的类型将所述消息转发给所述多媒体数据源管理进程或具体的信号源插件进程;The event receiving and forwarding module is used to be responsible for receiving messages from the mouse, keyboard, touch, and JS scripts, and forwards the messages to the multimedia data source management process or specific signal source plug-in process according to the type of the message;
音视频解码模块,用于对音视频数据进行解码处理;The audio and video decoding module is used to decode audio and video data;
调度模块,用于负责调度渲染模块;The scheduling module is responsible for scheduling the rendering module;
渲染模块,用于将解码后的音视频数据帧绘制到纹理上。The rendering module is used to draw the decoded audio and video data frames onto the texture.
进一步地,所述多媒体数据源管理进程包括:Further, the multimedia data source management process includes:
进程间通信建立模块,用于负责建立与所述多媒体渲染进程的IPC连接;An inter-process communication establishment module, configured to be responsible for establishing an IPC connection with the multimedia rendering process;
插件进程启动模块,用于根据所述多媒体渲染进程传入的信号源类型,启动相应的信号源插件进程;A plug-in process startup module, configured to start a corresponding signal source plug-in process according to the signal source type passed in by the multimedia rendering process;
事件接收与发送模块,用于接收来自所述多媒体渲染进程的消息。The event receiving and sending module is used for receiving messages from the multimedia rendering process.
进一步地,所述信号源插件进程包括:Further, the signal source plug-in process includes:
进程间通信建立模块,用于负责建立与所述多媒体数据源管理进程的IPC连接、负责建立与所述多媒体渲染进程的IPC连接;An inter-process communication establishing module, configured to be responsible for establishing an IPC connection with the multimedia data source management process, and for establishing an IPC connection with the multimedia rendering process;
事件接收发送模块,用于接收来自所述多媒体数据源管理进程、多媒体渲染进程的消息;An event receiving and sending module, configured to receive messages from the multimedia data source management process and the multimedia rendering process;
信号源数据帧发送模块,用于接收人机交互复杂的信号源数据,当一帧数据准备完毕以后,发送给所述多媒体渲染进程。The signal source data frame sending module is used to receive complex signal source data for human-computer interaction, and send it to the multimedia rendering process after a frame of data is prepared.
有益效果Beneficial effect
本发明与现有技术相比,具有的优点为:Compared with the prior art, the present invention has the advantages of:
1.信号源的解码与渲染都基于chromium内核完成,保证了系统的性能,人机交互是复杂的,而且经常变化,利用JavaScript来实现这部分功能,保证了系统的可维护性。1. The decoding and rendering of the signal source are completed based on the chromium kernel, which ensures the performance of the system. Human-computer interaction is complex and often changes. Using JavaScript to realize this part of the function ensures the maintainability of the system.
2.如果需要支持一种新的信号源,只需要编写相应的插件,这样保证了系统的可扩展性。2. If you need to support a new signal source, you only need to write the corresponding plug-in, which ensures the scalability of the system.
3.如果一个信号源插件崩溃,不会影响其它插件,保证了系统的稳定性。3. If a signal source plug-in crashes, it will not affect other plug-ins, ensuring the stability of the system.
附图说明Description of drawings
图1为本发明的总体架构图;Fig. 1 is the overall architecture diagram of the present invention;
图2为本发明中多媒体渲染进程的架构图;Fig. 2 is the architecture diagram of the multimedia rendering process in the present invention;
图3为本发明中多媒体数据源管理进程的架构图;Fig. 3 is the frame diagram of multimedia data source management process in the present invention;
图4本发明中信号源插件进程的架构图。Fig. 4 is a structure diagram of a signal source plug-in process in the present invention.
具体实施方式Detailed ways
下面结合附图中的具体实施例对本发明做进一步的说明。The present invention will be further described below in conjunction with the specific embodiments in the accompanying drawings.
参阅图1-4,一种可扩展的基于chromium内核的多媒体播控系统,包括:Refer to Figure 1-4, a scalable chromium-based multimedia broadcast control system, including:
Browser进程,Browser进程是使用chromium提供的content层进行定制的,除浏览器一些必备的功能以外,还用于对多媒体数据源管理进程用户进行授权处理,授权的多媒体数据源管理进程可以启动运行,Chromium是由Google主导开发的网页浏览器;Browser process, the Browser process is customized using the content layer provided by chromium. In addition to some necessary functions of the browser, it is also used to authorize the user of the multimedia data source management process. The authorized multimedia data source management process can start and run , Chromium is a web browser developed by Google;
多媒体渲染进程,多媒体渲染进程为基于chromium的PPAPI框架实现,本质上是一个PpapiPlugin进程,用于对多媒体信号进行解码、渲染;Multimedia rendering process, the multimedia rendering process is implemented by the chromium-based PPAPI framework, which is essentially a PpapiPlugin process for decoding and rendering multimedia signals;
多媒体数据源管理进程,多媒体数据源管理进程为基于chromium的PPAPI框架实现,本质上是一个PpapiPlugin进程,用于管理信号源插件进程;The multimedia data source management process is implemented by the chromium-based PPAPI framework, which is essentially a PpapiPlugin process used to manage the signal source plug-in process;
信号源插件进程,信号源插件进程用于采集信号源音视频数据,并发送给多媒体渲染进程;不同信号源类型对应于不同的信号源插件进程,如流媒体插件、Word插件、PPT插件、EXCEL插件等;Signal source plug-in process, the signal source plug-in process is used to collect signal source audio and video data, and send it to the multimedia rendering process; different signal source types correspond to different signal source plug-in processes, such as streaming media plug-ins, Word plug-ins, PPT plug-ins, EXCEL plug-ins, etc.;
多媒体渲染进程与多媒体数据源管理进程、信号源插件进程建立IPC连接,IPC即Inter-Process Communication,进程间通信。The multimedia rendering process establishes an IPC connection with the multimedia data source management process and the signal source plug-in process. IPC is Inter-Process Communication, inter-process communication.
系统还包括HTML文本、Render进程、GPU进程。Browser进程、Render进程读取HTML文本,Browser进程通过Render进程建立与多媒体数据源管理进程、多媒体渲染进程的IPC连接,GPU进程分别建立与Browser进程、Render进程IPC连接。The system also includes HTML text, Render process, and GPU process. The Browser process and the Render process read the HTML text, the Browser process establishes an IPC connection with the multimedia data source management process and the multimedia rendering process through the Render process, and the GPU process respectively establishes an IPC connection with the Browser process and the Render process.
多媒体渲染进程的架构如图2所示,包括:The architecture of the multimedia rendering process is shown in Figure 2, including:
进程间通信模块,用于负责建立与多媒体数据源管理进程的IPC连接、负责建立与信号源插件进程的IPC连接;The inter-process communication module is used to be responsible for establishing an IPC connection with the multimedia data source management process, and for establishing an IPC connection with the signal source plug-in process;
事件接收与转发模块,用于负责接收来自鼠标、键盘、触摸、JS脚本的消息,并根据消息的类型将消息转发给多媒体数据源管理进程或具体的信号源插件进程;The event receiving and forwarding module is responsible for receiving messages from the mouse, keyboard, touch, and JS scripts, and forwards the messages to the multimedia data source management process or specific signal source plug-in process according to the type of the message;
音视频解码模块,用于对音视频数据进行解码处理;The audio and video decoding module is used to decode audio and video data;
调度模块,用于负责调度渲染模块;The scheduling module is responsible for scheduling the rendering module;
渲染模块,用于将解码后的音视频数据帧绘制到纹理上;The rendering module is used to draw the decoded audio and video data frames onto the texture;
信号源的解码与渲染都基于chromium内核完成,保证了系统的性能。The decoding and rendering of the signal source are completed based on the chromium kernel, which ensures the performance of the system.
多媒体数据源管理进程的架构如图3所示,包括:The architecture of the multimedia data source management process is shown in Figure 3, including:
进程间通信建立模块,用于负责建立与多媒体渲染进程的IPC连接;The inter-process communication establishment module is responsible for establishing an IPC connection with the multimedia rendering process;
插件进程启动模块,用于根据多媒体渲染进程传入的信号源类型,启动相应的信号源插件进程;The plug-in process startup module is used to start the corresponding signal source plug-in process according to the signal source type passed in by the multimedia rendering process;
事件接收与发送模块,用于接收来自多媒体渲染进程的消息。The event receiving and sending module is used for receiving messages from the multimedia rendering process.
信号源插件进程的架构如图4所示,包括:The architecture of the signal source plug-in process is shown in Figure 4, including:
进程间通信建立模块,用于负责建立与多媒体数据源管理进程的IPC连接、负责建立与多媒体渲染进程的IPC连接;The inter-process communication establishment module is used for establishing the IPC connection with the multimedia data source management process and the IPC connection with the multimedia rendering process;
事件接收发送模块,用于接收来自多媒体数据源管理进程、多媒体渲染进程的消息;The event receiving and sending module is used to receive messages from the multimedia data source management process and the multimedia rendering process;
信号源数据帧发送模块,用于接收人机交互复杂的信号源数据,当一帧数据准备完毕以后,发送给多媒体渲染进程;The signal source data frame sending module is used to receive complex signal source data for human-computer interaction, and send it to the multimedia rendering process after a frame of data is prepared;
人机交互是复杂的,而且经常变化,利用JavaScript来实现这部分功能,保证了系统的可维护性,如果需要支持一种新的信号源,只需要编写相应的插件,这样保证了系统的可扩展性,如果一个信号源插件崩溃,不会影响其它插件,保证了系统的稳定性。Human-computer interaction is complex and often changes. Using JavaScript to realize this part of the function ensures the maintainability of the system. If you need to support a new signal source, you only need to write the corresponding plug-in, which ensures the system. Scalability, if a signal source plug-in crashes, it will not affect other plug-ins, ensuring the stability of the system.
以上仅是本发明的优选实施方式,应当指出对于本领域的技术人员来说,在不脱离本发明结构的前提下,还可以作出若干变形和改进,这些都不会影响本发明实施的效果和专利的实用性。The above are only preferred embodiments of the present invention, and it should be pointed out that for those skilled in the art, some deformations and improvements can be made without departing from the structure of the present invention, and these will not affect the effect and implementation of the present invention. Utility of patents.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310084766.1ACN116193211B (en) | 2023-02-02 | 2023-02-02 | Scalable chromium-kernel-based multimedia broadcast control system |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310084766.1ACN116193211B (en) | 2023-02-02 | 2023-02-02 | Scalable chromium-kernel-based multimedia broadcast control system |
| Publication Number | Publication Date |
|---|---|
| CN116193211Atrue CN116193211A (en) | 2023-05-30 |
| CN116193211B CN116193211B (en) | 2024-11-29 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310084766.1AActiveCN116193211B (en) | 2023-02-02 | 2023-02-02 | Scalable chromium-kernel-based multimedia broadcast control system |
| Country | Link |
|---|---|
| CN (1) | CN116193211B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682014A (en)* | 2011-03-14 | 2012-09-19 | 腾讯科技(深圳)有限公司 | Open-type plug-in module management platform implemented on browser and open-type plug-in module management method |
| US20120284632A1 (en)* | 2011-05-06 | 2012-11-08 | Cisco Technology, Inc. | Integrated Rendering of Streaming Media in Virtualized Desktop Environment |
| CN105681841A (en)* | 2015-12-30 | 2016-06-15 | 广东欧珀移动通信有限公司 | Video play control method and system based on browser |
| US20170358036A1 (en)* | 2016-06-10 | 2017-12-14 | Understory, LLC | Data processing system for managing activities linked to multimedia content |
| CN110446114A (en)* | 2019-08-19 | 2019-11-12 | 广州视源电子科技股份有限公司 | Multimedia data processing device, method, electronic equipment and storage medium |
| CN112422508A (en)* | 2020-10-19 | 2021-02-26 | 武汉烽火众智数字技术有限责任公司 | Method and system for playing video in browser webpage |
| CN112468883A (en)* | 2019-09-09 | 2021-03-09 | 浙江宇视科技有限公司 | Video stream playing method and video stream playing equipment |
| CN114491343A (en)* | 2022-01-27 | 2022-05-13 | 山东浪潮通软信息科技有限公司 | Method, system, device and readable medium for playing streaming media in browser |
| CN115328576A (en)* | 2022-09-03 | 2022-11-11 | 郭云开 | Device and method for browser loading plug-in |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682014A (en)* | 2011-03-14 | 2012-09-19 | 腾讯科技(深圳)有限公司 | Open-type plug-in module management platform implemented on browser and open-type plug-in module management method |
| US20120284632A1 (en)* | 2011-05-06 | 2012-11-08 | Cisco Technology, Inc. | Integrated Rendering of Streaming Media in Virtualized Desktop Environment |
| CN105681841A (en)* | 2015-12-30 | 2016-06-15 | 广东欧珀移动通信有限公司 | Video play control method and system based on browser |
| US20170358036A1 (en)* | 2016-06-10 | 2017-12-14 | Understory, LLC | Data processing system for managing activities linked to multimedia content |
| CN110446114A (en)* | 2019-08-19 | 2019-11-12 | 广州视源电子科技股份有限公司 | Multimedia data processing device, method, electronic equipment and storage medium |
| CN112468883A (en)* | 2019-09-09 | 2021-03-09 | 浙江宇视科技有限公司 | Video stream playing method and video stream playing equipment |
| CN112422508A (en)* | 2020-10-19 | 2021-02-26 | 武汉烽火众智数字技术有限责任公司 | Method and system for playing video in browser webpage |
| CN114491343A (en)* | 2022-01-27 | 2022-05-13 | 山东浪潮通软信息科技有限公司 | Method, system, device and readable medium for playing streaming media in browser |
| CN115328576A (en)* | 2022-09-03 | 2022-11-11 | 郭云开 | Device and method for browser loading plug-in |
| Title |
|---|
| 烟雨任平生: "PPAPI插件开发指南", Retrieved from the Internet <URL:https://www.cnblogs.com/fangkm/p/6628425.html>* |
| 罗升阳: "Chromium多进程架构简要介绍和学习计划", pages 1 - 5, Retrieved from the Internet <URL:https://blog.csdn.net/Luoshengyang/article/details/47364477>* |
| 胡济豪: "基于Gstreamer的Chromium音视频播放系统的设计与实现", 《中国优秀硕士学位论文全文数据库》, no. 9, 15 September 2019 (2019-09-15)* |
| Publication number | Publication date |
|---|---|
| CN116193211B (en) | 2024-11-29 |
| Publication | Publication Date | Title |
|---|---|---|
| US9035991B2 (en) | Collaboration system and method | |
| CN102158423B (en) | Upgrade via chat group keeping track of content item | |
| TW444167B (en) | Internet convolution audio/video server | |
| CN103200461B (en) | A kind of multiple stage playback terminal synchronous playing system and player method | |
| CN112653700B (en) | Website video communication method based on WEBRTC | |
| CN107707533B (en) | Company's wheat method, system and storage medium based on Web | |
| US8918827B2 (en) | Method and system for providing complete internet anywhere with partial server processing | |
| US20090290012A1 (en) | Methods for providing video content and associated service devices, systems thereof | |
| US20200274908A1 (en) | Screencast orchestration | |
| CN101242430A (en) | Fixed-point data prefetching method in peer-to-peer network on-demand system | |
| CN108055304B (en) | Synchronization method, apparatus, server, device and storage medium for remote data | |
| WO2017088394A1 (en) | Online live video player and playing method | |
| WO2023273713A1 (en) | Communication method and apparatus for multicast and broadcast service, medium, and electronic device | |
| KR20220133964A (en) | Method and apparatus for dynamic adaptive streaming over HTTP | |
| CN117294805A (en) | Video conference cloud recording method and device, electronic equipment and storage medium | |
| CN100571378C (en) | A method for improving the delivery speed of program pages for cable television video on demand | |
| CN110460889A (en) | A video projection method, device, system and storage medium | |
| CN101442627A (en) | Control method for peer-to-peer calculation set-top box player | |
| WO2013152686A1 (en) | Live video stream aggregation distribution method and device | |
| CN116193211A (en) | Scalable multimedia broadcast control system based on chromoum kernel | |
| CN118646905A (en) | A low-latency high-definition live web playback method | |
| CN102143132A (en) | Frequency channel switching method, switching agency, client and terminal for stream media | |
| CN113079386A (en) | Video online playing method and device, electronic equipment and storage medium | |
| CN100391149C (en) | Application management subsystem and method in multi-node intelligent network application service system | |
| CN105898621A (en) | Scalable video transmission method, device and system |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |