技术领域Technical Field
本发明涉及无线多媒体通信技术领域，具体涉及无线动态视频的跨层传输方法、装置、系统、介质及产品。The present invention relates to the technical field of wireless multimedia communication, and in particular to a cross-layer transmission method, device, system, medium and product of wireless dynamic video.
背景技术Background technique
随着车联网技术的蓬勃发展，视频数据在车联网系统中发挥着举足轻重的作用。它不仅能够为驾驶员提供实时的路况和交通信息，帮助他们做出更为精准的驾驶决策，同时也为乘客带来了丰富的视觉体验和娱乐内容。然而，在车辆高速行驶的过程中，网络延迟、带宽限制和数据冗余等问题时常困扰着视频数据的传输。这些问题导致预警和警示信息无法及时、准确地传达给驾驶员和乘客，从而增加了交通事故的风险。With the booming development of Internet of Vehicles technology, video data plays a vital role in the Internet of Vehicles system. It not only provides drivers with real-time road conditions and traffic information to help them make more accurate driving decisions, but also brings rich visual experience and entertainment content to passengers. However, when vehicles are driving at high speeds, network delays, bandwidth limitations, and data redundancy often plague the transmission of video data. These problems result in the inability to convey early warning and warning information to drivers and passengers in a timely and accurate manner, thereby increasing the risk of traffic accidents.
为了满足不同场景下高质量、高带宽视频数据的传输需求，研究者们不断探索新的技术手段。其中，可扩展视频编码(Scalable Video Coding,SVC)技术因其独特的优势而备受关注。SVC技术允许一个视频生成多种不同维度的码流，这些码流可以根据不同的网络环境和用户需求进行灵活调整，从而实现了视频数据的高效传输。In order to meet the transmission requirements of high-quality, high-bandwidth video data in different scenarios, researchers are constantly exploring new technical means. Among them, Scalable Video Coding (SVC) technology has attracted much attention due to its unique advantages. SVC technology allows a video to generate multiple streams of different dimensions, which can be flexibly adjusted according to different network environments and user needs, thereby achieving efficient transmission of video data.
然而，SVC技术的应用也面临着一些挑战。由于SVC生成的码流主要在应用层(APP层)中生成，缺乏与其他网络层的动态控制机制，这使得视频数据的传输效率和质量无法得到进一步优化。However, the application of SVC technology also faces some challenges. Since the bitstream generated by SVC is mainly generated in the application layer (APP layer), there is a lack of dynamic control mechanism with other network layers, which makes it impossible to further optimize the transmission efficiency and quality of video data.
因此，如何开发出一种无线动态视频的跨层传输方法，通过优化SVC技术与网络层的交互，实现视频数据的快速、稳定传输，已成为当前本领域技术人员亟待攻克的技术难题。Therefore, how to develop a cross-layer transmission method for wireless dynamic video and achieve fast and stable transmission of video data by optimizing the interaction between SVC technology and the network layer has become a technical problem that technicians in this field urgently need to overcome.
相关术语解释：Explanation of relevant terms:
可分级视频编码(Scalable Video Coding，SVC)：传统视频编码的延伸，具有可伸缩和可分层的特点，可以在帧率、分辨率、质量上进行划分，一次编码可以输出多层码流，分为基本层和增强层，适用不同的终端和网络状况。Scalable Video Coding (SVC): An extension of traditional video coding, it is scalable and layerable. It can be divided into frame rate, resolution, and quality. One encoding can output multiple layers of code streams, divided into a basic layer and an enhancement layer, suitable for different terminals and network conditions.
基础层(Base Layer，BL)：提供基本图像质量的数据流。Base Layer (BL): A data stream that provides basic image quality.
增强层(Enhancement Layer，EL)：提供在基础层的基础之上重构的更高图像质量所需的数据流。Enhancement Layer (EL): Provides the data stream required to reconstruct higher image quality based on the base layer.
编码：由于视频信息中存在冗余，因此可以在保证视频质量的前提下，减少用来表征视频信息的比特数。Coding: Since there is redundancy in video information, the number of bits used to represent the video information can be reduced while ensuring video quality.
冗余：图像的各像素数据之间存在着极强的相似性，因此视频信息中包含着大量重复或多余的信息。Redundancy: There is a strong similarity between the pixel data of the image, so the video information contains a lot of repeated or redundant information.
NALU数据包；Network Abstraction Layer Unit,网络抽象层单元。NALU data packet; Network Abstraction Layer Unit, network abstraction layer unit.
功分复用(Power Division Multiple,PDM)：将功率域划分为多个功率段，每个功率段之间满足线性可加性，发射端根据合理的功率控制算法同时同频地发射功率域的叠加信号，接收端通过利用多用户检测技术(SIC接收机)完成多个数据流的分离和解调。Power Division Multiplexing (PDM): The power domain is divided into multiple power segments, each of which satisfies linear additivity. The transmitter transmits the superimposed signals of the power domains simultaneously and at the same frequency according to a reasonable power control algorithm, and the receiver uses multi-user detection technology (SIC receiver) to separate and demodulate multiple data streams.
正交相移键控(QPSK)：又称四相移相键控，通过转换或调制来传达数据的调制方法，基准信号(载波)的定相，QPSK每个符号能够进行两位编码，以格雷编码的方式显示在图形上以最小化误码率。Quadrature Phase Shift Keying (QPSK): Also known as Quadrature Phase Shift Keying, a modulation method that conveys data by shifting or modulating the phase of a reference signal (carrier). QPSK is able to encode two bits per symbol and is displayed on a graph in a Gray-coded manner to minimize the bit error rate.
发明内容Summary of the invention
本发明的目的在于提供一种无线动态视频的跨层传输方法，以克服现有技术中由于SVC技术缺乏与其他网络层的动态控制机制，导致视频数据的传输效率和质量无法得到优化的问题。The purpose of the present invention is to provide a cross-layer transmission method for wireless dynamic video to overcome the problem in the prior art that the transmission efficiency and quality of video data cannot be optimized due to the lack of dynamic control mechanism with other network layers in SVC technology.
本发明通过下述技术方案来解决上述技术问题：The present invention solves the above technical problems through the following technical solutions:
一种无线动态视频的跨层传输方法，包括以下步骤：A cross-layer transmission method for wireless dynamic video, comprising the following steps:
S1：设置输入视频信号的GoP长度，使GoP的个数为整数；在APP层对视频信号进行编码，所述编码采用质量可分级视频编码；进行码流提取，获取PHY层的传输速率，使编码后的视频信号的总码率小于等于PHY层的传输速率；编码后，每帧视频信号包括一层基础层和多层增强层，将每层基础层、增强层封装为NALU数据包；S1: Set the GoP length of the input video signal so that the number of GoPs is an integer; encode the video signal at the APP layer, and the encoding adopts quality scalable video encoding; extract the bitstream and obtain the transmission rate of the PHY layer, so that the total bit rate of the encoded video signal is less than or equal to the transmission rate of the PHY layer; after encoding, each frame of the video signal includes a base layer and multiple enhancement layers, and each base layer and enhancement layer is encapsulated as a NALU data packet;
S2：在APP层设置NALU数据包的优先级，其中，基础层NALU数据包的优先级高于增强层NALU数据包的优先级，增强层的层数越低，增强层NALU数据包的优先级越高；S2: Set the priority of the NALU data packet at the APP layer, where the priority of the base layer NALU data packet is higher than that of the enhancement layer NALU data packet. The lower the number of enhancement layers, the higher the priority of the enhancement layer NALU data packet;
S3：在DLL层采用功分复用传输技术，根据每帧视频信号基础层和增强层的层数划分功率段，每一层为一个功率段；根据步骤S2设置NALU数据包的优先级分配每个功率段的功率；其中，NALU数据包的优先级越高，分配的功率越大；S3: using power division multiplexing transmission technology at the DLL layer, dividing the power segment according to the number of layers of the base layer and the enhancement layer of each frame of the video signal, each layer is a power segment; setting the priority of the NALU data packet according to step S2 to allocate the power of each power segment; wherein, the higher the priority of the NALU data packet, the greater the allocated power;
S4：在PHY层设置调制方式，对每帧视频信号的NALU数据包采用多级QPSK调制方案，得到调制后的视频信号；所述QPSK调制的级数等于每帧视频信号功率段的个数，即每帧视频编码的层数；S4: Setting the modulation mode at the PHY layer, adopting a multi-level QPSK modulation scheme for the NALU data packet of each frame of the video signal to obtain a modulated video signal; the number of levels of the QPSK modulation is equal to the number of power segments of each frame of the video signal, that is, the number of layers of each frame of video encoding;
S5：把调制后的视频信号通过射频传输至无线信道；S5: transmitting the modulated video signal to a wireless channel via radio frequency;
S6：接收端接收无线信道的视频信号，进行解调、解功分复用、码流合并及视频解码，而后输出视频信号。S6: The receiving end receives the video signal from the wireless channel, performs demodulation, demultiplexing, code stream merging and video decoding, and then outputs the video signal.
进一步地，步骤S1中，所述质量可分级视频编码采用中粒度质量可分级性。Furthermore, in step S1, the quality scalable video encoding adopts medium-granularity quality scalability.
进一步地，步骤S1中，所述编码为基于H.264和H.265编码标准中的一种。Furthermore, in step S1, the encoding is based on one of the H.264 and H.265 encoding standards.
进一步地，步骤S1中，所述GoP长度等于视频信号的帧数。Furthermore, in step S1, the GoP length is equal to the number of frames of the video signal.
进一步地，步骤S4中，所述调制方式为正交相移键控。Furthermore, in step S4, the modulation method is quadrature phase shift keying.
进一步地，步骤S4中，所述星座映射为格雷映射。Further, in step S4, the constellation mapping is Gray mapping.
一种无线动态视频的跨层传输装置，包括APP层处理模块、DLL层处理模块、PHY层处理模块以及无线传输模块；其中，APP层处理模块：用于对输入的视频信号进行编码、解码及设置NALU数据包的优先级；DLL层处理模块：用于对NALU数据包进行功分复用、解功分复用及功率分配；PHY层处理模块：用于对NALU数据包进行调制及解调；无线传输模块：用于进行射频传输。A cross-layer transmission device for wireless dynamic video includes an APP layer processing module, a DLL layer processing module, a PHY layer processing module and a wireless transmission module; wherein the APP layer processing module is used to encode and decode input video signals and set the priority of NALU data packets; the DLL layer processing module is used to perform power division multiplexing, demultiplexing and power distribution on NALU data packets; the PHY layer processing module is used to modulate and demodulate NALU data packets; and the wireless transmission module is used to perform radio frequency transmission.
一种计算机装置设备系统，包括存储器、处理器及存储在存储器上的计算机程序，所述处理器执行所述计算机程序以实现上述无线动态视频的跨层传输方法的步骤。A computer device system includes a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-mentioned cross-layer transmission method of wireless dynamic video.
一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时，实现上述无线动态视频的跨层传输方法的步骤。A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned cross-layer transmission method of wireless dynamic video are implemented.
一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现上述无线动态视频的跨层传输方法的步骤。A computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned cross-layer transmission method of wireless dynamic video are implemented.
与现有技术相比，本发明的积极进步效果在于：Compared with the prior art, the positive and progressive effects of the present invention are:
本发明方法通过利用PHY层的调制方式与无线信道的实时传输状态，结合DLL层的功率分配，以及APP层的编码结构，并充分利用NALU数据包的优先级特性，实现了在实际传输过程中的传输速率可控性。不仅确保了每路数据流的比特正交性和公平性，而且有更低的复杂度。同时还实现了PHY层、DLL层与APP层之间的动态联合控制，保证了视频数据的稳定传输，从而显著提高了视频的服务质量，满足用户对高质量视频服务的需求。将APP层、DLL层和PHY层三层联合跨层优化，使得传输系统的性能整体达到最优，保障视频高效传输。The method of the present invention realizes the controllability of the transmission rate in the actual transmission process by utilizing the modulation mode of the PHY layer and the real-time transmission state of the wireless channel, combining the power allocation of the DLL layer and the coding structure of the APP layer, and making full use of the priority characteristics of the NALU data packet. It not only ensures the bit orthogonality and fairness of each data stream, but also has lower complexity. At the same time, it also realizes the dynamic joint control between the PHY layer, the DLL layer and the APP layer, ensuring the stable transmission of video data, thereby significantly improving the service quality of the video and meeting the user's demand for high-quality video services. The APP layer, the DLL layer and the PHY layer are jointly cross-layer optimized, so that the overall performance of the transmission system is optimized, and efficient video transmission is guaranteed.
进一步地，本发明方法在APP层利用NALU数据包作为最小单元代替传统的以层作为最小单元，NALU数据包可以在不同层之间自由传递，无需受到固定层级的限制，使得数据传输和处理更加灵活，从而适应各种复杂的网络环境和应用需求。而且在任意GoP结构下，均可以合理分析和利用NALU的优先级，以实现更加精准和高效的功率段匹配。同时，引入速率控制机制，确保在不同网络条件下能够灵活调整传输速率，以适应多样化的车联网传输环境。Furthermore, the method of the present invention uses NALU data packets as the smallest unit at the APP layer instead of the traditional layer as the smallest unit. NALU data packets can be freely transmitted between different layers without being restricted by fixed levels, making data transmission and processing more flexible, thereby adapting to various complex network environments and application requirements. Moreover, under any GoP structure, the priority of NALU can be reasonably analyzed and utilized to achieve more accurate and efficient power segment matching. At the same time, a rate control mechanism is introduced to ensure that the transmission rate can be flexibly adjusted under different network conditions to adapt to the diverse Internet of Vehicles transmission environment.
进一步地，本发明方法在DLL层采用功分复用传输技术，通过精准划分功率段来进行功率控制，并且结合了PDM技术中不同功率段传输可靠性不同的特点，去匹配NALU的优先级，确保高优先级的NALU能够获得更可靠的传输，从而更有效地保障接收端的视频质量。进一步地，结合NALU数据包的优先级特性与无线信道的传输状态进行功率分配，当信道质量上乘时，能够高效发送符合传输速率的全部码流信息，确保数据的完整性和实时性；当信道质量出现波动或下降时，能够智能地调整并优化策略，仅发送基础层以及优先级较高的增强层数据，从而在复杂多变的网络环境中，始终保证视频信号的正确传输，并最大程度地维持视频的服务质量。Furthermore, the method of the present invention adopts power division multiplexing transmission technology at the DLL layer, performs power control by accurately dividing the power segment, and combines the characteristics of different transmission reliability of different power segments in PDM technology to match the priority of NALU, ensuring that high-priority NALU can obtain more reliable transmission, thereby more effectively guaranteeing the video quality of the receiving end. Furthermore, power allocation is performed in combination with the priority characteristics of the NALU data packet and the transmission status of the wireless channel. When the channel quality is excellent, all code stream information that meets the transmission rate can be efficiently sent to ensure the integrity and real-time nature of the data; when the channel quality fluctuates or decreases, the strategy can be intelligently adjusted and optimized to only send the base layer and the enhanced layer data with higher priority, thereby always ensuring the correct transmission of the video signal in a complex and changeable network environment, and maintaining the video service quality to the greatest extent.
进一步地，本发明方法在PHY层对NALU数据包均采用QPSK的调制方式，QPSK使用的符号数较少，复杂度和错误率较低；且每个符号的相位差异大，具有更高的灵敏度，因此抗干扰能力更强。若将每帧视频数据封装为L层NALU数据包，则采用L级QPSK叠加的方式进行调制。不仅保证了传输的可靠性，还避免了高阶调制带来的复杂性和不稳定性。通过这种策略，能够在不同信道条件下适应不同NALU的优先级，确保更高优先级的NALU具有更低的误码率，从而提高传输效率。Furthermore, the method of the present invention adopts QPSK modulation for NALU data packets at the PHY layer. QPSK uses fewer symbols, has lower complexity and error rate; and the phase difference of each symbol is large, has higher sensitivity, and therefore has stronger anti-interference ability. If each frame of video data is encapsulated as an L-layer NALU data packet, it is modulated by L-level QPSK superposition. Not only does it ensure the reliability of transmission, but it also avoids the complexity and instability caused by high-order modulation. Through this strategy, it is possible to adapt to the priorities of different NALUs under different channel conditions, ensuring that NALUs with higher priorities have lower bit error rates, thereby improving transmission efficiency.
进一步地，本发明方法在传输过程中仅利用一阶边的参考关系代替所有的高阶边的参考关系，可以降低传输过程中的复杂度，实现视频编码与传输的高效联合。Furthermore, the method of the present invention only uses the reference relationship of the first-order edges to replace the reference relationship of all high-order edges during the transmission process, which can reduce the complexity of the transmission process and achieve efficient combination of video encoding and transmission.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
说明书附图用来提供对本发明的进一步理解，构成本发明的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。The drawings in the specification are used to provide further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
图1为本发明采用H.264编码格式且码率固定时，视频1的GoP与PSNR的关系图；FIG1 is a graph showing the relationship between GoP and PSNR of video 1 when the present invention adopts the H.264 encoding format and the bit rate is fixed;
图2为本发明采用H.264编码格式且码率固定时，视频2的GoP与PSNR的关系图；FIG2 is a graph showing the relationship between GoP and PSNR of video 2 when the present invention adopts the H.264 encoding format and the bit rate is fixed;
图3为本发明采用H.264编码格式且码率固定时，视频3的GoP与PSNR的关系图；FIG3 is a graph showing the relationship between GoP and PSNR of video 3 when the present invention adopts the H.264 encoding format and the bit rate is fixed;
图4为本发明实施例中视频1的层间预测参考关系图；FIG4 is a diagram of inter-layer prediction reference relationships of video 1 according to an embodiment of the present invention;
图5为本发明采用H.265编码格式且码率固定时，视频1的GoP与PSNR的关系图；FIG5 is a graph showing the relationship between GoP and PSNR of video 1 when the present invention adopts the H.265 encoding format and the bit rate is fixed;
图6为本发明采用H.265编码格式且码率固定时，视频4的GoP与PSNR的关系图；FIG6 is a graph showing the relationship between GoP and PSNR of video 4 when the present invention adopts the H.265 encoding format and the bit rate is fixed;
图7为遍历两帧两层的bus视频的总功率结果图；Figure 7 is a graph showing the total power results of traversing two frames and two layers of bus video;
图8为本发明采用H.264编码格式时，视频1的NALU正确传输数量图；FIG8 is a diagram showing the correct transmission quantity of NALUs of Video 1 when the present invention adopts the H.264 encoding format;
图9为本发明采用H.264编码格式时，视频1的NALU正确解码比率图；FIG9 is a diagram showing the correct decoding ratio of NALU of Video 1 when the present invention adopts the H.264 encoding format;
图10为本发明采用H.265编码格式时，视频1的NALU正确传输数量图；FIG10 is a diagram showing the correct transmission quantity of NALUs of Video 1 when the present invention adopts the H.265 encoding format;
图11为本发明采用H.265编码格式时，视频1的NALU正确解码比率图；FIG11 is a diagram showing the correct decoding ratio of NALU of Video 1 when the present invention adopts the H.265 encoding format;
图12为本发明所述两种参考关系框架图；FIG12 is a diagram showing two reference relationship frameworks of the present invention;
图13为不同参考关系在不同位置出错可解码NALU数量的结果统计图，其中，(a)为一阶参考关系，(b)为二阶参考关系；FIG13 is a statistical diagram of the number of NALUs that can be decodable with errors at different positions under different reference relations, where (a) is a first-order reference relation and (b) is a second-order reference relation;
图14为本发明方法的流程图；FIG14 is a flow chart of the method of the present invention;
图15为本发明方法的框图。FIG15 is a block diagram of the method of the present invention.
具体实施方式Detailed ways
以下结合附图对本发明做进一步的详细说明，所述是对本发明的解释而不是限定。The present invention is further described in detail below in conjunction with the accompanying drawings, which are intended to explain the present invention rather than to limit it.
一种无线动态视频的跨层传输方法，包括以下步骤：A cross-layer transmission method for wireless dynamic video, comprising the following steps:
S1：利用PHY层(Physical Layer,物理层)反馈的传输速率去控制可分级视频编码的配置，控制方法为：使编码后的视频数据码流速率匹配或者小于等于PHY层的传输速率；S1: Using the transmission rate fed back by the PHY layer (Physical Layer) to control the configuration of scalable video coding, the control method is: making the encoded video data stream rate match or less than or equal to the transmission rate of the PHY layer;
可分级视频编码对含有M帧的输入视频进行编码时，将每帧视频数据编码为L层，包括1层基础层(Base Layer，BL)和L-1层增强层(Enhancement Layer，EL)，进一步将第i帧的第j层视频数据封装成NALU(Network Abstraction Layer Unit,网络抽象层单元)数据包，每层NALU数据包形成一个数据流，利用集合R表示某一帧视频数据(该帧视频数据含有L层NALU数据包)的服务速率为：When scalable video coding encodes an input video containing M frames, each frame of video data is encoded into L layers, including 1 base layer (Base Layer, BL) and L-1 enhancement layer (Enhancement Layer, EL), and the j-th layer of video data of the i-th frame is further encapsulated into a NALU (Network Abstraction Layer Unit) data packet. Each layer of NALU data packet forms a data stream. The set R is used to represent the service rate of a frame of video data (the frame of video data contains L layers of NALU data packets):
R＝{R1，R2，…RL}R＝{R1 , R2 , …RL }
S2：根据NALU数据包在GoP结构中的位置来决定NALU数据包的优先级，对于第i帧第j层的NALU数据包，i一定时，j越小，对应的NALU数据包的优先级越高；S2: Determine the priority of the NALU data packet according to its position in the GoP structure. For the NALU data packet of the jth layer of the i-th frame, when i is constant, the smaller j is, the higher the priority of the corresponding NALU data packet;
S3：采用功分复用(Power Division Multiple,PDM)传输技术，根据编码层数划分功率段的个数，并由NALU数据包的优先级决定功率段的大小；S3: Power Division Multiplex (PDM) transmission technology is used to divide the number of power segments according to the number of coding layers, and the size of the power segment is determined by the priority of the NALU data packet;
功率段的划分具体为：若对M帧视频进行编码，将每帧视频编码为L层，则功分复用时共划分为MxL个功率段。其中，如果第i帧第j层的NALU数据包优先级越高，那么该NALU数据包所分配的功率越大。The power segment is specifically divided as follows: if M frames of video are encoded, and each frame of video is encoded as L layers, then it is divided into MxL power segments during power division multiplexing. Among them, if the NALU data packet of the jth layer of the i-th frame has a higher priority, the power allocated to the NALU data packet is greater.
设总的发射功率为E,对一帧视频L个数据流进行功率控制，按照每一个数据流的优先级进行功率控制，其中功率因子表示为：Assume that the total transmission power is E, and perform power control on L data streams of a frame of video according to the priority of each data stream, where the power factor is expressed as:
a＝{a1,a2…aL}a＝{a1 ,a2 …aL }
则每个数据流的发射功率为：A＝{a1E,a2E,…,aLE}。将L个数据流分别乘以对应的发射功率之后，进行叠加编码并发射出去。在此过程中，叠加信号为x，发射信号可以表示为：Then the transmission power of each data stream is: A = {a1 E, a2 E, …, aL E}. After multiplying the L data streams by the corresponding transmission power, they are superimposed and encoded and transmitted. In this process, the superimposed signal is x, and the transmitted signal can be expressed as:
S4：根据DLL层(Data Link Layer,数据链路层)的反馈信息对第i帧第j层NALU数据包作基带调制，调制方式为QPSK。S4: baseband modulate the j-layer NALU data packet of the i-th frame according to the feedback information of the DLL layer (Data Link Layer), and the modulation mode is QPSK.
每帧视频功率段的个数决定了该帧视频数据在PHY层采取几重星座映射，即PHY层星座映射的重数等于每帧视频信号功率段的个数。The number of power segments in each frame of video determines the number of constellation mappings adopted by the frame of video data at the PHY layer, that is, the number of constellation mappings at the PHY layer is equal to the number of power segments in each frame of video signal.
若将每帧视频数据封装为L层NALU数据包，则对每帧视频进行L重星座映射，映射方法可以为格雷映射；If each frame of video data is encapsulated as an L-layer NALU data packet, L-fold constellation mapping is performed on each frame of video, and the mapping method may be Gray mapping;
假设发射端的数据流为x＝{x1,x2…xL}，总发射功率为Pt，功率控制因子为：a＝{a1,a2…aL}，且分配的功率段满足：a1>a2>…>aL,则每个数据流的发射功率为：Assume that the data stream at the transmitting end is x = {x1 , x2 …xL }, the total transmit power is Pt , the power control factor is: a = {a1 , a2 …aL }, and the allocated power range satisfies: a1 >a2 >…>aL , then the transmit power of each data stream is:
A＝{a1Pt,a2Pt,…,aLPt}。A={a1 Pt ,a2 Pt ,…,aL Pt }.
所有层的数据流I＝{I1，I2,…IL}进行QPSK调制得出传输流x＝{x1,x2,…xL}The data streams of all layers I = {I1 , I2 , ... IL } are modulated by QPSK to obtain the transmission stream x = {x1 , x2 , ... xL }
则叠加后的信号表示为：The superimposed signal is expressed as:
S5：把调制后的信号进行射频传输到无线信道。S5: Transmit the modulated signal to the wireless channel via radio frequency.
S6：接收端实现数据的接收以及还原，主要包括解调、解功分复用、码流合并、视频解码和视频输出五个过程。S6: The receiving end realizes data reception and restoration, which mainly includes five processes: demodulation, power division multiplexing, bit stream merging, video decoding and video output.
假设信道为AWGN信道，设AWGN方差为σ2，信道增益为hn，则SIC一级接收机的输入为：Assuming that the channel is an AWGN channel, the AWGN variance is σ2 , and the channel gain ishn , the input of the SIC primary receiver is:
SIC一级接收机接收到y′信号后，首先进行解调得出功率段最高的I1′信号，此时将x2,…,xL信号均看作干扰信号。其次将I1′信号进行调制得出x1′并从y′中分离出来，由此SIC二级接收机的输入为：After the SIC primary receiver receives the y′ signal, it first demodulates it to obtain the I1 ′ signal with the highest power range. At this time, the x2 ,…,xL signals are all regarded as interference signals. Secondly, the I1 ′ signal is modulated to obtain x1 ′ and separated from y′. Therefore, the input of the SIC secondary receiver is:
对二级接收机重复一级接收机的操作，可以得出功率段二级高的I2′信号，此时其他信号均为干扰信号。以此类推可以得出N级接收机时的输入信号为：Repeating the operation of the first-level receiver for the second-level receiver can obtain the I2 ′ signal with the second-highest power range. At this time, other signals are interference signals. By analogy, the input signal of the N-level receiver can be obtained as follows:
对此信号进行解调即可得出功率段最小的IN′信号。经过上述步骤，即可在接收端得到发射端发射的数据流表示为：x′＝{x1′,x2′…xL′}。By demodulating this signal, the IN ′ signal with the minimum power range can be obtained. After the above steps, the data stream transmitted by the transmitting end can be obtained at the receiving end and is expressed as: x′＝{x1 ′,x2 ′…xL ′}.
一种无线动态视频的跨层传输装置，包括APP层处理模块、在DLL层和PHY层进行调整的DLL层处理模块与PHY层处理模块以及无线传输模块，其中：APP层处理模块：包括视频编码过程以及对NALU数据包划分优先级的过程；DLL层处理模块：主要采用功分复用传输机制；PHY层处理模块：调制方式为QPSK，作联合星座映射，映射方法可以为格雷映射；无线传输模块：进行射频传输。A cross-layer transmission device for wireless dynamic video includes an APP layer processing module, a DLL layer processing module and a PHY layer processing module for adjusting at the DLL layer and the PHY layer, and a wireless transmission module, wherein: the APP layer processing module includes a video encoding process and a process of prioritizing NALU data packets; the DLL layer processing module mainly adopts a power division multiplexing transmission mechanism; the PHY layer processing module uses a modulation mode of QPSK and performs joint constellation mapping, and the mapping method can be Gray mapping; the wireless transmission module performs radio frequency transmission.
为使本发明所解决的技术问题、技术方案及有益效果更加清晰，以下结合附图及实施例，对本发明作进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本发明，并不限定本发明，详细说明如下。In order to make the technical problems, technical solutions and beneficial effects solved by the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the present invention. The detailed description is as follows.
视频1：BUS_176x144_30_orig_01.yuv；Video 1: BUS_176x144_30_orig_01.yuv;
视频2：akiyo_176×_30Frames.yuv；Video 2: akiyo_176×_30Frames.yuv;
视频3：akiyo_176×_300Frames.yuv；Video 3: akiyo_176×_300Frames.yuv;
视频4：BUS_176x144_75_orig_01.yuv。Video 4: BUS_176x144_75_orig_01.yuv.
首先，对GoP长度与视频质量关系进行说明：First, the relationship between GoP length and video quality is explained:
用BUS_176x144_30_orig_01.yuv(视频1)、akiyo_176×_30Frames.yuv(视频2)两种视频来表示帧间差异大和帧间差异小的视频，用akiyo_176×_30Frames.yuv(视频2)、akiyo_176×_300Frames.yuv(视频3)两种视频来表示帧数不同的视频，下面对三个视频进行分析。Use BUS_176x144_30_orig_01.yuv (video 1) and akiyo_176×_30Frames.yuv (video 2) to represent videos with large and small frame differences, and use akiyo_176×_30Frames.yuv (video 2) and akiyo_176×_300Frames.yuv (video 3) to represent videos with different frame numbers. The three videos are analyzed below.
为使码流在信道发生变化的情况下依然可以使视频正确传输，本发明选择质量可分级的编码方式，编码器为基于H.264和H.265编码标准的编码器。在信道质量好的情况下，发送符合传输速率的全部码流信息，在信道质量变差时，可发送基础层以及优先级较高的增强层，保证视频的正确传输并尽可能大的保证视频的质量。在码率固定的情况下，编码框架与量化参数两个因素会对视频质量产生影响。下面分别利用H.264以及H.265进行码率控制得出结论，为后续固定码率选择合理的GoP长度。In order to ensure that the bit stream can still transmit the video correctly when the channel changes, the present invention selects a quality-scalable encoding method, and the encoder is an encoder based on the H.264 and H.265 encoding standards. When the channel quality is good, all bit stream information that meets the transmission rate is sent. When the channel quality deteriorates, the base layer and the enhanced layer with a higher priority can be sent to ensure the correct transmission of the video and to ensure the quality of the video as much as possible. When the bit rate is fixed, the two factors of coding framework and quantization parameter will affect the video quality. The following uses H.264 and H.265 for bit rate control to draw conclusions and select a reasonable GoP length for the subsequent fixed bit rate.
采用H.264进行编码：视频1的视频帧间差异大，帧数为30，视频编码为三层，基础层码率设置为200kbps，增强层1码率设置为500kbps，增强层2码率设置为1000kbps，则视频1最大的码率为1000kbps，视频帧率为30帧/秒。视频2的视频帧间差异小，帧数为30，视频编码为三层，基础层码率固定为100kbps，增强层1码率固定为200kbps，增强层2码率固定为500kbps，则视频2最大的码率为500kbps，视频帧率为30帧/秒。视频3的帧数为300，视频编码为三层，基础层码率固定为100kbps，增强层1码率固定为200kbps，增强层2码率固定为500kbps，则视频2最大的码率为500kbps，视频帧率为30帧/秒。设置三个视频的误差允许范围最大迭代次数为10次，采用同样大小的量化参数(QP)，在不同的GoP设置下对视频进行编码，并且GoP长度的取值应符合视频完整GoP的个数。同时，为避免没有得到视频码率控制的最优码流，应尽可能保证视频迭代次数低于10，故对视频每一层QP大小的预估以及预编码来保证码率控制的精准性。H.264 encoding is used: Video 1 has a large difference between video frames, the number of frames is 30, the video encoding is three-layer, the base layer bit rate is set to 200kbps, the enhancement layer 1 bit rate is set to 500kbps, and the enhancement layer 2 bit rate is set to 1000kbps. The maximum bit rate of Video 1 is 1000kbps, and the video frame rate is 30 frames/second. Video 2 has a small difference between video frames, the number of frames is 30, the video encoding is three-layer, the base layer bit rate is fixed at 100kbps, the enhancement layer 1 bit rate is fixed at 200kbps, and the enhancement layer 2 bit rate is fixed at 500kbps. The maximum bit rate of Video 2 is 500kbps, and the video frame rate is 30 frames/second. The number of frames of video 3 is 300, and the video is encoded in three layers. The base layer bitrate is fixed at 100kbps, the enhancement layer 1 bitrate is fixed at 200kbps, and the enhancement layer 2 bitrate is fixed at 500kbps. The maximum bitrate of video 2 is 500kbps, and the video frame rate is 30 frames per second. The maximum number of iterations of the error tolerance range of the three videos is set to 10 times. The same quantization parameter (QP) is used to encode the videos under different GoP settings, and the GoP length value should be consistent with the number of complete GoPs of the video. At the same time, in order to avoid not obtaining the optimal bitstream for video bitrate control, the number of video iterations should be kept below 10 as much as possible. Therefore, the QP size of each layer of the video is estimated and pre-encoded to ensure the accuracy of bitrate control.
由图1、2及3可以得出，在进行码率控制编码任意视频时，GoP长度越长，PSNR(PeakSignal-to-Noise Ratio，峰值信噪比)越大，视频质量越好。It can be concluded from Figures 1, 2 and 3 that when encoding any video with bit rate control, the longer the GoP length, the greater the PSNR (Peak Signal-to-Noise Ratio), and the better the video quality.
采用H.265进行编码：将视频1编码为三层，基础层码率设置为200kbps，增强层1码率设置为500kbps，增强层2码率设置为1000kbps，则视频1最大的码率为1000kbps，视频帧率为30帧/秒。设置视频的误差允许范围最大迭代次数为10次，GoP长度的取值应符合视频完整GoP的个数来，为避免没有得到视频码率控制的最优码流，应尽可能保证视频迭代次数低于10，故对视频每一层预编码来保证码率控制的精准性。视频的参考关系如图4所示，在后续所有的编码框图中，每个方块表示视频第i帧第j层的数据，箭头表示参考的关系以及方向。此框图为一个三层质量可分级，其中EL2层数据仅利用EL1层的数据进行层间预测。H.265 is used for encoding: Video 1 is encoded into three layers, the base layer bit rate is set to 200kbps, the enhancement layer 1 bit rate is set to 500kbps, and the enhancement layer 2 bit rate is set to 1000kbps. The maximum bit rate of Video 1 is 1000kbps, and the video frame rate is 30 frames per second. The maximum number of iterations of the error tolerance range of the video is set to 10 times, and the value of the GoP length should be consistent with the number of complete GoPs of the video. In order to avoid not obtaining the optimal bit stream for video bit rate control, the number of video iterations should be kept as low as possible. Therefore, each layer of the video is pre-encoded to ensure the accuracy of bit rate control. The reference relationship of the video is shown in Figure 4. In all subsequent encoding block diagrams, each block represents the data of the jth layer of the i-th frame of the video, and the arrow represents the reference relationship and direction. This block diagram is a three-layer quality scalable, in which the EL2 layer data only uses the EL1 layer data for inter-layer prediction.
由图5可知，在码率固定的情况下，GoP越长，视频质量越好。As shown in Figure 5, when the bit rate is fixed, the longer the GoP, the better the video quality.
视频BUS_176x144_75_orig_01.yuv(视频4)的帧数为75帧，其余参数与视频1的编码参数设置相同，对视频4进行GoP从5到75间隔为10来取值进行实验，由图6可知，在码率固定的情况下，GoP越长，视频质量越好。参见图6，可以看到在GoP为35到65之间有波动，这是由于这几个GOP长度所对应的I帧个数相同，即最后一个GoP均不是完整的GoP所以造成的波动，因此在选择GoP长度时，尽可能选取完整的长度，使视频GoP的个数为整数。The number of frames of video BUS_176x144_75_orig_01.yuv (video 4) is 75, and the other parameters are the same as the encoding parameter settings of video 1. The GoP of video 4 is experimented with the interval of 10 from 5 to 75. As shown in Figure 6, when the bit rate is fixed, the longer the GoP, the better the video quality. Referring to Figure 6, it can be seen that there is a fluctuation between GoP 35 and 65. This is because the number of I frames corresponding to these GOP lengths is the same, that is, the last GoP is not a complete GoP, so the fluctuation is caused. Therefore, when selecting the GoP length, try to select the complete length so that the number of video GoPs is an integer.
在采用H.264进行速率控制时，每一层均采用同样大小的量化参数(QP)，因此在固定编码结构以及参考关系后，得到的码率会与要求码率存在误差。虽然误差可以进行设置，通过多次迭代可将误差尽量控制在允许范围内，但并不精准。而采用H.265编码进行速率控制时，会动态调整每一帧每一层的量化参数大小，来得出最符合速率要求的码流。虽然相较于H.264，采用H.265的速率控制更佳准确，误差相对很小。但同样需要预编码来确定第一帧的每层QP的大小，保证速率控制更加精准，视频质量更好。When using H.264 for rate control, each layer uses the same quantization parameter (QP). Therefore, after fixing the coding structure and reference relationship, the obtained bit rate will have an error with the required bit rate. Although the error can be set and the error can be controlled within the allowable range as much as possible through multiple iterations, it is not accurate. When using H.265 encoding for rate control, the size of the quantization parameter of each layer of each frame will be dynamically adjusted to obtain the bit stream that best meets the rate requirements. Although compared with H.264, the rate control of H.265 is more accurate and the error is relatively small. However, pre-coding is also required to determine the size of the QP of each layer of the first frame to ensure more accurate rate control and better video quality.
实施例一Embodiment 1
选用基于H.264标准的编码器，同时，为满足信道变化的情况下依然可以使视频正确的传输，采用MGS层分级方式(Medium-grain Quality Scalability，中粒度质量可分级)，在APP层对含有30帧的视频1进行编码，将每帧视频数据编码为3层，分别为1层基础层、第一层增强层及第2层增强层，进一步将每层视频数据封装成NALU数据包，其中，基础层NALU数据包的优先级高于增强层NALU数据包的优先级，第一层增强层NALU数据包的优先级高于第二层增强层NALU数据包的优先级。An encoder based on the H.264 standard is selected. At the same time, in order to ensure that the video can be transmitted correctly when the channel changes, the MGS layer grading method (Medium-grain Quality Scalability) is adopted. The video 1 containing 30 frames is encoded at the APP layer, and each frame of video data is encoded into 3 layers, namely, a base layer, a first enhancement layer and a second enhancement layer. Each layer of video data is further encapsulated into a NALU data packet, wherein the priority of the base layer NALU data packet is higher than that of the enhancement layer NALU data packet, and the priority of the first enhancement layer NALU data packet is higher than that of the second enhancement layer NALU data packet.
利用调制相关的比特率cn代替香农容量rn，可以得出每层的传输速率，cn＝Bwlog2(Mn)，其中Mn为4。参数设置如下：Using the modulation-dependent bit rate cn instead of the Shannon capacity rn, the transmission rate of each layer can be obtained, cn =Bw log2 (Mn ), where Mn is 4. The parameters are set as follows:
由于在速率控制过程中，编码后的视频信号的总码率应该在最佳范围内小于等于PHY层的传输速率720Kbps，可以得出迭代结果如下：Since the total bit rate of the encoded video signal should be less than or equal to the transmission rate of the PHY layer 720Kbps within the optimal range during the rate control process, the iterative results are as follows:
BL:Rate＝243.4000 QP＝32.430248 PSNR＝31.8445BL:Rate＝243.4000 QP＝32.430248 PSNR＝31.8445
EL1:Rate＝444.0720 QP＝29.446204 PSNR＝33.6602EL1:Rate＝444.0720 QP＝29.446204 PSNR＝33.6602
EL2:Rate＝705.4400 QP＝29.500027 PSNR＝33.7162EL2:Rate＝705.4400 QP＝29.500027 PSNR＝33.7162
而在编码过程中，QP应为整数，且QP越大，视频码率越小，因此为满足需求，将QP进行向上取整，可以得出以下数据：During the encoding process, QP should be an integer, and the larger the QP, the smaller the video bit rate. Therefore, to meet the requirements, QP is rounded up, and the following data can be obtained:
BL:Rate＝241.2 QP＝33 PSNR＝31.8445 a＝0.5％BL:Rate=241.2 QP=33 PSNR=31.8445 a=0.5%
EL1:Rate＝441.8 QP＝30 PSNR＝33.6602 a＝-7.9％EL1:Rate=441.8 QP=30 PSNR=33.6602 a=-7.9%
EL2:Rate＝702.5 QP＝30 PSNR＝33.7162 a＝-2.4％％EL2:Rate=702.5 QP=30 PSNR=33.7162 a=-2.4%%
其中，a表示速率误差，a均保留三位小数。根据上述实验提取满足不同码率的码流文件。Wherein, a represents the rate error, and a is rounded to three decimal places. According to the above experiment, the bitstream files meeting different bitrates are extracted.
将得到的码流文件进行数据分离，在PHY层设置调制方式为正交相移键控(QPSK)，对NALU数据包进行基带信号调制；采用功分复用传输技术，根据编码层数划分功率段的个数，并由NALU数据包的优先级决定功率段的大小；NALU数据包的优先级越高，分配的功率越大，其中，基础层分配的功率最大，第一层增强层分配的功率次之，第二层增强层分配的功率较小。本实施例将一帧视频数据封装为3层NALU数据包，则叠加3重QPSK调制，将调制后的视频1通过射频传输至无线信道，接收端接收无线信道的视频信号，进行解调、解功分复用、码流合并及视频解码，而后输出视频信号。The obtained bitstream file is subjected to data separation, the modulation mode is set to quadrature phase shift keying (QPSK) at the PHY layer, and the NALU data packet is subjected to baseband signal modulation; the power division multiplexing transmission technology is adopted, the number of power segments is divided according to the number of coding layers, and the size of the power segment is determined by the priority of the NALU data packet; the higher the priority of the NALU data packet, the greater the power allocated, among which the base layer is allocated the largest power, the first layer of the enhancement layer is allocated the second largest power, and the second layer of the enhancement layer is allocated the smallest power. In this embodiment, a frame of video data is encapsulated into 3 layers of NALU data packets, and 3 layers of QPSK modulation are superimposed, and the modulated video 1 is transmitted to the wireless channel via radio frequency. The receiving end receives the video signal of the wireless channel, performs demodulation, power division multiplexing, bitstream merging and video decoding, and then outputs the video signal.
传输过程的参数设置如下：The parameters of the transmission process are set as follows:
通过改变SNR(Signal-to-Noise Ratio，信噪比，表示信号的功率与噪声的功率之间的比值)的大小，即改变基础层功率P1,第一层增强层功率P2及第二层增强层功率P3的大小来统计输出NALU数据包的正确解码的数量S，统计结果参见图8-9。By changing the size of SNR (Signal-to-Noise Ratio, which represents the ratio between the power of the signal and the power of the noise), that is, changing the size of the base layer powerP1 , the first layer enhancement layer powerP2 and the second layer enhancement layer powerP3 , the number S of correctly decoded output NALU data packets is counted. The statistical results are shown in Figures 8-9.
可以得出基于H.264编码下，3层30帧的NALU数据包成功解码的平均数量与功率的关系，即S与P1、P2及P3之间的关系。由图8和图9可以看出S(输出NALU的正确解码的数量)是P3的拟凹函数，又由于P1,P2均可以由P3表示，故S是P1、P2和P3的拟凹函数，即S是关于P1,P2,...Pn的拟凹函数，本发明方法采用H.264编码存在最优策略进行无线动态视频的跨层传输。It can be obtained that the average number of successfully decoded NALU packets of 3 layers and 30 frames under H.264 encoding is related to power, that is, the relationship between S and P1 , P2 and P3. It can be seen from Figures 8 and 9 that S (the number of correctly decoded output NALUs) is a quasi-concave function of P3 , and since P1 and P2 can both be represented by P3 , S is a quasi-concave function of P1 , P2 and P3 , that is, S is a quasi-concave function of P1 , P2 , ... Pn . The method of the present invention adopts the optimal strategy of H.264 encoding to perform cross-layer transmission of wireless dynamic video.
实施例2Example 2
选用基于H.265标准的编码器，同时，为满足信道变化的情况下依然可以使视频正确的传输，采用MGS层分级方式(Medium-grain Quality Scalability，中粒度质量可分级)，在APP层对含有30帧的视频1进行编码，将每帧视频数据编码为3层，分别为1层基础层、第一层增强层及第二层增强层，编码框架参见图4，视频第三层仅利用第二层进行预测，即第二层增强层通过第一层增强层进行预测。进一步将每层视频数据封装成NALU数据包，其中，基础层NALU数据包的优先级高于增强层NALU数据包的优先级，第一层增强层的优先级高于第二层增强层的优先级。An encoder based on the H.265 standard is selected. At the same time, in order to ensure that the video can be transmitted correctly when the channel changes, the MGS layer grading method (Medium-grain Quality Scalability) is adopted. The video 1 containing 30 frames is encoded at the APP layer, and each frame of video data is encoded into 3 layers, namely, 1 base layer, the first enhancement layer and the second enhancement layer. The encoding framework is shown in Figure 4. The third layer of the video is predicted only by the second layer, that is, the second enhancement layer is predicted by the first enhancement layer. Each layer of video data is further encapsulated into a NALU data packet, wherein the priority of the base layer NALU data packet is higher than that of the enhancement layer NALU data packet, and the priority of the first enhancement layer is higher than that of the second enhancement layer.
利用调制相关的比特率cn代替香农容量rn，可以得出每层的传输速率，cn＝Bwlog2(Mn)，其中Mn为4。参数设置如下：Using the modulation-dependent bit rate cn instead of the Shannon capacity rn, the transmission rate of each layer can be obtained, cn =Bw log2 (Mn ), where Mn is 4. The parameters are set as follows:
由于在速率控制过程中，编码后的视频信号的总码率应该小于等于PHY层的传输速率720Kbps，可以得出迭代结果如下(确定参考关系后每一帧每一层的QP均为动态调整且不相同，故不作统计)：Since the total bit rate of the encoded video signal should be less than or equal to the transmission rate of the PHY layer 720Kbps during the rate control process, the iterative results can be obtained as follows (after determining the reference relationship, the QP of each frame and each layer is dynamically adjusted and different, so no statistics are made):
BL:Rate＝240.088 PSNR＝33.2921 a＝0.036％BL:Rate=240.088 PSNR=33.2921 a=0.036%
EL1:Rate＝480.208 PSNR＝37.7444 a＝0.043％EL1:Rate=480.208 PSNR=37.7444 a=0.043%
EL2:Rate＝720.152 PSNR＝40.7982 a＝0.021％EL2:Rate=720.152 PSNR=40.7982 a=0.021%
由于速率控制后的结果均在传输速率要求附近，但是均稍大一点，故调整要求低于传输速率1kbps。可以得出以下结论：Since the results after rate control are all close to the transmission rate requirement, but slightly larger, the adjustment requirement is 1kbps lower than the transmission rate. The following conclusions can be drawn:
BL:Rate＝239.256 PSNR＝33.2828 a＝-0.31％BL:Rate＝239.256 PSNR＝33.2828 a＝-0.31%
EL1:Rate＝479.160 PSNR＝37.7187 a＝-0.175％EL1:Rate＝479.160 PSNR＝37.7187 a＝-0.175%
EL2:Rate＝719.320 PSNR＝40.7817 a＝-0.094％EL2:Rate=719.320 PSNR=40.7817 a=-0.094%
其中，a表示速率误差，根据上述实验提取满足不同码率的码流文件。Wherein, a represents the rate error, and the bitstream files satisfying different bitrates are extracted according to the above experiments.
将得到的码流文件进行数据分离，在PHY层设置调制方式为正交相移键控(QPSK)，对NALU数据包进行基带信号调制；采用功分复用传输技术，根据编码层数划分功率段的个数，并由NALU数据包的优先级决定功率段的大小；NALU数据包的优先级越高，分配的功率越大，其中，基础层分配的功率最大，第一层增强层分配的功率次之，第二层增强层分配的功率较小，本实施例将一帧视频数据封装为3层NALU数据包，则叠加3重QPSK调制，将调制后的视频1通过射频传输至无线信道，接收端接收无线信道的视频信号，进行解调、解功分复用、码流合并及视频解码，而后输出视频信号。The obtained bitstream file is data separated, the modulation mode is set to quadrature phase shift keying (QPSK) at the PHY layer, and the NALU data packet is baseband signal modulated; the power division multiplexing transmission technology is adopted, the number of power segments is divided according to the number of coding layers, and the size of the power segment is determined by the priority of the NALU data packet; the higher the priority of the NALU data packet, the greater the allocated power, among which the base layer allocates the largest power, the first layer of the enhanced layer allocates the second largest power, and the second layer of the enhanced layer allocates a smaller power. In this embodiment, a frame of video data is encapsulated into 3 layers of NALU data packets, and 3 QPSK modulations are superimposed, and the modulated video 1 is transmitted to the wireless channel through radio frequency. The receiving end receives the video signal of the wireless channel, demodulates, demultiplexes, merges the code stream, and decodes the video, and then outputs the video signal.
实施例3Example 3
选用基于H.265标准的编码器，同时，为满足信道变化的情况下依然可以使视频正确的传输，采用MGS层分级方式(Medium-grain Quality Scalability，中粒度质量可分级)，在APP层对含有30帧的视频1进行编码，将每帧视频数据编码为3层，分别为1层基础层、第一层增强层及第二层增强层，编码框架参见图4，视频第三层利用第二层以及前一帧进行预测，即第二层增强层通过第一层增强层及前一帧进行预测。进一步将每层视频数据封装成NALU数据包，其中，基础层NALU数据包的优先级高于增强层NALU数据包的优先级，第一层增强层的优先级高于第二层增强层的优先级。An encoder based on the H.265 standard is selected. At the same time, in order to ensure that the video can be transmitted correctly when the channel changes, the MGS layer grading method (Medium-grain Quality Scalability) is adopted. The video 1 containing 30 frames is encoded at the APP layer, and each frame of video data is encoded into 3 layers, namely, 1 base layer, the first enhancement layer and the second enhancement layer. The encoding framework is shown in Figure 4. The third layer of the video is predicted by the second layer and the previous frame, that is, the second enhancement layer is predicted by the first enhancement layer and the previous frame. Each layer of video data is further encapsulated into a NALU data packet, wherein the priority of the base layer NALU data packet is higher than that of the enhancement layer NALU data packet, and the priority of the first enhancement layer is higher than that of the second enhancement layer.
利用调制相关的比特率cn代替香农容量rn，可以得出每层的传输速率，cn＝Bwlog2(Mn)，其中Mn为4。参数设置如下：Using the modulation-dependent bit rate cn instead of the Shannon capacity rn, the transmission rate of each layer can be obtained, cn =Bw log2 (Mn ), where Mn is 4. The parameters are set as follows:
由于在速率控制过程中，编码后的视频信号的总码率应该小于等于PHY层的传输速率720Kbps，速率控制结果如下(确定参考关系后每一帧每一层的QP均为动态调整且不相同，故不作统计)：Since the total bit rate of the encoded video signal should be less than or equal to the transmission rate of the PHY layer 720Kbps during the rate control process, the rate control results are as follows (after determining the reference relationship, the QP of each layer of each frame is dynamically adjusted and different, so it is not counted):
BL:Rate＝239.256 PSNR＝33.2828 a＝-0.31％BL:Rate＝239.256 PSNR＝33.2828 a＝-0.31%
EL1:Rate＝479.296 PSNR＝37.7304 a＝-0.146％EL1:Rate＝479.296 PSNR＝37.7304 a＝-0.146%
EL2:Rate＝719.112 PSNR＝40.6956 a＝-0.123％EL2:Rate=719.112 PSNR=40.6956 a=-0.123%
其中，a表示速率误差，根据上述实验提取满足不同码率的码流文件。Wherein, a represents the rate error, and the bitstream files satisfying different bitrates are extracted according to the above experiments.
将得到的码流文件进行数据分离，在PHY层设置调制方式为正交相移键控(QPSK)，对NALU数据包进行基带信号调制；采用功分复用传输技术，利用APP层的编码信息、PHY层的调制信息和PHY层的无线信道传输状态信息进行功率段的划分，包括对功率段的个数和功率段的功率大小进行划分，NALU数据包的优先级越高，分配的功率越大，其中，基础层分配的功率最大，第一层增强层分配的功率次之，第二层增强层分配的功率较小；本实施例将一帧视频数据封装为3层NALU数据包，则叠加3重QPSK调制，将调制后的视频1信号通过射频传输至无线信道，接收端接收无线信道的视频信号，进行解调、解功分复用、码流合并及视频解码，而后输出视频信号。The obtained bitstream file is data separated, the modulation mode is set to quadrature phase shift keying (QPSK) at the PHY layer, and the NALU data packet is baseband signal modulated; the power division multiplexing transmission technology is adopted, and the coding information of the APP layer, the modulation information of the PHY layer and the wireless channel transmission status information of the PHY layer are used to divide the power segment, including the number of power segments and the power size of the power segment. The higher the priority of the NALU data packet, the greater the allocated power, among which the base layer allocates the largest power, the first layer of the enhanced layer allocates the second largest power, and the second layer of the enhanced layer allocates the smallest power; in this embodiment, a frame of video data is encapsulated into 3 layers of NALU data packets, and 3 QPSK modulations are superimposed, and the modulated video 1 signal is transmitted to the wireless channel through radio frequency. The receiving end receives the video signal of the wireless channel, demodulates, demultiplexes, merges the code stream and decodes the video, and then outputs the video signal.
通过改变SNR的大小，来统计输出NALU数据包的正确解码的数量S，统计结果参见图10-11。By changing the size of SNR, the number S of correctly decoded output NALU data packets is counted. The statistical results are shown in Figures 10-11.
实施例2和实施例3均采用H.265进行编码，将H.265得出的速率的误差与H.264进行对比，可以得出，H.265的速率控制更佳精准。由于实施例2和实施例3存在参考关系的差异性，且在速率控制的过程中，基于H.265的量化参数为动态调整，故两种方法的误差不进行比较。Both Example 2 and Example 3 use H.265 for encoding. By comparing the rate error obtained by H.265 with that of H.264, it can be concluded that the rate control of H.265 is more accurate. Since there is a difference in the reference relationship between Example 2 and Example 3, and in the process of rate control, the quantization parameters based on H.265 are dynamically adjusted, the errors of the two methods are not compared.
由实施例3参考图10-11，可以得出基于H.265编码下，3层30帧的NALU成功解码的平均数量与功率的关系，即S与P1、P2及P3之间的关系。With reference to Figures 10-11 of Example 3, the relationship between the average number of successfully decoded NALUs and power of 3 layers and 30 frames under H.265 encoding can be obtained, that is, the relationship between S andP1 ,P2 andP3 .
结合实施例1与实施例3可以得出无论是采用H.264编码还是H.265编码，S都是关于P1,P2,...Pn的拟凹函数，即采用本发明方法存在最优策略进行无线动态视频的跨层传输。Combining Example 1 with Example 3, it can be concluded that no matter whether H.264 encoding or H.265 encoding is used, S is a quasi-concave function about P1 , P2 , ... Pn , that is, there is an optimal strategy for cross-layer transmission of wireless dynamic video using the method of the present invention.
同时通过仿真来验证该拟凹函数，仿真配置如下：以bus视频两帧两层为例验证推论，其中小区半径为D＝10m，带宽为Bw＝120kHz，信噪比为10dB，高斯白噪声密度为：174dBm/Hz,可以根据P2＝(snr×P_n oise)/h2，P1＝(snr×(P_noise+P2×h2))/h2，P_total＝P1+P2计算出总功率的大小。其中，P_total为总发射功率，P1为第一层发射功率，P2为第二层发射功率，P_noise为噪声功率，发射端和接收端的距离为h，最大平均可顺序解码NALU数据包的个数S；各功率段值经过遍历所有的情况得出的结果如图7所示，x轴表示第二层发射功率的大小，y轴代表第一层发射功率的大小，z轴代表为平均可顺序解码的NALU比例。可以看出S是关于P1，P2的拟凹函数。即采用本发明方法存在最优解、最优策略进行无线动态视频的跨层传输。At the same time, the quasi-concave function is verified by simulation, and the simulation configuration is as follows: take two frames and two layers of bus video as an example to verify the inference, where the cell radius is D = 10m, the bandwidth isBw = 120kHz, the signal-to-noise ratio is 10dB, and the Gaussian white noise density is: 174dBm/Hz. The total power can be calculated according toP2 = (snr × P_noise) /h2 ,P1 = (snr × (P_noise +P2 ×h2 )) /h2 , P_total =P1 +P2 . Among them, P_total is the total transmission power,P1 is the first layer transmission power,P2 is the second layer transmission power, P_noise is the noise power, the distance between the transmitter and the receiver is h, and the maximum average number of NALU packets that can be sequentially decoded is S; the results of each power segment value after traversing all the situations are shown in Figure 7, the x-axis represents the size of the second layer transmission power, the y-axis represents the size of the first layer transmission power, and the z-axis represents the average sequentially decodable NALU ratio. It can be seen that S is a quasi-concave function aboutP1 andP2 . That is, the method of the present invention has an optimal solution and an optimal strategy for cross-layer transmission of wireless dynamic video.
由于H.265/SHVC可以进行参考关系的调整，故利用SHVC对参考关系在传输过程中的影响来研究参考关系的简化。在实际传输过程中，由于信道的影响无法保证每次传输在同一个数据包出错，采取强制数据包出错，统计出错后可以进行正确顺序解码的NALU数据包的数量，并进行多组实验。Since H.265/SHVC can adjust the reference relationship, the influence of SHVC on the reference relationship during transmission is used to study the simplification of the reference relationship. In the actual transmission process, due to the influence of the channel, it is impossible to ensure that the same data packet will fail each time. The forced data packet error is adopted, and the number of NALU data packets that can be correctly decoded in sequence after the error is counted, and multiple groups of experiments are conducted.
视频采用视频1，帧数为30帧，共编码三层，基础层码率固定为200kbps，增强层1码率固定为500kbps，增强层2码率固定为1000kbps，则视频1最大的码率为1000kbps，视频帧率为30帧/秒。设置视频的误差允许范围最大迭代次数为10次，编码过程中只有一个GoP，也就是说NALU共有90个数据包，编码结构如图12所示，图(a)和图(b)为两种复杂程度不同的编码结构。实验通过控制不同NALU在传输过程中出错，验证两种编码框架的NALU传输效果，同时可以得出利用NALU的优先级作为功率分配的必要性。The video uses video 1, with 30 frames and three layers of coding. The base layer bit rate is fixed at 200kbps, the enhancement layer 1 bit rate is fixed at 500kbps, and the enhancement layer 2 bit rate is fixed at 1000kbps. The maximum bit rate of video 1 is 1000kbps, and the video frame rate is 30 frames per second. The maximum number of iterations of the error allowable range of the video is set to 10 times. There is only one GoP during the encoding process, which means that the NALU has a total of 90 data packets. The encoding structure is shown in Figure 12. Figure (a) and Figure (b) are two encoding structures with different complexities. The experiment verifies the NALU transmission effect of the two encoding frameworks by controlling the errors of different NALUs during the transmission process. At the same time, it can be concluded that it is necessary to use the priority of NALU as power allocation.
如图13可以得出，虽然两种编码结构的参考关系不同，但是NALU数据包出错的影响效果相同也就是说传输成功的数据包相同，视频的质量相同，由此可以验证在传输的过程中，在计算传输数据包个数时，任意高阶参考关系均可以简化为一阶参考关系，降低计算复杂度。同时，实验结果表明，基础层出错可以解码的数据包远小于增强层出错可以解码的数据包个数，由此可以看出基础层NALU的重要性远高于增强层，因此在进行功率分配时应合理利用NALU的优先级，更好的保证视频质量。As shown in Figure 13, although the reference relationships of the two coding structures are different, the effects of NALU data packet errors are the same, that is, the successfully transmitted data packets are the same and the video quality is the same. This verifies that in the transmission process, when calculating the number of transmitted data packets, any high-order reference relationship can be simplified to a first-order reference relationship to reduce the computational complexity. At the same time, the experimental results show that the number of data packets that can be decoded due to errors in the base layer is much smaller than the number of data packets that can be decoded due to errors in the enhancement layer. This shows that the importance of the base layer NALU is much higher than that of the enhancement layer. Therefore, the priority of the NALU should be reasonably utilized when allocating power to better ensure video quality.
通过以上描述，本领域技术人员可以清楚地了解到本发明可以通过硬件实现，也可以借助软件加必要的通用硬件平台的方式来实现。基于这样的理解，本发明技术方案可以以软件产品的形式体现出来，该产品可以存储在一个非易失性存储介质(可以是CD-ROM、U盘、移动硬盘等)中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，网络设备等)执行本发明各个实施例所述的方法。Through the above description, those skilled in the art can clearly understand that the present invention can be implemented by hardware, or by software plus necessary general hardware platforms. Based on such understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, a network device, etc.) to execute the methods described in each embodiment of the present invention.
本发明提出的优化问题旨在通过设计一种同时满足传输速率以及总传输功率约束的联合跨层传输数据的方案，使接收端接收到的可顺序解码NALU数据包数量最大，视频质量最好。与传统方案相比，首先利用NALU作为传输的最小单元代替层，因此本发明的目标是最大化GoP中可顺序解码的NALU个数。其次，在APP层，利用SVC自适应编码，根据DLL层以及PHY层联合反馈的传输速率要求，对视频编码结构以及参数进行自适应调整，得到合理的编码方案。在DLL层，由于采用的是基于QPSK的PDMA传输机制，因此利用NALU的优先级进行功率匹配，将优先级高的数据流放在优先级高的功率段上，同时按照优先级不同层的速率匹配合适的QPSK数量以及功率段个数。The optimization problem proposed in the present invention aims to maximize the number of sequentially decodable NALU data packets received by the receiving end and achieve the best video quality by designing a joint cross-layer data transmission scheme that satisfies the transmission rate and total transmission power constraints. Compared with the traditional scheme, NALU is first used as the minimum unit of transmission to replace the layer, so the goal of the present invention is to maximize the number of sequentially decodable NALUs in GoP. Secondly, at the APP layer, SVC adaptive coding is used to adaptively adjust the video coding structure and parameters according to the transmission rate requirements jointly fed back by the DLL layer and the PHY layer to obtain a reasonable coding scheme. At the DLL layer, since the PDMA transmission mechanism based on QPSK is adopted, the priority of NALU is used for power matching, and the data stream with high priority is placed on the power segment with high priority. At the same time, the appropriate number of QPSK and the number of power segments are matched according to the rate of different priority layers.
本发明实现了视频传输过程中的高效性，提高了传输的鲁棒性，缓解了网络传输的压力，且本发明经过多次实验验证，在保证传输质量的同时提高传输的稳定性。The present invention achieves high efficiency in the video transmission process, improves the robustness of transmission, and alleviates the pressure of network transmission. The present invention has been verified through multiple experiments and improves the stability of transmission while ensuring the transmission quality.
以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410589933.2ACN118368427A (en) | 2024-05-13 | 2024-05-13 | Cross-layer transmission method, device, system, medium and product of wireless dynamic video |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410589933.2ACN118368427A (en) | 2024-05-13 | 2024-05-13 | Cross-layer transmission method, device, system, medium and product of wireless dynamic video |
| Publication Number | Publication Date |
|---|---|
| CN118368427Atrue CN118368427A (en) | 2024-07-19 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410589933.2APendingCN118368427A (en) | 2024-05-13 | 2024-05-13 | Cross-layer transmission method, device, system, medium and product of wireless dynamic video |
| Country | Link |
|---|---|
| CN (1) | CN118368427A (en) |
| Publication | Publication Date | Title |
|---|---|---|
| US7944872B2 (en) | Adaptive coding and modulation aware network load balancing | |
| CN101902296B (en) | Coding/decoding method and device for fountain codes | |
| US7093028B1 (en) | User and content aware object-based data stream transmission methods and arrangements | |
| Chen et al. | A near optimal QoE-driven power allocation scheme for scalable video transmissions over MIMO systems | |
| KR100842544B1 (en) | Transmission Method Using Scalable Image Coding and Mobile Communication System Using The Same | |
| US20080144713A1 (en) | Acm aware encoding systems and methods | |
| Park et al. | Optimal DASH-multicasting over LTE | |
| US20100296428A1 (en) | A robust system and method for wireless data multicasting using superposition modulation | |
| JP7343915B2 (en) | FEC mechanism based on media content | |
| KR20040065234A (en) | Modulation algorithm for a MPEG-4 FGS wireless transmission system | |
| CN103795996B (en) | 3D delivery of video method and apparatus | |
| JPWO2006033404A1 (en) | Wireless multimedia communication method | |
| CN104219528B (en) | A kind of video transmission method for the mimo system for supporting gradable video encoding | |
| Zhao et al. | Adaptive hybrid digital–analog video transmission in wireless fading channel | |
| She et al. | A framework of cross-layer superposition coded multicast for robust IPTV services over WiMAX | |
| Deng et al. | Cross-layer DASH-based multipath video streaming over LTE and 802.11 ac networks | |
| Bajic | Efficient cross-layer error control for wireless video multicast | |
| CN103051980B (en) | A kind of resource allocation methods adopting the ofdm system of gradable video encoding | |
| Zhang et al. | Cross-Layer Optimization of Scalable Video Transmission for IoV in 6G | |
| CN118368427A (en) | Cross-layer transmission method, device, system, medium and product of wireless dynamic video | |
| Zhang et al. | Network-adaptive rate control and unequal loss protection with TCP-friendly protocol for scalable video over Internet | |
| Ghandi et al. | H. 264 layered coded video over wireless networks: Channel coding and modulation constraints | |
| Ahmedin et al. | Content and buffer aware scheduling for video delivery over lte | |
| Mukhtar et al. | An occupancy-based and channel-aware multi-level adaptive scheme for video communications over wireless channels | |
| Ji et al. | Cross-layer opportunistic scheduling for device-to-device video multicast services |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | Inventor after:Han Weijia Inventor after:Qiao Caifeng Inventor after:Yue Yuxiang Inventor after:Wang Yujia Inventor after:Zhong Yanan Inventor after:Ma Xiao Inventor after:Wu Weihua Inventor after:Teng Wei Inventor before:Han Weijia Inventor before:Qiao Caifeng Inventor before:Yue Zixiang Inventor before:Wang Yujia Inventor before:Zhong Yanan Inventor before:Ma Xiao Inventor before:Wu Weihua Inventor before:Teng Wei | |
| CB03 | Change of inventor or designer information |