Summary of the invention
The invention provides a kind of real time video data transmission method and electronic equipment, method and apparatus provided by the present invention solves in prior art the problem of the low and waste bandwidth of the retransmission efficiency of existence when using udp protocol to carry out transfer of data.
The invention provides a kind of real time video data transmission method, the method is applied in the first electronic equipment, when the first electronic equipment and the second electronic equipment carry out video data transmission, if the first electronic equipment sends video compression frame data to the second electronic equipment, when video compression frame data are divided into multiple data packet transmission, the method comprises:
The historical information of each packet that record sends, and described video compression frames analysis is determined to the importance of described each packet, and generate first priority tag corresponding with each packet according to the importance determined, described historical information and described priority tag are combined to form the first precedence information and are sent to the second electronic equipment;
Receive the re-transmission sequence of message information to the message lost that the second electronic equipment generates based on described first precedence information;
After receiving described re-transmission sequence of message information, according to this re-transmission sequence of message information, re-transmission message is sorted, and from historical record message, select corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
The scheme more optimized, described video compression frames analysis is determined that the importance of described each packet comprises:
The parameter detected in the heading of described video compression frames determines key frame and reference frame in this video compression frames;
Determine that the importance of packet corresponding to described key frame is higher than the packet corresponding to described reference frame.
The scheme more optimized, describedly according to described first precedence information, re-transmission message is carried out sequence and comprised:
According to the first precedence information received and the retransmission priority of the time factor determination retransmission data packet may of packet received, priority is entered retransmission queue higher than the packet priority of setting threshold.
The scheme more optimized, is combined to form the first precedence information and is sent to the second electronic equipment and comprises by described historical information and described priority tag:
Before every frame data message sends, by media transmission protocol, the first precedence information that described historical information and described priority tag are combined to form is sent to the second electronic equipment.
The embodiment of the present invention also provides another real time video data transmission method, the method is applied in the 3rd electronic equipment, when the 3rd electronic equipment and the 4th electronic equipment carry out video data transmission, if the 3rd electronic equipment receives the 4th electronic equipment and sends video compression frame data, when video compression frame data are divided into multiple data packet transmission, the method comprises:
3rd electronic equipment receives the first precedence information that the 4th electronic equipment sends, wherein, described first precedence information comprises the historical information of each packet that the 4th electronic equipment sends and first priority tag corresponding with each packet that generate according to the importance of each packet;
Determine the order of losing message retransmission according to described first precedence information, and generate re-transmission sequence of message information according to described order;
Described re-transmission sequence of message information is sent to the 4th electronic equipment, 4th electronic equipment is sorted to re-transmission message according to described re-transmission sequence of message, and from historical record message, selects corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
The scheme more optimized, is describedly sent to the 4th electronic equipment by described re-transmission sequence of message information and comprises:
By media control protocol, described re-transmission sequence of message information is sent to the 4th electronic equipment.
The scheme more optimized, the method also comprises:
The first packet received corresponding to default frame is determined whether according to described first precedence information, if do not had, then the Buffer output strategy of the first packet described in the first rule adjustment that basis is default.
A kind of electronic equipment of correspondence is also provided for first method the present invention, this electronic equipment is sending ending equipment, when sending ending equipment and receiving device carry out video data transmission, if sending ending equipment sends video compression frame data to receiving device, when video compression frame data are divided into multiple data packet transmission, this sending ending equipment comprises:
First sending module, for recording the historical information of each packet of transmission, and described video compression frames analysis is determined to the importance of described each packet, and generate first priority tag corresponding with each packet according to the importance determined, described historical information and described priority tag are combined to form the first precedence information and are sent to receiving device;
First receiver module, for receiving the re-transmission sequence of message information to the message lost that described receiving device generates based on described first precedence information;
Retransmission module, for after receiving described re-transmission sequence of message information, sorts to re-transmission message according to this re-transmission sequence of message information, and from historical record message, selects corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
The scheme more optimized, to described video compression frames analysis, described first sending module determines that the importance of described each packet comprises:
The parameter detected in the heading of described video compression frames determines key frame and reference frame in this video compression frames;
Determine that the importance of packet corresponding to described key frame is higher than the packet corresponding to described reference frame.
The scheme more optimized, the first precedence information that described historical information and described priority tag are combined to form also for before every frame data message transmission, is sent to receiving device by media transmission protocol by the first sending module.
Another electronic equipment is also provided for second method the present invention, this electronic equipment is receiving device, when sending ending equipment and receiving device carry out video data transmission, if receiving device receiving end/sending end equipment sends video compression frame data, when video compression frame data are divided into multiple data packet transmission, this receiving device comprises:
Second receiver module, for the first precedence information that receiving end/sending end equipment sends, wherein, described first precedence information comprises the historical information of each packet that sending ending equipment sends and first priority tag corresponding with each packet that generate according to the importance of each packet;
Sequence information generation module, for determining the order of losing message retransmission according to described first precedence information, and generates re-transmission sequence of message information according to described order;
Second sending module, for described re-transmission sequence of message information is sent to sending ending equipment, described sending ending equipment is sorted to re-transmission message according to described re-transmission sequence of message, and from historical record message, selects corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
The scheme more optimized, described second sending module is also for being sent to sending ending equipment by media control protocol by described re-transmission sequence of message information.
The scheme more optimized, electronic equipment also comprises:
Adjusting module, for the first packet determining whether to receive corresponding to default frame according to described first precedence information, if do not had, then the Buffer output strategy of the first packet described in the first rule adjustment that basis is default.
One or two in technique scheme, at least has following technique effect:
The method and apparatus that the embodiment of the present invention provides, first confirms the importance of each packet, then newly carries out prioritization according to packet is important; According to priority, receiving terminal, when receiving packet, determines that but those packets are important do not receive, then, when determining to retransmit, retransmitted by packet priority high for priority.Make receiving terminal can according to the data of the importance determination preemptive re-transmission of data message by such scheme, if so when network state is bad, the quality of video calling can be improved; And efficiency and the bandwidth availability ratio of re-transmission can be increased.
Embodiment
UDP is a connectionless protocol, and before transmission data, source and terminal do not connect, and just goes simply to capture the data from application program, and as quickly as possible it is thrown away on network when it wants to transmit as UDP.At transmitting terminal, the speed that UDP transmits data is only the restriction generating the speed of data, the ability of computer and transmission bandwidth by application program; At receiving terminal, UDP is placed on each message section in queue, and application program reads a message section at every turn from queue.UDP uses Best-Effort, does not namely ensure reliable payment.So the problem of the retransmission efficiency of existence low and waste bandwidth when using udp protocol to carry out transfer of data.
In addition, if video data forms video compression frames by interframe compression (Interframecompression), be have that the feature of very large correlation (namely having redundant information between its consecutive frame of continuous print video) realizes based on continuous front and back two frame of many videos or animation implement compression by the data compared on time shaft between different frame, further raising compression ratio is generally Lossless Compression.Condensed frame comprises key frame and reference frame, if key frame has been lost, can have a huge impact whole transfer of data.So need to provide the data of a kind of scheme to different importance to use different re-transmission policies, the invention provides a kind of real time video data transmission method based on this thought, the method specifically comprises:
The method is applied in the first electronic equipment, when the first electronic equipment and the second electronic equipment carry out video data transmission, if the first electronic equipment sends video compression frame data to the second electronic equipment, when video compression frame data are divided into multiple data packet transmission, the method comprises:
The historical information of each packet that record sends, and described video compression frames analysis is determined to the importance of described each packet, and generate first priority tag corresponding with each packet according to the importance determined, described historical information and described priority tag are combined to form the first precedence information and are sent to the second electronic equipment;
Receive the re-transmission sequence of message information to the message lost that the second electronic equipment generates based on described first precedence information;
After receiving described re-transmission sequence of message information, according to this re-transmission sequence of message information, re-transmission message is sorted, and from historical record message, select corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
Method provided by the present invention, first confirms the importance of each packet, then newly carries out prioritization according to packet is important; According to priority, receiving terminal, when receiving packet, determines that but those packets are important do not receive, then, when determining to retransmit, retransmitted by packet priority high for priority.Make receiving terminal can according to the data of the importance determination preemptive re-transmission of data message by such scheme, if so when network state is bad, the quality of video calling can be improved; And efficiency and the bandwidth availability ratio of re-transmission can be increased.
Be described various embodiment of the present invention and various aspects below with reference to following details, accompanying drawing will illustrate various embodiment.Explanation below and accompanying drawing are to exemplary illustration of the present invention, and should not regard limitation of the present invention as.Describe a large amount of detail to provide the detailed understanding to various embodiments of the invention.But in some cases, will known or traditional details be described, to provide brief description to embodiments of the invention.
Some part of following detailed description represents with the form of algorithm, and these algorithms comprise the operation carried out for the data stored in computer storage.Algorithm refer to substantially cause the operation of results needed be certainly in harmony sequence.These operations usually need or relate to physical manipulation or physical quantity.Usually (but not being inevitable), this tittle takes the form of the signal of telecommunication or magnetic signal, and these signals can be stored, transmit, merge, compare and otherwise be handled.Already proved, sometimes (mainly in order to normally used reason), these signals were called position, value, element, symbol, character, item, number etc. are easily.
But should keep firmly in mind, these and similar all terms are associated with suitable physical quantity, and be only the label being easily applied to this tittle.Unless outside being hereinafter otherwise noted with other forms significantly, the explanation using such as " process " or " calculating " or the term such as " judgement " or " display " to carry out in whole specification can refer to the action that data handling system or like carry out and process, measure the data that represent and convert thereof in the memory of this system or register (or other this category informations store, transmit or show device) similarly with other data that the form of physical quantity represents in the RS of described action and process operating computer with physics (electronics).
The present invention can relate to the equipment for performing the one or more operation in operation described in the application.This equipment can for required object and special configuration, or also can comprise all-purpose computer, described all-purpose computer optionally activates (activate) or reconstruct (reconfigure) by the computer program be stored in this computer.Such computer program can be stored in machine (such as computer) computer-readable recording medium or be stored in and be suitable for stored electrons instruction be coupled in any type media of bus respectively, and described computer-readable medium includes but not limited to the dish (comprising floppy disk, CD, CD-ROM and magneto optical disk) of any type, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, magnetic card or optical card.
Machine readable media comprises for be stored by the readable form of machine (such as computer) or any mechanism of transmission information.Such as, machine readable media comprises read-only memory (ROM); Random access memory (RAM), disk storage media, optical storage medium, flash memory device, with electricity, light, sound or other form propagate signal (such as carrier wave, infrared signal, digital signal etc.) etc.
Example one, as shown in Figure 1, the embodiment of the present invention provides a kind of real time video data transmission method, the method of this embodiment is the description carried out whole scheme based on transmitting terminal, is described in detail to the specific embodiment of the present invention below in conjunction with Figure of description:
The method is applied in the first electronic equipment, when the first electronic equipment and the second electronic equipment carry out video data transmission, if the first electronic equipment (i.e. sending ending equipment) sends video compression frame data to the second electronic equipment (i.e. receiving device), when video compression frame data are divided into multiple data packet transmission, the method comprises:
Step 101, the historical information of each packet that record sends, and described video compression frames analysis is determined to the importance of described each packet, and generate first priority tag corresponding with each packet according to the importance determined, described historical information and described priority tag are combined to form the first precedence information and are sent to the second electronic equipment;
In embodiments of the present invention, described video compression frames analysis is determined that the importance of described each packet comprises:
Mode one, determines whether the importance that key frame is corresponding different:
The parameter detected in the heading of described video compression frames determines key frame and reference frame in this video compression frames;
Determine that the importance of packet corresponding to described key frame is higher than the packet corresponding to described reference frame.
Mode two, for pre-setting the importance parameter of each packet, and this importance parameter to be stored in heading.Electronic equipment is when detection packet importance, then the importance parameter detecting setting can judge the importance of each packet.
In this embodiment, described historical information and described priority tag are combined to form the first precedence information and are sent to the second electronic equipment and comprise:
Before every frame data message sends, by media transmission protocol, the first precedence information that described historical information and described priority tag are combined to form is sent to the second electronic equipment.
Step 102, receives the re-transmission sequence of message information to the message lost that the second electronic equipment generates based on described first precedence information;
In embodiments of the present invention, the difference maximum with prior art is after data sending terminal (i.e. the first electronic equipment) carries out importance assessment (namely generating corresponding priority tag according to the importance of packet) according to transmitting terminal to each packet, when data receiver (i.e. the second electronic equipment) determines data retransmission, then the result can assessed according to described importance determines which packet priority retransmits.Described in inventive embodiments, carry out sequence according to described first precedence information to re-transmission message comprise:
According to the first precedence information received and the retransmission priority of the time factor determination retransmission data packet may of packet received, priority is entered retransmission queue higher than the packet priority of setting threshold.
In this embodiment, when determining the retransmission priority of some packets, first determine the importance priority of each retransmission data packet may according to the first priority tag in the first precedence information; In addition, the time factor (factor such as transmission delay as packet) considering packet is also needed; After two kinds of situations being considered, determine the retransmission priority of packet.
Step 103, after receiving described re-transmission sequence of message information, sorts to re-transmission message according to this re-transmission sequence of message information, and from historical record message, selects corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
Example two, as shown in Figure 2, the embodiment of the present invention provides a kind of real time video data transmission method, the method of this embodiment is the description carried out whole scheme based on receiving terminal, is described in detail to the specific embodiment of the present invention below in conjunction with Figure of description:
The method is applied in the 3rd electronic equipment, when the 3rd electronic equipment and the 4th electronic equipment carry out video data transmission, if the 3rd electronic equipment (i.e. receiving device) receives the 4th electronic equipment (i.e. sending ending equipment) and sends video compression frame data, when video compression frame data are divided into multiple data packet transmission, the method comprises:
Step 201,3rd electronic equipment receives the first precedence information that the 4th electronic equipment sends, wherein, described first precedence information comprises the historical information of each packet that the 4th electronic equipment sends and first priority tag corresponding with each packet that generate according to the importance of each packet;
Step 202, determines the order of losing message retransmission according to described first precedence information, and generates re-transmission sequence of message information according to described order;
Step 203, described re-transmission sequence of message information is sent to the 4th electronic equipment, 4th electronic equipment is sorted to re-transmission message according to described re-transmission sequence of message, and from historical record message, selects corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
In this embodiment, describedly by the mode that described re-transmission sequence of message information is sent to the 4th electronic equipment can be: by media control protocol, described re-transmission sequence of message information is sent to the 4th electronic equipment.
In this embodiment, except the priority by described first priority determination data re-transmission, after determining that those packets are important, adaptive adjustment can also be done to the condition of acceptance of important packet.So in the present embodiment data receiver after receiving the first precedence information that transmitting terminal sends, can also according to the Buffer output strategy in described first priority, the descriptor of packet importance further being adjusted to packet, specific implementation can be:
The first packet received corresponding to default frame is determined whether according to described first precedence information, if do not had, then the Buffer output strategy of the first packet described in the first rule adjustment that basis is default.
Example three, as shown in Figure 3, the embodiment of the present invention provides a kind of real time video data transmission method, the method of this embodiment is the description carried out according to the data interaction flow process of receiving terminal and transmitting terminal, is described in detail to the specific embodiment of the present invention below in conjunction with Figure of description:
Step 301, transmitting terminal generates history message accounting table and priority tag table;
Conveniently use in the present embodiment, Packet Generation situation can be carried out recording rear history of forming message accounting table; The priority tag table of importance priority and packet corresponding relation is generated according to the importance priority difference of each packet.
In this embodiment, during video compression frame data divided stator frame, the message sent is saved in history message accounting table, generate message priority label list according to the importance of message in frame and frame, the content of this table comprises the sequence number of data message, marks the label of this bag weights of importance simultaneously.Receiving terminal was sent to by media transmission protocol before every frame data message sends.
Step 302, receiving terminal generates and retransmits sequence of message;
The frame data of receiving terminal to buffering are added up, according to generate rule, message queue table is retransmitted to the message lost, this rule calculates according to factors such as the message priority label list received and times of receiving packet the priority retransmitted, the bag that retransmission priority is high preferentially enters retransmission queue, the bag that retransmission priority is low comes after queue, give up low priority message when packet loss is serious and bandwidth is inadequate and do not enter queue, and send re-transmission sequence of message table to transmitting terminal by media control protocol.On the other hand, according to message priority label list, receiving terminal can do Dynamic controlling to the Buffer output strategy of important frame.
Step 303, transmitting terminal data retransmission;
Receive after retransmitting response message (i.e. retransmission queue), transmitting terminal, according to the priority tag table of historical record message, does second time prioritization to re-transmission sequence of message table, and from historical record message, selects corresponding message retransmission according to order.
Method provided by the present invention, first confirms the importance of each packet, then newly carries out prioritization according to packet is important; According to priority, receiving terminal, when receiving packet, determines that but those packets are important do not receive, then, when determining to retransmit, retransmitted by packet priority high for priority.Make receiving terminal can according to the data of the importance determination preemptive re-transmission of data message by such scheme, if so when network state is bad, the quality of video calling can be improved; And efficiency and the bandwidth availability ratio of re-transmission can be increased.
As shown in Figure 4, the method the present invention provided for embodiment one also provides a kind of electronic equipment, this electronic equipment is sending ending equipment, when sending ending equipment and receiving device carry out video data transmission, if sending ending equipment sends video compression frame data to receiving device, when video compression frame data are divided into multiple data packet transmission, this sending ending equipment comprises:
First sending module 401, for recording the historical information of each packet of transmission, and described video compression frames analysis is determined to the importance of described each packet, and generate first priority tag corresponding with each packet according to the importance determined, described historical information and described priority tag are combined to form the first precedence information and are sent to receiving device;
Wherein, to described video compression frames analysis, this first sending module 401 determines that the importance of described each packet comprises:
The parameter detected in the heading of described video compression frames determines key frame and reference frame in this video compression frames;
Determine that the importance of packet corresponding to described key frame is higher than the packet corresponding to described reference frame.
In addition, the first precedence information that described historical information and described priority tag are combined to form also for before every frame data message transmission, is sent to receiving device by media transmission protocol by this first sending module 401.
First receiver module 402, for receiving the re-transmission sequence of message information to the message lost that described receiving device generates based on described first precedence information;
Retransmission module 403, for after receiving described re-transmission sequence of message information, according to this re-transmission sequence of message information, re-transmission message is sorted, and from historical record message, select corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
As shown in Figure 5, the method the present invention provided for embodiment two also provides another electronic equipment, this electronic equipment is receiving device, when sending ending equipment and receiving device carry out video data transmission, if receiving device receiving end/sending end equipment sends video compression frame data, when video compression frame data are divided into multiple data packet transmission, this receiving device comprises:
Second receiver module 501, for the first precedence information that receiving end/sending end equipment sends, wherein, described first precedence information comprises the historical information of each packet that sending ending equipment sends and first priority tag corresponding with each packet that generate according to the importance of each packet;
Sequence information generation module 502, for determining the order of losing message retransmission according to described first precedence information, and generates re-transmission sequence of message information according to described order;
Second sending module 503, for described re-transmission sequence of message information is sent to sending ending equipment, described sending ending equipment is sorted to re-transmission message according to described re-transmission sequence of message, and from historical record message, selects corresponding message to retransmit according to described sequence and described re-transmission sequence of message information.
Wherein, this second sending module 503 is also for being sent to sending ending equipment by media control protocol by described re-transmission sequence of message information.
In addition, in this embodiment, except the priority by described first priority determination data re-transmission, after determining that those packets are important, adaptive adjustment can also be done to the condition of acceptance of important packet.So the electronic equipment in this embodiment also comprises:
Adjusting module 504, for the first packet determining whether to receive corresponding to default frame according to described first precedence information, if do not had, then the Buffer output strategy of the first packet described in the first rule adjustment that basis is default.
Above-mentioned one or more technical scheme in the embodiment of the present application, at least has following technique effect:
Method provided by the present invention and electronic equipment, first confirm the importance of each packet, then newly carries out prioritization according to packet is important; According to priority, receiving terminal, when receiving packet, determines that but those packets are important do not receive, then, when determining to retransmit, retransmitted by packet priority high for priority.Make receiving terminal can according to the data of the importance determination preemptive re-transmission of data message by such scheme, if so when network state is bad, the quality of video calling can be improved; And efficiency and the bandwidth availability ratio of re-transmission can be increased.
Method of the present invention is not limited to the embodiment described in embodiment, and those skilled in the art's technical scheme according to the present invention draws and other execution mode belongs to technological innovation scope of the present invention equally.
Obviously, those skilled in the art can carry out various change and modification to the present invention and not depart from the spirit and scope of the present invention.Like this, if these amendments of the present invention and modification belong within the scope of the claims in the present invention and equivalent technologies thereof, then the present invention is also intended to comprise these change and modification.