File accelerated transmission method and deviceTechnical Field
The invention relates to the field of network data transmission, in particular to a file accelerated transmission method and device.
Background
Today, with the rapid development of cloud computing technology, data shows an explosive growth trend, and the demand of people for data increases video information and picture information in a large amount. Similarly, new data is generated during the use of the mass data, and the new data needs to be analyzed and processed. Whether the data is used or the mining and analysis based on the data are carried out, how to firstly ensure the fast and stable transmission of mass data is an important subject.
Taking the distributed log statistical processing system as an example, the central server needs to obtain the original log data on the log generation end for further statistical analysis. However, a large amount of bandwidth and time are required for transmitting the mass log data, and especially when the central server and the log generating end are located in different operators, the network environment is complex and variable, so that the uploading process becomes extremely difficult. Therefore, how to achieve high-performance, high-reachability, and bandwidth-saving data uploading becomes a hot spot of interest.
In the existing transmission process, especially in the cross-network environment, when a file needs to be forwarded through a transit node when being transmitted, how to effectively improve the transmission performance is a technical problem to be solved.
Disclosure of Invention
In order to solve the above problems, the present invention provides a method and an apparatus for accelerating file transmission.
The invention provides a file accelerated transmission method, which comprises the following steps:
the method comprises the steps that a source node uploads target information to a server, when the target information needs to be forwarded to the server through a transfer node, at least one transfer node is selected from a friend node list of the source node, and the target information is forwarded to the server through the transfer node.
The file accelerated transmission method also has the following characteristics:
the method further comprises the following steps: after the target information is forwarded to the server through the transit nodes, the source node receives transmission effect parameters of the target information received by the server from the transit nodes from the server, and scores of the transit nodes are determined or updated according to the transmission effect parameters corresponding to the transit nodes;
the selecting at least one transit node from the friend node list of the source node comprises: and selecting the transfer node with the maximum score from the friend node list, or selecting the transfer node with the score larger than a preset value, or selecting N transfer nodes ranked as the top N according to the ranking of the score from high to low, wherein N is an integer larger than 1.
The file accelerated transmission method also has the following characteristics:
the transmission effect parameter includes at least one of the following parameters: the average speed of transmitting the target information, the time for transmitting the target information and the retransmission times;
the transmission effect parameter is positively correlated with the grade of the transit node.
The file accelerated transmission method also has the following characteristics:
the forwarding, by the transit node, the target information to the server comprises: and forwarding the target information to a server through the transfer node, downloading the fragment data of the target information from the transfer node by the server, downloading the fragment data from each transfer node by the server at different downloading speeds, wherein the downloading speeds of different transfer nodes and the downloaded fragment quantity are positively correlated.
The file accelerated transmission method also has the following characteristics:
the source node uploading the target information to the server comprises the following steps:
and the source node sends an uploading request for uploading the target information to the server, receives a fragment downloading request of the target information from the server, and sends each fragment data to the server according to the fragment downloading request.
The file accelerated transmission method also has the following characteristics:
the source node uploading the target information to the server comprises the following steps: the source node and the server establish at least one transmission channel, target information is uploaded through the transmission channel, and the transmission channel adopts at least one communication protocol.
The invention also provides a file accelerated transmission device, which comprises:
the uploading module is used for uploading the target information to the server;
the judging module is used for judging whether the target information needs to be forwarded to the server through a transfer node;
a selection module, configured to select at least one transit node from the friend node list of the source node;
and the transfer module is used for sending the target information to the transfer node selected by the selection module.
The file accelerating device also has the following characteristics:
the system also comprises a receiving module and a friend node list maintenance module;
the receiving module is configured to receive, from the server, transmission effect parameters of the target information received by the server from each transfer node;
the friend node list maintenance module is used for determining or updating the scores of the transfer nodes according to the transmission effect parameters corresponding to the transfer nodes;
the selection module is further configured to select at least one transit node from the friend node list of the source node using: and selecting the transfer node with the maximum score from the friend node list, or selecting the transfer node with the score larger than a preset value, or selecting N transfer nodes ranked as the top N according to the ranking of the score from high to low, wherein N is an integer larger than 1.
The file accelerated transmission device also has the following characteristics:
the uploading module is further used for sending an uploading request for uploading the target information to the server, receiving a fragment downloading request of the target information from the server, and sending each fragment data to the server according to the fragment downloading request.
The file accelerated transmission device also has the following characteristics:
the uploading module is also used for establishing at least one transmission channel with the server, uploading target information through the transmission channel, and the transmission channel adopts at least one communication protocol.
When the forwarding needs to be carried out through the transit node, the best transit node can be intelligently selected, so that the optimal transit path is obtained, the data transmission performance can be improved, the network processing performance can be improved, and the user experience can be improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart of a file accelerated transfer method in an embodiment;
FIG. 2 is a block diagram of an apparatus for accelerated file transfer according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
FIG. 1 is a flowchart illustrating an embodiment of a method for accelerated file transfer, the method comprising:
step 101, a source node uploads target information to a server;
and 102, when the target information is judged to be forwarded to the server through the transit node, selecting at least one transit node from the friend node list of the source node, and forwarding the target information to the server through the transit node.
The target information may be data in various formats, for example, multimedia information such as text information, picture information, or video information, and may further include analysis data such as log data.
A friend node list initialization step may be further included before step 101, and this step includes setting each transit node and initializing the score of each transit node.
The method further includes step 103, after the target information is forwarded to the server through the relay nodes, the source node receives, from the server, transmission effect parameters of the target information received by the server from each relay node, and determines or updates scores of each relay node according to the transmission effect parameters corresponding to each relay node. Wherein selecting at least one transit node from the friend node list of the source node comprises: and selecting the transfer node with the maximum score from the friend node list, or selecting the transfer node with the score larger than a preset value, or selecting N transfer nodes ranked as the top N according to the ranking of the score from high to low, wherein N is an integer larger than 1.
The transmission effect parameter includes at least one of the following parameters: average speed of transmitting target information, time taken to transmit the target information, and retransmission times.
The transmission effect parameter is positively correlated with the grade of the transit node. Examples are as follows: updating the score of each transit node according to the transmission effect parameter corresponding to each transit node comprises: and when the transmission effect parameter corresponding to a transit node is judged to be at the corresponding preset parameter threshold level, taking the product of the current grade of the transit node and the coefficient corresponding to the preset parameter threshold level as the updated grade of the transit node.
For example: the transmission effect parameter includes an average speed of transmission target information. The average speed threshold level of the transmission target information includes three third levels: a first grade: 5 Mbits per second to 2 Mbits per second; a second stage: 2 Mbits per second to 1 Mbits per second; third level: 1 mbit/sec to 1000 bits/sec. The coefficient corresponding to the first threshold level is a first coefficient having a value of 1.1. The coefficient corresponding to the second threshold level is a second coefficient having a value of 1. The coefficient corresponding to the third threshold level is a third coefficient having a value of 0.9. In the transit node score maintenance process of the method, the transit node has a good transmission effect, the higher the score of the transit node is, the poorer the transmission effect of the transit node is, and the lower the score of the transit node is, so that the capability of the transit node can be reasonably determined according to historical data.
By the scoring mode of the transit node and the updating mode of the scoring of the transit node in the method, the file transmission efficiency and the file transmission quality can be continuously improved along with the increase of the uploading times.
The method aims to prevent the file from being too large, so that the transmission time is long, and even the uploading is failed due to time-out. The transmission in this method is a way of fragmented transmission. The source node uploading the target information to the server comprises the following steps: and the source node sends an uploading request for uploading the target information to the server, receives a fragment downloading request of the target information from the server, and sends each fragment data to the server according to the fragment downloading request.
Forwarding the target information to the server through the transit node specifically includes: and forwarding the target information to a server through the transfer node, downloading the fragment data of the target information from the transfer node by the server, downloading the fragment data from each transfer node by the server at different downloading speeds, wherein the downloading speeds of different transfer nodes and the downloaded fragment quantity are positively correlated.
In order to prevent the file transmission from being charged by a process or a thread to realize sequential synchronization from beginning to end, the processing resources of the system cannot be effectively utilized, in the method, the source node and the server establish at least one transmission channel, target information is uploaded through the transmission channel, and the transmission channel adopts at least one communication protocol (such as a TCP protocol or a UDP protocol).
FIG. 2 is a block diagram of an embodiment of an apparatus for accelerated document transport, the apparatus comprising:
the uploading module is used for uploading the target information to the server;
the judging module is used for judging whether the target information needs to be forwarded to the server through a transfer node;
the selection module is used for selecting at least one transit node from a friend node list of the source node;
and the transfer module is used for sending the target information to the transfer node selected by the selection module.
Wherein, this device still includes: the system comprises a receiving module and a friend node list maintaining module.
The receiving module is used for receiving the transmission effect parameters of the target information received by the server from each transfer node from the server;
the friend node list maintenance module is used for determining or updating the scores of the transfer nodes according to the transmission effect parameters corresponding to the transfer nodes;
the selection module is further configured to select at least one transit node from the friend node list of the source node using: and selecting the transfer node with the maximum score from the friend node list, or selecting the transfer node with the score larger than a preset value, or selecting N transfer nodes ranked as the top N according to the ranking of the score from high to low, wherein N is an integer larger than 1.
Wherein,
the uploading module is also used for sending an uploading request for uploading the target information to the server, receiving a fragment downloading request of the target information from the server, and sending each fragment data to the server according to the fragment downloading request.
The uploading module is also used for establishing at least one transmission channel with the server, uploading the target information through the transmission channel, and the transmission channel adopts at least one communication protocol.
When the forwarding needs to be carried out through the transit node, the best transit node can be intelligently selected, so that the optimal transit path is obtained, the data transmission performance can be improved, the network processing performance can be improved, and the user experience can be improved.
The above-described aspects may be implemented individually or in various combinations, and such variations are within the scope of the present invention.
It will be understood by those skilled in the art that all or part of the steps of the above methods may be implemented by instructing the relevant hardware through a program, and the program may be stored in a computer readable storage medium, such as a read-only memory, a magnetic or optical disk, and the like. Alternatively, all or part of the steps of the foregoing embodiments may also be implemented by using one or more integrated circuits, and accordingly, each module/unit in the foregoing embodiments may be implemented in the form of hardware, and may also be implemented in the form of a software functional module. The present invention is not limited to any specific form of combination of hardware and software.
It is to be noted that, in this document, the terms "comprises", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion, so that an article or apparatus including a series of elements includes not only those elements but also other elements not explicitly listed or inherent to such article or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of additional like elements in the article or device comprising the element.
The above embodiments are merely to illustrate the technical solutions of the present invention and not to limit the present invention, and the present invention has been described in detail with reference to the preferred embodiments. It will be understood by those skilled in the art that various modifications and equivalent arrangements may be made without departing from the spirit and scope of the present invention and it should be understood that the present invention is to be covered by the appended claims.