Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The FastDFS generally comprises a tracking Server (Tracker Server), a Storage Server (Storage Server) and a Client (Client), wherein the tracking Server is responsible for receiving a data transmission request of the Client, selecting a proper Storage Server, and the Client establishes a connection with the Storage Server and then uploads or downloads data.
More specifically, when a client needs to upload or download data, a data transmission request may be sent to the tracking server, and the tracking server may select a certain storage server in a certain storage server group according to the data transmission request, and send an IP address and a port number of the storage server to the client, and then the client establishes a connection with the storage server according to the IP address and the port number, and completes the data upload or download based on the connection.
In the prior art, each storage server can upload own state information to the tracking server at regular time, and the tracking server and the storage server are usually deployed in the same local area network environment, so that the storage server usually uses an intranet IP address when uploading own state information, the tracking server only knows the intranet IP address of the storage server, and further the tracking server sends the intranet IP address of the storage server when sending the IP address and the port number of the storage server to the client.
For example, there is a tracking server with an intranet IP address of 10.0.0.1, a mapped extranet IP address of 192.168.100.1, and a service port of 22122; in addition, the intranet IP address of one storage server is 10.0.0.2, the mapped extranet IP address is 192.168.100.2, the port number is 23000, and the intranet IP address is 10.0.0.2 when the storage server uploads the state information of the storage server to the tracking server at regular time; in the data uploading process, a client side initiates a file uploading request to a tracking server, and the tracking server can return an intranet IP address of 10.0.0.2 and a port number of 23000 of a storage server to the client side; the client is connected with the storage server according to the received IP address and the port number, when the client is positioned in an intranet, the client can be connected with the storage server, and when the client is positioned in an extranet, the client cannot be connected with the storage server, so that data uploading fails. The data downloading process is similar and will not be described herein.
In order to solve the technical problem, the embodiment of the invention adds a step of judging whether the mapping of the internal and external network communication address information is needed, if the mapping of the internal and external network communication address information is determined to be needed, the external network communication address information of the storage server is obtained according to the internal network communication address information of the storage server, so that the client can establish connection with the storage server according to the external network communication address information of the storage server to finish the data transmission.
The step of judging whether the mapping of the internal and external network communication address information is needed can be executed on the tracking server, or can be executed on any other equipment of the internal network, or can be executed on the client, but if the step of judging whether the mapping of the internal and external network communication address information is needed is executed on the tracking server, the program of the tracking server needs to be modified, the modification workload is large, the load of the tracking server is possibly increased, and the tracking server is unstable, and if the step of judging whether the mapping of the internal and external network communication address information is needed is executed on any other equipment of the internal network, other equipment needs to be additionally deployed, and the cost is increased.
The data transmission method provided by the embodiment of the present invention is applicable to the communication system shown in fig. 1, and as shown in fig. 1, the method includes: atracking server 11, astorage server 12 and aclient 13.
Theclient 13 may send a data transmission request to thetracking server 11; thetracking server 11 determines the intranet communication address information of thestorage server 12 according to the data transmission request and sends the intranet communication address information to theclient 13; theclient 13 judges whether the mapping of the internal and external network communication address information is needed; if the fact that the internal and external network communication address information mapping is needed is determined, acquiring corresponding external network communication address information according to the internal network communication address information; and performing data transmission with thestorage server 12 according to the external network communication address information.
A plurality oftracking servers 11 can be deployed to form a tracking server cluster, eachtracking server 11 can be in a peer-to-peer (equal status) mode and is easy to expand, information managed by eachtracking server 11 is also stored in a memory, when aclient 13 accesses thetracking server 11, onetracking server 11 can be randomly allocated from the tracking server cluster to interact with theclient 13, and whether the other side normally operates can be detected between thetracking server 11 and astorage server 12 through a heartbeat mechanism; a plurality ofstorage servers 12 may be deployed to form a storage server cluster, which may be divided into a plurality of groups (groups), each group may isolate data of different applications, that is, data of different applications are stored in different groups, and thestorage servers 12 in the same group are synchronized and backed up with each other; theclient 13 may be a server or any other electronic device, which is not described herein.
The following describes the technical solutions of the present invention and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
Fig. 2 is a flowchart of a data transmission method according to an embodiment of the present invention. The embodiment provides a data transmission method, an execution main body of which is a client, and the data transmission method comprises the following specific steps:
s201, sending a data transmission request to the tracking server.
In this embodiment, the tracking server is connected to the client and the storage server, and when the client needs to upload or download data, the tracking server may send a data transmission request to the tracking server, where the data transmission request may include a data upload request or a data download request, and correspondingly, the data upload request may include related information of the data to be uploaded, such as an application to which the data to be uploaded belongs, so that the tracking server determines the storage server according to the related information of the data to be uploaded; the data downloading request may include information related to the data to be downloaded, such as an application to which the data to be downloaded belongs, an identifier (id) of the data to be downloaded, and the like, so that the tracking server determines the storage server according to the information related to the data to be downloaded.
S202, receiving intranet communication address information of the storage server returned by the tracking server; and the intranet communication address information of the storage server is determined by the tracking server according to the data transmission request.
In this embodiment, after receiving a data transmission request sent by a client, a tracking server may determine a storage server according to the data transmission request, for example, for the data upload request, the tracking server may determine a group of storage servers according to related information of data to be uploaded included in the data upload request, then determine an available storage server in the group, for example, randomly select, or poll, or select an idle storage server, and send intranet communication address information of the storage server to the client; for the data downloading request, the tracking server may determine a group of storage servers according to the related information of the data to be downloaded included in the data downloading request, then determine an available storage server in the group, for example, randomly select, or select a polling selection, or select an idle storage server, and further send the intranet communication address information of the storage server to the client, optionally, when determining the storage server, it may also detect whether the data in the group of the storage server keeps synchronous. The intranet communication address information may include, but is not limited to, an intranet IP address, a port number, and the like.
S203, judging whether the internal and external network communication address information mapping is needed or not; and if the fact that the internal and external network communication address information mapping is needed is determined, acquiring corresponding external network communication address information according to the internal network communication address information.
In this embodiment, the client may determine whether the mapping of the communication address information of the internal and external networks is required, and when the client is in the internal network, that is, in the same local area network as the tracking server and the storage server, the mapping of the communication address information of the internal and external networks is not required; when the client is in an external network, that is, not in the same local area network as the tracking server and the storage server, the internal and external network communication address information mapping is required, and under the condition that the internal and external network communication address information mapping is determined to be required, the corresponding external network communication address information can be obtained according to the internal network communication address information, for example, an internal network IP address is converted into an external network IP address.
In this embodiment, the method for determining whether to map the intranet and extranet communication address information is not limited.
The determination of whether the mapping of the intranet and extranet communication address information is required is not limited to be performed after steps S201 and S202, and may be performed at any time before the corresponding extranet communication address information is acquired from the intranet communication address information.
And S204, carrying out data transmission with the storage server according to the external network communication address information.
In this embodiment, after the corresponding extranet communication address information is acquired according to the intranet communication address information, data transmission may be performed with the storage server according to the extranet communication address information.
Specifically, in an optional embodiment, as shown in fig. 3, for the data uploading process, the performing data transmission with the storage server according to the extranet communication address information may specifically include:
s301, establishing connection with the storage server according to the extranet communication address information;
s302, sending data to be uploaded to the storage server, so that the storage server stores the data to be uploaded and generates a data identifier, and synchronizing the data to be uploaded to other storage servers in a group where the storage server is located;
s303, receiving the data identifier returned by the storage server.
In this embodiment, the client may establish a connection with the storage server through the extranet communication address information, where the connection is not limited to a wired connection or a wireless connection, and then the client may send the data to be uploaded to the storage server through the connection, and the storage server stores the data to be uploaded, and generates a data identifier (id) for the data at the same time, and then sends the data identifier to the client, and the client may store or record the data identifier, so as to query, access or download the data subsequently according to the data, and at the same time, it may also determine that the data is uploaded successfully when the data identifier is received; after the storage server finishes storing, the data to be uploaded can be synchronized to other storage servers in the group, so that the data in the storage servers in the whole group are kept consistent. It should be noted that synchronizing the data to be uploaded to other storage servers in the group where the storage server is located is not necessarily performed before step S303, and may be performed at any time after the storage server stores the data to be uploaded.
In another alternative embodiment, as shown in fig. 4, for the data downloading process, the performing data transmission with the storage server according to the extranet communication address information may specifically include:
s401, establishing connection with the storage server according to the external network communication address information;
s402, sending a data identifier of data to be downloaded to the storage server;
s403, receiving the data to be downloaded returned by the storage server according to the data identifier.
In this embodiment, the client may establish a connection with the storage server through the extranet communication address information, where the connection mode is not limited to wired connection or wireless connection, and then the client sends a data identifier (id) of the data to be downloaded to the storage server through the connection, and the storage server sends the data to be downloaded to the client according to the data identifier.
In a possible embodiment, if the data to be downloaded does not exist in the storage server, it may be that the storage server does not keep data synchronization with other storage servers in the group where the storage server is located, and after performing data synchronization with other storage servers in the group, the data to be downloaded is sent to the client according to the data identifier.
On the basis of any of the above embodiments, when it is determined that the mapping of the intranet and extranet communication address information is not required, data transmission is performed with the storage server according to the intranet communication address information. The data uploading process and the data downloading process at this time may be similar to S301-S303 and S401-S403, and the communication address information of the external network may be replaced by the communication address information of the internal network.
In the data transmission method provided by the embodiment, a data transmission request is sent to a tracking server through a client; receiving intranet communication address information of a storage server returned by a tracking server, wherein the intranet communication address information of the storage server is determined by the tracking server according to a data transmission request; the client judges whether the mapping of the internal and external network communication address information is needed or not; if the fact that the internal and external network communication address information mapping is needed is determined, acquiring corresponding external network communication address information according to the internal network communication address information; and carrying out data transmission with the storage server according to the external network communication address information. In the embodiment, whether the mapping of the internal and external network communication address information is needed or not is judged by the client, so that the external network communication address information of the tracking server can be acquired under the condition that the client is in the external network, the connection with the storage server is realized, the data transmission is completed, and the stable operation of the system is ensured.
On the basis of any of the foregoing embodiments, the determining whether the intra-extranet communication address information mapping is required in S203 may specifically include:
and judging whether the internal and external network communication address information mapping is needed or not according to the preset configuration information.
In this embodiment, the client generally needs to set some configuration information related to the FastDFS service in advance by a technician, including but not limited to the following items:
1) service establishment connection timeout: if connect _ timeout is 2
2) Time-out time of transmission and reception data: such as network _ timeout ═ 30
3) Character encoding method: such as charSet ═ UTF-8
4) Tracking the external network ip address provided by the server: such as tracker _ server 192.168.100.1:22122
When the connection establishment time exceeds the service connection establishment timeout time, an error prompt can be returned; when the time for sending or receiving data exceeds the time-out time for sending and receiving data, an error prompt can be returned; and the outer network ip address provided by the tracking server is used for establishing connection between the client and the tracking server, sending a data transmission request to the tracking server and receiving the inner network communication address information of the storage server returned by the tracking server.
In this embodiment, in order to determine whether the mapping of the intranet and extranet communication address information is needed, relevant configuration information may be added in advance in the client.
In an optional embodiment, a target configuration item may be added in advance in preset configuration information of a client, where the target configuration item includes a field for indicating whether to perform internal and external network mapping of communication address information, where, for example, flag _ mapping ═ true represents the internal and external network mapping of the communication address information, and flag _ mapping ═ false represents the internal and external network mapping of the communication address information, which is not needed, and further, it may be determined whether to perform internal and external network communication address information mapping according to the target configuration item included in the preset configuration information, where the target configuration item includes a field for indicating whether to perform internal and external network mapping of the communication address information.
In another optional embodiment, communication address information of a client may be added to preset configuration information of the client in advance, and then whether the client and the storage server are in the same local area network environment may be determined according to the communication address information of the client included in the preset configuration information; and if the storage server is not in the same local area network environment as the storage server, mapping the communication address information of the internal network and the external network.
In addition to any of the above embodiments, the client may also preset an intranet-extranet communication address information mapping relationship, for example, 10.0.0.2 ═ 192.168.100.2, where the former is intranet communication address information of the storage server and the latter is corresponding extranet communication address information. Optionally, the preset mapping relationship between the intranet and extranet communication address information may also be included in the preset configuration information of the client. But may be obtained by other means.
Further, the acquiring, according to the intranet communication address information, corresponding extranet communication address information in S203 may specifically include:
and acquiring the external network communication address information corresponding to the internal network communication address information according to the internal network communication address information and a preset internal and external network communication address information mapping relation.
In this embodiment, after receiving the intranet communication address information sent by the tracking server, the client may query, from the mapping relationship between the intranet and intranet communication address information, the extranet communication address information corresponding to the intranet communication address information, so as to obtain the extranet communication address information of the tracking server, so as to establish a connection with the tracking server.
On the basis of any of the above embodiments, as shown in the signaling diagram shown in fig. 5, in the data uploading process, the data transmission method specifically includes:
s501, the storage server uploads state information to the tracking server at regular time;
s501 is executed in a timed mode, can be executed simultaneously with the following steps, and has no conflict;
s502, the client sends a data uploading request to the tracking server;
s503, the tracking server determines the intranet communication address information of the storage server according to the data transmission request;
s504, the tracking server returns the intranet communication address information of the storage server to the client;
s505, the client judges whether the mapping of the internal and external network communication address information is needed; if the fact that the internal and external network communication address information mapping is needed is determined, acquiring corresponding external network communication address information according to the internal network communication address information;
s506, the client establishes connection with the storage server according to the extranet communication address information and sends data to be uploaded to the storage server;
s507, the storage server stores the data to be uploaded and generates a data identifier;
s508, the storage server sends the data identifier to the client;
s509, the storage server synchronizes the data to be uploaded to other storage servers in the group where the storage server is located;
s509 may be performed before S508, or may be performed simultaneously with S508.
On the basis of any of the above embodiments, as shown in the signaling diagram shown in fig. 6, in the data downloading process, the data transmission method specifically includes:
s511, the storage server uploads the state information to the tracking server at regular time;
wherein, the step S511 is executed at regular time, and can be executed simultaneously with the following steps, and there is no conflict;
s512, the client sends a data downloading request to the tracking server;
s513, the tracking server detects the synchronous state of the storage server and determines the intranet communication address information of the storage server according to the data transmission request;
s514, the tracking server returns the intranet communication address information of the storage server to the client;
s515, the client judges whether the mapping of the internal and external network communication address information is needed; if the fact that the internal and external network communication address information mapping is needed is determined, acquiring corresponding external network communication address information according to the internal network communication address information;
s516, the client establishes connection with the storage server according to the external network communication address information and sends a data identifier of data to be downloaded to the storage server;
s517, the storage server inquires the data to be downloaded according to the data identification;
s518, the storage server sends the data to be downloaded to a client.
Fig. 7 is a block diagram of a client according to an embodiment of the present invention. The client provided in this embodiment may execute the processing flow provided in the data transmission method embodiment, as shown in fig. 7, theclient 700 includes a sendingmodule 701, a receivingmodule 702, aprocessing module 703, and adata transmission module 704.
A sendingmodule 701, configured to send a data transmission request to a tracking server;
areceiving module 702, configured to receive intranet communication address information of the storage server returned by the tracking server; the intranet communication address information of the storage server is determined by the tracking server according to the data transmission request;
theprocessing module 703 is configured to determine whether mapping of the intranet and extranet communication address information is required; if the fact that the internal and external network communication address information mapping is needed is determined, acquiring corresponding external network communication address information according to the internal network communication address information;
adata transmission module 704, configured to perform data transmission with the storage server according to the extranet communication address information.
On the basis of any of the above embodiments, when determining whether the mapping of the intranet and extranet communication address information is required, theprocessing module 703 is configured to:
and judging whether the internal and external network communication address information mapping is needed or not according to the preset configuration information.
On the basis of any of the above embodiments, when determining whether the intra-internet communication address information mapping is required according to the preset configuration information, theprocessing module 703 is configured to:
judging whether internal and external network communication address information mapping is needed or not according to a target configuration item included in the preset configuration information, wherein the target configuration item includes a field for indicating whether internal and external network mapping of the communication address information is needed or not; or
Judging whether the client side and the storage server are in the same local area network environment or not according to the communication address information of the client side, which is included in the preset configuration information; and if the storage server is not in the same local area network environment as the storage server, mapping the communication address information of the internal network and the external network.
On the basis of any of the above embodiments, when acquiring corresponding extranet communication address information according to the intranet communication address information, theprocessing module 703 is configured to:
and acquiring the external network communication address information corresponding to the internal network communication address information according to the internal network communication address information and a preset internal and external network communication address information mapping relation.
On the basis of any of the above embodiments, the data transmission request is a data upload request; thedata transmission module 704 is configured to, when performing data transmission with the storage server according to the extranet communication address information:
establishing connection with the storage server according to the external network communication address information;
sending data to be uploaded to the storage server, so that the storage server stores the data to be uploaded and generates a data identifier, and synchronizing the data to be uploaded to other storage servers in a group where the storage server is located;
and receiving the data identification returned by the storage server.
On the basis of any of the above embodiments, the data transmission request is a data download request; thedata transmission module 704 is configured to, when performing data transmission with the storage server according to the extranet communication address information:
establishing connection with the storage server according to the external network communication address information;
sending a data identifier of data to be downloaded to the storage server;
and receiving the data to be downloaded returned by the storage server according to the data identification.
On the basis of any of the above embodiments, thedata transmission module 704 is further configured to:
and if the fact that the internal and external network communication address information mapping is not needed is determined, data transmission is carried out between the internal network communication address information and the storage server.
The client provided in the embodiment of the present invention may be specifically configured to execute the method embodiments provided in fig. 2 to 4, and specific functions are not described herein again.
The client terminal provided by the embodiment of the invention sends a data transmission request to the tracking server through the client terminal; receiving intranet communication address information of a storage server returned by a tracking server, wherein the intranet communication address information of the storage server is determined by the tracking server according to a data transmission request; the client judges whether the mapping of the internal and external network communication address information is needed or not; if the fact that the internal and external network communication address information mapping is needed is determined, acquiring corresponding external network communication address information according to the internal network communication address information; and carrying out data transmission with the storage server according to the external network communication address information. In the embodiment, whether the mapping of the internal and external network communication address information is needed or not is judged by the client, so that the external network communication address information of the tracking server can be acquired under the condition that the client is in the external network, the connection with the storage server is realized, the data transmission is completed, and the stable operation of the system is ensured.
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device provided by the embodiment of the present invention may execute the processing flow provided by the data transmission method embodiment, as shown in fig. 8, theelectronic device 80 includes amemory 81, aprocessor 82, and a computer program; wherein the computer program is stored in thememory 81 and is configured to be executed by theprocessor 82 for the data transmission method described in the above embodiments. In addition, theelectronic device 80 may also have acommunication interface 83 for transmitting control instructions and data.
The electronic device in the embodiment shown in fig. 8 may be configured to implement the technical solution of the data transmission method embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.
In addition, the present embodiment also provides a computer-readable storage medium on which a computer program is stored, the computer program being executed by a processor to implement the data transmission method described in the above embodiment.
In addition, the present embodiment also provides a computer program product, which includes computer instructions, and the computer instructions, when executed by a processor, implement the data transmission method described in the foregoing embodiment.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It is obvious to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to perform all or part of the above described functions. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not described herein again.
The above embodiments are only used for illustrating the technical solutions of the embodiments of the present invention, and are not limited thereto; although embodiments of the present invention have been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.