Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a method, an apparatus, a storage medium and a system for transparent transmission of data in a distributed system, which are used for solving the problem that flexible and extensible transparent transmission of data cannot be performed in the distributed system in the prior art.
In order to achieve the above and other related objects, the present invention provides a distributed system data transparent transmission method, which includes the steps of obtaining a first transmission protocol request sent by a client, applying a preset transparent transmission protocol layer module based on the first transmission protocol request to determine transparent data to be transmitted, adding the transparent data to a transparent data container, taking out the transparent data from the transparent data container based on a second transmission protocol, identifying the transparent data, and adding the identified transparent data to a second transmission protocol request sent by a gateway to send to a destination address.
In an embodiment of the invention, the determining the transparent data to be transmitted based on the first transmission protocol request by applying a preset transparent protocol layer module includes obtaining a transparent identifier based on the first transmission protocol request header, determining the transparent data corresponding to the transparent identifier when the transparent identifier is obtained, obtaining a field corresponding to the transparent data defined by the logic assembly module in the first transmission protocol request header when the transparent identifier is not obtained, determining the transparent data corresponding to the field based on the field, and resolving the predefined resolvable field to obtain the transparent data corresponding to the field after resolving when the transparent data is not obtained based on the field.
In an embodiment of the invention, the method for adding the identified transparent data to the second transmission protocol request sent by the gateway to be sent to the destination address comprises the steps that a server of the second transmission protocol receives the second transmission protocol request, the transparent data is determined by applying the transparent protocol layer module based on the second transmission protocol request, the transparent data is added to a transparent data container corresponding to the server, the server takes the transparent data out of the transparent data container corresponding to the server based on the second transmission protocol, and the transparent data is identified and added to the second transmission protocol request sent by the server to be sent to the destination address.
In an embodiment of the invention, the first transmission protocol and the second transmission protocol are different.
In an embodiment of the present invention, after the transparent data is added to the transparent data container, the method further includes the steps of retrieving the transparent data from the transparent data container, identifying the transparent data, adding the identified transparent data to a second request of the first transmission protocol sent by the gateway, and directly sending the second request to the destination address.
In an embodiment of the present invention, the transparent data container is configured in a memory of the gateway.
Correspondingly, the invention provides a distributed system data transparent transmission device which comprises an acquisition module, a determination module, a first processing module and a second processing module, wherein the acquisition module is used for acquiring a first transmission protocol request sent by a client, the determination module is used for determining transparent transmission data to be transmitted based on the first transmission protocol request by applying a preset transparent transmission protocol layer module, the transparent transmission data is added into a transparent transmission data container, the first processing module is used for taking out the transparent transmission data from the transparent transmission data container based on a second transmission protocol, and the second processing module is used for identifying the transparent transmission data and adding the transparent transmission data after identification into a second transmission protocol request sent by a gateway so as to send the transparent transmission data to a destination address.
The transparent transmission protocol layer module comprises a protocol management module, a protocol butt joint module, a logic assembly module and a logic processing module, wherein the protocol management module is used for carrying out the definition management of the butt joint protocol registration and processing logic, the protocol butt joint module is used for defining the transparent transmission entrance of the transparent transmission protocol and the execution of the transparent transmission processing flow in different interactive protocols, the logic assembly module is used for creating a logic processing object based on the definition of the processing logic in the protocol management module and comprises a transparent transmission data receiving processor, an interactive protocol parameter transparent transmission data grabbing processor, an interactive protocol head transparent transmission data grabbing processor and an interactive protocol head field analysis transparent transmission processor, and the logic processing module is used for realizing the logic processing of the interactive protocol butt joint transparent transmission protocol and comprises a transparent transmission data receiving processor and a transparent transmission data sending processor.
The determining module is specifically configured to obtain a transparent identifier based on the first transport protocol request header, determine transparent data corresponding to the transparent identifier when the transparent identifier is obtained, obtain a field corresponding to the transparent data defined by the logic assembling module in the first transport protocol request header when the transparent identifier is not obtained, determine the transparent data corresponding to the field based on the field, and parse a token field to obtain the transparent data when the transparent data is not obtained based on the field.
The present invention provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the distributed system data transparent transmission method described above.
The invention provides a distributed system data transparent transmission system which comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor is used for running the computer program to realize the distributed system data transparent transmission method.
As described above, the distributed system data transparent transmission method and device of the invention have the following beneficial effects:
(1) The method can provide transparent transmission of various remote program call interaction data, and has high expandability.
(2) The method is suitable for the data transparent transmission scene of the heterogeneous distributed system, and low-cost access and use are provided for the data transparent transmission of different development languages.
(3) The support capability of data transparent transmission can be provided for the middleware and the base frame, the research and development cost of the data transparent transmission module is saved, and the research and development efficiency is improved.
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.
It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.
The data transparent transmission method and device for the distributed system can provide transparent transmission of various remote program call interaction data, has high expandability, can be suitable for data transparent transmission scenes of heterogeneous distributed systems, provides low-cost access and use for data transparent transmission of different development languages, and can provide supporting capability for data transparent transmission for middleware and a base frame, so that research and development cost of a data transparent transmission module in the method and device is saved, and research and development efficiency is improved.
As shown in fig. 1, in this embodiment, the distributed system data transparent transmission method of the present invention includes the following steps:
step S1, a first transmission protocol request sent by a client is obtained.
Specifically, when there is a data transparent transmission requirement in the distributed system, the transmission protocol from the client to the gateway is a first transmission protocol, and before the client sends the first transmission protocol request, transparent transmission data is added into a first transmission protocol request header of the first transmission protocol request, and is sent to the gateway together with the transmission data.
And step S2, based on the first transmission protocol request, a preset transparent transmission protocol layer module is applied to determine transparent transmission data to be transmitted, and the transparent transmission data is added into a transparent transmission data container.
The transparent transmission protocol layer module comprises a protocol management module, a protocol butt joint module, a logic assembly module and a logic processing module, wherein the protocol management module is used for carrying out the definition management of the butt joint protocol registration and processing logic, the protocol butt joint module is used for defining transparent transmission inlets for executing transparent transmission protocols in different interactive protocols and carrying out transparent transmission processing flows, the logic assembly module is used for creating logic processing objects based on the definition of the processing logic in the protocol management module and comprises a transparent transmission data receiving processor, an interactive protocol parameter transparent transmission data grabbing processor, an interactive protocol head transparent transmission data grabbing processor and an interactive protocol head field analysis transparent transmission processor, and the logic processing module is used for realizing the logic processing of the interactive protocol butt joint transparent transmission protocol and comprises a transparent transmission data receiving processor and a transparent transmission data sending processor.
More specifically, as shown in fig. 2, in this embodiment, the transmission-through data transmission flow chart of the present invention is shown. After the modules of the transparent transmission protocol layer are set, the transparent transmission protocol and interaction protocol based butt joint and the data transmission network flow are used for determining the transparent transmission data transmission flow, and the method comprises the steps of preprocessing transparent transmission data by a logic processing layer 21, sending a transmission protocol request containing the transparent transmission data through a transparent transmission protocol layer 22, transmitting the transmission protocol request added with the transparent transmission data to a communication interaction layer 23, sending the transmission protocol request to a receiving end through a network transmission 24, receiving the transmission protocol request containing the transparent transmission data by the receiving end through the communication interaction layer 23, acquiring the transparent transmission data from the transmission protocol request through the transparent transmission protocol layer 22, and operating the transparent transmission data through the logic processing layer 21, so that the full-link transparent transmission of the transparent transmission data is completed.
Further specifically, based on the first transmission protocol request header, a transparent identifier is obtained, transparent data corresponding to the transparent identifier is determined when the transparent identifier is obtained, a field corresponding to the transparent data defined by the logic assembly module in the first transmission protocol request header is obtained when the transparent identifier is not obtained, the transparent data corresponding to the field is determined based on the field, and when the corresponding transparent data is not obtained based on the field, a predefined resolvable field is resolved to obtain the transparent data corresponding to the field after the field is resolved.
For example, the preset transparent transmission identifier of transparent transmission data is_tt_m_, and the client needs to transmit transparent transmission data from the communication network of the hypertext transfer protocol (Hyper Text transfer protocol, http) to the communication network of the distributed service framework protocol Dubbo. The method comprises the steps that firstly, a client adds transparent transmission data into a first transmission protocol request, the first transmission protocol request containing the transparent transmission data and transmitted data transmission are sent to a gateway, wherein the first transmission protocol between the client and the gateway is an Http protocol, the gateway is in communication connection with a Dubbo-protocol server, and the Dubbo-protocol server is connected with a requested destination address through a second transmission protocol Dubbo protocol. 4 logical definitions of transparent transmission processing are defined in a logical assembly module of the transparent transmission protocol layer, wherein the logical definitions correspond to 4 transparent transmission input processors respectively and comprise transparent transmission data receiving processors of the Http protocol, transparent transmission data grabbing processors of Http protocol parameters, transparent transmission data grabbing processors of the Http protocol header, and analysis transparent transmission processors for analyzing fields in the Http protocol header. The data submitted by the Http protocol is provided with interface service data, public data and the like, wherein the fields corresponding to the public data comprise token fields and the like, 4 transparent transmission processing logics are defined in a transparent transmission protocol layer logic assembly module based on Dubbo protocol and transparent transmission protocol layer modules in advance, the 4 transparent transmission processing logics respectively correspond to 4 transparent transmission input processors and comprise a transparent transmission data receiving processor of Dubbo protocol, a Dubbo protocol parameter transparent transmission data grabbing processor, a Dubbo protocol head transparent transmission data grabbing processor and a transparent transmission analysis processor for analyzing the fields in the Dubbo protocol head. The gateway is a network server, a set of transparent transmission receiving processors related to the Http protocol and a set of transparent transmission sending processors related to the Dubbo protocol are created in advance through the abstract object of protocol transparent transmission butt joint and the logic definition of transparent transmission processing. In an actual transparent data transmission scene, firstly, transparent data is acquired from an Http protocol request header according to whether a request header name contains a transparent identifier, when the transparent identifier is obtained, transparent data corresponding to the transparent identifier is determined, the transparent data is added into a transparent data container, when the transparent identifier cannot be detected from the Http protocol request header, a field corresponding to the transparent data defined by transparent processing logic is acquired from the Http protocol request, the transparent data corresponding to the field is determined based on the field, and when the transparent data cannot be acquired through the field, the predefined resolvable field is resolved to obtain the transparent data corresponding to the field after the field is resolved.
More specifically, a transparent data container is configured in the memory in advance, and when transparent data is identified, the transparent data is stored in the transparent data container; the transparent transmission data container is realized in a higher level, and can be compatible with the realization of various interactive protocols and transparent transmission among asynchronous threads.
And step S3, based on a second transmission protocol, taking out the transparent transmission data from the transparent transmission data container.
Specifically, the transparent transmission data is obtained from the transparent transmission data container so as to mark the transparent transmission data.
Further specifically, when the transparent data is not acquired from the transparent data container, a transparent data container is newly created in the memory, the transparent data is added into the transparent data container by a code adding mode, and then the newly added transparent data is acquired from the transparent data container so as to mark the transparent data.
And S4, marking the transparent transmission data, and adding the marked transparent transmission data to a second transmission protocol request sent by the gateway so as to send the second transmission protocol request to a destination address.
The method comprises the steps of receiving a second transmission protocol request by a server of a second transmission protocol, applying a transparent transmission protocol layer module based on the second transmission protocol request to determine transparent transmission data, adding the transparent transmission data into a transparent transmission data container corresponding to the server, taking out the transparent transmission data from the transparent transmission data container corresponding to the server based on the second transmission protocol, identifying the transparent transmission data, and adding the identified transparent transmission data into the second transmission protocol request sent by the server to send the transparent transmission data to the destination address.
For example, in the above example, after the gateway sends the Dubbo protocol request with the transparent data added to the Dubbo protocol server, the Dubbo protocol server applies the transparent protocol layer module based on the Dubbo protocol request to determine the transparent data, specifically, firstly, whether the transparent data is acquired from the Dubbo protocol request header according to whether the request header name includes the transparent identifier, when the transparent identifier is obtained, the transparent data corresponding to the transparent identifier is determined, the transparent data is added to the transparent data container, when the transparent identifier cannot be detected from the Dubbo protocol request header, the field corresponding to the transparent data defined by the transparent processing is acquired from the Dubbo protocol request, the transparent data corresponding to the field is determined based on the field, and when the transparent data cannot be acquired through the field, the predefined resolvable field is again resolved to obtain the transparent data corresponding to the field after the field resolution. And adding the transparent transmission data into a transparent transmission data container, taking out the transparent transmission data from the transparent transmission data container by the Dubbo protocol server based on Dubbo protocol, identifying the transparent transmission data, adding the identified transparent transmission data into a Dubbo protocol request, and sending the request to a destination address.
Further specifically, the first transmission protocol and the second transmission protocol are different. In the above examples, more than one interactive protocol is used, wherein the first transmission protocol is Http protocol, the second transmission protocol is Dubbo protocol, the data transmission processing is performed in all links associated with multiple interactive protocols, in other example scenarios, the interactive protocol of the data transmission processing can be extended to more protocols according to actual network transmission scenarios, or the data transmission processing can also be performed in the same interactive protocol.
The data transparent transmission processing process of the same interactive protocol is as follows.
The method comprises the steps of obtaining a first transmission protocol request sent by a client, determining transmission data to be transmitted by applying a preset transmission protocol layer module based on the first transmission protocol request, adding the transmission data into a transmission data container, taking out the transmission data from the transmission data container, identifying the transmission data, adding the identified transmission data into a second request of the first transmission protocol sent by a gateway, and directly sending the second request to the destination address.
For example, the preset transparent transmission identifier of the transparent transmission data is_tt_m_, and the client needs to transmit the transparent transmission data to the destination address in the communication network of the Http protocol. Firstly, the client adds the transparent data into the Http protocol request, and sends the Http protocol request containing the transparent data and the transmitted data transmission to the gateway. 4 logic definitions are defined for transparent transmission processing in a logic assembly module of the transparent transmission protocol layer, wherein the logic definition corresponds to 4 transparent transmission input processors respectively and comprises a transparent transmission data receiving processor of the Http protocol, a transparent transmission data grabbing processor of the Http protocol parameter, a transparent transmission data grabbing processor of the Http protocol header and a transparent transmission resolving processor for resolving fields in the Http protocol header. The data submitted by the Http protocol has interface service data, public data and the like, wherein the fields corresponding to the public data comprise token fields and the like, and a group of transparent receiving processors related to the Http protocol are created for the gateway through the abstract objects of the transparent transmission butt joint of the protocol and the logic definition of transparent transmission processing in advance. In an actual transparent data transmission scene, firstly, transparent data is acquired from an Http protocol request header according to whether a request header name contains a transparent identifier, when the transparent identifier is obtained, transparent data corresponding to the transparent identifier is determined, the transparent data is added into a transparent data container, when the transparent identifier cannot be detected from the Http protocol request header, a field corresponding to the transparent data defined by transparent processing logic is acquired from the Http protocol request, the transparent data corresponding to the field is determined based on the field, and when the transparent data cannot be acquired through the field, the predefined resolvable field is resolved to obtain the transparent data corresponding to the field after the field is resolved. The gateway takes out the transparent transmission data from the transparent transmission data container when the gateway needs to transmit the transparent transmission data to the destination address, identifies the transparent transmission data, adds the transparent transmission data after identification to a second request of an Http protocol sent by the gateway, and directly sends the second request to the destination address. As shown in fig. 3, in the present embodiment, an example of the present invention is schematically shown. When a request for receiving the transparent transmission data is received, based on the first transmission protocol request, a preset transparent transmission protocol layer module is applied to determine the transparent transmission data 31 to be transmitted. Specifically, the method includes the steps of firstly acquiring transparent data from a first transmission protocol request header according to whether a request header name contains transparent identification, determining transparent data corresponding to the transparent identification when the transparent identification is obtained, adding the transparent data into a transparent data container, acquiring a field corresponding to the transparent data defined by logic of transparent processing from the first transmission protocol request when the transparent identification cannot be detected from the first transmission protocol request header, determining the transparent data corresponding to the field based on the field, analyzing a predefined resolvable field to obtain the transparent data corresponding to the field analysis when the transparent data cannot be obtained through the field, and adding the transparent data into the transparent data container 33 after the transparent data 32 is obtained. When there is a request for sending the transparent data, for example, when the gateway needs to transmit the transparent data to a certain destination address 34, the gateway takes out the transparent data 35 from the transparent data container, identifies the transparent data, adds the transparent data after identification to a second request 36 of a second transmission protocol sent by the gateway, and sends the second request to the corresponding destination address.
As shown in fig. 4, in this embodiment, the distributed system data transparent transmission device of the present invention includes:
An obtaining module 41, configured to obtain a first transport protocol request sent by a client;
A determining module 42, configured to determine, based on the first transmission protocol request, transmission data to be transmitted by applying a preset transmission protocol layer module;
A first processing module 43, configured to extract the transparent data from the transparent data container based on a second transmission protocol;
the second processing module 44 is configured to identify the transparent transmission data, and add the transparent transmission data after identification to a second transport protocol request sent by the gateway, so as to send the second transport protocol request to the destination address.
The transparent transmission protocol layer module comprises a protocol management module, a protocol butt joint module, a logic assembly module and a logic processing module, wherein the protocol management module is used for carrying out the definition management of the butt joint protocol registration and processing logic, the protocol butt joint module is used for defining the transparent transmission entrance of the transparent transmission protocol and the execution of the transparent transmission processing flow in different interactive protocols, the logic assembly module is used for creating a logic processing object based on the definition of the processing logic in the protocol management module and comprises a transparent transmission data receiving processor, an interactive protocol parameter transparent transmission data grabbing processor, an interactive protocol head transparent transmission data grabbing processor and an interactive protocol head field analysis transparent transmission processor, and the logic processing module is used for realizing the logic processing of the interactive protocol butt joint transparent transmission protocol and comprises a transparent transmission data receiving processor and a transparent transmission data sending processor. The determining module 42 is specifically configured to obtain a transparent identifier based on the first transport protocol request header, determine transparent data corresponding to the transparent identifier when the transparent identifier is obtained, obtain a field corresponding to the transparent data defined by the logic assembling module in the first transport protocol request header when the transparent identifier is not obtained, determine the transparent data corresponding to the field based on the field, and parse a token field to obtain the transparent data when the transparent data is not obtained based on the field.
In further detail, as shown in fig. 5, in the embodiment, the transparent transmission protocol layer module of the present invention includes a protocol management module 51, a protocol docking module 52, a logic assembling module 53 and a logic processing module 54. The protocol management module 51 is configured to create transparent-transmission protocol implementation of different protocols when a module of the transparent-transmission protocol layer defined by application works, effectively improve acquisition performance of the protocol implementation by using a cache policy, and update definition of the processing logic through an interface.
And the protocol docking module 52 is used for docking the interactive protocol and the transparent transmission protocol, and when the definition of the processing logic of the protocol management module is changed, the processing logic in the logic assembly module is newly assembled through the protocol docking module. The protocol transparent transmission method comprises the steps of including an abstract object of protocol transparent transmission and connection, and the interactive protocol realizes the transparent transmission protocol connection according to the abstract object.
The logic assembling module 53 is configured to create a logic processing object by defining processing logic in the protocol management module, and package the logic processing object into a list, and the logic processing module uses an iterator to perform reasonable arrangement on logic processing, so as to reduce judgment of an intermediate process and improve execution speed. The method comprises the steps of transmitting a logical definition of processing, a data-transmitting receiver and a data-transmitting transmitter, maintaining the data-transmitting receiver and the data-transmitting transmitter by an abstract object in a protocol docking module, wherein the data-transmitting receiver is used for maintaining a data-transmitting input processor in a logical processing module appointed by the logical definition of processing, the data-transmitting transmitter is used for maintaining a data-transmitting output processor in a logical processing module appointed by the logical definition of processing, and the logical definition of processing creates a plurality of logical processors of transmitting according to the definition, and the logical definition of processing comprises a data-transmitting receiving processor, an interactive protocol parameter data-transmitting grabbing processor, an interactive protocol header data-transmitting grabbing processor and an interactive protocol header field analysis data-transmitting processor.
The logic processing module 54, i.e., the specific pass-through logic, is completed by the module, which packages each individual data pass-through logic into a pass-through ingress and a pass-through egress, each individual data pass-through logic for receiving requests containing pass-through data and for delivering requests containing pass-through data. The system comprises a transparent transmission data receiving processor and a transparent transmission data transmitting processor.
The technical features of the specific implementation of the distributed system data transparent transmission device in this embodiment are basically the same as the principle of each step in the distributed system data transparent transmission method in embodiment 1, and the general technical content between the method and the device is not repeated.
The storage medium of the present invention stores a computer program which, when executed by a processor, implements the distributed system data transparent transmission method described above.
As shown in fig. 6, in the present embodiment, the distributed system data transmission system of the present invention includes a processor 61 and a memory 62.
The memory 62 is used for storing a computer program.
The memory 62 includes a ROM, RAM, magnetic disk, U-disk, memory card, or optical disk, etc. various media that can store program codes.
The processor 61 is connected to the memory 62, and is configured to execute the computer program stored in the memory 62, so that the distributed system data transparent transmission system executes the distributed system data transparent transmission method described above.
Preferably, the processor 61 may be a general-purpose processor, including a central Processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), a graphics processor (Graphic Processing Unit, GPU) and the like, where in practical applications, the general-purpose processor is mainly a graphics processor, and may also be a digital signal processor (DIGITAL SIGNAL Processing, DSP for short), an Application SPECIFIC INTEGRATED Circuit (ASIC for short), a Field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, or discrete hardware component.
In summary, the distributed system data transparent transmission method and device can provide transparent transmission of various remote program call interaction data, have high expandability, can be suitable for data transparent transmission scenes of heterogeneous distributed systems, provide low-cost access and use for data transparent transmission of different development languages, and can provide supporting capability for data transparent transmission for middleware and a base frame, so that research and development cost of a data transparent transmission module in the method and device is saved, and research and development efficiency is improved. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.