Page data transmission method and systemTechnical Field
The invention relates to the technical field of internet communication, in particular to a page data transmission method and a page data transmission system.
Background
When a client of a browser is installed on a mobile terminal, when a page is browsed through the client, how to quickly return page data to the client by a server is a key for influencing the response speed of the page. And the page data includes the main document page and web resources related to the page resolution. The web resources include non-image web resources such as picture resources, external JavaScript (script) files and CSS (Cascading Style Sheets) Style files. For a site accessed by a user at a high frequency, the web resources under each path of the site are the same or similar, so that the key for improving the page response speed is to avoid repeatedly issuing the same web resources.
At present, a web resource transmission method based on a client caching technology is generally adopted, in the method, a server obtains a web resource of a page corresponding to a page domain name accessed by a client, judges whether the web resource can be cached, and returns a cache identifier, the web resource and a unique identifier thereof to the client if the web resource can be cached. And caching the page domain name, the web resource and the unique identifier thereof by the client according to the cache identifier. When the client initiates the page request under the page domain name again, all the web resources related to the page domain name are determined from the cache, the unique identifications of the web resources are combined into an identification set, and the identification set is contained in the page request. And after receiving the page request, the server calculates the unique identifiers of all the web resources corresponding to the page domain name, compares the calculated unique identifiers with the unique identifiers in the identifier set contained in the page request, and if the identifier set contains a certain calculated unique identifier, determines that the client caches the web resources corresponding to the unique identifiers, and only returns indication information indicating that the client uses the locally cached web resources without returning the web resources. After receiving the indication information, the client directly uses the locally cached web resource to cooperate with page resolution.
The method for transmitting the web resources has a certain effect on high-frequency access of a user to certain specific sites, but the user does not necessarily have fixed access to a certain site. And the cache capacity of the client is limited, the cache amount of the non-image web resources influencing the page analysis is very low probably because a large number of picture resources which do not influence the page analysis are cached, and the server must issue complete data of the non-image web resources when the client does not cache the non-image web resources, so that the page response time is prolonged, and the transmission flow is wasted.
Disclosure of Invention
In view of this, an object of the embodiments of the present invention is to provide a page data transmission method and system, where when a client does not cache a non-graphic web resource, a server only issues incremental data corresponding to the non-graphic web resource to the client, so as to avoid issuing complete data of the non-graphic web resource.
The embodiment of the invention provides a page data transmission method, which comprises the following steps:
the server establishes one or more different super webpage templates and stores the one or more different super webpage templates in the server and the client, wherein each super webpage template comprises the same type of non-image web resources under each path of a certain type of website;
the server receives and analyzes a page request sent by the client;
when the local cache of the client stores the required non-image web resource, the server instructs the client to directly read the required non-image web resource from the local cache of the client, otherwise
The server performs incremental compression operation on the non-image web resources acquired according to the page request and the super webpage template to obtain incremental data;
and the server transmits the incremental data serving as page response data to the client.
Preferably, the conditions met by the certain category of websites include: the number of the similar non-graphic web resources under each path of the website is less than a preset value, and the total capacity of the similar non-graphic web resources is less than a preset capacity; the user access amount of the website is higher than a preset threshold value; the frequency of the web site revising is lower than the preset frequency.
Preferably, the method further comprises the following steps: the data structure of the super webpage template comprises a non-image web resource type, a website domain name list of a website, the version number of the super webpage template and text data combined by the content of the non-image web resource type in the website.
Preferably, the method further comprises the following steps: the non-graphical web resource types include a script class and a style class.
Preferably, in the step of establishing one or more different super web page templates by the server, the establishing method includes:
the server determines a plurality of websites with completely consistent organizational structures and forms site domain names of the websites into a site domain name list;
the server determines the non-image web resource types under each path of the website and sets the version number of the super webpage template;
the server combines the contents of the non-image web resource types in the website into text data;
and the server encrypts and encodes the text data through a preset encryption algorithm and a preset encoding algorithm to obtain an identifier of the super webpage template, and the identifier is used as the file name of the super webpage template.
Preferably, the step of merging the contents of the non-graph web resource types in the website into text data by the server includes:
the server directly merges the contents of the non-image web resource types under each path of the website into text data based on the file level; or,
the server determines the similarity degree between the data blocks in the content of the non-image web resource type under each path of the website, extracts the data blocks with the similarity degree larger than a preset similarity value from the content of the non-image web resource type under each path, and combines the extracted data blocks into text data.
Preferably, the method further comprises the following steps: when a client side is ready to access a website, one or more super webpage templates stored by the client side are retrieved, the super webpage template suitable for the website is found out, a client side cache list corresponding to a website domain name of the website is obtained from a local cache, and the client side cache list comprises a non-image web resource identifier corresponding to the website domain name cached by the client side; and the client sends a page request to the server, and the site domain name of the website, the identifier of the super webpage template, the type and the version number of the non-image web resource and the client cache list are provided in the page request.
Preferably, the server parses the website domain name of the website, the identifier of the super webpage template, the non-image web resource type, the version number and the client cache list from the page request;
the server acquires a non-graph web resource which corresponds to the site domain name and is the same as the analyzed non-graph web resource in type;
and calculating the identification of the acquired non-image web resource, determining whether the client cache list contains the calculated identification, if so, judging that the required non-image web resource exists in the local cache of the client, otherwise, judging that the required non-image web resource does not exist in the local cache of the client.
Preferably, in the step of calculating the identification of the acquired non-graph web resource, the method comprises: and carrying out hash operation on the address corresponding to the non-graph web resource and the content of the non-graph web resource to obtain a hash value corresponding to the non-graph web resource, and using the hash value as the identifier of the non-graph web resource.
Preferably, in the step of instructing, by the server, the client to directly read the required non-graph web resource from the local cache of the client, the method includes:
the server sends cached indication information to the client, wherein the cached indication information comprises the identification of the required non-graph web resource and the cached identification;
and the client receives the cached indication information and directly acquires the required non-image web resources from the local cache of the client according to the identification of the required non-image web resources.
Preferably, the step of performing incremental compression operation on the non-image web resource acquired according to the page request and the super web page template by the server to obtain incremental data includes:
the server analyzes the identifier and the version number of the super webpage template corresponding to the site domain name from the page request;
and the server acquires a corresponding super webpage template from one or more super webpage templates stored in the server according to the analyzed identifier and version number of the super webpage template, and performs increment compression operation according to the acquired non-image web resource and the super webpage template to obtain increment data.
Preferably, the method further comprises the following steps: the server caches the identification of the non-image web resource subjected to the incremental compression operation, the identifier of the super webpage template and the incremental compression operation result in a local cache system of the server, wherein the incremental compression operation result is successful identification and incremental data, or the incremental compression operation result is failed identification.
Preferably, the method further comprises the following steps: before the server carries out increment compression operation, inquiring whether a local cache system of the server caches an identification of a non-image web resource and an increment compression operation result corresponding to an identifier of a super webpage template;
when the local cache system of the server does not cache the corresponding increment compression operation result, the server performs increment compression operation on the non-image web resource and the super webpage template to obtain increment data;
and when the server local cache system caches the corresponding increment compression operation result, directly acquiring the corresponding increment compression operation result from the server local cache system.
Preferably, when the server local cache system does not cache the corresponding increment compression operation result and the server performs the increment compression operation to obtain the increment data,
the server calculates the ratio of the capacity of the incremental data to the capacity of the non-image web resource to obtain an incremental compression rate corresponding to the non-image web resource;
if the incremental compression rate is smaller than a preset compression threshold value, the server caches the identification of the non-image web resource, the identifier of the super webpage template, the successful identification and the incremental data to a local cache system of the server;
if the incremental compression rate is larger than or equal to the preset compression threshold value, the server caches the identification of the non-image web resource, the identifier of the super webpage template and the failure identification to a local cache system of the server through the incremental cache service.
Preferably, the method further comprises the following steps: responding the page request according to the increment compression operation result,
when the corresponding increment compression operation result comprises a successful identifier and increment data, the server responds to the page request in an increment mode;
and when the corresponding increment compression operation result comprises a failure identifier, the server responds to the page request in a complete data mode.
Preferably, the step of the server responding to the page request in an incremental manner includes: and the server encapsulates the page response head information of the non-image web resource, the client cache judgment identifier, the incremental operation identifier, the identifier of the super webpage template, the original size of the non-image web resource, the size of the incremental data and the incremental data through a preset protocol format, and transmits the encapsulated data serving as page response data to the client.
Preferably, the method further comprises the following steps: the client receives the page response data transmitted by the server and analyzes the page response data through the preset protocol format;
restoring the required non-image web resources according to the locally cached super web page template corresponding to the received identifier of the super web page template and the incremental data;
judging whether the reduction operation is successful according to the size of the reduced non-image web resource and the original size included by the page response data;
and if the restoration is successful and the client cache judgment identifier indicates that caching is needed, caching the needed non-image web resources in the local client.
Preferably, the step of the server responding to the page request in a complete data manner includes:
the server compresses the non-image web resource in a preset compression mode to obtain compressed data;
and encapsulating the page response head information of the non-image web resource, the client cache judgment identifier, the identifier of the preset compression mode and the compressed data through a preset protocol format, and transmitting the encapsulated data serving as page response data to the client.
Preferably, the method further comprises the following steps: the client receives the page response data transmitted by the server and analyzes the page response data through the preset protocol format;
decompressing the compressed data by adopting a corresponding preset compression mode according to the identifier of the preset compression mode to obtain the required non-image web resource;
and if the client cache judgment identifier indicates that caching is needed, caching the needed non-image web resource in the local client.
Preferably, the method further comprises the following steps: the client manages one or more stored super webpage templates independently from a local cache of the client; the client closes or opens the super webpage template corresponding to a certain type of website through the control parameters issued by the background.
Preferably, the method further comprises the following steps: the server detects the version changing degree of the website corresponding to each super webpage template every other preset time period, and when the version changing degree of a website is detected to exceed the preset version changing degree, the server generates a new version of the super webpage template corresponding to the website according to the website after version changing.
Preferably, the method further comprises the following steps: the server determines whether the version of the super webpage template corresponding to the website to be accessed cached by the client is the latest version or not according to the page request sent by the client;
when the version of the super webpage template cached by the client is determined not to be the latest version, the server transmits updating indication information to the client;
and the client downloads and stores the latest version of the super webpage template from the server according to the update indication information.
The embodiment of the invention also provides a page data transmission system, which comprises: a server and a client, wherein:
the server is used for establishing one or more different super webpage templates and storing the one or more different super webpage templates in the server and the client, and each super webpage template comprises the same type of non-image web resources under each path of a certain type of websites;
the server is used for receiving and analyzing the page request sent by the client;
the server is used for indicating the client to directly read the required non-image web resources from the local cache of the client when the required non-image web resources are stored in the local cache of the client, otherwise, performing increment compression operation on the non-image web resources acquired according to the page request and the super webpage template to obtain increment data; and transmitting the incremental data serving as page response data to the client.
Preferably, the conditions met by the certain category of websites include:
the number of the similar non-graphic web resources under each path of the website is less than a preset value, and the total capacity of the similar non-graphic web resources is less than a preset capacity; the user access amount of the website is higher than a preset threshold value; the frequency of the web site revising is lower than the preset frequency.
Preferably, the method further comprises the following steps: the data structure of the super webpage template comprises a non-image web resource type, a website domain name list of a website, the version number of the super webpage template and text data combined by the content of the non-image web resource type in the website.
Preferably, the method further comprises the following steps: the non-graphical web resource types include a script class and a style class.
Preferably, the server is configured to determine a plurality of websites with completely consistent organizational structures and form a list of site domain names of the websites; determining the types of non-image web resources under each path of a website, and setting the version number of the super webpage template; merging the content of the non-graph web resource type in the website into text data; and carrying out encryption and coding operation on the text data through a preset encryption algorithm and a preset coding algorithm to obtain an identifier of the super webpage template, and simultaneously taking the identifier as the file name of the super webpage template.
Preferably, the server is used for directly merging the contents of the non-graph web resource types under each path of the website into text data based on the file level; or,
the server is used for determining the similarity degree between the data blocks in the content of the non-image web resource type under each path of the website, extracting the data blocks with the similarity degree larger than a preset similarity value from the content of the non-image web resource type under each path, and combining the extracted data blocks into text data.
Preferably, the method further comprises the following steps: the client is used for searching one or more super webpage templates stored by the client when a website is ready to be accessed, finding out the super webpage template suitable for the website, and acquiring a client cache list corresponding to a website domain name of the website from a local cache, wherein the client cache list comprises a mark of a non-image web resource corresponding to the website domain name cached by the client; and sending a page request to a server, wherein the page request provides the website domain name of the website, the identifier of the super webpage template, the type and the version number of the non-image web resource and the client cache list.
Preferably, the server is configured to parse, from the page request, a site domain name of the website, an identifier of a super webpage template, a non-image web resource type, a version number, and the client cache list; acquiring a non-graph web resource which corresponds to the site domain name and is the same as the analyzed non-graph web resource in type; and calculating the identification of the acquired non-image web resource, determining whether the client cache list contains the calculated identification, if so, judging that the required non-image web resource exists in the local cache of the client, otherwise, judging that the required non-image web resource does not exist in the local cache of the client.
Preferably, the server is further configured to perform a hash operation on the address corresponding to the non-graph web resource and the content of the non-graph web resource to obtain a hash value corresponding to the non-graph web resource, and use the hash value as the identifier of the non-graph web resource.
Preferably, the server is configured to send cached indication information to the client, where the cached indication information includes an identifier of the required non-graph web resource and a cached identifier;
and the client is used for receiving the cached indication information and directly acquiring the required non-image web resources from the local cache of the client according to the identification of the required non-image web resources.
Preferably, the server is configured to parse an identifier and a version number of a super webpage template corresponding to the site domain name from the page request; and acquiring a corresponding super webpage template from one or more super webpage templates stored in the super webpage template according to the analyzed identifier and the version number of the super webpage template, and performing increment compression operation according to the acquired non-image web resource and the super webpage template to obtain increment data.
Preferably, the method further comprises the following steps: the server is also used for caching the identification of the non-image web resource subjected to the incremental compression operation, the identifier of the super webpage template and the incremental compression operation result in a local cache system of the server, wherein the incremental compression operation result is successful identification and incremental data, or the incremental compression operation result is failed identification.
Preferably, the method further comprises the following steps: the server is also used for inquiring whether a local cache system of the server caches the identification of the non-image web resource and the incremental compression operation result corresponding to the identifier of the super webpage template before incremental compression operation is carried out; when the local cache system of the server does not cache the corresponding increment compression operation result, carrying out increment compression operation on the non-image web resource and the super webpage template to obtain increment data; and when the server local cache system caches the corresponding increment compression operation result, directly acquiring the corresponding increment compression operation result from the server local cache system.
Preferably, the server is further configured to calculate a ratio between a capacity of the incremental data and a capacity of the non-image web resource to obtain an incremental compression rate corresponding to the non-image web resource, when the server local cache system does not cache a corresponding incremental compression operation result and the server performs an incremental compression operation to obtain the incremental data; if the incremental compression rate is smaller than a preset compression threshold value, caching the identification of the non-image web resource, the identifier of the super webpage template, the successful identification and the incremental data into a local cache system of a server; if the incremental compression rate is larger than or equal to the preset compression threshold value, caching the identification of the non-image web resource, the identifier of the super webpage template and the failure identification to a local cache system of a server.
Preferably, the server is configured to respond to the page request in an incremental manner when the corresponding incremental compression operation result includes the successful identifier and the incremental data; and when the corresponding increment compression operation result comprises a failure identifier, responding to the page request in a complete data mode.
Preferably, the server is further configured to encapsulate, by using a preset protocol format, the page response header information of the non-image web resource, the client cache judgment identifier, the incremental operation identifier, the identifier of the super web page template, the original size of the non-image web resource, the size of the incremental data, and transmit the encapsulated data to the client as page response data.
Preferably, the method further comprises the following steps: the client is used for receiving the page response data transmitted by the server and analyzing the page response data through the preset protocol format; restoring the required non-image web resources according to the locally cached super web page template corresponding to the received identifier of the super web page template and the incremental data; judging whether the reduction operation is successful according to the size of the reduced non-image web resource and the original size included by the page response data; and if the restoration is successful and the client cache judgment identifier indicates that caching is needed, caching the needed non-image web resources in the local client.
Preferably, the server is further configured to compress the non-graph web resource in a preset compression manner to obtain compressed data; and encapsulating the page response head information of the non-image web resource, the client cache judgment identifier, the identifier of the preset compression mode and the compressed data through a preset protocol format, and transmitting the encapsulated data serving as page response data to the client.
Preferably, the method further comprises the following steps: the client is used for receiving the page response data transmitted by the server and analyzing the page response data through the preset protocol format; decompressing the compressed data by adopting a corresponding preset compression mode according to the identifier of the preset compression mode to obtain the required non-image web resource; and if the client cache judgment identifier indicates that caching is needed, caching the needed non-image web resource in the local client.
Preferably, the method further comprises the following steps: the client is also used for managing one or more super webpage templates stored by the client in a mode independent of local cache of the client; and closing or opening the super webpage template corresponding to a certain type of website through the control parameters issued by the background.
Preferably, the method further comprises the following steps: the server is further used for detecting the version changing degree of the website corresponding to each super webpage template every other preset time period, and when the version changing degree of a website is detected to exceed the preset version changing degree, the server generates a new version of the super webpage template corresponding to the website according to the website after version changing.
Preferably, the method further comprises the following steps: the server is also used for determining whether the version of the super webpage template corresponding to the website to be accessed cached by the client is the latest version according to the page request sent by the client; when the version of the super webpage template cached by the client is determined not to be the latest version, the server transmits updating indication information to the client;
and the client is also used for downloading and storing the latest version of the super webpage template from the server according to the updating indication information.
The page data transmission method and the system provided by the embodiment of the invention have the beneficial effects that: and establishing one or more different super webpage templates and storing the one or more different super webpage templates in a server and a client, wherein the super webpage templates comprise the same type of non-graphic web resources under each path of a certain type of websites. And when the required non-image web resource does not exist in the local cache of the client, the server performs increment compression operation on the non-image web resource acquired according to the page request and the super webpage template to obtain increment data, and transmits the increment data to the client. Because the server only transmits the incremental data between the non-image web resources and the super webpage template to the client, the transmission of complete data of the non-image web resources is avoided, the page response time is shortened, and the transmission flow is saved.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
FIG. 1A is a flow chart of a page data transmission method of the present invention;
FIG. 1B illustrates a flow chart of updating a super webpage template according to the present invention;
fig. 2 is a schematic diagram illustrating a page data transmission system according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Considering that in the web resource transmission method based on client-side caching in the related art, the client-side caching capacity is limited, the caching quantity affecting the non-graphic web resources is very low probably due to the fact that a large number of picture resources are cached, and when the client-side does not cache the non-graphic web resources, the server must issue complete data of the non-graphic web resources, so that the page response time is prolonged, and the transmission flow is wasted.
Based on this, the embodiment of the invention provides a page data transmission method and a page data transmission system. The following is described by way of example.
Example 1
Referring to fig. 1A, an embodiment of the present invention provides a page data transmission method. The method specifically comprises the following steps:
step 101: the server establishes one or more different super webpage templates and stores the different super webpage templates in the server and the client, wherein each super webpage template comprises the same type of non-image web resources under each path of a certain type of websites.
In the embodiment of the present invention, the server may be a background server or a proxy server corresponding to the browser.
The data structure of the super webpage template comprises a non-image web resource type, a website domain name list of a website, the version number of the super webpage template and text data combined by the content of the non-image web resource type in the website. The non-image web resources comprise an external JavaScript script and a CSS style sheet, the non-image web resource types comprise a script type and a style type, and the super webpage template can be a script super template and a style super template.
The use of the super web page template is based on the site domain name of the web site because the convergence of non-graphic web resources and data size of different web sites are very different. The super webpage template is specific to all similar non-graphic web resources under each path of a website, and mainly specific to a JavaScript script and a CSS style sheet, namely for a given site domain name, the super webpage template corresponding to the site domain name at most comprises a script super template and a style super template.
In the embodiment of the present invention, the conditions that the website suitable for the super webpage template needs to satisfy include:
(1) the number of the similar non-graphic web resources under each path of the website is less than a preset value, and the total capacity of the similar non-graphic web resources is less than a preset capacity, namely the similar non-graphic web resources under each path of the website are aggregated and converged, so that the excessive system overhead of the client and the server for storing and maintaining the super webpage template is avoided;
(2) the user access amount of the website is higher than a preset threshold value, namely the website belongs to a high-frequency visited site, and the utilization rate of the super webpage template corresponding to the high-frequency visited site is very high;
(3) the version change frequency of the website is lower than the preset frequency, and the version change frequency refers to the frequency of great changes of technologies or frames used by scripts of the website, for example, the website is transferred from a script library to another script library, or the layout of the website is greatly adjusted, or the CSS file of the website is greatly changed. Since the non-image web resources corresponding to the website may be changed greatly when the website is modified, the super web page template corresponding to the modified website needs to be regenerated, so that the modification frequency of the website needs to be lower than the preset frequency, the super web page template with a new version is prevented from being frequently generated, and the storage and maintenance costs of the super web page template are reduced.
In the embodiment of the invention, the server determines a plurality of websites with completely consistent organizational structures, forms the site domain names of the websites into a site domain name list, determines the types of non-image web resources under each path of the websites, and sets the version number of the super webpage template.
The server can set the version number of the super webpage template according to a preset version naming rule. The preset version naming rules may be numbered in natural number or alphabetical order.
The server merges the content of the non-graphic web resource types in the web site into text data. Specifically, the server directly merges the contents of the non-image web resource types under each path of the website into text data based on the file level; or the server firstly determines the similarity degree between the data blocks in the content of the non-image web resource type under each path of the website, extracts the data blocks with the similarity degree larger than a preset similarity value from the content of the non-image web resource type under each path, and combines the extracted data blocks into text data.
And the server encrypts and encodes the text data obtained by combination through a preset encryption algorithm and a preset encoding algorithm to obtain an identifier of the super webpage template, and the identifier is used as the file name of the super webpage template.
The identifier is used as a key basis for identifying two parties of the server and the client to maintain the same super webpage template and issue incremental data.
The predetermined encryption Algorithm may be MD5(Message-Digest Algorithm 5), and the predetermined encoding Algorithm may be base64 (encoding method for transmitting 8-Bit byte codes). The server encrypts the text data through an MD5 algorithm to obtain a ciphertext, obtains the characters with the preset number in front of the ciphertext, transcodes the obtained characters through base64 to obtain 8 characters, and determines the 8 characters as the identifier of the super webpage template. In the embodiment of the present invention, the first 6 characters of the ciphertext can be taken and encoded into an identifier of 8 characters through base 64.
The server combines the text data obtained by the combination, the generated identifier, the formed site domain name list, the non-image web resource type and the version number to form a super webpage template, and the super webpage template is stored in the server and the client.
The client manages one or more of the hypertext web page templates it stores in a manner that is independent of the client's local cache. When the client is started, the super webpage template is loaded, and double indexes of the super webpage template are respectively established by the site domain name and the identifier, so that the corresponding super webpage template can be quickly acquired from two dimensions of the site domain name and the identifier.
The client closes or opens the super webpage template corresponding to a certain type of website through the control parameters issued by the background. The server can also close or open the super webpage template corresponding to a certain type of website by controlling the parameters.
In order to ensure the effectiveness of the super webpage template, in a preferred embodiment of the present invention, the server needs to periodically detect whether the website is modified, and if so, the server generates a new version of the super webpage template corresponding to the modified website, and instructs the client to update the new version of the super webpage template.
As shown in FIG. 1B, the process of updating the super webpage template comprises:
s1: the method comprises the steps that a server detects the version changing degree of a website corresponding to each super webpage template every other preset time period, and when the version changing degree of a website is detected to exceed the preset version changing degree, a new version of the super webpage template corresponding to the website is generated according to the website after version changing;
the preset time period may be 5 hours or 10 hours, etc. For each super webpage template locally stored by the server, the server loads all non-graph web resources under each site domain name in a site domain name list corresponding to the super webpage template every preset time period, performs incremental compression operation on each non-graph web resource and the super webpage template to obtain incremental compressed data, and calculates the incremental compression ratio of each incremental compressed data, wherein the incremental compression ratio can reflect the version change degree of the website. And when the incremental compression rate of all the non-image web resources of a certain website exceeds a compression rate threshold value, determining that the version change degree of the website exceeds a preset version change degree, and generating a new version of the super web page template corresponding to the website based on the non-image web resources of the website after version change.
S2: the server determines whether the version of the super webpage template corresponding to the website to be accessed cached by the client is the latest version or not according to the page request sent by the client, if so, the server responds to the page request, and if not, the server executes the step S3;
the server analyzes the site domain name and the version number from the page request sent by the client, and locally acquires the version number of the super webpage template corresponding to the site domain name from the server according to the site domain name. And if the local version number of the server is higher than the version number in the page request, determining that the super webpage template corresponding to the site domain name cached by the client is not the latest version.
When the server determines that the version cached by the client is not the latest version, the server sends the complete data of the non-image web resource required by the page request at this time.
S3: the server transmits updating indication information to the client;
the update indication information includes an identifier and a version number of the old version of the super webpage template, an identifier and a version number of the new version of the super webpage template, and a website domain name list corresponding to the new version of the super webpage template.
S4: and the client downloads and stores the latest version of the super webpage template from the server according to the update indication information.
The client side can update immediately after receiving the update indication information sent by the server, or can update at a subsequent appropriate time, for example, when switching to a WiFi network.
The specific process of updating the super webpage template by the client is as follows:
1, the client marks the invalidity of the old version of the super webpage template in a super webpage template manager according to the identifier of the old version of the super webpage template in the updating indication information;
2, the client sends an update request to the server;
the update request carries an identifier and a version number of the old version of the super webpage template, an identifier and a version number of the new version of the super webpage template, and a site domain name list applicable to the new version of the super webpage template.
3, the server receives the update request sent by the client, compares the update request with the server, and then responds to the update request according to a response protocol format:
and after confirming that the information written in the updating request is correct, the server compresses the new version of the super webpage template in a certain compression mode, then encapsulates the identifier and the version number of the new version of the super webpage template, the site domain name list corresponding to the new version of the super webpage template, the compressed data and the compression mode of the new version of the super webpage template and the original data size of the new version of the super webpage template in a response protocol format, and transmits the encapsulated data serving as updating response data to the client.
4, the client receives the update response data, analyzes the update response data according to a response protocol format, and judges whether the analysis of the new version of the super webpage template is successful after analyzing the new version of the super webpage template, if so, the step 6 is executed, and if not, the step 5 is executed;
the method for judging whether the new version of the super webpage template is successfully analyzed may include: and judging whether the size of the analyzed new version of the super webpage template is consistent with the size of original data included in the update response data or not, or judging whether the identifier of the analyzed new version of the super webpage template is consistent with the identifier included in the update response data or not.
And 5, the client re-marks the effectiveness of the old version of the super webpage template in the super webpage template manager, records that the update is wrong, and then updates based on the response indication of the server.
And 6, adding the new version of the super webpage template into the super webpage template manager by the client, deleting the old version of the super webpage template, and finishing the updating of the super webpage template.
In another preferred embodiment of the present invention, a client caching policy may be set in advance in the server and the client, where the client caching policy specifies a condition that the non-graph web resource that can be cached by the client needs to meet, for example, the non-graph web resource is only cacheable when the capacity of the non-graph web resource is less than a capacity threshold. Before issuing the non-graph web resources to the client, the server firstly judges whether the client can cache the non-graph web resources according to the client caching strategy, if so, the client is instructed to cache the non-graph web resources, otherwise, the client is instructed not to cache the non-graph web resources. The client caching policy referred to herein may use any client caching policy known in the art and will not be described in great detail herein.
When the server determines that the client can cache the non-image web resource, the server performs hash operation on the address and the content of the non-image web resource to obtain a hash value, the hash value is used as the identifier of the non-image web resource, and the cache indication, the non-image web resource and the identifier thereof are issued to the client. And caching the non-graph web resource, the site domain name corresponding to the non-graph web resource and the identification thereof in a local cache of the client by the client.
When a user needs to browse a webpage of a certain website and click a link corresponding to the webpage, a client determines the website corresponding to the link as the website needing to be visited, acquires the identifier of the non-image web resource corresponding to the website domain name of the website from a local cache of the client, forms the acquired identifier of the non-image web resource into a client cache list, and acquires the identifier, the version number and the type of the non-image web resource of the super webpage template corresponding to the website domain name. The client sends a page request to the server, and the page request carries the site domain name, the client cache list, the identifier, the version number and the non-image web resource type.
Step 102: the server receives and analyzes a page request sent by the client;
the server receives a page request sent by the client, analyzes a website domain name of a website accessed by the client from the page request, and analyzes a client cache list, an identifier, a version number and a non-image web resource type. Step 103: when the required non-graph web resources are stored in the local cache of the client, the server instructs the client to directly read the required non-graph web resources from the local cache of the client;
the server obtains a non-image web resource which corresponds to the website domain name and is the same as the analyzed non-image web resource type according to the analyzed website domain name and the non-image web resource type, carries out hash operation on an address corresponding to the non-image web resource and the content of the non-image web resource to obtain a hash value corresponding to the non-image web resource, and determines the hash value as the identification of the non-image web resource. And after calculating the identifier of the non-image web resource, determining whether the analyzed client cache list contains the calculated identifier, if so, judging that the local cache of the client stores the required non-image web resource, otherwise, judging that the local cache of the client does not store the required non-image web resource.
The server sends cached indication information to the client, the cached indication information comprising an identification of the required non-graph web resource and a cached identification. And the client receives the cached indication information, and directly acquires the required non-graph web resources from the local cache of the client according to the required non-graph web resources identifier included in the cached indication information.
The server can encapsulate the http response header of the non-image web resource, the identifier of the non-image web resource and the cached identifier according to a preset protocol format, and send the encapsulated information to the client. After receiving the information sent by the server, the client analyzes the http response header of the non-image web resource, the identifier of the non-image web resource and the cached identifier from the information according to the preset protocol format, and directly acquires the corresponding non-image web resource from the local cache of the client according to the identifier of the non-image web resource. The preset protocol format may be a standard protocol format of HTML 5.
In a preferred embodiment of the present invention, a client cache policy may be set in advance between the server and the client, and when the server needs to issue the non-image web resource to the client and the non-image web resource hits the local cache of the client, the server does not need to issue the non-image web resource but directly instructs the client to obtain the non-image web resource from the local cache, so that no other operation is needed, and the transmission flow is greatly saved.
Step 104: when the required non-image web resources are not stored in the local cache of the client, the server performs incremental compression operation on the non-image web resources acquired according to the page request and the super webpage template to obtain incremental data;
the server acquires corresponding non-image web resources from a website server corresponding to the website domain name according to the analyzed website domain name, acquires corresponding super webpage templates from one or more super webpage templates stored in the super webpage templates according to the identifiers and version numbers of the super webpage templates, and performs incremental compression operation according to the acquired non-image web resources and the super webpage templates to obtain incremental data.
In the embodiment of the present invention, the server may perform incremental compression operation on the non-graph web resource and the super web page template through an SDCH (Shared dictionary compression over HTTP, a mechanism for performing HTTP data compression based on a dictionary).
SDCH is an HTTP/1.1 (hypertext transfer protocol version 1.1) compatible extension that supports compression of HTTP response data by sharing a dictionary between the client and server. The scenarios for SDCH application are as follows: the server firstly sends the dictionary to the client through the extension; when a client requests a certain http resource, the server can perform incremental compression on the resource based on the same dictionary stored by both sides, and the server issues incremental data to the client; the client restores the incremental data to the requested http resource using a corresponding incremental compression algorithm.
In the embodiment of the invention, the super webpage templates stored in the server and the client are the dictionaries shared between the client and the server. For a given pair of data, one data is used as a dictionary or template, and the other data is compressed by using an incremental compression algorithm, and identification is performed based on the same data flow in the two data, so that the same data flow is reduced in the output compressed data. The higher the similarity between the two data, the less incremental data. Meanwhile, the incremental compression algorithm also provides self compression of different levels based on the generated incremental data, so that the size of the incremental data is reduced, and the compression effect is further improved.
Because all pages of a certain website under various paths use the same super webpage template, the difference data of each page is very little, and the similarity of each page is assumed to be close to 90%, then by using the SDCH technology, the server firstly issues the super webpage template of the website to the client, and when the client accesses a certain page of the website, the client can use an increment compression mode to respond to the increment data of the page based on the super webpage template, thereby achieving the purpose of reducing the transmission data flow.
In another preferred embodiment of the present invention, an incremental caching service is set in the server, and the identifier of the non-image web resource that has undergone the incremental compression operation, the identifier of the super web page template, and the incremental compression operation result are cached in the local cache system of the server, where the incremental compression operation result is a successful identifier and incremental data, or the incremental compression operation result is a failed identifier.
The success flag and the failure flag are determined by the delta compression rate of the delta data. The delta compression ratio is the ratio between the capacity of the delta data and the capacity of the non-graphic web resource. And if the incremental compression rate of the incremental data is smaller than the preset compression threshold, the incremental compression operation result stored in the local cache system of the server comprises a successful identifier and the incremental data. And if the increment compression rate of the increment data is greater than or equal to the preset compression threshold, the increment compression operation result stored in the local cache system of the server only comprises a failure identifier. The preset compression threshold may be 5%, etc.
The increment cache service can enable a plurality of transit browsing service processes running on the server to share the increment compression operation result of the increment operation. Mainly, for a given non-graph web resource, multiple users may need to issue the non-graph web resource, or the same user needs to issue the non-graph web resource in different time periods, and the non-graph web resource and the matched super web page template thereof are all fixed and unchangeable in a limited time period, so that the increment compression operation result of the increment operation of the non-graph web resource relative to the super web page template is cached in a local cache system of a server through the increment cache service, the increment compression operation of the non-graph web resource can be effectively reduced, and the computing resource of a processor is saved.
Because the server is provided with the increment cache service, before the server performs increment compression operation, the server can firstly inquire whether the local cache system of the server caches the increment compression operation result corresponding to the identification of the non-image web resource and the identification of the super web page template through the increment cache service. And when the local cache system of the server does not cache the corresponding increment compression operation result, the server performs increment compression operation on the non-image web resource and the super webpage template to obtain increment data. And when the server local cache system caches the corresponding increment compression operation result, directly acquiring the corresponding increment compression operation result from the server local cache system, and responding the page request by the subsequent server according to the increment compression operation result.
In the embodiment of the invention, when the local cache system of the server does not cache the corresponding incremental compression operation result and the server performs the incremental compression operation to obtain the incremental data, the server calculates the ratio of the capacity of the incremental data to the capacity of the non-image web resource to obtain the incremental compression rate corresponding to the non-image web resource. If the incremental compression rate is smaller than the preset compression threshold value, the server caches the identification of the non-image web resource, the identifier of the super webpage template, the successful identification and the incremental data to a local cache system of the server through an incremental cache service. If the incremental compression rate is larger than or equal to the preset compression threshold value, the server caches the identification of the non-image web resource, the identifier of the super webpage template and the failure identification to a local cache system of the server through an incremental cache service.
Step 105: and the server transmits the incremental data to the client as page response data.
And when the local cache system of the server does not cache the corresponding incremental compression operation result and the server performs incremental compression operation to obtain incremental data, responding to the page request of the client in an incremental mode if the incremental compression rate of the incremental data is greater than a preset compression threshold value. Otherwise, responding to the page request of the client in a complete data mode.
And when the local cache system of the server caches the corresponding increment compression operation result which comprises the successful identifier and the increment data, the server responds to the page request in an increment mode. And when the local cache system of the server caches the corresponding increment compression operation result and the increment compression operation result comprises the failure identifier, the server responds to the page request in a complete data mode.
In a preferred embodiment of the present invention, a client cache policy may be set in advance between the server and the client, and before the server responds to the page request of the client, whether the non-graph web resource required by the page request is suitable for local cache at the client is determined according to the existing client cache policy between the server and the client, and a client cache determination identifier is generated according to the determination result.
The operation of responding in an incremental manner comprises the following steps: and the server packages the page response head information of the non-image web resource, the client cache judgment identifier, the incremental operation identifier, the identifier of the super webpage template, the original size of the non-image web resource, the size of the incremental data and the incremental data through a preset protocol format, and transmits the packaged data serving as page response data to the client.
The client receives the page response data transmitted by the server, analyzes the page response data through a preset protocol format, acquires a corresponding super webpage template from a local cache of the client according to an identifier included in the page response data, and restores the required non-image web resources according to the webpage template and the incremental data in the page response data.
In the embodiment of the invention, the client further judges whether the reduction operation is successful according to the size of the reduced non-image web resource and the original size included in the page response data, if the size of the reduced non-image web resource is consistent with the original size, the reduction is successful, otherwise, the reduction is failed. And the failure of restoration indicates that the data transmission is in problem, and the non-image web resources obtained by restoration are discarded.
And if the restoration is successful and the client cache judgment identifier in the page response data indicates that the cache is required, the client caches the restored non-image web resources in the local client.
The above operation of responding in a complete data manner is as follows: the server compresses the non-image web resource in a preset compression mode to obtain compressed data, encapsulates page response head information of the non-image web resource, client cache judgment identification, identification of the preset compression mode and the compressed data in a preset protocol format, and transmits the encapsulated data serving as page response data to the client. The predetermined compression method may be gzip or 7 zip.
The client receives the page response data transmitted by the server, analyzes the page response data through a preset protocol format, and decompresses the compressed data by adopting a corresponding preset compression mode according to the identifier of the preset compression mode in the page response data to obtain the required non-image web resource. And if the client cache judgment identifier indicates that caching is needed, caching the needed non-image web resources in the local client.
The preset protocol format and the response protocol format mentioned in the embodiment of the present invention may be HTML5, WebApp protocol, H264 protocol, or the like. The threshold values in the embodiments of the present invention mentioned above may be set based on empirical values.
The embodiment of the invention has the beneficial effects that: and establishing one or more different super webpage templates and storing the one or more different super webpage templates in a server and a client, wherein the super webpage templates comprise the same type of non-graphic web resources under each path of a certain type of websites. And when the required non-image web resource does not exist in the local cache of the client, the server performs increment compression operation on the non-image web resource acquired according to the page request and the super webpage template to obtain increment data, and transmits the increment data to the client. Because the server only transmits the incremental data between the non-image web resources and the super webpage template to the client, the transmission of complete data of the non-image web resources is avoided, the page response time is shortened, and the transmission flow is saved.
Example 2
Referring to fig. 2, an embodiment of the present invention provides a page data transmission system, which may execute the page data transmission method described in embodiment 1 above. The system comprises a server 201 and a client 202;
the server 201 is used for establishing one or more different super webpage templates and storing the super webpage templates in the server and the client 202, wherein each super webpage template comprises the same type of non-image web resources under each path of a certain type of website; receiving and analyzing a page request sent by a client 202; when the local cache of the client 202 stores the required non-image web resource, the client 202 is instructed to directly read the required non-image web resource from the local cache of the client 202, otherwise, the non-image web resource acquired according to the page request and the super webpage template are subjected to incremental compression operation to obtain incremental data; the delta data is transmitted to client 202 as page response data.
The conditions met by the certain type of websites include: the number of the similar non-graphic web resources under each path of the website is less than a preset value, and the total capacity of the similar non-graphic web resources is less than a preset capacity; the user access amount of the website is higher than a preset threshold value; the frequency of the web site revising is lower than the preset frequency.
The data structure of the super webpage template comprises a non-image web resource type, a website domain name list of a website, the version number of the super webpage template and text data combined by the content of the non-image web resource type in the website.
The non-graphical web resources include an outlying JavaScript script and a CSS style sheet. The non-graphical web resource types described above include script classes and style classes. The super webpage template can be a script super template and a style super template.
Further, in the process of establishing one or more different super web page templates, the server 201 is configured to determine multiple websites with completely consistent organizational structures and form a list of site domains of the multiple websites; determining the types of non-graphic web resources under each path of a website; setting the version number of the super webpage template; merging the contents of the non-image web resource types in the websites into text data; and carrying out encryption and coding operation on the text data through a preset encryption algorithm and a preset coding algorithm to obtain an identifier of the super webpage template, and taking the identifier as the file name of the super webpage template. The server 201 forms the text data, the identifier, the site domain name list, the non-image web resource type and the version number into a super web page template, and stores the super web page template in itself and the client 202.
When the content of the non-graphic web resource type in the website is merged into the text data, the server 201 is configured to directly merge the content of the non-graphic web resource type in each path of the website into the text data based on the file level; or,
the server 201 is configured to determine a similarity degree between data blocks in the content of the non-graphic web resource type in each path of the website, extract a data block with a similarity degree greater than a preset similarity value from the content of the non-graphic web resource type in each path, and merge the extracted data block into text data.
Further, the client 202 is configured to, when a website is ready to be accessed, retrieve one or more super web templates stored therein, find a super web template applicable to the website, and obtain a client 202 cache list corresponding to a website domain name of the website from a local cache, where the client 202 cache list includes an identifier of a non-graph web resource corresponding to the website domain name that has been cached by the client 202; a page request is sent to server 201, where the site domain name of the website, the identifier of the super web page template, the type of the non-image web resource, the version number, and the client cache list are provided.
Further, the server 201 is configured to parse a site domain name of the website, an identifier of a super webpage template, a non-image web resource type, a version number, and the client cache list from the page request, and acquire a non-image web resource that corresponds to the site domain name and is the same as the parsed non-image web resource type; calculating the obtained identifier of the non-graph web resource, determining whether the cache list of the client 202 contains the calculated identifier, if yes, judging that the required non-graph web resource exists in the local cache of the client 202, otherwise, judging that the required non-graph web resource does not exist in the local cache of the client 202.
The server 201 is further configured to perform a hash operation on the address corresponding to the non-graph web resource and the content of the non-graph web resource to obtain a hash value corresponding to the non-graph web resource, and use the hash value as the identifier of the non-graph web resource.
When judging that the required non-map web resource exists in the local cache of the client 202, the server 201 is configured to send cached indication information to the client 202, where the cached indication information includes an identifier of the required non-map web resource and a cached identifier;
and the client 202 is configured to receive the cached indication information, and directly obtain the required non-graph web resource from the local cache of the client 202 according to the identifier of the required non-graph web resource.
Further, the server 201 is configured to parse the identifier and the version number of the super webpage template corresponding to the site domain name from the page request; and acquiring a corresponding super webpage template from one or more super webpage templates stored in the super webpage template according to the analyzed identifier and version number of the super webpage template, and performing increment compression operation according to the acquired non-image web resource and the super webpage template to obtain increment data.
Further, the server 201 is further configured to cache, in the local cache system of the server 201, the identifier of the non-image web resource on which the incremental compression operation has been performed, the identifier of the super web page template, and the incremental compression operation result, where the incremental compression operation result is a successful identifier and incremental data, or the incremental compression operation result is a failed identifier.
The server 201 is further configured to, before performing the incremental compression operation, query whether the local cache system of the server 201 caches an identifier of the non-graph web resource and an incremental compression operation result corresponding to the identifier of the super web page template; when the local cache system of the server 201 does not cache the corresponding incremental compression operation result, performing incremental compression operation on the non-image web resource and the super webpage template to obtain incremental data; when the local cache system of the server 201 caches the corresponding incremental compression operation result, the corresponding incremental compression operation result is directly obtained from the local cache system of the server 201.
When the server local cache system does not cache the corresponding incremental compression operation result and the server 201 performs the incremental compression operation to obtain the incremental data, the server 201 is further configured to calculate a ratio between the capacity of the incremental data and the capacity of the non-image web resource to obtain an incremental compression rate corresponding to the non-image web resource when the server 201 local cache system does not cache the corresponding incremental compression operation result and the server 201 performs the incremental compression operation to obtain the incremental data; if the incremental compression rate is smaller than the preset compression threshold, caching the identification of the non-image web resource, the identifier of the super webpage template, the successful identification and the incremental data to a local cache system of the server 201 through an incremental cache service; if the incremental compression rate is greater than or equal to the preset compression threshold, the identifier of the non-graph web resource, the identifier of the super webpage template and the failure identifier are cached to a local cache system of the server 201 through an incremental cache service.
Further, when the server 201 responds to the page request according to the incremental compression operation result, the server 201 is configured to respond to the page request in an incremental manner when the corresponding incremental compression operation result includes the successful identifier and the incremental data; and when the corresponding increment compression operation result comprises the failure identification, responding to the page request in a complete data mode.
Further, in the process that the server 201 responds to the page request in an incremental manner, the server 201 is further configured to encapsulate, by using a preset protocol format, the page response header information of the non-graphical web resource, the client cache determination identifier, the incremental operation identifier, the identifier of the super web page template, the original size of the non-graphical web resource, the size of the incremental data, and transmit the encapsulated data as page response data to the client 202.
Further, the client 202 is configured to receive the page response data transmitted by the server 201, and parse the page response data through a preset protocol format; restoring the required non-image web resources according to the locally cached super webpage template corresponding to the identifier of the received super webpage template and the incremental data; judging whether the reduction operation is successful according to the size of the reduced non-image web resource and the original size included by the page response data; if the restoration is successful and the client cache judgment identifier indicates that caching is needed, caching the needed non-image web resource locally in the client 202.
Further, in the process that the server 201 responds to the page request in a complete data manner, the server 201 is further configured to compress the non-graph web resource in a preset compression manner to obtain compressed data; the page response header information, the client cache judgment identifier, the identifier of the preset compression mode and the compressed data of the non-image web resource are encapsulated through a preset protocol format, and the encapsulated data are transmitted to the client 202 as page response data.
Further, the client 202 is configured to receive the page response data transmitted by the server 201, and parse the page response data through a preset protocol format; decompressing the compressed data by adopting a corresponding preset compression mode according to the identifier of the preset compression mode to obtain the required non-image web resource; if the client cache judgment identifier indicates that caching is required, caching the required non-image web resource locally in the client 202.
Further, the client 202 is also configured to manage one or more of the super web templates stored therein in a manner independent of the local cache of the client 202; and closing or opening the super webpage template corresponding to a certain type of website through the control parameters issued by the background.
Further, the server 201 is further configured to detect, every preset time period, a revision degree of the website corresponding to each super webpage template, and when it is detected that the revision degree of a website exceeds the preset revision degree, generate a new version of the super webpage template corresponding to the website according to the website after revision.
Further, the server 201 is further configured to determine, according to the page request sent by the client 202, whether the version of the super webpage template corresponding to the website to be accessed, cached by the client 202, is the latest version; when determining that the version of the super webpage template cached by the client 202 is not the latest version, the server 201 transmits update indication information to the client 202;
the client 202 is further configured to download and store the latest version of the super webpage template from the server 201 according to the update indication information.
The embodiment of the invention has the beneficial effects that: and establishing one or more different super webpage templates and storing the one or more different super webpage templates in a server and a client, wherein the super webpage templates comprise the same type of non-graphic web resources under each path of a certain type of websites. And when the required non-image web resource does not exist in the local cache of the client, the server performs increment compression operation on the non-image web resource acquired according to the page request and the super webpage template to obtain increment data, and transmits the increment data to the client. Because the server only transmits the incremental data between the non-image web resources and the super webpage template to the client, the transmission of complete data of the non-image web resources is avoided, the page response time is shortened, and the transmission flow is saved.
The page data transmission system provided in the embodiment of the present invention is only a specific system embodiment of the present invention, and the scope of protection of the present invention is not limited thereto. It is clear to those skilled in the art that for convenience and brevity of description, the specific operation of the system described above may refer to the corresponding process in the above method embodiments.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.