技术领域technical field
本发明涉及数据业务技术领域，更具体的涉及一种网关流量压缩处理方法与装置、网络服务提供系统。The present invention relates to the technical field of data services, and more specifically relates to a gateway traffic compression processing method and device, and a network service providing system.
背景技术Background technique
WAP的全称是“无线应用协议(Wireless Application Protocol)”，提供了通过手机访问互联网的途径。只要有了一个支持WAP的手机，就可以通过WAP网关(WAP Gateway)作为上网代理访问互联网。移动终端通过WAP上网时，一般遵循WAP1.x协议或WAP2.0协议。随着WAP日益向互联网规范的靠拢和融合，最新的版本WAP2.0全面支持：XHTML，，TCP/IP，，超文本传输协议(HTTP/1.1)，和传输层安全(TLS)。The full name of WAP is "Wireless Application Protocol (Wireless Application Protocol)", which provides a way to access the Internet through mobile phones. As long as you have a mobile phone that supports WAP, you can use the WAP Gateway (WAP Gateway) as an Internet proxy to access the Internet. When a mobile terminal accesses the Internet through WAP, it generally follows the WAP1.x protocol or the WAP2.0 protocol. As WAP increasingly moves closer to and integrates with Internet norms, the latest version WAP2.0 fully supports: XHTML, TCP/IP, Hypertext Transfer Protocol (HTTP/1.1), and Transport Layer Security (TLS).
WAP网关是作为上网的代理存在的，作用就是为手机终端(客户端)向目标网站中转浏览请求，，并把目标网站返回的内容中转给客户端。The WAP gateway exists as an agent for surfing the Internet, and its function is to transfer browsing requests to the target website for the mobile terminal (client), and transfer the content returned by the target website to the client.
在RFC2616定义了HTTP/1.1协议的内容压缩的处理机制。通过压缩从WEB网络服务提供系统返回给客户端的流量内容，将有效提升传输效率和客户感知。WEB网络服务提供系统一般采用通用的压缩算法，如zip和compress对内容进行压缩，取得预期的效果。网络流量类型包括文本、图像和应用程序等多种类型，据统计，对文本类型的流量内容进行压缩的效率最高。RFC2616 defines the content compression processing mechanism of the HTTP/1.1 protocol. By compressing the traffic content returned from the WEB network service providing system to the client, the transmission efficiency and customer perception will be effectively improved. WEB network service provision systems generally use general compression algorithms, such as zip and compress, to compress content and achieve expected results. Types of network traffic include text, images, and applications. According to statistics, the efficiency of compressing text-type traffic is the highest.
由于并非所有浏览器都完全支持网页压缩机制，因此，能够支持网页压缩功能的浏览器将在HTTP请求包中以专用字段给予标志。根据RFC2616的定义，浏览器可以用Accept-Encoding字段进行声明，WAP网关收到客户端的请求后，将把客户端的请求转发到目标URL所在网站，并把网站的相应内容返回给客户端。在给客户端返回网页内容时，将根据客户端的请求头中是否包含Accept-Encoding字段以决定是否对返回给客户端的内容压缩。如果客户端的请求中包含了Accept-Encoding并说明了支持的压缩算法，如gzip或deflate，则WAP网关可以将内容经gzip或deflate压缩后返回给客户端。否则，WAP网关将把网站返回的内容透传给客户端。Since not all browsers fully support the webpage compression mechanism, browsers that can support the webpage compression function will be marked with a dedicated field in the HTTP request packet. According to the definition of RFC2616, the browser can use the Accept-Encoding field to declare. After receiving the client's request, the WAP gateway will forward the client's request to the website where the target URL is located, and return the corresponding content of the website to the client. When returning webpage content to the client, it will be determined whether to compress the content returned to the client according to whether the client's request header contains the Accept-Encoding field. If the client's request includes Accept-Encoding and describes the supported compression algorithms, such as gzip or deflate, the WAP gateway can return the content to the client after being compressed by gzip or deflate. Otherwise, the WAP gateway will transparently transmit the content returned by the website to the client.
现有WAP网关流量压缩处理机制是依据客户端(Accept-Encoding：gzip，deflate)提出的压缩能力的支持情况来决定是否对返回的流量压缩。如果客户端不声明支持压缩，则将把流量透传给客户端。The existing WAP gateway traffic compression processing mechanism decides whether to compress the returned traffic according to the support of the compression capability proposed by the client (Accept-Encoding: gzip, deflate). If the client does not declare support for compression, the traffic will be transparently passed to the client.
由于网络上客户端种类、数量众多，客户端在处理HTTP流量压缩时采取的机制各有不同。经分析，网络中有大量客户端虽然支持对压缩后的网页内容进行解压的能力，但是实际在提交网页浏览的请求中却没有包含Accept-Encoding字段，从而导致SP网络服务提供系统或WAP网关不会对这一部分流量进行处理，造成网络流量资源的浪费，并降低了客户的网页浏览感知。Due to the large number and types of clients on the network, the mechanisms adopted by the clients to process HTTP traffic compression are different. After analysis, there are a large number of clients in the network that support the ability to decompress compressed web content, but actually do not include the Accept-Encoding field in the request submitted for web browsing, resulting in the SP network service providing system or WAP gateway not being able to This part of the traffic will be processed, resulting in a waste of network traffic resources and reducing the customer's perception of web browsing.
另一方面，由于完全不支持HTTP流量压缩的客户端的数量也有很多，为了避免用户引压缩问题带来的投诉，WAP网关也不能简单的对所有流量进行直接压缩，而需要采用一定的策略进行处理。On the other hand, since there are many clients that do not support HTTP traffic compression at all, in order to avoid complaints caused by users' compression problems, WAP gateways cannot simply compress all traffic directly, but need to adopt certain strategies to deal with it .
因此，现有技术中，存在仅依据客户请求的简单声明来决定是否采用压缩机制，而没有充分挖掘客户端的处理能力的问题。Therefore, in the prior art, there is a problem that whether to adopt the compression mechanism is determined only based on the simple statement requested by the client, but the processing capability of the client is not fully exploited.
发明内容Contents of the invention
本发明提供一种网关流量压缩处理方法与装置、网络服务提供系统，用于解决现有技术中，存在仅依据客户请求的简单声明来决定是否采用压缩机制，而没有充分挖掘客户端的处理能力的问题。The present invention provides a gateway traffic compression processing method and device, and a network service providing system, which are used to solve the problem in the prior art that whether to use a compression mechanism is determined only based on a simple statement requested by a client, without fully exploiting the processing capability of the client question.
为实现本发明的上述目的，根据本发明的第一个方面，提供一种网关流量压缩处理方法，并采用以下技术方案：In order to achieve the above purpose of the present invention, according to the first aspect of the present invention, a gateway traffic compression processing method is provided, and the following technical solutions are adopted:
网关流量压缩处理方法包括：a：网关判断接入的终端请求中是否包含压缩支持字段，并得一判断结果；b：在判断结果为终端请求中不包含压缩支持字段时，网关查询预设的终端压缩能力存储单元，得第一查询结果；c：在第一查询结果为终端支持压缩时，网关在预设的终端压缩能力存储单元中获取终端所支持的压缩编码方式；以及d：网关使用压缩编码方式所支持的压缩编码算法将对应终端请求的返回内容压缩后返回给终端。The gateway traffic compression processing method includes: a: the gateway judges whether the terminal request contains the compression support field, and obtains a judgment result; b: when the judgment result is that the terminal request does not contain the compression support field, the gateway queries the preset The terminal compression capability storage unit obtains the first query result; c: when the first query result is that the terminal supports compression, the gateway obtains the compression encoding method supported by the terminal from the preset terminal compression capability storage unit; and d: the gateway uses The compression coding algorithm supported by the compression coding mode compresses the returned content corresponding to the terminal request and returns it to the terminal.
进一步地，在步骤c之后，网关流量压缩处理方法还包括：e：网关查询预设的客户行为数据库，得第二查询结果；f：在第二查询结果为预设的客户行为数据库中终端的标识为允许压缩时，网关继续执行步骤d。Further, after step c, the gateway traffic compression processing method also includes: e: the gateway queries the preset customer behavior database to obtain a second query result; f: the second query result is the terminal in the preset customer behavior database When identified as allowing compression, the gateway proceeds to step d.
进一步地，在步骤e之后，网关流量压缩处理方法还包括：g：在第二查询结果为预设的客户行为数据库中终端的标识为禁止压缩时，网关将终端请求对应的返回内容透传给终端。Further, after step e, the gateway traffic compression processing method further includes: g: when the second query result is that the identification of the terminal in the preset customer behavior database is compression prohibition, the gateway transparently transmits the returned content corresponding to the terminal request to terminal.
进一步地，预设的终端压缩能力存储单元记录接入网关的所有终端的型号以及对应型号的压缩能力信息。Further, the preset terminal compression capability storage unit records the models of all terminals accessing the gateway and the compression capability information of the corresponding models.
进一步地，在步骤a之后，网关流量压缩处理方法还包括：h：在判断结果为请求中包含压缩支持字段时，网关根据压缩支持字段所支持的压缩方式将终端请求对应的返回内容压缩后返回给终端；j：所述网关更新预设的客户行为数据库中所述终端的压缩标识；以及k：所述网关将所述终端的支持编码方式更新至所述终端压缩能力存储单元。Further, after step a, the gateway traffic compression processing method further includes: h: when the judgment result is that the request contains a compression support field, the gateway compresses the returned content corresponding to the terminal request according to the compression method supported by the compression support field and returns to the terminal; j: the gateway updates the compression identifier of the terminal in the preset customer behavior database; and k: the gateway updates the supported encoding mode of the terminal to the terminal compression capability storage unit.
进一步地，预设的客户行为数据库中保存的终端包括：网关判断接入的第一终端是否在预设时间范围内对接收到的压缩数据包中所包含的网页发起链接请求；在第一终端未在预设时间内对接收到的压缩数据包中所包含的网页发起链接请求时，网关将第一终端记录在预设的客户行为数据库中，并将第一终端的标识更新为禁用压缩。Further, the terminals stored in the preset customer behavior database include: whether the first terminal accessed by the gateway initiates a link request for the web page contained in the received compressed data package within the preset time range; When no link request is initiated for the webpage contained in the received compressed data package within the preset time, the gateway records the first terminal in the preset customer behavior database, and updates the identification of the first terminal to disable compression.
进一步地，在步骤b之后，网关流量压缩处理方法还包括：i：在第一查询结果为终端不支持压缩时，网关将请求对应的返回内容透传给终端。Further, after step b, the gateway traffic compression processing method further includes: i: when the first query result shows that the terminal does not support compression, the gateway transparently transmits the returned content corresponding to the request to the terminal.
根据本发明的另外一个方面，提供一种网关流量压缩处理装置，并采用以下技术方案：According to another aspect of the present invention, a gateway traffic compression processing device is provided, and the following technical solutions are adopted:
网关流量压缩处理装置包括：第一判断模块，用于判断接入的终端请求中是否包含压缩支持字段，并得一判断结果；第一查询模块，用于在判断结果为终端请求中不包含压缩支持字段时，查询预设的终端压缩能力存储单元，得第一查询结果；获取模块，用于在第一查询结果为终端支持压缩时，在预设的终端压缩能力存储单元中获取终端所支持的压缩编码方式；以及压缩模块，用于使用压缩编码方式所支持的压缩编码算法将对应终端请求的返回内容压缩后返回给终端。The gateway traffic compression processing device includes: a first judging module for judging whether the terminal request for access includes a compression support field, and obtaining a judging result; a first query module for judging that the terminal request does not contain a compression field. When the field is supported, query the preset terminal compression capability storage unit to obtain the first query result; the acquisition module is used to obtain the terminal supported by the terminal from the preset terminal compression capability storage unit when the first query result is that the terminal supports compression a compression coding method; and a compression module, configured to use a compression coding algorithm supported by the compression coding method to compress the returned content corresponding to the terminal request and return it to the terminal.
进一步地，网关流量压缩处理装置还包括：第二查询模块，用于查询终端预设的客户行为数据库，得第二查询结果；继续执行模块，用于在第二查询结果为客户行为数据库中终端的标志为允许压缩时，继续执行步骤c。Further, the gateway traffic compression processing device also includes: a second query module, used to query the customer behavior database preset by the terminal, to obtain a second query result; a continuing execution module, used to perform the second query result if the terminal is in the customer behavior database. When the flag of is to allow compression, proceed to step c.
进一步地，网关流量压缩处理装置还包括透传模块，用于在第二查询结果为预设的客户行为数据库中终端的标志为禁止压缩时，将终端请求对应的返回内容透传给终端。Further, the gateway traffic compression processing device also includes a transparent transmission module, which is used to transparently transmit the returned content corresponding to the terminal request to the terminal when the second query result is that the terminal flag in the preset customer behavior database is compression prohibited.
进一步地，网关流量压缩处理装置还包括维护模块，用于维护预设的客户行为数据库，维护模块包括：第二判断模块，用于判断接入的第一终端是否在预设时间范围内对接收到的压缩数据包中所包含的网页发起链接请求；标识模块，用于在第一终端未在预设时间内对接收到的压缩数据包中所包含的网页发起链接请求时，网关将第一终端记录在预设的客户行为数据库中，并将第一终端的标识更新为禁用压缩。Further, the gateway traffic compression processing device also includes a maintenance module for maintaining a preset customer behavior database, and the maintenance module includes: a second judging module for judging whether the first terminal that is connected to the receiving terminal within a preset time range The webpage contained in the received compressed data packet initiates a link request; the identification module is configured to, when the first terminal does not initiate a link request for the webpage contained in the received compressed data packet within a preset time, the gateway sends the first The terminal records in the preset customer behavior database, and updates the identifier of the first terminal to disable compression.
进一步地，网关流量压缩处理装置还包括：记录模块，用于记录接入网关的所有终端的型号以及对应型号压缩能力信息。Further, the gateway traffic compression processing device further includes: a recording module, configured to record the models of all terminals accessing the gateway and the corresponding model compression capability information.
根据本发明的又一个方面，提供一种网络服务提供系统，并采用以下技术方案：According to another aspect of the present invention, a network service providing system is provided, and the following technical solutions are adopted:
网络服务提供系统包括WAP网关，WAP网关包括上述的网关流量压缩处理装置，网络服务提供系统还包括：客户行为分析器，连接WAP网关，用于分析接入WAP网关的终端行为，得出终端为支持压缩的终端或不支持压缩的终端；终端压缩能力存储单元，连接第一查询模块，用于在终端请求中不包含压缩支持字段时，查询终端在终端压缩能力存储单元中对应的压缩能力信息；以及客户行为数据库，连接第二查询模块，用于查询终端在客户行为数据库中的压缩标识。The network service providing system includes a WAP gateway, and the WAP gateway includes the above-mentioned gateway traffic compression processing device. The network service providing system also includes: a customer behavior analyzer connected to the WAP gateway for analyzing the behavior of terminals connected to the WAP gateway, and obtaining the terminal as A terminal that supports compression or a terminal that does not support compression; the terminal compression capability storage unit, connected to the first query module, used to query the corresponding compression capability information of the terminal in the terminal compression capability storage unit when the terminal request does not contain a compression support field ; and the customer behavior database, connected to the second query module, for querying the compressed identification of the terminal in the customer behavior database.
进一步地，客户行为分析器分析在收到WAP网关返回的经压缩处理的页面后，能正常解压并发起对页面的链接请求的终端支持压缩的终端；客户行为分析器分析在收到WAP网关返回的经压缩处理的页面后，等待发起页面链路超时的终端为不支持压缩的终端。Further, after receiving the compressed page returned by the WAP gateway, the client behavior analyzer analyzes the terminals that can decompress normally and initiate a link request to the page to support compressed terminals; After the compressed page is processed, the terminal that waits for the timeout of the link to initiate the page is a terminal that does not support compression.
可以发现，本发明的技术方案具有如下优点：It can be found that the technical solution of the present invention has the following advantages:
1)根据客户端的实际处理能力而不是仅依据其在终端请求中的简单声明来决定是否采用压缩机制，可充分挖掘各种客户端的处理能力，提升网络流量的压缩效率。1) Decide whether to use the compression mechanism based on the actual processing capabilities of the client rather than just based on its simple statement in the terminal request, which can fully tap the processing capabilities of various clients and improve the compression efficiency of network traffic.
2)由于现网客户端的种类和数量具有多样性，本发明提出了对各种终端和客户行为进行分析和跟踪的做法，避免和减少客户端的投诉。2) Due to the diversity of types and quantities of clients on the existing network, the present invention proposes a method of analyzing and tracking various terminals and client behaviors to avoid and reduce client complaints.
3)本发明提出的客户行为数据库，实现了精确到个人用户的压缩管理能力。3) The customer behavior database proposed by the present invention realizes the compression management capability accurate to individual users.
除了上面所描述的目的、特征和优点之外，本发明还有其它的目的、特征和优点。下面将参照图，对本发明作进一步详细的说明。In addition to the objects, features and advantages described above, the present invention has other objects, features and advantages. Hereinafter, the present invention will be described in further detail with reference to the drawings.
附图说明Description of drawings
图1表示本发明实施中网关流量压缩处理方法的主要流程图；Fig. 1 represents the main flow chart of gateway traffic compression processing method in the implementation of the present invention;
图2表示本发明实施中终端支持压缩的处理流程示意图；Fig. 2 shows a schematic diagram of the processing flow of the terminal supporting compression in the implementation of the present invention;
图3表示本发明实施中终端不支持压缩的处理流程示意图；FIG. 3 shows a schematic diagram of a processing flow in which the terminal does not support compression during the implementation of the present invention;
图4表示本发明实施中支持主动压缩终端的上网流程；Fig. 4 represents the online process of supporting active compression terminal in the implementation of the present invention;
图5表示本发明实施中网关流量压缩处理装置的主要结构图；以及Fig. 5 represents the main structural diagram of the gateway traffic compression processing device in the implementation of the present invention; and
图6表示本发明实施中网络服务提供系统的结构示意图。Fig. 6 shows a schematic structural diagram of the network service providing system in the implementation of the present invention.
具体实施方式detailed description
以下结合附图对本发明的实施例进行详细说明，但是本发明可以由权利要求限定和覆盖的多种不同方式实施。The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways defined and covered by the claims.
图1表示本发明实施中网关流量压缩处理方法的主要流程图。Fig. 1 shows the main flow chart of the gateway traffic compression processing method in the implementation of the present invention.
参见图1所示，网关流量压缩处理方法包括：Referring to Figure 1, the gateway traffic compression processing method includes:
a：网关判断接入的终端请求中是否包含压缩支持字段，并得一判断结果；a: The gateway judges whether the incoming terminal request contains a compression support field, and obtains a judgment result;
b：在判断结果为终端请求中不包含压缩支持字段时，网关查询预设的终端压缩能力存储单元，得第一查询结果；b: When the judgment result is that the compression support field is not included in the terminal request, the gateway queries the preset terminal compression capability storage unit to obtain the first query result;
c：在第一查询结果为终端支持压缩时，网关在预设的终端压缩能力存储单元中获取终端所支持的压缩编码方式；以及c: when the first query result shows that the terminal supports compression, the gateway obtains the compression encoding method supported by the terminal from the preset terminal compression capability storage unit; and
d：网关使用压缩编码方式所支持的压缩编码算法将对应终端请求的返回内容压缩后返回给终端。d: The gateway uses the compression coding algorithm supported by the compression coding method to compress the returned content corresponding to the terminal request and return it to the terminal.
在本实施例的上述技术方案中，步骤a中，网关可以通过硬件或软件的方式判断终端请求的HTTP请求包中是否以专用字段示出终端的压缩能力，如果终端支持压缩，可以在HTTP请求包含Accept-Encoding字段以决定是否对返回给终端的内容压缩，如果在HTTP请求中未包含类似Accept-Encoding字段，网关通过查询预设的终端压缩能力存储单元，以了解该终端的压缩能力，如果客户端支持压缩，则选取该终端支持的内容进行压缩后返回给客户端。该实施例是通过一个终端压缩能力存储单元记录了网内终端的型号及对应的压缩能力，来补充执行对于在请求中未示出压缩字段的返回内容是否压缩问题。In the above technical solution of this embodiment, in step a, the gateway can determine whether the terminal’s compression capability is shown in a dedicated field in the HTTP request packet requested by the terminal through hardware or software. If the terminal supports compression, it can be used in the HTTP request Include the Accept-Encoding field to determine whether to compress the content returned to the terminal. If no similar Accept-Encoding field is included in the HTTP request, the gateway will query the preset terminal compression capability storage unit to know the compression capability of the terminal. If If the client supports compression, select the content supported by the terminal to compress and return to the client. In this embodiment, a terminal compression capability storage unit records the model of the terminal in the network and the corresponding compression capability to supplement the execution of the question of whether to compress the returned content that does not show the compression field in the request.
优选地，终端压缩能力存储单元记录了现网所有终端对流量压缩算法的支持能力。记录的内容还包括终端的型号、是否有发出Accept-Encoding信息、在未发出Accept-Encoding信息的情况下可否支持压缩以及如果支持压缩，具体支持哪些编解码算法。Preferably, the terminal compression capability storage unit records the support capability of all terminals on the live network to the traffic compression algorithm. The recorded content also includes the model of the terminal, whether Accept-Encoding information is sent, whether it can support compression without sending Accept-Encoding information, and if compression is supported, which codec algorithms are specifically supported.
优选地，网关流量压缩处理方法可以包括：Preferably, the gateway traffic compression processing method may include:
a：网关判断接入的终端请求中是否包含压缩支持字段，并得一判断结果；a: The gateway judges whether the incoming terminal request contains a compression support field, and obtains a judgment result;
b：在判断结果为终端请求中不包含压缩支持字段时，网关查询预设的终端压缩能力存储单元，得第一查询结果；b: When the judgment result is that the compression support field is not included in the terminal request, the gateway queries the preset terminal compression capability storage unit to obtain the first query result;
c：在第一查询结果为终端支持压缩时，网关在预设的终端压缩能力存储单元中获取终端所支持的压缩编码方式；c: When the first query result shows that the terminal supports compression, the gateway obtains the compression encoding method supported by the terminal from the preset terminal compression capability storage unit;
e：网关查询预设的客户行为数据库，得第二查询结果；e: The gateway queries the preset customer behavior database and obtains the second query result;
f：在第二查询结果为预设的客户行为数据库中终端的标识为允许压缩时，网关继续执行步骤d；以及f: when the second query result is that the identification of the terminal in the preset customer behavior database is to allow compression, the gateway continues to execute step d; and
d：网关使用压缩编码方式所支持的压缩编码算法将对应终端请求的返回内容压缩后返回给终端。d: The gateway uses the compression coding algorithm supported by the compression coding method to compress the returned content corresponding to the terminal request and return it to the terminal.
在本实施例的上述技术方案中，终端型号在终端压缩能力存储单元显示为支持压缩之后，网关还将查询一预设的客户行为数据库，该客户行为数据库通过分析以往该终端的行为，记录了该终端是否支持压缩的信息，如果要求访问的终端在该客户行为数据库中未被记录为禁止压缩，则可以按照上述实施例的步骤将返回内容压缩后返回给终端。In the above technical solution of this embodiment, after the terminal model is displayed as supporting compression in the terminal compression capability storage unit, the gateway will also query a preset customer behavior database, which records the customer behavior database by analyzing the past behavior of the terminal Whether the terminal supports compression information, if the terminal requiring access is not recorded as prohibiting compression in the customer behavior database, the returned content can be compressed and returned to the terminal according to the steps in the above embodiment.
优选地，预设的客户行为数据库中终端的获取方法为：网关判断接入的任一终端是否在预设时间范围内对接收到的压缩数据包中所包含的网页发起链接请求；在任一终端未在预设时间内对接收到的压缩数据包中所包含的网页发起链接请求时，网关将任一终端在预设的客户行为数据库中的标识更新为禁用压缩。Preferably, the method for obtaining terminals in the preset customer behavior database is as follows: the gateway judges whether any terminal connected within the preset time range initiates a link request to the webpage contained in the received compressed data packet; When no link request is initiated for the webpage contained in the received compressed data package within the preset time, the gateway updates the identification of any terminal in the preset customer behavior database to disable compression.
在本实施例的上述技术方案中，并联跨接在客户端和WAP网关之间的客户行为分析器对客户端的上网行为进行分析和记录，如果发现用户在接收压缩流量后的上网行为异常，则更新客户行为数据库。具体做法是：网关监控每个用户的上网交互过程。如果客户端在请求HTML/WML/XHTML等网页文件且请求头中未包含Accept-Encoding字段时，网络服务提供系统返回了压缩内容，且其后一定的时间范围内该客户端就没有发起页面内包含的链接请求，则说明该用户不能正常解压网络服务提供系统返回的压缩的网页文件，即不支持“WAP网关主动压缩”，于是更新客户行为数据库，在客户行为数据库中标志该用户号码及相应的终端型号对压缩的需求标志为禁用。In the above technical solution of this embodiment, the client behavior analyzer connected in parallel between the client and the WAP gateway analyzes and records the client's online behavior, if it is found that the user's online behavior after receiving the compressed traffic is abnormal, then Update customer behavior database. The specific method is: the gateway monitors the Internet interaction process of each user. If the client requests HTML/WML/XHTML and other webpage files and the request header does not contain the Accept-Encoding field, the network service provider system returns the compressed content, and the client does not initiate the in-page encoding within a certain period of time thereafter. If the link request is included, it means that the user cannot normally decompress the compressed webpage file returned by the network service provider system, that is, it does not support "WAP gateway active compression", so the customer behavior database is updated, and the user number and the corresponding user number are marked in the customer behavior database. Terminal models that require compression are disabled.
可选地，网关流量压缩处理方法可以包括：Optionally, the gateway traffic compression processing method may include:
a：网关判断接入的终端请求中是否包含压缩支持字段，并得一判断结果；a: The gateway judges whether the incoming terminal request contains a compression support field, and obtains a judgment result;
h：在判断结果为请求中包含压缩支持字段时，网关根据压缩支持字段所支持的压缩方式将终端请求对应的返回内容压缩后返回给终端。h: When it is judged that the request contains a compression support field, the gateway compresses the returned content corresponding to the terminal request and returns it to the terminal according to the compression method supported by the compression support field.
在本实施例的技术方案中，客户端通过TCP/IP网络通过HTTP协议，经WAP网关向目标网络服务提供系统发出网页内容访问请求。WAP网关在收到客户端请求后，首先根据目标URL到目标网站提取网页、图像等内容。如果WAP网关收到客户端的请求中包含Accept-Encoding的压缩支持字段，则将内容压缩后返回给客户端，流程结束。In the technical solution of this embodiment, the client sends a webpage content access request to the target network service providing system through the HTTP protocol through the TCP/IP network and through the WAP gateway. After receiving the request from the client, the WAP gateway first extracts content such as webpages and images from the target website according to the target URL. If the WAP gateway receives the client's request containing the Accept-Encoding compression support field, it will compress the content and return it to the client, and the process ends.
可选地，网关流量压缩处理方法可以包括：Optionally, the gateway traffic compression processing method may include:
a：网关判断接入的终端请求中是否包含压缩支持字段，并得一判断结果；a: The gateway judges whether the incoming terminal request contains a compression support field, and obtains a judgment result;
h：在判断结果为请求中包含压缩支持字段时，网关根据压缩支持字段所支持的压缩方式将终端请求对应的返回内容压缩后返回给终端。h: When it is judged that the request contains a compression support field, the gateway compresses the returned content corresponding to the terminal request and returns it to the terminal according to the compression method supported by the compression support field.
j：网关更新预设的客户行为数据库中终端的压缩标识；以及j: the gateway updates the compressed identifier of the terminal in the preset customer behavior database; and
k：网关将终端的支持编码方式更新至终端压缩能力存储单元。k: The gateway updates the encoding mode supported by the terminal to the terminal compression capability storage unit.
在本实施例的上述技术方案中，终端在收到WAP网关返回的经过压缩处理的页面(html/xhtml/wml)后，能正常解压并发起页面包含的链接(如image/js/css等)，则说明终端支持压缩能力，WAP网关将该终端的压缩需求标志更新至客户行为数据库，将终端支持的编解码方式更新至终端压缩能力存储单元。In the above technical solution of this embodiment, after receiving the compressed page (html/xhtml/wml) returned by the WAP gateway, the terminal can decompress normally and initiate the links contained in the page (such as image/js/css, etc.) , it indicates that the terminal supports the compression capability, and the WAP gateway updates the compression requirement flag of the terminal to the customer behavior database, and updates the encoding and decoding methods supported by the terminal to the terminal compression capability storage unit.
可选地，网关流量压缩处理方法可以包括：Optionally, the gateway traffic compression processing method may include:
a：网关判断接入的终端请求中是否包含压缩支持字段，并得一判断结果；a: The gateway judges whether the incoming terminal request contains a compression support field, and obtains a judgment result;
b：在判断结果为终端请求中不包含压缩支持字段时，网关查询预设的终端压缩能力存储单元，得第一查询结果；b: When the judgment result is that the compression support field is not included in the terminal request, the gateway queries the preset terminal compression capability storage unit to obtain the first query result;
i：在第一查询结果为终端不支持压缩时，网关将请求对应的返回内容透传给终端。i: When the first query result shows that the terminal does not support compression, the gateway transparently transmits the returned content corresponding to the request to the terminal.
在本实施例的上述技术方案中，WAP网关通过查询终端压缩能力存储单元，了解客户端是否支持压缩能力，如果客户端不支持压缩，则将网站内容透传返回给客户端。In the above technical solution of this embodiment, the WAP gateway learns whether the client supports the compression capability by querying the terminal compression capability storage unit, and if the client does not support compression, transparently transmits the website content back to the client.
图2表示本发明实施中终端支持压缩的处理流程示意图。Fig. 2 shows a schematic diagram of the processing flow of the terminal supporting compression in the implementation of the present invention.
根据上述实施例，图2给出了终端支持压缩的处理流程为：According to the above-mentioned embodiment, Fig. 2 provides the processing flow of the terminal supporting compression as follows:
步骤201：WAP网关接收终端发起html/xhtml/wml的HTTP Req；Step 201: WAP gateway receiving terminal initiates HTTP Req of html/xhtml/wml;
步骤202：WAP网关向SP网络服务提供系统中转上述html/xhtml/wml的HTTP Req；Step 202: The WAP gateway forwards the HTTP Req of the above html/xhtml/wml to the SP network service providing system;
步骤203：WAP网关接收SP网络服务提供系统返回的没有被压缩的HTTPResq；Step 203: the WAP gateway receives the uncompressed HTTPResq returned by the SP network service providing system;
步骤204：WAP网关对返回的HTTP Resq做压缩处理后返回给终端；Step 204: the WAP gateway compresses the returned HTTP Resq and returns it to the terminal;
步骤205：WAP网关接收终端对压缩HTTP Resq中包含的链接(如image/js/css等)，则说明终端支持压缩能力；Step 205: the WAP gateway receives the link (such as image/js/css, etc.) contained in the compressed HTTP Resq from the terminal, indicating that the terminal supports the compression capability;
步骤206：WAP网关上述连接中转给SP网络服务提供系统；Step 206: The WAP gateway transfers the above connection to the SP network service providing system;
步骤207：WAP网关更新客户行为数据库中压缩需求标识；Step 207: The WAP gateway updates the compressed demand identifier in the customer behavior database;
步骤208：WAP网关接收客户行为数据库返回的更新成功消息；Step 208: the WAP gateway receives the update success message returned by the customer behavior database;
步骤209：WAP网关将该终端支持的编解码方式更新至终端压缩能力存储单元。Step 209: The WAP gateway updates the codec mode supported by the terminal to the terminal compression capability storage unit.
图3表示本发明实施中终端不支持压缩的处理流程示意图。Fig. 3 is a schematic diagram showing the processing flow of the terminal not supporting compression in the implementation of the present invention.
参见图3所示，为终端不支持压缩的处理流程：See Figure 3 for the processing flow for terminals that do not support compression:
步骤301：WAP网关接收终端发起html/xhtml/wml的HTTP Req；Step 301: WAP gateway receiving terminal initiates HTTP Req of html/xhtml/wml;
步骤302：WAP网关向SP网络服务提供系统中转上述html/xhtml/wml的HTTP Req；Step 302: The WAP gateway forwards the HTTP Req of the above html/xhtml/wml to the SP network service providing system;
步骤303：WAP网关接收SP网络服务提供系统返回的没有被压缩的HTTPResq；Step 303: the WAP gateway receives the uncompressed HTTPResq returned by the SP network service providing system;
步骤304：WAP网关对返回的HTTP Resq做压缩处理后返回给终端；Step 304: the WAP gateway compresses the returned HTTP Resq and returns it to the terminal;
步骤305：WAP网关等待终端发起页面中包含的链接(如image/js/css等)超时，则说明终端解压页面内容失败，该终端不支持压缩，WAP网关将该终端的压缩需求标志更新至客户行为数据库；Step 305: When the WAP gateway waits for the link (such as image/js/css, etc.) contained in the page initiated by the terminal to time out, it means that the terminal fails to decompress the content of the page, and the terminal does not support compression. The WAP gateway updates the terminal's compression requirement flag to the client Behavioral database;
步骤305：WAP网关接收客户行为数据库返回的更新成功信息。Step 305: The WAP gateway receives the update success information returned by the customer behavior database.
图4表示本发明实施中支持主动压缩终端的上网流程。Fig. 4 shows the online flow of terminals supporting active compression in the implementation of the present invention.
参见图4所示，支持主动压缩终端的上网流程为：As shown in Figure 4, the Internet access process of terminals that support active compression is as follows:
步骤401：WAP网关接收终端发起的无压缩标志的HTTP Req；Step 401: the WAP gateway receives the HTTP Req without compression flag initiated by the terminal;
步骤402：WAP网关向SP网络服务提供系统中转上述无压缩标志的HTTPReq；Step 402: The WAP gateway transfers the above HTTPReq without compression flag to the SP network service providing system;
步骤403：WAP网关接收SP网络服务提供系统返回的没有被压缩的HTTPResq；Step 403: the WAP gateway receives the uncompressed HTTPResq returned by the SP network service providing system;
步骤404：WAP网关查询客户行为数据库中的压缩需求标志；Step 404: the WAP gateway queries the compression demand flag in the customer behavior database;
步骤405：WAP网关接收客户行为数据库返回的终端压缩需求标志；Step 405: the WAP gateway receives the terminal compression demand flag returned by the customer behavior database;
步骤406：WAP网关查询终端压缩能力存储单元中终端支持的编码方式；Step 406: WAP gateway inquires the encoding mode supported by the terminal in the terminal compression capability storage unit;
步骤407：WAP网关接收终端压缩能力存储单元返回的终端支持的编码方式；Step 407: The WAP gateway receives the coding mode supported by the terminal returned by the terminal compression capability storage unit;
步骤408：WAP网关对返回的HTTP Resq做压缩处理后返回给终端。Step 408: The WAP gateway compresses the returned HTTP Resq and returns it to the terminal.
图5表示本发明实施中网关流量压缩处理装置的主要结构图。Fig. 5 shows the main structural diagram of the gateway traffic compression processing device in the implementation of the present invention.
参见图5所示，网关流量压缩处理装置包括：第一判断模块51，用于判断接入的终端请求中是否包含压缩支持字段，并得一判断结果；第一查询模块53，用于在判断结果为终端请求中不包含压缩支持字段时，查询预设的终端压缩能力存储单元，得第一查询结果；获取模块55，用于在第一查询结果为终端支持压缩时，在预设的终端压缩能力存储单元中获取终端所支持的压缩编码方式；以及压缩模块57，用于使用压缩编码方式所支持的压缩编码算法将对应终端请求的返回内容压缩后返回给终端。Referring to Fig. 5, the gateway flow compression processing device includes: a first judging module 51 for judging whether the terminal request for access includes a compression support field, and obtaining a judging result; a first query module 53 for judging When the result is that the compression support field is not included in the terminal request, query the preset terminal compression capability storage unit to obtain the first query result; the obtaining module 55 is configured to, when the first query result is that the terminal supports compression, in the preset terminal The compression encoding method supported by the terminal is obtained from the compression capability storage unit; and the compression module 57 is configured to use the compression encoding algorithm supported by the compression encoding method to compress the returned content corresponding to the terminal request and return it to the terminal.
优选地，网关流量压缩处理装置还包括：第二查询模块(图中未示)，用于查询终端预设的客户行为数据库，得第二查询结果；继续执行模块(图中未示)，用于在第二查询结果为客户行为数据库中终端的标志为允许压缩时，继续执行步骤c。Preferably, the gateway traffic compression processing device further includes: a second query module (not shown in the figure), used to query the customer behavior database preset by the terminal to obtain a second query result; a continuation module (not shown in the figure), used to When the second query result shows that the terminal flag in the customer behavior database is to allow compression, continue to execute step c.
优选地，网关流量压缩处理装置还包括透传模块(图中未示)，用于在第二查询结果为预设的客户行为数据库中终端的标志为禁止压缩时，将终端请求对应的返回内容透传给终端。Preferably, the gateway traffic compression processing device also includes a transparent transmission module (not shown in the figure), which is used to request the corresponding return content of the terminal request when the second query result is that the terminal flag in the preset customer behavior database is prohibiting compression. Transparently transmitted to the terminal.
优选地，网关流量压缩处理装置还包括维护模块(图中未示)，用于维护预设的客户行为数据库，维护模块包括：第二判断模块(图中未示)，用于判断接入的第一终端是否在预设时间范围内对接收到的压缩数据包中所包含的网页发起链接请求；标识模块(图中未示)，用于在所述第一终端未在所述预设时间内对接收到的压缩数据包中所包含的网页发起链接请求时，所述网关将所述第一终端记录在所述预设的客户行为数据库中，并将所述第一终端的标识更新为禁用压缩。Preferably, the gateway traffic compression processing device also includes a maintenance module (not shown in the figure), for maintaining a preset customer behavior database, and the maintenance module includes: a second judging module (not shown in the figure), for judging the access Whether the first terminal initiates a link request to the webpage contained in the received compressed data package within the preset time range; an identification module (not shown in the figure) is used for when the first terminal is not within the preset time When a link request is initiated for the webpage contained in the received compressed data packet, the gateway records the first terminal in the preset customer behavior database, and updates the identification of the first terminal to Disable compression.
优选地，网关流量压缩处理装置还包括记录模块(图中未示)，用于记录接入网关的所有终端的型号以及对应型号压缩能力信息。Preferably, the gateway traffic compression processing device further includes a recording module (not shown in the figure), configured to record the models of all terminals accessing the gateway and the corresponding model compression capability information.
图6表示本发明实施中网络服务提供系统的结构示意图。Fig. 6 shows a schematic structural diagram of the network service providing system in the implementation of the present invention.
参见图6所示，网络服务提供系统包括WAP网关60，WAP网关60包括上述的网关流量压缩处理装置，网络服务提供系统还包括：客户行为分析器66，连接WAP网关60，用于分析接入WAP网关60的终端行为，得出得出所述终端为支持压缩的终端或不支持压缩的终端；终端压缩能力存储单元68，连接第一查询模块(图5中标号为53，图6中未示)，用于在终端请求中不包含压缩支持字段时，查询终端在终端压缩能力存储单元中对应的压缩能力信息；以及客户行为数据库64，连接第二查询模块(图中未示)，用于查询终端在客户行为数据库中的压缩标识。Referring to shown in Figure 6, the network service providing system includes a WAP gateway 60, and the WAP gateway 60 includes the above-mentioned gateway traffic compression processing device, and the network service providing system also includes: a customer behavior analyzer 66, connected to the WAP gateway 60, for analyzing access The terminal behavior of WAP gateway 60 draws that said terminal is a terminal that supports compression or a terminal that does not support compression; terminal compression capability storage unit 68 is connected to the first query module (marked as 53 in Fig. 5, not shown in Fig. 6 shown), for when the terminal request does not include the compression support field, query the corresponding compression capability information of the terminal in the terminal compression capability storage unit; and the customer behavior database 64 is connected to the second query module (not shown in the figure), used It is used to query the compressed identification of the terminal in the customer behavior database.
优选地，客户行为分析器分析在收到WAP网关返回的经压缩处理的页面后，能正常解压并发起对页面的链接请求的终端为支持压缩的终端；客户行为分析器分析在收到WAP网关返回的经压缩处理的页面后，等待发起页面链路超时的终端为不支持压缩的终端。Preferably, after receiving the compressed page returned by the WAP gateway, the client behavior analyzer analysis can normally decompress and initiate a terminal that supports the compression of the page link request; the client behavior analyzer analysis receives the WAP gateway After returning the compressed page, the terminal that waits for the timeout of initiating the page link is a terminal that does not support compression.
在本发明的所有技术方案中，WAP网关均是根据终端支持的压缩方式进行压缩处理，对应终端不支持的部分，WAP网关会以透传的方式将这部分内容返回给终端。In all the technical solutions of the present invention, the WAP gateway performs compression processing according to the compression methods supported by the terminal, and for the part not supported by the terminal, the WAP gateway will return this part of content to the terminal in a transparent transmission manner.
可以发现，通过上述实施例的技术方案，本领域的技术人员应该明白，本发明根据客户端的实际处理能力而不是仅依据其在HTTP中的简单声明来决定是否采用压缩机制，可充分挖掘各种客户端的处理能力，提升网络流量的压缩效率；由于现网客户端的种类和数量具有多样性，本发明提出了对各种终端和客户行为进行分析和跟踪的做法，避免和减少客户端的投诉；而客户行为分析分析器和客户行为数据库，实现了精确到个人用户的压缩管理能力。It can be found that, through the technical solutions of the above-mentioned embodiments, those skilled in the art should understand that the present invention decides whether to adopt the compression mechanism based on the actual processing capability of the client rather than only based on its simple statement in HTTP, and can fully exploit various The processing capability of the client improves the compression efficiency of network traffic; due to the diversity of the types and quantities of the existing network clients, the present invention proposes a method of analyzing and tracking various terminals and client behaviors to avoid and reduce client complaints; and The customer behavior analysis analyzer and customer behavior database realize the compression management capability accurate to individual users.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110410825.7ACN103167554B (en) | 2011-12-09 | 2011-12-09 | Gateway flow constriction processing method and device, network service providing system |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110410825.7ACN103167554B (en) | 2011-12-09 | 2011-12-09 | Gateway flow constriction processing method and device, network service providing system |
| Publication Number | Publication Date |
|---|---|
| CN103167554A CN103167554A (en) | 2013-06-19 |
| CN103167554Btrue CN103167554B (en) | 2016-09-07 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110410825.7AActiveCN103167554B (en) | 2011-12-09 | 2011-12-09 | Gateway flow constriction processing method and device, network service providing system |
| Country | Link |
|---|---|
| CN (1) | CN103167554B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105101140A (en)* | 2015-06-30 | 2015-11-25 | 北京奇虎科技有限公司 | Method for saving stream access to network, proxy server and network access system |
| CN104902462B (en)* | 2015-06-30 | 2019-02-12 | 北京奇虎科技有限公司 | A method for a mobile terminal to access a network with less flow and the mobile terminal |
| CN107592553B (en)* | 2017-09-01 | 2020-06-12 | 清华大学 | Image compression agent balance processing method and device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101365189A (en)* | 2008-09-02 | 2009-02-11 | 中国电信股份有限公司 | Method and system implementing speech passing through in mobile network |
| CN101610268A (en)* | 2009-07-16 | 2009-12-23 | 杭州华三通信技术有限公司 | A kind of implementation method of keyword filtration and equipment |
| CN101640869A (en)* | 2009-08-27 | 2010-02-03 | 中兴通讯股份有限公司 | Content conversion method and wireless application protocol gateway |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101365158A (en)* | 2007-08-10 | 2009-02-11 | 华为技术有限公司 | Parameter negotiation and implementation method and system for header compression feedback |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101365189A (en)* | 2008-09-02 | 2009-02-11 | 中国电信股份有限公司 | Method and system implementing speech passing through in mobile network |
| CN101610268A (en)* | 2009-07-16 | 2009-12-23 | 杭州华三通信技术有限公司 | A kind of implementation method of keyword filtration and equipment |
| CN101640869A (en)* | 2009-08-27 | 2010-02-03 | 中兴通讯股份有限公司 | Content conversion method and wireless application protocol gateway |
| Publication number | Publication date |
|---|---|
| CN103167554A (en) | 2013-06-19 |
| Publication | Publication Date | Title |
|---|---|---|
| CN107025234B (en) | Information pushing method and cache server | |
| US10387521B2 (en) | Creation and delivery of pre-rendered web pages for accelerated browsing | |
| US6907429B2 (en) | Satellite-based internet access system with remote prefetching of inline objects of web pages | |
| US10686726B2 (en) | Method for optimizing resource loading at mobile browsers based on cloud-client cooperation | |
| JP5230642B2 (en) | Web page display method and system | |
| US8930806B2 (en) | Providing toolbars | |
| US8694609B2 (en) | Method and apparatus for improving wireless data networks performance | |
| US11356533B2 (en) | Facilitation of efficient web site page loading | |
| US11841910B2 (en) | Token-based authentication for a proxy web scraping service | |
| CN102184231A (en) | Method and device for acquiring page resources | |
| CN102638580A (en) | Webpage information processing method and webpage information processing device | |
| CN110083616A (en) | Page data processing method, device, equipment and computer readable storage medium | |
| CN102929985A (en) | Method and system for displaying collected webpage | |
| CN103220371A (en) | Method and system for conducting content adaptation | |
| US20230018983A1 (en) | Traffic counting for proxy web scraping | |
| EP4227828A1 (en) | Web scraping through use of proxies, and applications thereof | |
| US20180302489A1 (en) | Architecture for proactively providing bundled content items to client devices | |
| CN103167554B (en) | Gateway flow constriction processing method and device, network service providing system | |
| US9400547B2 (en) | Processing device and method thereof | |
| US11182452B2 (en) | Web acceleration via learning | |
| WO2012119496A1 (en) | Pre-reading method and equipment | |
| EP2850809B1 (en) | Providing data to a network terminal | |
| WO2011067769A1 (en) | Shared dictionary compression over http proxy | |
| CN102790749A (en) | Method and system for processing data service, WAP (wireless application protocol) gateway and user terminal | |
| David et al. | An optimized cloud based proxy server techniques to improve user experience of spectrum utilization |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |