技术领域technical field
本发明涉及到计算机应用技术领域，具体的的说是一种Web服务的双路缓存机制设计方法。The invention relates to the field of computer application technology, in particular to a design method of a two-way cache mechanism for Web services.
背景技术Background technique
由于传统的基于B/S（Browser/Server）架构的Web应用系统没有采用负载均衡机制，尤其是那种大用户量访问的应用系统在高峰时段的系统访问容易带来web信息系统事务请求响应时间延长，吞吐量降低，资源利用率不高等性能问题。因此，人们在不断寻找一些优化方法来提高web信息系统的性能。Because the traditional web application system based on B/S (Browser/Server) architecture does not adopt a load balancing mechanism, especially the system access of the application system accessed by a large number of users during peak hours is likely to bring about the transaction request response time of the web information system Performance issues such as extension, reduced throughput, and low resource utilization. Therefore, people are constantly looking for some optimization methods to improve the performance of web information systems.
在现阶段，基于B/S架构的Web应用系统的高速缓存技术主要存在如下缺点：At the present stage, the caching technology of web application systems based on B/S architecture mainly has the following shortcomings:
1、现在主要使用的动态页面缓存的特点是将经常访问的内容放到内存中，下次访问时只需要动态加载内存中的内容，从而减轻了应用系统的访问压力。这种缓存机制缺点是过分使用内存容易导致系统性能下降，效率不高，缓存是占用一定内存单元的，大数据量的访问会占用大量的内存单元。这样就会导致数据被频繁加载，有可能造成缓存失效，而且内存使用过大一旦缓存不及时的释放，会造成服务器内存资源耗尽，会造成Web应用系统出现假死现象。1. The feature of the dynamic page cache that is mainly used now is to store frequently accessed content in the memory, and only need to dynamically load the content in the memory during the next visit, thereby reducing the access pressure of the application system. The disadvantage of this caching mechanism is that excessive use of memory will easily lead to system performance degradation, and the efficiency is not high. The cache occupies a certain memory unit, and the access of a large amount of data will occupy a large number of memory units. This will lead to frequent loading of data, which may cause cache failure, and if the cache is not released in time if the memory usage is too large, the server memory resources will be exhausted, and the web application system will appear suspended animation.
2、现在主要使用的静态页面缓存的特点是采用预编译模板技术，使用数据变更触发机制，生成关于数据内容的静态页，用户访问数据时直接访问所生成的静态页面而不是访问数据库。这种机制的缺点在于对于以前生成的静态页面的内容很难再修改，这就导致了这种方法的重用性比较差，很难做到修改数据的及时更新。 2. The static page cache mainly used now is characterized by the use of precompiled template technology and the data change trigger mechanism to generate static pages about data content. When users access data, they directly access the generated static pages instead of accessing the database. The disadvantage of this mechanism is that it is difficult to modify the content of the previously generated static pages, which leads to poor reusability of this method, and it is difficult to update the modified data in time.
发明内容Contents of the invention
本发明针对目前Web应用系统的缓存技术存在的不足之处，提出一种Web服务的双路缓存机制设计方法。Aiming at the deficiencies of the cache technology of the current Web application system, the invention proposes a design method for a two-way cache mechanism of the Web service.
本发明是基于动态页面和静态页面两种缓存机制相结合，而提出的一种双路缓存机制，该缓存机制是按如下方式解决所述技术问题的：通过数据库和xml缓存文件进行双路读写，同步操作触发更新的缓存机制，以xml缓存模块作为用户访问时的使用库，应用系统中的数据库作为备用库，数据的读写操作均在xml缓存文件中完成；该缓存机制的设计内容由双路缓存机制的管理过程、双路缓存机制的读取流程和双路缓存机制的写入流程组成。The present invention is based on the combination of two caching mechanisms of dynamic pages and static pages, and proposes a two-way caching mechanism. Write, synchronous operation triggers the update caching mechanism, the xml cache module is used as the library when the user accesses, the database in the application system is used as the backup library, and the data read and write operations are all completed in the xml cache file; the design content of the cache mechanism It consists of the management process of the two-way cache mechanism, the read process of the two-way cache mechanism, and the write process of the two-way cache mechanism.
下面对本发明的双路缓存机制的各个组成部分进行说明：The various components of the two-way cache mechanism of the present invention are described below:
所述双路缓存机制的管理过程是：对于xml缓存和页面缓存的管理都是由系统缓存管理模块统一进行的，关于页面缓存模块，对于在一定时间内没有访问的页面缓存信息会进行删除，释放内存；对于xml缓存，专门建立了一个分区来进行xml缓存数据的存储管理，对于xml的删除时间是根据应用系统的规模设定一个xml缓存文件个数的临界值，当xml缓存的xml文件个数达到阀值的时候，缓存管理模块的管理进程会触发xml缓存模块的删除机制。这个机制是按照最长时间不调用的原理进行删除，它首先会按照访问时间进行排序，把最长时间不访问的临界值10%的xml文件进行删除操作。The management process of the two-way cache mechanism is: the management of the xml cache and the page cache is carried out uniformly by the system cache management module. Regarding the page cache module, the page cache information that has not been accessed within a certain period of time will be deleted. Release memory; for xml cache, a partition is specially established to store and manage xml cache data. For xml deletion time, a critical value for the number of xml cache files is set according to the scale of the application system. When the xml cache xml file When the number reaches the threshold, the management process of the cache management module will trigger the deletion mechanism of the xml cache module. This mechanism is to delete according to the principle of not calling for the longest time. It will first sort according to the access time, and delete the xml files with a critical value of 10% that have not been accessed for the longest time.
所述双路缓存机制的读取流程是：当用户进行数据的读取操作时，首先应用系统的缓存管理模块调用管理进程，来判断应用系统的xml缓存模块中是否存在所访问数据的xml文件，若存在，则直接读取xml缓存模块中的这个文件，然后解析出数据，以html方式返回给用户；若不存在，则需要管理进程去页面缓存中查看是否存在该数据的缓存信息，若存在，则把这些信息引入到xml缓存中，然后再根据xml的处理机制去提供用户的使用，若不存在，则需要管理进程从数据库中调用相关的数据表信息，然后进行页面缓存的保存。The reading process of the two-way cache mechanism is: when the user performs a data reading operation, first the cache management module of the application system invokes the management process to determine whether there is an xml file of the accessed data in the xml cache module of the application system , if it exists, directly read the file in the xml cache module, then parse out the data, and return it to the user in html; if it does not exist, the management process needs to go to the page cache to check whether the cache information of the data exists, if If it exists, import this information into the xml cache, and then provide users with the use of the xml processing mechanism. If it does not exist, the management process needs to call the relevant data table information from the database, and then save the page cache.
所述双路缓存机制的写入流程是：当用户进行的是注册等数据的录入操作时，用户在页面填写完数据信息后，则会调用缓存管理进程，将数据存入数据库，同时在页面缓存和xml缓存中存储一个缓存数据；若在页面缓存和xml缓存中已经存在该数据的缓存信息，则更新这个缓存数据。The writing process of the two-way cache mechanism is: when the user performs registration and other data entry operations, after the user fills in the data information on the page, the cache management process will be called to store the data in the database, and at the same time, the page will A cache data is stored in the cache and the xml cache; if the cache information of the data already exists in the page cache and the xml cache, the cache data is updated.
此外，本发明的Web服务双路缓存机制可以根据Web服务的业务量进行扩充和设定，具有较大的灵活性。所述xml机制可以满足跨平台的应用需求，具有很好的移植性。In addition, the two-way caching mechanism of the Web service of the present invention can be expanded and set according to the business volume of the Web service, and has greater flexibility. The xml mechanism can meet cross-platform application requirements and has good portability.
本发明的一种Web服务的双路缓存机制设计方法的有益效果是：该Web服务双路缓存机制动静结合的双路缓存机制能较大程度上提高应用系统服务器性能，提高系统对海量数据处理的实时性和准确性，减轻应用系统服务器的访问压力，提高对用户的服务质量。The beneficial effect of the design method of a two-way caching mechanism for Web services of the present invention is that the two-way caching mechanism of the Web service two-way caching mechanism combined with dynamic and static can greatly improve the performance of the application system server and improve the system's ability to process massive data. Real-time and accuracy, reduce the access pressure of the application system server, and improve the service quality for users.
附图说明Description of drawings
附图1为该Web服务的双路缓存机制的读取流程图；Accompanying drawing 1 is the reading flowchart of the two-way cache mechanism of this Web service;
附图2为该Web服务的双路缓存机制的写入流程图。Accompanying drawing 2 is the writing flowchart of the two-way cache mechanism of this Web service.
具体实施方式detailed description
下面结合附图对本发明的一种Web服务的双路缓存机制设计方法作进一步详细说明。A method for designing a two-way caching mechanism for Web services according to the present invention will be further described in detail below in conjunction with the accompanying drawings.
本发明是基于动态页面和静态页面两种缓存机制相结合，而提出的一种双路缓存机制，是通过数据库和xml缓存文件进行双路读写，同步操作触发更新的缓存机制，以xml缓存模块作为用户访问时的使用库，应用系统中的数据库作为备用库，数据的读写操作均在xml缓存文件中完成；其设计内容由双路缓存机制的管理过程、双路缓存机制的读取流程和双路缓存机制的写入流程三部分组成。The present invention is based on the combination of two caching mechanisms of dynamic pages and static pages, and proposes a two-way caching mechanism, which is a two-way caching mechanism for reading and writing through the database and xml cache files, synchronous operation triggering updates, and xml caching The module is used as a library for user access, and the database in the application system is used as a backup library. Data read and write operations are all completed in the xml cache file; its design content consists of the management process of the two-way cache mechanism and the reading of the two-way cache mechanism The process and the writing process of the two-way cache mechanism consist of three parts.
通过本发明的实施方案进一步了解该缓存机制的技术特征：Further understand the technical characteristics of this caching mechanism through the embodiment of the present invention:
首先，在Web应用系统中创建一个分区用来作为xml缓存模块数据存储的地方，这个分区的大小需要根据本系统的实际数据产生量来设定，当然，也不能太大，会造成用户空间的浪费，这个xml缓存空间的创建，大小的更改以及xml缓存文件的创建、删除和管理都有缓存管理模块进行统一的管理。First, create a partition in the web application system to store the data of the xml cache module. The size of this partition needs to be set according to the actual amount of data generated by the system. Of course, it should not be too large, which will cause user space Waste, the creation of the xml cache space, the change of the size, and the creation, deletion and management of the xml cache file are all managed by the cache management module.
其次，缓存管理模块利用数组创建动态页面缓存机制，先定义一个一维数组，数组中放置两个元素，第一个元素是用来存放缓存数据的，第二个元素是表示该缓存数据的已经存在时间的。第一个元素是在数据调用时加载显示，在显示时首先要判断该缓存数据是否过期，这个操作主要是通过元素的第二个元素的时间来判断，没过期就加载该缓存单元过期就删除该数据，重新从数据库中进行读取，并引入缓存函数，生成缓存数据和标志时间。Secondly, the cache management module uses an array to create a dynamic page cache mechanism. First, a one-dimensional array is defined, and two elements are placed in the array. The first element is used to store cached data, and the second element represents the cached data. Existence of time. The first element is loaded and displayed when the data is called. When displaying, it is first necessary to judge whether the cached data has expired. This operation is mainly judged by the time of the second element of the element. If it is not expired, the cache unit will be loaded and deleted. The data is read from the database again, and a cache function is introduced to generate cache data and mark time.
再次，对于双路缓存的管理过程是由系统的缓存管理模块统一的调度和管理，对于页面缓存模块，对于一定时间没有访问的页面缓存信息会进行删除，因为xml缓存占用的不是内存信息而是分区空间，因此在本发明的设计中，对于xml的删除时间没有做严格的规定，而是根据应用系统的规模也就是数据量的大小建立了一个xml文件个数的临界值，当xml缓存模块中的xml文件达到一定个数的时候，在本方案中，对于xml缓存占用的空间做了最大量的估计，即最大量的缓存内容只占整个缓存分区的60%-80%，这样保证了xml的缓存模块的稳定运行，消除空间占满导致xml缓存失效的危险。当xml缓存的xml文件个数达到阀值的时候，缓存管理模块的管理进程会触发xml缓存模块的删除机制，这个机制是按照最长时间不调用的原理进行删除，它首先会按照访问时间进行排序，把最长时间不访问的临界值10%的xml文件进行删除操作，这样就保证了最大空间的利用和xml缓存的最大数量的保存，消除了内存不断变换引起的系统性能下降的问题，提高了用户访问的效率。Again, the management process of the two-way cache is uniformly scheduled and managed by the cache management module of the system. For the page cache module, the page cache information that has not been accessed for a certain period of time will be deleted, because the xml cache occupies not memory information but partition space, therefore in the design of the present invention, there is no strict regulation for the deletion time of xml, but according to the scale of the application system, that is, the size of the data volume, a critical value of the number of xml files is established, when the xml cache module When the number of xml files in the xml file reaches a certain number, in this solution, the maximum amount of space occupied by the xml cache is estimated, that is, the maximum amount of cache content only accounts for 60%-80% of the entire cache partition, which ensures The stable operation of the xml cache module eliminates the danger of xml cache invalidation due to space occupation. When the number of xml files in the xml cache reaches the threshold, the management process of the cache management module will trigger the deletion mechanism of the xml cache module. This mechanism is to delete according to the principle of not calling for the longest time. It will first be deleted according to the access time Sort, delete the xml files with a critical value of 10% that have not been accessed for the longest time, so as to ensure the maximum utilization of space and the storage of the maximum number of xml caches, and eliminate the problem of system performance degradation caused by continuous memory changes. Improve the efficiency of user access.
此外，如图1所示，本发明双路缓存机制的读取流程如下：当用户进行数据的读取操作时，首先应用系统的管理模块会调用管理进程，由此进程去判断应用系统中的xml缓存模块中是否存在所访问数据的xml文件，如果存在，就直接读取xml缓存模块中的这个文件，然后解析出数据，以html方式返回给用户；如果在xml缓存模块中不存在所要访问数据的xml文件，这就需要管理进程去页面缓存中查看是否存在该数据的缓存信息，如果存在就需要把这些信息引入到xml缓存中，然后再根据xml的处理机制去提供用户的使用，如果不存在，这时候就需要管理进程从数据库中调用相关的数据表信息，然后进行页面缓存的保存，以方便以后用户对该信息的访问。In addition, as shown in Figure 1, the reading process of the two-way cache mechanism of the present invention is as follows: when the user performs a data reading operation, the management module of the application system will first call the management process, and the process will determine the data in the application system. Whether there is an xml file of the accessed data in the xml cache module, if it exists, directly read the file in the xml cache module, then parse the data, and return it to the user in html; if there is no xml file in the xml cache module The xml file of the data, which requires the management process to check whether the cache information of the data exists in the page cache. If it exists, it needs to import this information into the xml cache, and then provide the user with the use of the xml processing mechanism. If If it does not exist, at this time, the management process needs to call the relevant data table information from the database, and then save the page cache to facilitate the user's access to the information in the future.
如图2所示，本发明双路缓存机制的写入流程如下：如果用户进行的是注册等数据的录入操作，用户在页面填写完数据信息后，会调用缓存管理进程，由此进程去判断XML缓存中是否存在该数据的缓存信息，如果存在，就更新该数据的内容和时间，如果不存在，就将该数据写入XML缓存中；再次调用缓存管理进程去判断页面缓存中是否存在该数据的缓存信息，如果存在，就更新该数据的内容和时间，如果不存在，就将该数据写入页面缓存，最终将数据存入数据库，整个写入过程保证用户数据的一致性，以方便用户的查看和使用。As shown in Figure 2, the writing process of the two-way cache mechanism of the present invention is as follows: if the user performs data entry operations such as registration, after the user fills in the data information on the page, the cache management process will be called, and the process will judge Whether there is the cache information of the data in the XML cache, if it exists, update the content and time of the data, if not, write the data into the XML cache; call the cache management process again to determine whether the page cache exists If the cache information of the data exists, the content and time of the data will be updated. If it does not exist, the data will be written into the page cache, and finally the data will be stored in the database. The entire writing process ensures the consistency of user data to facilitate Viewing and Use by Users.
除说明书所述技术特征之外，均为本专业技术人员的已知技术。Except for the technical features described in the description, all are known technologies by those skilled in the art.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210475846.1ACN103049478B (en) | 2012-11-22 | 2012-11-22 | Double-way buffering mechanism design method for Web services |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210475846.1ACN103049478B (en) | 2012-11-22 | 2012-11-22 | Double-way buffering mechanism design method for Web services |
| Publication Number | Publication Date |
|---|---|
| CN103049478A CN103049478A (en) | 2013-04-17 |
| CN103049478Btrue CN103049478B (en) | 2017-05-03 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210475846.1AActiveCN103049478B (en) | 2012-11-22 | 2012-11-22 | Double-way buffering mechanism design method for Web services |
| Country | Link |
|---|---|
| CN (1) | CN103049478B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281668A (en)* | 2014-09-28 | 2015-01-14 | 墨仕（厦门）电子商务有限公司 | Data processing method |
| CN109582895A (en)* | 2018-12-04 | 2019-04-05 | 山东浪潮通软信息科技有限公司 | A kind of cache implementing method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6542967B1 (en)* | 1999-04-12 | 2003-04-01 | Novell, Inc. | Cache object store |
| CN101154230A (en)* | 2006-09-30 | 2008-04-02 | 中兴通讯股份有限公司 | Responding method for large data volume specified searching web pages |
| CN102081650A (en)* | 2010-12-29 | 2011-06-01 | 上海网达软件有限公司 | Method for rapidly displaying user interface of embedded type platform |
| CN102184236A (en)* | 2011-05-16 | 2011-09-14 | 复旦大学 | Distance-based adaptive webpage replacement method in spatial database |
| CN102609364A (en)* | 2012-02-14 | 2012-07-25 | 华中科技大学 | Cache exchange method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6542967B1 (en)* | 1999-04-12 | 2003-04-01 | Novell, Inc. | Cache object store |
| CN101154230A (en)* | 2006-09-30 | 2008-04-02 | 中兴通讯股份有限公司 | Responding method for large data volume specified searching web pages |
| CN102081650A (en)* | 2010-12-29 | 2011-06-01 | 上海网达软件有限公司 | Method for rapidly displaying user interface of embedded type platform |
| CN102184236A (en)* | 2011-05-16 | 2011-09-14 | 复旦大学 | Distance-based adaptive webpage replacement method in spatial database |
| CN102609364A (en)* | 2012-02-14 | 2012-07-25 | 华中科技大学 | Cache exchange method |
| Publication number | Publication date |
|---|---|
| CN103049478A (en) | 2013-04-17 |
| Publication | Publication Date | Title |
|---|---|---|
| CN102014158B (en) | Cloud storage service client high-efficiency fine-granularity data caching system and method | |
| CN103116472B (en) | The method, system and device of ttl value in dynamic change data cache | |
| CN101916289B (en) | Method for establishing digital library storage system supporting mass small files and dynamic backup number | |
| CN104580437A (en) | Cloud storage client and high-efficiency data access method thereof | |
| CN105183839A (en) | Hadoop-based storage optimizing method for small file hierachical indexing | |
| US10089317B2 (en) | System and method for supporting elastic data metadata compression in a distributed data grid | |
| CN103455577A (en) | Multi-backup nearby storage and reading method and system of cloud host mirror image file | |
| CN102023973A (en) | Method, device and system for synchronizing database with application cache server | |
| Kim et al. | Improving the storage performance of smartphones through journaling in non-volatile memory | |
| CN104657143A (en) | High-performance data caching method | |
| CN104657461A (en) | File system metadata search caching method based on internal memory and SSD (Solid State Disk) collaboration | |
| CN104281673B (en) | A kind of caching structure system of database and corresponding construction method | |
| CN104519103A (en) | Synchronous network data processing method, server and related system | |
| WO2021008425A1 (en) | System startup method and related device | |
| CN111966511A (en) | Message queue data read-write processing method and device | |
| CN108959500A (en) | A kind of object storage method, device, equipment and computer readable storage medium | |
| CN112650694B (en) | Data reading method and device, cache proxy server and storage medium | |
| CN103020077A (en) | Method for managing memory of real-time database of power system | |
| CN103049478B (en) | Double-way buffering mechanism design method for Web services | |
| CN113448897B (en) | An optimization method suitable for pure user mode remote direct memory access | |
| CN101377788A (en) | Method and system of caching management in cluster file system | |
| US11586353B2 (en) | Optimized access to high-speed storage device | |
| CN103209212B (en) | Based on the data cache method in the Web network management client of RIA and system | |
| CN112559568B (en) | Method, device and computer-readable storage medium for determining virtual items | |
| CN115080625A (en) | Caching method, device, device and storage medium based on Spring Cache caching framework |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |