











技术领域technical field
本申请涉及计算机技术领域,尤其涉及一种缓存管理方法、装置、存储介质及电子设备。The present application relates to the field of computer technology, and in particular to a cache management method, device, storage medium and electronic equipment.
背景技术Background technique
随着互联网的普及,在大批用户涌入服务平台的同时,也带来了平台访问数据量几何级的飞速增长,对于服务平台的管理带来了挑战和机遇。为了保障服务平台与客户端之间的事务正常进行,会基于服务平台采用数据缓存更新技术,以使得服务平台能具有更高的对外处理速率。With the popularization of the Internet, while a large number of users are pouring into the service platform, it has also brought about a rapid growth in the amount of platform access data, which has brought challenges and opportunities to the management of the service platform. In order to ensure the normal progress of transactions between the service platform and the client, data cache update technology will be adopted based on the service platform, so that the service platform can have a higher external processing rate.
发明内容Contents of the invention
本说明书提供了一种缓存管理方法、装置、存储介质及电子设备,所述技术方案如下:This specification provides a cache management method, device, storage medium and electronic equipment, and the technical solution is as follows:
第一方面,本说明书提供了一种缓存管理方法,所述方法包括:In a first aspect, this specification provides a cache management method, the method comprising:
监测针对热数据和冷数据的缓存波动状态,基于所述缓存波动状态获取访问用户分布信息;Monitoring the cache fluctuation state for hot data and cold data, and obtaining access user distribution information based on the cache fluctuation state;
基于所述访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节,所述第一数据缓存空间为缓存所述热数据的数据存储空间,所述第二数据缓存空间为缓存所述冷数据的数据缓存空间。Based on the access user distribution information, the cache space adjustment is performed on the first data cache space and/or the second data cache space, the first data cache space is a data storage space for caching the hot data, and the second data cache space The cache space is a data cache space for caching the cold data.
第二方面,本说明书提供了一种缓存管理装置,所述装置包括:In a second aspect, this specification provides a cache management device, the device comprising:
信息获取模块,用于监测针对热数据和冷数据的缓存波动状态,基于所述缓存波动状态获取访问用户分布信息;An information acquisition module, configured to monitor the cache fluctuation state for hot data and cold data, and obtain access user distribution information based on the cache fluctuation state;
数据处理模块,用于基于所述访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节,所述第一数据缓存空间为缓存所述热数据的数据存储空间,所述第二数据缓存空间为缓存所述冷数据的数据缓存空间。A data processing module, configured to adjust the cache space of the first data cache space and/or the second data cache space based on the access user distribution information, the first data cache space is a data storage space for caching the hot data , the second data cache space is a data cache space for caching the cold data.
第三方面,本说明书提供一种计算机存储介质,所述计算机存储介质存储有多条指令,所述指令适于由处理器加载并执行上述的方法步骤。In a third aspect, the specification provides a computer storage medium, where a plurality of instructions are stored in the computer storage medium, and the instructions are suitable for being loaded by a processor to execute the above-mentioned method steps.
第四方面,本说明书提供一种电子设备,可包括:处理器和存储器;其中,所述存储器存储有计算机程序,所述计算机程序适于由所述处理器加载并执行上述的方法步骤。In a fourth aspect, the specification provides an electronic device, which may include: a processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute the above-mentioned method steps.
本说明书一些实施例提供的技术方案带来的有益效果至少包括:The beneficial effects brought by the technical solutions provided by some embodiments of this specification at least include:
在本说明书一个或多个实施例中,服务平台通过监测针对热数据和冷数据的缓存波动状态以获取当前的访问用户分布信息,基于访问用户分布信息对第一数据缓存空间和/或第二数据缓存空间进行缓存空间动态调节。基于访问用户分布信息一定程度上可预测冷热数据的缓存量的波动,可预先对相应缓存空间进行调节,提高了请求数据时缓存命中率,降低服务平台的缓存处理压力。In one or more embodiments of this specification, the service platform obtains the current access user distribution information by monitoring the cache fluctuation status for hot data and cold data, and based on the access user distribution information, caches the first data and/or the second The data cache space is dynamically adjusted for the cache space. Based on the access user distribution information, the fluctuation of the cache volume of hot and cold data can be predicted to a certain extent, and the corresponding cache space can be adjusted in advance, which improves the cache hit rate when requesting data and reduces the cache processing pressure of the service platform.
附图说明Description of drawings
为了更清楚地说明本说明书或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in this specification or the prior art, the following will briefly introduce the drawings that need to be used in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only the present invention. For some embodiments of the application, those skilled in the art can also obtain other drawings based on these drawings without creative work.
图1是本说明书提供的一种缓存管理系统的场景示意图;FIG. 1 is a schematic diagram of a scene of a cache management system provided in this manual;
图2是本说明书提供的一种缓存管理方法的流程示意图;FIG. 2 is a schematic flow diagram of a cache management method provided in this specification;
图3是本说明书提供的另一种缓存管理方法的流程示意图;FIG. 3 is a schematic flowchart of another cache management method provided in this specification;
图4是本说明书提供的另一种缓存管理方法的流程示意图;FIG. 4 is a schematic flowchart of another cache management method provided in this specification;
图5是本说明书提供的一种缓存管理装置的结构示意图;FIG. 5 is a schematic structural diagram of a cache management device provided in this specification;
图6是本说明书提供的一种信息获取模块的结构示意图;Fig. 6 is a schematic structural diagram of an information acquisition module provided in this specification;
图7是本说明书提供的一种数据处理模块的结构示意图;Fig. 7 is a schematic structural diagram of a data processing module provided in this specification;
图8是本说明书提供的另一种缓存管理装置的结构示意图;FIG. 8 is a schematic structural diagram of another cache management device provided in this specification;
图9是本说明书提供的一种电子设备的结构示意图;Fig. 9 is a schematic structural diagram of an electronic device provided in this specification;
图10是本说明书提供的操作系统和用户空间的结构示意图;Figure 10 is a schematic structural diagram of the operating system and user space provided in this manual;
图11是图10中安卓操作系统的架构图;Fig. 11 is the architecture diagram of the Android operating system in Fig. 10;
图12是图10中IOS操作系统的架构图。FIG. 12 is a structural diagram of the IOS operating system in FIG. 10 .
具体实施方式Detailed ways
下面将结合本说明书中的附图,对本说明书中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in this specification will be clearly and completely described below in conjunction with the accompanying drawings in this specification. Obviously, the described embodiments are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.
在本申请的描述中,需要理解的是,术语“第一”、“第二”等仅用于描述目的,而不能理解为指示或暗示相对重要性。在本申请的描述中,需要说明的是,除非另有明确的规定和限定,“包括”和“具有”以及它们任何变形,意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元,而是可选地还包括没有列出的步骤或单元,或可选地还包括对于这些过程、方法、产品或设备固有的其他步骤或单元。对于本领域的普通技术人员而言,可以具体情况理解上述术语在本申请中的具体含义。此外,在本申请的描述中,除非另有说明,“多个”是指两个或两个以上。“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。In the description of the present application, it should be understood that the terms "first", "second" and so on are used for descriptive purposes only, and should not be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise specified and limited, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally further includes For other steps or units inherent in these processes, methods, products or devices. Those of ordinary skill in the art can understand the specific meanings of the above terms in this application in specific situations. In addition, in the description of the present application, unless otherwise specified, "plurality" means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. The character "/" generally indicates that the contextual objects are an "or" relationship.
在相关技术中,缓存管理作为提高服务平台响应速度、降低服务压力重要手段,通常由于成本因素服务平台所能提供的缓存空间有限,由于缓存空间的限制,服务平台一般在为若干类型缓存数据分别分配缓存空间之后基于设置的缓存淘汰策略进行缓存优化,然而采用这种方式在面临诸如大量用户涌入所带来的缓存压力时,无法有效缓解缓存压力。In related technologies, cache management is an important means to improve the response speed of the service platform and reduce service pressure. Usually, due to cost factors, the cache space that the service platform can provide is limited. Due to the limitation of cache space, the service platform generally caches data for several types separately. After the cache space is allocated, cache optimization is performed based on the set cache elimination strategy. However, this method cannot effectively alleviate the cache pressure when facing the cache pressure caused by the influx of a large number of users.
下面结合具体的实施例对本申请进行详细说明。The present application will be described in detail below in conjunction with specific embodiments.
请参见图1,为本说明书提供的一种缓存管理系统的场景示意图。如图1所示,所述缓存管理系统至少可以包括客户端集群和服务平台100。Please refer to FIG. 1 , which is a schematic diagram of a scene of a cache management system provided in this specification. As shown in FIG. 1 , the cache management system may at least include a client cluster and a service platform 100 .
所述客户端集群可以包括至少一个客户端,如图1所示,具体包括用户1对应的客户端1、用户2对应的客户端2、…、用户n对应的客户端n,n为大于0的整数。The client cluster may include at least one client, as shown in FIG. 1 , specifically including
客户端集群中的各客户端可以是具备通信功能的电子设备,该电子设备包括但不限于:可穿戴设备、手持设备、个人电脑、平板电脑、车载设备、智能手机、计算设备或连接到无线调制解调器的其它处理设备等。在不同的网络中电子设备可以叫做不同的名称,例如:用户设备、接入终端、用户单元、用户站、移动站、移动台、远方站、远程终端、移动设备、用户终端、终端、无线通信设备、用户代理或用户装置、蜂窝电话、无绳电话、个人数字处理(personal digital assistant,PDA)、5G网络或未来演进网络中的电子设备等。Each client in the client cluster may be an electronic device capable of communication, including but not limited to: wearable device, handheld device, personal computer, tablet computer, vehicle-mounted device, smart phone, computing device or connected to a wireless Other processing equipment for modems, etc. Electronic equipment can be called by different names in different networks, such as: user equipment, access terminal, subscriber unit, subscriber station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication Devices, user agents or user devices, cellular phones, cordless phones, personal digital assistants (PDAs), electronic devices in 5G networks or future evolution networks, etc.
所述服务平台100可以是单独的服务器设备,例如:机架式、刀片、塔式、或者机柜式的服务器设备,或采用工作站、大型计算机等具备较强计算能力硬件设备;也可以是采用多个服务器组成的服务器集群,所述服务集群中的各服务器可以是以对称方式组成的,其中每台服务器在事务链路中功能等价、地位等价,各服务器均可单独对外提供服务,所述单独提供服务可以理解为无需另外的服务器的辅助。The service platform 100 can be a separate server device, for example: a rack-mounted, blade, tower-type, or cabinet-type server device, or a workstation, a large computer, and other hardware devices with strong computing capabilities; A server cluster composed of two servers, each server in the service cluster can be formed in a symmetrical manner, wherein each server has equivalent functions and statuses in the transaction link, and each server can provide external services independently, so The aforementioned independent provision of services can be understood as not requiring the assistance of another server.
在本说明书的一个或多个实施例中,服务平台100与客户端集群中的至少一个客户端可建立通信连接,基于该通信连接完成服务数据的交互。服务平台100可以对客户端集群中的至少一个客户端提供相应事务服务,事务服务包括但不限于消费服务、购物服务、金融服务、授信服务等等,具体事务服务类型基于实际应用情况确定。In one or more embodiments of this specification, the service platform 100 can establish a communication connection with at least one client in the client cluster, and complete the interaction of service data based on the communication connection. The service platform 100 can provide corresponding transaction services to at least one client in the client cluster. Transaction services include but are not limited to consumer services, shopping services, financial services, credit services, etc., and the specific transaction service types are determined based on actual application conditions.
在服务平台100对若干客户端提供事务服务的过程,在诸如相关事务活动发布、平台热点事件等场景下,会出现大量的用户通过所持有的客户端访问服务平台100。为了更好的对外提供事务服务,服务平台100可以是预先引入分布式服务部署以及平台缓存建设,而在某些场景的突发客户访问流量下,其新老用户的访问差异较大,使得通常的缓存策略很难灵活地优化缓存,基于此服务平台100可通过执行本说明书一个或多个实施例的缓存管理方法,对服务平台100用于存储热数据的第一缓存空间以及用户存储冷数据的第二缓存空间进行优化。In the process of the service platform 100 providing transaction services to several clients, a large number of users will access the service platform 100 through the clients they hold in scenarios such as release of related transaction activities and hot events on the platform. In order to better provide transaction services externally, the service platform 100 can pre-introduce distributed service deployment and platform cache construction. However, under sudden customer access traffic in certain scenarios, the access differences between new and old users are relatively large, making the usual It is difficult to flexibly optimize the cache based on the caching strategy of this specification. Based on this, the service platform 100 can implement the cache management method of one or more embodiments of this specification to store the first cache space used by the service platform 100 to store hot data and the user to store cold data. The second cache space is optimized.
在本说明书一个或多个实施例中,服务平台100会将各类型的服务数据进行存储,客户端会基于实际场景访问或查询服务平台100的相应服务数据,为避免在数据量增大、访问集中等场景下,出现服务平台100维护的数据库的负担加重,数据库响应恶化,网站显示延迟等重大影响。服务平台100可采用缓存机制进行数据访问的优化,In one or more embodiments of this specification, the service platform 100 will store various types of service data, and the client will access or query the corresponding service data of the service platform 100 based on actual scenarios. In scenarios such as centralization, the burden on the database maintained by the service platform 100 increases, the response of the database deteriorates, and the display of the website is delayed. The service platform 100 can use the cache mechanism to optimize data access,
示意性的,一种缓存机制缓存原理可以是检查客户端对应用户请求的数据时,服务平台100维护的缓存空间中是否存在该请求数据,若存在直接返回,无需查询数据库。Schematically, a caching mechanism caching principle may be to check whether the requested data exists in the cache space maintained by the service platform 100 when the data requested by the client corresponds to the user, and if so, return it directly without querying the database.
若请求数据在缓存空间中查询不到,服务平台100查询数据库,返回数据,同时把数据存储到缓存中一份。If the requested data cannot be found in the cache space, the service platform 100 queries the database, returns the data, and stores a copy of the data in the cache.
示意性的,服务平台100可以保持缓存的“新鲜性”,服务平台100每当数据发生变化的时候,同步更新缓存空间的缓存信息,确保用户不会在缓存空间取到旧的数据。Schematically, the service platform 100 can maintain the "freshness" of the cache. Whenever the data changes, the service platform 100 will synchronously update the cache information in the cache space to ensure that users will not get old data in the cache space.
需要说明的是,服务平台100与客户端集群中的至少一个客户端通过网络建立通信连接进行交互通信,其中,网络可以是无线网络,也可以是有线网络,无线网络包括但不限于蜂窝网络、无线局域网、红外网络或蓝牙网络,有线网络包括但不限于以太网、通用串行总线(universal serial bus,USB)或控制器局域网络。在说明书一个或多个实施例中,使用包括超文本标记语言(Hyper Text Mark-up Language,HTML)、可扩展标记语言(Extensible Markup Language,XML)等的技术和/或格式来代表通过网络交换的数据(如目标压缩包)。此外还可以使用诸如安全套接字层(Secure Socket Layer,SSL)、传输层安全(Transport Layer Security,TLS)、虚拟专用网络(Virtual Private Network,VPN)、网际协议安全(Internet Protocol Security,IPsec)等常规加密技术来加密所有或者一些链路。在另一些实施例中,还可以使用定制和/或专用数据通信技术取代或者补充上述数据通信技术。It should be noted that the service platform 100 establishes a communication connection with at least one client in the client cluster for interactive communication through a network, wherein the network may be a wireless network or a wired network, and the wireless network includes but is not limited to a cellular network, Wireless local area network, infrared network or bluetooth network, wired network includes but not limited to Ethernet, universal serial bus (universal serial bus, USB) or controller area network. In one or more embodiments of the specification, technologies and/or formats including Hyper Text Markup Language (Hyper Text Mark-up Language, HTML), Extensible Markup Language (Extensible Markup Language, XML) etc. are used to represent data (such as the target archive). In addition, you can also use methods such as Secure Socket Layer (Secure Socket Layer, SSL), Transport Layer Security (Transport Layer Security, TLS), Virtual Private Network (Virtual Private Network, VPN), Internet Protocol Security (Internet Protocol Security, IPsec) and other conventional encryption techniques to encrypt all or some links. In some other embodiments, customized and/or dedicated data communication technologies may also be used to replace or supplement the above data communication technologies.
本说明书所提供的缓存管理系统实施例与一个或多个实施例中的所述缓存管理方法属于同一构思,在说明书一个或多个实施例涉及的所述缓存管理方法对应的执行主体可以是上述服务平台100;在说明书一个或多个实施例涉及的所述缓存管理方法对应的执行主体也可以其他电子设备,具体基于实际应用环境确定。缓存管理系统实施例其体现实现过程可详见下述的方法实施例,这里不再赘述。The cache management system embodiment provided in this specification and the cache management method in one or more embodiments belong to the same concept, and the execution subject corresponding to the cache management method involved in one or more embodiments of the specification may be the above-mentioned The service platform 100; the execution subject corresponding to the cache management method involved in one or more embodiments of the specification may also be other electronic devices, which are specifically determined based on the actual application environment. The implementation process of the embodiment of the cache management system can be referred to the following method embodiment in detail, and will not be repeated here.
基于图1所示的场景示意图,下面对本说明书一个或多个实施例提供的模型处理方法进行详细介绍。Based on the schematic diagram of the scene shown in FIG. 1 , the model processing method provided by one or more embodiments of this specification will be described in detail below.
请参见图2,为本说明书一个或多个实施例提供了一种缓存管理方法的流程示意图,该方法可依赖于计算机程序实现,可运行于基于冯诺依曼体系的缓存管理装置上。该计算机程序可集成在应用中,也可作为独立的工具类应用运行。所述缓存管理装置可以为服务平台。Please refer to FIG. 2 , which provides a schematic flowchart of a cache management method for one or more embodiments of this specification. The method can be implemented by relying on a computer program and can run on a cache management device based on the von Neumann architecture. The computer program can be integrated in the application, or run as an independent utility application. The cache management device may be a service platform.
具体的,该缓存管理方法包括:Specifically, the cache management method includes:
S102:监测针对热数据和冷数据的缓存波动状态,基于所述缓存波动状态获取访问用户分布信息;S102: Monitor the cache fluctuation state for hot data and cold data, and obtain access user distribution information based on the cache fluctuation state;
在基于服务平台的实际应用场景下,通常根据访问频率或者基于用户的访问操作收集操作日志、服务信息等服务数据,将服务数据根据数据访问热度分为冷数据和热数据,然后根据冷热数据类型将数据存储或迁移至相应的存储空间(如数据库)中。In the actual application scenario based on the service platform, service data such as operation logs and service information are usually collected according to the access frequency or user-based access operations, and the service data is divided into cold data and hot data according to the data access popularity, and then according to the hot and cold data Type stores or migrates data to corresponding storage spaces (such as databases).
根据一些实施例中,服务平台通常采用缓存机制进行数据访问的优化,服务平台可维护用于存储服务数据的缓存空间以及数据库,一种缓存机制缓存原理可以是在客户端发起用户请求以请求相应数据时,服务平台检查客户端对应用户请求的请求数据,服务平台首先在维护的缓存空间中查询是否存在该请求数据,若存在直接向客户端返回请求数据,无需查询数据库。若请求数据在缓存空间中查询不到,服务平台100查询数据库,返回请求数据,同时把请求数据存储到缓存空间中一份。According to some embodiments, the service platform usually uses a cache mechanism to optimize data access. The service platform can maintain a cache space and a database for storing service data. A cache mechanism cache principle can be to initiate a user request on the client to request the corresponding When processing data, the service platform checks the client’s request data corresponding to the user’s request. The service platform first checks whether the requested data exists in the maintained cache space. If there is, it directly returns the requested data to the client without querying the database. If the requested data cannot be found in the cache space, the service platform 100 queries the database, returns the requested data, and stores a copy of the requested data in the cache space.
可以理解的,服务平台在数据处理过程中可以保持缓存的“新鲜性”,服务平台每当服务数据发生变化的时候,可以同步更新缓存空间的缓存信息,确保用户不会在缓存空间取到旧的服务数据。It is understandable that the service platform can maintain the "freshness" of the cache during data processing. Whenever the service data changes, the service platform can synchronously update the cache information in the cache space to ensure that users will not get old data from the cache space. service data.
可以理解的,在服务平台的实际应用场景下,服务平台通常根据数据热度将服务数据划分为冷数据以及热数据,在缓存空间中可以存储有冷数据以及热数据,在数据库中可以存储有冷数据以及热数据。It is understandable that in the actual application scenario of the service platform, the service platform usually divides the service data into cold data and hot data according to the data heat, and the cold data and hot data can be stored in the cache space, and the cold data can be stored in the database. data and hot data.
所述缓存波动状态用于表征用户访问缓存中冷数据和/或热数据的缓存波动情况。所述缓存波动状态可以是基于冷数据和/或热数据的访问波动量进行表征。The cache fluctuation state is used to represent the cache fluctuation situation of the user accessing cold data and/or hot data in the cache. The cache fluctuation state may be characterized based on access fluctuations of cold data and/or hot data.
可以理解的,服务平台可以实时或周期性监测冷数据和/或热数据的缓存波动量来确定缓存波动状态,在冷热数据波动比例比较大或者平稳的状态下,服务平台能够自动监测到,并且通过执行本说明书涉及的缓存管理方法进行处理。It can be understood that the service platform can monitor the buffer fluctuations of cold data and/or hot data in real time or periodically to determine the buffer fluctuation status. When the fluctuation ratio of cold and hot data is relatively large or stable, the service platform can automatically monitor. And it is processed by executing the cache management method involved in this specification.
具体的,服务平台可以实时或周期性监测冷数据和/或热数据的缓存波动量来确定缓存波动状态,基于所述缓存波动状态获取访问用户分布信息。Specifically, the service platform can monitor the cache fluctuations of cold data and/or hot data in real time or periodically to determine the cache fluctuation status, and obtain access user distribution information based on the cache fluctuation status.
所述访问用户分布信息用于反馈平台不同访问用户类型的用户其访问用户类型维度的用户类型分布情况,通过访问用户分布信息可以是诸如若干访问用户类型的用户量、用户类型分布状况、新老用户波动数据等信息中的一种或多种的拟合。The access user distribution information is used to feed back the user type distribution in the access user type dimension of users of different access user types on the platform. The access user distribution information can be such as the number of users of several access user types, the distribution of user types, new and old Fitting of one or more types of information such as user fluctuation data.
可选的,服务平台获取访问用户分布信息,可以是根据针对所述服务平台的访问用户所划分的用户类型,分别获取各用户类型在监测窗口时间内的用户访问数据,基于各用户类型的用户访问数据确定相应类型用户波动数据,将其作为访问用户分布信息。Optionally, the service platform obtains access user distribution information, which may be based on the user types divided by the access users of the service platform, and obtains the user access data of each user type within the monitoring window time, based on the user types of each user type The access data determines the corresponding type of user fluctuation data, which is used as access user distribution information.
示意性的,访问用户分布信息可以是新老用户波动数据,例如新老用户波动比例、新老用户波动数量等等。服务平台可以(在监测窗口时间内)获取针对新用户类型的第一用户访问数据,以及针对老用户类型的第二用户访问数据;Schematically, the access user distribution information may be new and old user fluctuation data, such as the fluctuation ratio of new and old users, the fluctuation number of new and old users, and the like. The service platform can (within the monitoring window time) acquire the first user access data for the new user type, and the second user access data for the old user type;
所述第一用户访问数据可以是(在监测窗口时间内)新用户类型所对应的用户访问数量、用户访问增长数量等用户访问数据;所述第二用户访问数据可以是(在监测窗口时间内)老用户类型所对应的用户访问数量、用户访问增长数量等用户访问数据。The first user access data can be (within the monitoring window time) user access data such as the number of user visits corresponding to the new user type, the number of user access increases; the second user access data can be (within the monitoring window time) ) User access data such as the number of user visits corresponding to the old user type, the number of user access growth, etc.
其中,新用户类型和老用户类型为针对所述服务平台的访问用户所划分的用户类型,如将访问平台次数小于次数阈值(如1次)的访问用户划分为新用户类型,将访问平台次数大于次数阈值的访问用户划分为老用户类型,新用户类型和老用户类型的类型划分标准可基于实际应用场景进行自定义,此处不作具体限定。Wherein, the new user type and the old user type are the user types divided for the visiting users of the service platform, such as dividing the visiting users whose number of visits to the platform is less than the number of times threshold (such as 1 time) into new user types, and dividing the number of visits to the platform Access users greater than the number of times threshold are classified as old user types, and the type classification criteria for new user types and old user types can be customized based on actual application scenarios, and are not specifically limited here.
然后基于第一用户访问数据以及第二用户访问数据确定新老用户波动数据,新老用户波动数据可以是新老用户波动比例参数,如新用户(增长/减少)波动比例、老用户(增长/减少)波动比例等。然后将新老用户波动数据作为访问用户分布信息;Then determine new and old user fluctuation data based on the first user access data and the second user access data, the new and old user fluctuation data can be new and old user fluctuation ratio parameters, such as new user (growth/decrease) fluctuation ratio, old user (growth/ reduction) fluctuation ratio, etc. Then use the fluctuation data of new and old users as the distribution information of visiting users;
其中,监测窗口时间可以理解为服务平台所设置的用于监测用户访问流量的窗口时间,通常针对流量平稳的情况,服务平台动态设置的是小时级别或者分钟级别的调整窗口时间,针对突发的剧增流量(如访问流量大于阈值),服务平台动态设置的调整窗口时间较小,例如调整窗口时间是以秒级别进行设置。Among them, the monitoring window time can be understood as the window time set by the service platform for monitoring user access traffic. Usually, when the traffic is stable, the service platform dynamically sets the adjustment window time at the hour or minute level. In the case of a sharp increase in traffic (for example, the access traffic is greater than the threshold), the adjustment window time dynamically set by the service platform is relatively small, for example, the adjustment window time is set at the second level.
S104:基于所述访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节,所述第一数据缓存空间为缓存所述热数据的数据存储空间,所述第二数据缓存空间为缓存所述冷数据的数据缓存空间。S104: Based on the access user distribution information, adjust the cache space of the first data cache space and/or the second data cache space, the first data cache space is a data storage space for caching the hot data, and the second data cache space is a data storage space for caching the hot data. The second data cache space is a data cache space for caching the cold data.
可以理解的,在本说明书一个或多个实施例中,可动态基于实际情况对缓存热数据的第一数据存储空间或对缓存冷数据的第二数据存储空间进行缓存调整,可提高服务平台所维护缓存空间的利用效率,缓解缓存负载状况,提高缓存处理的鲁棒性,通过调整缓存空间,进而可提高用户请求访问数据时的缓存命中率,降低对后端数据库的压力。It can be understood that, in one or more embodiments of this specification, the first data storage space for caching hot data or the second data storage space for caching cold data can be dynamically adjusted based on actual conditions, which can improve the performance of the service platform. Maintain the utilization efficiency of the cache space, alleviate the cache load, and improve the robustness of the cache processing. By adjusting the cache space, the cache hit rate when users request to access data can be improved, and the pressure on the back-end database can be reduced.
在本说明书一个或多个实施例中,服务平台通过监测针对热数据和冷数据的缓存波动状态,基于所述缓存波动状态来动态获取当前的访问用户分布信息,基于访问用户分布信息所预测的后续冷热缓存变化情况,可预先对第一数据缓存空间和/或第二数据缓存空间进行缓存空间动态调节;可以理解的,一方面,基于访问用户分布信息一定程度上可预测热数据或冷数据的缓存量的波动情况,在实际应用中根据访问用户分布信息可指示在老用户类型的用户访问激增的情况下:热数据缓存管理后续通常处理计算量更大以及更多的热数据需要进行存储、根据访问用户分布信息可指示在新用户类型的用户访问激增的情况下,冷数据缓存管理后续通常处理计算量更大,更多的冷数据需要进行存储。基于此,可基于访问用户分布信息来实现后续热数据或冷数据的需求量提前预测,进而可以提前调整相应数据的缓存空间。另一方面,采用对相应数据的数据缓存空间进行调整的方式,可避免在一些访问波动较大场景下避免对冷数据/热数据的频繁迁移缓存位置,以避免加剧缓存管理的时间复杂度以及缓存系统的负载,采用对应调整缓存空间的方式一定程度大幅降低了缓存管理的开销量。In one or more embodiments of this specification, the service platform dynamically obtains the current access user distribution information based on the cache fluctuation status for hot data and cold data by monitoring the cache fluctuation status, and based on the predicted access user distribution information Subsequent changes in hot and cold caches, the first data cache space and/or the second data cache space can be dynamically adjusted in advance; it can be understood that, on the one hand, hot data or cold data can be predicted to a certain extent based on access user distribution information The fluctuation of data cache volume, in actual application, according to the access user distribution information, can indicate that in the case of a surge of old user types of user access: hot data cache management usually requires more calculations and more hot data to be processed later. Storage, according to access user distribution information can indicate that in the case of a surge in user access of new user types, the follow-up processing of cold data cache management usually requires more calculations, and more cold data needs to be stored. Based on this, based on access user distribution information, the demand for subsequent hot data or cold data can be predicted in advance, and then the cache space of the corresponding data can be adjusted in advance. On the other hand, the method of adjusting the data cache space of the corresponding data can avoid frequent migration of cold data/hot data cache locations in some scenarios with large access fluctuations, so as to avoid aggravating the time complexity of cache management and For the load of the cache system, the way of adjusting the cache space correspondingly reduces the overhead of cache management to a certain extent.
在一种可行的实施方式中,服务平台基于访问用户分布信息进行分析处理,可确定第一数据缓存空间和/或第二数据缓存空间中待调整的目标调节缓存空间,然后对目标调节缓存空间进行缓存空间调整。例如,对访问用户分布信息进行分析处理,若访问用户分布信息指示后续热数据管理通常处理计算量大以及更多的热数据需要进行存储,则目标调节缓存空间为第一数据缓存空间;若访问用户分布信息指示后续冷数据管理通常处理计算量大以及更多的冷数据需要进行存储,则目标调节缓存空间为第二数据缓存空间;In a feasible implementation manner, the service platform performs analysis and processing based on access user distribution information, and can determine the target adjustment cache space to be adjusted in the first data cache space and/or the second data cache space, and then adjust the cache space for the target Make cache size adjustments. For example, analyze and process access user distribution information, if the access user distribution information indicates that follow-up hot data management usually has a large amount of calculation and more hot data needs to be stored, then the target adjustment cache space is the first data cache space; if access User distribution information indicates that follow-up cold data management usually deals with a large amount of calculation and more cold data needs to be stored, then the target adjustment cache space is the second data cache space;
可选的,基于访问用户分布信息进行分析处理可以是,根据访问用户分布信息确定后续冷数据缓存管理的需求大或冷数据缓存管理的需求大,进而确定第一数据缓存空间和第二数据缓存空间中待调整的目标调节缓存空间。Optionally, the analysis and processing based on the access user distribution information may be, according to the access user distribution information, it is determined that the demand for subsequent cold data cache management is large or the demand for cold data cache management is large, and then the first data cache space and the second data cache space are determined. Target adjustment cache space to be adjusted in space.
示意性的,根据访问用户分布信息可确定诸如新用户类型/老用户类型的用户分布占比或用户分布波动较大,在新用户类型的用户分布占比或用户分布波动较大的情况下,可确定后续冷数据缓存管理的需求大或冷数据缓存管理的需求大,然后增加用于存储冷数据的第二数据缓存空间;在老用户类型的用户分布占比或用户分布波动较大的情况下,可确定后续热数据缓存管理的需求大或热数据缓存管理的需求大,然后增加用于存储热数据的第一数据缓存空间。Schematically, according to the access user distribution information, it can be determined such as the user distribution proportion of new user type/old user type or the user distribution fluctuates greatly. It can be determined that the demand for subsequent cold data cache management is large or the demand for cold data cache management is large, and then increase the second data cache space for storing cold data; in the case where the proportion of old user types or user distribution fluctuates greatly In this case, it may be determined that the demand for subsequent hot data cache management is large or the demand for hot data cache management is large, and then the first data cache space for storing hot data is increased.
在一种可行的实施方式中,采用最近最少使用缓存机制对所述第一数据缓存空间进行缓存淘汰处理,和采用最不经常使用缓存机制对所述第二数据缓存空间进行缓存淘汰处理。可以理解的,基于监测到的冷热数据的缓存波动情况,对用于缓存热数据的第一数据缓存空间以及用于缓存冷数据的第二数据缓存空间采用不同的淘汰策略进行缓存淘汰处理,提高缓存命中率。In a feasible implementation manner, the least recently used cache mechanism is used to perform cache elimination processing on the first data cache space, and the least frequently used cache mechanism is used to perform cache elimination processing on the second data cache space. It can be understood that based on the monitored cache fluctuations of hot and cold data, different elimination strategies are used for cache elimination processing for the first data cache space for caching hot data and the second data cache space for caching cold data. Improve cache hit ratio.
在本说明书一个或多个实施例中,服务平台通过监测针对热数据和冷数据的缓存波动状态以获取当前的访问用户分布信息,基于访问用户分布信息对第一数据缓存空间和/或第二数据缓存空间进行缓存空间动态调节。基于访问用户分布信息一定程度上可预测冷热数据的缓存量的波动,可预先对相应缓存空间进行调节,提高了请求数据时缓存命中率,降低服务平台的处理压力;以及避免采用对缓存数据进行缓存空间迁移的方式,可以降低缓存管理的时间复杂度以及缓存系统的负载,以及节省缓存管理的开销量。In one or more embodiments of this specification, the service platform obtains the current access user distribution information by monitoring the cache fluctuation status for hot data and cold data, and based on the access user distribution information, caches the first data and/or the second The data cache space is dynamically adjusted for the cache space. Based on the access user distribution information, the fluctuation of the cache volume of hot and cold data can be predicted to a certain extent, and the corresponding cache space can be adjusted in advance, which improves the cache hit rate when requesting data, reduces the processing pressure of the service platform; and avoids the use of cached data The way of migrating the cache space can reduce the time complexity of cache management and the load of the cache system, and save the overhead of cache management.
请参见图3,图3是本说明书一个或多个实施例提出的一种缓存管理方法的另一种实施例的流程示意图。具体的:Please refer to FIG. 3 . FIG. 3 is a schematic flowchart of another embodiment of a cache management method proposed by one or more embodiments of this specification. specific:
S202:监测针对热数据和冷数据的缓存波动比例;S202: Monitor the cache fluctuation ratio for hot data and cold data;
所述缓存波动比例可以是监测周期内热数据缓存波动(增长/减少)比例、冷数据缓存(增长/减少)波动比例、热数据缓存与冷数据缓存的波动比例等类型数据中的一种或多种的拟合。The cache fluctuation ratio can be one or more of the types of data such as the fluctuation ratio of hot data cache (increase/decrease), the fluctuation ratio of cold data cache (increase/decrease), the fluctuation ratio of hot data cache and cold data cache, etc. kind of fitting.
可以理解的,服务平台可以实时或周期性监测冷数据和/或热数据的缓存波动量来确定缓存波动比例,在缓存波动比例比较大或者平稳的状态下,服务平台能够自动监测到,并且通过执行本说明书涉及的缓存管理方法进行处理。It can be understood that the service platform can monitor the cache fluctuation of cold data and/or hot data in real time or periodically to determine the cache fluctuation ratio. When the cache fluctuation ratio is relatively large or stable, the service platform can automatically monitor it and pass Execute the cache management method involved in this specification for processing.
S204:若所述缓存波动比例大于或等于波动比例阈值,则获取访问用户分布信息;S204: If the cache fluctuation ratio is greater than or equal to the fluctuation ratio threshold, obtain access user distribution information;
所述波动比例阈值可以理解为针对缓存波动比例所设置的门限值或临界值,不同类的缓存波动比例可以分别设置相应的波动比例阈值。波动比例阈值用于在缓存波动比例指示到达临界时,监测服务平台当前的访问用户分布状况以预测或评估冷热数据缓存变化趋势。The fluctuation ratio threshold can be understood as a threshold or critical value set for buffer fluctuation ratios, and corresponding fluctuation ratio thresholds can be set for different types of buffer fluctuation ratios. The fluctuation ratio threshold is used to monitor the current distribution of access users on the service platform to predict or evaluate the change trend of hot and cold data caches when the cache fluctuation ratio indication reaches a critical point.
所述访问用户分布信息可以是诸如若干访问用户类型的用户量、用户类型分布状况、新老用户波动数据等信息中的一种或多种的拟合。The access user distribution information may be a fitting of one or more of information such as the number of users of several types of access users, the distribution status of user types, and fluctuation data of new and old users.
在本说明书一个或多个实施例中,所述访问用户分布信息可以为新老用户波动数据,例如新老用户波动比例、新老用户波动数量等等,服务平台可以(在监测窗口时间内)获取针对新用户类型的第一用户访问数据,以及针对老用户类型的第二用户访问数据,然后基于第一用户访问数据以及第二用户访问数据确定新老用户波动数据,该新老用户波动数据也即访问用户分布信息,其可以反馈在新用户类型和老用户类型上的用户访问量的分布变化。In one or more embodiments of this specification, the access user distribution information may be new and old user fluctuation data, such as the fluctuation ratio of new and old users, the fluctuation number of new and old users, etc., and the service platform may (within the monitoring window time) Obtain first user access data for new user types and second user access data for old user types, and then determine new and old user fluctuation data based on the first user access data and the second user access data, the new and old user fluctuation data That is, access user distribution information, which can feed back changes in the distribution of user visits on new user types and old user types.
可选的,若所述缓存波动比例大于或等于波动比例阈值,则服务平台可以忽略处理。Optionally, if the cache fluctuation ratio is greater than or equal to the fluctuation ratio threshold, the service platform may ignore the processing.
S206:基于所述访问用户分布信息,确定目标缓存调节方式;S206: Determine a target cache adjustment method based on the access user distribution information;
所述目标缓存调节方式基于实际应用情况至少包括缓存动态调节方式以及缓存周期调节方式。The target cache adjustment method includes at least a cache dynamic adjustment method and a cache period adjustment method based on actual application conditions.
在本说明书一个或多个实施例中,缓存动态调节方式通常适用于波动剧烈状态下的对缓存空间进行调整的方式,在缓存动态调节方式下其在监测窗口时间内实时基于诸如新老用户波动数据等访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间进行缓存空间动态调节。In one or more embodiments of this specification, the cache dynamic adjustment method is generally suitable for adjusting the cache space in a highly fluctuating state. In the cache dynamic adjustment mode, it is based on real-time changes in the monitoring window such as new and old user fluctuations Data and other access user distribution information, and dynamically adjust the cache space of the first data cache space and/or the second data cache space.
在本说明书一个或多个实施例中,缓存周期调节方式通常适用于波动平稳状态下的对缓存空间进行调整的方式。在缓存周期调节方式基于一定周期时间触发在固定周期时刻基于诸如新老用户波动数据等访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间的缓存空间调节。In one or more embodiments of this specification, the caching period adjustment method is generally applicable to the caching space adjustment method in a fluctuating and stable state. In the cache cycle adjustment mode, based on a certain cycle time, the cache space adjustment of the first data cache space and/or the second data cache space is triggered at a fixed cycle time based on access user distribution information such as new and old user fluctuation data.
在一种可行的实施方式中,以所述访问用户分布信息为新老用户波动数据为例,服务平台执行基于所述访问用户分布信息,确定目标缓存调节方式,具体可以是:In a feasible implementation manner, taking the access user distribution information as the fluctuation data of new and old users as an example, the service platform determines the target cache adjustment method based on the access user distribution information, which may specifically be:
服务平台可以基于新老用户波动数据来确定针对所述服务平台的用户流量波动状态;所述用户流量波动状态可以基于实际应用情况至少设置波动剧烈状态和波动平稳状态。The service platform can determine the user traffic fluctuation state for the service platform based on the fluctuation data of new and old users; the user traffic fluctuation state can be set at least as a severe fluctuation state and a stable fluctuation state based on actual application conditions.
示意性的,可以针对新老用户波动数据设置波动阈值数据,在新老用户波动数据满足波动阈值数据时,则认为用户流量波动状态为波动平稳状态;在新老用户波动数据不满足波动阈值数据时,则认为用户流量波动状态为波动剧烈状态。Schematically, the fluctuation threshold data can be set for the fluctuation data of new and old users. When the fluctuation data of new and old users meet the fluctuation threshold data, the user flow fluctuation state is considered to be a stable fluctuation state; when the fluctuation data of new and old users does not meet the fluctuation threshold data , the user traffic fluctuation state is considered to be a state of severe fluctuation.
例如,新老用户波动数据可以是新老用户波动比例参数,如新用户(增长/减少)波动比例、老用户(增长/减少)波动比例等。波动阈值数据可以是针对新用户(增长/减少)波动比例设置的新用户(增长/减少)波动阈值,波动阈值数据可以是针对老用户(增长/减少)波动比例设置的老用户(增长/减少)波动阈值。For example, the fluctuation data of new and old users may be a fluctuation ratio parameter of new and old users, such as the fluctuation ratio of new users (increase/decrease), the fluctuation ratio of old users (increase/decrease), and the like. The fluctuation threshold data can be the new user (growth/decrease) fluctuation threshold set for the new user (growth/decrease) fluctuation ratio, and the fluctuation threshold data can be the old user (growth/decrease) set for the old user (growth/decrease) fluctuation ratio ) fluctuation threshold.
若用户流量波动状态为波动剧烈状态,则服务平台确定目标缓存调节方式为缓存动态调节方式;If the user traffic fluctuation state is a state of severe fluctuation, the service platform determines that the target cache adjustment method is a cache dynamic adjustment method;
若用户流量波动状态为波动平稳状态,则服务平台确定目标缓存调节方式为缓存周期调节方式。If the user traffic fluctuation state is a stable fluctuation state, the service platform determines that the target cache adjustment mode is the cache cycle adjustment mode.
在一种可行的实施方式中,所述新老用户波动数据包括用户增长量以及新老用户比重;用户增长量可以理解为在监测窗口时间内新用户类型或老用户类型的增长量;新老用户比重可以理解为新用户类型或老用户类型占所有用户数量的比值数据。服务平台可以结合用户增长量以及新老用户比重联合进行用户流量波动状态的判决。In a feasible implementation, the new and old user fluctuation data includes user growth and the proportion of new and old users; user growth can be understood as the growth of new user types or old user types within the monitoring window time; new and old user types User proportion can be understood as the ratio data of new user types or old user types to all users. The service platform can combine the user growth and the proportion of new and old users to jointly judge the user traffic fluctuation status.
示意性的,服务平台可以比较用户增长量与增量阈值的大小,以及新老用户比重与用户比重阈值的大小。增量阈值为针对用户增长量设置的门限值,用户比重阈值为针对新老用户比重设置的门限值。Schematically, the service platform can compare the amount of user growth with the incremental threshold, and the proportion of new and old users with the threshold of user proportion. The increment threshold is the threshold value set for user growth, and the user proportion threshold is the threshold value set for the proportion of new and old users.
若所述用户增长量大于增量阈值,且所述新老用户比重大于用户比重阈值,则确定针对所述服务平台的用户流量波动状态为所述波动剧烈状态。反之,则确定针对所述服务平台的用户流量波动状态为波动平稳状态。If the user growth amount is greater than the increment threshold, and the proportion of new and old users is greater than the user proportion threshold, it is determined that the fluctuation state of user traffic for the service platform is the severe fluctuation state. On the contrary, it is determined that the fluctuation state of the user traffic of the service platform is a fluctuation steady state.
S208:采用所述目标缓存调节方式对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节。S208: Adjust the cache space of the first data cache space and/or the second data cache space by using the target cache adjustment manner.
在一种可行的实施方式中,以目标缓存调节方式为缓存周期调节方式为例,通常缓存周期调节方式适用于波动平稳状态下的对缓存空间进行调整的方式。在缓存周期调节方式下基于一定周期时间触发在固定周期时刻基于诸如新老用户波动数据等访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间的缓存空间进行空间微调。进一步的,在处于波动平稳状态下,第一数据缓存空间以及第二数据缓存空间通常高概率可以满足当前冷热数据缓存需求,服务平台仅需结合访问用户分布信息或当前冷热数据波动参数(如冷热数据波动幅度、冷热数据波动量)来适应性对第一数据缓存空间和第二数据缓存空间进行微调,所述微调的空间调整幅度小于设置的微调阈值。In a feasible implementation manner, taking the target cache adjustment mode as an example of a cache period adjustment mode, usually the cache period adjustment mode is suitable for adjusting the cache space in a fluctuating and stable state. In the caching cycle adjustment mode, based on a certain cycle time, the space fine-tuning of the first data cache space and/or the cache space of the second data cache space is carried out based on access user distribution information such as new and old user fluctuation data at a fixed cycle time. Furthermore, in the steady state of fluctuations, the first data cache space and the second data cache space can usually meet the current hot and cold data cache requirements with a high probability, and the service platform only needs to combine access user distribution information or current hot and cold data fluctuation parameters ( Adaptively fine-tune the first data cache space and the second data cache space, such as cold and hot data fluctuation range, cold and hot data fluctuation amount), and the fine-tuned space adjustment range is smaller than the set fine-tuning threshold.
示意性的,在缓存周期调节方式下,服务平台通常微调的空间调整幅度小于缓存动态调节方式的空间调整幅度。Schematically, in the cache period adjustment mode, the space adjustment range of fine-tuning by the service platform is usually smaller than the space adjustment range of the cache dynamic adjustment mode.
在一种可行的实施方式中,以目标缓存调节方式为缓存动态调节方式为例,缓存动态调节方式通常适用于波动剧烈状态下的对缓存空间进行调整的方式,服务平台执行所述采用所述目标缓存调节方式对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节,可以是:In a feasible implementation manner, taking the target cache adjustment mode as an example of a cache dynamic adjustment mode, the cache dynamic adjustment mode is usually suitable for adjusting the cache space in a violently fluctuating state, and the service platform executes the above-mentioned The target cache adjustment mode adjusts the cache space of the first data cache space and/or the second data cache space, which may be:
1、服务平台可以采用所述缓存动态调节方式确定滑动窗口调节区域以及目标调节缓存空间;1. The service platform can use the cache dynamic adjustment method to determine the sliding window adjustment area and the target adjustment cache space;
所述滑动窗口调节区域为向确定需要调节的缓存空间(第一数据缓存空间和第二数据缓存空间中的其一)所分配的缓存区域,可以理解为将滑动窗口调节区域分配给需要调节的缓存空间。The sliding window adjustment area is a buffer area allocated to the buffer space (one of the first data buffer space and the second data buffer space) that needs to be adjusted, which can be understood as assigning the sliding window adjustment area to the buffer space that needs to be adjusted. cache space.
所述目标调节缓存空间可以理解为前述需要调节的缓存空间,目调节缓存空间为第一数据缓存空间和第二数据缓存空间中的其一,如冷数据缓存需求相对于热数据缓存需求较高,则目标调节缓存空间通常为第二数据缓存空间,也即将确定的滑动窗口调节区域分配给第二数据缓存空间;又如热数据缓存需求相对于冷数据缓存需求较高,则目标调节缓存空间通常为第一数据缓存空间,也即将确定的滑动窗口调节区域分配给第一数据缓存空间。The target adjustment cache space can be understood as the aforementioned cache space that needs to be adjusted. The target adjustment cache space is one of the first data cache space and the second data cache space. For example, the demand for cold data cache is higher than that for hot data cache. , then the target adjustment cache space is usually the second data cache space, that is, the determined sliding window adjustment area is allocated to the second data cache space; and if the demand for hot data cache is higher than that for cold data cache, then the target adjustment cache space Usually it is the first data cache space, that is, the determined sliding window adjustment area is allocated to the first data cache space.
可选的,可以设置每次缓存动态调节方式的缓存调节比例,在缓存动态调节方式下根据该缓存调节比例确定滑动窗口调节区域,如缓存调节比例可以是10%,则从可以调配的缓存空间(如闲置缓存空间)中确定10%缓存空间作为滑动窗口调节区域,以类似滑动窗口的形式将滑动窗口调节区域分配至目标调节缓存空间,实际应用中,通常将滑动窗口调节区域与目标调节缓存空间进行关联。Optionally, you can set the cache adjustment ratio of each cache dynamic adjustment mode, and determine the sliding window adjustment area according to the cache adjustment ratio in the cache dynamic adjustment mode. For example, the cache adjustment ratio can be 10%, then the buffer space that can be allocated (such as idle cache space), determine 10% of the buffer space as the sliding window adjustment area, and allocate the sliding window adjustment area to the target adjustment buffer space in a form similar to the sliding window. In practical applications, the sliding window adjustment area and the target adjustment buffer are usually Space is associated.
示意性的,可以设置基准缓存调节比例,在每次缓存动态调节时,按照基准缓存调节比例进行调整。Schematically, a base cache adjustment ratio can be set, and each time the cache is dynamically adjusted, the adjustment is performed according to the base cache adjustment ratio.
示意性的,缓存动态调节方式通常会涉及到监测过程中的多轮调整,在首轮按照基准缓存调节比例确定滑动窗口调节区域并分配至目标调节缓存空间之后,可以监测新老用户波动数据或冷热数据波动参数,来衡量波动剧烈状态的程度,如可以基于新老用户波动数据或冷热数据波动参数评估波动因子,基于该波动因子对基准缓存调节比例进行调节,基于调节之后的缓存调节比例在下一轮缓存空间调节过程中,确定下一轮的滑动窗口调节区域,继而对下一轮的目标调节缓存空间进行调节。Schematically, the cache dynamic adjustment method usually involves multiple rounds of adjustments in the monitoring process. After the first round of adjusting the sliding window adjustment area is determined according to the benchmark cache adjustment ratio and allocated to the target adjustment cache space, the fluctuation data of new and old users or Hot and cold data fluctuation parameters to measure the degree of severe fluctuations. For example, the fluctuation factor can be evaluated based on the fluctuation data of new and old users or the fluctuation parameters of cold and hot data. During the next round of buffer space adjustment, the ratio determines the next round of sliding window adjustment area, and then adjusts the next round of target adjustment buffer space.
在一种可行的实施方式中,缓存调节比例可以是动态的,服务平台可以基于实时监测到的访问用户分布信息来动态确定缓存调节比例,基于所述缓存调节比例确定滑动窗口调节区域,如基于所述缓存调节比例从可以调配的缓存空间(如闲置缓存空间)中确定滑动窗口调节区域。In a feasible implementation, the cache adjustment ratio can be dynamic, and the service platform can dynamically determine the cache adjustment ratio based on the real-time monitored access user distribution information, and determine the sliding window adjustment area based on the cache adjustment ratio, such as based on The cache adjustment ratio determines the sliding window adjustment area from the allocated cache space (such as idle cache space).
示意性的,服务平台可以实时监测用户访问分布信息中的至少一个监测参数,如监测参数可以是新老用户波动比例、新老用户波动数量、若干访问用户类型的用户量、用户类型分布状况参数等一种或多种的拟合,可以通过对每类监测参数进行加权,基于加权的评估值来确定缓存调节比例,如直接将评估值作为缓存调节比例。Schematically, the service platform can monitor at least one monitoring parameter in the user access distribution information in real time, for example, the monitoring parameter can be the fluctuation ratio of new and old users, the number of new and old users fluctuations, the number of users of several access user types, and the distribution status parameters of user types For one or more fittings, each type of monitoring parameter can be weighted, and the cache adjustment ratio can be determined based on the weighted evaluation value, for example, the evaluation value can be directly used as the cache adjustment ratio.
示意性的,可设置若干个参考评估范围,每个参考评估范围对应一个参考缓存调节比例,通过确定评估值所落入的参考评估范围,获取所落入参考评估范围所对应的参考缓存调节比例作为前述缓存调节比例。Schematically, several reference evaluation ranges can be set, and each reference evaluation range corresponds to a reference buffer adjustment ratio. By determining the reference evaluation range where the evaluation value falls into, the reference buffer adjustment ratio corresponding to the reference evaluation range that falls into can be obtained As the aforementioned cache adjustment ratio.
2、基于所述滑动窗口调节区域对目标调节缓存空间进行缓存空间调节,所述目标调节缓存空间为所述第一数据缓存空间和第二数据缓存空间中的至少其一。2. Perform buffer space adjustment on a target adjustment buffer space based on the sliding window adjustment area, where the target adjustment buffer space is at least one of the first data buffer space and the second data buffer space.
具体的,服务平台在从可以调配的缓存空间中确定滑动窗口调节区域之后,然后将滑动窗口调节区域与目标调节缓存空间进行关联,也即将滑动窗口调节区域分配给目标调节缓存空间。Specifically, after the service platform determines the sliding window adjustment area from the buffer space that can be allocated, it then associates the sliding window adjustment area with the target adjustment buffer space, that is, assigns the sliding window adjustment area to the target adjustment buffer space.
在一种具体的实施场景中,服务平台的缓存总空间可以由第一数据缓存空间和第二数据缓存空间组成,也即服务平台预先将缓存总空间分为缓存热数据的第一数据缓存空间以及第二数据缓存空间。在缓存动态调节方式下,采用类似滑动窗口的形式,比如决策出当前需要更多第一数据缓存空间也即将第一数据缓存空间作为目标调节缓存空间,则可以从第二数据缓存空间中选取滑动窗口调节区域划分目标调节缓存空间中,整个缓存调节过程可以避免加剧缓存管理的时间复杂度以及缓存系统的负载,采用对应调整缓存空间的方式一定程度大幅降低了缓存管理的开销量,提升了缓存空间的利用效率。In a specific implementation scenario, the total cache space of the service platform can be composed of the first data cache space and the second data cache space, that is, the service platform divides the total cache space into the first data cache space for caching hot data in advance and a second data cache space. In the cache dynamic adjustment mode, a form similar to a sliding window is adopted. For example, if it is determined that more first data cache space is currently needed and the first data cache space is used as the target to adjust the cache space, then the sliding window can be selected from the second data cache space. In the window adjustment area division target adjustment cache space, the entire cache adjustment process can avoid increasing the time complexity of cache management and the load of the cache system. The corresponding adjustment of the cache space can greatly reduce the overhead of cache management to a certain extent and improve the cache. Space utilization efficiency.
进一步的,服务平台执行所述采用所述缓存动态调节方式确定滑动窗口调节区域以及目标调节缓存空间,基于所述滑动窗口调节区域对目标调节缓存空间进行缓存空间调节,可以是:Further, the service platform executes the method of using the buffer dynamic adjustment method to determine the sliding window adjustment area and the target adjustment buffer space, and adjusts the buffer space for the target adjustment buffer space based on the sliding window adjustment area, which may be:
A2:服务平台可以获取新老用户波动数据和/或冷热数据波动参数,基于所述新老用户波动数据和/或所述冷热数据波动参数,从第一数据缓存空间和第二数据缓存空间中确定目标调节缓存空间;A2: The service platform can obtain new and old user fluctuation data and/or cold and hot data fluctuation parameters, based on the new and old user fluctuation data and/or the cold and hot data fluctuation parameters, from the first data cache space and the second data cache Determine the target in the space to adjust the cache space;
可以理解的,根据新老用户波动数据和/或冷热数据波动参数可以确定后续冷热数据缓存管理的需求,进而基于冷热数据缓存管理的需求从第一数据缓存空间和第二数据缓存空间中待调整的目标调节缓存空间。如:根据访问用户分布信息可确定诸如新用户类型/老用户类型的用户分布占比或用户分布波动较大,示意性的在新用户类型的用户分布占比或用户分布波动较大的情况下,可确定后续冷数据缓存管理的需求大或冷数据缓存管理的需求大,然后增加用于存储冷数据的第二数据缓存空间;在老用户类型的用户分布占比或用户分布波动较大的情况下,可确定后续热数据缓存管理的需求大或热数据缓存管理的需求大,然后增加用于存储热数据的第一数据缓存空间。It can be understood that according to the fluctuation data of new and old users and/or the fluctuation parameters of cold and hot data, the requirements for subsequent cold and hot data cache management can be determined, and then the first data cache space and the second data cache space can be selected based on the requirements for cold and hot data cache management. The target to be adjusted in the adjustment cache space. For example, according to the access user distribution information, it can be determined such as the proportion of new user types/old user types or the user distribution fluctuates greatly. Schematically, in the case of the user distribution proportion of new user types or the user distribution fluctuates greatly , it can be determined that the demand for follow-up cold data cache management is large or the demand for cold data cache management is large, and then increase the second data cache space for storing cold data; in the case where the proportion of old user types or user distribution fluctuates greatly In some cases, it may be determined that the demand for subsequent hot data cache management is large or the demand for hot data cache management is large, and then the first data cache space for storing hot data is increased.
A4:服务平台可以确定针对第三数据缓存空间的第一缓存调节比例;所述第三数据缓存空间为所述第一数据缓存空间和所述第二数据缓存空间中除所述目标调节缓存空间之外的数据缓存空间。A4: The service platform can determine the first cache adjustment ratio for the third data cache space; the third data cache space is the first data cache space and the second data cache space except the target adjustment cache space Data cache space outside.
可以理解的,第三数据缓存空间作为前述可调配的缓存空间,如缓存调节比例可以是10%,则从可以调配的第三数据缓存空间中确定10%缓存空间作为滑动窗口调节区域。It can be understood that the third data cache space is used as the previously configurable cache space, and if the cache adjustment ratio can be 10%, then 10% of the cache space can be determined from the configurable third data cache space as the sliding window adjustment area.
可选的,第一缓存调节比例可以是设置的基准缓存调节比例,在每次缓存动态调节时,按照基准缓存调节比例进行调整。Optionally, the first cache adjustment ratio may be a set reference cache adjustment ratio, which is adjusted according to the reference cache adjustment ratio each time the cache is dynamically adjusted.
可选的,第一缓存调节比例可以是动态的,服务平台可以基于实时监测到的访问用户分布信息来动态确定第一缓存调节比例,基于第一缓存调节比例确定滑动窗口调节区域,如基于第一缓存调节比例从可以调配的第三数据缓存空间中确定滑动窗口调节区域。具体动态确定第一缓存调节比例可参考前述动态确定缓存调节比例的释义,此处不再赘述。Optionally, the first cache adjustment ratio may be dynamic. The service platform may dynamically determine the first cache adjustment ratio based on the real-time monitored access user distribution information, and determine the sliding window adjustment area based on the first cache adjustment ratio, such as based on the first cache adjustment ratio. A cache adjustment ratio determines the sliding window adjustment area from the allocated third data cache space. For specific dynamic determination of the first cache adjustment ratio, reference may be made to the aforementioned definition of dynamically determining the cache adjustment ratio, which will not be repeated here.
A6:服务平台在确定第一缓存调节比例之后,可以基于第一缓存调节比例从第三数据缓存空间中选取第一滑动窗口调节区域,将第一滑动窗口调节区域与所述目标调节缓存空间相关联,A6: After determining the first cache adjustment ratio, the service platform can select the first sliding window adjustment area from the third data cache space based on the first cache adjustment ratio, and correlate the first sliding window adjustment area with the target adjustment cache space couplet,
可以理解的,第一滑动窗口调节区域通常不为已经缓存冷/热数据的缓存区域。It can be understood that the first sliding window adjustment area is generally not a cache area where cold/hot data has been cached.
在本说明书一个或多个实施例中,服务平台的总缓存空间可以针对第一缓存空间以及第二缓存空间设置一个相对于总缓存空间的下限比例,第一缓存空间以及第二缓存空间的空间比例需要大于或等于该下限比例。In one or more embodiments of this specification, the total cache space of the service platform can set a lower limit ratio relative to the total cache space for the first cache space and the second cache space, and the space of the first cache space and the second cache space The scale needs to be greater than or equal to this lower bound scale.
可选的,针对用于缓存冷热数据的缓存空间而言,通常各类型缓存空间采用同种缓存淘汰机制进行缓存空间中冷/热数据的更新,但在波动剧烈状态下,采取前述方式难以适配当下场景,基于此,基于当前的目标缓存调节方式,对第一数据缓存空间和/或第二数据缓存空间进行缓存淘汰机制调整。Optionally, for the cache space used to cache hot and cold data, usually all types of cache space use the same cache elimination mechanism to update the cold/hot data in the cache space, but in a state of severe fluctuations, it is difficult to adopt the aforementioned method Adapt to the current scenario, based on this, adjust the cache elimination mechanism for the first data cache space and/or the second data cache space based on the current target cache adjustment method.
示意性的,通常各类型缓存空间采用一种缓存淘汰机制以便缓存管理,在本说明书一个或多个实施例中,若所述目标缓存调节方式为缓存动态调节方式,可对第一数据缓存空间和第二数据缓存空间采用不同的缓存淘汰机制。若目标缓存调节方式为缓存周期调节方式,可不启用不同的缓存淘汰机制进行缓存淘汰管理,可以原有缓存淘汰机制进行控制管理。Schematically, each type of cache space usually adopts a cache elimination mechanism for cache management. In one or more embodiments of this specification, if the target cache adjustment method is a cache dynamic adjustment method, the first data cache space can be It adopts a different cache elimination mechanism from the second data cache space. If the target cache adjustment mode is the cache cycle adjustment mode, different cache elimination mechanisms may not be enabled for cache elimination management, and the original cache elimination mechanism may be used for control and management.
在本说明书一个或多个实施例中,考虑到新老用户类型对后续冷热数据缓存需求的影响,若所述目标缓存调节方式为缓存动态调节方式,可以采用最近最少使用缓存机制(LRU机制)对所述第一数据缓存空间进行缓存淘汰处理,和采用最不经常使用缓存机制(LFU机制)对所述第二数据缓存空间进行缓存淘汰处理,这样可结合冷热数据缓存需求进行后续冷热数据的高效更新。In one or more embodiments of this specification, considering the impact of new and old user types on subsequent cold and hot data cache requirements, if the target cache adjustment method is a cache dynamic adjustment method, the least recently used cache mechanism (LRU mechanism) can be used ) performs cache elimination processing on the first data cache space, and performs cache elimination processing on the second data cache space by using a least frequently used cache mechanism (LFU mechanism), so that subsequent cooling can be performed in combination with cold and hot data cache requirements. Efficient update of hot data.
在一种具体的实施场景中,服务平台的缓存总空间可以由第一数据缓存空间、第二数据缓存空间和预留缓存空间组成,通过设置预留缓存空间可以实时基于冷热数据缓存需求动态调整区域数据缓存类型以及更新相应数据缓存淘汰策略。In a specific implementation scenario, the total cache space of the service platform can be composed of the first data cache space, the second data cache space, and the reserved cache space. By setting the reserved cache space, it can be dynamically updated in real time based on the hot and cold data cache requirements. Adjust the type of regional data cache and update the corresponding data cache elimination strategy.
在缓存动态调节方式下,采用类似滑动窗口的形式从预留缓存空间中选取全部或部分作为滑动窗口调节区域,比如决策出当前需要更多第一数据缓存空间也即将第一数据缓存空间作为目标调节缓存空间,则可以从预留缓存空间中选取滑动窗口调节区域划分目标调节缓存空间中,整个缓存调节过程可以避免加剧缓存管理的时间复杂度以及缓存系统的负载,采用对应调整缓存空间的方式一定程度大幅降低了缓存管理的开销量,提升了缓存空间的利用效率。In the cache dynamic adjustment mode, select all or part of the reserved cache space as the sliding window adjustment area in a form similar to the sliding window. For example, it is determined that more first data cache space is currently needed, and the first data cache space is set as the target. To adjust the cache space, you can select the sliding window adjustment area from the reserved cache space to divide the target adjustment cache space. The entire cache adjustment process can avoid increasing the time complexity of cache management and the load of the cache system, and adopt a corresponding way to adjust the cache space To a certain extent, the overhead of cache management is greatly reduced, and the utilization efficiency of cache space is improved.
进一步的,服务平台执行所述采用所述缓存动态调节方式确定滑动窗口调节区域以及目标调节缓存空间,基于所述滑动窗口调节区域对目标调节缓存空间进行缓存空间调节,可以是:Further, the service platform executes the method of using the buffer dynamic adjustment method to determine the sliding window adjustment area and the target adjustment buffer space, and adjusts the buffer space for the target adjustment buffer space based on the sliding window adjustment area, which may be:
B2:服务平台可以获取新老用户波动数据和/或冷热数据波动参数,基于所述新老用户波动数据和/或所述冷热数据波动参数,从第一数据缓存空间和第二数据缓存空间中确定目标调节缓存空间;B2: The service platform can obtain new and old user fluctuation data and/or cold and hot data fluctuation parameters, based on the new and old user fluctuation data and/or the cold and hot data fluctuation parameters, from the first data cache space and the second data cache Determine the target in the space to adjust the cache space;
B4:获取预留缓存空间,确定针对所述预留缓存空间的第二缓存调节比例;所述预留缓存空间为除所述第一数据缓存空间和第二数据缓存空间之外的数据缓存空间;B4: Obtain the reserved cache space, and determine the second cache adjustment ratio for the reserved cache space; the reserved cache space is the data cache space other than the first data cache space and the second data cache space ;
可以理解的,预留缓存空间作为前述可调配的缓存空间,如缓存调节比例可以是10%,则从可以调配的预留缓存空间中确定10%缓存空间作为滑动窗口调节区域。It can be understood that the reserved buffer space is used as the previously configurable buffer space. For example, the buffer adjustment ratio may be 10%, and then 10% of the buffer space that can be allocated is determined as the sliding window adjustment area.
可选的,第二缓存调节比例可以是设置的基准缓存调节比例,在每次缓存动态调节时,按照基准缓存调节比例进行调整。Optionally, the second cache adjustment ratio may be a set reference cache adjustment ratio, which is adjusted according to the reference cache adjustment ratio each time the cache is dynamically adjusted.
可选的,第二缓存调节比例可以是动态的,服务平台可以基于实时监测到的访问用户分布信息来动态确定第二缓存调节比例,基于第二缓存调节比例确定滑动窗口调节区域,如基于第二缓存调节比例从可以调配的预留缓存空间中确定滑动窗口调节区域。具体动态确定第二缓存调节比例可参考前述动态确定缓存调节比例的释义,此处不再赘述。Optionally, the second cache adjustment ratio may be dynamic, and the service platform may dynamically determine the second cache adjustment ratio based on the real-time monitored access user distribution information, and determine the sliding window adjustment area based on the second cache adjustment ratio, such as based on the first The second cache adjustment ratio determines the sliding window adjustment area from the reserved buffer space that can be allocated. For specific dynamic determination of the second cache adjustment ratio, reference may be made to the definition of the aforementioned dynamic determination of the cache adjustment ratio, which will not be repeated here.
可选的,服务平台可持续进行监测,在监测到服务平台的用户流量波动状态从波动剧烈状态更新为波动平稳状态时,服务平台可以释放前述所确定的滑动窗口调节区域,也即释放滑动窗口调节区域重新设置为预留缓存空间对应的缓存区域。Optionally, the service platform can continuously monitor, and when it detects that the user traffic fluctuation state of the service platform is updated from a state of severe fluctuation to a state of stable fluctuation, the service platform can release the previously determined sliding window adjustment area, that is, release the sliding window The adjustment area is reset to the cache area corresponding to the reserved cache space.
B6:基于所述第二缓存调节比例从所述预留缓存空间中选取第二滑动窗口调节区域,将所述第二滑动窗口调节区域与目标调节缓存空间相关联。B6: Selecting a second sliding window adjustment area from the reserved buffer space based on the second buffer adjustment ratio, and associating the second sliding window adjustment area with a target adjustment buffer space.
可以理解的,第二滑动窗口调节区域通常不为已经缓存冷/热数据的缓存区域。It can be understood that the second sliding window adjustment area is generally not a cache area where cold/hot data has been cached.
在本说明书一个或多个实施例中,服务平台通过监测针对热数据和冷数据的缓存波动状态以获取当前的访问用户分布信息,基于访问用户分布信息对第一数据缓存空间和/或第二数据缓存空间进行缓存空间动态调节。基于访问用户分布信息一定程度上可预测冷热数据的缓存量的波动,可预先对相应缓存空间进行调节,提高了请求数据时缓存命中率,降低服务平台的处理压力;以及避免采用对缓存数据进行缓存空间迁移的方式,可以降低缓存管理的时间复杂度以及缓存系统的负载,以及节省缓存管理的开销量;以及,在波动剧烈状态下,平台能够自动精准监测,并且决策出合适的缓存淘汰机制进行缓存管理。In one or more embodiments of this specification, the service platform obtains the current access user distribution information by monitoring the cache fluctuation status for hot data and cold data, and based on the access user distribution information, caches the first data and/or the second The data cache space is dynamically adjusted for the cache space. Based on the access user distribution information, the fluctuation of the cache volume of hot and cold data can be predicted to a certain extent, and the corresponding cache space can be adjusted in advance, which improves the cache hit rate when requesting data, reduces the processing pressure of the service platform; and avoids the use of cached data The way to migrate the cache space can reduce the time complexity of cache management and the load of the cache system, and save the overhead of cache management; and, in the case of severe fluctuations, the platform can automatically and accurately monitor and make decisions on appropriate cache elimination mechanism for cache management.
请参见图3,图3是本说明书一个或多个实施例提出的一种缓存管理方法的另一种实施例的流程示意图。具体的:Please refer to FIG. 3 . FIG. 3 is a schematic flowchart of another embodiment of a cache management method proposed by one or more embodiments of this specification. specific:
S302:监测针对热数据和冷数据的缓存波动状态,基于所述缓存波动状态获取访问用户分布信息;S302: Monitor the cache fluctuation state for hot data and cold data, and obtain access user distribution information based on the cache fluctuation state;
S304:基于所述访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节;S304: Adjust the cache space of the first data cache space and/or the second data cache space based on the access user distribution information;
S306:获取针对第一数据缓存空间和/或第二数据缓存空间的待存储的第一缓存数据;S306: Obtain first cache data to be stored for the first data cache space and/or the second data cache space;
所述第一缓存数据可以理解为服务平台待存储的缓存数据,可以是待存储至第一数据缓存空间的热数据类型,也可以是待存储至第二数据缓存空间的冷数据类型。The first cache data can be understood as the cache data to be stored by the service platform, which may be a hot data type to be stored in the first data cache space, or a cold data type to be stored in the second data cache space.
示意性的,服务平台通常采用缓存机制进行数据访问的优化,服务平台可维护用于存储服务数据的缓存空间以及数据库,在客户端发起用户请求以请求相应数据时,服务平台检查客户端对应用户请求的请求数据,服务平台首先在维护的缓存空间(第一缓存空间、第二缓存空间)中查询是否存在该请求数据,若请求数据在缓存空间中查询不到,服务平台查询数据库,返回请求数据,可以理解的,服务平台可将该请求数据在缓存空间(第一缓存空间、第二缓存空间)中进行存储,进一步的,待存储的请求数据也即待存储至缓存空间(第一缓存空间、第二缓存空间)的第一缓存数据。Schematically, the service platform usually uses a caching mechanism to optimize data access. The service platform can maintain the cache space and database for storing service data. When the client initiates a user request to request the corresponding data, the service platform checks the corresponding user of the client. For the requested data, the service platform first checks whether the requested data exists in the maintained cache space (the first cache space and the second cache space). If the requested data cannot be found in the cache space, the service platform queries the database and returns the request Data, it can be understood that the service platform can store the requested data in the cache space (the first cache space, the second cache space), and further, the request data to be stored is also to be stored in the cache space (the first cache space) space, the second cache space) of the first cache data.
示意性的,服务平台每当服务数据发生变化的时候,可以同步更新缓存空间的缓存信息数据,以确保用户不会在缓存空间取到旧的缓存信息数据,进一步的,前述待同步更新的缓存信息数据也即待存储至缓存空间(第一缓存空间、第二缓存空间)对旧的缓存信息数据进行同步更新的第一缓存数据。Schematically, whenever the service data changes, the service platform can synchronously update the cache information data in the cache space to ensure that the user will not get the old cache information data in the cache space. Further, the aforementioned cache to be updated synchronously The information data is also the first cache data to be stored in the cache space (the first cache space, the second cache space) to synchronously update the old cache information data.
S308:对所述第一缓存数据进行目标编码处理,得到第二缓存编码数据;所述第二缓存编码数据为不含键值对中关键字元素数据的缓存编码数据。S308: Perform target encoding processing on the first cached data to obtain second cached encoded data; the second cached encoded data is cached encoded data without key element data in the key-value pair.
在相关技术中,待存储的第一缓存信息在存入缓存空间时通常以键值对(key-value)等缓存编码方式进行编码存储,key是关键字,value是值,Key-value是一种以键值对存储数据的数据编码存储形式;In related technologies, when the first cached information to be stored is stored in the cache space, it is usually coded and stored in a key-value pair (key-value) or other cache coding method, where key is a keyword, value is a value, and Key-value is a A data encoding storage form that stores data in key-value pairs;
示意性的,第一缓存数据为用户信息为例,存入缓存空间时会存“用户索引+用户信息”,而用户信息通常以key-value编码,比如常见的json格式,用户索引为用户的id,用户信息存取{"姓名":"张三","岁数":20}。Schematically, the first cache data is user information as an example. When it is stored in the cache space, "user index + user information" will be stored, and user information is usually encoded with key-value, such as the common json format, and the user index is the user's id, user information access {"name":"Zhang San","age":20}.
在本说明书一个或多个实施例中,不采用前述key-value编码对第一缓存数据进行缓存处理,而是采用目标编码处理方式,使得经目标编码处理后的第二缓存编码数据不含键值对中关键字元素数据。In one or more embodiments of this specification, instead of using the aforementioned key-value encoding to cache the first cache data, a target encoding processing method is used, so that the second cache encoded data after target encoding processing does not contain a key Key element data in value pairs.
示意性的,以用户信息存取{"姓名":"张三","岁数":20}为例,仅目标编码处理后,仅会缓存对象的值value-"张三"以及“20”,而不会缓存对象的关键字元素key-"姓名"以及"岁数"。也即第二缓存编码数据中不含键值对中关键字元素数据key。Schematically, take user information access {"name":"Zhang San","age":20} as an example, only the object value-"Zhang San" and "20" will be cached after the target encoding is processed , without caching the key element key-"name" and "age" of the object. That is, the second cache coded data does not contain the key element data key in the key-value pair.
在一种可行的实施方式中,服务平台执行所述对所述第一缓存数据进行目标编码处理,得到第二缓存编码数据,具体可以是:In a feasible implementation manner, the service platform executes the target encoding process on the first cached data to obtain the second cached encoded data, which may specifically be:
C2:确定所述第一缓存数据对应的第一缓存关键信息,所述第一缓存关键信息包括基于键值对编码方式生成的关键字元素数据以及值元素数据。C2: Determine first cache key information corresponding to the first cache data, where the first cache key information includes key element data and value element data generated based on key-value pair encoding.
所述第一缓存关键信息可以理解为以键值对编码方式编码的缓存信息,以前述第一缓存数据为用户信息为例,第一缓存数据“用户索引+用户信息”中的“用户信息”也即第一缓存关键信息。The first cached key information can be understood as cached information encoded in a key-value pair coding manner. Taking the aforementioned first cached data as user information as an example, the "user information" in the first cached data "user index + user information" That is, key information is first cached.
第一缓存关键信息基于键值对编码方式生成,其包括关键字元素数据以及值元素数据。数据{"姓名":"张三","岁数":20}中:"姓名"和"岁数"为关键字元素数据,"张三"和20为值元素数据。The first cache key information is generated based on key-value pair encoding, which includes key element data and value element data. In the data {"name":"Zhang San","age":20}: "name" and "age" are key element data, "Zhang San" and 20 are value element data.
C4:确定针对关键字元素数据的分隔符,基于所述值元素数据和所述分隔符进行编码处理,得到第二缓存数据。C4: Determine the delimiter for the key element data, perform encoding processing based on the value element data and the delimiter, and obtain the second cache data.
可以理解的,可以以分隔符实现对关键字元素数据的替代,分隔符占用空间小于关键字元素数据,基于此可大幅节省缓存编码所占空间。It can be understood that the keyword element data can be replaced by a delimiter, and the space occupied by the delimiter is smaller than that of the key element data. Based on this, the space occupied by the cache code can be greatly saved.
可选的,分割符可以仅起到区分开不同类值元素数据的作用,分割符前后指示的值元素数据为属于不同类关键字元素所对应的值。数据{"姓名":"张三","岁数":20},则可以经目标编码后输出"张三##20",其中“##”即为分割符。Optionally, the delimiter may only serve to distinguish different types of value element data, and the value element data indicated before and after the delimiter are values corresponding to key elements belonging to different types. The data {"name":"Zhang San","age": 20} can output "Zhang San##20" after target encoding, where "##" is the separator.
可选的,分割符可以与关键字元素相关联,也即预先设置不同类别关键字元素与其对应参考分割符的映射关系,基于该映射关系可确定针对关键字元素数据的分割符。Optionally, the separator may be associated with the keyword element, that is, the mapping relationship between different types of keyword elements and their corresponding reference separators is preset, and the separator for the keyword element data can be determined based on the mapping relationship.
可以理解的,在确定针对关键字元素数据的分隔符之后,基于所述值元素数据和所述分隔符进行编码处理,以采用分隔符对关键字元素数据进行替换,在基于替换后的分隔符以及值元素数据编码,可以生成第二缓存数据。示意性的,以数据{"姓名":"李四","岁数":30}为例进行编码处理,采用分割符“??”替代关键字元素"姓名"和"岁数",得到的第二缓存数据可以是“李四??30”。It can be understood that after determining the delimiter for the key element data, encoding processing is performed based on the value element data and the delimiter, so as to replace the key element data with the delimiter, and based on the delimiter after replacement And the value element data encoding can generate the second cache data. Schematically, take the data {"name":"Li Si","age":30} as an example for encoding processing, and use the separator "??" to replace the keyword elements "name" and "age", and the obtained first The second buffer data may be "Li Si?? 30".
S310:对所述第二缓存数据进行缓存存储。S310: Perform cache storage on the second cache data.
可以理解的,基于第二缓存数据对应的数据类型将其缓存至数据类型对应的数据缓存空间中。如第二缓存数据对应的数据类型为热数据类型,则将其缓存至第一数据缓存空间中,可适用于缓存空间有限的服务场景,可有效解决服务平台负载压力大的问题。It can be understood that, based on the data type corresponding to the second cached data, it is cached in the data cache space corresponding to the data type. If the data type corresponding to the second cache data is a hot data type, it is cached in the first data cache space, which is applicable to service scenarios with limited cache space and can effectively solve the problem of heavy load pressure on the service platform.
示意性的,第二缓存数据对应的数据类型与第一缓存数据对应的数据类型相同。Schematically, the data type corresponding to the second cache data is the same as the data type corresponding to the first cache data.
S312:响应于客户端的数据查询请求,获取第二缓存数据;S312: Obtain second cached data in response to the data query request of the client;
所述数据查询请求为客户端向服务平台所发起的,服务平台对外提供相应事务服务,客户端的用户基于实际需求可在客户端在上向服务平台发起数据查询请求,以获取相应服务数据。The data query request is initiated by the client to the service platform, and the service platform provides corresponding transaction services externally. Based on actual needs, the user of the client can initiate a data query request to the service platform on the client to obtain corresponding service data.
可以理解的,服务平台接收到客户端的数据查询请求,响应客户端的数据查询请求,服务平台首先在维护的缓存空间(第一缓存空间、第二缓存空间)中查询是否存在数据查询请求指示的请求数据,若存在该请求数据,则从缓存空间(第一缓存空间、第二缓存空间)中获取该请求数据并作为查询到的第二缓存数据。It can be understood that the service platform receives the data query request from the client and responds to the data query request from the client. The service platform first checks whether there is a request indicated by the data query request in the maintained cache space (first cache space, second cache space) data, if the requested data exists, the requested data is obtained from the cache space (the first cache space, the second cache space) and used as the queried second cache data.
示意性的,服务平台对外可以提供近端服务,通常,近端服务是一个sdk(软件开发工具包)安装到客户端中,客户端发起的数据查询请求,由近端服务与服务平台协同处理,服务平台可以基于该近端服务接收到数据查询请求,并获取第二缓存数据。Schematically, the service platform can provide near-end services externally. Usually, the near-end service is an sdk (software development kit) installed in the client, and the data query request initiated by the client is processed by the near-end service and the service platform. , the service platform may receive the data query request based on the near-end service, and acquire the second cached data.
S314:对所述第二缓存数据进行目标解码处理得到第三缓存数据,将所述第三缓存数据发送至客户端;S314: Perform target decoding processing on the second cached data to obtain third cached data, and send the third cached data to the client;
所述目标解码处理可以理解为目标编码处理的逆过程。The target decoding process can be understood as the inverse process of the target encoding process.
在一种可行的实施方式中,所述对所述第二缓存数据进行目标解码处理,得到第三缓存数据,包括:In a feasible implementation manner, the performing target decoding processing on the second cached data to obtain the third cached data includes:
确定所述第二缓存数据对应的用户索引信息和第二用户信息,所述第二用户信息包括值元素数据和分隔符;Determine user index information and second user information corresponding to the second cache data, where the second user information includes value element data and separators;
基于所述值元素数据和所述分隔符确定关键字元素数据,基于所述用户索引信息、所述值元素数据和所述关键字元素数据进行编码处理,得到第三缓存编码数据。Key element data is determined based on the value element data and the delimiter, and encoding processing is performed based on the user index information, the value element data, and the key element data to obtain third cache encoded data.
服务平台执行所述对所述第二缓存数据进行目标解码处理,得到第三缓存数据,可以是:The service platform executes the target decoding process on the second cached data to obtain the third cached data, which may be:
D2:确定所述第二缓存数据对应的第二缓存关键信息,所述第二缓存关键信息包括值元素数据和分隔符;D2: Determine the second cache key information corresponding to the second cache data, the second cache key information includes value element data and a separator;
在实际应用中,第二缓存数据的数量为多个,基于实际事务需要通常缓存数据可由索引信息以及缓存关键信息构成,索引信息用于从多个缓存数据中索引到请求查询的缓存数据。基于此,可以从第二缓存数据中提取出第二缓存关键信息。第二缓存关键信息是经目标编码处理之后得到数据,第二缓存关键信息包括值元素数据和分隔符。In practical applications, there are multiple second cached data. Based on actual transaction needs, the cached data can generally consist of index information and cache key information. The index information is used to index from multiple cached data to the cached data requested for query. Based on this, the second cache key information may be extracted from the second cache data. The second cache key information is data obtained after object encoding processing, and the second cache key information includes value element data and a delimiter.
D4:基于所述值元素数据和/或所述分隔符确定关键字元素数据,基于所述值元素数据和所述关键字元素数据进行编码处理,得到第三缓存数据,其中,所述第三缓存编码数据为包含键值对中关键字元素数据的缓存编码数据。D4: Determine key element data based on the value element data and/or the delimiter, perform encoding processing based on the value element data and the key element data, and obtain third cache data, wherein the third Cache-encoded data is cache-encoded data that includes key element data in key-value pairs.
可以理解的,可以确定值元素数据的数据类型,根据值元素数据的数据类型来确定关键字元素数据,值元素数据的数据类型与关键字元素数据强相关。例如值元素数据“张三”,其数据类型通常为姓名,则关键字元素数据即为姓名关键字元素。It can be understood that the data type of the value element data can be determined, and the key element data can be determined according to the data type of the value element data, and the data type of the value element data is strongly related to the key element data. For example, the data type of the value element data "Zhang San" is usually a name, and the key element data is the name key element.
可以理解的,可以根据分隔符来确定关键字元素数据,在目标编码处理时,若基于不同的关键字元素采用不同的分隔符,则可以基于不同分隔符与相应关键字元素的映射关系,来确定分隔符所对应的关键字元素数据。It can be understood that the key element data can be determined according to the delimiter. During the target encoding process, if different delimiters are used based on different key elements, then based on the mapping relationship between different delimiters and corresponding key elements, the Determines the key element data corresponding to the delimiter.
可选的,可以同时基于值元素数据和所述分隔符确定关键字元素数据,也即对两种关键字元素数据进行数据拟合,得到最终的关键字元素数据。Optionally, the key element data may be determined based on the value element data and the delimiter at the same time, that is, data fitting is performed on the two key element data to obtain the final key element data.
进一步的,在确定关键字元素数据之后,基于所述值元素数据和所述关键字元素数据进行编码处理,一种方式可以是直接基于所述值元素数据和所述关键字元素数据组合生成第三缓存数据,一种方式可以是对第二缓存关键信息中的分隔符采用关键字元素数据进行替换,基于替换之后的数据来生成第三缓存数据。Further, after the keyword element data is determined, encoding processing is performed based on the value element data and the keyword element data. One way may be to directly generate the first key element data based on the combination of the value element data and the keyword element data. For the third cache data, one way may be to replace the delimiter in the key information of the second cache with key element data, and generate the third cache data based on the replaced data.
进一步的,服务平台生成第三缓存数据之后,然后可以将第三缓存数据发送至客户端。Further, after the service platform generates the third cache data, it may then send the third cache data to the client.
S316:将所述第二缓存数据发送至客户端,以使所述客户端对所述第二缓存数据进行目标解码处理得到第三缓存数据。S316: Send the second cached data to the client, so that the client performs target decoding processing on the second cached data to obtain third cached data.
可以理解的,服务平台响应客户端的数据查询请求,获取第二缓存数据之后,可以不再本地进行解码,以减轻服务平台的缓存管理压力。通过直接将第二缓存数据发送至客户端,从而指示客户端对第二缓存数据进行目标解码处理,来得到第三缓存数据。It can be understood that after the service platform responds to the data query request of the client and obtains the second cached data, it may no longer perform local decoding, so as to reduce the cache management pressure of the service platform. The third cached data is obtained by directly sending the second cached data to the client, thereby instructing the client to perform target decoding processing on the second cached data.
在本说明书一个或多个实施例中,客户端对所述第二缓存数据进行目标解码处理得到第三缓存数据的执行过程与服务平台对所述第二缓存数据进行目标解码处理得到第三缓存数据的执行过程相同,区别仅在于执行主体不同,此处不再进行赘述。In one or more embodiments of this specification, the execution process of the client performing target decoding processing on the second cache data to obtain the third cache data and the service platform performing target decoding processing on the second cache data to obtain the third cache data The execution process of the data is the same, and the only difference is that the execution subject is different, which will not be repeated here.
在一种具体的实施场景中,服务平台可以对外提供近端服务,近端服务可以用于解码,以解决服务平台压力大的现象,数据查询请求直接请求到近端服务中。缓存查询命中,近端服务再基于服务平台反馈的第二缓存数据进行目标解码处理,以第二缓存数据中的第二缓存关键信息为"张三##20"为例,则解码还原为{"姓名":"张三","岁数":20}。In a specific implementation scenario, the service platform can provide near-end services externally, and the near-end services can be used for decoding to solve the phenomenon of high pressure on the service platform, and data query requests are directly requested to the near-end services. When the cache query hits, the near-end service performs target decoding processing based on the second cache data fed back by the service platform. Taking the second cache key information in the second cache data as "Zhang San##20" as an example, the decoding is restored to { "Name": "Zhang San", "age": 20}.
在本说明书一个或多个实施例中,服务平台通过监测针对热数据和冷数据的缓存波动状态以获取当前的访问用户分布信息,基于访问用户分布信息对第一数据缓存空间和/或第二数据缓存空间进行缓存空间动态调节。基于访问用户分布信息一定程度上可预测冷热数据的缓存量的波动,可预先对相应缓存空间进行调节,提高了请求数据时缓存命中率,降低服务平台的处理压力;以及避免采用对缓存数据进行缓存空间迁移的方式,可以降低缓存管理的时间复杂度以及缓存系统的负载,以及节省缓存管理的开销量;以及,在波动剧烈状态下,平台能够自动精准监测,并且决策出合适的缓存淘汰机制进行缓存管理;以及,在缓存空间不足的情况下,利用编解码可以在有限的缓存空间里缓存更多的数据,提高缓存空间的资源利用率。In one or more embodiments of this specification, the service platform obtains the current access user distribution information by monitoring the cache fluctuation status for hot data and cold data, and based on the access user distribution information, caches the first data and/or the second The data cache space is dynamically adjusted for the cache space. Based on the access user distribution information, the fluctuation of the cache volume of hot and cold data can be predicted to a certain extent, and the corresponding cache space can be adjusted in advance, which improves the cache hit rate when requesting data, reduces the processing pressure of the service platform; and avoids the use of cached data The way to migrate the cache space can reduce the time complexity of cache management and the load of the cache system, and save the overhead of cache management; and, in the case of severe fluctuations, the platform can automatically and accurately monitor and make decisions on appropriate cache elimination mechanism for cache management; and, in the case of insufficient cache space, more data can be cached in the limited cache space by using codecs, improving the resource utilization of the cache space.
下面将结合图5,对本说明书提供的缓存管理装置进行详细介绍。需要说明的是,图5所示的缓存管理装置,用于执行本申请图1~图4所示实施例的方法,为了便于说明,仅示出了与本说明书相关的部分,具体技术细节未揭示的,请参照本申请图1~图4所示的实施例。The cache management device provided in this specification will be described in detail below with reference to FIG. 5 . It should be noted that the cache management device shown in FIG. 5 is used to execute the methods shown in the embodiments shown in FIGS. 1 to 4 of this application. For the convenience of description, only the parts related to this specification are shown, and the specific technical details are not included. For disclosure, please refer to the embodiments shown in FIGS. 1 to 4 of this application.
请参见图5,其示出本说明书的缓存管理装置的结构示意图。该缓存管理装置1可以通过软件、硬件或者两者的结合实现成为用户终端的全部或一部分。根据一些实施例,该缓存管理装置1包括信息获取模块11、数据处理模块12,具体用于:Please refer to FIG. 5 , which shows a schematic structural diagram of a cache management device in this specification. The
信息获取模块11,用于监测针对热数据和冷数据的缓存波动状态,基于所述缓存波动状态获取访问用户分布信息;The
数据处理模块12,用于基于所述访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节,所述第一数据缓存空间为缓存所述热数据的数据存储空间,所述第二数据缓存空间为缓存所述冷数据的数据缓存空间。The
可选的,如图6所示,所信息获取模块11,包括:Optionally, as shown in Figure 6, the
数据监测单元111,用于监测针对热数据和冷数据的缓存波动比例;A
信息获取单元112,用于若所述缓存波动比例大于或等于波动比例阈值,则获取访问用户分布信息。The
可选的,所述信息获取模块11,具体用于:Optionally, the
获取针对新用户类型的第一用户访问数据,以及针对老用户类型的第二用户访问数据;Acquiring first user access data for new user types and second user access data for old user types;
基于所述第一用户访问数据以及第二用户访问数据确定新老用户波动数据,将所述新老用户波动数据作为访问用户分布信息;Determine new and old user fluctuation data based on the first user access data and the second user access data, and use the new and old user fluctuation data as access user distribution information;
其中,所述新用户类型和所述老用户类型为针对所述服务平台的访问用户所划分的用户类型。Wherein, the new user type and the old user type are user types classified for access users of the service platform.
可选的,如图7所示,所述数据处理模块12,包括:Optionally, as shown in Figure 7, the
方式确定单元121,用于基于所述访问用户分布信息,确定目标缓存调节方式;A
空间调节单元122,用于采用所述目标缓存调节方式对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节。The
可选的,所述访问用户分布信息为新老用户波动数据,所述方式确定单元121,具体用于:Optionally, the access user distribution information is new and old user fluctuation data, and the
基于所述新老用户波动数据确定针对所述服务平台的用户流量波动状态;determining the user traffic fluctuation state for the service platform based on the fluctuation data of new and old users;
若所述用户流量波动状态为波动剧烈状态,则确定目标缓存调节方式为缓存动态调节方式;If the user traffic fluctuation state is a state of severe fluctuation, then determine that the target cache adjustment mode is a cache dynamic adjustment mode;
若所述用户流量波动状态为波动平稳状态,则确定目标缓存调节方式为缓存周期调节方式。If the user traffic fluctuation state is a stable fluctuation state, it is determined that the target cache adjustment mode is a cache period adjustment mode.
可选的,所述方式确定单元121,具体用于:Optionally, the
所述新老用户波动数据包括用户增长量以及新老用户比重;The new and old user fluctuation data includes user growth and the proportion of new and old users;
若所述用户增长量大于增量阈值,且所述新老用户比重大于用户比重阈值,则确定针对所述服务平台的用户流量波动状态为所述波动剧烈状态。If the user growth amount is greater than the increment threshold, and the proportion of new and old users is greater than the user proportion threshold, it is determined that the fluctuation state of user traffic for the service platform is the severe fluctuation state.
可选的,所述目标缓存调节方式为缓存动态调节方式,所述空间调节单元122,具体用于:Optionally, the target cache adjustment mode is a cache dynamic adjustment mode, and the
采用所述缓存动态调节方式确定滑动窗口调节区域以及目标调节缓存空间,基于所述滑动窗口调节区域对目标调节缓存空间进行缓存空间调节,所述目标调节缓存空间为所述第一数据缓存空间和第二数据缓存空间中的至少其一。Using the buffer dynamic adjustment method to determine the sliding window adjustment area and the target adjustment buffer space, and based on the sliding window adjustment area to adjust the buffer space for the target adjustment buffer space, the target adjustment buffer space is the first data buffer space and the first data buffer space. At least one of the second data cache spaces.
可选的,所述空间调节单元122,具体用于:Optionally, the
基于所述访问用户分布信息确定缓存调节比例,基于所述缓存调节比例确定滑动窗口调节区域。A cache adjustment ratio is determined based on the access user distribution information, and a sliding window adjustment area is determined based on the cache adjustment ratio.
可选的,所述空间调节单元122,具体用于:Optionally, the
获取新老用户波动数据和/或冷热数据波动参数,基于所述新老用户波动数据和/或所述冷热数据波动参数,从第一数据缓存空间和第二数据缓存空间中确定目标调节缓存空间;Obtain new and old user fluctuation data and/or cold and hot data fluctuation parameters, and determine target adjustment from the first data cache space and the second data cache space based on the new and old user fluctuation data and/or the cold and hot data fluctuation parameters cache space;
确定针对第三数据缓存空间的第一缓存调节比例;determining a first cache adjustment ratio for the third data cache space;
基于所述第一缓存调节比例从第三数据缓存空间中选取第一滑动窗口调节区域,将所述第一滑动窗口调节区域与所述目标调节缓存空间相关联,所述第三数据缓存空间为所述第一数据缓存空间和所述第二数据缓存空间中除所述目标调节缓存空间之外的数据缓存空间。Selecting a first sliding window adjustment area from the third data buffer space based on the first buffer adjustment ratio, associating the first sliding window adjustment area with the target adjustment buffer space, the third data buffer space is A data cache space in the first data cache space and the second data cache space except the target adjustment cache space.
可选的,所述空间调节单元122,具体用于:Optionally, the
获取新老用户波动数据和/或冷热数据波动参数,基于所述新老用户波动数据和/或所述冷热数据波动参数,从第一数据缓存空间和第二数据缓存空间中确定目标调节缓存空间;Obtain new and old user fluctuation data and/or cold and hot data fluctuation parameters, and determine target adjustment from the first data cache space and the second data cache space based on the new and old user fluctuation data and/or the cold and hot data fluctuation parameters cache space;
获取预留缓存空间,确定针对所述预留缓存空间的第二缓存调节比例;所述预留缓存空间为除所述第一数据缓存空间和第二数据缓存空间之外的数据缓存空间;Obtain a reserved cache space, and determine a second cache adjustment ratio for the reserved cache space; the reserved cache space is a data cache space other than the first data cache space and the second data cache space;
基于所述第二缓存调节比例从所述预留缓存空间中选取第二滑动窗口调节区域,将所述第二滑动窗口调节区域与目标调节缓存空间相关联。Selecting a second sliding window adjustment area from the reserved buffer space based on the second buffer adjustment ratio, and associating the second sliding window adjustment area with a target adjustment buffer space.
可选的,所述装置1,还用于:Optionally, the
基于所述目标缓存调节方式,对第一数据缓存空间和/或第二数据缓存空间进行缓存淘汰机制调整。Based on the target cache adjustment manner, a cache elimination mechanism adjustment is performed on the first data cache space and/or the second data cache space.
可选的,所述装置1,还用于:Optionally, the
若所述目标缓存调节方式为缓存动态调节方式,则采用最近最少使用缓存机制对所述第一数据缓存空间进行缓存淘汰处理,和采用最不经常使用缓存机制对所述第二数据缓存空间进行缓存淘汰处理。If the target cache adjustment mode is a dynamic cache adjustment mode, the least recently used cache mechanism is used to perform cache elimination processing on the first data cache space, and the least frequently used cache mechanism is used to perform cache elimination processing on the second data cache space Cache eviction handling.
可选的,如图8所示,所述装置1,包括:Optionally, as shown in Figure 8, the
数据缓存模块13,用于获取针对第一数据缓存空间和/或第二数据缓存空间的待存储的第一缓存数据;A
所述数据缓存模块13,还用于对所述第一缓存数据进行目标编码处理,得到第二缓存编码数据;所述第二缓存编码数据为不含键值对中关键字元素数据的缓存编码数据。The
对所述第二缓存数据进行缓存存储。Perform cache storage on the second cache data.
可选的,所述数据缓存模块13,还用于:Optionally, the
确定所述第一缓存数据对应的第一缓存关键信息,所述第一缓存关键信息包括基于键值对编码方式生成的关键字元素数据以及值元素数据;Determine first cache key information corresponding to the first cache data, where the first cache key information includes key element data and value element data generated based on key-value pair encoding;
确定针对关键字元素数据的分隔符,基于所述值元素数据和所述分隔符进行编码处理,得到第二缓存数据。Determining a delimiter for the key element data, performing encoding processing based on the value element data and the delimiter, to obtain second cache data.
可选的,所述数据缓存模块13,还用于:Optionally, the
响应于客户端的数据查询请求,获取第二缓存数据;In response to the data query request of the client, obtain the second cache data;
对所述第二缓存数据进行目标解码处理得到第三缓存数据,将所述第三缓存数据发送至客户端;或,将所述第二缓存数据发送至客户端,以使所述客户端对所述第二缓存数据进行目标解码处理得到第三缓存数据;performing target decoding processing on the second cached data to obtain third cached data, and sending the third cached data to the client; or, sending the second cached data to the client, so that the client can performing target decoding processing on the second cached data to obtain third cached data;
其中,所述第三缓存数据为包含键值对中关键字元素数据的缓存编码数据。Wherein, the third cache data is cache coded data including key element data in the key-value pair.
可选的,所述数据缓存模块13,还用于:Optionally, the
确定所述第二缓存数据对应的第二缓存关键信息,所述第二缓存关键信息包括值元素数据和分隔符;Determine second cache key information corresponding to the second cache data, where the second cache key information includes value element data and a delimiter;
基于所述值元素数据和/或所述分隔符确定关键字元素数据,基于所述值元素数据和所述关键字元素数据进行编码处理,得到第三缓存数据。Key element data is determined based on the value element data and/or the delimiter, and encoding processing is performed based on the value element data and the key element data to obtain third cache data.
需要说明的是,上述实施例提供的缓存管理装置在执行缓存管理方法时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将设备的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。另外,上述实施例提供的缓存管理装置与缓存管理方法实施例属于同一构思,其体现实现过程详见方法实施例,这里不再赘述。It should be noted that, when the cache management device provided in the above embodiment executes the cache management method, the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional modules according to needs. , which divides the internal structure of the device into different functional modules to complete all or part of the functions described above. In addition, the cache management device and the cache management method embodiments provided in the above embodiments belong to the same idea, and the implementation process thereof is detailed in the method embodiments, and will not be repeated here.
上述本说明书序号仅仅为了描述,不代表实施例的优劣。The above serial numbers in this specification are for description only, and do not represent the advantages and disadvantages of the embodiments.
在本说明书一个或多个实施例中,服务平台通过监测针对热数据和冷数据的缓存波动状态以获取当前的访问用户分布信息,基于访问用户分布信息对第一数据缓存空间和/或第二数据缓存空间进行缓存空间动态调节。基于访问用户分布信息一定程度上可预测冷热数据的缓存量的波动,可预先对相应缓存空间进行调节,提高了请求数据时缓存命中率,降低服务平台的处理压力。In one or more embodiments of this specification, the service platform obtains the current access user distribution information by monitoring the cache fluctuation status for hot data and cold data, and based on the access user distribution information, caches the first data and/or the second The data cache space is dynamically adjusted for the cache space. Based on the access user distribution information, the fluctuation of the cache volume of hot and cold data can be predicted to a certain extent, and the corresponding cache space can be adjusted in advance, which improves the cache hit rate when requesting data and reduces the processing pressure of the service platform.
本说明书还提供了一种计算机存储介质,所述计算机存储介质可以存储有多条指令,所述指令适于由处理器加载并执行如上述图1~图4所示实施例的所述缓存管理方法,具体执行过程可以参见图1~图4所示实施例的具体说明,在此不进行赘述。This specification also provides a computer storage medium, the computer storage medium can store a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the cache management in the embodiments shown in the above-mentioned Figures 1 to 4 For the method, for the specific execution process, refer to the specific description of the embodiments shown in FIGS. 1 to 4 , and details are not repeated here.
本申请还提供了一种计算机程序产品,该计算机程序产品存储有至少一条指令,所述至少一条指令由所述处理器加载并执行如上述图1~图4所示实施例的所述缓存管理方法,具体执行过程可以参见图1~图4所示实施例的具体说明,在此不进行赘述。The present application also provides a computer program product, the computer program product stores at least one instruction, and the at least one instruction is loaded by the processor and executes the cache management described in the embodiments shown in Fig. 1 to Fig. 4 above. For the method, for the specific execution process, refer to the specific description of the embodiments shown in FIGS. 1 to 4 , and details are not repeated here.
请参考图9,其示出了本申请一个示例性实施例提供的电子设备的结构方框图。本申请中的电子设备可以包括一个或多个如下部件:处理器110、存储器120、输入装置130、输出装置140和总线150。处理器110、存储器120、输入装置130和输出装置140之间可以通过总线150连接。Please refer to FIG. 9 , which shows a structural block diagram of an electronic device provided by an exemplary embodiment of the present application. The electronic device in this application may include one or more of the following components: a
处理器110可以包括一个或者多个处理核心。处理器110利用各种接口和线路连接整个电子设备内的各个部分,通过运行或执行存储在存储器120内的指令、程序、代码集或指令集,以及调用存储在存储器120内的数据,执行电子设备100的各种功能和处理数据。可选地,处理器110可以采用数字信号处理(digital signal processing,DSP)、现场可编程门阵列(field-programmable gate array,FPGA)、可编程逻辑阵列(programmable logicArray,PLA)中的至少一种硬件形式来实现。处理器110可集成中心处理器(centralprocessing unit,CPU)、图像处理器(graphics processing unit,GPU)和调制解调器等中的一种或几种的组合。其中,CPU主要处理操作系统、用户界面和应用程序等;GPU用于负责显示内容的渲染和绘制;调制解调器用于处理无线通信。可以理解的是,上述调制解调器也可以不集成到处理器110中,单独通过一块通信芯片进行实现。
存储器120可以包括随机存储器(random Access Memory,RAM),也可以包括只读存储器(read-only memory,ROM)。可选地,该存储器120包括非瞬时性计算机可读介质(non-transitory computer-readable storage medium)。存储器120可用于存储指令、程序、代码、代码集或指令集。存储器120可包括存储程序区和存储数据区,其中,存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等,该操作系统可以是安卓(Android)系统,包括基于Android系统深度开发的系统、苹果公司开发的IOS系统,包括基于IOS系统深度开发的系统或其它系统。存储数据区还可以存储电子设备在使用中所创建的数据比如电话本、音视频数据、聊天记录数据,等。The
参见图10所示,存储器120可分为操作系统空间和用户空间,操作系统即运行于操作系统空间,原生及第三方应用程序即运行于用户空间。为了保证不同第三方应用程序均能够达到较好的运行效果,操作系统针对不同第三方应用程序为其分配相应的系统资源。然而,同一第三方应用程序中不同应用场景对系统资源的需求也存在差异,比如,在本地资源加载场景下,第三方应用程序对磁盘读取速度的要求较高;在动画渲染场景下,第三方应用程序则对GPU性能的要求较高。而操作系统与第三方应用程序之间相互独立,操作系统往往不能及时感知第三方应用程序当前的应用场景,导致操作系统无法根据第三方应用程序的具体应用场景进行针对性的系统资源适配。Referring to FIG. 10 , the
为了使操作系统能够区分第三方应用程序的具体应用场景,需要打通第三方应用程序与操作系统之间的数据通信,使得操作系统能够随时获取第三方应用程序当前的场景信息,进而基于当前场景进行针对性的系统资源适配。In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up the data communication between the third-party applications and the operating system, so that the operating system can obtain the current scene information of the third-party applications at any time, and then based on the current scene. Targeted system resource adaptation.
以操作系统为Android系统为例,存储器120中存储的程序和数据如图11所示,存储器120中可存储有Linux内核层320、系统运行时库层340、应用框架层360和应用层380,其中,Linux内核层320、系统运行库层340和应用框架层360属于操作系统空间,应用层380属于用户空间。Linux内核层320为电子设备的各种硬件提供了底层的驱动,如显示驱动、音频驱动、摄像头驱动、蓝牙驱动、Wi-Fi驱动、电源管理等。系统运行库层340通过一些C/C++库来为Android系统提供了主要的特性支持。如SQLite库提供了数据库的支持,OpenGL/ES库提供了3D绘图的支持,Webkit库提供了浏览器内核的支持等。在系统运行时库层340中还提供有安卓运行时库(Android runtime),它主要提供了一些核心库,能够允许开发者使用Java语言来编写Android应用。应用框架层360提供了构建应用程序时可能用到的各种API,开发者也可以通过使用这些API来构建自己的应用程序,比如活动管理、窗口管理、视图管理、通知管理、内容提供者、包管理、通话管理、资源管理、定位管理。应用层380中运行有至少一个应用程序,这些应用程序可以是操作系统自带的原生应用程序,比如联系人程序、短信程序、时钟程序、相机应用等;也可以是第三方开发者所开发的第三方应用程序,比如游戏类应用程序、即时通信程序、相片美化程序等。Taking the operating system as an Android system as an example, the programs and data stored in the
以操作系统为IOS系统为例,存储器120中存储的程序和数据如图12所示,IOS系统包括:核心操作系统层420(Core OS layer)、核心服务层440(Core Services layer)、媒体层460(Media layer)、可触摸层480(Cocoa Touch Layer)。核心操作系统层420包括了操作系统内核、驱动程序以及底层程序框架,这些底层程序框架提供更接近硬件的功能,以供位于核心服务层440的程序框架所使用。核心服务层440提供给应用程序所需要的系统服务和/或程序框架,比如基础(Foundation)框架、账户框架、广告框架、数据存储框架、网络连接框架、地理位置框架、运动框架等等。媒体层460为应用程序提供有关视听方面的接口,如图形图像相关的接口、音频技术相关的接口、视频技术相关的接口、音视频传输技术的无线播放(AirPlay)接口等。可触摸层480为应用程序开发提供了各种常用的界面相关的框架,可触摸层480负责用户在电子设备上的触摸交互操作。比如本地通知服务、远程推送服务、广告框架、游戏工具框架、消息用户界面接口(User Interface,UI)框架、用户界面UIKit框架、地图框架等等。Taking the operating system as the IOS system as an example, the programs and data stored in the
在图12所示出的框架中,与大部分应用程序有关的框架包括但不限于:核心服务层440中的基础框架和可触摸层480中的UIKit框架。基础框架提供许多基本的对象类和数据类型,为所有应用程序提供最基本的系统服务,和UI无关。而UIKit框架提供的类是基础的UI类库,用于创建基于触摸的用户界面,iOS应用程序可以基于UIKit框架来提供UI,所以它提供了应用程序的基础架构,用于构建用户界面,绘图、处理和用户交互事件,响应手势等等。Among the frameworks shown in FIG. 12 , frameworks related to most applications include, but are not limited to: the basic framework in the core service layer 440 and the UIKit framework in the touchable layer 480 . The basic framework provides many basic object classes and data types, and provides the most basic system services for all applications, regardless of UI. The class provided by the UIKit framework is a basic UI class library for creating a touch-based user interface. iOS applications can provide UI based on the UIKit framework, so it provides the infrastructure of the application for building user interfaces, drawing , Handle and user interaction events, respond to gestures, and more.
其中,在IOS系统中实现第三方应用程序与操作系统数据通信的方式以及原理可参考Android系统,本申请在此不再赘述。Wherein, the method and principle of implementing the data communication between the third-party application program and the operating system in the IOS system can refer to the Android system, and this application will not repeat them here.
其中,输入装置130用于接收输入的指令或数据,输入装置130包括但不限于键盘、鼠标、摄像头、麦克风或触控设备。输出装置140用于输出指令或数据,输出装置140包括但不限于显示设备和扬声器等。在一个示例中,输入装置130和输出装置140可以合设,输入装置130和输出装置140为触摸显示屏,该触摸显示屏用于接收用户使用手指、触摸笔等任何适合的物体在其上或附近的触摸操作,以及显示各个应用程序的用户界面。触摸显示屏通常设置在电子设备的前面板。触摸显示屏可被设计成为全面屏、曲面屏或异型屏。触摸显示屏还可被设计成为全面屏与曲面屏的结合,异型屏与曲面屏的结合,本说明书对此不加以限定。Wherein, the
除此之外,本领域技术人员可以理解,上述附图所示出的电子设备的结构并不构成对电子设备的限定,电子设备可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。比如,电子设备中还包括射频电路、输入单元、传感器、音频电路、无线保真(wireless fidelity,WiFi)模块、电源、蓝牙模块等部件,在此不再赘述。In addition, those skilled in the art can understand that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or less components than those shown in the illustration, or combine certain some components, or a different arrangement of components. For example, the electronic device also includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, and a bluetooth module, which will not be repeated here.
在本说明书中,各步骤的执行主体可以是上文介绍的电子设备。可选地,各步骤的执行主体为电子设备的操作系统。操作系统可以是安卓系统,也可以是IOS系统,或者其它操作系统,本说明书对此不作限定。In this specification, the execution subject of each step may be the electronic device introduced above. Optionally, each step is executed by an operating system of the electronic device. The operating system may be an Android system, an IOS system, or other operating systems, which is not limited in this manual.
本说明书的电子设备,其上还可以安装有显示设备,显示设备可以是各种能实现显示功能的设备,例如:阴极射线管显示器(cathode ray tubedisplay,简称CR)、发光二极管显示器(light-emitting diode display,简称LED)、电子墨水屏、液晶显示屏(liquidcrystal display,简称LCD)、等离子显示面板(plasma display panel,简称PDP)等。用户可以利用电子设备101上的显示设备,来查看显示的文字、图像、视频等信息。所述电子设备可以是服务器、服务平台、智能手机、平板电脑、游戏设备、AR(Augmented Reality,增强现实)设备、汽车、数据存储装置、音频播放装置、视频播放装置、笔记本、桌面计算设备等设备。The electronic equipment in this manual can also be equipped with a display device on it, and the display device can be various devices that can realize display functions, such as: cathode ray tube display (cathode ray tube display, CR for short), light-emitting diode display (light-emitting diode display (LED for short), electronic ink screen, liquid crystal display (LCD for short), plasma display panel (PDP for short), etc. The user can use the display device on the electronic device 101 to view displayed text, images, videos and other information. The electronic device may be a server, a service platform, a smart phone, a tablet computer, a game device, an AR (Augmented Reality, augmented reality) device, a car, a data storage device, an audio playback device, a video playback device, a notebook, a desktop computing device, etc. equipment.
在图9所示的电子设备中,其中电子设备可以是一种服务平台,处理器110可以用于调用存储器120中存储的应用程序,并具体执行以下操作:In the electronic device shown in FIG. 9 , where the electronic device can be a service platform, the
监测针对热数据和冷数据的缓存波动状态,基于所述缓存波动状态获取访问用户分布信息;Monitoring the cache fluctuation state for hot data and cold data, and obtaining access user distribution information based on the cache fluctuation state;
基于所述访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节,所述第一数据缓存空间为缓存所述热数据的数据存储空间,所述第二数据缓存空间为缓存所述冷数据的数据缓存空间。Based on the access user distribution information, the cache space adjustment is performed on the first data cache space and/or the second data cache space, the first data cache space is a data storage space for caching the hot data, and the second data cache space The cache space is a data cache space for caching the cold data.
在一个实施例中,所述处理器110在执行所述监测针对热数据和冷数据的缓存波动状态,基于所述缓存波动状态获取访问用户分布信息时,具体执行以下操作:In one embodiment, when the
监测针对热数据和冷数据的缓存波动比例;Monitor the cache fluctuation ratio for hot data and cold data;
若所述缓存波动比例大于或等于波动比例阈值,则获取访问用户分布信息。If the cache fluctuation ratio is greater than or equal to the fluctuation ratio threshold, access user distribution information is acquired.
在一个实施例中,所述处理器110在执行所述获取访问用户分布信息时,执行以下操作:In one embodiment, the
获取针对新用户类型的第一用户访问数据,以及针对老用户类型的第二用户访问数据;Acquiring first user access data for new user types and second user access data for old user types;
基于所述第一用户访问数据以及第二用户访问数据确定新老用户波动数据,将所述新老用户波动数据作为访问用户分布信息;Determine new and old user fluctuation data based on the first user access data and the second user access data, and use the new and old user fluctuation data as access user distribution information;
其中,所述新用户类型和所述老用户类型为针对所述服务平台的访问用户所划分的用户类型。Wherein, the new user type and the old user type are user types classified for access users of the service platform.
在一个实施例中,所述处理器110在执行所述基于所述访问用户分布信息,对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节时,执行以下步骤:In one embodiment, the
基于所述访问用户分布信息,确定目标缓存调节方式;Determine a target cache adjustment method based on the access user distribution information;
采用所述目标缓存调节方式对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节。The cache space adjustment is performed on the first data cache space and/or the second data cache space by using the target cache adjustment manner.
在一个实施例中,所述访问用户分布信息为新老用户波动数据,所述处理器110在执行所述基于所述访问用户分布信息,确定目标缓存调节方时,执行以下步骤:In one embodiment, the access user distribution information is new and old user fluctuation data, and the
基于所述新老用户波动数据确定针对所述服务平台的用户流量波动状态;determining the user traffic fluctuation state for the service platform based on the fluctuation data of new and old users;
若所述用户流量波动状态为波动剧烈状态,则确定目标缓存调节方式为缓存动态调节方式;If the user traffic fluctuation state is a state of severe fluctuation, then determine that the target cache adjustment mode is a cache dynamic adjustment mode;
若所述用户流量波动状态为波动平稳状态,则确定目标缓存调节方式为缓存周期调节方式。If the user traffic fluctuation state is a stable fluctuation state, it is determined that the target cache adjustment mode is a cache period adjustment mode.
在一个实施例中,所述处理器110在执行所述基于所述新老用户波动数据确定针对所述服务平台的用户流量波动状态时,执行以下步骤:In one embodiment, the
所述新老用户波动数据包括用户增长量以及新老用户比重;The new and old user fluctuation data includes user growth and the proportion of new and old users;
若所述用户增长量大于增量阈值,且所述新老用户比重大于用户比重阈值,则确定针对所述服务平台的用户流量波动状态为所述波动剧烈状态。If the user growth amount is greater than the increment threshold, and the proportion of new and old users is greater than the user proportion threshold, it is determined that the fluctuation state of user traffic for the service platform is the severe fluctuation state.
在一个实施例中,所述目标缓存调节方式为缓存动态调节方式,所述处理器110在执行所述采用所述目标缓存调节方式对第一数据缓存空间和/或第二数据缓存空间进行缓存空间调节时,执行以下步骤:In one embodiment, the target cache adjustment method is a cache dynamic adjustment method, and the
采用所述缓存动态调节方式确定滑动窗口调节区域以及目标调节缓存空间,基于所述滑动窗口调节区域对目标调节缓存空间进行缓存空间调节,所述目标调节缓存空间为所述第一数据缓存空间和第二数据缓存空间中的至少其一。Using the buffer dynamic adjustment method to determine the sliding window adjustment area and the target adjustment buffer space, and based on the sliding window adjustment area to adjust the buffer space for the target adjustment buffer space, the target adjustment buffer space is the first data buffer space and the first data buffer space. At least one of the second data cache spaces.
在一个实施例中,所述处理器110在执行所述确定滑动窗口调节区域时,执行以下步骤:In one embodiment, the
基于所述访问用户分布信息确定缓存调节比例,基于所述缓存调节比例确定滑动窗口调节区域。A cache adjustment ratio is determined based on the access user distribution information, and a sliding window adjustment area is determined based on the cache adjustment ratio.
在一个实施例中,所述处理器110在执行所述采用所述缓存动态调节方式确定滑动窗口调节区域以及目标调节缓存空间,基于所述滑动窗口调节区域对目标调节缓存空间进行缓存空间调节时,执行以下步骤:In one embodiment, when the
获取新老用户波动数据和/或冷热数据波动参数,基于所述新老用户波动数据和/或所述冷热数据波动参数,从第一数据缓存空间和第二数据缓存空间中确定目标调节缓存空间;Obtain new and old user fluctuation data and/or cold and hot data fluctuation parameters, and determine target adjustment from the first data cache space and the second data cache space based on the new and old user fluctuation data and/or the cold and hot data fluctuation parameters cache space;
确定针对第三数据缓存空间的第一缓存调节比例;determining a first cache adjustment ratio for the third data cache space;
基于所述第一缓存调节比例从第三数据缓存空间中选取第一滑动窗口调节区域,将所述第一滑动窗口调节区域与所述目标调节缓存空间相关联,所述第三数据缓存空间为所述第一数据缓存空间和所述第二数据缓存空间中除所述目标调节缓存空间之外的数据缓存空间。Selecting a first sliding window adjustment area from the third data buffer space based on the first buffer adjustment ratio, associating the first sliding window adjustment area with the target adjustment buffer space, the third data buffer space is A data cache space in the first data cache space and the second data cache space except the target adjustment cache space.
在一个实施例中,所述处理器110在执行所述采用所述缓存动态调节方式确定滑动窗口调节区域以及目标调节缓存空间,基于所述滑动窗口调节区域对目标调节缓存空间进行缓存空间调节时,执行以下步骤:In one embodiment, when the
获取新老用户波动数据和/或冷热数据波动参数,基于所述新老用户波动数据和/或所述冷热数据波动参数,从第一数据缓存空间和第二数据缓存空间中确定目标调节缓存空间;Obtain new and old user fluctuation data and/or cold and hot data fluctuation parameters, and determine target adjustment from the first data cache space and the second data cache space based on the new and old user fluctuation data and/or the cold and hot data fluctuation parameters cache space;
获取预留缓存空间,确定针对所述预留缓存空间的第二缓存调节比例;所述预留缓存空间为除所述第一数据缓存空间和第二数据缓存空间之外的数据缓存空间;Obtain a reserved cache space, and determine a second cache adjustment ratio for the reserved cache space; the reserved cache space is a data cache space other than the first data cache space and the second data cache space;
基于所述第二缓存调节比例从所述预留缓存空间中选取第二滑动窗口调节区域,将所述第二滑动窗口调节区域与目标调节缓存空间相关联。Selecting a second sliding window adjustment area from the reserved buffer space based on the second buffer adjustment ratio, and associating the second sliding window adjustment area with a target adjustment buffer space.
在一个实施例中,所述处理器110在执行所述缓存管理方法时,还执行以下步骤:In one embodiment, when the
基于所述目标缓存调节方式,对第一数据缓存空间和/或第二数据缓存空间进行缓存淘汰机制调整。Based on the target cache adjustment manner, a cache elimination mechanism adjustment is performed on the first data cache space and/or the second data cache space.
在一个实施例中,所述处理器110在执行所述基于所述目标缓存调节方式,对第一数据缓存空间和/或第二数据缓存空间进行缓存淘汰机制调整时,执行以下步骤:In one embodiment, when the
若所述目标缓存调节方式为缓存动态调节方式,则采用最近最少使用缓存机制对所述第一数据缓存空间进行缓存淘汰处理,和采用最不经常使用缓存机制对所述第二数据缓存空间进行缓存淘汰处理。If the target cache adjustment mode is a dynamic cache adjustment mode, the least recently used cache mechanism is used to perform cache elimination processing on the first data cache space, and the least frequently used cache mechanism is used to perform cache elimination processing on the second data cache space Cache eviction handling.
在一个实施例中,所述处理器110在执行所述缓存管理方法时,还执行以下步骤:In one embodiment, when the
获取针对第一数据缓存空间和/或第二数据缓存空间的待存储的第一缓存数据;Acquiring first cache data to be stored for the first data cache space and/or the second data cache space;
对所述第一缓存数据进行目标编码处理,得到第二缓存编码数据;所述第二缓存编码数据为不含键值对中关键字元素数据的缓存编码数据。Target encoding processing is performed on the first cached data to obtain second cached encoded data; the second cached encoded data is cached encoded data without key element data in the key-value pair.
对所述第二缓存数据进行缓存存储。Perform cache storage on the second cache data.
在一个实施例中,所述处理器110在执行所述对所述第一缓存数据进行目标编码处理,得到第二缓存数据时,具体执行以下步骤:In one embodiment, when the
确定所述第一缓存数据对应的第一缓存关键信息,所述第一缓存关键信息包括基于键值对编码方式生成的关键字元素数据以及值元素数据;Determine first cache key information corresponding to the first cache data, where the first cache key information includes key element data and value element data generated based on key-value pair encoding;
确定针对关键字元素数据的分隔符,基于所述值元素数据和所述分隔符进行编码处理,得到第二缓存数据。Determining a delimiter for the key element data, performing encoding processing based on the value element data and the delimiter, to obtain second cache data.
在一个实施例中,所述处理器110在执行所述缓存管理方法时,还执行以下步骤:In one embodiment, when the
响应于客户端的数据查询请求,获取第二缓存数据;In response to the data query request of the client, obtain the second cache data;
对所述第二缓存数据进行目标解码处理得到第三缓存数据,将所述第三缓存数据发送至客户端;或,将所述第二缓存数据发送至客户端,以使所述客户端对所述第二缓存数据进行目标解码处理得到第三缓存数据;performing target decoding processing on the second cached data to obtain third cached data, and sending the third cached data to the client; or, sending the second cached data to the client, so that the client can performing target decoding processing on the second cached data to obtain third cached data;
其中,所述第三缓存数据为包含键值对中关键字元素数据的缓存编码数据。Wherein, the third cache data is cache coded data including key element data in the key-value pair.
在一个实施例中,所述处理器110在执行所述对所述第二缓存数据进行目标解码处理,得到第三缓存数据时,具体执行以下步骤:In one embodiment, when the
确定所述第二缓存数据对应的第二缓存关键信息,所述第二缓存关键信息包括值元素数据和分隔符;Determine second cache key information corresponding to the second cache data, where the second cache key information includes value element data and a delimiter;
基于所述值元素数据和/或所述分隔符确定关键字元素数据,基于所述值元素数据和所述关键字元素数据进行编码处理,得到第三缓存数据Determine key element data based on the value element data and/or the delimiter, perform encoding processing based on the value element data and the key element data, and obtain third cache data
在本说明书一个或多个实施例中,服务平台通过监测针对热数据和冷数据的缓存波动状态以获取当前的访问用户分布信息,基于访问用户分布信息对第一数据缓存空间和/或第二数据缓存空间进行缓存空间动态调节。基于访问用户分布信息一定程度上可预测冷热数据的缓存量的波动,可预先对相应缓存空间进行调节,提高了请求数据时缓存命中率,降低服务平台的处理压力。In one or more embodiments of this specification, the service platform obtains the current access user distribution information by monitoring the cache fluctuation status for hot data and cold data, and based on the access user distribution information, caches the first data and/or the second The data cache space is dynamically adjusted for the cache space. Based on the access user distribution information, the fluctuation of the cache volume of hot and cold data can be predicted to a certain extent, and the corresponding cache space can be adjusted in advance, which improves the cache hit rate when requesting data and reduces the processing pressure of the service platform.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机程序来指令相关的硬件来完成,所述的程序可存储于一计算机可读取存储介质中,该程序在执行时,可包括如上述各方法的实施例的流程。其中,所述的存储介质可为磁碟、光盘、只读存储记忆体或随机存储记忆体等。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the programs can be stored in a computer-readable storage medium. During execution, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory or a random access memory, and the like.
以上所揭露的仅为本申请较佳实施例而已,当然不能以此来限定本申请之权利范围,因此依本申请权利要求所作的等同变化,仍属本申请所涵盖的范围。The above disclosures are only preferred embodiments of the present application, which certainly cannot limit the scope of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope of the present application.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210907778.5ACN115334158A (en) | 2022-07-29 | 2022-07-29 | A cache management method, device, storage medium and electronic equipment |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210907778.5ACN115334158A (en) | 2022-07-29 | 2022-07-29 | A cache management method, device, storage medium and electronic equipment |
| Publication Number | Publication Date |
|---|---|
| CN115334158Atrue CN115334158A (en) | 2022-11-11 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210907778.5APendingCN115334158A (en) | 2022-07-29 | 2022-07-29 | A cache management method, device, storage medium and electronic equipment |
| Country | Link |
|---|---|
| CN (1) | CN115334158A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120315784A (en)* | 2025-06-10 | 2025-07-15 | 北京科杰科技有限公司 | A generation management method for API |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107426302A (en)* | 2017-06-26 | 2017-12-01 | 腾讯科技(深圳)有限公司 | Access scheduling method, apparatus, system, terminal, server and storage medium |
| CN107562804A (en)* | 2017-08-08 | 2018-01-09 | 上海数据交易中心有限公司 | Data buffer service system and method, terminal |
| CN108459821A (en)* | 2017-02-21 | 2018-08-28 | 中兴通讯股份有限公司 | A kind of method and device of data buffer storage |
| CN109344092A (en)* | 2018-09-11 | 2019-02-15 | 天津易华录信息技术有限公司 | A kind of method and system improving cold storing data reading speed |
| CN110442309A (en)* | 2019-07-24 | 2019-11-12 | 广东紫晶信息存储技术股份有限公司 | A kind of cold and hot method for interchanging data and system based on optical storage |
| CN112015343A (en)* | 2020-08-27 | 2020-12-01 | 杭州宏杉科技股份有限公司 | Cache space management method and device of storage volume and electronic equipment |
| CN113688160A (en)* | 2021-09-08 | 2021-11-23 | 北京沃东天骏信息技术有限公司 | Data processing method, processing device, electronic device and storage medium |
| CN113742131A (en)* | 2020-05-29 | 2021-12-03 | 伊姆西Ip控股有限责任公司 | Method, electronic device and computer program product for storage management |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108459821A (en)* | 2017-02-21 | 2018-08-28 | 中兴通讯股份有限公司 | A kind of method and device of data buffer storage |
| US20210133103A1 (en)* | 2017-02-21 | 2021-05-06 | Zte Corporation | Data caching method and apparatus |
| CN107426302A (en)* | 2017-06-26 | 2017-12-01 | 腾讯科技(深圳)有限公司 | Access scheduling method, apparatus, system, terminal, server and storage medium |
| CN107562804A (en)* | 2017-08-08 | 2018-01-09 | 上海数据交易中心有限公司 | Data buffer service system and method, terminal |
| CN109344092A (en)* | 2018-09-11 | 2019-02-15 | 天津易华录信息技术有限公司 | A kind of method and system improving cold storing data reading speed |
| CN110442309A (en)* | 2019-07-24 | 2019-11-12 | 广东紫晶信息存储技术股份有限公司 | A kind of cold and hot method for interchanging data and system based on optical storage |
| CN113742131A (en)* | 2020-05-29 | 2021-12-03 | 伊姆西Ip控股有限责任公司 | Method, electronic device and computer program product for storage management |
| CN112015343A (en)* | 2020-08-27 | 2020-12-01 | 杭州宏杉科技股份有限公司 | Cache space management method and device of storage volume and electronic equipment |
| CN113688160A (en)* | 2021-09-08 | 2021-11-23 | 北京沃东天骏信息技术有限公司 | Data processing method, processing device, electronic device and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120315784A (en)* | 2025-06-10 | 2025-07-15 | 北京科杰科技有限公司 | A generation management method for API |
| Publication | Publication Date | Title |
|---|---|---|
| US11310331B2 (en) | Optimizing user interface data caching for future actions | |
| CN108491275B (en) | Program optimization method, device, terminal and storage medium | |
| US11057490B2 (en) | Systems and methods for utilizing unused network capacity for prefetch requests | |
| US10684995B2 (en) | Storage optimization in computing devices | |
| CN109684358B (en) | Data query method and device | |
| US8874700B2 (en) | Optimizing storage of data files | |
| CN109542614B (en) | Resource allocation method, device, terminal and storage medium | |
| CN112260853B (en) | Disaster recovery switching method, device, storage medium and electronic equipment | |
| US20130179930A1 (en) | Method and system for visualizing an adaptive screen according to a terminal | |
| CN105205014A (en) | Data storage method and device | |
| EP4525444A1 (en) | Video transcoding method and apparatus, and device, storage medium and video on-demand system | |
| US20190114989A1 (en) | Systems and methods for image optimization | |
| CN106095417A (en) | A kind of prospect Application Program Interface refresh synchronization method and system | |
| CN109522131A (en) | Resource allocation method, device and equipment | |
| CN112686528A (en) | Method, apparatus, server and medium for allocating customer service resources | |
| CN112650804B (en) | Big data access method, device, system and storage medium | |
| CN115334158A (en) | A cache management method, device, storage medium and electronic equipment | |
| US20160037509A1 (en) | Techniques to reduce bandwidth usage through multiplexing and compression | |
| CN112286930A (en) | Method, device, storage medium and electronic equipment for resource sharing of redis business side | |
| CN114519158B (en) | A page display method, device, storage medium and terminal | |
| CN115730224A (en) | Service processing method, device, equipment and storage medium | |
| CN104572409A (en) | Application parameter management method and system and electronic device | |
| CN115378938A (en) | Network resource scheduling method, gateway equipment, edge and cloud data center server | |
| CN113849255B (en) | Data processing method, device and storage medium | |
| CN114928608B (en) | Method, device, equipment and storage medium for processing multimedia resources |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |