CN114860724A - Processing method, storage medium and equipment for database shared memory buffer pool - Google Patents
Processing method, storage medium and equipment for database shared memory buffer poolDownload PDFInfo
- Publication number
- CN114860724A CN114860724ACN202210451352.3ACN202210451352ACN114860724ACN 114860724 ACN114860724 ACN 114860724ACN 202210451352 ACN202210451352 ACN 202210451352ACN 114860724 ACN114860724 ACN 114860724A
- Authority
- CN
- China
- Prior art keywords
- fixed
- buffer
- buffer pool
- pool
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明涉及数据库技术，特别是涉及一种数据库共享内存缓冲池的处理方法、存储介质与设备。The invention relates to database technology, in particular to a processing method, storage medium and device for a database shared memory buffer pool.
背景技术Background technique
数据库中通常会在内存中设置一个共享缓冲池区域，其在所有服务会话间共享，为所有关系表提供缓冲服务，以避免每次读取关系表时需要去磁盘中读取。在缓冲池中读取某页面称为命中缓冲池，而去磁盘中读取页面称为未命中。众所周知，对内存区域的访问远远快于对磁盘的访问，因此有必要尽可能降低未命中的概率。A shared buffer pool area is usually set in memory in the database, which is shared among all service sessions to provide buffer services for all relational tables, so as to avoid the need to read from disk every time the relational table is read. Reading a page in the buffer pool is called a hit in the buffer pool, while reading a page from disk is called a miss. It is known that accesses to memory regions are much faster than accesses to disk, so it is necessary to reduce the probability of misses as much as possible.
然而，由于缓冲池的容量是有限的，当缓冲池耗尽时，我们不得不将一部分页面移除出缓冲池，该过程称为缓冲区置换。显然，判断哪些页面是最有可能不再被访问的页面是缓冲区置换技术的核心要点。However, since the capacity of the buffer pool is limited, when the buffer pool is exhausted, we have to remove a portion of the pages from the buffer pool, a process called buffer swapping. Obviously, judging which pages are most likely to no longer be accessed is the core point of the buffer replacement technique.
现有技术中存在一些缓冲区置换技术，例如LRU(Least Recently Used，最近最少使用)、LFU(Least Frequently Used，最不经常使用)、FIFO(First Input First Output，先入先出)等。这些缓冲区置换技术分别有各自的一些特点，分别具有各自所适应的场景下。其中最常见的置换算法为LRU，则能在大多数访问序列下均有不错的表现。但不可避免地，由于访问序列的难以预测性以及缓冲区容量本身所带来的限制，这些置换技术总会存在某些特定的访问序列，出现非最优的置换页面判断，导致非必要的缓冲区换出，进而降低访问的命中率。There are some buffer replacement technologies in the prior art, such as LRU (Least Recently Used, least recently used), LFU (Least Frequently Used, least frequently used), FIFO (First Input First Output, first in first out) and the like. These buffer replacement technologies have their own characteristics, and they have their own suitable scenarios. The most common permutation algorithm is LRU, which can perform well in most access sequences. But inevitably, due to the unpredictability of the access sequence and the limitation of the buffer capacity itself, there will always be some specific access sequences in these replacement technologies, and there will be non-optimal replacement page judgments, resulting in unnecessary buffering. The area is swapped out, thereby reducing the hit rate of the access.
理论上的最优置换算法，称为OPT(OPTimal replacement，最佳页面替换)，这种理论上的最佳算法基于能提前确定所有访问序列的假设，因此通常不适用于实际场景。The theoretical optimal replacement algorithm is called OPT (OPTimal replacement, optimal page replacement). This theoretical optimal algorithm is based on the assumption that all access sequences can be determined in advance, so it is usually not suitable for practical scenarios.
因此，现有技术的缓冲区置换技术都无法保证所有访问序列下的最优，容易出现后续可能经常访问的页面被自动置换出缓冲区的情况，在该情况下频繁的进行缓冲区置换，造成大量多余的I/O(Input/Output,输入/输出)开销，影响数据库的整体效率。Therefore, none of the buffer replacement technologies in the prior art can guarantee the optimality under all access sequences, and it is easy to occur that the pages that may be frequently accessed in the future are automatically replaced from the buffer. A large amount of redundant I/O (Input/Output, input/output) overhead affects the overall efficiency of the database.
发明内容SUMMARY OF THE INVENTION
本发明的一个目的是要提供一种减少因缓冲池置换导致的多余I/O开销。It is an object of the present invention to provide a way to reduce excess I/O overhead caused by buffer pool replacement.
本发明一个进一步的目的是要使得提高数据库的整体性能。A further object of the present invention is to improve the overall performance of the database.
特别地，本发明提供了一种数据库共享内存缓冲池的处理方法，其包括：In particular, the present invention provides a method for processing a database shared memory buffer pool, which includes:
获取用于使数据库的固定缓冲关系解除固定的指令；Get the command for unpinning the fixed buffer relationship of the database;
判断固定缓冲关系的全部页面是否已经缓存在固定缓冲池中；Determine whether all pages of the fixed buffer relationship have been cached in the fixed buffer pool;
若是，将固定缓冲关系的页面从固定缓冲池移动至普通缓冲池，其中数据库的共享内存预先开辟有固定缓冲池以及普通缓冲池，并且固定缓冲池和普通缓冲池相互独立；If so, move the pages of the fixed buffer relationship from the fixed buffer pool to the common buffer pool, in which the shared memory of the database is pre-opened with a fixed buffer pool and a common buffer pool, and the fixed buffer pool and the common buffer pool are independent of each other;
将固定缓冲关系的固定标记设置为非固定状态。Set the fixed flag of the fixed buffer relationship to the non-fixed state.
可选地，在获取用于使数据库的固定缓冲关系解除固定的指令的步骤之后还包括：解析指令，确定指令对应的固定缓冲关系；打开固定缓冲关系，并确定固定缓冲关系的固定缓冲池标记是否为固定状态；若是，则将固定缓冲池标记置为非固定状态。Optionally, after the step of acquiring the instruction for unfixing the fixed buffer relationship of the database, it further includes: parsing the instruction, and determining the fixed buffer relationship corresponding to the instruction; opening the fixed buffer relationship, and determining the fixed buffer pool mark of the fixed buffer relationship. Whether it is in a fixed state; if so, set the fixed buffer pool flag to a non-fixed state.
可选地，在固定缓冲关系的固定缓冲池标记为非固定状态的情况下，还包括：输出报错信息，以报告指令对应的关系并非固定缓冲关系。Optionally, when the fixed buffer pool of the fixed buffer relationship is marked as a non-fixed state, the method further includes: outputting error reporting information to report that the relationship corresponding to the instruction is not a fixed buffer relationship.
可选地，将固定缓冲关系的页面从固定缓冲池移动至普通缓冲池的步骤之后还包括：解除对固定缓冲关系的引用计数器的限制，使得固定缓冲关系参与普通缓冲池的缓冲置换。Optionally, after the step of moving the pages of the fixed buffer relationship from the fixed buffer pool to the common buffer pool, the step further includes: releasing the restriction on the reference counter of the fixed buffer relationship, so that the fixed buffer relationship participates in the buffer replacement of the common buffer pool.
可选地，在将固定缓冲关系的页面从固定缓冲池移动至普通缓冲池的步骤之后还包括：清理固定缓冲池的空间，并确定固定缓冲池的剩余空间，以供存储其他固定缓冲关系。Optionally, after the step of moving the pages of the fixed buffer relationship from the fixed buffer pool to the common buffer pool, the method further includes: clearing the space of the fixed buffer pool, and determining the remaining space of the fixed buffer pool for storing other fixed buffer relationships.
可选地，在获取用于使数据库的固定缓冲关系解除固定的指令的步骤之前还包括：获取缓冲池设置指令；解析缓冲池设置指令，确定出需要指定的固定缓冲关系；将固定缓冲关系的固定缓冲池标记修改为固定状态；在获取到对固定缓冲关系的访问后，启动将固定缓冲关系移入固定缓冲池的流程。Optionally, before the step of acquiring the instruction for unfixing the fixed buffer relationship of the database, it further includes: acquiring the buffer pool setting instruction; parsing the buffer pool setting instruction to determine the fixed buffer relationship that needs to be specified; The fixed buffer pool flag is modified to the fixed state; after obtaining access to the fixed buffer relationship, the process of moving the fixed buffer relationship into the fixed buffer pool is started.
可选地，在将固定缓冲关系移入固定缓冲池的步骤之后还包括：将固定缓冲关系的引用计数器置为大于等于1，并在共享内存的缓存空间进行缓冲置换的过程中，维持固定缓冲关系的引用计数器大于等于1。Optionally, after the step of moving the fixed buffer relationship into the fixed buffer pool, the method further includes: setting the reference counter of the fixed buffer relationship to be greater than or equal to 1, and maintaining the fixed buffer relationship in the process of performing buffer replacement in the cache space of the shared memory. 's reference counter is greater than or equal to 1.
可选地，固定缓冲池的容量大小通过配置文件进行预先配置。Optionally, the capacity size of the fixed buffer pool is pre-configured through a configuration file.
根据本发明的另一个方面，还提供了一种机器可读存储介质，其上存储有机器可执行程序，机器可执行程序被处理器执行时实现上述任一种的数据库共享内存缓冲池的处理方法。According to another aspect of the present invention, a machine-readable storage medium is also provided, on which a machine-executable program is stored, and when the machine-executable program is executed by a processor, any one of the above-mentioned processing of the database shared memory buffer pool is implemented method.
根据本发明的又一个方面，还提供了一种计算机设备，包括存储器、处理器及存储在存储器上并在处理器上运行的机器可执行程序，并且处理器执行机器可执行程序时实现上述任一种的数据库共享内存缓冲池的处理方法。According to yet another aspect of the present invention, a computer device is also provided, including a memory, a processor, and a machine-executable program stored in the memory and running on the processor, and the processor implements any of the above when executing the machine-executable program. A processing method of database shared memory buffer pool.
本发明的数据库共享内存缓冲池的处理方法，在数据库的共享内存的缓存空间预先开辟固定缓冲池以及普通缓冲池，其中普通缓冲池使用传统的缓冲置换算法，进行页面的普通置换。固定缓冲池用于供数据库用户指定的关系进行缓冲。在需要将关系移出固定缓冲池，实现对固定缓冲池内页面的置换时，首先确定用于解除固定的指令所对应的固定缓冲关系的全部页面是否已经缓存在固定缓冲池中，然后将固定缓冲关系的页面从固定缓冲池移动至普通缓冲池，使固定缓冲关系退出固定缓存，参与普通缓冲池的普通置换。从而为固定缓冲池腾出空闲空间，使得数据库可以灵活高效地利用缓存机制，减少了因页面置换导致的多余I/O开销。In the processing method of the database shared memory buffer pool of the present invention, a fixed buffer pool and a common buffer pool are opened in advance in the cache space of the shared memory of the database, wherein the common buffer pool uses a traditional buffer replacement algorithm to perform common page replacement. The fixed buffer pool is used for buffering of relations specified by the database user. When it is necessary to move the relationship out of the fixed buffer pool and replace the pages in the fixed buffer pool, first determine whether all the pages of the fixed buffer relationship corresponding to the unfixed instruction have been cached in the fixed buffer pool, and then replace the fixed buffer relationship The pages of the fixed buffer pool are moved from the fixed buffer pool to the common buffer pool, so that the fixed buffer relationship exits the fixed cache and participates in the common replacement of the common buffer pool. This frees up free space for the fixed buffer pool, enabling the database to utilize the caching mechanism flexibly and efficiently, reducing redundant I/O overhead caused by page replacement.
进一步地，本发明的数据库共享内存缓冲池的处理方法，固定缓冲关系根据解除固定的指令退出固定缓冲池，在数据库用户正确进行设置的情况下，可以有效提升数据库的整体性能。Further, in the processing method of the database shared memory buffer pool of the present invention, the fixed buffer relationship exits the fixed buffer pool according to the unfixed instruction, and the overall performance of the database can be effectively improved when the database user sets correctly.
更进一步地，本发明的数据库共享内存缓冲池的处理方法，为关系增加了固定缓冲池标记和固定标记，以标识标识固定缓冲关系的设置状态和缓冲状态，为固定缓冲关系移入固定缓冲池、移出固定缓冲池、在普通缓冲池内的等待等操作提供了支持。Furthermore, in the method for processing the database shared memory buffer pool of the present invention, a fixed buffer pool mark and a fixed mark are added to the relationship, so as to identify the setting state and buffer state of the fixed buffer relationship, and move the fixed buffer relationship into the fixed buffer pool, Operations such as moving out of a fixed buffer pool, waiting in a normal buffer pool, etc. are supported.
根据下文结合附图对本发明具体实施例的详细描述，本领域技术人员将会更加明了本发明的上述以及其他目的、优点和特征。The above and other objects, advantages and features of the present invention will be more apparent to those skilled in the art from the following detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings.
附图说明Description of drawings
后文将参照附图以示例性而非限制性的方式详细描述本发明的一些具体实施例。附图中相同的附图标记标示了相同或类似的部件或部分。本领域技术人员应该理解，这些附图未必是按比例绘制的。附图中：Hereinafter, some specific embodiments of the present invention will be described in detail by way of example and not limitation with reference to the accompanying drawings. The same reference numbers in the figures designate the same or similar parts or parts. It will be understood by those skilled in the art that the drawings are not necessarily to scale. In the attached picture:
图1是根据本发明一个实施例的数据库共享内存缓冲池的处理方法的示意流程图；1 is a schematic flowchart of a method for processing a database shared memory buffer pool according to an embodiment of the present invention;
图2根据本发明一个实施例的数据库共享内存缓冲池的处理方法中数据库共享内存的示意图；2 is a schematic diagram of a database shared memory in a method for processing a database shared memory buffer pool according to an embodiment of the present invention;
图3是根据本发明一个实施例的数据库共享内存缓冲池的处理方法取消关系固定的流程示意图；3 is a schematic flow chart of canceling a fixed relationship in a method for processing a database shared memory buffer pool according to an embodiment of the present invention;
图4是根据本发明一个实施例的数据库共享内存缓冲池的处理方法进行关系固定的流程示意图；FIG. 4 is a schematic flowchart of a relationship fixing process in a method for processing a database shared memory buffer pool according to an embodiment of the present invention;
图5是根据本发明一个实施例的机器可读存储介质的示意图；以及FIG. 5 is a schematic diagram of a machine-readable storage medium according to one embodiment of the present invention; and
图6是根据本发明一个实施例的计算机设备的示意图。Figure 6 is a schematic diagram of a computer device according to one embodiment of the present invention.
具体实施方式Detailed ways
图1是根据本发明一个实施例的数据库共享内存缓冲池的处理方法的示意流程图，图2根据本发明一个实施例的数据库共享内存缓冲池的处理方法中数据库共享内存的示意图。数据库共享内存缓冲池的处理方法一般性地可包括：1 is a schematic flowchart of a method for processing a database shared memory buffer pool according to an embodiment of the present invention, and FIG. 2 is a schematic diagram of a database shared memory in a method for processing a database shared memory buffer pool according to an embodiment of the present invention. The processing method of the database shared memory buffer pool can generally include:
步骤S102，获取用于使数据库的固定缓冲关系解除固定的指令；Step S102, obtaining an instruction for unfixing the fixed buffer relationship of the database;
步骤S104，判断固定缓冲关系的全部页面是否已经缓存在固定缓冲池中；Step S104, judging whether all pages of the fixed buffer relationship have been cached in the fixed buffer pool;
步骤S106，若是，将固定缓冲关系的页面从固定缓冲池移动至普通缓冲池，其中数据库的共享内存预先开辟有固定缓冲池以及普通缓冲池，并且固定缓冲池和普通缓冲池相互独立；Step S106, if yes, move the pages of the fixed buffer relationship from the fixed buffer pool to the common buffer pool, wherein the shared memory of the database is pre-opened with a fixed buffer pool and a common buffer pool, and the fixed buffer pool and the common buffer pool are independent of each other;
步骤S108，将固定缓冲关系的固定标记设置为非固定状态。Step S108, setting the fixed flag of the fixed buffer relationship to a non-fixed state.
在本实施例的数据库中，共享内存的缓冲空间预先开辟有普通缓冲池以及固定缓冲池。普通缓冲池使用传统的缓冲置换算法，进行页面的普通置换，例如采用LRU、LFU等置换算法。固定缓冲池用于供数据库用户指定的关系进行缓冲。固定缓冲池与普通缓冲池互相独立，在关系移入固定缓冲池后，该固定缓冲关系不再参与普通页面置换，避免了因页面置换导致的多余I/O开销。In the database of this embodiment, the buffer space of the shared memory is pre-opened with a common buffer pool and a fixed buffer pool. The ordinary buffer pool uses the traditional buffer replacement algorithm to perform ordinary page replacement, such as LRU, LFU and other replacement algorithms. The fixed buffer pool is used for buffering of relations specified by the database user. The fixed buffer pool and the common buffer pool are independent of each other. After the relationship is moved into the fixed buffer pool, the fixed buffer relationship no longer participates in the normal page replacement, which avoids the redundant I/O overhead caused by page replacement.
在需要将关系移出固定缓冲池，实现对固定缓冲池内页面的置换时，首先确定用于解除固定的指令所对应的固定缓冲关系的全部页面是否已经缓存在固定缓冲池中，然后将固定缓冲关系的页面从固定缓冲池移动至普通缓冲池，使固定缓冲关系退出固定缓存，参与普通缓冲池的普通置换。从而为固定缓冲池腾出空闲空间，使得数据库可以灵活高效地利用缓存机制。When it is necessary to move the relationship out of the fixed buffer pool and replace the pages in the fixed buffer pool, first determine whether all the pages of the fixed buffer relationship corresponding to the unfixed instruction have been cached in the fixed buffer pool, and then replace the fixed buffer relationship The pages of the fixed buffer pool are moved from the fixed buffer pool to the common buffer pool, so that the fixed buffer relationship exits the fixed cache and participates in the common replacement of the common buffer pool. This frees up free space for the fixed buffer pool, so that the database can utilize the caching mechanism flexibly and efficiently.
固定缓冲池的大小可以通过配置文件配置，且与普通缓冲池独立存在。一个数据库页面的缓存要么存在于固定缓冲池中，要么存在于普通缓冲池中。数据库可以通过一个哈希表建立从磁盘页面号到缓冲池具体位置的映射，缓冲池具体位置则包含了所在缓冲池以及相对于缓冲池基地址的偏移，则通过缓冲池基地址与上述偏移量就可以确定页面在缓冲池具体位置。The size of the fixed buffer pool can be configured through the configuration file and exists independently of the normal buffer pool. A cache of database pages exists either in the fixed buffer pool or in the normal buffer pool. The database can establish a mapping from the disk page number to the specific location of the buffer pool through a hash table. The specific location of the buffer pool includes the buffer pool and the offset relative to the base address of the buffer pool. The amount of shift can determine the specific location of the page in the buffer pool.
布置数据库的磁盘上可以存储数据文件21、日志文件22、配置文件23，数据库的共享内存10中可以开辟有普通缓冲池11、固定缓冲池12、其他共享内存。如本领域技术人员所习知的，磁盘的读取速度明显慢于内存的读取速度。本实施例的数据库，在共享内存10内开辟出普通缓冲池11、固定缓冲池12。普通缓冲池11可以用于页面的正常缓存，固定缓冲池12可以按照数据库用户的设置使指定的关系被固定缓存。Data files 21 , log files 22 , and
缓冲池内关系页面的页面头上可以设置记录该页面的相关状态的存储位，记录的信息可以包括页面是否存在于固定缓冲池中、引用计数器、是否可以被置换和清理等。例如关系页面的页面头上可以设置固定标记以及固定缓冲池标记，其中固定标记用于标记该关系已经移入固定缓冲池，固定缓冲池标记用于标记该关系已经被指定为固定缓冲关系。The page header of the relational page in the buffer pool may set a storage bit that records the relevant state of the page, and the recorded information may include whether the page exists in the fixed buffer pool, the reference counter, whether it can be replaced and cleaned, and so on. For example, a fixed mark and a fixed buffer pool mark may be set on the page header of the relation page, wherein the fixed mark is used to mark that the relation has been moved into the fixed buffer pool, and the fixed buffer pool mark is used to mark that the relation has been designated as a fixed buffer relation.
如果步骤S104的判断结果为固定缓冲关系的全部页面并未缓存在固定缓冲池中，则表明固定缓冲关系并未存入固定缓冲池，在该情况下，不需要进行取消固定的操作。If the determination result in step S104 is that all pages of the fixed buffer relationship are not cached in the fixed buffer pool, it indicates that the fixed buffer relationship is not stored in the fixed buffer pool, and in this case, the unfixing operation is not required.
步骤S102的指令可以使用SQL(Structured Query Language，结构化查询语言)，并包含需要取消固定的固定缓冲关系的信息，例如关系标识、名称等。在步骤S102之后，还可以解析指令，确定指令对应的固定缓冲关系；打开固定缓冲关系，并确定固定缓冲关系的固定缓冲池标记是否为固定状态；若是，则将固定缓冲池标记置为非固定状态。通过修改固定缓冲池标记，将关系标识为取消固定的状态。在固定缓冲关系的固定缓冲池标记为非固定状态的情况下，还可以包括输出报错信息，以报告指令对应的关系并非固定缓冲关系。也即在取消固定的指令所指示的关系，自身并未指示为固定缓冲关系的情况下，指令可能存在错误，此时利用报错信息，向数据库用户报告指令对应的关系并非固定缓冲关系，以便进行查证。The instruction in step S102 may use SQL (Structured Query Language, Structured Query Language), and includes information that needs to cancel the fixed fixed buffer relationship, such as a relationship identifier, name, and the like. After step S102, the instruction can also be parsed to determine the fixed buffer relationship corresponding to the instruction; the fixed buffer relationship is opened, and it is determined whether the fixed buffer pool flag of the fixed buffer relationship is in a fixed state; if so, the fixed buffer pool flag is set to non-fixed state. Identify the relationship as unpinned by modifying the pinned buffer pool flag. In the case where the fixed buffer pool of the fixed buffer relationship is marked as a non-fixed state, an error message may also be output to report that the relationship corresponding to the instruction is not a fixed buffer relationship. That is to say, if the relationship indicated by the fixed command is canceled, but it does not indicate a fixed buffer relationship, the command may have errors. At this time, the error message is used to report to the database user that the relationship corresponding to the command is not a fixed buffer relationship, so that the Verify.
步骤S106将固定缓冲关系的页面从固定缓冲池移动至普通缓冲池的步骤之后还包括：解除对固定缓冲关系的引用计数器的限制，使得固定缓冲关系参与普通缓冲池的缓冲置换。另外在步骤S106之后还可以清理固定缓冲池的空间，并确定固定缓冲池的剩余空间，以供存储其他固定缓冲关系。Step S106, after the step of moving the pages of the fixed buffer relationship from the fixed buffer pool to the common buffer pool, further includes: releasing the restriction on the reference counter of the fixed buffer relationship, so that the fixed buffer relationship participates in the buffer replacement of the common buffer pool. In addition, after step S106, the space of the fixed buffer pool may be cleared, and the remaining space of the fixed buffer pool may be determined for storing other fixed buffer relationships.
图3是根据本发明一个实施例的数据库共享内存缓冲池的处理方法取消关系固定的流程示意图，对一个数据库关系取消固定缓存的过程包括：3 is a schematic flow chart of canceling a fixed relationship in a processing method for a database shared memory buffer pool according to an embodiment of the present invention, and the process of canceling a fixed cache for a database relationship includes:
步骤S302，获取用于使数据库的固定缓冲关系解除固定的指令；Step S302, obtaining an instruction for unfixing the fixed buffer relationship of the database;
步骤S304，解析指令，确定并打开指令对应的固定缓冲关系；Step S304, parse the instruction, determine and open the fixed buffer relationship corresponding to the instruction;
步骤S306，确定固定缓冲关系的固定缓冲池标记是否为固定状态；Step S306, determining whether the fixed buffer pool mark of the fixed buffer relationship is in a fixed state;
步骤S308，在固定缓冲关系的固定缓冲池标记为非固定状态的情况下，还包括：输出报错信息，以报告指令对应的关系并非固定缓冲关系；Step S308, when the fixed buffer pool of the fixed buffer relationship is marked as a non-fixed state, further comprising: outputting error reporting information to report that the relationship corresponding to the instruction is not a fixed buffer relationship;
步骤S310，判断固定缓冲关系的全部页面是否已经缓存在固定缓冲池中；Step S310, judging whether all pages of the fixed buffer relationship have been cached in the fixed buffer pool;
步骤S312，将固定缓冲关系的页面从固定缓冲池移动至普通缓冲池；Step S312, moving the pages of the fixed buffer relationship from the fixed buffer pool to the common buffer pool;
步骤S314，将固定缓冲关系的固定标记设置为非固定状态；Step S314, setting the fixed flag of the fixed buffer relationship to a non-fixed state;
步骤S316，解除对固定缓冲关系的引用计数器的限制，使得固定缓冲关系参与普通缓冲池的缓冲置换；Step S316, lifting the restriction on the reference counter of the fixed buffer relationship, so that the fixed buffer relationship participates in the buffer replacement of the common buffer pool;
步骤S318，清理固定缓冲池的空间，并确定固定缓冲池的剩余空间，以供存储其他固定缓冲关系；Step S318, clearing the space of the fixed buffer pool, and determining the remaining space of the fixed buffer pool for storing other fixed buffer relationships;
本实施例的数据库共享内存缓冲池的处理方法还提供了对固定缓冲关系进行固定的过程。也即在在获取用于使数据库的固定缓冲关系解除固定的指令的步骤之前还包括：获取缓冲池设置指令；解析缓冲池设置指令，确定出需要指定的固定缓冲关系；将固定缓冲关系的固定缓冲池标记修改为固定状态；在获取到对固定缓冲关系的访问后，启动将固定缓冲关系移入固定缓冲池的流程。The processing method for the database shared memory buffer pool in this embodiment also provides a process of fixing the fixed buffer relationship. That is, before the step of acquiring the instruction for unfixing the fixed buffer relationship of the database, it further includes: acquiring the buffer pool setting instruction; parsing the buffer pool setting instruction to determine the fixed buffer relationship that needs to be specified; The buffer pool mark is modified to the fixed state; after obtaining access to the fixed buffer relationship, the process of moving the fixed buffer relationship into the fixed buffer pool is started.
在将固定缓冲关系移入固定缓冲池的步骤之后还可以包括：将固定缓冲关系的引用计数器置为大于等于1，并在共享内存的缓存空间进行缓冲置换的过程中，维持固定缓冲关系的引用计数器大于等于1。After the step of moving the fixed buffer relationship into the fixed buffer pool, the method may further include: setting the reference counter of the fixed buffer relationship to be greater than or equal to 1, and maintaining the reference counter of the fixed buffer relationship in the process of performing buffer replacement in the cache space of the shared memory greater than or equal to 1.
当一个关系指定使用固定缓冲池时，该关系的当前所有页面以及后续扩展的所有页面均会被加入固定缓冲池中，除非固定缓冲池的容量已经耗尽。在固定缓冲池的容量耗尽的情况下，则页面仍然被存储在普通缓冲池中，直到固定缓冲池出现额外空间后，该关系的下一次被访问时，才会重新固定缓存这部分页面。When a relationship specifies the use of a fixed buffer pool, all pages of the relationship and all pages of subsequent extensions are added to the fixed buffer pool, unless the fixed buffer pool is exhausted. When the capacity of the fixed buffer pool is exhausted, the page is still stored in the common buffer pool, and the page will be fixed and cached again when the relationship is accessed next time after the fixed buffer pool has additional space.
图4是根据本发明一个实施例的数据库共享内存缓冲池的处理方法进行关系固定的流程示意图，对一个数据库关系执行固定缓存的过程包括：FIG. 4 is a schematic flowchart of a relationship fixing process for a database shared memory buffer pool processing method according to an embodiment of the present invention. The process of performing fixed caching on a database relationship includes:
步骤S402，获取数据库用户输入的缓冲池设置指令，该缓冲池设置指令用于指定数据库的一个关系使用固定缓冲池；Step S402, obtaining the buffer pool setting instruction input by the database user, and the buffer pool setting instruction is used to specify that a relation of the database uses a fixed buffer pool;
步骤S404，数据库解析缓冲池设置指令，确定需要指定的固定缓冲关系；Step S404, the database parses the buffer pool setting instruction, and determines the fixed buffer relationship that needs to be specified;
步骤S406，打开指定的固定缓冲关系，将其固定缓冲池标记置为固定状态；Step S406, open the specified fixed buffer relationship, and set its fixed buffer pool mark to a fixed state;
步骤S408，获取对数据库的访问，确定该访问是上述固定缓冲关系被指定后的首次访问；Step S408, obtaining access to the database, and determining that the access is the first access after the above-mentioned fixed buffer relationship is specified;
步骤S410，将该固定缓冲关系的所有页面读取到共享内存的普通缓冲池中；通过执行步骤S412至步骤S418，尝试将固定缓冲关系固定到固定缓冲池中；Step S410, read all pages of the fixed buffer relationship into the common buffer pool of the shared memory; try to fix the fixed buffer relationship into the fixed buffer pool by executing steps S412 to S418;
步骤S412，判断固定缓冲池是否存在剩余空间；Step S412, judging whether there is remaining space in the fixed buffer pool;
步骤S414，若存在剩余空间，则将上述读取到共享内存的普通缓冲池中的页面移动至固定缓冲池中，并将固定标记设置为固定状态，将引用计数器设置为1；Step S414, if there is remaining space, move the page read into the common buffer pool of the shared memory to the fixed buffer pool, set the fixed flag to a fixed state, and set the reference counter to 1;
步骤S416，若不存在剩余空间，放弃固定缓冲池的操作；Step S416, if there is no remaining space, abandon the operation of fixing the buffer pool;
步骤S418，后续缓冲池置换流程中，对于固定标记为固定状态的页面，其引用计数器限制为不会被减少到1以下，避免该页面被置换。普通缓冲池模中的页面正常参与页面置换。Step S418 , in the subsequent buffer pool replacement process, for a page that is fixed and marked as a fixed state, its reference counter is limited to not be reduced below 1, so as to prevent the page from being replaced. Pages in the normal buffer pool mode normally participate in page replacement.
通过上述步骤，数据库用户可以基于已知的访问频率信息设置数据库的置换算法，尤其是可以针对数据库大小较小且访问频率较高的表或索引，为其赋予较高的内存缓冲区粘性。如果指定了合适的高频访问表或索引，则可以有效降低由缓冲区置换带来的多余I/O开销，并提高数据库整体性能。Through the above steps, the database user can set the database replacement algorithm based on the known access frequency information, especially for tables or indexes with small database size and high access frequency, which can be assigned higher memory buffer stickiness. If a suitable high-frequency access table or index is specified, the redundant I/O overhead caused by buffer replacement can be effectively reduced, and the overall performance of the database can be improved.
本实施例还提供了一种机器可读存储介质和计算机设备。图5是根据本发明一个实施例的机器可读存储介质40的示意图，图6是根据本发明一个实施例的计算机设备50的示意图。This embodiment also provides a machine-readable storage medium and a computer device. FIG. 5 is a schematic diagram of a machine-readable storage medium 40 according to an embodiment of the present invention, and FIG. 6 is a schematic diagram of a computer device 50 according to an embodiment of the present invention.
机器可读存储介质40其上存储有机器可执行程序41，机器可执行程序41被处理器执行时实现上述任一实施例的数据库共享内存缓冲池的处理方法。The machine-readable storage medium 40 stores a machine-executable program 41 thereon, and when the machine-executable program 41 is executed by the processor, implements the processing method of the database shared memory buffer pool in any of the foregoing embodiments.
计算机设备50可以包括存储器520、处理器510及存储在存储器520上并在处理器510上运行的机器可执行程序41，并且处理器510执行机器可执行程序41时实现上述任一实施例的数据库共享内存缓冲池的处理方法。The computer device 50 may include a memory 520, a processor 510, and a machine-executable program 41 stored on the memory 520 and running on the processor 510, and the processor 510 executing the machine-executable program 41 implements the database of any of the above-described embodiments How to handle the shared memory buffer pool.
需要说明的是，在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何机器可读存储介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as an ordered listing of executable instructions for implementing the logical functions, and may be embodied in any machine in a readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with These instructions are used to execute a system, apparatus or device.
就本实施例的描述而言，机器可读存储介质40可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质40甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。For the purposes of the description of this embodiment, machine-readable storage medium 40 may be any instruction execution system, apparatus or apparatus that can contain, store, communicate, propagate, or transmit the program for use in or in conjunction with such instruction execution system, apparatus, or apparatus. installation. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). Additionally, the computer readable medium 40 may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, and then edited, interpreted or otherwise as necessary. Process in a suitable manner to obtain the program electronically and then store it in computer memory.
应当理解，本发明的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system.
计算机设备50可以是例如服务器、台式计算机、笔记本式计算机、平板计算机或智能手机。在一些示例中，计算机设备50可以是云计算节点。计算机设备50可以在由计算机系统执行的计算机系统可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括执行特定的任务或者实现特定的抽象数据类型的例程、程序、目标程序、组件、逻辑、数据结构等。计算机设备50可以在通过通信网络链接的远程处理设备执行任务的分布式云计算环境中实施。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算系统存储介质上。Computer device 50 may be, for example, a server, desktop computer, notebook computer, tablet computer, or smartphone. In some examples, computer device 50 may be a cloud computing node. Computer device 50 may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, object programs, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer device 50 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.
计算机设备50可以包括适于执行存储的指令的处理器510、在操作期间为所述指令的操作提供临时存储空间的存储器520。处理器510可以是单核处理器、多核处理器、计算集群或任何数量的其他配置。存储器520可以包括随机存取存储器(RAM)、只读存储器、闪存或任何其他合适的存储系统。Computer device 50 may include a processor 510 adapted to execute stored instructions, a memory 520 that provides temporary storage space for the operation of the instructions during operation. Processor 510 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Memory 520 may include random access memory (RAM), read only memory, flash memory, or any other suitable storage system.
处理器510可以通过系统互连(例如PCI、PCI-Express等)连接到适于将计算机设备50连接到一个或多个I/O设备(输入/输出设备)的I/O接口(输入/输出接口)。I/O设备可以包括例如键盘和指示设备，其中指示设备可以包括触摸板或触摸屏等等。I/O设备可以是计算机设备50的内置组件，或者可以是外部连接到计算设备的设备。Processor 510 may be connected through a system interconnect (eg, PCI, PCI-Express, etc.) to I/O interfaces (input/output devices) suitable for connecting computer device 50 to one or more I/O devices (input/output devices). interface). I/O devices may include, for example, keyboards and pointing devices, which may include touch pads or touch screens, and the like. The I/O device may be a built-in component of computer device 50, or may be a device externally connected to the computing device.
处理器510也可以通过系统互连链接到适于将计算机设备50连接到显示设备的显示接口。显示设备可以包括作为计算机设备50的内置组件的显示屏。显示设备还可以包括外部连接到计算机设备50的计算机监视器、电视机或投影仪等。此外，网络接口控制器(network interface controller，NIC)可以适于通过系统互连将计算机设备50连接到网络。在一些实施例中，NIC可以使用任何合适的接口或协议(诸如因特网小型计算机系统接口等)来传输数据。网络可以是蜂窝网络、无线电网络、广域网(WAN))、局域网(LAN)或因特网等等。远程设备可以通过网络连接到计算设备。Processor 510 may also be linked through a system interconnect to a display interface suitable for connecting computer device 50 to a display device. The display device may include a display screen as a built-in component of the computer device 50 . The display device may also include a computer monitor, a television or a projector, etc. externally connected to the computer device 50 . Additionally, a network interface controller (NIC) may be adapted to connect the computer device 50 to the network through a system interconnect. In some embodiments, the NIC may use any suitable interface or protocol (such as the Internet Small Computer System Interface, etc.) to transmit data. The network may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, among others. The remote device can be connected to the computing device through a network.
本实施例提供的流程图并不旨在指示方法的操作将以任何特定的顺序执行，或者方法的所有操作都包括在所有的每种情况下。此外，方法可以包括附加操作。在本实施例方法提供的技术思路的范围内，可以对上述方法进行附加的变化。The flowcharts provided in this embodiment are not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in all cases. Furthermore, methods may include additional operations. Within the scope of the technical idea provided by the method in this embodiment, additional changes may be made to the above method.
至此，本领域技术人员应认识到，虽然本文已详尽示出和描述了本发明的多个示例性实施例，但是，在不脱离本发明精神和范围的情况下，仍可根据本发明公开的内容直接确定或推导出符合本发明原理的许多其他变型或修改。因此，本发明的范围应被理解和认定为覆盖了所有这些其他变型或修改。By now, those skilled in the art will recognize that although various exemplary embodiments of the present invention have been shown and described in detail herein, the present invention may still be implemented in accordance with the present disclosure without departing from the spirit and scope of the present invention. The content directly determines or derives many other variations or modifications consistent with the principles of the invention. Accordingly, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210451352.3ACN114860724B (en) | 2022-04-26 | 2022-04-26 | Database shared memory buffer pool processing method, storage medium and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210451352.3ACN114860724B (en) | 2022-04-26 | 2022-04-26 | Database shared memory buffer pool processing method, storage medium and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114860724Atrue CN114860724A (en) | 2022-08-05 |
| CN114860724B CN114860724B (en) | 2025-06-03 |
Family
ID=82632676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210451352.3AActiveCN114860724B (en) | 2022-04-26 | 2022-04-26 | Database shared memory buffer pool processing method, storage medium and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114860724B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050050443A1 (en)* | 2003-08-26 | 2005-03-03 | International Business Machines Corporation | System and method for starting a buffer pool |
| US20080140937A1 (en)* | 2006-12-12 | 2008-06-12 | Sybase, Inc. | System and Methodology Providing Multiple Heterogeneous Buffer Caches |
| CN103365794A (en)* | 2012-03-28 | 2013-10-23 | 国际商业机器公司 | Data processing method and system |
| US20170103109A1 (en)* | 2013-06-05 | 2017-04-13 | International Business Machines Corporation | Method, program, and system for processing space-time database |
| CN110489425A (en)* | 2019-08-26 | 2019-11-22 | 上海达梦数据库有限公司 | A kind of data access method, device, equipment and storage medium |
| CN112948286A (en)* | 2019-12-10 | 2021-06-11 | 阿里巴巴集团控股有限公司 | Data caching method and device, electronic equipment and computer readable medium |
- 2022
- 2022-04-26CNCN202210451352.3Apatent/CN114860724B/enactiveActive
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050050443A1 (en)* | 2003-08-26 | 2005-03-03 | International Business Machines Corporation | System and method for starting a buffer pool |
| US20080140937A1 (en)* | 2006-12-12 | 2008-06-12 | Sybase, Inc. | System and Methodology Providing Multiple Heterogeneous Buffer Caches |
| CN103365794A (en)* | 2012-03-28 | 2013-10-23 | 国际商业机器公司 | Data processing method and system |
| US20170103109A1 (en)* | 2013-06-05 | 2017-04-13 | International Business Machines Corporation | Method, program, and system for processing space-time database |
| CN110489425A (en)* | 2019-08-26 | 2019-11-22 | 上海达梦数据库有限公司 | A kind of data access method, device, equipment and storage medium |
| CN112948286A (en)* | 2019-12-10 | 2021-06-11 | 阿里巴巴集团控股有限公司 | Data caching method and device, electronic equipment and computer readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114860724B (en) | 2025-06-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210089470A1 (en) | Address translation methods and systems | |
| US8122216B2 (en) | Systems and methods for masking latency of memory reorganization work in a compressed memory system | |
| US7472252B2 (en) | Merging identical memory pages | |
| JP4791458B2 (en) | System and method for maintaining objects in a lookup cache | |
| US8185692B2 (en) | Unified cache structure that facilitates accessing translation table entries | |
| US8370604B2 (en) | Method and system for caching attribute data for matching attributes with physical addresses | |
| CN113641596B (en) | Cache management method, cache management device and processor | |
| CN115061955B (en) | Processor, electronic device, address translation method and cache page table entry method | |
| US7103747B2 (en) | Memory table and memory manager for use in managing memory | |
| US20170147508A1 (en) | Device, system and method of accessing data stored in a memory | |
| US9003127B2 (en) | Storing data in a system memory for a subsequent cache flush | |
| CN114860724A (en) | Processing method, storage medium and equipment for database shared memory buffer pool | |
| CN115080464B (en) | Data processing method and data processing device | |
| CN117033446A (en) | Deletion method, storage medium and computer equipment for database execution plan cache | |
| CN114880356B (en) | Processing method, storage medium and equipment for database shared memory buffer pool | |
| CN109165172B (en) | Cache data processing method and related equipment | |
| US7421536B2 (en) | Access control method, disk control unit and storage apparatus | |
| CN112673359B (en) | A data processing method and device | |
| CN114791913B (en) | Shared memory buffer pool processing method, storage medium and equipment for database | |
| CN114860723A (en) | Database shared memory buffer pool processing method, storage medium and device | |
| US20070150653A1 (en) | Processing of cacheable streaming data | |
| US10977176B2 (en) | Prefetching data to reduce cache misses | |
| CN119719165A (en) | Data query method, device, equipment and storage medium | |
| CN117056378A (en) | Synchronization method of cache data and global instance data and related equipment | |
| CN116595010A (en) | Database index optimization method, storage medium and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | Country or region after:China Address after:100102 201, 2 / F, 101, No. 5 building, No. 7 Rongda Road, Chaoyang District, Beijing Applicant after:China Electronics Technology Group Jincang (Beijing) Technology Co.,Ltd. Address before:100102 201, 2 / F, 101, No. 5 building, No. 7 Rongda Road, Chaoyang District, Beijing Applicant before:BEIJING KINGBASE INFORMATION TECHNOLOGIES Inc. Country or region before:China | |
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |