Movatterモバイル変換


[0]ホーム

URL:


CN119248808B - Database table structure modification method and device, electronic equipment and storage medium - Google Patents

Database table structure modification method and device, electronic equipment and storage medium
Download PDF

Info

Publication number
CN119248808B
CN119248808BCN202411783770.8ACN202411783770ACN119248808BCN 119248808 BCN119248808 BCN 119248808BCN 202411783770 ACN202411783770 ACN 202411783770ACN 119248808 BCN119248808 BCN 119248808B
Authority
CN
China
Prior art keywords
data
lock
target data
database table
modification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202411783770.8A
Other languages
Chinese (zh)
Other versions
CN119248808A (en
Inventor
殷骏
杨蔚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Huaku Data Technology Co ltd
Original Assignee
Jiangsu Huaku Data Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Huaku Data Technology Co ltdfiledCriticalJiangsu Huaku Data Technology Co ltd
Priority to CN202411783770.8ApriorityCriticalpatent/CN119248808B/en
Publication of CN119248808ApublicationCriticalpatent/CN119248808A/en
Application grantedgrantedCritical
Publication of CN119248808BpublicationCriticalpatent/CN119248808B/en
Activelegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Classifications

Landscapes

Abstract

Translated fromChinese

本发明提供了一种数据库表结构修改方法及装置、电子设备和存储介质,可用于数据库技术领域。数据库表结构修改方法包括:根据第一数据组和第二数据组构建第一目标数据表,第一数据组包括在修改之前原数据库表中的基础数据,第二数据组包括在修改过程中在原数据库表中产生的增量数据;在检测到对第一目标数据表的读操作的情况下,对第一目标数据表添加第一锁,和/或在第二数据组的数据量满足预设数值条件的情况下,对第一目标数据表添加第二锁;对第一目标数据表中的元数据进行修改,得到第二目标数据表。本发明方法通过在第一目标数据表上添加第一锁和第二锁,可以防止在数据表结构修改过程中发生数据不一致的情况,确保数据的完整性和准确性。

The present invention provides a method and device for modifying the structure of a database table, an electronic device, and a storage medium, which can be used in the field of database technology. The method for modifying the structure of a database table includes: constructing a first target data table according to a first data group and a second data group, the first data group includes the basic data in the original database table before modification, and the second data group includes the incremental data generated in the original database table during the modification process; when a read operation on the first target data table is detected, adding a first lock to the first target data table, and/or when the amount of data in the second data group meets a preset numerical condition, adding a second lock to the first target data table; modifying the metadata in the first target data table to obtain a second target data table. By adding the first lock and the second lock to the first target data table, the method of the present invention can prevent data inconsistency from occurring during the modification of the data table structure, thereby ensuring the integrity and accuracy of the data.

Description

Database table structure modification method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for modifying a database table structure, an electronic device, and a storage medium.
Background
Many database systems have database structure modification operations that are mutually exclusive of database read and write operations, and all old data is migrated to a new structure after the database structure is modified, resulting in significant performance consumption over a period of time. So that most database system administrators currently modify the database structure only at night or with very little traffic.
Therefore, a method for ensuring the availability of the database and the consistency of the data is needed, wherein the read-write operation of the database can be continued when the database structure is modified.
Disclosure of Invention
In view of the above, the present invention provides a database table structure modification method and apparatus, an electronic device, and a storage medium.
According to a first aspect of the invention, a database table structure modification method is provided, which comprises the steps of responding to a modification instruction of an original database table structure, constructing a first target data table according to a first data group and a second data group, wherein the first data group comprises basic data in the original database table before modification, the second data group comprises incremental data generated in the original database table in the modification process, the table structures of the first target data table and the original database table are different, adding a first lock to the first target data table when a read operation of the first target data table is detected, and/or adding a second lock to the first target data table when the data amount of the second data group meets a preset numerical condition, and modifying metadata in the first target data table added with the first lock and/or the second lock to obtain a second target data table.
Optionally, the first target data table comprises a first lock, a second lock and a plurality of locks except the second lock of the first lock, the first lock and the second lock are mutually exclusive, the first lock coexists with the other plurality of locks, and the second lock and the first lock and the other plurality of locks are mutually exclusive.
Optionally, the first lock is a multi-version shared lock and the second lock is a multi-version exclusive lock.
Optionally, the first data set includes pre-image data of the base data in the original database table, and the generation of the pre-image data is responsive to an operation of modifying the structure of the original database table.
Optionally, constructing the first target data table according to the first data group and the second data group comprises constructing a third target data table with a different table structure from the original database table according to a modification target preset by a user, wherein the third target data table does not have any data row, constructing a basic target data table according to the first data group and the third target data table, and constructing the first target data table on the basis of the basic target data table according to the second data group and the corresponding incremental data operation log.
Optionally, the preset numerical condition includes that the number of corresponding incremental data operation logs is less than or equal to 5.
The second aspect of the invention provides a database table structure modifying device, which comprises a constructing module and a locking module, wherein the constructing module is used for responding to a modifying instruction of an original database table structure, constructing a first target data table according to a first data group and a second data group, the first data group comprises basic data in the original database table before modification, the second data group comprises incremental data generated in the original database table in the modifying process, the table structures of the first target data table and the original database table are different, the locking module is used for adding a first lock to the first target data table when a reading operation of the first target data table is detected, and/or adding a second lock to the first target data table when the data amount of the second data group meets a preset numerical condition, and the modifying module is used for modifying metadata in the first target data table added with the first lock and/or the second lock to obtain a second target data table.
A third aspect of the invention provides an electronic device comprising one or more processors and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.
A fourth aspect of the invention also provides a computer readable storage medium having stored thereon a computer program or instructions which when executed by a processor performs the steps of the above method.
The fifth aspect of the invention also provides a computer program product comprising a computer program or instructions which, when executed by a processor, carries out the steps of the method described above.
According to the invention, the first lock and the second lock are added on the first target data table, so that the condition of inconsistent data in the data migration and modification process can be prevented, and the integrity and accuracy of the data are ensured. The method has the advantages that when the second lock is added, namely when the metadata is started to be modified, is determined by presetting numerical conditions, can be flexibly adjusted according to the load and performance requirements of the system so as to adapt to different workloads, and simultaneously controls the modification process of the whole metadata at millisecond and second levels, so that the database performance jitter caused by online modification of the database table structure is reduced. When the database structure is modified to the last step, the newly added lock type is used for monopolizing the metadata information until the database structure operation transaction is submitted, so that the consistency of data reading is ensured.
Drawings
The foregoing and other objects, features and advantages of the invention will be apparent from the following description of embodiments of the invention, which proceeds with reference to the accompanying drawings.
Fig. 1 shows an application scenario diagram of a database table structure modification method according to an embodiment of the present invention.
Fig. 2 shows a flowchart of a database table structure modification method according to an embodiment of the invention.
FIG. 3 shows a flowchart of a first target data table construction method according to an embodiment of the invention.
Fig. 4 shows a block diagram of a database table structure modifying apparatus according to an embodiment of the present invention.
Fig. 5 shows a block diagram of an electronic device adapted to implement a database table structure modification method according to an embodiment of the invention.
Detailed Description
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the invention. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present invention.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
In the concept of implementing the invention, when the database structure is modified, the problems to be solved include that the read-write operation of the database can be continued when the database structure is modified, so that the availability of the database is ensured, and the consistency of the data is ensured. The method mainly comprises the steps of firstly, after the operation of modifying a database structure is started, concurrent writing operation for a database object is continuously performed and incremental data can be continuously generated, if a traditional exclusive lock mode is used, the concurrency performance of the database and normal use of services are greatly blocked, secondly, the metadata information is required to be modified at the tail of the process of modifying the database structure, the query plan mode is synchronously modified, but the mode of reading the version of the database object is that lock query is not added, so that consistency of the metadata information query cannot be guaranteed, and thirdly, the database structure is required to be modified, data is migrated after being upgraded, and a large amount of system resources are consumed.
Fig. 1 illustrates an exemplary application scenario 100 in which a database table structure modification method may be applied according to an embodiment of the present invention. It should be noted that fig. 1 is only an example of an application scenario where an embodiment of the present invention may be applied, so as to help those skilled in the art understand the technical content of the present invention, but it does not mean that the embodiment of the present invention may not be applied to other devices, systems, environments, or scenarios.
As shown in fig. 1, an application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, the network 104 and the server 105. The network 104 may include various connection types, such as wired and/or wireless communication links, and the like.
The user may interact with the server 105 via the network 104 using the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages etc. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, and/or social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, the third terminal device 103.
The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the database table structure modification method provided by the embodiment of the present invention may be generally executed by the server 105. Accordingly, the database table structure modifying apparatus provided in the embodiment of the present invention may be generally disposed in the server 105. The database table structure modification method provided by the embodiment of the present invention may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and/or the server 105. Accordingly, the database table structure modifying apparatus provided by the embodiment of the present invention may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and/or the server 105.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The database table structure modification method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 5 based on the scenario described in fig. 1.
Fig. 2 shows a flowchart of a database table structure modification method according to an embodiment of the invention.
As shown in FIG. 2, the method includes operations S201-S203.
In operation S201, in response to a modification instruction to the original database table structure, a first target data table is constructed from a first data set including basic data in the original database table before modification and a second data set including delta data generated in the original database table during modification, wherein the table structures of the first target data table and the original database table are different.
According to an embodiment of the present invention, an original database table structure refers to a table in an already existing database, which contains data structures and data that need to be modified or expanded, before any modification is made. The modification instruction refers to a series of operation commands for changing the original database table structure, such as adding, deleting or modifying columns or fields, changing data types, adding or deleting indexes, etc. The first data set refers to all data already present in the original database table, which is the underlying data, to be used in constructing the new target data table, prior to table structure modification. The second data set refers to new data generated during the modification of the table structure, which may be due to new data insertion, update or deletion operations of existing data, or incremental data generated by user operations on the database during the construction of the new table by the first data set, i.e. the second data set contains all incremental changes made to the original database table during the modification. The first target data table is a new table created according to the table structure modification instruction, and the structure of the new table is different from that of the original database table, and the new table contains a modified data structure and is used for storing data of the first data group and the second data group, for example, the first target data table is added with three columns compared with the original database table structure, namely, the change on the database table structure, namely, the modification on the table structure, or the first target data table is different from the original database table in the aspects of columns or fields, data types, indexes, constraints and the like, and can be adjusted according to actual demands of users, for example, the structure adjustment caused by service demand change, performance optimization or other factors.
According to the embodiment of the invention, when an instruction for modifying the table structure of the data table is received, the data table with different structures preset by a user, and the first data set and the second data set are constructed to obtain a first target data table. For example, the data of the first target data table is stored by adding a data base structure storage fragment, and then the corresponding data fragments of the first target data table are constructed by the first data group and the second data group in a data page mode.
In operation S202, a first lock is added to the first target data table in case a read operation to the first target data table is detected, and/or a second lock is added to the first target data table in case the data amount of the second data set satisfies a preset numerical condition.
According to an embodiment of the invention, a read operation refers to the act of querying or retrieving data that allows a user to view the data in a table without modifying it, and when a read operation to a first target data table is detected, the database adds a lock to manage concurrent access. The first lock is a lock that allows multiple transactions to read data at the same time but does not allow modification of the data, and when a read operation occurs, the first lock is added to the first target data table to ensure that the data is not modified by other transactions when read. The second lock is a lock for preventing other transactions from reading or modifying data, and when the data volume of the second data set meets the preset numerical condition, the second lock is added on the first target data table so as to ensure the consistency and the integrity of the data. The preset value condition is a threshold set by the database administrator for deciding when to apply the second lock, e.g. set in processing the second data set, i.e. the incremental data, to reach a threshold, i.e. to add the second lock to the first target data table.
According to an embodiment of the invention, the first lock is added when a read operation is detected, and the first lock is not added all the way to the database provided that the user has not performed a read operation on the database during online modification of the entire data table structure.
In operation S203, metadata in the first target data table to which the first lock and/or the second lock is added is modified to obtain a second target data table.
According to embodiments of the present invention, metadata refers to information that provides information about the content, structure, origin, and other characteristics of the data. In a database table, metadata may include column names, data types, constraints, etc., and modifications to metadata may involve altering these attributes or definitions. The second target data table refers to a data table subjected to metadata modification based on the first target data table, and is also a target database table subjected to online modification of a database table structure by a user, and the target database table has a structure different from the metadata table and corresponding metadata.
In some embodiments of the present invention, the first lock responds to a user's read operation on data in the data table, and the second lock is a necessary condition at the stage of modifying metadata, which may be understood that the second lock is added before the metadata is modified, or that no second lock does not modify the metadata of the first target data table. Further, during the operation of modifying metadata, the database table must have a second lock, whether the first lock exists or not that does not affect the metadata's modification operation.
In some embodiments of the present invention, after obtaining the second target data table, the method further includes releasing the second lock and generating a new target access path based on the second target data table, where the operation execution plan is used to update an operation execution plan of data in the second target data table, and the operation execution plan refers to a manner in which the database uses to read the database object information, such as a full table scan, an index scan, etc., where updating the database object may change the execution plan of the database read operation.
Further, modifying the corresponding data page to adapt to the modified metadata is also included in the case that the metadata is modified. When the database object is updated, if the metadata information is judged to be changed, the data object to be updated is updated to be changed into the latest database object structure. In the prior art, after metadata is changed, a database object (usually a user table) is subjected to exclusive locking, then full table data is updated and then is migrated to a new storage space, while the metadata information is changed, so long as the table data object is not changed, the old data is not changed as much as possible, and only when the old data object is updated, the old data object is updated to the data object of the latest version database structure, so that the performance jitter caused by the database structure modification operation is reduced to the minimum as much as possible.
According to the embodiment of the invention, the first lock and the second lock are added on the first target data table, so that the condition of inconsistent data in the data migration and modification process can be prevented, and the integrity and accuracy of the data are ensured. The method has the advantages that when the second lock is added, namely when the metadata is started to be modified, is determined by presetting numerical conditions, can be flexibly adjusted according to the load and performance requirements of the system so as to adapt to different workloads, and simultaneously controls the modification process of the whole metadata at millisecond and second levels, so that the database performance jitter caused by online modification of the database table structure is reduced. When the database structure is modified to the last step, the newly added lock type is used for monopolizing the metadata information until the database structure operation transaction is submitted, so that the consistency of data reading is ensured.
According to the embodiment of the invention, the first target data table comprises a first lock, a second lock and a plurality of locks except the second lock of the first lock, the first lock and the second lock are mutually exclusive, the first lock and the other locks coexist, and the second lock and the first lock and the other locks are mutually exclusive.
According to an embodiment of the present invention, the locks other than the first lock and the second lock refer to locks on the data table caused by other user operations, and may or may not exist during modification of the data table. The mutual exclusivity of the first lock and the second lock can control concurrent access to a first target data table, the first lock can be shared with other locks of other operations, the first lock is added when a read operation is detected to ensure that data cannot be modified simultaneously when the data is read, but the addition of the second lock means that metadata modification is required, and no read, write and other operations are allowed at the moment so as to protect metadata information from version reading problems. Moreover, the risk of deadlock can be reduced through a lock strategy, and the hierarchical structure of the lock can avoid cyclic waiting possibly caused by multi-granularity lock operation because the first lock is mutually exclusive with the second lock, so that the probability of deadlock occurrence is reduced.
According to an embodiment of the invention, the first lock is a multi-version shared lock and the second lock is a multi-version exclusive lock.
According to an embodiment of the invention, the first lock is a multi-version shared lock (multi-version shared lock, MS) and the second lock is a multi-version exclusive lock (multi-version exclusive lock, MX). The MS lock allows the version read operation to be parallel to other read and write operations, and the MS lock IS compatible with various locks, such as an intention shared lock (IS), an intention exclusive lock (IX), a shared lock (S), and an exclusive lock (X), which means that the version read operation can be executed in parallel with other transactions without modifying the database structure, thereby improving concurrency performance of the database, and only mutually exclusive with the MX lock ensures that all locks are blocked before the metadata IS modified, and data inconsistency IS avoided.
According to an embodiment of the invention, the first data set includes pre-image data of the base data in the original database table, and the generation of the pre-image data is responsive to an operation of modifying the structure of the original database table.
According to embodiments of the present invention, pre-image data refers to a snapshot or copy of a piece of data that the system would hold prior to performing a data modification operation. The pre-image of the data is used for recovering the data to a state before modification when the transaction fails or needs to roll back, so that the atomicity and consistency of the transaction are ensured. When the first data set is called to construct a first target data table with a new structure, if the corresponding data has the pre-image data, the pre-image data is called. The generation of the pre-image data is related to the instruction for modifying the database table structure, and the basic data is judged to have change or influence according to the modification instruction, so that the pre-image data page of the data page is generated, and if the change or influence is not generated, the pre-image data is not generated, namely the pre-image page of all the data pages of the whole data table is not required to be generated.
In some embodiments of the present invention, a write operation of a database object temporarily opens up a temporary data fragment to save pre-image data when an online modification of the database structure operation is detected. The write operation generates a pre-image data page of a data page affected by the operation of modifying the database structure online and stores the pre-image data page in a temporary data partition, when a first target data table is constructed, all data of the database object is scanned to construct a corresponding storage partition of the first target data table, if the pre-image of the data page is stored in the temporary data partition during scanning, the data page in the temporary data partition is used for constructing a newly added database structure, and otherwise, the data page is constructed by unchanged data page partitions.
For example, the original database table has data page 1, data page 2 and data page 3, and according to the modification instruction, the online modification of the table structure only has an effect on the data of the data page 2, so that the pre-image data page of the data page 2, namely, the data page 2', is generated and stored in the temporary data partition opened temporarily. The first data set at this time will consist of data page 1, data page 2', data page 3.
According to the embodiment of the invention, the front image data and the incremental data ensure that the data generated in the whole database structure modification process is not lost and the normal use of the database service is not influenced, and the data rollback can be performed on the assumption that errors occur in the modification process. And the original data is used by the data which only has influence on the modification structure, but the data which has no influence or no change can avoid overlarge system pressure caused by a large amount of pre-construction image data.
FIG. 3 shows a flowchart of a first target data table construction method according to an embodiment of the invention.
As shown in fig. 3, constructing a first target data table according to the first data set and the second data set includes operations S301 to 303.
In operation S301, a third target data table having a different table structure from the original database table is constructed according to a modification target preset by the user, wherein the third target data table does not have any data row.
In operation S302, a base target data table is constructed from the first data set and the third target data table.
In operation S303, a first target data table is constructed based on the base target data table according to the second data set and the corresponding incremental data operation log.
In operation S301, the modification targets preset by the user are which modifications the user wishes to make to the original database table structure, such as adding new columns, deleting existing columns, modifying the data type of columns, changing the index of the table, and the like. The third target data table is a new data table which is different from the original database table in structure and is constructed according to the user modification requirement, and is an empty table with a specific structure but without any data. Further, a corresponding data storage partition may be opened up accordingly.
In operation S302, after the third target data table is constructed, the basic data in the original database table is imported first, that is, the basic target data table is constructed through the first data set. For example, the first data set is called or copied to a storage partition corresponding to the third target data table in the form of a data page, so that the construction of the basic target data table is realized.
In operation S303, the incremental data in the second data set will generate an incremental data operation log corresponding to the incremental data during the modification process, and the incremental data operation log records the specific operations of the data changes. An incremental data temporary partition can be opened up for storing the incremental data and the corresponding incremental data operation log. The incremental data and the corresponding incremental data operation log in the incremental data temporary fragments are added and updated in real time according to the operation of an actual user, and in the process of constructing the first target data table for the second data set, the data page is constructed according to the operation log of the incremental data, or the data in the basic target data table constructed by the first data set can be understood as modification, so that the first target data table is obtained.
According to the embodiment of the invention, the empty third target data table is firstly created, then the basic target data table is built based on the first data group, and finally the first target data table is built according to the second data group and the incremental data operation log, and the first target data table is built in stages, so that the data migration process is more orderly and efficient. By creating an empty third target data table and introducing the data step by step, the integrity and consistency of the data can be better controlled, especially during multi-step data migration.
According to an embodiment of the present invention, the preset numerical condition includes that the number of corresponding incremental data operation logs is less than or equal to 5.
According to the embodiment of the present invention, in the process of constructing the first target data table in operation S303, since there may be a continuous operation performed on the database by the user, incremental data is continuously generated, but at the same time, when the incremental data is consumed, the consumption speed is generally far greater than the incremental data generation speed, and if it is monitored that the incremental data operation logs are less than or equal to 5, a second lock is added to the first target data table, and a subsequent metadata modification operation is performed.
According to the embodiment of the invention, when the number of the incremental data operation logs is small, the modification operation on the first target data table is relatively small, and at the moment, the addition of the second lock can quickly complete the modification of metadata, reduce the execution time of the transaction, improve the overall transaction processing efficiency, and greatly reduce the system jitter caused by the modification of the database table structure when the modification operation of the whole metadata is in a millisecond level.
Based on the database table structure modification method, the invention also provides a database table structure modification device. The device will be described in detail below in connection with fig. 4.
Fig. 4 shows a block diagram of a database table structure modifying apparatus according to an embodiment of the present invention.
As shown in fig. 4, the database table structure modifying apparatus 400 of this embodiment includes a building module 410, a locking module 420, and a modifying module 430.
The construction module 410 is configured to construct a first target data table according to a first data set and a second data set in response to a modification instruction to the original database table structure, wherein the first data set includes basic data in the original database table before modification, and the second data set includes incremental data generated in the original database table during modification, and wherein the table structures of the first target data table and the original database table are different. In an embodiment, the construction module 410 may be configured to perform the operation S201 described above, which is not described herein.
The locking module 420 is configured to add a first lock to the first target data table if a read operation to the first target data table is detected, and/or add a second lock to the first target data table if the data amount of the second data set satisfies a preset numerical condition. In an embodiment, the locking module 420 may be used to perform the operation S202 described above, which is not described herein.
The modification module 430 is configured to modify metadata in the first target data table to which the first lock and/or the second lock are added, to obtain a second target data table. In an embodiment, the modification module 430 may be configured to perform the operation S203 described above, which is not described herein.
According to the embodiment of the invention, the first lock and the second lock are added on the first target data table, so that the condition of inconsistent data in the data migration and modification process can be prevented, and the integrity and accuracy of the data are ensured. The method has the advantages that when the second lock is added, namely when the metadata is started to be modified, is determined by presetting numerical conditions, can be flexibly adjusted according to the load and performance requirements of the system so as to adapt to different workloads, and simultaneously controls the modification process of the whole metadata at millisecond and second levels, so that the database performance jitter caused by online modification of the database table structure is reduced. When the database structure is modified to the last step, the newly added lock type is used for monopolizing the metadata information until the database structure operation transaction is submitted, so that the consistency of data reading is ensured.
Any of the building module 410, locking module 420, and modifying module 430 may be combined in one module to be implemented, or any of the modules may be split into multiple modules, according to embodiments of the present invention. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules. At least one of the building module 410, the locking module 420, and the modifying module 430 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-a-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or by hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Or at least one of the construction module 410, the locking module 420 and the modification module 430 may be at least partially implemented as a computer program module which, when executed, may perform the corresponding functions.
Fig. 5 shows a block diagram of an electronic device adapted to implement a database table structure modification method according to an embodiment of the invention.
Fig. 5 shows a block diagram of an electronic device adapted to implement the method described above, according to an embodiment of the invention. The electronic device shown in fig. 5 is only an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.
As shown in fig. 5, an electronic device 500 according to an embodiment of the present invention includes a processor 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. The processor 501 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 501 may also include on-board memory for caching purposes. The processor 501 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flow according to an embodiment of the invention.
In the RAM 503, various programs and data required for the operation of the electronic apparatus 500 are stored. The processor 501, ROM 502, and RAM 503 are connected to each other by a bus 504. The processor 501 performs various operations of the method flow according to an embodiment of the present invention by executing programs in the ROM 502 and/or the RAM 503. Note that the program may be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 may also perform various operations of the method flow according to embodiments of the present invention by executing programs stored in the one or more memories.
According to an embodiment of the invention, the electronic device 500 may further comprise an input/output (I/O) interface 505, the input/output (I/O) interface 505 also being connected to the bus 504. The electronic device 500 may also include one or more of an input portion 506 including a keyboard, a mouse, etc., an output portion 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc., a storage portion 508 including a hard disk, etc., and a communication portion 509 including a network interface card such as a LAN card, a modem, etc., connected to an input/output (I/O) interface 505. The communication section 509 performs communication processing via a network such as the internet. The drive 510 is also connected to an input/output (I/O) interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.
According to an embodiment of the present invention, the method flow according to an embodiment of the present invention may be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 509, and/or installed from the removable media 511. The above-described functions defined in the system of the embodiment of the present invention are performed when the computer program is executed by the processor 501. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the invention.
The present invention also provides a computer-readable storage medium that may be included in the apparatus/device/system described in the above embodiments, or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present invention.
According to an embodiment of the present invention, the computer-readable storage medium may be a nonvolatile computer-readable storage medium. Such as, but not limited to, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the invention, the computer-readable storage medium may include ROM 502 and/or RAM 503 and/or one or more memories other than ROM 502 and RAM 503 described above.
Embodiments of the present invention also include a computer program product comprising a computer program comprising program code for performing the method provided by the embodiments of the present invention, when the computer program product is run on an electronic device, for causing the electronic device to implement the method for determining a database running state provided by the embodiments of the present invention.
The above-described functions defined in the system/apparatus of the embodiment of the present invention are performed when the computer program is executed by the processor 501. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the invention.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed, and downloaded and installed in the form of a signal on a network medium, and/or installed from a removable medium 511 via the communication portion 509. The computer program may comprise program code that is transmitted using any appropriate network medium, including but not limited to wireless, wireline, etc., or any suitable combination of the preceding.
According to embodiments of the present invention, program code for carrying out computer programs provided by embodiments of the present invention may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or in assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. Those skilled in the art will appreciate that the features recited in the various embodiments of the invention can be combined and/or combined in a variety of ways, even if such combinations or combinations are not explicitly recited in the present invention. In particular, the features recited in the various embodiments of the invention can be combined and/or combined in various ways without departing from the spirit and teachings of the invention. All such combinations and/or combinations fall within the scope of the invention.
The embodiments of the present invention are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the invention, and such alternatives and modifications are intended to fall within the scope of the invention.

Claims (10)

Translated fromChinese
1.一种数据库表结构修改方法,其特征在于,所述数据库表结构修改方法包括:1. A method for modifying a database table structure, characterized in that the method for modifying a database table structure comprises:响应于对原数据库表结构的修改指令,根据第一数据组和第二数据组构建第一目标数据表,所述第一数据组包括在修改之前原数据库表中的基础数据,所述第二数据组包括在修改过程中,在所述原数据库表中产生的增量数据,其中,所述第一目标数据表和所述原数据库表的表结构不同;In response to an instruction to modify the structure of an original database table, a first target data table is constructed according to a first data group and a second data group, wherein the first data group includes basic data in the original database table before modification, and the second data group includes incremental data generated in the original database table during the modification process, wherein the first target data table and the original database table have different table structures;在检测到对所述第一目标数据表的读操作的情况下,对所述第一目标数据表添加第一锁,和在所述第二数据组的数据量满足预设数值条件的情况下,对所述第一目标数据表添加第二锁;When a read operation on the first target data table is detected, a first lock is added to the first target data table, and when the amount of data of the second data group meets a preset numerical condition, a second lock is added to the first target data table;对添加第一锁和第二锁的所述第一目标数据表中的元数据进行修改,得到第二目标数据表。The metadata in the first target data table to which the first lock and the second lock are added is modified to obtain a second target data table.2.根据权利要求1所述的数据库表结构修改方法,其特征在于,所述第一目标数据表包括第一锁、第二锁和除第一锁、第二锁以外的其他多个锁;2. The method for modifying the database table structure according to claim 1, wherein the first target data table includes a first lock, a second lock, and multiple locks other than the first lock and the second lock;所述第一锁与所述第二锁互斥,所述第一锁与所述其他多个锁共存;The first lock and the second lock are mutually exclusive, and the first lock and the other multiple locks coexist;所述第二锁与所述第一锁、所述其他多个锁互斥。The second lock is mutually exclusive with the first lock and the other multiple locks.3.根据权利要求1所述的数据库表结构修改方法,其特征在于,3. The method for modifying the database table structure according to claim 1, characterized in that:所述第一锁为多版本共享锁,所述第二锁为多版本独占锁。The first lock is a multi-version shared lock, and the second lock is a multi-version exclusive lock.4.根据权利要求1所述的数据库表结构修改方法,其特征在于,4. The method for modifying the database table structure according to claim 1, characterized in that:所述第一数据组中包括所述原数据库表中基础数据的前像数据,所述前像数据是响应于对所述原数据库表结构进行修改的操作生成的。The first data group includes pre-image data of basic data in the original database table, and the pre-image data is generated in response to an operation of modifying the structure of the original database table.5.根据权利要求1或4所述的数据库表结构修改方法,其特征在于,所述根据第一数据组和第二数据组构建第一目标数据表包括:5. The method for modifying the database table structure according to claim 1 or 4, wherein constructing the first target data table according to the first data group and the second data group comprises:根据用户预设的修改目标,构建具有与所述原数据库表的表结构不同的第三目标数据表,其中所述第三目标数据表不具有数据行;According to the modification target preset by the user, construct a third target data table having a table structure different from that of the original database table, wherein the third target data table has no data rows;根据所述第一数据组和所述第三目标数据表构建基础目标数据表;constructing a basic target data table according to the first data group and the third target data table;根据所述第二数据组和对应的增量数据操作日志在所述基础目标数据表的基础上构建所述第一目标数据表。The first target data table is constructed based on the basic target data table according to the second data group and the corresponding incremental data operation log.6.根据权利要求5所述的数据库表结构修改方法,其特征在于,6. The method for modifying the database table structure according to claim 5, characterized in that:所述预设数值条件包括所述增量数据操作日志的数量小于或等于5个。The preset numerical condition includes that the number of the incremental data operation logs is less than or equal to 5.7.一种数据库表结构修改装置,其特征在于,所述数据库表结构修改装置包括:7. A database table structure modification device, characterized in that the database table structure modification device comprises:构建模块,用于响应于对原数据库表结构的修改指令,根据第一数据组和第二数据组构建第一目标数据表,所述第一数据组包括在修改之前,原数据库表中的基础数据,所述第二数据组包括在修改过程中,在所述原数据库表中产生的增量数据,其中,所述第一目标数据表和所述原数据库表的表结构不同;a construction module, configured to construct a first target data table according to a first data group and a second data group in response to a modification instruction for an original database table structure, wherein the first data group includes basic data in the original database table before modification, and the second data group includes incremental data generated in the original database table during the modification process, wherein the first target data table and the original database table have different table structures;加锁模块,用于在检测到对所述第一目标数据表的读操作的情况下,对所述第一目标数据表添加第一锁,和在所述第二数据组的数据量满足预设数值条件的情况下,对所述第一目标数据表添加第二锁;a locking module, configured to add a first lock to the first target data table when a read operation on the first target data table is detected, and to add a second lock to the first target data table when the data volume of the second data group meets a preset numerical condition;修改模块,用于对添加第一锁和第二锁的所述第一目标数据表中的元数据进行修改,得到第二目标数据表。The modification module is used to modify the metadata in the first target data table to which the first lock and the second lock are added to obtain the second target data table.8.一种电子设备,包括:8. An electronic device, comprising:一个或多个处理器;one or more processors;存储器,用于存储一个或多个计算机程序,a memory for storing one or more computer programs,其特征在于,所述一个或多个处理器执行所述一个或多个计算机程序以实现根据权利要求1~6中任一项所述数据库表结构修改方法的步骤。It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the database table structure modification method according to any one of claims 1 to 6.9.一种计算机可读存储介质,其上存储有可执行指令,其特征在于,该指令被处理器执行时使处理器实现权利要求1~6中任一项所述的数据库表结构修改方法。9. A computer-readable storage medium having executable instructions stored thereon, characterized in that when the instructions are executed by a processor, the processor implements the database table structure modification method according to any one of claims 1 to 6.10.一种计算机程序产品,包括计算机程序,其特征在于,所述计算机程序在被处理器执行时实现根据权利要求1~6中任一项所述的数据库表结构修改方法。10. A computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the method for modifying the database table structure according to any one of claims 1 to 6 is implemented.
CN202411783770.8A2024-12-062024-12-06Database table structure modification method and device, electronic equipment and storage mediumActiveCN119248808B (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN202411783770.8ACN119248808B (en)2024-12-062024-12-06Database table structure modification method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN202411783770.8ACN119248808B (en)2024-12-062024-12-06Database table structure modification method and device, electronic equipment and storage medium

Publications (2)

Publication NumberPublication Date
CN119248808A CN119248808A (en)2025-01-03
CN119248808Btrue CN119248808B (en)2025-06-06

Family

ID=94033288

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN202411783770.8AActiveCN119248808B (en)2024-12-062024-12-06Database table structure modification method and device, electronic equipment and storage medium

Country Status (1)

CountryLink
CN (1)CN119248808B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN115422188A (en)*2022-08-252022-12-02中国工商银行股份有限公司Table structure online changing method and device, electronic equipment and storage medium
CN116383195A (en)*2023-02-212023-07-04北京奥星贝斯科技有限公司Data storage method and device and electronic equipment

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
US10503727B2 (en)*2016-04-262019-12-10Servicenow, Inc.System and method for mapping database changes
CN116204330A (en)*2022-12-292023-06-02金篆信科有限责任公司Processing method and device of distributed database, electronic equipment and readable medium
CN117349258A (en)*2023-09-222024-01-05上海利唐信息科技有限公司Database version management method, device, computing equipment and storage medium
CN118796948B (en)*2024-09-122025-01-14苏州吉呗思数据技术有限公司 Method, device and electronic device for modifying sharding rules of distributed database

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN115422188A (en)*2022-08-252022-12-02中国工商银行股份有限公司Table structure online changing method and device, electronic equipment and storage medium
CN116383195A (en)*2023-02-212023-07-04北京奥星贝斯科技有限公司Data storage method and device and electronic equipment

Also Published As

Publication numberPublication date
CN119248808A (en)2025-01-03

Similar Documents

PublicationPublication DateTitle
US10552372B2 (en)Systems, methods, and computer-readable media for a fast snapshot of application data in storage
EP4254183A1 (en)Transaction processing method and apparatus, computer device, and storage medium
US10678649B2 (en)Interfacing with a virtual database system
US11704335B2 (en)Data synchronization in a data analysis system
WO2021249207A1 (en)Database transaction processing method and apparatus, and server and storage medium
US8832028B2 (en)Database cloning
EP2797013B1 (en)Database update execution according to power management schemes
US10275347B2 (en)System, method and computer program product for managing caches
CN110019251A (en)A kind of data processing system, method and apparatus
WO2020181810A1 (en)Data processing method and apparatus applied to multi-level caching in cluster
US20170220642A1 (en)Methods for updating reference count and shared objects in a concurrent system
US12164496B2 (en)Transaction execution method, computing device, and storage medium
US11797523B2 (en)Schema and data modification concurrency in query processing pushdown
US12210514B2 (en)Schema and data modification concurrency in query processing pushdown
US11822552B2 (en)Methods for updating reference count and shared objects in a concurrent system
CN112579698A (en)Data synchronization method, device, gateway equipment and storage medium
CN115794819A (en)Data writing method and electronic equipment
CN115422188A (en)Table structure online changing method and device, electronic equipment and storage medium
US20220067065A1 (en)Providing instant and distributed access to a source blob via copy-on-read blobs and link blobs
CN119248808B (en)Database table structure modification method and device, electronic equipment and storage medium
US12045474B2 (en)Efficient dirty page expiration
US10942898B2 (en)System and method for a persistent hierarchical work manager
US12147412B2 (en)Concurrent optimistic transactions for tables with deletion vectors
CN119537481A (en) Database synchronization method, device, equipment, medium and program product
Diaz et al.Working with NoSQL Alternatives

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination
GR01Patent grant
GR01Patent grant

[8]ページ先頭

©2009-2025 Movatter.jp