CN106354830B - Method and device for data synchronization between database cluster nodes - Google Patents

Abstract

The invention provides a method and a device for data synchronization among database cluster nodes, wherein the method comprises the following steps: acquiring information of a table to be synchronized from a metadata server; checking the lock queue to determine whether a lock to delete the table is included; and if the lock queue does not comprise the deletion information of the table, performing data synchronization. The table can be deleted before the table is modified, so that the data exchange synchronization is reduced, and the waste of time and system resources is avoided. The database synchronization is efficiently completed.

Description

Method and device for data synchronization between database cluster nodes

Technical Field

The invention belongs to the field of database clusters, and particularly relates to a method and a device for data synchronization among database cluster nodes.

Background

When the database cluster is used, a user needs to frequently perform operations of table building, calculation and table deletion. Before deleting a table, if a table on a certain node of the database cluster is damaged, the table on the database node needs to be synchronized.

The following methods can be currently used for synchronization: and locking the synchronized table, and deleting the table after the synchronization is finished. If the user has already done the calculation during the synchronization and needs to delete the table, the user needs to wait until the deleted table is synchronized before deleting the table, which wastes a lot of time and system resources.

Disclosure of Invention

The embodiment of the invention provides a method and a device for data synchronization among database cluster nodes, which are used for solving the existing technical problems.

In one aspect, an embodiment of the present invention provides a method for synchronizing data among database cluster nodes, including:

acquiring information of a table to be synchronized from a metadata server;

checking the lock queue to determine whether a lock to delete the table is included;

and if the lock queue does not comprise the deletion information of the table, performing data synchronization.

Further, the method further comprises:

and if the lock queue does not comprise the deletion information of the table, abandoning the data synchronization.

Further, after the data synchronization, the method further includes:

confirming whether the data synchronization is successful, and if the data synchronization is unsuccessful, writing the name of the table and the node corresponding to the failure into a metadata server to generate damage information;

adding a delete lock to the lock queue;

deleting the delete lock after the table is successfully deleted;

further, after the data synchronization, the method further includes:

and deleting the damage information of the table in the metadata server.

On the other hand, an embodiment of the present invention further provides a device for data synchronization between database cluster nodes, including:

an information acquisition unit for acquiring information of a table to be synchronized from a metadata server;

a lock checking unit for checking a lock queue to confirm whether a lock for deleting the table is included;

and the synchronization unit is used for carrying out data synchronization if the deletion information of the table is not included in the lock queue.

Further, the apparatus further comprises:

and the abandoning unit is used for abandoning the data synchronization if the lock queue comprises the deletion information of the table.

Further, the apparatus further comprises:

a writing unit, configured to determine whether data synchronization is successful, and if the data synchronization is unsuccessful, write the name of the table and a node corresponding to the failure into the metadata server, so as to generate damage information;

an adding unit, configured to add a delete lock to a lock queue;

and the lock deleting unit is used for deleting the deleting lock after the table is successfully deleted.

Further, the apparatus further comprises:

and the damage information deleting unit is used for deleting the damage information of the table in the metadata server.

The embodiment of the invention acquires the information of the table needing to be synchronized from the metadata server; checking a lock queue to determine whether the lock of the table is deleted; and if the lock queue does not comprise the deletion information of the table, performing data synchronization. The table can be deleted before the table is modified, so that the data exchange synchronization is reduced, and the waste of time and system resources is avoided. The database synchronization is efficiently completed.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.

Fig. 1 is a schematic flowchart of a method for synchronizing data between database cluster nodes according to an embodiment of the present invention;

fig. 2 is a schematic flowchart of a method for synchronizing data between database cluster nodes according to a second embodiment of the present invention;

fig. 3 is a schematic flowchart of a method for synchronizing data between database cluster nodes according to a third embodiment of the present invention;

fig. 4 is a schematic structural diagram of a data synchronization apparatus between database-partitioned cluster nodes according to a fourth embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Example one

Fig. 1 is a schematic flowchart of a method for synchronizing data between database cluster nodes according to an embodiment of the present invention. The method can be executed by a data synchronization device between the database cluster nodes, can be realized by a software/hardware mode, and can be integrated in the database cluster nodes.

Referring to fig. 1, the distributed database data backup segment deployment method includes:

s110, obtaining information of the table to be synchronized from the metadata server.

Data in any file system is divided into data and metadata. Data refers to actual data in a general file, and metadata refers to system data used to describe characteristics of a file, such as access rights, file owner, and distribution information of file data blocks, etc. A node in the system is specially managed by the metadata, and all the metadata is stored on a storage device of the node. All clients request the metadata from the metadata manager before requesting the file. Information of the tables that need to be synchronized can be obtained from the metadata server. Illustratively, a storage location for the synchronization table is included.

S120, checking the lock queue, and confirming whether the lock of the table is deleted or not.

The database cluster comprises a lock queue manager which is used for processing SQL locking requests of the database, queuing and storing the requests and providing a query interface for a synchronization tool. When the database executes data change operation, if some database nodes fail and the corresponding data copies on the failed nodes are successfully executed on other nodes, the failure information (table name and database node name) is recorded in the metadata server. The database performs a delete table operation while adding the delete lock to the lock queue manager. By querying the lock queue manager, it may be determined whether a lock to delete the table is included.

S130, if the lock queue does not include the deletion information of the table, data synchronization is carried out.

If the deletion information of the table is not included in the lock queue, the table is not damaged, and a synchronization tool can be used for carrying out data synchronization on the table.

The embodiment of the invention acquires the information of the table needing to be synchronized from the metadata server; checking a lock queue to determine whether the lock of the table is deleted; and if the lock queue does not comprise the deletion information of the table, performing data synchronization. The table can be deleted before the table is modified, so that the data exchange synchronization is reduced, and the waste of time and system resources is avoided. The database synchronization is efficiently completed.

In a preferred implementation of this embodiment, the method further includes: and if the lock queue comprises the deletion information of the table, abandoning the data synchronization. And if the lock queue comprises the deletion information of the table, the table is damaged, and a synchronization tool is not required to be adopted to perform data synchronization on the table. Or terminate synchronization when synchronizing a batch of data, avoiding wasting time and system resources.

Example two

Fig. 2 is a schematic flow chart of a method for synchronizing data between database cluster nodes according to a second embodiment of the present invention, where the second embodiment of the present invention is based on the foregoing embodiment, and further, the following steps are added after data synchronization is performed: confirming whether the data synchronization is successful, and if the data synchronization is unsuccessful, writing the name of the table and the node corresponding to the failure into a metadata server to generate damage information; adding a delete lock to the lock queue; and deleting the deletion lock after the table is deleted successfully.

Referring to fig. 2, the method for data synchronization between database cluster nodes includes:

s210, obtaining the information of the table needing to be synchronized from the metadata server.

S220, checking the lock queue, and confirming whether the lock of the table is deleted or not.

S230, if the lock queue does not include the deletion information of the table, data synchronization is carried out.

S240, confirming whether the data synchronization is successful, and if the data synchronization is unsuccessful, writing the name of the table and the node corresponding to the failure into the metadata server to generate damage information.

The database is used for receiving the SQL request, and if the table on a certain node is damaged in the request execution process, the database records the damaged table name and the database node name into the metadata server.

S250, adding the deletion lock to the lock queue.

Since the table has been corrupted, the table may be deleted and the database performs the delete table operation while the delete lock is added to the lock queue manager.

S260, deleting the deletion lock after the table is deleted successfully.

If the table is deleted successfully, the metadata server is informed to delete the corresponding table damage information in the metadata, and the lock queue server is informed to delete the deletion lock of the table in the lock queue.

The present embodiment adds the following steps after performing data synchronization: confirming whether the data synchronization is successful, and if the data synchronization is unsuccessful, writing the name of the table and the node corresponding to the failure into a metadata server to generate damage information; adding a delete lock to the lock queue; and deleting the deletion lock after the table is deleted successfully. The list with unsuccessful synchronization can be deleted to avoid the time and system resource waste caused by synchronization again in the later period.

EXAMPLE III

Fig. 3 is a schematic flow chart of a method for synchronizing data between database cluster nodes according to a third embodiment of the present invention, where the third embodiment of the present invention is based on the foregoing embodiment, and further, the following steps are added after data synchronization is performed: and deleting the damage information of the table in the metadata server.

Referring to fig. 3, the method for data synchronization between database cluster nodes includes:

s310, obtain the information of the table to be synchronized from the metadata server.

S320, checking the lock queue, and confirming whether the lock of the table is deleted or not.

S330, if the lock queue does not include the deletion information of the table, data synchronization is carried out.

S340, deleting the damage information of the table in the metadata server.

During synchronous operation, after a batch of data is synchronized, whether synchronization is completed is judged: if the synchronization is complete, the table corruption information on the metadata server is deleted. Since the data of the table is complete and correct due to the synchronization, the corruption information of the table in the metadata server can be deleted.

The present embodiment adds the following steps after performing data synchronization: and deleting the damage information of the table in the metadata server. After the table synchronization is successful, the metadata corresponding to the table can be modified in time, and the table which is successfully synchronized is prevented from being deleted by mistake.

Example four

Fig. 4 is a schematic structural diagram of an apparatus for synchronizing data between database cluster nodes according to a fourth embodiment of the present invention, as shown in fig. 4, the apparatus includes:

aninformation acquisition unit 410 for acquiring information of a table that needs synchronization from a metadata server;

alock checking unit 420 for checking the lock queue to see if a lock to delete the table is included;

asynchronizing unit 430, configured to perform data synchronization if the lock queue does not include the deletion information of the table.

The embodiment of the invention acquires the information of the table needing to be synchronized from the metadata server; checking a lock queue to determine whether the lock of the table is deleted; and if the lock queue does not comprise the deletion information of the table, performing data synchronization. The table can be deleted before the table is modified, so that the data exchange synchronization is reduced, and the waste of time and system resources is avoided. The database synchronization is efficiently completed.

Further, the apparatus further comprises:

and the abandoning unit is used for abandoning the data synchronization if the lock queue comprises the deletion information of the table.

Further, the apparatus further comprises:

a writing unit, configured to determine whether data synchronization is successful, and if the data synchronization is unsuccessful, write the name of the table and a node corresponding to the failure into the metadata server, so as to generate damage information;

an adding unit, configured to add a delete lock to a lock queue;

and the lock deleting unit is used for deleting the deleting lock after the table is successfully deleted.

Still further, the apparatus further comprises:

and the damage information deleting unit is used for deleting the damage information of the table in the metadata server.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (8)

1. A method for data synchronization between database cluster nodes is characterized by comprising the following steps:

acquiring information of a table to be synchronized from a metadata server;

checking the lock queue to determine whether a lock to delete the table is included; when a database executes data change operation, if partial database nodes fail and copies of corresponding data on the failed nodes are successfully executed on other nodes, recording failure information to a metadata server, and adding a lock for deleting a table to a lock queue manager when the database executes table deletion operation;

and if the lock queue does not comprise the deletion information of the table, performing data synchronization.

2. The method of claim 1, further comprising:

and if the lock queue comprises the deletion information of the table, abandoning the data synchronization.

3. The method of claim 1, further comprising, after performing data synchronization:

confirming whether the data synchronization is successful, and if the data synchronization is unsuccessful, writing the name of the table and the node corresponding to the failure into a metadata server to generate damage information;

adding a delete lock to the lock queue;

and deleting the deletion lock after the table is deleted successfully.

4. The method of claim 1, further comprising, after performing data synchronization:

and deleting the damage information of the table in the metadata server.

5. An apparatus for synchronizing data between database cluster nodes, comprising:

an information acquisition unit for acquiring information of a table to be synchronized from a metadata server;

a lock checking unit for confirming whether a lock for deleting the table is included; when a database executes data change operation, if partial database nodes fail and copies of corresponding data on the failed nodes are successfully executed on other nodes, recording failure information to a metadata server, and adding a lock for deleting a table to a lock queue manager when the database executes table deletion operation;

and the synchronization unit is used for carrying out data synchronization if the deletion information of the table is not included in the lock queue.

6. The apparatus of claim 5, further comprising:

and the abandoning unit is used for abandoning the data synchronization if the lock queue comprises the deletion information of the table.

7. The apparatus of claim 5, further comprising:

a writing unit, configured to determine whether data synchronization is successful, and if the data synchronization is unsuccessful, write the name of the table and a node corresponding to the failure into the metadata server, so as to generate damage information;

an adding unit, configured to add a delete lock to a lock queue;

and the lock deleting unit is used for deleting the deleting lock after the table is successfully deleted.

8. The apparatus of claim 5, further comprising:

and the damage information deleting unit is used for deleting the damage information of the table in the metadata server.

Images