CN103365987A - Clustered database system and data processing method based on shared-disk framework - Google Patents

Abstract

The invention relates to a clustered database system and data processing method based on a shared-disk framework. The system comprises main control nodes and transaction nodes, wherein the transaction nodes are used for processing transactions and querying static data in a clustered database; the main control nodes are used for carrying out update operation on data in the clustered database, generating dynamic data, saving and querying the dynamic data. According to a novel database management system framework and the data processing method under the cluster environment of shared-disk framework, advantages (such as large internal storage, SSD (Solid State Disk) and other latest hardware technologies) of latest hardware technologies are adopted, so that the performance of the database is improved remarkably, the cooperative work of multiple computes is utilized fully, limitation of the database on the aspect of expansibility is broken through, and the system performance and throughput of transaction processing are improved.

Description

A kind of clustered database system and data processing method based on the shared disk framework

Technical field

The present invention relates to database technical field, particularly a kind of clustered database system and data processing method based on the shared disk framework.

Background technology

Database is the net result of preserving calculating, so be the most important components of whole infosystem.Satisfying ever-increasing Transaction Processing application facet, also there are many urgent technical issues that need to address in fact in current database technology.

For all databases, except recording correct result, they all are faced with the challenge of four aspects: how to improve processing speed, availability of data, data security and data set extensibility, that is to say, the scalability that how to make current database have this four aspect makes the client can obtain simultaneously higher processing speed, higher availability of data, higher data security and larger data set.A plurality of databases are linked togather form data-base cluster and reach above-mentioned target and should say a very natural idea.Clustering is to use specific connected mode, and the hardware device that price is relatively low combines, and provides high-performance suitable task processing power, guarantees simultaneously the correctness of issued transaction.

Data-base cluster mainly contains following three kinds of structures:

1) shared drive (primary memory) structure (Shared_Memory, SM)

2) shared-disk architecture (Shared_Disk, SD)

3) shared nothing structure (Shared_Nothing, SN)

Under shared-disk architecture, more typical system is Oracle Real Application Cluster(Oracl e RAC at present), this system is the parallel cluster of Oracle, the Oracle example that is positioned at the different server system is accessed same oracle database simultaneously, communicate by private network between the node, all control documents, online daily record and data file leave on the shared equipment, can be read while write by all nodes in the cluster.Each node among the RAC is peer-to-peer, and each node receives user's request, and gives user's return results.RAC has adopted Cache Fusion (Database cache fusion) technology, and the transmission of data block is carried out by high speed, the low internal network that postpones in the data buffer of each node.This scheme Main Problems is that inter-node traffic is large, and node buffer zone simultaneous techniques is complicated.

Summary of the invention

The objective of the invention is for the problems referred to above, the present invention proposes a kind of clustered database system based on the shared disk framework and data processing method, and is large to overcome in the prior art inter-node traffic, the problem of node buffer zone simultaneous techniques complexity.

For achieving the above object, the present invention proposes a kind of clustered database system based on the shared disk framework, and this system comprises: main controlled node and affairs node;

Described affairs node is for the treatment of the inquiry of the issued transaction in the Cluster Database and static data;

Described main controlled node is used for Cluster Database is carried out the Data Update operation, produces dynamic data, preserves and inquire about described dynamic data.

Optionally, in an embodiment of the present invention, described main controlled node is further used for inquiring about described dynamic data, and Query Result is back to described affairs node;

Described affairs node is further used for obtaining the Static Inquiry result according to the static data on the data processing request inquiry disk, obtains result set according to the Query Result and the described Static Inquiry result merging that identify described main controlled node.

Optionally, in an embodiment of the present invention, described main controlled node is used for the data of inserting are preserved.

Optionally, in an embodiment of the present invention, described main controlled node is used for recording the sign of the data that need modification and the data after the corresponding renewal;

Optionally, in an embodiment of the present invention, described main controlled node is used for the sign that record needs the data of deletion.

Optionally, in an embodiment of the present invention, described affairs node adopts the relational database management system engine.

Optionally, in an embodiment of the present invention, described main controlled node adopts the main storage data base engine.

Optionally, in an embodiment of the present invention, described main controlled node is further used for regularly or is static data on the disk according to the operating position of core buffer with described daynamic transformation.

Optionally, in an embodiment of the present invention, described main controlled node is further used for regularly or according to the operating position of core buffer described dynamic data is forwarded to described affairs node;

Described affairs node is further used for the data that described main controlled node forwards are converted to static data on the disk.

For achieving the above object, the present invention also proposes a kind of data processing method, and the method comprises:

The query manipulation of the static data in the described affairs node processing Cluster Database;

Described main controlled node carries out the Data Update operation to Cluster Database, produces dynamic data, preserves and inquire about described dynamic data.

Optionally, in an embodiment of the present invention, the method also comprises:

Described main controlled node is inquired about described dynamic data and is obtained the first Query Result, and is forwarded to described affairs node;

Described affairs node obtains the Static Inquiry result according to the static data on the data processing request inquiry disk, according to sign described the first Query Result and described Static Inquiry result merging is obtained result set.

Optionally, in an embodiment of the present invention, described main controlled node is preserved the data of inserting.

Optionally, in an embodiment of the present invention, the sign of the data that described main controlled node record need to be modified and the data after corresponding the renewal;

Optionally, in an embodiment of the present invention, described main controlled node record needs the sign of the data of deletion.

Technique scheme has following beneficial effect: under a kind of cluster environment of new shared disk framework, database management system architecture and data processing method thereof, utilize the advantage (such as up-to-date hardware technologies such as large internal memory, SSD) of up-to-date hardware technology, significantly improved the performance of database, and take full advantage of many collaborative computer work, break through the limitation of database aspect extendability, improve the performance of system and the handling capacity of issued transaction.

Description of drawings

In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, the below will do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art, apparently, accompanying drawing in the following describes only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.

Fig. 1 is a kind of clustered database system block diagram based on the shared disk framework that the present invention proposes;

Fig. 2 is one of a kind of data processing method process flow diagram of proposing of the present invention;

Fig. 3 is two of a kind of data processing method process flow diagram of proposing of the present invention;

Fig. 4 is that the embodiment of the invention is based on the clustered database system Organization Chart of shared disk framework;

Fig. 5 is data processing method process flow diagram in the embodiment of the invention;

Fig. 6 is that the embodiment of the invention is based on the data processing method process flow diagram of the system architecture of Fig. 4;

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is clearly and completely described.Obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that obtains under the creative work prerequisite.

As shown in Figure 1, a kind of clustered database system block diagram based on the shared disk framework that proposes for the present invention.This system comprises: main controllednode 101 andaffairs node 102;

Describedaffairs node 102 is for the treatment of the query manipulation of the static data in the Cluster Database;

Described main controllednode 101 is used for Cluster Database is carried out the Data Update operation, produces dynamic data, preserves and inquire about described dynamic data.

Optionally, in an embodiment of the present invention, described main controllednode 101 is further used for inquiring about described dynamic data, and Query Result is back to described affairs node;

Describedaffairs node 102 is further used for obtaining the Static Inquiry result according to the static data on the data processing request inquiry disk, obtains result set according to the Query Result and the described Static Inquiry result merging that identify described main controlled node.

Optionally, in an embodiment of the present invention, described main controllednode 101 is used for the data of inserting are preserved.

Optionally, in an embodiment of the present invention, described main controllednode 101 is used for recording the sign of the data that are modified and the data after corresponding the renewal;

Optionally, in an embodiment of the present invention, described main controllednode 101 is used for the sign that record needs the data of deletion.

Optionally, in an embodiment of the present invention, describedaffairs node 102 adopts the relational database management system engine.

Optionally, in an embodiment of the present invention, described main controllednode 101 adopts the main storage data base engine.

Optionally, in an embodiment of the present invention, it regularly is static data on the disk with described daynamic transformation that described main controllednode 101 is further used for.

Optionally, in an embodiment of the present invention, described main controllednode 101 is further used for regularly described dynamic data being forwarded to described affairs node;

Describedaffairs node 102 is further used for the data that described main controllednode 101 forwards are converted to static data on the disk.

As shown in Figure 2, one of a kind of data processing method process flow diagram that proposes for the present invention.The method comprises:

Step 201): the query manipulation of the static data in the described affairs node processing Cluster Database;

Step 202): described main controlled node carries out the Data Update operation to Cluster Database, produces dynamic data, preserves and inquire about described dynamic data.

As shown in Figure 3, two of a kind of data processing method process flow diagram that proposes for the present invention.The method also comprises:

Step 203): described main controlled node is inquired about described dynamic data and is obtained the first Query Result, and is forwarded to described affairs node;

Step 204): described affairs node obtains the Static Inquiry result according to the static data on the data processing request inquiry disk, according to sign described the first Query Result and described Static Inquiry result merging is obtained result set.

Embodiment:

Present embodiment provides database management system architecture and data processing method thereof under a kind of cluster environment of new shared disk framework, utilize the advantage (such as up-to-date hardware technologies such as large internal memory, SSD) of up-to-date hardware technology, significantly improved the performance of database, and take full advantage of many collaborative computer work, break through the limitation of database aspect extendability, improve the performance of system and the handling capacity of issued transaction.

As shown in Figure 4, be a kind of clustered database system Organization Chart based on the shared disk framework of the embodiment of the invention.The computer node that forms data-base cluster has two types: affairs node and main controlled node; The affairs node adopts relational database management system (RDB) engine, and main controlled node adopts main storage data base (MMDB) engine.The affairs node is accepted user's data processing request.Static data on the affairs querying node disk, all increase, delete, change all and carry out at main controlled node data-base cluster, upgrade the result and are kept in the internal memory of main controlled node.When the data buffer of main controlled node is full of or system when idle, system then merges on the disk upgrading batch data, forms new static data.

The operation that increases, deletes, changes in the request is processed by main controlled node; For query manipulation in user's the request, then be divided into two parts and carry out: the static data on the affairs querying node disk, master control node inquiry dynamic data is merged into correct result set to these two parts, returns to the user.

For the update (INSERT statement) of database, processed by main controlled node, be stored in the internal storage data buffer zone;

For the retouching operation (UPDATE statement) of database, at first inquire the old tuple that needs modification, generate the new tuple after upgrading; New tuple is processed by main controlled node, the sign of record modification tuple and corresponding new tuple in the data buffer of main controlled node;

For the deletion action (DELETE statement) of database, at first inquiry needs the old tuple of modification, if exist, then the tuple of record deletion identifies in the data buffer of main controlled node;

Query manipulation (SELECT statement) for database, be divided into two parts: static data inquiry and dynamic data retrieval, the static data inquiry is carried out at the affairs node, dynamic data retrieval carries out at main controlled node, according to the tuple sign these two parts is merged the final correct result of formation and returns to the user;

Generally, a typical shared disk framework data-base cluster is made of a main controlled node and a plurality of affairs node.In high available configuration situation, also can increase by a station server, as the standby host of main controlled node, for the special two-shipper of main controlled node configuration is done memory mirror, thereby guarantee the real uninterrupted of Transaction Service, standby host also can be accepted read request simultaneously.

In addition, can also be on same server simultaneously operational relation database (RDB) engine and main storage data base (M MDB) engine, be that main controlled node and affairs node are integrated on the same station server, the in the manner described above simultaneously user of process database request.

As shown in Figure 5, be data processing method process flow diagram in the embodiment of the invention.The method comprises:

Step 501): client sends the SQL query instruction to the affairs node;

Step 502): the affairs node is resolved the generated query plan to described SQL query instruction;

Step 503): when inquiry plan inserted data, described affairs node was sent to main controlled node with the data of inserting;

When inquiry plan was deleted data, described affairs node and described main controlled node were carried out the sign that query manipulation obtains the data that need deletion, and will need the sign of the data of deleting to be sent to main controlled node;

When inquiry plan was revised data, the sign of the data that described affairs querying node goes out to be modified was sent to described main controlled node with the sign of the data that are modified and the data after corresponding the renewal;

Step 504): when inquiry plan was inquired about data, the static data on the described affairs querying node disk obtained the Static Inquiry result;

Step 505): described main controlled node is carried out the operation of inquiry plan, and returns a result set to described affairs node;

Step 506): described affairs node obtains the net result collection according to sign with result set and the described Static Inquiry result merging that described main controlled node returns, and described net result collection is back to described client.

Preferably, also comprise: described affairs node abnormality processing occurs after the generated query plan is resolved in described SQL query instruction, and then described affairs node sends wrong indication to described client.

As shown in Figure 6, be the data processing method process flow diagram of the embodiment of the invention based on the system architecture of Fig. 4.The below uses a concrete database manipulation example to further specify in conjunction with Fig. 6:

Tentation data storehouse cluster is comprised of three node N1, N2 and N3, and wherein N1, N2 are the affairs nodes, and N3 is main controlled node;

Tentation data has Table A in the storehouse, and three tuples (1,1,1) are arranged among the A, and (2,2,2), (3,3,3) suppose that first row is the unique identification of tuple in the table;

When node N1 received the user and inquires about the request of A table, three tuples of Table A were read the data buffer of node N1 from disk, inquire 3 tuples from node N1, query node N3 then, and the tuple without the A table then returns to three tuples of user;

When node N1 received the user to the A table new tuple of insertion (4,4,4) request, node N1 constructed new tuple, sends to node N3, is inserted into the data buffer of node N3;

When node N1 received the user tuple (1,1,1) in the A table is modified as the request of (1,2,2), then node N1 constructed new tuple (1,2,2), sends to node N3, and new tuple is inserted into the data buffer of node N3;

When node N1 receives the user and deletes tuple (2,2,2) request in the A table, then node N1 at first acknowledgment of your inquiry have tuple (2,2,2), then notify node N3, in the data buffer of node N3, insert a special tuple (2, del);

When node N1 receives when the user inquires about the A table again, from node N1, inquire three tuples (1,1,1), (2,2,2), (3,3,3), from node N3, inquire two tuples (1,2,2), (2, del), node N1 merges two result sets, can obtain result set (1,2,2), (3,3,3), return to the user.

Above-described embodiment; purpose of the present invention, technical scheme and beneficial effect are further described; institute is understood that; the above only is the specific embodiment of the present invention; the protection domain that is not intended to limit the present invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (14)

1. the clustered database system based on the shared disk framework is characterized in that, this system comprises: main controlled node and affairs node;

Described affairs node is used for the issued transaction of Cluster Database and the inquiry of static data;

Described main controlled node is used for Cluster Database is carried out the Data Update operation, produces dynamic data, preserves and inquire about described dynamic data.

2. the system as claimed in claim 1 is characterized in that, described main controlled node is further used for inquiring about described dynamic data, and Query Result is back to described affairs node;

Described affairs node is further used for obtaining the Static Inquiry result according to the static data on the data processing request inquiry disk, obtains result set according to the Query Result and the described Static Inquiry result merging that identify described main controlled node.

3. system as claimed in claim 1 or 2 is characterized in that, described main controlled node is used for the data of inserting are preserved.

4. system as claimed in claim 1 or 2 is characterized in that, described main controlled node is used for recording the sign of the data that need modification and the data after the corresponding renewal.

5. system as claimed in claim 1 or 2 is characterized in that, described main controlled node is used for the sign that record needs the data of deletion.

6. system as claimed in claim 1 or 2 is characterized in that, described affairs node adopts the relational database management system engine.

7. system as claimed in claim 1 or 2 is characterized in that, described main controlled node adopts the main storage data base engine.

8. system as claimed in claim 1 or 2 is characterized in that, described main controlled node is further used for regularly or is static data on the disk according to the operating position of core buffer with described daynamic transformation.

9. system as claimed in claim 1 or 2 is characterized in that, described main controlled node is further used for regularly or according to the operating position of core buffer described dynamic data is forwarded to described affairs node;

Described affairs node is further used for the data that described main controlled node forwards are converted to static data on the disk.

10. data processing method is characterized in that the method comprises:

The query manipulation of the static data in the described affairs node processing Cluster Database;

Described main controlled node carries out the Data Update operation to Cluster Database, produces dynamic data, preserves and inquire about described dynamic data.

11. method as claimed in claim 10 is characterized in that, the method also comprises:

Described main controlled node is inquired about described dynamic data and is obtained the first Query Result, and is forwarded to described affairs node;

Described affairs node obtains the Static Inquiry result according to the static data on the data processing request inquiry disk, according to sign described the first Query Result and described Static Inquiry result merging is obtained result set.

12., it is characterized in that described main controlled node is preserved the data of inserting such as claim 10 or 11 described methods.

13., it is characterized in that the sign of the data that described main controlled node record need to be modified and the data after corresponding the renewal such as claim 10 or 11 described methods.

14., it is characterized in that described main controlled node record needs the sign of the data of deletion such as claim 10 or 11 described methods.

Images