Summary of the invention
The object of the invention is to overcome the shortcoming of above-mentioned prior art, providing a kind of is carried out cache metadata and corresponding querying method is provided by NoSQL database, save system resource, improved the performance of data access, accelerate the response speed of metadata analysis, Optimizing Queries performance based on NoSQL, realize the system and method for metadata cache and analysis.
To achieve these goals, the system and method for realizing metadata cache and analysis based on NoSQL of the present invention has following formation:
Should based on NoSQL, realize the system of metadata cache and analysis, its principal feature is that described system comprises:
Metadata analysis client, in order to respond the metadata analysis instruction of user's input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and show described metadata analysis result;
Metadata analysis service end, the metadata analysis message sending in order to resolve described communication module, and the metadata analysis task obtaining according to parsing inquiry NoSQL database server obtains metadata analysis result, and the corresponding analysis result message of described metadata analysis result is sent to described communication module;
NoSQL database server, in order to generate NoSQL metadata table, and by the data buffer storage in metadatabase in described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response;
Communication module, in order to described metadata analysis message is sent to described metadata analysis server, and is sent to described metadata analysis client by described analysis result message.
Further, described metadata analysis client comprises graphic interface module and task parsing module, wherein:
Described graphic interface module, graphic interface in order to display element data analysis, and the metadata analysis instruction of the input of the user described in response, with the data of metadata analysis task corresponding to described metadata analysis instruction are sent to described task parsing module, and show the metadata analysis result that described task parsing module sends;
Described task parsing module, in order to the data of described metadata analysis task are converted to corresponding metadata analysis message, and described metadata analysis message is sent to described communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and described metadata analysis result is sent to described graphic interface module.
Further, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, wherein:
Described task scheduling modules, in order to resolve the metadata analysis message of described communication module transmission and to obtain corresponding metadata analysis task, and by described NoSQL database interface, inquire about described NoSQL database server according to described metadata analysis task, and obtain described metadata analysis result corresponding to metadata analysis task, and be institute's correspondence analysis results messages by corresponding metadata analysis results conversion;
Described NoSQL database interface, in order to access described NoSQL database server.
Further, it is characterized in that, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, wherein:
Described metadata management module, in order to generate NoSQL metadata table and NoSQL database, and by the data buffer storage in described metadatabase in described NoSQL database, and obtain related data information by described metadata query module inquiry, and described related data information is saved in described NoSQL metadata table, and the inquiry request of the metadata analysis service end described in response;
Described metadata query module, in order to inquire about described NoSQL database according to row key word by described application programming interfaces, and obtains related data information;
Described application programming interfaces, in order to access the application program of described NoSQL database server.
Further, described NoSQL database server comprises name module, described name module in order to the NoSQL database server to described register, add, the management of monitoring and fault detect.
Further, described NoSQL database comprises metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
Wherein, described metadata analysis instruction comprises metadata analysis function type, the information of metadata type and metadata title, described metadata analysis task is consanguinity analysis task, impact analysis task, association analysis task, topological analysis task or index analysis task, described metadata analysis task comprises client the Internet protocol address information, client end slogan information, task sequence information, task names information, task type information and mission bit stream, described metadatabase comprises metadata example table, metadata combination relation table and metadata dependence table.
In addition, the present invention also provides a kind of and utilizes described system to realize the metadata cache method based on NoSQL, and described method is specially:
Described NoSQL database server generates NoSQL metadata table, and by the data buffer storage in metadatabase to described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response.
Further, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, described NoSQL database server generate NoSQL metadata table and by the data buffer storage in metadatabase to described NoSQL metadata table, comprise the following steps:
(a) described metadata management module generates NoSQL metadata table and NoSQL database;
(b) described metadata management module by the data buffer storage in described metadatabase to described NoSQL database;
(c) described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information;
(d) described metadata management module is saved to described related data information in described NoSQL metadata table.
Further, described metadata management module generates NoSQL metadata table and NoSQL database, is specially:
Described metadata management module generates NoSQL metadata table, metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
Further, described metadata management module to described NoSQL database, comprises the following steps the data buffer storage in described metadatabase:
(b.1) the metadata example table in described metadata management module query metadata storehouse, and by the data buffer storage of described metadata example table to described metadata instance objects storehouse;
(b.2) the metadata dependence table in the described metadatabase of described metadata management module inquiry, and by the data buffer storage of described metadata dependence table to described metadata dependence library of object;
(b.3) the metadata combination relation table in the metadatabase described in described metadata management module inquiry, and the data buffer storage that described metadata is combined to relation table is to described metadata combination relationship object storehouse.
Further, described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information, comprises the following steps:
(c.1) described metadata query module be take metadata exemplary sequences number for the metadata instance objects storehouse described in described row key word is inquired about by described application programming interfaces, and obtains in metadata instance objects storehouse the data message except other attribute of metadata metadata exemplary sequences number;
(c.2) described metadata query module be take dependence metadata sequence number as described row key word is by the described described metadata dependence library of object of application programming interfaces inquiry, and obtains the data message that relies on metadata sequence number;
(c.3) described metadata query module be take composite unit data sequence number as described row key word is by the described metadata combination relationship object storehouse of described application programming interfaces inquiry, and obtains the data message of composite unit data sequence number.
Further, described metadata management module is saved to described related data information in described NoSQL metadata table, is specially:
Described metadata management module is saved to the data message of the data message of the data message of described other attribute of metadata, described dependence metadata sequence number and described composite unit data sequence number in described NoSQL metadata table.
In addition, the present invention is also a kind of utilizes above-mentioned method to realize the metadata processing method based on NoSQL, and described method comprises the following steps:
(1) the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module;
(2) described communication module is sent to described metadata analysis server by described metadata analysis message;
(3) described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result;
(4) described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result;
(5) described communication module is sent to described metadata analysis client by described analysis result message;
(6) the analysis result message that the described described communication module of metadata analysis client parsing sends obtains described metadata analysis result, and the metadata analysis result described in showing.
Further, described metadata analysis client comprises graphic interface module and task parsing module, the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, comprise the following steps:
(1.1) described graphic interface module display graphics interface;
(1.2) the metadata analysis instruction that described graphic interface module responds user inputs also obtains corresponding metadata analysis task;
(1.3) described graphic interface module is sent to described task parsing module by the data of metadata analysis task corresponding to described metadata analysis instruction;
(1.4) described task parsing module is converted to corresponding metadata analysis message by the data of described metadata analysis task, and described metadata analysis message is sent to described communication module.
Further, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result, comprise the following steps:
(3.1) described task scheduling modules is resolved the metadata analysis message of described communication module transmission and is obtained corresponding metadata analysis task;
(3.2) described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtains described metadata analysis result corresponding to metadata analysis task;
(3.3) described task scheduling modules is institute's correspondence analysis results messages by corresponding metadata analysis results conversion.
Further, described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtain described metadata analysis result corresponding to metadata analysis task, comprise the following steps:
(3.2.1) described task scheduling modules generates corresponding query task according to described query task;
(3.2.2) described task scheduling modules, according to described query task, by the described described NoSQL metadata table of NoSQL database interface inquiry, and obtains the target metadata exemplary sequences number that will analyze;
(3.2.3) described task scheduling modules generates analysis result object according to the data in described NoSQL metadata table, and take described analysis result object as root object establishment analysis result object tree;
(3.2.4) described task scheduling modules reads target-dependent metadata sequence number and the objective cross metadata sequence number in described analysis result object and joins in described query task;
(3.2.5) to take described target metadata exemplary sequences number be row key word to described task scheduling modules, and by the NoSQL metadata table described in described application programming interfaces inquiry;
(3.2.6) described task scheduling modules adds metadata corresponding to described row key word in described analysis result object tree;
(3.2.7) described task scheduling modules is deleted the target metadata exemplary sequences number that has completed inquiry, target-dependent metadata sequence number or objective cross metadata sequence number from query task;
(3.2.8) described task scheduling modules is by remaining target metadata exemplary sequences number in described query task, target-dependent metadata sequence number or objective cross metadata sequence number are described row key word, return to above-mentioned steps (3.2.6), until described query task is empty.
Further, described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result, comprises the following steps:
(4.1) described metadata analysis service end is using described analysis result object tree as described analysis result;
(4.2) described metadata analysis service end encapsulates described analysis result with XML form, and obtains described analysis result message;
(4.3) described metadata analysis service end is sent to described communication module by described analysis result message.
Further, described metadata analysis client comprises graphic interface module and task parsing module, the analysis result message that described metadata analysis client is resolved described communication module transmission obtains described metadata analysis result, and show described metadata analysis result, comprise the following steps:
(6.1) described task parsing module is resolved described metadata analysis result, and described metadata analysis result is sent to described graphic interface module;
(6.2) described graphic interface module shows described metadata analysis result.
Adopted and based on NoSQL, realized the system and method for metadata cache and analysis in this invention, with NoSQL database come Preservation Metadata example and relation information data, do not need to carry out save data with a plurality of tables in relevant database, without the Mapping and Converting instrument that uses Object-Relation, there is not the problem of Impedance Mismatch.Meanwhile, NoSQL metadata table will often need adjacent the depositing of data rows of access simultaneously, the time of having reduced magnetic head tracking, improved the efficiency of magnetic disc i/o, the response time of data query greatly reduces.
In addition, the redundant storage of NoSQL database supported data, by the incidence relation between metadata is kept in metadata example information, can reduce the number of times of data base querying, reduces the resource consumption of database server.Final system framework is simple, has high scalability, can increase as required NoSQL database server; Working service is convenient and swift, stable and reliable working performance, and the scope of application is comparatively extensive, for further developing of enterprise's metadata analysis application established solid foundation.
Embodiment
In order more clearly to describe technology contents of the present invention, below in conjunction with specific embodiment, conduct further description.
Refer to Fig. 1, in one embodiment, the system that realizes metadata cache and analysis based on NoSQL of the present invention comprises:
Metadata analysis client, in order to respond the metadata analysis instruction of user's input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and show described metadata analysis result;
Metadata analysis service end, the metadata analysis message sending in order to resolve described communication module, and the metadata analysis task obtaining according to parsing inquiry NoSQL database server obtains metadata analysis result, and the corresponding analysis result message of described metadata analysis result is sent to described communication module;
NoSQL database server, in order to generate NoSQL metadata table, and by the data buffer storage in metadatabase in described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response;
Communication module, in order to described metadata analysis message is sent to described metadata analysis server, and is sent to described metadata analysis client by described analysis result message.
Wherein, NoSQL database server provides the monitoring of NoSQL data-base cluster and management, the inquiry request of response to NoSQL database, availability and the reliability of guarantee NoSQL data-base cluster.Communication module can also provide reliable message stores and monitoring mechanism.
A kind of preferred embodiment in, described metadata analysis client comprises graphic interface module and task parsing module, wherein:
Described graphic interface module, graphic interface in order to display element data analysis, and the metadata analysis instruction of the input of the user described in response, with the data of metadata analysis task corresponding to described metadata analysis instruction are sent to described task parsing module, and show the metadata analysis result that described task parsing module sends;
Described task parsing module, in order to the data of described metadata analysis task are converted to corresponding metadata analysis message, and described metadata analysis message is sent to described communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and described metadata analysis result is sent to described graphic interface module.
A kind of preferred embodiment in, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, wherein:
Described task scheduling modules, in order to resolve the metadata analysis message of described communication module transmission and to obtain corresponding metadata analysis task, and by described NoSQL database interface, inquire about described NoSQL database server according to described metadata analysis task, and obtain described metadata analysis result corresponding to metadata analysis task, and be institute's correspondence analysis results messages by corresponding metadata analysis results conversion;
Described NoSQL database interface, in order to access described NoSQL database server.
In a kind of preferred embodiment, it is characterized in that, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, wherein:
Described metadata management module, in order to generate NoSQL metadata table and NoSQL database, and by the data buffer storage in described metadatabase in described NoSQL database, and obtain related data information by described metadata query module inquiry, and described related data information is saved in described NoSQL metadata table, and the inquiry request of the metadata analysis service end described in response;
Described metadata query module, in order to inquire about described NoSQL database according to row key word by described application programming interfaces, and obtains related data information;
Described application programming interfaces, in order to access the application program of described NoSQL database server.
In a kind of preferred embodiment, described NoSQL database server comprises name module, described name module in order to the NoSQL database server to described register, add, the management of monitoring and fault detect.
In a kind of preferred embodiment, described NoSQL database comprises metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
Wherein, described metadata analysis instruction comprises metadata analysis function type, the information of metadata type and metadata title, described metadata analysis task is consanguinity analysis task, impact analysis task, association analysis task, topological analysis task or index analysis task, described metadata analysis task comprises client the Internet protocol address information, client end slogan information, task sequence information, task names information, task type information and mission bit stream, described metadatabase comprises metadata example table, metadata combination relation table and metadata dependence table.
In addition, the present invention also provides a kind of and utilizes described system to realize the metadata cache method based on NoSQL, and described method is specially:
Described NoSQL database server generates NoSQL metadata table, and by the data buffer storage in metadatabase to described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response.
A kind of preferred embodiment in, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, described NoSQL database server generate NoSQL metadata table and by the data buffer storage in metadatabase to described NoSQL metadata table, comprise the following steps:
(a) described metadata management module generates NoSQL metadata table and NoSQL database;
(b) described metadata management module by the data buffer storage in described metadatabase to described NoSQL database;
(c) described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information;
(d) described metadata management module is saved to described related data information in described NoSQL metadata table.
A kind of preferred embodiment in, described metadata management module generates NoSQL metadata table and NoSQL database, is specially:
Described metadata management module generates NoSQL metadata table, metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
In a kind of preferred embodiment, described metadata management module to described NoSQL database, comprises the following steps the data buffer storage in described metadatabase:
(b.1) the metadata example table in described metadata management module query metadata storehouse, and by the data buffer storage of described metadata example table to described metadata instance objects storehouse;
(b.2) the metadata dependence table in the described metadatabase of described metadata management module inquiry, and by the data buffer storage of described metadata dependence table to described metadata dependence library of object;
(b.3) the metadata combination relation table in the metadatabase described in described metadata management module inquiry, and the data buffer storage that described metadata is combined to relation table is to described metadata combination relationship object storehouse.
In a kind of preferred embodiment, described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information, comprises the following steps:
(c.1) described metadata query module be take metadata exemplary sequences number for the metadata instance objects storehouse described in described row key word is inquired about by described application programming interfaces, and obtains in metadata instance objects storehouse the data message except other attribute of metadata metadata exemplary sequences number;
(c.2) described metadata query module be take dependence metadata sequence number as described row key word is by the described described metadata dependence library of object of application programming interfaces inquiry, and obtains the data message that relies on metadata sequence number;
(c.3) described metadata query module be take composite unit data sequence number as described row key word is by the described metadata combination relationship object storehouse of described application programming interfaces inquiry, and obtains the data message of composite unit data sequence number.
In a kind of preferred embodiment, described metadata management module is saved to described related data information in described NoSQL metadata table, is specially:
Described metadata management module is saved to the data message of the data message of the data message of described other attribute of metadata, described dependence metadata sequence number and described composite unit data sequence number in described NoSQL metadata table.
In addition, the present invention is also a kind of utilizes above-mentioned method to realize the metadata processing method based on NoSQL, and as shown in Figure 2, described method comprises the following steps:
(1) the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module;
(2) described communication module is sent to described metadata analysis server by described metadata analysis message;
(3) described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result;
(4) described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result;
(5) described communication module is sent to described metadata analysis client by described analysis result message;
(6) the analysis result message that the described described communication module of metadata analysis client parsing sends obtains described metadata analysis result, and the metadata analysis result described in showing.
A kind of preferred embodiment in, described metadata analysis client comprises graphic interface module and task parsing module, the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, comprise the following steps:
(1.1) described graphic interface module display graphics interface;
(1.2) the metadata analysis instruction that described graphic interface module responds user inputs also obtains corresponding metadata analysis task;
(1.3) described graphic interface module is sent to described task parsing module by the data of metadata analysis task corresponding to described metadata analysis instruction;
(1.4) described task parsing module is converted to corresponding metadata analysis message by the data of described metadata analysis task, and described metadata analysis message is sent to described communication module.
A kind of preferred embodiment in, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result, comprise the following steps:
(3.1) described task scheduling modules is resolved the metadata analysis message of described communication module transmission and is obtained corresponding metadata analysis task;
(3.2) described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtains described metadata analysis result corresponding to metadata analysis task;
(3.3) described task scheduling modules is institute's correspondence analysis results messages by corresponding metadata analysis results conversion.
In a kind of preferred embodiment, described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtain described metadata analysis result corresponding to metadata analysis task, comprise the following steps:
(3.2.1) described task scheduling modules generates corresponding query task according to described query task;
(3.2.2) described task scheduling modules, according to described query task, by the described described NoSQL metadata table of NoSQL database interface inquiry, and obtains the target metadata exemplary sequences number that will analyze;
(3.2.3) described task scheduling modules generates analysis result object according to the data in described NoSQL metadata table, and take described analysis result object as root object establishment analysis result object tree;
(3.2.4) described task scheduling modules reads target-dependent metadata sequence number and the objective cross metadata sequence number in described analysis result object and joins in described query task;
(3.2.5) to take described target metadata exemplary sequences number be row key word to described task scheduling modules, and by the NoSQL metadata table described in described application programming interfaces inquiry;
(3.2.6) described task scheduling modules adds metadata corresponding to described row key word in described analysis result object tree;
(3.2.7) described task scheduling modules is deleted the target metadata exemplary sequences number that has completed inquiry, target-dependent metadata sequence number or objective cross metadata sequence number from query task;
(3.2.8) described task scheduling modules is by remaining target metadata exemplary sequences number in described query task, target-dependent metadata sequence number or objective cross metadata sequence number are described row key word, return to above-mentioned steps (3.2.6), until described query task is empty.
In a kind of preferred embodiment, described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result, comprises the following steps:
(4.1) described metadata analysis service end is using described analysis result object tree as described analysis result;
(4.2) described metadata analysis service end encapsulates described analysis result with XML form, and obtains described analysis result message;
(4.3) described metadata analysis service end is sent to described communication module by described analysis result message.
A kind of preferred embodiment in, described metadata analysis client comprises graphic interface module and task parsing module, the analysis result message that described metadata analysis client is resolved described communication module transmission obtains described metadata analysis result, and show described metadata analysis result, comprise the following steps:
(6.1) described task parsing module is resolved described metadata analysis result, and described metadata analysis result is sent to described graphic interface module;
(6.2) described graphic interface module shows described metadata analysis result.
In a preferred specific embodiment, the method that realizes metadata cache and analysis based on NoSQL of the present invention comprises in fact two operations: metadata import operation (being metadata cache) and metadata analysis are processed operation.Data importing operation imports to the metadata being kept in relevant database in NoSQL database and carries out corresponding data-switching.Metadata analysis is processed and is operated in to carry out inquiry in the metadata of NoSQL database caches, carries out relevant metadata analysis, and operation is as follows in detail:
First, the step of metadata import operation is as follows:
1, create metadata instance objects storehouse, wherein, the field of the Property Name of object, attribute type and metadata example table is corresponding one by one;
2, query metadata example table, is saved in the value of metadatabase record in metadata instance objects storehouse;
Wherein, the form in metadata instance objects storehouse is: metadata example ID (sequence number), metadata title, metadata type, metadata attributes 1, metadata attributes 2, metadata attributes 3
3, call the API (application programming interfaces) that NoSQL database server provides, in NoSQL database server, create NoSQL metadata table, for depositing metadata example and relation data;
Wherein, NoSQL metadata tableau format is: row key word, and metadata example row family, dependence row family, syntagmatic row family, the data in row family comprise the value of a plurality of versions, the corresponding timestamp of each value;
In metadata example row family, comprise a plurality of row such as metadata title, metadata type, metadata attributes 1, metadata attributes 2, metadata attributes 3; Dependence row family comprises row of dependence metadata ID; Syntagmatic row family comprises row of composite unit data ID;
4, call the API that NoSQL database server provides, take metadata example ID as row key word, other attribute except metadata example ID in metadata instance objects storehouse is all saved in NoSQL database instance;
5, create metadata dependence library of object, wherein, the field of the Property Name of object, attribute type and metadata dependence table is corresponding one by one;
6, query metadata dependence table, is saved in the value of metadatabase record in metadata dependence library of object;
Wherein, metadata dependence library of object form is: metadata example ID, relies on metadata ID;
7, call the API that NoSQL database server provides, the metadata example ID of take in dependence object is inquired about in NoSQL database as row key word, by relying on metadata ID, is updated in the dependence metadata ID row in dependence row family;
8, create metadata combination relationship object storehouse, wherein, the field of the Property Name of object, attribute type and metadata combination relation table is corresponding one by one;
9, query metadata syntagmatic table, is saved in the value of metadatabase record in metadata combination relationship object storehouse;
Wherein, metadata combination relationship object library format is: metadata example ID, composite unit data ID;
10, call the API that NoSQL database server provides, the metadata example ID of take in syntagmatic library of object is inquired about in NoSQL database as row key word, and composite unit data ID is updated in the composite unit data ID row in syntagmatic row family.
Secondly, suppose to have a plurality of metadata analysis clients and metadata analysis service end, the step that metadata analysis is processed operation is as follows:
1, communication module initialization
11, read configuration file, obtain a plurality of analysis client ids and Analysis Service end ID;
12, according to analyzing client id, for each metadata analysis client, set up the queue of message sending and receiving;
13, according to Analysis Service end ID, for each metadata analysis service end, set up the queue of message sending and receiving;
14, watcher thread starts, the arrival waiting for the arrival of news;
2, NoSQL database server carries out initialization;
21, start naming service module in NoSQL database server;
22, start metadata management module in NoSQL database server;
23, start NoSQL metadatabase example in NoSQL database server;
24, after NoSQL metadata table starts successfully, to the name module information that reports in;
25, the information that metadata query module must all available NoSQL database instances;
26, metadata management module is communicated by letter with available NoSQL database instance, sets up the database table of NoSQL database instance and storage and the corresponding relation of data field;
27, metadata management module is inquired about all data field information, calculates unappropriated data field, and they are added to data field table to be allocated;
18, metadata management module is inquired about available NoSQL database instance information, unappropriated data separation dispensing NoSQL database instance is stored, and preserved corresponding relation;
3, metadata analysis service end is carried out initialization;
31, to communication module, send test post, if receive response, illustrate normally with being connected of communication module, confiscate to respond and provide miscue;
32, to the name module in NoSQL database server, send test post, confirm the normally operation of NoSQL data-base cluster;
33, the task scheduling thread of initiating task scheduler module, waits for the submission of metadata analysis task;
4, user submits to the data that will analyze to metadata analysis client;
41, the metadata analysis function that user is chosen metadata node and determined to carry out by graphic interface;
Wherein, metadata analysis function type comprises: consanguinity analysis, impact analysis, association analysis, topological analysis, index analysis etc.;
42, graphic interface module receives user's action and submits to related data to metadata analysis client;
Wherein, the data layout that graphic interface module is submitted to is: metadata analysis function type, metadata type, metadata title;
5, metadata analysis client is submitted to metadata analysis service end by analysis data and is analyzed;
51, the data that metadata analysis client sends to graphic interface module;
52, metadata analysis client encapsulates data in metadata analysis task;
Wherein, the form of metadata analysis task is: client ip address, client end slogan, task ID, task names, task type, mission bit stream
53, metadata analysis client sends to metadata analysis service end by metadata analysis message by communication module;
6, metadata analysis service end is carried out metadata analysis method, obtains analysis result;
61, the metadata analysis message that the task scheduling modules received communication module of metadata analysis service end is sent, starts new analysis thread execution metadata analysis task;
62, the parameter in metadata analysis service end analytical element data analysis message, obtains the metadata analysis task that will analyze;
Wherein, the form of metadata analysis message is: metadata analysis function type, metadata type, metadata title;
63, metadata analysis service end is used different query strategies according to the type of metadata analysis function, generates different query tasks;
Wherein, because the type of metadata analysis function mainly comprises: consanguinity analysis (blood lineage's analysis), impact analysis, entity associated analysis, entity impact analysis, main frame topological analysis, indicator consilience analysis etc., therefore corresponding query strategy comprises: consanguinity analysis strategy, impact analysis strategy, association analysis strategy, topological analysis strategy, consistency analysis strategy etc.;
64, metadata analysis service end is according to the query task generating, data in the inquiry API Access NoSQL metadata table that use NoSQL database server provides, the metadata example ID that acquisition will be analyzed, wherein, described inquiry API refers to the API of data query in the performing database that NoSQL provides;
65, metadata analysis service end obtains after metadata example ID, to in NoSQL database, take metadata example ID as row key word, in metadata example row family, dependence row family and syntagmatic row family, the value of the latest edition of each row generates analysis result object, and take this object as root object establishment analysis result object tree;
66, metadata analysis service end directly reads dependence metadata ID and the syntagmatic metadata ID in analysis result object and joins in query task;
67, metadata analysis service end reads the concentrated metadata ID of query task, the inquiry API that calling NoSQL database server provides inquires about the metadata record that the metadata ID of take in task-set is row key word and joins in analysis result object tree in metadata table, take that to have the metadata of mutual relationship be parent object;
68, metadata analysis service end directly reads dependence metadata ID and the syntagmatic metadata ID of metadata record in Query Result and joins query task and concentrates;
69, from query task, concentrate deletion to complete the metadata example ID of inquiry;
610, using the concentrated metadata ID of query task as row key word, carry out query steps above, until query task collection be sky
7, metadata analysis service end returns to analysis result to metadata analysis client;
71, metadata analysis service end encapsulates analysis result object tree with XML form, forms analysis result message content;
72, analysis result message is sent to metadata analysis client by communication module;
8, metadata analysis client represents;
81, metadata analysis client, from communication module receipt message, sends to graphic interface module by analysis result object tree after parsing;
82, graphic interface module represents analysis result object tree with graphical form;
Adopted and based on NoSQL, realized the system and method for metadata cache and analysis in this invention, with NoSQL database come Preservation Metadata example and relation information data, do not need to carry out save data with a plurality of tables in relevant database, without the Mapping and Converting instrument that uses Object-Relation, there is not the problem of Impedance Mismatch.Meanwhile, NoSQL metadata table will often need adjacent the depositing of data rows of access simultaneously, the time of having reduced magnetic head tracking, improved the efficiency of magnetic disc i/o, the response time of data query greatly reduces.
In addition, the redundant storage of NoSQL database supported data, by the incidence relation between metadata is kept in metadata example information, can reduce the number of times of data base querying, reduces the resource consumption of database server.Final system framework is simple, has high scalability, can increase as required NoSQL database server; Working service is convenient and swift, stable and reliable working performance, and the scope of application is comparatively extensive, for further developing of enterprise's metadata analysis application established solid foundation.
In this instructions, the present invention is described with reference to its specific embodiment.But, still can make various modifications and conversion obviously and not deviate from the spirit and scope of the present invention.Therefore, instructions and accompanying drawing are regarded in an illustrative, rather than a restrictive.