Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, with the increase of the number of users and the increase of personalized function requirements of the users, different structures and data can be obtained for data stored in a database for realizing a certain function after modification or supplement, and common functions may exist in the same industry or different industries, so that different project functions can be obtained according to the same product function, and the project functions are obtained by supplementing, deleting or modifying on the basis of the product functions so as to meet various personalized requirements of the users. If the related data of the developed product functions can be reused, the design and development time of new functions can be greatly saved, and the production efficiency is improved.
Based on this, the information synchronization method and apparatus provided in the embodiments of the present invention can improve the reusability of the product function database structure, save design and development time, improve production efficiency, and can also solve the problem of synchronization between a distributed product and a multi-item database.
For the convenience of understanding the embodiment, a detailed description will be given to an information synchronization method disclosed in the embodiment of the present invention.
An embodiment of the present invention provides an information synchronization method, referring to a flow chart of the information synchronization method shown in fig. 1, the method includes the following steps:
and step S102, acquiring the first information and the second information according to preset time.
In an embodiment of the invention, the first information is stored in a first database; the second information is stored in a second database. The second information can be processed by one or more of deleting, adding and modifying to generate a plurality of kinds of first information, and each kind of first information can be stored in different first databases. The second information of different kinds may be stored in different second databases. According to scanning at a preset time, a first database, first information in the first database and second information in a corresponding second database can be obtained.
It should be noted that, in order to implement a certain service function, the second information may be a general version, that is, the second database may open access rights to multiple users, and the first information is a customized version generated based on the second information, and may open access rights of the first database only to specified users.
It should be further noted that the second information may include a table structure, a field included in the table structure, data included in the field, and other information, and one or more service functions may be implemented through a combination of the table structure, the field, the data, and other information. For example, the first information is generated according to the second information, if the second information includes table 1, where table 1 includes field a, field b, and field c, and field a includes functional data d, functional data e, and functional data f, the first information obtained by field addition and functional data reduction may include table 1, where table 1 includes field a, field b, field c, and field d, and field a includes functional data d and functional data e; the other type of first information obtained through the modification of the functional data may include table 1, where table 1 includes field a, field b, and field c, and field a includes functional data d, functional data e, and functional data h. The second information can be processed according to actual requirements to obtain different first information.
In addition, for multiple types of first information corresponding to the same second information, the multiple types of first information may be stored in a distributed manner, and different times may be set to scan different types of first information.
And step S104, comparing the first information with the second information to obtain difference function data.
In the embodiment of the present invention, the functional data is a part of the first information or the second information, and the functional data may be one or a combination of any several of a table structure, a field, or data. By contrast, if the first information includes some function data and the second information does not include the function data, the function data may be used as difference function data.
It should be noted that comparing includes comparing the table structure with the data. For example, the data comparison may be performed by comparing the first information with the second information, where the second information includes table 1, table 1 includes field a, field b, and field c, field a includes function data d, function data e, and function data f, the first information includes table 1, table 1 includes field a, field b, field c, and field d, and field a includes function data d and function data e, and the obtained difference function data is: a field d. However, the functional data f is not present in the first information because it is present in the second information, and cannot be regarded as difference functional data.
And step S106, synchronizing the difference function data to a second database.
In the embodiment of the present invention, if there is a difference after comparison, the difference is automatically synchronized. Synchronization includes synchronizing the table structure and data of the database into the database. And copying and sending the difference function data to a second database, and establishing a table in the second database to realize the synchronization of the table structure when the table structure included in the difference function data is synchronously operated. The second database may be tabulated using SQL (Structured Query Language).
It should be noted that, in the embodiment of the present invention, for a plurality of pieces of first information corresponding to the same second information, parallel comparison may be performed to improve comparison efficiency, but synchronization needs to be performed one by one to ensure accuracy of a synchronization result.
In addition, after the data in the first database is synchronized to the second database, the user can conveniently access the second database to obtain the synchronized information, and after multiple times of synchronization, the second database comprises multiple kinds of first information in different first databases, so that the user can combine and use the multiple kinds of first information in the second database, further realize the multiplexing of the data, and save the design and development time.
The embodiment of the invention provides an information synchronization scheme, which is characterized in that first information in a first database and second information in a second database are obtained according to preset time, the first information is generated according to the second information, then the first information is compared with the second information, content which is included in the first information but not included in the second information is used as difference function data, and finally the difference function data are synchronized to the second database so as to enrich information in the second database. The embodiment of the invention can automatically compare the related information in different databases to realize the automatic synchronization of the difference function data from the first database to the second database, thereby relieving the complexity and the error of manual operation and improving the efficiency and the accuracy of data synchronization.
In order to improve the data processing efficiency of the comparison operation, before the first information and the second information are compared to obtain the difference function data, the following steps may be further performed:
the first information and the second information are loaded to a Remote Dictionary service (Redis) database.
In the embodiment of the invention, the first information and the second information to be compared are loaded into the Redis database, so that data comparison is carried out in the Redis database, and the pressure of a DB (database) database of the DBA system is reduced.
In order to further improve the accuracy of data synchronization, after the first information and the second information are compared to obtain difference function data, the following operations may be further performed:
locking the first information; judging whether the second information is locked; and if not, locking the second information.
In the embodiment of the invention, the first information is locked, so that the loading authority of the first information can be managed. After a certain execution engine acquires a certain first message, the first message is locked, and then the next execution engine can not acquire the first message any more. Similarly, details regarding the content of locking the second information are not repeated.
It should be noted that a lock is a mutually exclusive mechanism, and implements forced control over resources in a multi-thread environment to help implement concurrent control. And locking operation, so that a plurality of threads cannot be executed simultaneously, and when a certain thread is in a running state, other threads are in ready or blocking states.
In addition, in the embodiment of the present invention, if the second information is in an unlocked state, the second information is locked. If the second information is in the operation of synchronizing with other first information, it is indicated that the second information is in the locked state, and the second information needs to be locked after the second information is unlocked (i.e. synchronization of other first information and the second information is completed).
After the synchronization of the first information and the second information is completed, in order to facilitate the synchronization operation of other first information and second information, the method may further perform the following steps:
carrying out unlocking processing on the first information; and unlocking the second information.
In the embodiment of the invention, after the table structure and the data synchronization in the second database are completed, explanation processing operation is started. The unlocking processing can remove the loading authority limit of the first information and the second information, facilitate other processing of the first information, facilitate loading of the second information by other first information, and synchronize with the second information.
Considering that the first information and the second information are loaded into the Redis database for processing, the related information of the processing result is still stored in the Redis database, and in order to facilitate the user to obtain the difference function data, the synchronizing of the difference function data to the second database can be performed according to the following steps:
the difference function data is synchronized from the Redis database to the second database.
In the embodiment of the invention, the difference function data is copied and sent to the second database, and if a table structure exists in the difference function data, a table needs to be built in the second database, so that the synchronized information in the second database can realize the function which can be realized by the information in the first database.
In order to improve the correctness of the synchronized information, after the difference function data is synchronized to the second database, the following steps can be further executed:
comparing the first database with a second database after synchronizing the difference function database to obtain a verification result; the verification result is used for verifying the correctness of database synchronization.
In the embodiment of the present invention, the second database after synchronizing the difference function database should include all information in the first data, so as to implement all functions that can be implemented by the first information in the first database, and if it is found, according to the verification result obtained after the comparison, that the second database after synchronizing the difference function database does not include all information in the first data, it is not possible to implement all functions that can be implemented by the first information in the first database, and it is proved that data synchronization has a problem and needs to be paid attention.
In order to improve the efficiency of data processing in each step, before comparing the first information with the second information and obtaining the difference function data, the following steps may be further performed:
generating a plurality of micro services; the plurality of microservices includes at least: one or more of distributed timing microservice, contrast service microservice, plan synchronization microservice and verification microservice; registering various micro services through a registration center; and executing the information synchronization method according to the registered multiple microservices.
In the embodiment of the present invention, the micro service is a small service composed of a single Application, and has its own process and lightweight processing, and the service is deployed in a fully automatic manner according to a business function design, and communicates with other services using an HTTP API (Application Programming Interface). Meanwhile, the service uses a minimum scale centralized management (such as Docker) technology, the service can use different programming languages and databases, and the like. The microservices may be deployed on different servers or on different containers of the same server.
The micro-services are registered in the registration center, information such as addresses of the services can be obtained, and the registered micro-services can be mutually called.
Different kinds of micro-services can achieve different functions, for example, the distributed timing micro-service can be used for acquiring first information in a first database and second information in a second database according to preset time; the contrast service microservice can be used for contrasting the first information and the second information to obtain difference function data; the plan synchronization microservice may be used to synchronize the differential function data to a second database; the validation microservice may be used to compare the first database with a second database after synchronizing the difference function database to obtain a validation result.
Referring to the schematic diagram of the information synchronization method shown in fig. 2, the method will be described with reference to a specific embodiment.
(1) Distributed timing, contrast service, plan synchronization, verification and other micro services register services through the registration center and call each other.
(2) And scanning information such as a certain item table, a field and data in the DB management database at regular time, loading the information into the memory database Redis and locking the item.
(3) And checking whether the product corresponding to the item is locked or not, if not, immediately locking the product, and if so, waiting for the product to be unlocked.
(4) After the product is unlocked, information such as tables, fields and data of the items is compared with information related to the product, and redundant tables, fields and data in the items are merged into the information related to the product which is added to Redis.
(5) And after the plan synchronization micro-service receives the merging completion notification, the micro-service synchronizes relevant information to the product management database.
(6) After the table, the field and the data are synchronized into the database, the micro service executes to generate SQL and synchronizes to the product database.
(7) The validation microservice checks the database synchronization correctness by comparing the product database to the project database.
The embodiment of the invention provides an information synchronization method and device, the method can regularly scan information such as various project tables and fields, service data and the like in first information through micro-service and compare the information with tables and fields and the like in second information, and the information is regularly synchronized to a product database according to redundant project tables, fields, data and the like after comparison. The specified product library is not deleted, but only newly added. The method can be used for solving the problem of automatic comparison and synchronization in the management of multiple projects (customized versions for a certain user based on the universal versions) of the same product (universal version), the project functions are synchronized to the product functions by combing version information, the existing functions in the project are not required to be re-developed by workers, the data of the existing project functions can be obtained from the second database where the product function data are located, and the production efficiency is improved; the problems that the table structures of multiple items of the same product are various and complex, the complexity of manual comparison and the error are easy to achieve, the items are automatically synchronized to the product and are synchronized to other items can be solved, and the efficiency and the accuracy of data synchronization are improved.
An embodiment of the present invention further provides an information synchronization apparatus, referring to a structural block diagram of the information synchronization apparatus shown in fig. 3, where the apparatus includes:
an obtainingmodule 71, configured to obtain the first information and the second information according to a preset time; the first information is generated according to the second information; the first information is stored in a first database; the second information is stored in a second database; acomparison module 72, configured to compare the first information with the second information to obtain difference function data; the first information includes difference function data and the second information does not include difference function data; asynchronization module 73 for synchronizing the difference function data to the second database.
In one embodiment, referring to another block diagram of the information synchronization apparatus shown in fig. 4, the apparatus includes: the device also includes: and aloading module 74, configured to load the first information and the second information into a Redis database.
In one embodiment, referring to another block diagram of the information synchronization apparatus shown in fig. 4, the apparatus includes: the device also includes: a lockingmodule 75, configured to perform locking processing on the first information; judging whether the second information is locked; and if not, locking the second information.
In one embodiment, referring to another block diagram of the information synchronization apparatus shown in fig. 4, the apparatus includes: the device also includes: an unlockingmodule 76 for: carrying out unlocking processing on the first information; and unlocking the second information.
In one embodiment, the synchronization module is further configured to: the difference function data is synchronized from the Redis database to the second database.
In one embodiment, referring to another block diagram of the information synchronization apparatus shown in fig. 4, the apparatus includes: the device also includes: averification module 77 for: comparing the first database with a second database after synchronizing the difference function database to obtain a verification result; the verification result is used for verifying the correctness of database synchronization.
In one embodiment, referring to another block diagram of the information synchronization apparatus shown in fig. 4, the apparatus includes: the device also includes: amicroservice module 78 for: generating a plurality of micro services; the plurality of microservices includes at least: one or more of distributed timing microservice, contrast service microservice, plan synchronization microservice and verification microservice; registering various micro services through a registration center; and executing the information synchronization method according to the registered multiple microservices.
The embodiment of the present invention further provides a computer device, referring to the schematic block diagram of the structure of the computer device shown in fig. 5, the computer device includes amemory 81 and aprocessor 82, the memory stores a computer program that can be executed on the processor, and the processor implements the steps of any one of the methods when executing the computer program.
It is clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the computer device described above may refer to the corresponding process in the foregoing method embodiments, and no further description is provided herein
Embodiments of the present invention also provide a computer readable medium having non-volatile program code executable by a processor, the program code causing the processor to perform any of the steps of the above-described method.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.