Column name	Data type	Whether or not it is empty	Note
				id	bigint	Whether or not	Primary key id
aMsgConfigId	bigint	Whether or not	Service one
				bMsgConfigId	bigint	Whether or not	Service two
checkType	int	Whether or not	Type of collation
				aTriggerRule	text	Is that	Business-trigger rules
bTriggerRule	text	Is that	Business two-trigger rules
				aRootKey	varchar	Whether or not	Service-triggering main key
bRootKey	varchar	Is that	Service two-trigger main key
				expireTime	long	Is that	Checking the age
checkRuleConfigId	bigint	Whether or not	Data content collation rule id
				aFields	varchar	Whether or not	Business-to-collation data item
bFields	varchar	Whether or not	Business two-check data item
				alarmRecipient	varchar	Whether or not	Alarm receiver

The primary key id may be an identifier corresponding to the comparison configuration information.

The first service and the second service may refer to service descriptions or service identifiers corresponding to the first data to be compared and the second data to be compared.

The first business triggering rule and the second business triggering rule may be another comparison triggering condition, where the first business triggering rule may be configured for a database table in the first data provider or the first to-be-compared data, and is used to compare the first to-be-compared data with the database table in the other data provider or the to-be-compared data when the first business triggering rule is satisfied. Correspondingly, the second business triggering rule may be configured for a database table or second to-be-compared data in the second data provider, and is used to compare the second to-be-compared data with the database table or the to-be-compared data in the other data providers when the database table or the to-be-compared data meets the second business triggering rule.

The checking time limit can be an effective time range within which the first data to be compared and the second data to be compared can be compared. Specifically, the first data to be compared and the second data to be compared may have a certain time delay from generation to comparison, and the comparison timeliness may be used to compare the first data to be compared and the second data to be compared within a preset time delay range, thereby improving timeliness and accuracy of the data comparison result.

The collation data item may also be referred to as a collation attribute. In practice, the first data to be compared and the second data to be compared may both include a plurality of data items, that is, may include a plurality of fields, but during data comparison, each data item does not necessarily need to be compared one by one, and therefore, the first service check data item and the second service check data item may respectively designate the data items to be compared in the first data to be compared and the second data to be compared, so as to avoid comparing data contents of all data items, save comparison time, and improve data comparison efficiency.

The alarm receiver may be a receiver of the comparison result, or may be a receiver of the abnormal comparison result when the comparison result is abnormal, for example, if the comparison result is not matched, the alarm receiver may send alarm information to the alarm receiver.

The data content collation rule may be configured by the information as shown in the following table 2:

TABLE 2

Column name	Data type	Whether or not it is empty	Note
				id	bigint	Whether or not	Primary key id
ruleName	varchar	Whether or not	Rule name
				ruleDesc	varchar	Whether or not	Rule description
ruleScript	text	Whether or not	Script file
				scriptType	varchar	Whether or not	Rule type
ruleStatus	varchar	Whether or not	Status of state

The primary key id may be a data content check rule id, the rule type may be any one of groovy, avertor, or java, and the state may be used to set whether the data content check rule is currently in a use state.

In this disclosure, when a first to-be-compared data provided by a first data provider includes a corresponding first set data item and a second to-be-compared data provided by a second data provider includes a corresponding second set data item, the data comparison platform may acquire a first data content associated with the data content comparison rule in the first to-be-compared data and acquire a second data content associated with the data content comparison rule in the second to-be-compared data, and may compare the first data content and the second data content based on the data content comparison rule to determine whether the first data content and the second data content are consistent. According to the scheme, the first data to be compared and the second data to be compared can be compared when the comparison triggering conditions are met, comparison under the condition that designated information is lacked is avoided, and reliability and efficiency of data comparison are improved.

In an exemplary embodiment, in step S210, the acquiring and storing the data to be compared corresponding to each of the plurality of data providers includes:

acquiring original data in data sources corresponding to the data providers; and filtering the original data, and acquiring and storing the data to be compared corresponding to the plurality of data providers based on the filtered original data.

In a specific implementation, after a plurality of data providers are determined, data sources corresponding to data to be compared by the plurality of data providers can be determined, and original data is acquired from the data sources corresponding to the data providers.

After the original data is acquired, because data which does not need to be compared may exist in the original data, a preset data filtering rule can be acquired, the original data is omitted, the original data of irrelevant data is screened out based on the filtered original data, the original data is used as data to be compared corresponding to a plurality of data providers, and the currently acquired data to be compared is stored.

When the original data is filtered, different data providers can use different data filtering rules to respectively filter the original data corresponding to each data provider, and the same data filtering rules can also be used to filter the original data corresponding to different data providers.

In practical application, a worker can set a filtering rule of data to be compared in advance through a configuration interface, the data filtering rule can be stored in a configuration file as configuration information for filtering the data, and then the data comparing platform can read the data filtering rule from the configuration file for filtering. For example, a filtering rule "$. amount >0& & & $. state ═ SUCCESS" may be configured in advance to filter out data to be aligned in which "amount" and "state" in the acquired data to be aligned satisfy a condition.

According to the data comparison method and device, after the data comparison platform obtains the original data, the original data can be filtered, the data to be compared corresponding to a plurality of data providers are obtained and stored based on the filtered original data, the data which do not need to be compared can be filtered, and the storage and calculation cost of subsequent links is reduced.

In an exemplary embodiment, in step S210, the acquiring original data in the data source corresponding to each of the plurality of data providers includes:

acquiring pre-configured storage paths corresponding to the data providers respectively, and determining data sources corresponding to the storage paths; and acquiring original data in each data source through a preset common message queue.

The storage path is a storage path of a data source corresponding to the data provider.

In a specific implementation, a worker may perform storage path configuration in advance, where the storage path is a storage path of a data source corresponding to original data in a data provider, and by configuring the storage path in advance, a corresponding data source may be determined based on the storage path. In one example, the data path configuration may be made through table 3 as shown below:

TABLE 3

The primary key identification and the name are the primary key identification and the name corresponding to the original data, and the service description is used for representing the service described by the original data.

The data provider is a data provider configured by the staff and to-be-compared with the data, and the data comparison platform can determine the plurality of data providers 120 to be compared with the data by reading the data provider in the data path configuration. Since each data provider may have one or more databases for storing data, the databases may be used to determine the specific database associated with the data provider 120.

The database table rule can be used for screening out one or more data tables meeting the database table rule in the database, when the database table records a plurality of data about different types of services, the database table can be divided to obtain a plurality of partitions, the corresponding partitions are determined through the distribution keys, and the distribution keys can also be partition keys. The data specified in the database table may then be determined based on the service identification and the distribution key.

In this example, after determining a plurality of data providers 120 to be subjected to data comparison, a pre-configured storage path corresponding to each data provider 120 may be obtained, and original data in a data source corresponding to the storage path may be obtained through a preset common message queue.

For example, as shown in fig. 3, a first data provider and a second data provider are included, and the first data provider and the second data provider may provide log files (e.g., binlog binary log files) as the data to be compared. When performing data reconciliation, the data comparison platform 110 may read path configuration information including the contents in table 3, and determine, according to the path configuration information, a first data provider and a second data provider that are to perform data comparison, further, may also obtain, from the path configuration information, respective storage paths corresponding to the first data provider and the second data provider, and may further send a data acquisition request including the storage path to the first data provider and the second data provider.

After receiving the data acquisition request, the first data provider and the second data provider can acquire log data under the corresponding storage path, analyze the log data through a log analyzer, respectively use the data obtained after analysis as first original data and second original data, and further the first data provider and the second data provider can transmit the first original data and the second original data to the data comparison platform through a common message queue.

In one example, the common message queue may be one or more common message queues, and by setting the common message queue, application for a special message queue before performing data comparison may be avoided. The storage path corresponding to each data provider may be a storage path corresponding to two or more data providers, for example, in a scenario of comparing a single data table with a single data table, the data comparison of the data table a may only depend on the data table B, and for example, in a scenario of comparing a single data table with multiple data tables, the data comparison of the data table C may depend on the data table D and the data table E, and then the storage paths of the data table C, the data table D, and the data table E may be configured at the same time.

In this disclosure, the data comparison platform can acquire the respective storage path that corresponds of a plurality of data providers of pre-configuration, and determine the data source that the storage path corresponds, and then can acquire the raw data in each data source through the public message queue of presetting, can be through the mode of configuration, compare the changeable data of in-process and compare the flow phase separation with more stable data, when the data comparison scene changes, such as data provider or when waiting to compare the data change, can compare different data through modifying corresponding storage path, effectively improve data comparison efficiency.

In an exemplary embodiment, after the obtaining and storing the to-be-compared data corresponding to each data provider through the callable message queue, the method may further include:

The identification queue may store data identifications corresponding to currently acquired data to be compared, and the identification queue serves as a queue and may access the data identifications in a first-in first-out manner.

In practical application, after the data comparison platform 110 obtains the data to be compared, the data identifier corresponding to the data to be compared may be obtained.

Specifically, after the data to be compared is obtained, the data comparison platform 110 may obtain a check module identifier (checkUnitId), a storage path identifier (triggerMsgConfigId), and a unique identifier (rootKey) corresponding to the data to be compared, where the check module identifier is an identifier corresponding to a check module for performing data comparison on subsequent data to be compared, the storage path configuration identifier is an identifier corresponding to a storage path corresponding to the data to be compared, and the unique identifier is an identifier capable of uniquely identifying current data to be compared in a plurality of data to be compared. Furthermore, the data comparison platform can splice the check module identifier, the storage path identifier and the unique identifier, and use the spliced character string as the data identifier of the data to be compared.

After the data identifier corresponding to the data to be compared is obtained, the data identifier may be stored in a preset identifier queue. Specifically, after the data comparison platform 110 obtains all the data to be compared, the data identifier corresponding to the data to be compared can be obtained in time and stored in the identifier queue, so that a plurality of successively obtained data identifiers can be sequentially stored in the identifier queue.

In the disclosure, the data comparison platform obtains the data identifier corresponding to the compared data, and stores the data identifier in a preset identifier queue, so as to provide a basis for subsequently implementing the ordered comparison of the data to be compared.

acquiring a target data identifier of which the waiting time reaches a preset duration from a preset identifier queue; and acquiring data to be compared corresponding to the target data identification from the stored data to be compared, and taking the data to be compared as the data to be compared of the target with the waiting time reaching the preset duration.

The identification queue stores data identifications corresponding to the currently acquired data to be compared.

In practical application, after the data identifier is stored in the identifier queue, the data identifier may not be immediately taken out from the identifier queue, but after waiting for a certain time, the target data identifier whose waiting time reaches the preset duration is obtained from the preset identifier queue. After the target data identifier is obtained, the data to be compared corresponding to the target data identifier may be obtained from the stored data to be compared, and the obtained data is used as the data to be compared of the target when the waiting time reaches the preset duration.

As shown in fig. 4, after the data to be compared is obtained, the data to be compared may be stored in a cache, and a data identifier corresponding to the data to be compared may be stored in an identifier queue, where the identifier queue may be a delay queue (triggerQueue), and a worker may adjust a preset delay of the delay queue according to different service scenarios. After receiving the data comparison instruction, the data comparison platform may load a corresponding script file from the database, generate a comparison rule, and simultaneously determine whether the waiting time of the data identifier in the identifier queue expires, and when the waiting time expires, may query the target data to be compared associated with the target data identifier according to the target data identifier currently taken out of the delay queue.

In the present disclosure, the data comparison platform 110 obtains a target data identifier whose waiting time reaches a preset duration from a preset identifier queue, obtains data to be compared corresponding to the target data identifier from stored data to be compared, and uses the data to be compared as the target data to be compared.

In an exemplary embodiment, the data identifier may include a checking module identifier associated with a checking module, the checking module may be configured to perform data comparison on target data to be compared, and performing data comparison on the target data to be compared corresponding to each of the plurality of data providers based on the comparison rule may include:

determining a corresponding target checking module based on the checking module identification in the currently acquired data identification; and calling the target checking module to trigger the target checking module to perform data comparison on the target data to be compared corresponding to the plurality of data providers based on the comparison rule.

In practical application, a plurality of checking modules can be preset, for example, when the number of data to be compared is large, the data to be compared under the same storage path of different data providers can be compared in parallel through the plurality of checking modules, and therefore the data comparison efficiency is greatly improved.

In this embodiment, after the data identifier is obtained from the identifier queue, the target matching module corresponding to the matching module identifier may be determined based on the matching module identifier in the currently obtained data identifier, and then the target matching module may be called to trigger the target matching module to perform data matching on the target to-be-compared data corresponding to each data provider based on the comparison rule. In a specific implementation, the plurality of checking modules can perform data comparison on the target data to be compared by using the same comparison rule.

In the disclosure, the data comparison platform may determine the corresponding target comparison module based on the comparison module identifier in the currently acquired data identifiers, and call the target comparison module to trigger the target comparison module to perform data comparison on the target data to be compared based on the comparison rule, so that a plurality of comparison modules can be reasonably scheduled to perform data comparison, and the speed of data comparison is increased.

In an exemplary embodiment, after the target data to be compared is compared, a corresponding comparison result may be obtained. Specifically, if the comparison result is data match, the target data to be compared may be continuously obtained for comparison, and if the comparison result is data mismatch, the corresponding target data to be compared may be stored, so as to perform any one or more of query, check, and alarm on the target data to be compared having an error in the following. In addition, the recording method of the comparison result may be configured in advance, and as shown in the following table 4, a table is configured for the recording method of the comparison result:

TABLE 4

Column name	Data type	Whether or not it is empty	Note
				id	bigint	Whether or not	Identification corresponding to comparison result
resultCode	varchar	Whether or not	Comparison result coding (coding rule)
				resultMsg	varchar	Whether or not	Comparison results
checkUnitId	bigint	Whether or not	Verification module identification

In practical application, the alarm operation may be performed according to a preset time interval, for example, the alarm may be performed on data with abnormal comparison results every minute.

It should be understood that although the steps in the flowcharts of fig. 2, 3 and 4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 3 and 4 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least some of the other steps.

It is understood that the same/similar parts between the embodiments of the method described above in this specification can be referred to each other, and each embodiment focuses on the differences from the other embodiments, and it is sufficient that the relevant points are referred to the descriptions of the other method embodiments.

Fig. 5 is a block diagram illustrating an information processing apparatus 500 according to an example embodiment. Referring to fig. 5, the apparatus includes adata acquisition unit 501, an alignmentrule generation unit 502, and adata alignment unit 503.

Adata obtaining unit 501, configured to perform determining a plurality of data providers to be subjected to data comparison, and obtain and store to-be-compared data corresponding to each of the plurality of data providers;

a comparisonrule generating unit 502 configured to execute generating a comparison rule corresponding to a configuration file acquired in advance in response to the data comparison instruction;

adata comparison unit 503 configured to perform data comparison on the target data to be compared corresponding to each of the plurality of data providers based on the comparison rule if the waiting time corresponding to the target data to be compared in the data to be compared reaches a preset duration; the target data to be compared is at least part of the stored data to be compared.

In an exemplary embodiment, the apparatus further comprises:

thedata comparing unit 503 includes:

In an exemplary embodiment, thedata obtaining unit 501 includes:

In an exemplary embodiment, the raw data acquisition module includes:

In an exemplary embodiment, the apparatus further comprises:

thedata comparing unit 503 includes:

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 6 is a block diagram illustrating anelectronic device 600 for implementing an information processing method according to an example embodiment. For example, theelectronic device 600 may be a server. Referring to fig. 6,electronic device 600 includes aprocessing component 620 that further includes one or more processors, and memory resources, represented bymemory 622, for storing instructions, such as application programs, that are executable by processingcomponent 620. The application programs stored inmemory 622 may include one or more modules that each correspond to a set of instructions. Further, theprocessing component 620 is configured to execute instructions to perform the above-described methods.

Theelectronic device 600 may further include: thepower component 624 is configured to perform power management for theelectronic device 600, the wired orwireless network interface 626 is configured to connect theelectronic device 600 to a network, and the input/output (I/O)interface 628. Theelectronic device 600 may operate based on an operating system stored in thememory 622, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.

In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as thememory 622 comprising instructions, executable by the processor of theelectronic device 600 to perform the above-described method is also provided. The storage medium may be a computer-readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided, which includes instructions executable by a processor of theelectronic device 600 to perform the above-described method.

Fig. 7 is a block diagram illustrating anelectronic device 700 for implementing an information processing method according to an example embodiment. For example, theelectronic device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a gaming console, a tablet device, a medical device, an exercise device, a personal digital assistant, and so forth.

Referring to fig. 7,electronic device 700 may include one or more of the following components: processingcomponent 702,memory 704,power component 706,multimedia component 708,audio component 710, input/output (I/O)interface 712,sensor component 714, andcommunication component 716.

Theprocessing component 702 generally controls overall operation of theelectronic device 700, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. Theprocessing components 702 may include one ormore processors 720 to execute instructions to perform all or a portion of the steps of the methods described above. Further, theprocessing component 702 may include one or more modules that facilitate interaction between theprocessing component 702 and other components. For example, theprocessing component 702 may include a multimedia module to facilitate interaction between themultimedia component 708 and theprocessing component 702.

Thememory 704 is configured to store various types of data to support operations at theelectronic device 700. Examples of such data include instructions for any application or method operating on theelectronic device 700, contact data, phonebook data, messages, pictures, videos, and so forth. Thememory 704 may be implemented by any type or combination of volatile or non-volatile storage devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, optical disk, or graphene memory.

Thepower supply component 706 provides power to the various components of theelectronic device 700. Thepower components 706 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for theelectronic device 700.

Themultimedia component 708 includes a screen providing an output interface between theelectronic device 700 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, themultimedia component 708 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when theelectronic device 700 is in an operation mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

Theaudio component 710 is configured to output and/or input audio signals. For example, theaudio component 710 includes a Microphone (MIC) configured to receive external audio signals when theelectronic device 700 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in thememory 704 or transmitted via thecommunication component 716. In some embodiments,audio component 710 also includes a speaker for outputting audio signals.

The I/O interface 712 provides an interface between theprocessing component 702 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

Thesensor assembly 714 includes one or more sensors for providing various aspects of status assessment for theelectronic device 700. For example, thesensor assembly 714 may detect an open/closed state of theelectronic device 700, the relative positioning of components, such as a display and keypad of theelectronic device 700, thesensor assembly 714 may also detect a change in the position of theelectronic device 700 or components of theelectronic device 700, the presence or absence of user contact with theelectronic device 700, the orientation or acceleration/deceleration of thedevice 700, and a change in the temperature of theelectronic device 700. Thesensor assembly 714 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. Thesensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, thesensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Thecommunication component 716 is configured to facilitate wired or wireless communication between theelectronic device 700 and other devices. Theelectronic device 700 may access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G, or 5G), or a combination thereof. In an exemplary embodiment, thecommunication component 716 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, thecommunication component 716 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, theelectronic device 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as thememory 704 comprising instructions, executable by theprocessor 720 of theelectronic device 700 to perform the above-described method is also provided. For example, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided that includes instructions executable by theprocessor 720 of theelectronic device 700 to perform the above-described method.

It should be noted that the descriptions of the above-mentioned apparatus, the electronic device, the computer-readable storage medium, the computer program product, and the like according to the method embodiments may also include other embodiments, and specific implementations may refer to the descriptions of the related method embodiments, which are not described in detail herein.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.