Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Referring to fig. 1, fig. 1 is a schematic application scenario diagram of an automatic encoding method of check logic according to an embodiment of the present invention, where the application scenario includes:
(1) And the server is used for providing a back-end service for data transmission. The server is a computer device, can be a single server or a server cluster, can be a cloud server or a special webpage server, receives the access of an external terminal, and is connected with the terminal through a wired network or a wireless network.
(2) The terminal shown in fig. 1 comprises a terminal 1, a terminal 2 and a terminal 3, the automatic coding method of the verification logic provided by the invention is applied to any terminal and can be realized through a software program configured on the terminal, specifically, the terminal obtains the data table verification logic and a preset normalized SQL statement from a server by accessing the server, and the obtained normalized SQL statement is injected into the converted data table verification logic on the terminal to generate an SQL code. The terminal can be electronic equipment such as a smart phone, a smart watch, a notebook computer, a tablet computer or a desktop computer, and the terminal accesses the server through a wired network or a wireless network.
Referring to fig. 2, fig. 2 is a schematic flow chart of an automatic encoding method of check logic according to an embodiment of the present invention. As shown in fig. 2, the method includes the following steps S101 to S104.
S101, acquiring data table verification logic aiming at a target data table.
In the embodiment of the invention, in order to acquire the data table checking logic in the target data table, the target data table needs to be determined first, wherein the target data table can be all data tables in the target database or part of data tables in the target database; if the target data table is a partial data table in the target database, the partial data table is determined by the user through selection.
It should be noted that, in order to store the data table check logic, a preset data table check logic library needs to be established in advance, where the data table check logic is stored in the preset data table check logic library in advance, where the preset data table check logic library is composed of data table check logic of one or more data tables, and each data table check logic is associated with a table name of a corresponding data table and forms an association relationship.
Further, as shown in fig. 3, the step S101 includes the following steps S201 to S202:
S201, obtaining the table name of the target data table.
In the embodiment of the present invention, after the user determines the selected target data table, the table name of the target data table is acquired, for example, in the selection sentence "SELECT LASTNAME from Persons", the table name of the acquired data table is "Persons".
S202, according to the table name of the target data table and the association relation formed by the data table verification logic and the table name of the data table in the preset data table verification logic library, obtaining the data table verification logic corresponding to the target data table from the preset data table verification logic library.
In the embodiment of the invention, the table name of the target data table is determined, the table name of the target data table is obtained, and the data table verification logic corresponding to the target data table is obtained from the preset data table verification logic library by utilizing the obtained table name and the association relationship formed by the data table verification logic and the table name of the data table in the preset data table verification logic library. In the preset data table check logic library, each data table may correspond to one or more data table check logic, for example: for the target data table A, the following three data table checking logics exist in a preset data table checking logic library: 1. whether the field a1 in the target data table A is a data table check logic of a digital type; 2. data table checking logic for checking whether the null rate of the field a1 in the target data table A is less than 10%; 3. whether the field a2 in the target data table A is data table checking logic of date type; when the table name of the target data table is determined to be a, the following three data table checking logics can be obtained from the preset data table checking logic library, namely: 1. whether the field a1 in the target data table A is a data table check logic of a digital type; 2. data table checking logic for checking whether the null rate of the field a1 in the target data table A is less than 10%; 3. whether field a2 in target data table a is date type data table check logic.
S102, determining the verification level of the data table verification logic, and classifying the data table verification logic according to the verification level.
In the embodiment of the present invention, the check level of the data table check logic refers to a level classified according to a table structure (table name, field type, primary key, null rate, etc.) of the target data table, for example: the different table structures have the following several check levels, namely a table name check level and a field check level. The determining the check level of the data table check logic is specifically determined by analyzing and judging whether the data table check logic is specific to the table name or the logic of the field, wherein, for the data table check logic of the table name check level, for example, there are: whether the data of the data table is repeated (for example, find the abnormal data of the repeated ID card number in the personnel information table), for the data table check logic of the field level, for example, there are: whether the duty ratio of the check field is excessive (for example, the field value is the number of empty lines/the total number of lines of the table) >0.2 is a number type/date type (for example, whether the content of a certain numerical field only contains 0-9, decimal point and negative sign) which judges whether the duty ratio of a certain field exceeds 20 percent), and whether the check field type is defined.
After determining the check level of the data table check logic, the data table check logic needs to be classified according to the check level, and in this embodiment, classifying the data table check logic according to the check level specifically includes: determining the class of the data table check logic through the check level, and classifying the data table check logic into a table name check class if the check level is a table name check level; and if the check level is a field check level, classifying the data table check logic into a field check class. For example: 3 data table check logics are respectively 1, and whether a field a1 in the target data table A is of a digital type or not; 2. data table checking logic for checking whether the null rate of the field a1 in the target data table A is less than 10%; 3. whether the field a2 in the target data table A is data table check logic of date type or not, and all the above 3 data table check logics belong to the field check class.
S103, logically describing the classified data table verification logic by using a preset format so as to convert the classified data table verification logic into a format structure which can be identified by a program.
In the embodiment of the present invention, the operation form of the preset format is: category of data table check logic + target data table name + field + logic condition. The step of logically describing the classified data table verification logic by using a preset format specifically includes: determining the running form of the preset format, and converting the existence form of the text content in the classified data table checking logic into the running form of the preset format. Namely, converting the text content in the classified data table verification logic into the text content in the preset format, for example: for 3 data table check logic belonging to the field check class: 1. whether the field a1 in the target data table A is a data table check logic of a digital type; 2. data table checking logic for checking whether the null rate of the field a1 in the target data table A is less than 10%; 3. whether the field a2 in the target data table A is data table checking logic of date type; the logic description can be performed by using a preset format of 'category of data table check logic + target data table name + field + logic condition', and the following steps can be obtained respectively: a. whether the field check class + the target data table a + the field a1+ is a digital type; b. whether the field check class + target data table a + field a1+ null rate is less than 10%; c. field check class + target data table a + whether field a2+ is a date type.
S104, acquiring a preset normalized SQL statement, and injecting the converted data table verification logic to generate an SQL code.
In the embodiment of the present invention, the preset normalized SQL (Structured Query Language ) statement may be a select, from, where, group by four-part SQL statement. The injection of the converted data table check logic to generate SQL code, for example: a. whether the field check class + the target data table a + the field a1+ is a digital type; b. whether the field check class + target data table a + field a1+ null rate is less than 10%; c. whether the field check class + the target data table a + the field a2+ is a date type; the following SQL codes can be obtained by injecting according to a preset normalized SQL statement: whether "select" is a digital type "from target data table a where field a1", "select" null rate is less than 10% "from target data table a where field a1", "select" is a date type "from target data table a where field a2".
From the above, the embodiment of the present invention obtains the data table verification logic for the target data table; determining the verification level of the data table verification logic, and classifying the data table verification logic according to the verification level; logically describing the classified data table verification logic by using a preset format so as to convert the classified data table verification logic into a format structure which can be identified by a program; and acquiring a preset normalized SQL statement, and injecting the converted data table verification logic to generate an SQL code. The invention provides an automatic coding method of check logic based on data processing, which realizes the check of mass data by automatically writing the check logic, reduces the manual coding cost and realizes the automatic updating and maintenance of automatically generated codes.
Referring to fig. 4, fig. 4 is a schematic flowchart of a method for automatically encoding check logic according to another embodiment of the present invention, as shown in fig. 4, the method includes steps S301 to S308 as follows:
S301, data table verification logic for a target data table is obtained.
In the embodiment of the invention, in order to acquire the data table checking logic in the target data table, the target data table needs to be determined first, wherein the target data table can be all data tables in the target database or part of data tables in the target database; if the target data table is a partial data table in the target database, the partial data table is determined by the user through selection.
It should be noted that, in order to store the data table check logic, a preset data table check logic library needs to be established in advance, where the data table check logic is stored in the preset data table check logic library in advance, where the preset data table check logic library is composed of data table check logic of one or more data tables, and each data table check logic is associated with a table name of a corresponding data table and forms an association relationship.
S302, determining the check level of the data table check logic, and classifying the data table check logic according to the check level.
In the embodiment of the present invention, the check level of the data table check logic refers to a level classified according to a table structure (table name, field type, primary key, null rate, etc.) of the target data table, for example: the different table structures have the following several check levels, namely a table name check level and a field check level. The determining the check level of the data table check logic is specifically determined by analyzing and judging whether the data table check logic is specific to the table name or the logic of the field, wherein, for the data table check logic of the table name check level, for example, there are: whether the data of the data table is repeated (for example, find the abnormal data of the repeated ID card number in the personnel information table), for the data table check logic of the field level, for example, there are: whether the duty ratio of the check field is excessive (for example, the field value is the number of empty lines/the total number of lines of the table) >0.2 is a number type/date type (for example, whether the content of a certain numerical field only contains 0-9, decimal point and negative sign) which judges whether the duty ratio of a certain field exceeds 20 percent), and whether the check field type is defined.
After determining the check level of the data table check logic, the data table check logic needs to be classified according to the check level, and in this embodiment, classifying the data table check logic according to the check level specifically includes: determining the class of the data table check logic through the check level, and classifying the data table check logic into a table name check class if the check level is a table name check level; and if the check level is a field check level, classifying the data table check logic into a field check class. For example: 3 data table check logics are respectively 1, and whether a field a1 in the target data table A is of a digital type or not; 2. data table checking logic for checking whether the null rate of the field a1 in the target data table A is less than 10%; 3. whether the field a2 in the target data table A is data table check logic of date type or not, and all the above 3 data table check logics belong to the field check class.
S303, logically describing the classified data table verification logic by using a preset format so as to convert the classified data table verification logic into a format structure which can be identified by a program.
In the embodiment of the present invention, the operation form of the preset format is: category of data table check logic + target data table name + field + logic condition. The step of logically describing the classified data table verification logic by using a preset format specifically includes: determining the running form of the preset format, and converting the existence form of the text content in the classified data table checking logic into the running form of the preset format. Namely, converting the text content in the classified data table verification logic into the text content in the preset format, for example: for 3 data table check logic belonging to the field check class: 1. whether the field a1 in the target data table A is a data table check logic of a digital type; 2. data table checking logic for checking whether the null rate of the field a1 in the target data table A is less than 10%; 3. whether the field a2 in the target data table A is data table checking logic of date type; the logic description can be performed by using a preset format of 'category of data table check logic + target data table name + field + logic condition', and the following steps can be obtained respectively: a. whether the field check class + the target data table a + the field a1+ is a digital type; b. whether the field check class + target data table a + field a1+ null rate is less than 10%; c. field check class + target data table a + whether field a2+ is a date type.
S304, acquiring a preset normalized SQL statement, and injecting the converted data table verification logic to generate an SQL code.
In the embodiment of the present invention, the preset normalized SQL (Structured Query Language ) statement may be a select, from, where, group by four-part SQL statement. The injection of the converted data table check logic to generate SQL code, for example: a. whether the field check class + the target data table a + the field a1+ is a digital type; b. whether the field check class + target data table a + field a1+ null rate is less than 10%; c. whether the field check class + the target data table a + the field a2+ is a date type; the following SQL codes can be obtained by injecting according to a preset normalized SQL statement: whether "select" is a digital type "from target data table a where field a1", "select" null rate is less than 10% "from target data table a where field a1", "select" is a date type "from target data table a where field a2".
S305, if a verification request is received, obtaining data to be verified in a target data table according to the verification request.
In the embodiment of the invention, the execution subject for verification may receive, in a wired connection manner or in a wireless connection manner, a verification request sent by a target user, where the target user may be a user that uses communication connection with the execution subject, and the verification request may be data to be verified in a target data table for verification.
Note that the verification request may be information transmitted in various protocols, such as HTTP (HyperText Transfer Protocol ), FTP (FILE TRANSFER Protocol, file transfer Protocol), and the like.
S306, determining data table verification logic of the data to be verified.
In the embodiment of the present invention, according to the acquired data to be checked, the data table checking logic for the target data table is searched and determined from the preset data table checking logic library, and the specific implementation process may be: and searching and acquiring the data table verification logic corresponding to the target data table from the preset data table verification logic library by utilizing the table name of the target data table and the association relation formed by the data table verification logic and the table name of the data table in the preset data table verification logic library.
S307, acquiring SQL codes corresponding to the data table check logic.
In the embodiment of the present invention, in step S305, a preset normalized SQL statement is used to perform SQL encoding on the data table verification logic and generate an SQL code, so that the corresponding SQL code can be obtained through the data table verification logic, specifically: and determining the type of the data table verification logic and the corresponding target data table name according to the acquired data table verification logic, and searching the corresponding target SQL code from the generated SQL code according to the type of the data table verification logic and the corresponding target data table name.
And S308, executing the SQL codes to verify the data to be verified in the target data table.
In the embodiment of the invention, the SQL codes obtained according to the configuration of the structured query language are adopted to verify the data to be verified, so that batch verification of the records existing through the data table can be realized, and in the batch verification process, if the verification rule changes, the rule is only required to be adjusted and added in a mode of modifying the SQL codes, so that the application does not need to be deployed again, and the verification flexibility is improved. In the verification process, the verification of the character type, null value and length may be accomplished using, for example, the following SQL code: whether "select" is a digital type "from target data table a where field a1", "select" null rate is less than 10% "from target data table a where field a1", "select" is a date type "from target data table a where field a2".
Therefore, according to the embodiment of the invention, the data to be verified provided by the user is verified through the SQL codes obtained through configuration of the structured query language, so that batch verification of the records existing through the data table can be realized, in addition, if the SQL codes obtained through configuration change in the verification process, the rules are required to be adjusted and added in a mode of modifying the SQL codes, therefore, the application is not required to be deployed again, and the verification flexibility is improved.
Referring to fig. 5, in response to the above-mentioned method for automatically encoding check logic, an embodiment of the present invention further provides an apparatus for automatically encoding check logic, where the apparatus 100 includes: a first acquisition unit 101, a classification unit 102, a logic description unit 103, and an encoding unit 104.
Wherein, the first obtaining unit 101 is configured to obtain data table verification logic for a target data table;
The classifying unit 102 is configured to determine a check level of the data table check logic, and classify the data table check logic according to the check level;
A logic description unit 103, configured to logically describe the classified data table verification logic by using a preset format, so as to convert the classified data table verification logic into a format structure that can be identified by a program;
the encoding unit 104 is configured to obtain a preset normalized SQL statement, and inject the converted data table verification logic to generate an SQL code.
From the above, the embodiment of the present invention obtains the data table verification logic for the target data table; determining the verification level of the data table verification logic, and classifying the data table verification logic according to the verification level; logically describing the classified data table verification logic by using a preset format so as to convert the classified data table verification logic into a format structure which can be identified by a program; and acquiring a preset normalized SQL statement, and injecting the converted data table verification logic to generate an SQL code. The invention provides an automatic coding method of check logic based on data processing, which realizes the check of mass data by automatically writing the check logic, reduces the manual coding cost and realizes the automatic updating and maintenance of automatically generated codes.
Referring to fig. 6, the first obtaining unit 101 includes:
a first obtaining subunit 101a, configured to obtain a table name of the target data table;
The second obtaining subunit 101b is configured to obtain, from the preset data table check logic library, a data table check logic corresponding to the target data table according to the table name of the target data table and an association relationship formed by the data table check logic and the table name of the data table in the preset data table check logic library.
Referring to fig. 7, the logic description unit 103 includes:
And the conversion unit 103a is configured to determine an operation form of the preset format, and convert the existence form of the text content in the classified data table verification logic into the operation form of the preset format.
Referring to fig. 8, in response to the above-mentioned method for automatically encoding check logic, an embodiment of the present invention further provides an apparatus for automatically encoding check logic, where the apparatus 200 includes: a first acquisition unit 201, a classification unit 202, a logic description unit 203, an encoding unit 204, a third acquisition unit 205, a determination unit 206, a fourth acquisition unit 207, and an execution unit 208.
Wherein, the first obtaining unit 201 is configured to obtain data table verification logic for a target data table;
a classification unit 202, configured to determine a check level of the data table check logic, and classify the data table check logic according to the check level;
A logic description unit 203, configured to logically describe the classified data table verification logic by using a preset format, so as to convert the classified data table verification logic into a format structure identifiable by a program;
the encoding unit 204 is configured to obtain a preset normalized SQL statement, and inject the converted data table verification logic to generate an SQL code;
A third obtaining unit 205, configured to obtain, if a verification request is received, data to be verified in a target data table according to the verification request;
a determining unit 206, configured to determine data table verification logic of the data to be verified;
A fourth obtaining unit 207, configured to obtain an SQL code corresponding to the data table verification logic;
And the execution unit 208 is used for executing the SQL code to verify the data to be verified in the target data table.
The automatic coding device of the check logic corresponds to the automatic coding method of the check logic one by one, and the specific principle and process are the same as those of the method described in the above embodiment, and are not repeated.
The automatic encoding means of the above-described verification logic may be implemented in the form of a computer program which is executable on a computer device as shown in fig. 9.
Fig. 9 is a schematic structural diagram of a computer device according to the present application. The device may be a terminal or a server, where the terminal may be a smart phone, a tablet computer, a notebook computer, or a desktop computer. The server may be an independent server or a server cluster formed by a plurality of servers. With reference to FIG. 9, the computer device 500 includes a processor 502, a non-volatile storage medium 503, internal memory 504, and a network interface 505, connected by a system bus 501. The non-volatile storage medium 503 of the computer device 500 may store an operating system 5031 and a computer program 5032, where the computer program 5032, when executed, may cause the processor 502 to perform a method of automatically encoding verification logic. The processor 502 of the computer device 500 is used to provide computing and control capabilities, supporting the operation of the entire computer device 500. The internal memory 504 provides an environment for the execution of a computer program 5032 in the non-volatile storage medium 503, which when executed by the processor, causes the processor 502 to perform a method of automatically encoding check logic. The network interface 505 of the computer device 500 is for network communication. It will be appreciated by persons skilled in the art that the architecture shown in fig. 9 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting as to the computer device to which the present inventive arrangements are applicable, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
Wherein the processor 502, when executing the computer program, performs the following operations:
acquiring data table verification logic aiming at a target data table;
determining the verification level of the data table verification logic, and classifying the data table verification logic according to the verification level;
logically describing the classified data table verification logic by using a preset format so as to convert the classified data table verification logic into a format structure which can be identified by a program;
And acquiring a preset normalized SQL statement, and injecting the converted data table verification logic to generate an SQL code.
In one embodiment, the obtaining data table check logic for the target data table includes:
acquiring the table name of the target data table;
And acquiring the data table verification logic corresponding to the target data table from the preset data table verification logic library according to the table name of the target data table and the association relation formed by the data table verification logic and the table name of the data table in the preset data table verification logic library.
In one embodiment, the logically describing the classified data table verification logic using a preset format includes:
Determining the running form of the preset format, and converting the existence form of the text content in the classified data table checking logic into the running form of the preset format.
In one embodiment, the preset normalized SQL statement includes: select, from, where and group by four-part SQL statements.
In one embodiment, the processor 502 further performs the following operations when executing the computer program:
If a verification request is received, acquiring data to be verified in a target data table according to the verification request;
determining data table verification logic of the data to be verified;
Acquiring SQL codes corresponding to the data table verification logic;
and executing the SQL code to verify the data to be verified in the target data table.
Those skilled in the art will appreciate that the embodiment of the computer device shown in fig. 9 is not limiting of the specific construction of the computer device, and in other embodiments, the computer device may include more or less components than those shown, or certain components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device includes only a memory and a processor, and in such embodiments, the structure and function of the memory and the processor are consistent with the embodiment shown in fig. 9, and will not be described again.
The present invention provides a computer readable storage medium storing one or more computer programs executable by one or more processors to implement the steps of:
acquiring data table verification logic aiming at a target data table;
determining the verification level of the data table verification logic, and classifying the data table verification logic according to the verification level;
logically describing the classified data table verification logic by using a preset format so as to convert the classified data table verification logic into a format structure which can be identified by a program;
And acquiring a preset normalized SQL statement, and injecting the converted data table verification logic to generate an SQL code.
In one embodiment, the obtaining data table check logic for the target data table includes:
acquiring the table name of the target data table;
And acquiring the data table verification logic corresponding to the target data table from the preset data table verification logic library according to the table name of the target data table and the association relation formed by the data table verification logic and the table name of the data table in the preset data table verification logic library.
In one embodiment, the logically describing the classified data table verification logic using a preset format includes:
Determining the running form of the preset format, and converting the existence form of the text content in the classified data table checking logic into the running form of the preset format.
In one embodiment, the preset normalized SQL statement includes: select, from, where and group by four-part SQL statements.
In one embodiment, the one or more computer programs may be executed by one or more processors, further implementing the steps of:
If a verification request is received, acquiring data to be verified in a target data table according to the verification request;
determining data table verification logic of the data to be verified;
Acquiring SQL codes corresponding to the data table verification logic;
and executing the SQL code to verify the data to be verified in the target data table.
The aforementioned storage medium of the present invention includes: various media capable of storing program codes, such as magnetic disk, optical disk, read-Only Memory (ROM), etc.
The units in all embodiments of the invention may be implemented by general-purpose integrated circuits, such as a CPU (Central Processing Unit ), or by ASIC (Application SPECIFIC INTEGRATED Circuit, application specific integrated Circuit).
The steps in the automatic coding method of the check logic in the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs.
The units in the automatic coding device of the check logic can be combined, divided and deleted according to actual needs.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.