Disclosure of Invention
In view of the foregoing, it is necessary to provide a data processing method, apparatus, terminal and computer storage medium for solving the problem of low efficiency of data processing in a database in the prior art.
In order to solve the above problems, the present invention provides a data processing method, including:
when a data change request is received, adding the data change request into a preset data change cache set according to the data change type of the data change request;
when a data query request of target data corresponding to the data change request stored in the data change cache set is received, integrating the data change request in the data change cache set according to the data change type to generate a corresponding data batch statement;
and executing corresponding data change operation in the database according to the data batch statement.
In one possible implementation manner, according to a data change type of the data change request, adding the data change request to a preset data change cache set includes:
According to a preset data change type, a data change cache set is established in a preset cache layer;
And storing the data characteristics of the target modification data corresponding to the data change request into a data change cache set according to the data change type of the data change request.
In one possible implementation manner, according to a data modification type of the data modification request, storing the data characteristics of the target modification data corresponding to the data modification request into the data modification cache set, including:
when the data change request is a new data request, storing table names, main key column names and key value pairs of column data of data to be newly built into a newly-added data cache set;
When the data change request is a data deleting request, storing the table name and the main key value of the data to be deleted into a data deleting cache set;
when the data change request is a data modification request, the table name, the primary key value, the column name and the key value pair of the column data of the data to be modified are stored into the modified data cache set.
In one possible implementation manner, integrating the data change requests in the data change cache set according to the data change types to generate corresponding data batch processing sentences, including:
When the data change request is a new data request, generating a data batch new sentence of batch new data according to the table name, the main key column name and the key value pair of the column data of the data to be newly created;
When the data change request is a data deleting request, a data batch deleting statement for batch deleting data is generated according to the table name and the main key value of the data to be deleted.
In one possible implementation manner, integrating the data change requests in the data change cache set according to the data change types to generate corresponding data batch processing sentences, including:
When the data change request is a data modification request, if the data to be modified is the data to be deleted in the data cache set, the data modification request is not executed;
when the modified data is not the data to be deleted in the data cache set, corresponding data modification sentences are respectively generated according to the table names, the main key values, the column names and the key values of the column data of the data to be modified.
In one possible implementation, after performing a corresponding data modification operation in the database according to the data batch statement, the method includes:
The data change cache set is emptied.
In one possible implementation, after performing a corresponding data modification operation in the database according to the data batch statement, the method includes:
and generating a finished change data table according to the table name of the data to be newly built, the table name of the data to be deleted and the table name of the data to be changed.
The invention also provides a data processing device, comprising:
the request cache module is used for adding the data change request into a preset data change cache set according to the data change type of the data change request when the data change request is received;
The request integration module is used for integrating the data change requests in the data change cache set according to the data change types when receiving the data query requests of the target data corresponding to the data change requests stored in the data change cache set, and generating corresponding data batch processing sentences;
And the request processing module is used for executing corresponding data changing operation in the database according to the data batch processing statement.
The invention also provides a database information management terminal comprising a memory and a processor, wherein,
A memory for storing a program;
And a processor coupled to the memory for executing programs stored in the memory to implement the steps of the data processing method of any of the embodiments described above.
The present invention also provides a computer-readable storage medium storing a computer-readable program or instructions which, when executed by a processor, enable the steps in the data processing method of any one of the above embodiments to be carried out.
The data processing method has the advantages that when a data change request is received, the data change request is added into a preset data change cache set according to the data change type of the data change request, when a data query request of target data corresponding to the data change request stored in the data change cache set is received, the data change request in the data change cache set is integrated according to the data change type to generate a corresponding data batch processing statement, and corresponding data change operation is executed in a database according to the data batch processing statement. By carrying out batch processing on the data change request, the data processing efficiency of the database can be greatly improved, and the modern requirements are met.
Detailed Description
The following detailed description of preferred embodiments of the invention is made in connection with the accompanying drawings, which form a part hereof, and together with the description of the embodiments of the invention, are used to explain the principles of the invention and are not intended to limit the scope of the invention.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The invention discloses a data processing method, which comprises the following steps:
S101, when a data change request is received, adding the data change request into a preset data change cache set according to the data change type of the data change request;
S102, when a data query request of target data corresponding to a data change request stored in a data change cache set is received, integrating the data change request in the data change cache set according to a data change type to generate a corresponding data batch statement;
s103, executing corresponding data change operation in the database according to the data batch statement.
In the embodiment of the invention, the data processing method can be applied to computer equipment as a computer program for processing operations such as data query, access, change and the like of a database. Specifically, the database may be an existing database or a newly built database, and the type and volume of the database are not limited by the present invention.
In the embodiment of the invention, the data change request refers to a request for changing data in a database, generally, the access to the database comprises operations such as data change, data query and the like, the data processing method provided by the invention is mainly used for processing the data change request, because the data change request needs to change the data in the database, compared with the data query request, the data processing flow is more complicated and the data processing efficiency is slower, and the data processing method provided by the invention can process the data change request in batches so as to improve the data processing efficiency.
In the embodiment of the invention, when the data change request is received, the data change request is cached into a corresponding data change cache set according to a specific data change type in the data change request, and the data change cache set is established in advance according to the possible data change type and can respectively store different data change requests.
In the embodiment of the invention, when receiving the data query request of the target data corresponding to the data change request stored in the data change cache set, the data change requests of the same type in each data change cache set are integrated to generate corresponding data batch processing sentences, and the data batch processing sentences can process the data in batches in the database so as to improve the data processing efficiency.
The data processing method provided by the invention comprises the steps of adding a data change request into a preset data change cache set according to the data change type of the data change request when the data change request is received, integrating the data change request in the data change cache set according to the data change type to generate a corresponding data batch statement when a data query request of target data corresponding to the data change request stored in the data change cache set is received, and executing corresponding data change operation in a database according to the data batch statement. By carrying out batch processing on the data change request, the data processing efficiency of the database can be greatly improved, and the modern requirements are met.
As one possible embodiment of the present invention, in this embodiment, as shown in fig. 2, according to a data modification type of a data modification request, adding the data modification request to a preset data modification cache set includes:
s201, establishing a data change cache set in a preset cache layer according to a preset data change type;
s202, according to the data change type of the data change request, storing the data characteristics of the target change data corresponding to the data change request into a data change cache set.
In the embodiment of the invention, a data caching layer is arranged for storing the data change caching set. Specifically, according to the possible data change types, the corresponding data change cache sets are set, for example, the data change types include newly added data, deleted data and modified data, and then the newly added data cache sets, the deleted data cache sets and the modified data cache sets can be correspondingly set. When a data change request is received, according to the data change type of the data change request, storing the data characteristics of target modified data corresponding to the data change request into a corresponding data change cache set. The details of which data features of the target modification data need to be stored are described later in this invention.
According to the invention, the data change cache set is arranged in the cache layer, so that the data change request can be cached into the corresponding data change cache set according to different data change types of the data change request, and the data characteristics of the target change data corresponding to the data change request are stored, thereby facilitating the subsequent data batch processing.
As one possible embodiment of the present invention, in this embodiment, as shown in fig. 3, according to a data modification type of a data modification request, storing data features of target modification data corresponding to the data modification request into a data modification cache set, including:
S301, when the data change request is a new data request, storing a table name, a main key column name and a key value pair of column data of the data to be newly created into a newly added data cache set;
S302, when the data change request is a data deleting request, storing a table name and a main key value of data to be deleted into a data deleting cache set;
s303, when the data change request is a data modification request, the table name, the primary key value, the column name and the key value pair of the column data of the data to be modified are stored into the modified data cache set.
In the embodiment of the present invention, when the data change request is a new data request, the table name, the primary key column name and the key value pair of the column data of the data to be newly created are stored in the new data cache set, for example, student information is created for students, teacher information is created for teachers, and the corresponding stored data features are [{"STUDENTINFO","101",row1}, {"STUDENTINFO","102",row2}, {"TEACHERINFO","001",row3},{"TEACHERINFO","002",row4}],, where "STUDENTINFO" and "TEACHERINFO" are list names, "101", "102", "001" and "002" are corresponding column names, and row1, row2, row13 and row4 are corresponding column data, respectively. When the data change request is a data deletion request, the table name and the primary key value of the data to be deleted are stored in a data deletion cache set, for example, student information is deleted, and the corresponding stored data is characterized by [ { "STUDENTINFO", "97" }, { "TEACHERINFO", "98" }, { "TEACHERINFO", "001" }, { "TEACHERINFO", "002" } ], wherein "STUDENTINFO" and "TEACHERINFO" are list names, and 97, 98, 001 and 002 are the primary key values of the corresponding data to be deleted. When the data change request is a data modification request, the table name, the primary key value, the column name and the key value pair of the column data to be modified are stored in the modified data cache set, for example, the student information and the teacher information are modified, the feature data to be stored is ：[{"STUDENTINFO","102",row5}, {"STUDENTINFO","103",row6}, {"TEACHERINFO","003",row7},{"TEACHERINFO","004",row8}],, wherein 'STUDENTINFO' and 'TEACHERINFO' are list names, 102103003004 is a corresponding column name, and row5, row6, row7 and row8 are the key value pair of the corresponding column data respectively.
According to the invention, the data characteristics of the data to be changed are stored in the data change cache set, so that the data can be conveniently processed according to the data characteristics.
As a possible implementation manner of the present invention, in this implementation manner, as shown in fig. 4, the integrating the data change requests in the data change cache set according to the data change types, to generate the corresponding data batch statement includes:
S401, when the data change request is a new data request, generating a data batch new sentence of batch new data according to the list name, the main key column name and the key value pair of the column data of the data to be newly created;
S402, when the data change request is a data deleting request, a data batch deleting statement for batch deleting data is generated according to the table name and the main key value of the data to be deleted.
In the embodiment of the present invention, when the data change request is a new data request, a data batch new sentence of batch new data may be generated according to the data characteristics of the data to be newly created, and as in the previous embodiment, two data batch new sentences may be generated in batch for the set [{"STUDENTINFO","101",row1}, {"STUDENTINFO","102",row2}, {"TEACHERINFO","001",row3},{"TEACHERINFO","002",row4}],:
INSERT INTO STUDENTINFO (ID, NAME, GENDER) VALUE ("101", "Zhangsan", "Man"), ("102", "Liqu", "Man")
INSERT INTO TEACHERINFO (ID, NAME, GENDER) VALUE ("001", "king", men "), (" 002"," Zhao Liu "," men ")
Of course, the embodiment of the invention is only a simple implementation mode which is convenient to describe, in practical application, generally new student information and new teacher information are data processing in large batch, so that dozens or hundreds of data processing batches can be generated into two data batch new sentences, data processing events are greatly saved, and data processing efficiency is improved.
On the other hand, when the data change request is a data deletion request, a data batch deletion statement for batch deleting data is generated according to the table name and the primary key value of the data to be deleted. As in the previous embodiment, two data bulk delete statements are generated for the set [ { "STUDENTINFO", "97" }, { "STUDENTINFO", "98" }, { "TEACHERINFO", "001" }, { "TEACHERINFO", "002" } ]:
DELETE FROM STUDENTINFO WHERE ID IN (”97”,“98”)
DELETE FROM TEACHERINFO WHERE ID IN (”001”,“002”)
Similarly, the batch processing mode for deleting the data requests can also improve the efficiency of deleting the data in the database.
As a possible implementation manner of the present invention, in this implementation manner, as shown in fig. 5, the integrating the data change requests in the data change cache set according to the data change types, to generate the corresponding data batch statement includes:
s501, when the data change request is a data modification request, if the data to be modified is the data to be deleted in the data cache set, the data modification request is not executed;
s502, when the modified data is not the data to be deleted in the data cache set, corresponding data modification sentences are respectively generated according to the table names, the main key values, the column names and the key values of the column data of each data to be modified.
In the embodiment of the invention, when the data change request is a data change request, whether the data to be changed corresponding to the data change request is the data to be deleted needs to be judged first, and if the data to be changed is the data to be deleted, the data to be changed can be directly deleted without processing the data change request. When the data to be modified is not the data to be deleted, corresponding data modification sentences are respectively generated according to the table names, the main key values, the column names and the key values of the column data of the data to be modified. Unlike the new data request and the delete data request in the foregoing embodiment, for the data to be modified, a corresponding data modification statement needs to be generated for each data modification request, for example, four data modification statements are generated for the set [{"STUDENTINFO","102",row5}, {"STUDENTINFO","103",row6}, {"TEACHERINFO","003",row7},{"TEACHERINFO","004",row8}]:
UPDATE STUDENTINFO SET NAME = "wang wu" WHERE id= '102'
UPDATE STUDENTINFO SET GENDER = "female" WHERE id= '103'
UPDATE TEACHERINFO SET NAME = “XXX” WHERE ID=‘003’
UPDATE STUDENTINFO SET GENDER = "female" WHERE id= '004'
The method and the device judge whether the data to be modified corresponding to the data modification request is the data to be deleted or not, directly delete the data to be modified to be deleted, omit data modification operation and improve data processing efficiency.
As a possible embodiment of the present invention, after performing a corresponding data modification operation in a database according to a data batch statement, the method includes:
The data change cache set is emptied.
In the embodiment of the present invention, after the corresponding data modification operation is executed in the database based on the data batch processing statement in the foregoing embodiment, in order to process the subsequent data modification request, the data modification cache set needs to be emptied, and specifically, the newly added data cache set, the deleted data cache set and the modified data cache set need to be all emptied.
Further, after the corresponding data modification operation is executed in the database according to the data batch statement, the method includes:
and generating a finished change data table according to the table name of the data to be newly built, the table name of the data to be deleted and the table name of the data to be changed.
In the embodiment of the invention, in order to enable the data batch processing statement to return a correct result and record the data change request, a completed change data table can be generated according to the table name of the data to be newly added, the table name of the data to be deleted and the table name of the data to be changed, so that the data change condition can be queried based on the completed change data table.
According to the embodiment of the invention, after the data is processed in batches, the data change cache set is emptied, and the completed change data table is constructed, so that the data change condition can be conveniently inquired.
In order to better implement the data processing method in the embodiment of the present invention, correspondingly, as shown in fig. 6, on the basis of the data processing method, an embodiment of the present invention further provides a data processing apparatus, where a data processing apparatus 600 includes:
The request buffer module 601 is configured to add, when receiving a data change request, the data change request to a preset data change buffer set according to a data change type of the data change request;
The request integrating module 602 is configured to integrate, when receiving a data query request of target data corresponding to a data change request stored in the data change cache set, the data change request in the data change cache set according to a data change type, and generate a corresponding data batch statement;
The request processing module 603 is configured to perform a corresponding data modification operation in the database according to the data batch statement.
The data processing apparatus 600 provided in the foregoing embodiments may implement the technical solutions described in the foregoing data processing method embodiments, and the specific implementation principles of the foregoing modules or units may be referred to the corresponding content in the foregoing data processing method embodiments, which is not described herein again.
The data processing device provided by the invention adds the data change request into a preset data change cache set according to the data change type of the data change request when the data change request is received, integrates the data change request in the data change cache set according to the data change type to generate a corresponding data batch statement when the data query request of target data corresponding to the data change request stored in the data change cache set is received, and executes corresponding data change operation in a database according to the data batch statement. By carrying out batch processing on the data change request, the data processing efficiency of the database can be greatly improved, and the modern requirements are met.
As shown in fig. 7, the present invention also provides a database information management terminal 700 accordingly. The database information management terminal 700 includes a processor 701, a memory 702, and a display 703. Fig. 7 shows only some of the components of the database information management terminal 700, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead.
The processor 701 may in some embodiments be a central processing unit (Central Processing Unit, CPU), microprocessor or other data processing chip for executing program code or processing data stored in the memory 702, such as the data processing method of the present invention.
In some embodiments, the processor 701 may be a single server or a group of servers. The server farm may be centralized or distributed. In some embodiments, the processor 701 may be local or remote. In some embodiments, the processor 701 may be implemented in a cloud platform. In some embodiments, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multiple cloud, or the like, or any combination thereof.
The memory 702 may be an internal storage unit of the database information management terminal 700 in some embodiments, such as a hard disk or a memory of the database information management terminal 700. The memory 702 may also be an external storage device of the database information management terminal 700 in other embodiments, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the database information management terminal 700.
Further, the memory 702 may also include both an internal storage unit and an external storage device of the database information management terminal 700. The memory 702 is used for storing application software and various types of data of the installation database information management terminal 700.
The display 703 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like in some embodiments. The display 703 is used for displaying information of the database information management terminal 700 and for displaying a visualized user interface. The components 701-703 of the database information management terminal 700 communicate with each other via a system bus.
In some embodiments, when the processor 701 executes a data processing program in the memory 702, the following steps may be implemented:
when a data change request is received, adding the data change request into a preset data change cache set according to the data change type of the data change request;
when a data query request of target data corresponding to the data change request stored in the data change cache set is received, integrating the data change request in the data change cache set according to the data change type to generate a corresponding data batch statement;
and executing corresponding data change operation in the database according to the data batch statement.
It will be appreciated that the processor 701, when executing a data processing program in the memory 702, may perform other functions in addition to the above, as may be seen in particular from the description of the corresponding method embodiments above.
Further, the type of the database information management terminal 700 is not particularly limited in the embodiment of the present invention, and the database information management terminal 700 may be a portable database information management terminal such as a mobile phone, a tablet computer, a personal digital assistant (personaldigital assistant, PDA), a wearable device, a laptop computer (laptop), and the like. Exemplary embodiments of portable database information management terminals include, but are not limited to, portable database information management terminals that carry IOS, android, microsoft or other operating systems. The portable database information management terminal described above may also be other portable database information management terminals, such as a laptop computer (laptop) or the like having a touch-sensitive surface (e.g., a touch panel). It should also be appreciated that in other embodiments of the present invention, the database information management terminal 700 may be a desktop computer having a touch-sensitive surface (e.g., a touch panel) instead of a portable database information management terminal.
Correspondingly, the embodiment of the invention also provides a computer readable storage medium, which is used for storing a computer readable program or instructions, and when the program or instructions are executed by a processor, the steps or functions in the data processing method provided by each method embodiment can be realized.
Those skilled in the art will appreciate that all or part of the processes implementing the methods of the above embodiments may be implemented by a computer program to instruct related hardware, and the program may be stored in a computer readable storage medium. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention.