Disclosure of Invention
The embodiment of the application provides a data processing method, a data processing device, electronic equipment and a storage medium, and provides a general data processing method which can be used in a dynamic approval process, so that data conversion processing of various data structures in the dynamic approval process is realized.
In one aspect, the present application provides a data processing method, including:
acquiring to-be-processed source data uploaded by a user at a current approval node;
determining the data structure type of the source data and the target data structure type needing to be derived;
performing first conversion processing on the source data to convert the source data into intermediate data with a preset data structure type;
and performing second conversion processing on the intermediate data to convert the intermediate data into target data with a target data structure type needing to be exported.
In an optional embodiment, the determining the data structure type of the source data includes:
performing data analysis processing on the source data to obtain type definition information of the source data;
and determining the data structure type of the source data according to the type definition information of the source data.
In an optional embodiment, the type definition information includes: data attribute information, source information of data, and data content.
In an optional embodiment, the performing a first conversion process on the source data to convert the source data into intermediate data having a preset data structure type includes:
determining a first mapping relation between the data structure type of the source data and the preset data structure type;
and calling a data conversion component corresponding to the first mapping relation, and performing data conversion processing on the source data to obtain the intermediate data.
In an optional embodiment, the performing a second conversion process on the intermediate data to convert the intermediate data into target data having a target data structure type that needs to be derived includes:
determining a second mapping relation between the preset data structure type and the target data structure type;
and calling a data conversion component corresponding to the second mapping relation, and performing data conversion processing on the intermediate data to obtain the target data.
In a second aspect, the present application provides a data processing apparatus comprising:
the acquisition module is used for acquiring to-be-processed source data uploaded by a user at a current approval node;
the determining module is used for determining the data structure type of the source data and the target data structure type needing to be derived;
the conversion module is used for carrying out first conversion processing on the source data so as to convert the source data into intermediate data with a preset data structure type; and performing second conversion processing on the intermediate data to convert the intermediate data into target data with a target data structure type required to be derived.
In an optional embodiment, the determining module is specifically configured to:
performing data analysis processing on the source data to obtain type definition information of the source data; and determining the data structure type of the source data according to the type definition information of the source data.
In an optional embodiment, the conversion module is specifically configured to:
determining a first mapping relation between the data structure type of the source data and the preset data structure type; calling a data conversion component corresponding to the first mapping relation, and performing data conversion processing on the source data to obtain intermediate data; determining a second mapping relation between the preset data structure type and the target data structure type; and calling a data conversion component corresponding to the second mapping relation, and performing data conversion processing on the intermediate data to obtain the target data.
In a third aspect, the present application provides an electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the data processing method of the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, implement the data processing method of the first aspect.
In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the data processing method of the first aspect.
The embodiment of the application provides a data processing method, a data processing device, an electronic device and a storage medium, and the method comprises the steps of acquiring to-be-processed source data uploaded by a user at a current approval node; determining the data structure type of the source data and the target data structure type needing to be derived; performing first conversion processing on the source data to convert the source data into intermediate data with a preset data structure type; and performing second conversion processing on the intermediate data to convert the intermediate data into target data with a target data structure type needing to be exported, so that conversion of data of a non-specific type is realized in a mode of converting source data into the intermediate data firstly and then converting the intermediate data to obtain the target data, and the mode is suitable for various scenes including a dynamic approval process and meets different requirements of different users on data export.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the appended claims.
First, terms of art related to the present application will be described:
structured data refers to data with a standard format, and data in a relational database is typical structured data. The structured data is generally stored in a two-dimensional table mode, and is characterized in that data records are row data, one record represents complete entity information, the data stores fixed field numbers, each field has fixed data types (int, varchar, datatime and the like), and the length of the field is specified at the same time.
The unstructured data refers to data without a standard format, and common unstructured data comprises plain text or multimedia data such as Word files and images.
Semi-structured data refers to the type of data structure between structured data and unstructured data, and the relative format specification, such as HTML, JSON, EXCEL files, and the like. In this case, the semi-structured data may be parsed into structured data by using a certain parsing method.
With the development of technology, office automation becomes possible. Under various scenes, a possible existence form of office automation is realized during the circulation of the process through the examination and approval process on the line.
In the conventional approval process, the approval process is generally fixed, for example, for the source data uploading, when the user needs to upload the source data, the approval node will have a clear data format requirement for the source data, e.g., a certain approval node only supports the uploading of the source data in a certain data format. The reason for this is that the approval node can only support the conversion of data from one data format to another data format and export the converted data.
With the diversification of the approval process, the data format of the data has more diversified choices. Based on business requirements, dynamic approval processes become possible.
In a dynamic approval process, a process node is defined by self, and file resource information to be uploaded by the node, such as forms, Word files, Excel files and the like, is defined at the same time, and in the face of different data file formats, the traditional data export only considers the conversion of a certain data file into another data file in a specific form, and can not support the export of various types of metadata; similarly, in the dynamic approval process, due to different emphasis points of the users, the data export requirements are different, even if the same source data form is adopted, different users may require to export different target data, for example, for report information in the approval process, a common department only needs to export a table in an Excel format, and for a decision department, a chart needs to be exported for decision making.
Different from the traditional approval process, in the dynamic approval process, the data format of the source data uploaded by the user and the data format of the derived target data are not limited, and the data formats of the source data and the target data depend on the actual requirements of the approval nodes.
Under such a requirement, how to reasonably convert the data in different formats becomes a key problem for realizing a dynamic approval process.
Based on this, the embodiments of the present application provide a data processing method, an apparatus, an electronic device, and a storage medium, which implement conversion of unspecific type data in a manner of converting source data into intermediate data and then converting the intermediate data to obtain target data, and the manner is applicable to various scenarios including a dynamic approval process, and meets different requirements of different users for data export.
The method provided by the present application will be described below with reference to different implementations.
Referring to fig. 1, fig. 1 is a schematic diagram of a network architecture on which the present application is based, where the network architecture shown in fig. 1 may specifically include aterminal device 1 and aserver 2.
Theterminal device 1 is a hardware device that can be used for performing an online approval process, and includes, but is not limited to, a desktop computer, a tablet computer, a smart phone, and the like having an arithmetic function.
Theserver 2 may specifically be a cloud service cluster that can be used to store data and process data, and the data processing device provided in the present application may be integrated in theserver 2, and the data processing device interacts with theterminal device 1 through a network, may obtain source data uploaded by a user through theterminal device 1, and provides an approval service and a target data export service for theterminal device 1.
The data processing device provided by the present application will be installed in theserver 2 described above in the form of software, or the data processing device provided by the present application will be integrated in theserver 2 described above in the form of a logical unit.
Example one
Fig. 2 is a schematic flowchart of a data processing method provided in the present application, and as shown in fig. 2, the method includes:
step 201, obtaining to-be-processed source data uploaded by a user at a current approval node.
Step 202, determining the data structure type of the source data and the target data structure type to be derived.
Step 203, performing a first conversion process on the source data to convert the source data into intermediate data having a preset data structure type.
And 204, performing second conversion processing on the intermediate data to convert the intermediate data into target data with a target data structure type needing to be exported.
It should be noted that the data processing method provided in the present application is particularly applicable to a data processing apparatus, and the data processing apparatus may be installed or integrated in theserver 2 in the network architecture as shown in fig. 1.
In the data processing method provided by the embodiment of the application, the data processing device first obtains to-be-processed source data uploaded by a user at a current approval node.
The current approval node may be specifically a node in a preconfigured approval process, or a processing process. When configuring the current approval node, a process developer or a process manager can construct an interface of the approval node and a data uploading interface, so that a user can upload source data to a server where the node is located through the interface provided by the interface of the approval node, and the source data can be captured and processed by a data processing device.
Fig. 3 is a schematic interface diagram of a data processing method provided in the present application, and as shown in fig. 3, a user may input basic information required for approval on an approval interface provided by an approval node, and upload a related file resource at a data interface provided by the approval interface.
It should be noted that the basic information on the approval interface shown in fig. 3 will form page data, where the basic information includes: approval of the applicant, time of approval, items of approval, reviewers, and the like. The file resources on the approval interface will form file data, such as "2020 statement data, doc" in the attachment upload, etc. The page data and the document data described above will constitute the source data described in this application.
After the data processing device obtains the source data, the data processing device determines a data structure type of the source data and a target data structure type that needs to be exported, so as to respectively perform conversion processing on page data and file data in the source data.
Specifically, a parser for parsing the source data may be further provided in the data processing apparatus.
By using the analyzer, the data processing device can perform data analysis processing on the source data to obtain the type definition information of the source data, and then determine the data structure type of the source data according to the type definition information of the source data.
Further, the type definition information includes data attribute information, data source information, and data content. The data attribute information may specifically include basic information of source data such as a data attribute type (file data or page data), a data name suffix, and a data path. The source information of the data includes a name of the data source, an address of the data source, and the like. The data content is used to define the content to be used in the subsequent conversion of the source data, such as a partial column in Excel, a partial paragraph in Word, and the like, and the data content further includes a data item name, a data item type, a data item length, other descriptions, and the like.
By parsing the source data into type definition information, the parser may be caused to further classify the source data based on the type definition information to determine a data structure type to which the source data belongs. According to the structural characteristics (structured data, semi-structured data and unstructured data) of different data structures, the source data can be classified by combining the type definition information of the source data, and the file type, the source and the structural characteristics of the source data are determined.
Table 1 shows the file types, sources, and structural characteristics under different data structure types.
Illustratively, the page data in the aforementioned source data generally belongs to semi-structured data with a document type of HTML and a light data exchange format as a source.
After determining the data structure type of the source data, the data processing device performs a first conversion process on the source data to convert the source data into intermediate data having a preset data structure type. Specifically, a first mapping relation between a data structure type of the source data and the preset data structure type is determined; and then, calling a data conversion component corresponding to the first mapping relation, and performing data conversion processing on the source data to obtain the intermediate data.
Furthermore, a converter is also arranged in the data processing device, and the converter comprises mapping relations among a plurality of data structure types and a data conversion component corresponding to each mapping relation.
For example, the components NPOI and Aspose are two common data transformation components. The NPOI can be used for converting Excel data of semi-structured data into structured data SqlServer or Oracle; and the Aspose can realize the processing of converting Word data of unstructured data into structured data SqlServer or Oracle.
That is to say, in the embodiment of the present application, the data processing apparatus may determine the data structure type of the source data, then establish a mapping between the data structure type and a preset data structure type, and then find the conversion component corresponding to the mapping from the converter, so as to start the data conversion component and perform the conversion processing on the source data.
The source data will be converted into intermediate data existing in a preset data structure type by the processing of the data conversion components including the components NPOI and expose, etc.
It should be noted that the storage form of the intermediate data at least includes the following items: an intermediate data item name, an intermediate data item type, an intermediate data item length, and a description.
The name of the intermediate data item, the type of the intermediate data item, and the length of the intermediate data item correspond to the related information in the type definition information in the aforementioned source data, and the description refers to the specific data content of the source data under the data structure type of the source data.
Through the above processing, the data processing apparatus will obtain intermediate data having a preset data structure type. It should be noted that, although there is a difference between different source data and the data structure type of the source data, the source data will be converted into intermediate data having the same data structure type. Generally, the intermediate data is of a structured data structure type in order to simplify the conversion process.
After the data processing device obtains the intermediate data, the data processing device also converts the intermediate data again to obtain the data under the target data structure type required by the examination and approval.
Specifically, the data processing device determines a second mapping relationship between the preset data structure type and the target data structure type; and calling a data conversion component corresponding to the second mapping relation, and performing data conversion processing on the intermediate data to obtain the target data.
Similar to the foregoing conversion of the source data, when the intermediate data is processed, a mapping relationship between a data structure type of the intermediate data and a target data structure type is determined, so that a corresponding data conversion component is called from the converter according to the mapping relationship to perform conversion processing on the intermediate data.
The data processing method provided by the embodiment of the application realizes the conversion of the data of the unspecific type by converting the source data into the intermediate data and then converting the intermediate data to obtain the target data, and the method is suitable for various scenes including a dynamic approval process and meets different requirements of different users on data derivation. The intermediate data are of a structured data structure type, the data of other data structure types are converted into the intermediate data of the structured data structure type, the intermediate data of the structured data structure type are converted into the target data of other data structure types, the conversion processing processes adopted by the intermediate data and the target data of the structured data structure type are simple and efficient, and the processing speed of the data in the approval process is convenient to improve.
Example two
On the basis of the first embodiment, the second embodiment provides a data processing apparatus, fig. 4 is a schematic structural diagram of the data processing apparatus provided in the present application, and as shown in fig. 4, the data processing apparatus includes: anacquisition module 410, adetermination module 420, and aconversion module 430.
An obtainingmodule 410, configured to obtain to-be-processed source data that is uploaded by a user at a current approval node;
a determiningmodule 420, configured to determine a data structure type of the source data and a target data structure type that needs to be derived;
aconversion module 430, configured to perform a first conversion process on the source data to convert the source data into intermediate data having a preset data structure type; and performing second conversion processing on the intermediate data to convert the intermediate data into target data with a target data structure type required to be derived.
In an optional embodiment, the determiningmodule 420 is specifically configured to:
performing data analysis processing on the source data to obtain type definition information of the source data; and determining the data structure type of the source data according to the type definition information of the source data.
In an optional embodiment, the type definition information includes: data attribute information, source information of data, and data content.
In an optional embodiment, theconversion module 430 is specifically configured to:
determining a first mapping relation between the data structure type of the source data and the preset data structure type; calling a data conversion component corresponding to the first mapping relation, and performing data conversion processing on the source data to obtain intermediate data; determining a second mapping relation between the preset data structure type and the target data structure type; and calling a data conversion component corresponding to the second mapping relation, and performing data conversion processing on the intermediate data to obtain the target data.
The embodiment of the application provides a data processing device, which is used for acquiring source data to be processed, which is uploaded by a user at a current approval node; determining the data structure type of the source data and the target data structure type needing to be derived; performing first conversion processing on the source data to convert the source data into intermediate data with a preset data structure type; and performing second conversion processing on the intermediate data to convert the intermediate data into target data with a target data structure type needing to be exported, so that conversion of data of a non-specific type is realized in a mode of converting source data into the intermediate data firstly and then converting the intermediate data to obtain the target data, and the mode is suitable for various scenes including a dynamic approval process and meets different requirements of different users on data export.
EXAMPLE III
Fig. 5 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, and as shown in fig. 5, an embodiment of the present application further provides an electronic device 1400, which includes:memory 1401,processor 1402, and computer programs.
Wherein the computer program is stored in thememory 1401 and configured to be executed by theprocessor 1402 to implement the data processing method provided by any one of the embodiments of the present application. The related descriptions and effects corresponding to the steps in the drawings can be correspondingly understood, and redundant description is not repeated here.
In this embodiment, thememory 1401 and theprocessor 1402 are connected by a bus.
Example four
The embodiment of the application provides a computer readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the data processing method provided by any one of the embodiments of the application.
In the several embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, a division of modules is merely a logical division, and an actual implementation may have another division, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, systems or modules, and may be in an electrical, mechanical or other form.
Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware form, and can also be realized in a form of hardware and a software functional module.
Program code for implementing the methods of the present application may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable question answering system, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Furthermore, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the data processing method as described before.
Further, while operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.