CN116166645A - A method for automating the data governance process of university business systems - Google Patents

Abstract

The invention discloses a method for automatizing a data management flow of a college business system, which comprises the following steps: s1: collecting metadata of each service system of a college at regular time; s2: automatically cleaning, filtering, converting and layering metadata of each service system; s3: formulating a data standard with unified measurement and scale; s4: configuring a corresponding quality inspection scheme to automatically circulate quality inspection and outputting a quality inspection report; s5: and a built-in mature process engine is adopted to automatically push the problem data to the corresponding responsible person. The method for automatizing the data treatment process of the college business system provided by the invention can quickly find out the quality problem, quickly trace the source and process the quality problem by automatically collecting metadata, constructing data standards, establishing quality rules and the like to form a closed-loop process management of data treatment.

Description

Translated fromChinese

一种用于高校业务系统数据治理流程自动化的方法A method for automating the data governance process of university business systems

技术领域technical field

本发明涉及一种数据流转方法，尤其涉及一种用于高校业务系统数据治理流程自动化的方法。The invention relates to a data transfer method, in particular to a method for automating the data management process of a university business system.

背景技术Background technique

数据治理是将数据作为组织资产进行管理，通过持续提高数据的质量，最终实现数据价值的最大化，数据治理是所有数据应用的基础。Data governance is to manage data as an organizational asset. By continuously improving the quality of data, the value of data can be maximized. Data governance is the basis of all data applications.

随着高校信息化的不断发展，“数字化校园”向“智慧校园”转型的大背景下，传统的业务系统建设及数据仓库模式已经无法满足高校对教育大数据的深层次挖掘需求。原有模式下的高校业务系统通常由各业务部门独立分散建设，缺乏统一的规划和数据标准，导致所积累的大量业务数据存在碎片化、数据打架、不规范、数据质量低下等弊端。With the continuous development of college informatization and the transformation from "digital campus" to "smart campus", the traditional business system construction and data warehouse model can no longer meet the deep mining needs of colleges and universities for educational big data. Under the original model, the university business system is usually built independently and decentralized by various business departments, lacking unified planning and data standards, resulting in fragmentation, data conflicts, irregularities, and low data quality in a large amount of accumulated business data.

发明内容Contents of the invention

本发明要解决的技术问题是提供一种用于高校业务系统数据治理流程自动化的方法，能够快速发现质量问题，快速溯源并处理，形成一个数据治理闭环流程管理。The technical problem to be solved by the present invention is to provide a method for automating the data governance process of the university business system, which can quickly find quality problems, quickly trace the source and deal with them, and form a data governance closed-loop process management.

本发明为解决上述技术问题而采用的技术方案是提供一种用于高校业务系统数据治理流程自动化的方法，包括如下步骤：S1：定时采集高校各业务系统元数据；S2：将各业务系统元数据自动清洗过滤转换并分层；S3：制定具备统一度量和标尺的数据标准；S4：配置相应质检方案自动循环质检，输出质检报告；S5：采用内置成熟流程引擎，自动推送问题数据到对应责任人。The technical solution adopted by the present invention to solve the above-mentioned technical problems is to provide a method for automating the data management process of university business systems, including the following steps: S1: regularly collect the metadata of each business system of the university; S2: collect the metadata of each business system Data is automatically cleaned, filtered, converted, and layered; S3: Formulate data standards with unified metrics and scales; S4: Configure the corresponding quality inspection plan to automatically cycle quality inspections and output quality inspection reports; S5: Use a built-in mature process engine to automatically push problem data to the corresponding responsible person.

进一步地，所述步骤S1自动增量或全量采集元数据，对采集到的元数据进行变更差异比对，展示血缘关系和管理数据字典，并基于元数据血缘构建全链路追溯管控数据。Further, the step S1 automatically collects metadata incrementally or in full, compares the changes and differences of the collected metadata, displays the blood relationship and manages the data dictionary, and constructs full-link traceability management and control data based on the blood relationship of the metadata.

进一步地，所述步骤S2采用可视化的数据集成工具转换数据，通过绑定的规则自动清洗数据，并分层分主题管理数据，形成数据资产全景图。Further, the step S2 converts the data using a visualized data integration tool, automatically cleans the data through bound rules, manages the data hierarchically and sub-subjects, and forms a panorama of data assets.

进一步地，所述步骤S3还包括绑定源头业务系统数据，并自动同步数据，以图形化的方式展示标准数据与元数据之间的共享及映射关系。Further, the step S3 also includes binding source business system data, automatically synchronizing data, and displaying the sharing and mapping relationship between standard data and metadata in a graphical manner.

进一步地，所述步骤S4包括联动元数据、数据标准自动引入质量规则，配置质检方案输出质量评分报告；所述步骤S5通过成熟流程引擎定时推送问题数据，联动元数据血缘关系快速溯源核查问题并生成整改问题数据推送到对应责任人，流程化跟踪整改问题数据实现数据治理的自动化闭环处理。Further, the step S4 includes linking metadata and data standards to automatically introduce quality rules, and configuring a quality inspection plan to output a quality scoring report; the step S5 regularly pushes problem data through a mature process engine, and links the blood relationship of the metadata to quickly trace and verify the problem The rectification problem data is generated and pushed to the corresponding responsible person, and the rectification problem data is tracked in a process to realize the automatic closed-loop processing of data governance.

进一步地，还包括将数据整合成一张表，通知用户自行核查。Further, it also includes integrating the data into a table and notifying users to check by themselves.

本发明对比现有技术有如下的有益效果：本发明提供的用于高校业务系统数据治理流程自动化的方法，通过自动采集元数据、构建数据标准和建立质量规则等一系列流程，能够快速发现质量问题，快速溯源并处理，形成一个数据治理闭环流程管理。Compared with the prior art, the present invention has the following beneficial effects: the method provided by the present invention for the automation of the data governance process of the university business system can quickly discover the quality through a series of processes such as automatic collection of metadata, construction of data standards, and establishment of quality rules. Problems can be quickly traced and processed to form a closed-loop process management of data governance.

附图说明Description of drawings

图1为本发明实施例中用于高校业务系统数据治理流程自动化的方法流程图。FIG. 1 is a flow chart of a method for automating the data governance process of a university business system in an embodiment of the present invention.

具体实施方式Detailed ways

下面结合附图和实施例对本发明作进一步的描述。The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

图1为本发明实施例中用于高校业务系统数据治理流程自动化的方法流程图。FIG. 1 is a flow chart of a method for automating the data governance process of a university business system in an embodiment of the present invention.

请参见图1，本发明实施例的用于高校业务系统数据治理流程自动化的方法，包括如下步骤：Please refer to Fig. 1, the method for the automation of the data governance process of the university business system according to the embodiment of the present invention includes the following steps:

本发明提供的用于高校业务系统数据治理流程自动化的方法，包括元数据、数据标准、数据质量和治理管控，四者相辅相成，形成治理流程闭环管理。The method for automating the data governance process of the university business system provided by the present invention includes metadata, data standards, data quality and governance control, and the four complement each other to form a closed-loop management of the governance process.

1、元数据。自动增量或全量采集元数据，构建血缘关系图，自动检测比对元数据变更情况，从而达到可视化全链路分析能力，上下游数据关系一目了然，绑定质量规则，联动数据集成工具，实现自动化数据清洗。1. Metadata. Automatic incremental or full collection of metadata, construction of kinship relationship diagrams, automatic detection and comparison of metadata changes, so as to achieve visual full-link analysis capabilities, clear upstream and downstream data relationships at a glance, binding quality rules, and linkage data integration tools to achieve automation Data cleaning.

2、数据标准。建立数据标准，自动同步到数仓，并可图形化展示标准与元数据和共享的映射关系。2. Data standards. Establish data standards, automatically synchronize to the data warehouse, and graphically display the mapping relationship between standards, metadata and sharing.

3、数据质量。建立质量规则，可从标准引入快速生成，配置质检方案后自动循环检测并输出报告和质量评分，并将问题数据自动推送到对应负责人。3. Data quality. Establish quality rules, which can be quickly generated from standard introduction. After configuring the quality inspection plan, automatic loop detection and output of reports and quality scores will be automatically pushed to the corresponding responsible person.

4、治理管控。流程化跟踪并整改问题数据，并进行部门绩效评估，全面监督治理工作。4. Governance and control. Track and rectify problem data in a procedural manner, conduct departmental performance evaluation, and comprehensively supervise governance work.

具体实施过程如下：The specific implementation process is as follows:

S1)定时采集高校各业务系统元数据；采集并自动变更元数据，进行变更差异比对，展示血缘关系和管理数据字典，绑定标准代码和质量规则，全链路追溯管控数据，具备全局性和统一性。S1) Regularly collect metadata of various business systems in colleges and universities; collect and automatically change metadata, compare changes, display blood relationship and manage data dictionary, bind standard codes and quality rules, and track and control data throughout the link, with globalization and unity.

S2)将各业务系统元数据自动清洗过滤转换并分层；联合数据集成工具转换数据，通过绑定的规则自动清洗数据，并分层分主题管理数据，形成数据资产全景图，具备可视化。S2) Automatically clean, filter, transform and layer the metadata of each business system; unite data integration tools to convert data, automatically clean data through bound rules, and manage data layer by topic to form a panoramic view of data assets with visualization.

S3)制定数据标准；绑定源头业务系统数据，并自动同步数据，标准一旦变更动态推送负责人审核。自动获取标准代码并可自动同步到数仓，具备统一度量和标尺。S3) Formulate data standards; bind the source business system data, and automatically synchronize the data. Once the standard is changed, it will be dynamically pushed to the person in charge for review. Automatically obtain standard codes and automatically synchronize to the data warehouse, with unified measurement and scale.

S4)配置相应质检方案自动循环质检，输出质检报告；由质检输出报告并推送问题到治理管控，可流程化跟踪并整改，具备自动化及可持续性。S4) Configure the corresponding quality inspection plan to automatically cycle the quality inspection and output the quality inspection report; the quality inspection outputs the report and pushes the problem to the governance control, which can be tracked and rectified in a process-based manner, with automation and sustainability.

S5)内置成熟流程引擎，自动推送问题数据到对应责任人；联动元数据、数据标准自动引入质量规则，配置质检进行质量评分输出报告，具备集中管控和管理的能力；由质检输出报告并推送问题到治理管控，可流程化跟踪并整改，具备自动化及可持续性。配置质检方案定时推送问题数据，通过治理管控核查问题，联动元数据血缘关系快速溯源并进行整改数据，从而实现数据治理流程闭环和自动化。S5) The built-in mature process engine automatically pushes problem data to the corresponding responsible person; linkage metadata and data standards automatically introduce quality rules, configure quality inspection to output quality scoring reports, and have the ability of centralized control and management; output reports by quality inspection and Push problems to governance control, which can be tracked and rectified in a process-based manner, with automation and sustainability. Configure the quality inspection plan to regularly push problem data, and through governance, control and verification of problems, link metadata blood relationship to quickly trace the source and rectify the data, so as to realize the closed-loop and automation of the data governance process.

S6)根据一张表用户自检数据；将数据整合成一张表，用户可自行核查或通过其他方式使用数据，从而提高数据质量，体现数据价值，赋能数据。S6) According to a table, the user self-checks the data; the data is integrated into a table, and the user can check it by himself or use the data in other ways, so as to improve the data quality, reflect the value of the data, and empower the data.

综上所述，本发明实施例的用于高校业务系统数据治理流程自动化的方法，通过自动采集元数据、构建数据标准和建立质量规则等一系列流程，能够快速发现质量问题，快速溯源并处理，形成一个数据治理闭环流程管理。In summary, the method for automating the data governance process of the university business system in the embodiment of the present invention can quickly discover quality problems, quickly trace the source and deal with them through a series of processes such as automatic collection of metadata, construction of data standards, and establishment of quality rules. , forming a data governance closed-loop process management.

虽然本发明已以较佳实施例揭示如上，然其并非用以限定本发明，任何本领域技术人员，在不脱离本发明的精神和范围内，当可作些许的修改和完善，因此本发明的保护范围当以权利要求书所界定的为准。Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the present invention The scope of protection should be defined by the claims.

Claims (6)

1. The method for automatizing the data management flow of the college business system is characterized by comprising the following steps:

s1: collecting metadata of each service system of a college at regular time;

s2: automatically cleaning, filtering, converting and layering metadata of each service system;

s3: formulating a data standard with unified measurement and scale;

s4: configuring a corresponding quality inspection scheme to automatically circulate quality inspection and outputting a quality inspection report;

s5: and a built-in mature process engine is adopted to automatically push the problem data to the corresponding responsible person.

2. The method for data management flow automation of a college business system according to claim 1, wherein the step S1 automatically collects metadata in an increment or full quantity, performs change difference comparison on the collected metadata, displays a blood-margin relation and a management data dictionary, and constructs full-link traceback management data based on the blood-margin of the metadata.

3. The method for data management flow automation of a college business system according to claim 1, wherein the step S2 uses a visualized data integration tool to convert data, automatically cleans the data through binding rules, and hierarchically divides theme management data to form a data asset panorama.

4. The method for data management process automation of a college business system according to claim 1, wherein the step S3 further comprises binding source business system data and automatically synchronizing the data, and graphically displaying the sharing and mapping relationship between standard data and metadata.

5. The method for data governance process automation of a college business system according to claim 1, wherein the step S4 includes automatically introducing quality rules in linkage metadata and data standards, and configuring a quality inspection scheme to output a quality score report; and step S5, the problem data is pushed regularly through a mature flow engine, the linkage metadata blood relationship is traced and checked rapidly, the correction problem data is generated and pushed to corresponding responsible persons, and the automatic closed-loop processing of data treatment is realized by tracing the correction problem data in a flow mode.

6. The method for automated data governance flow in a college business system according to claim 1, further comprising integrating the data into a table informing the user of self-verification.

Images