技术领域technical field
本发明电力运营监测领域，具体涉及一种电力运营监控数据质量评估系统及方法。The invention relates to the field of electric power operation monitoring, and specifically relates to a system and method for evaluating the quality of electric power operation monitoring data.
背景技术Background technique
作为一种战略资源与生产要素的信息资源正在逐渐成为企业正常运转的基础,缺失、错误等异常数据必定产生错误或不准确的处理结果,导致错误或不准确的决策，对企业可能导致严重的甚至是致命的后果。因此，数据质量对于企业来说，显得至关重要。As a strategic resource and a factor of production, information resources are gradually becoming the basis for the normal operation of enterprises. Abnormal data such as missing and wrong data will inevitably produce wrong or inaccurate processing results, leading to wrong or inaccurate decisions, which may cause serious damage to the enterprise. even fatal consequences. Therefore, data quality is very important for enterprises.
电力运营监控数据具有体量大、类型多、速度快等特点，存在数据统计口径不一致、数据质量参差不齐、业务功能重复、数据质量管控滞后等问题。然而，针对数据质量问题，电力运营监控数据质量目前还没有形成统一的评估标准，缺乏数据质量核查和评价方法，严重影响数据资产化进程和数据深层次应用。Power operation monitoring data has the characteristics of large volume, many types, and fast speed. There are problems such as inconsistent data statistics, uneven data quality, duplication of business functions, and lagging data quality control. However, in view of the data quality problem, there is no unified evaluation standard for the quality of power operation monitoring data, and the lack of data quality verification and evaluation methods seriously affects the process of data capitalization and deep application of data.
因此，亟需制定一种电力运营监控数据质量评估方法，用于完善和固化数据质量评估的通用准则。Therefore, it is urgent to develop a method for evaluating the quality of power operation monitoring data to improve and solidify the general criteria for data quality evaluation.
发明内容Contents of the invention
有鉴于此，本发明提供的一种电力运营监控数据质量评估系统及方法，该系统及方法避免了现有数据质量评估过程中一旦出错、需重新执行的问题，大大提高了整体数据质量评估效率；能够进行直观的量化评估和结果反馈，从而提供了系统全面的数据质量评估标准，保障数据的准确、及时、有效和可信，促进了数据质量的提升，进而提高了电力系统的运行稳定性。In view of this, the present invention provides a power operation monitoring data quality evaluation system and method, which avoids the problem of re-execution once an error occurs in the existing data quality evaluation process, and greatly improves the overall data quality evaluation efficiency ;It can carry out intuitive quantitative evaluation and result feedback, thus providing a systematic and comprehensive data quality evaluation standard, ensuring the accuracy, timeliness, effectiveness and credibility of the data, promoting the improvement of data quality, and thus improving the operation stability of the power system .
本发明的目的是通过以下技术方案实现的：The purpose of the present invention is achieved through the following technical solutions:
一种电力运营监控数据质量评估系统，所述系统包括均独立执行脚本代码的数据集抽取模块、异常甄别模块和数据质量评估模块；A power operation monitoring data quality evaluation system, the system includes a data set extraction module, an abnormality screening module, and a data quality evaluation module that independently execute script codes;
所述数据集抽取模块划分评估任务并进行数据集抽取；The data set extraction module divides evaluation tasks and performs data set extraction;
所述异常甄别模块筛选数据质量问题，并进行数据质量评价和量化计算；The abnormal screening module screens data quality problems, and performs data quality evaluation and quantitative calculation;
所述数据质量评估模块进行数据质量结果的可视化操作，输出数据质量报告。The data quality assessment module performs a visualization operation of data quality results and outputs a data quality report.
优选的，所述数据集抽取模块从电力运营监控数据中心抽取所需的数据集，其中，所述数据集由任意组合的单个数据表或子表组成；所述数据集抽取模块包括数据质量评估任务单元、选择评估指标与规则单元及抽取目标数据集单元；Preferably, the data set extraction module extracts the required data set from the power operation monitoring data center, wherein the data set is composed of a single data table or sub-table in any combination; the data set extraction module includes data quality assessment Task units, selection of evaluation indicators and rule units, and extraction of target data set units;
所述数据质量评估任务单元用于发布数据质量评估任务，所述数据质量评估任务包括若干子任务，每个所述子任务均为数据质量评估对象与评估指标的组合；The data quality assessment task unit is used to issue a data quality assessment task, and the data quality assessment task includes several subtasks, each of which is a combination of a data quality assessment object and an assessment index;
所述选择评估指标与规则单元根据用户需求，将相应的数据质量评估指标及约束规则添加到所述数据质量评估中；The selection evaluation index and rule unit adds the corresponding data quality evaluation index and constraint rules to the data quality evaluation according to user requirements;
所述抽取目标数据集单元基于所述数据质量评估指标和约束规则，获取需实施数据质量评估的目标数据集，形成数据质量问题的甄别任务。The unit for extracting a target data set obtains a target data set that needs to be evaluated for data quality based on the data quality evaluation index and constraint rules, and forms a task of identifying data quality problems.
优选的，所述异常甄别模块针对所述目标数据集，利用数据质量评估指标和约束规则甄别出异常电力数据，处理检查出的数据质量问题并确定评价方法，存储计算结果；所述异常甄别模块包括检查约束规则单元、异常数据定位与处理单元、确定评价方法单元及获取方法参数单元；Preferably, the abnormality screening module uses data quality assessment indicators and constraint rules to identify abnormal power data for the target data set, processes the detected data quality problems, determines an evaluation method, and stores calculation results; the abnormality screening module Including inspection constraint rule unit, abnormal data location and processing unit, determination evaluation method unit and acquisition method parameter unit;
所述检查约束规则单元根据约束规则库，对所述目标数据集启动异常数据甄别任务，扫描数据对象并定位数据质量问题，将检查出的数据质量问题信息返回给异常数据处理模块；The check constraint rule unit starts the abnormal data screening task for the target data set according to the constraint rule library, scans the data object and locates the data quality problem, and returns the detected data quality problem information to the abnormal data processing module;
所述异常数据定位与处理单元对数据质量问题信息的统计函数进行分类处理，将不同数据质量问题添加至相应评估指标对应的缓存数据库中；The abnormal data location and processing unit classifies the statistical functions of the data quality problem information, and adds different data quality problems to the cache database corresponding to the corresponding evaluation index;
所述确定评价方法单元用于确定数据质量评价计算方法；The determination evaluation method unit is used to determine the data quality evaluation calculation method;
所述获取方法参数单元用于提取缓存库中的所需统计信息，根据评价方法计算得到评价参数信息并缓存至结果数据库中。The acquisition method parameter unit is used to extract the required statistical information in the cache library, calculate the evaluation parameter information according to the evaluation method and cache it in the result database.
优选的，所述数据质量评估模块根据异常甄别模块数据获取数据质量评估指标的结果信息，并选择可视化操作，输出数据质量评估报告，所述数据质量评估模块包括获取评估指标单元、选择可视化方式单元及评估报告单元；Preferably, the data quality evaluation module obtains the result information of the data quality evaluation index according to the abnormality screening module data, and selects a visualization operation to output a data quality evaluation report, and the data quality evaluation module includes an evaluation index unit and a visualization mode selection unit and assessment report unit;
所述获取评估指标单元根据评估任务提结果数据库中的评估结果信息，并发送至选择可视化方式单元；The obtaining evaluation indicator unit extracts the evaluation result information in the result database according to the evaluation task, and sends it to the selection visualization mode unit;
所述选择可视化方式单元根据结果信息选择数据可视化方式，调整数据可视化参数，绘制可视化图形，输出到数据质量评估报告模块中；The selection visualization mode unit selects a data visualization mode according to the result information, adjusts data visualization parameters, draws a visualization graph, and outputs it to the data quality assessment report module;
所述评估报告单元根据预先设定的报告模板出具数据质量评估报告。The evaluation report unit issues a data quality evaluation report according to a preset report template.
优选的，所述约束规则库包括数据质量评估指标单元及数据质量约束规则单元；Preferably, the constraint rule library includes a data quality evaluation index unit and a data quality constraint rule unit;
所述数据质量评估指标单元中的指标包括数据质量基础评估指标及数据质量扩展评估指标；The indicators in the data quality assessment indicator unit include data quality basic assessment indicators and data quality extended assessment indicators;
所述数据质量约束规则单元包括及时性规则、完整性规则、准确性规则、数据字典完备性规则及设计实现一致性规则。The data quality constraint rule unit includes timeliness rules, integrity rules, accuracy rules, data dictionary completeness rules and design and implementation consistency rules.
一种电力运营监控数据质量评估方法，所述方法在一种电力运营监控数据质量评估系统中实现，所述系统包括均独立执行脚本代码的数据集抽取模块、异常甄别模块和数据质量评估模块；所述数据集抽取模块划分评估任务并进行数据集抽取；所述异常甄别模块筛选数据质量问题，并进行数据质量评价和量化计算；所述数据质量评估模块进行数据质量结果的可视化操作，输出数据质量报告；所述方法包括如下步骤：A method for evaluating the quality of electric power operation monitoring data, the method is implemented in a system for evaluating the quality of electric power operation monitoring data, and the system includes a data set extraction module, an abnormality screening module, and a data quality evaluation module that independently execute script codes; The data set extraction module divides evaluation tasks and performs data set extraction; the abnormality screening module screens data quality problems, and performs data quality evaluation and quantitative calculation; the data quality evaluation module performs visualization operations on data quality results, and outputs data quality report; said method comprises the steps of:
步骤1.所述数据集抽取模块从电力运营监控数据中心抽取所需的数据集，其中，所述数据集由任意组合的单个数据表或子表组成；Step 1. The data set extraction module extracts the required data set from the power operation monitoring data center, wherein the data set is composed of a single data table or sub-table in any combination;
步骤2.所述异常甄别模块针对所述目标数据集，利用数据质量评估指标和约束规则甄别出异常电力数据，处理检查出的数据质量问题并确定评价方法，存储计算结果；Step 2. The abnormality screening module uses data quality assessment indicators and constraint rules to identify abnormal power data for the target data set, processes the detected data quality problems, determines an evaluation method, and stores calculation results;
步骤3.根据异常甄别模块数据获取数据质量评估指标的结果信息，并选择可视化操作，输出数据质量评估报告。Step 3. Obtain the result information of the data quality assessment index according to the data of the abnormality screening module, and select the visualization operation to output the data quality assessment report.
优选的，所述步骤1包括：Preferably, said step 1 includes:
1-1.发布数据质量评估任务，所述数据质量评估任务包括若干子任务，每个所述子任务均为数据质量评估对象与评估指标的组合；1-1. Release a data quality assessment task, the data quality assessment task includes several subtasks, each of which is a combination of data quality assessment objects and assessment indicators;
1-2.根据用户需求，将相应的数据质量评估指标及约束规则添加到所述数据质量评估中；1-2. According to user needs, add corresponding data quality evaluation indicators and constraint rules to the data quality evaluation;
1-3.基于所述数据质量评估指标和约束规则，获取需实施数据质量评估的目标数据集，形成数据质量问题的甄别任务。1-3. Based on the data quality evaluation indicators and constraint rules, obtain the target data set that needs to be evaluated for data quality, and form a screening task for data quality problems.
优选的，所述步骤2包括：Preferably, said step 2 includes:
2-1.根据约束规则库，对所述目标数据集启动异常数据甄别任务，扫描数据对象并定位数据质量问题，将检查出的数据质量问题信息返回给异常数据处理模块；2-1. According to the constraint rule library, start the abnormal data screening task for the target data set, scan the data object and locate the data quality problem, and return the detected data quality problem information to the abnormal data processing module;
2-2.对数据质量问题信息的统计函数进行分类处理，将不同数据质量问题添加至相2-2. Classify the statistical functions of the data quality problem information, and add different data quality problems to the corresponding
应评估指标对应的缓存数据库中；In the cache database corresponding to the evaluation indicator;
2-3.确定数据质量评价计算方法；2-3. Determine the calculation method for data quality evaluation;
2-4.提取缓存库中的所需统计信息，根据评价方法计算得到评价参数信息并缓存至结果数据库中。2-4. Extract the required statistical information in the cache library, calculate the evaluation parameter information according to the evaluation method and cache it in the result database.
优选的，所述步骤3包括：Preferably, said step 3 includes:
3-1.根据评估任务提结果数据库中的评估结果信息，并发送至选择可视化方式单元；3-1. Extract the evaluation result information in the result database according to the evaluation task, and send it to the selection visualization mode unit;
3-2.根据结果信息选择数据可视化方式，调整数据可视化参数，绘制可视化图形，输出到数据质量评估报告模块中；3-2. Select the data visualization method according to the result information, adjust the data visualization parameters, draw the visualization graph, and output it to the data quality assessment report module;
3-3.根据预先设定的报告模板出具数据质量评估报告。3-3. Issue a data quality assessment report according to the preset report template.
优选的，所述2-4中的所述评价方法包括数据表级评价方法、任务表级评估量化计算方法及数据质量扩展指标评价方法；Preferably, the evaluation method in the above 2-4 includes a data table-level evaluation method, a task table-level evaluation quantitative calculation method, and a data quality expansion index evaluation method;
所述数据表级评价方法包括数据表级质量评价结果数据方法和数据表级质量评价计算方法；The data table-level evaluation method includes a data table-level quality evaluation result data method and a data table-level quality evaluation calculation method;
所述任务表级评估量化计算方法包括任务表级质量评估结果数据计算方法和任务表级质量评估量化计算方法；The task list-level evaluation quantitative calculation method includes a task list-level quality assessment result data calculation method and a task list-level quality assessment quantitative calculation method;
所述数据质量扩展指标评价方法包括扩展级评估结果数据评价方法和扩展质量评估计算方法。The data quality extended index evaluation method includes an extended level evaluation result data evaluation method and an extended quality evaluation calculation method.
从上述的技术方案可以看出，本发明提供了一种电力运营监控数据质量评估系统及方法，系统包括均独立执行脚本代码的数据集抽取模块、异常甄别模块和数据质量评估模块；方法包括从电力运营监控数据中心抽取所需的数据集；针对目标数据集，利用数据质量评估指标和约束规则甄别出异常电力数据，处理检查出的数据质量问题并确定评价方法，存储计算结果；获取数据质量评估指标的结果信息并选择可视化操作。本发明提出的系统及方法避免了现有数据质量评估过程中的问题，大大提高了整体数据质量评估效率；能够进行直观的量化评估和结果反馈，从而提供了系统全面的数据质量评估标准，保障数据的准确、及时、有效和可信，促进了数据质量的提升，进而提高了电力系统的运行稳定性。It can be seen from the above technical solutions that the present invention provides a system and method for evaluating the quality of power operation monitoring data. The system includes a data set extraction module, an abnormality screening module, and a data quality evaluation module that independently execute script codes; the method includes: The power operation monitoring data center extracts the required data sets; for the target data set, uses data quality evaluation indicators and constraint rules to identify abnormal power data, handles the detected data quality problems, determines the evaluation method, and stores the calculation results; obtains data quality Evaluate the resulting information for the indicator and choose a visualization action. The system and method proposed by the present invention avoid the problems in the existing data quality evaluation process, and greatly improve the overall data quality evaluation efficiency; it can perform intuitive quantitative evaluation and result feedback, thereby providing a systematic and comprehensive data quality evaluation standard to ensure The accuracy, timeliness, effectiveness and credibility of the data have promoted the improvement of data quality, thereby improving the operation stability of the power system.
与最接近的现有技术比，本发明提供的技术方案具有以下优异效果：Compared with the closest prior art, the technical solution provided by the present invention has the following excellent effects:
1、本发明所提供的技术方案中，根据用户数据质量需求，将评估任务划分为数据表级和任务级进行数据集抽取；从目标数据集中检查和甄别出所有要求的数据质量问题，进行数据质量评价和量化计算；针对评估结果进行数据可视化操作，输出数据质量报告。其中，数据集抽取、异常甄别和数据质量评估模块的脚本代码将会相互独立执行，避免了现有数据质量评估过程中一旦出错、需重新执行的问题，大大提高了整体数据质量评估效率和容错能力。1. In the technical solution provided by the present invention, according to user data quality requirements, the evaluation task is divided into data table level and task level for data set extraction; check and identify all required data quality problems from the target data set, and perform data Quality evaluation and quantitative calculation; perform data visualization operations on the evaluation results, and output data quality reports. Among them, the script codes of data set extraction, anomaly screening and data quality assessment modules will be executed independently of each other, avoiding the problem of re-execution once an error occurs in the existing data quality assessment process, and greatly improving the overall data quality assessment efficiency and fault tolerance ability.
2、本发明所提供的技术方案，构建了电力运监监控数据评估指标以及约束规则，并给出了数据质量评价的具体方法。其中，定义了基础性评估指标(及时性、完整性、准确性)和扩展性评估指标(数据字典完备性、设计实现一致性)，限定了业务及时性约束、传输及时性约束、非空约束、值域约束等约束规则，并对数据质量评估指标定义了具体的评价方法，能够进行直观的量化评估和结果反馈，从而提供了系统全面的数据质量评估标准，促进数据质量的提升。2. The technical solution provided by the present invention constructs the monitoring data evaluation index and constraint rules of electric power operation monitoring, and provides a specific method for data quality evaluation. Among them, the basic evaluation indicators (timeliness, completeness, accuracy) and scalability evaluation indicators (data dictionary completeness, design and implementation consistency) are defined, and the business timeliness constraints, transmission timeliness constraints, and non-empty constraints are defined. , value range constraints and other constraint rules, and define specific evaluation methods for data quality evaluation indicators, which can perform intuitive quantitative evaluation and result feedback, thus providing systematic and comprehensive data quality evaluation standards and promoting the improvement of data quality.
3、本发明所提供的技术方案，电力运营监控数据质量评估指标给出了具体的数据质量评价方法与实现过程，通过统计信息量化电力数据在每个数据质量评估指标上的状态情况，进而得出评估数据集的质量情况，能够将抽象的电力运营监控数据质量评估转化为直观的量化评估结果反馈，实现了数据质量的定性和定量评估，提供了综合多维的数据质量评价方法。3. In the technical solution provided by the present invention, the power operation monitoring data quality evaluation index provides a specific data quality evaluation method and implementation process, quantifies the status of power data on each data quality evaluation index through statistical information, and then obtains By evaluating the quality of the data set, it can transform the abstract power operation monitoring data quality evaluation into intuitive quantitative evaluation result feedback, realize the qualitative and quantitative evaluation of data quality, and provide a comprehensive multi-dimensional data quality evaluation method.
4、本发明提供的技术方案，应用广泛，具有显著的社会效益和经济效益。4. The technical solution provided by the present invention is widely used and has significant social and economic benefits.
附图说明Description of drawings
图1是本发明的一种电力运营监控数据质量评估系统的示意图；Fig. 1 is a schematic diagram of a power operation monitoring data quality evaluation system of the present invention;
图2是本发明的系统中的数据质量评估指标和约束规则的关系示意图；Fig. 2 is a schematic diagram of the relationship between data quality evaluation indicators and constraint rules in the system of the present invention;
图3是本发明的一种电力运营监控数据质量评估方法的流程图；Fig. 3 is a flow chart of a method for evaluating the quality of power operation monitoring data of the present invention;
图4是本发明的方法中步骤1的流程示意图；Fig. 4 is the schematic flow sheet of step 1 in the method of the present invention;
图5是本发明的方法中步骤2的流程示意图；Fig. 5 is the schematic flow sheet of step 2 in the method of the present invention;
图6是本发明的方法中步骤3的流程示意图。Fig. 6 is a schematic flow chart of step 3 in the method of the present invention.
具体实施方式detailed description
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.
如图1所示，本发明提供一种电力运营监控数据质量评估系统，系统包括均独立执行脚本代码的数据集抽取模块、异常甄别模块和数据质量评估模块；As shown in Figure 1, the present invention provides a power operation monitoring data quality assessment system, the system includes a data set extraction module, an abnormality screening module and a data quality assessment module that independently execute script codes;
数据集抽取模块划分评估任务并进行数据集抽取；The data set extraction module divides the evaluation tasks and performs data set extraction;
异常甄别模块筛选数据质量问题，并进行数据质量评价和量化计算；The anomaly screening module screens data quality problems, and performs data quality evaluation and quantitative calculations;
数据质量评估模块进行数据质量结果的可视化操作，输出数据质量报告。The data quality assessment module visualizes the data quality results and outputs a data quality report.
其中，数据集抽取模块从电力运营监控数据中心抽取所需的数据集，其中，数据集由任意组合的单个数据表或子表组成；数据集抽取模块包括数据质量评估任务单元、选择评估指标与规则单元及抽取目标数据集单元；Among them, the data set extraction module extracts the required data set from the power operation monitoring data center, wherein the data set is composed of a single data table or sub-table in any combination; the data set extraction module includes a data quality evaluation task unit, selection of evaluation indicators and Rule unit and extraction target data set unit;
数据质量评估任务单元用于发布数据质量评估任务，数据质量评估任务包括若干子任务，每个子任务均为数据质量评估对象与评估指标的组合；The data quality assessment task unit is used to issue data quality assessment tasks. The data quality assessment task includes several subtasks, and each subtask is a combination of data quality assessment objects and assessment indicators;
选择评估指标与规则单元根据用户需求，将相应的数据质量评估指标及约束规则添加到数据质量评估中；Select the evaluation index and rule unit to add the corresponding data quality evaluation index and constraint rules to the data quality evaluation according to user needs;
抽取目标数据集单元基于数据质量评估指标和约束规则，获取需实施数据质量评估的目标数据集，形成数据质量问题的甄别任务。The target data set unit is extracted based on the data quality evaluation indicators and constraint rules, and the target data set that needs to be evaluated for data quality is obtained to form the identification task of data quality problems.
其中，异常甄别模块针对目标数据集，利用数据质量评估指标和约束规则甄别出异常电力数据，处理检查出的数据质量问题并确定评价方法，存储计算结果；异常甄别模块包括检查约束规则单元、异常数据定位与处理单元、确定评价方法单元及获取方法参数单元；Among them, the abnormality screening module uses data quality assessment indicators and constraint rules to identify abnormal power data for the target data set, handles the detected data quality problems, determines the evaluation method, and stores the calculation results; the abnormality screening module includes checking constraint rule units, abnormal Data positioning and processing unit, determination evaluation method unit and acquisition method parameter unit;
检查约束规则单元根据约束规则库，对目标数据集启动异常数据甄别任务，扫描数据对象并定位数据质量问题，将检查出的数据质量问题信息返回给异常数据处理模块；The check constraint rule unit starts the abnormal data screening task for the target data set according to the constraint rule library, scans the data object and locates the data quality problem, and returns the detected data quality problem information to the abnormal data processing module;
异常数据定位与处理单元对数据质量问题信息的统计函数进行分类处理，将不同数据质量问题添加至相应评估指标对应的缓存数据库中；The abnormal data location and processing unit classifies and processes the statistical functions of the data quality problem information, and adds different data quality problems to the cache database corresponding to the corresponding evaluation index;
确定评价方法单元用于确定数据质量评价计算方法；Determining the evaluation method unit is used to determine the calculation method for data quality evaluation;
获取方法参数单元用于提取缓存库中的所需统计信息，根据评价方法计算得到评价参数信息并缓存至结果数据库中。The acquisition method parameter unit is used to extract the required statistical information in the cache library, calculate the evaluation parameter information according to the evaluation method and cache it in the result database.
其中，数据质量评估模块根据异常甄别模块数据获取数据质量评估指标的结果信息，并选择可视化操作，输出数据质量评估报告，数据质量评估模块包括获取评估指标单元、选择可视化方式单元及评估报告单元；Among them, the data quality evaluation module obtains the result information of the data quality evaluation index according to the data of the abnormality screening module, and selects a visualization operation to output a data quality evaluation report. The data quality evaluation module includes an evaluation index unit, a visualization mode selection unit and an evaluation report unit;
获取评估指标单元根据评估任务提结果数据库中的评估结果信息，并发送至选择可视化方式单元；The evaluation indicator unit obtains the evaluation result information in the result database according to the evaluation task, and sends it to the selection visualization mode unit;
选择可视化方式单元根据结果信息选择数据可视化方式，调整数据可视化参数，绘制可视化图形，输出到数据质量评估报告模块中；Select the visualization method unit to select the data visualization method according to the result information, adjust the data visualization parameters, draw the visualization graph, and output it to the data quality assessment report module;
评估报告单元根据预先设定的报告模板出具数据质量评估报告。The evaluation report unit issues a data quality evaluation report according to a preset report template.
如图2所示，约束规则库包括数据质量评估指标单元及数据质量约束规则单元；As shown in Figure 2, the constraint rule library includes a data quality evaluation index unit and a data quality constraint rule unit;
数据质量评估指标单元中的指标包括数据质量基础评估指标及数据质量扩展评估指标；The indicators in the data quality assessment indicator unit include data quality basic assessment indicators and data quality extended assessment indicators;
数据质量约束规则单元包括及时性规则、完整性规则、准确性规则、数据字典完备性规则及设计实现一致性规则；The data quality constraint rule unit includes timeliness rules, integrity rules, accuracy rules, data dictionary completeness rules and design implementation consistency rules;
及时性规则、完整性规则及准确性规则均为数据质量基础评估指标的规则；Timeliness rules, completeness rules and accuracy rules are the rules of basic evaluation indicators of data quality;
数据字典完备性规则及设计实现一致性规则均为数据质量扩展评估指标的规则。The completeness rules of data dictionary and the consistency rules of design and implementation are the rules of data quality extended evaluation index.
如图3所示，本发明提供一种电力运营监控数据质量评估方法，方法在一种电力运营监控数据质量评估系统中实现，系统包括均独立执行脚本代码的数据集抽取模块、异常甄别模块和数据质量评估模块；数据集抽取模块划分评估任务并进行数据集抽取；异常甄别模块筛选数据质量问题，并进行数据质量评价和量化计算；数据质量评估模块进行数据质量结果的可视化操作，输出数据质量报告；As shown in Figure 3, the present invention provides a method for evaluating the quality of power operation monitoring data. The method is implemented in a system for evaluating the quality of power operation monitoring data. The system includes a data set extraction module that independently executes script codes, an abnormality screening module and Data quality evaluation module; data set extraction module divides evaluation tasks and performs data set extraction; anomaly screening module screens data quality problems, and performs data quality evaluation and quantitative calculation; data quality evaluation module performs visualization operation of data quality results, and outputs data quality Report;
包括如下步骤：Including the following steps:
步骤1.数据集抽取模块从电力运营监控数据中心抽取所需的数据集，其中，数据集由任意组合的单个数据表或子表组成；Step 1. The data set extraction module extracts the required data set from the power operation monitoring data center, wherein the data set is composed of a single data table or sub-table in any combination;
步骤2.异常甄别模块针对目标数据集，利用数据质量评估指标和约束规则甄别出异常电力数据，处理检查出的数据质量问题并确定评价方法，存储计算结果；Step 2. The abnormality screening module uses data quality assessment indicators and constraint rules to identify abnormal power data for the target data set, handles the detected data quality problems, determines the evaluation method, and stores the calculation results;
步骤3.根据异常甄别模块数据获取数据质量评估指标的结果信息，并选择可视化操作，输出数据质量评估报告。Step 3. Obtain the result information of the data quality assessment index according to the data of the abnormality screening module, and select the visualization operation to output the data quality assessment report.
如图4所示，步骤1包括：As shown in Figure 4, step 1 includes:
1-1.发布数据质量评估任务，数据质量评估任务包括若干子任务，每个子任务均为数据质量评估对象与评估指标的组合；1-1. Release the data quality assessment task. The data quality assessment task includes several subtasks, and each subtask is a combination of data quality assessment objects and assessment indicators;
1-2.根据用户需求，将相应的数据质量评估指标及约束规则添加到数据质量评估中；1-2. According to user needs, add corresponding data quality evaluation indicators and constraint rules to data quality evaluation;
1-3.基于数据质量评估指标和约束规则，获取需实施数据质量评估的目标数据集，形成数据质量问题的甄别任务。1-3. Based on the data quality evaluation indicators and constraint rules, obtain the target data set that needs to be evaluated for data quality, and form a screening task for data quality problems.
如图5所示，步骤2包括：As shown in Figure 5, step 2 includes:
2-1.根据约束规则库，对目标数据集启动异常数据甄别任务，扫描数据对象并定位数据质量问题，将检查出的数据质量问题信息返回给异常数据处理模块；2-1. According to the constraint rule library, start the abnormal data screening task for the target data set, scan the data object and locate the data quality problem, and return the detected data quality problem information to the abnormal data processing module;
2-2.对数据质量问题信息的统计函数进行分类处理，将不同数据质量问题添加至相应评估指标对应的缓存数据库中；2-2. Classify and process the statistical functions of data quality problem information, and add different data quality problems to the cache database corresponding to the corresponding evaluation indicators;
2-3.确定数据质量评价计算方法；2-3. Determine the calculation method for data quality evaluation;
2-4.提取缓存库中的所需统计信息，根据评价方法计算得到评价参数信息并缓存至结果数据库中。2-4. Extract the required statistical information in the cache library, calculate the evaluation parameter information according to the evaluation method and cache it in the result database.
如图6所示，步骤3包括：As shown in Figure 6, step 3 includes:
3-1.根据评估任务提结果数据库中的评估结果信息，并发送至选择可视化方式单元；3-1. Extract the evaluation result information in the result database according to the evaluation task, and send it to the selection visualization mode unit;
3-2.根据结果信息选择数据可视化方式，调整数据可视化参数，绘制可视化图形，输出到数据质量评估报告模块中；3-2. Select the data visualization method according to the result information, adjust the data visualization parameters, draw the visualization graph, and output it to the data quality assessment report module;
3-3.根据预先设定的报告模板出具数据质量评估报告。3-3. Issue a data quality assessment report according to the preset report template.
其中，2-4中的评价方法包括数据表级评价方法、任务表级评估量化计算方法及数据质量扩展指标评价方法；Among them, the evaluation methods in 2-4 include data table-level evaluation methods, task table-level evaluation quantitative calculation methods and data quality expansion index evaluation methods;
数据表级评价方法包括数据表级质量评价结果数据方法和数据表级质量评价计算方法；The data table-level evaluation method includes the data table-level quality evaluation result data method and the data table-level quality evaluation calculation method;
任务表级评估量化计算方法包括任务表级质量评估结果数据计算方法和任务表级质量评估量化计算方法；Quantitative calculation methods for task list-level assessment include task list-level quality assessment result data calculation methods and task list-level quality assessment quantitative calculation methods;
数据质量扩展指标评价方法包括扩展级评估结果数据评价方法和扩展质量评估计算方法。The data quality extended index evaluation method includes an extended level evaluation result data evaluation method and an extended quality evaluation calculation method.
本发明提供一种应用在电力运营监控数据质量评估系统中的电力运营监控数据质量评估方法的具体应用例，如下：The present invention provides a specific application example of a power operation monitoring data quality evaluation method applied in a power operation monitoring data quality evaluation system, as follows:
将电力运营监控数据划分为数据表级和任务级数据集，按照数据质量评估指标和约束规则，从源数据中提取目标数据集，进行全面的系统的数据质量评估。The power operation monitoring data is divided into data table-level and task-level data sets, and the target data set is extracted from the source data according to the data quality evaluation indicators and constraint rules to conduct a comprehensive and systematic data quality evaluation.
1.数据质量评估系统1. Data quality assessment system
图1为本发明实施例的数据质量评估系统的示意图，包含数据集抽取模块、异常甄别模块和数据质量评估模块，具体实施过程如下：Fig. 1 is a schematic diagram of a data quality assessment system according to an embodiment of the present invention, including a data set extraction module, an abnormality screening module and a data quality assessment module, and the specific implementation process is as follows:
(1)数据集抽取模块：以数据质量评估为指导，从电力运营监控数据中心抽取所需的数据集，其中，数据集由任意组合的单个数据表(或子表)所组成，主要包含以下部分：(1) Data set extraction module: Guided by data quality assessment, the required data set is extracted from the power operation monitoring data center, where the data set consists of any combination of single data tables (or sub-tables), mainly including the following part:
a)数据质量评估任务：该模块主要发布数据质量评估任务。数据质量评估过程是以评估任务为单位，一个数据质量评估任务又可以包含若干子任务，一个子任务是数据质量评估对象与评估指标的组合。a) Data quality assessment tasks: This module mainly publishes data quality assessment tasks. The data quality assessment process is based on assessment tasks. A data quality assessment task can contain several subtasks. A subtask is a combination of data quality assessment objects and assessment indicators.
b)选择评估指标与规则：根据用户需求，把相应的数据质量评估指标和数据质量约束规则(形式如：<评估指标1(规则1，规则2，...)>，<评估指标2(规则1，规则2，...)>，...)添加到评估任务模块中，为目标数据添加限定条件。b) Selection of evaluation indicators and rules: According to user needs, the corresponding data quality evaluation indicators and data quality constraint rules (forms such as: <evaluation index 1 (rule 1, rule 2, ...)>, <evaluation index 2 ( rule1, rule2, ...)>, ...) are added to the evaluation task module to add qualifications for the target data.
c)抽取目标数据集：该模块主要获取目标数据集。基于评估指标和约束规则，获取要实施数据质量评估的目标对象，从而形成数据质量问题的甄别任务。c) Extract the target data set: This module mainly obtains the target data set. Based on the evaluation indicators and constraint rules, the target objects to be evaluated for data quality are obtained, thereby forming the identification task of data quality problems.
(2)异常甄别模块：针对目标评估对象，利用评估指标和约束规则甄别出异常电力数据，处理检查出的数据质量问题，并确定评价方法，存储相关计算结果，主要包含以下部分：(2) Abnormality screening module: For the target evaluation object, use evaluation indicators and constraint rules to identify abnormal power data, deal with the detected data quality problems, determine the evaluation method, and store related calculation results. It mainly includes the following parts:
a)检查约束规则：该模块主要涉及基于约束规则的数据质量检查函数。针对目标数据集，检查函数启动异常数据甄别任务，扫描数据对象，发现并定位数据质量问题，并将检查出的数据质量问题信息返回给异常数据处理模块。a) Check constraint rules: This module mainly involves data quality check functions based on constraint rules. For the target data set, the check function starts the abnormal data screening task, scans the data object, finds and locates data quality problems, and returns the detected data quality problem information to the abnormal data processing module.
b)异常数据处理：该模块主要涉及基于评估指标的统计函数。针对异常数据，统计函数进行分类处理，将不同数据质量问题添加相应评估指标对应的缓存数据库中，以供评价结果计算。b) Abnormal data processing: This module mainly involves statistical functions based on evaluation indicators. For abnormal data, statistical functions are used to classify and process, and different data quality problems are added to the cache database corresponding to the corresponding evaluation index for calculation of evaluation results.
c)确定评价方法：该模块主要确定数据质量评价计算方法。根据评估指标，映射到具体的数据质量量化计算方式，为评价结果计算提供实际可行的方法，具体评价方法见下一节。c) Determine the evaluation method: This module mainly determines the calculation method for data quality evaluation. According to the evaluation index, it is mapped to the specific quantitative calculation method of data quality, and provides a practical and feasible method for the evaluation result calculation. The specific evaluation method is shown in the next section.
d)获取方法参数：该模块主要计算评价参数信息。根据b)和d)，提取缓存库中的所需统计信息，利用评价方法计算评估结果，并缓存到结果数据库中。d) Obtaining method parameters: This module mainly calculates evaluation parameter information. According to b) and d), the required statistical information in the cache library is extracted, the evaluation result is calculated by the evaluation method, and cached in the result database.
(3)数据质量评估模块：根据异常甄别模块数据，获取数据质量评估指标的结果信息，并选择可视化操作，输出数据质量评估报告，主要包含以下部分：(3) Data quality assessment module: According to the data of the abnormality screening module, the result information of the data quality assessment indicators is obtained, and the visualization operation is selected to output the data quality assessment report, which mainly includes the following parts:
a)获取评估指标：该模块主要获取评估指标的结果信息。根据评估任务，提结果数据库中的评估结果信息，并反馈给可视化模块。a) Obtain evaluation indicators: This module mainly obtains the result information of evaluation indicators. According to the evaluation task, the evaluation result information in the result database is extracted and fed back to the visualization module.
b)选择可视化方式：该模块主要进行数据可视化操作。根据结果信息，选择数据可视化方式，调整数据可视化参数，绘制可视化图形，输出到数据质量评估报告模块中。b) Select a visualization method: This module mainly performs data visualization operations. According to the result information, select the data visualization method, adjust the data visualization parameters, draw the visualization graph, and output it to the data quality assessment report module.
c)评估报告：根据预先设定的报告模板，该模块出具数据质量评估报告，反映整体数据质量情况。c) Evaluation report: According to the pre-set report template, this module issues a data quality evaluation report to reflect the overall data quality.
该发明的实施离不开数据中心、约束规则库和评估指标的支撑。数据中心存储所有的电力运营监控数据，所有评估数据集均是它的子集；约束规则库存储所有数据质量问题的判别规则，为数据质量评估提供标准；评估指标是数据质量评估的基本对象，直接对应不同的约束规则。有关约束规则和评估指标的详细情况见下一节。The implementation of the invention is inseparable from the support of the data center, constraint rule library and evaluation index. The data center stores all power operation monitoring data, and all evaluation data sets are a subset of it; the constraint rule library stores the discriminant rules for all data quality problems, providing standards for data quality evaluation; evaluation indicators are the basic objects of data quality evaluation, Directly correspond to different constraint rules. See the next section for details on constraint rules and evaluation metrics.
2.数据质量评估指标及约束规则2. Data quality evaluation indicators and constraint rules
A.数据质量评估指标A. Data Quality Evaluation Indicators
数据质量评估指标包含基础和扩展评估指标：Data quality evaluation indicators include basic and extended evaluation indicators:
1)数据质量基础评估指标是判别数据质量的基本要素，包含三个方面：1) Data quality basic evaluation indicators are the basic elements for judging data quality, including three aspects:
a)及时性反映了数据对于应用领域的更新程度，包括业务层及时性和数据层及时性。前者关注各个业务状态是否及时更新，后者关注数据本身的及时更新情况。a) Timeliness reflects the update degree of data for the application field, including the timeliness of the business layer and the timeliness of the data layer. The former focuses on whether the status of each business is updated in time, while the latter focuses on the timely update of the data itself.
b)完整性用来描述数值信息的完整程度。b) Integrity is used to describe the completeness of numerical information.
c)准确性是描述待测数据与真实数据的符合程度，体现在合规性和一致性两方面。前者反映同一数据表内部字段上数据内容的合规程度，后者关注不同数据表之间字段上数据内容的一致程度。c) Accuracy is to describe the degree of conformity between the data to be tested and the real data, which is reflected in two aspects of compliance and consistency. The former reflects the degree of compliance of data content on fields within the same data table, while the latter focuses on the degree of consistency of data content on fields between different data tables.
2)数据质量扩展评估指标是与技术发展相关的数据质量评估要素，包含两个方面：2) The data quality expansion evaluation index is an element of data quality evaluation related to technological development, which includes two aspects:
a)数据字典完备性反映了对于数据模型中的数据对象或者项目描述的完备程度。a) The completeness of the data dictionary reflects the completeness of the data object or item description in the data model.
b)设计实现一致性是描述业务数据的实际实现与顶层模型设计的一致程度。b) Design and implementation consistency refers to the degree of consistency between the actual implementation of business data and the top-level model design.
B.数据质量约束规则B. Data Quality Constraint Rules
数据质量评估指标通过多个维度的约束规则进行描述。一个数据质量评估指标能被多个约束规则描述，一个约束规则能描述多个数据质量评估指标，具体关系如图2所示。The data quality evaluation index is described by the constraint rules of multiple dimensions. A data quality evaluation indicator can be described by multiple constraint rules, and a constraint rule can describe multiple data quality assessment indicators. The specific relationship is shown in Figure 2.
1)及时性1) Timeliness
a)业务及时性约束规则：是对数据集的业务状态从发生到形成的滞后时间约束规则，体现为对所允许的数据滞后形成时间的定义。a) Business timeliness constraint rule: It is a lag time constraint rule for the business state of the data set from occurrence to formation, which is reflected in the definition of the allowed data lag formation time.
b)传输及时性约束规则：是对数据集的业务数据从形成到进库的滞后时间约束规则，体现为对所允许的数据滞后进库时间的定义。b) Transmission timeliness constraint rule: it is the lag time constraint rule for the business data of the data set from formation to storage, which is reflected in the definition of the allowed data lag time into the storage.
2)完整性2) Integrity
a)记录完整性约束规则：是对数据集数据长度的约束规则。一个数据集期望的数据记录数可以通过期望的值来进行约束，这个期望的值可以根据一定的业务需要进行计算。a) Record integrity constraint rule: it is a constraint rule on the data length of the data set. The expected number of data records in a data set can be constrained by the expected value, which can be calculated according to certain business needs.
b)字段完整性约束规则：是对记录之间字段连续取值的约束规则。一个数据集字段的值必须(或在满足某一条件时必须)出现连续的取值。b) Constraint rules for field integrity: It is a constraint rule for continuous values of fields between records. The value of a data set field must (or must meet a certain condition) appear consecutive values.
c)非空约束规则：是对字段上的空值的约束规则。字段上的数据内容(或字段值)不允许(或在满足某种条件时不允许)出现空值。c) Non-empty constraint rule: it is a constraint rule for the null value on the field. The data content (or field value) on the field does not allow (or does not allow when a certain condition is met) to have a null value.
d)外键约束规则：是对数据集之间关联程度的约束规则，反映一个表的主键值需被另一个表的引用程度。d) Foreign key constraint rule: It is a constraint rule on the degree of association between data sets, reflecting the degree to which the primary key value of a table needs to be referenced by another table.
3)准确性3) Accuracy
准确性包含合规性和一致性两方面，前者反映同一数据集质量，后者反映不同数据集质量。二者通过不同的约束规则进行评估，也通过相同约束规则(但不同维度)进行评估。其中，交叉约束规则有等值性和逻辑性存在性约束规则。Accuracy includes compliance and consistency. The former reflects the quality of the same dataset, and the latter reflects the quality of different datasets. Both are evaluated by different constraint rules, and also by the same constraint rules (but different dimensions). Among them, the cross constraint rules include equivalence and logical existence constraint rules.
a)合规性a) Compliance
●值域约束规则：是对字段上的数据取值范围的约束规则。字段上的数据取值必须在某一范围内出现。● Value domain constraint rules: It is a constraint rule on the range of data values on the field. The data value on the field must appear within a certain range.
●类型约束规则：是对字段上的数据类型的约束规则。字段上的数据内容的数据类型必须与字段定义的数据类型相符合。●Type constraint rule: It is a constraint rule for the data type on the field. The data type of the data content on the field must match the data type defined by the field.
●格式约束规则：是对字段上的数据格式的约束规则。字段上的数据内容的数据类型必须与字段定义的数据格式相符合。● Format constraint rule: it is a constraint rule on the data format on the field. The data type of the data content on the field must match the data format defined by the field.
●精度约束规则：是对字段上的数据精度的约束规则。字段上的数据内容的数据类型必须与字段定义的精度相符合。●Precision Constraint Rules: Constraint rules for data precision on fields. The data type of the data content on the field must match the precision defined by the field.
●等值性约束规则：是对记录内部字段上数据取值的约束规则。在同一记录上，该字段上的数据必须与另一个或多个字段上的数据通过函数计算得出结果相同。● Equivalence constraint rule: It is a constraint rule for the value of data on the internal field of the record. On the same record, the data on this field must be the same as the data on another or more fields calculated by the function.
●逻辑性约束规则：是对记录内部字段上数据取值逻辑关系的约束规则。在同一记录上，该字段上的数据必须与另一个或多个字段上的数据满足某种逻辑关系(如等于、大于、小于等)。● Logical constraint rules: It is a constraint rule for the logical relationship of data values on the internal fields of the record. On the same record, the data on this field must satisfy a certain logical relationship (such as equal to, greater than, less than, etc.) with the data on another or more fields.
●唯一性约束规则：是对数据集内部是否存在重复记录的约束规则。一个数据表中的一条记录必须在该表内唯一，不允许重复记录出现。●Unique constraint rule: It is a constraint rule on whether there are duplicate records in the data set. A record in a data table must be unique within the table, duplicate records are not allowed.
b)一致性b) Consistency
●等值性约束规则：是对数据集之间字段上数据取值的约束规则。一个数据集字段上的数据取值必须与另一个或多个数据集的一个或多个字段的数据取值计算得出结果相同。● Equivalence constraint rule: it is a constraint rule for the value of data on fields between data sets. The data values on a field of a dataset must be calculated to be the same as the data values of one or more fields in another or more datasets.
●逻辑性依赖约束规则：是对数据集之间字段上数据逻辑关系的约束规则。该字段上的数据必须与另一个或多个数据表中的一个字段上的数据满足某种逻辑关系(如等于、大于、小于等)。●Logical dependency constraint rule: it is a constraint rule for the logical relationship of data on fields between data sets. The data on this field must satisfy a certain logical relationship (such as equal to, greater than, less than, etc.) with the data on one field in another or more data tables.
●存在性依赖约束规则：是对数据集之间字段上的数据存在关系的约束规则。该字段上的数据必须在另一个或多个数据表中的一个字段上出现。●Existence dependency constraint rule: It is a constraint rule for the relationship between the data on the fields between the datasets. The data on this field must appear on a field in another data table or tables.
4)数据字典完备性4) Data dictionary completeness
a)域完备性约束规则：是对数据字典上域(或子域)内容的整体说明的约束规则。数据字典必须对各个域(或子域)做出详细的解释说明。其中，数据字典是指对数据的数据项、数据结构、数据流、数据存储、处理逻辑、外部实体等进行定义和描述，其目的是对数据流程图中的各个评估指标做出详细的说明。a) Domain completeness constraint rule: it is a constraint rule for the overall description of the domain (or sub-domain) content on the data dictionary. The data dictionary must make detailed explanations for each domain (or subdomain). Among them, the data dictionary refers to the definition and description of data items, data structure, data flow, data storage, processing logic, external entities, etc., and its purpose is to make a detailed description of each evaluation indicator in the data flow diagram.
b)表完备性约束规则：是对数据字典上基本表设计的约束规则。数据字典必须(或在满足某一条件时必须)详细地定义和描述各个数据表的涵义。b) Constraint rules for completeness of tables: Constraint rules for basic table design on the data dictionary. The data dictionary must (or must meet a certain condition) define and describe the meaning of each data table in detail.
c)字段完备性约束规则：是对数据字典上字段标准化的约束规则。数据字典必须(或在满足某一条件时必须)对所有字段予以统一的名称、格式和涵义。c) Constraint rules for completeness of fields: Constraint rules for field standardization on the data dictionary. The data dictionary must (or when a certain condition is met) give uniform names, formats and meanings to all fields.
5)设计实现一致性5) Consistency by design
a)表一致性约束规则：是对数据库中数据表实现的约束规则。数据库必须遵守数据字典规范实现每个数据表。a) Constraint rule of table consistency: it is a constraint rule implemented on the data tables in the database. The database must implement each data table following the data dictionary specification.
b)字段一致性约束规则：是数据表中字段实现的约束规则。数据表必须遵守数据字典规范实现每个数据字段。b) Field consistency constraint rule: it is the constraint rule implemented by the field in the data table. The data table must comply with the data dictionary specification to implement each data field.
c)关系一致性约束规则：是对数据库中对象关系实现的约束规则。数据库必须遵守数据字典规范实现每个对象关系。c) Relational Consistency Constraint Rules: Constraint rules for the realization of object relationships in the database. The database must implement each object relation according to the data dictionary specification.
3.数据质量评价方法3. Data quality evaluation method
A.数据表级评价方法A. Data sheet-level evaluation method
该方法对单个数据表(或单个数据表的子表)组成的一个数据集进行质量评估指标计算，是数据质量评估的基本单位。This method calculates the quality evaluation index for a data set composed of a single data table (or a sub-table of a single data table), which is the basic unit of data quality evaluation.
1)数据表级质量评价结果数据1) Data sheet-level quality evaluation result data
首先，给出以下反映数据表级质量评估实际情况的结果数据：First, the following result data reflecting the actual situation of data sheet-level quality assessment are given:
a)问题分类数：在对一个数据集进行质量评估时，出现违反数据质量评估指标中约束规则种类的个数，记做Sq。a) The number of problem classifications: when evaluating the quality of a data set, the number of types that violate the constraint rules in the data quality evaluation index is recorded as Sq .
b)问题记录数：在对一个数据集进行质量评估时，存在违反数据质量评估指标中约束规则的记录个数，同一条记录出现多个违反约束规则的不重复计算，记做Rq。b) Number of problem records: When evaluating the quality of a data set, the number of records that violate the constraint rules in the data quality assessment index, and the same record has multiple non-repeated calculations that violate the constraint rules, which is recorded as Rq .
c)问题数据个数：在对一个数据集进行质量评估时，存在违反数据质量评估指标中约束规则的记录个数，同一条记录出现多个违反约束规则的重复计算，记做Dq。c) Number of problem data: When evaluating the quality of a data set, the number of records that violate the constraint rules in the data quality assessment index, and the same record has multiple double calculations that violate the constraint rules, recorded as Dq .
d)问题数据字段个数：在对一个数据集进行质量评估时，出现违反数据质量评估指标中约束规则的数据字段个数，一个字段违反多个约束规则不重复计算，记做Cq。d) Number of problem data fields: When evaluating the quality of a data set, the number of data fields that violate the constraint rules in the data quality assessment index, one field that violates multiple constraint rules will not be counted repeatedly, and it will be recorded as Cq .
e)缺少数据字段个数：一个数据集中数据字典定义数据字段在实际数据字段中没有出现的个数，记做Cs。e) The number of missing data fields: the number of data fields defined by the data dictionary in a data set that do not appear in the actual data fields, denoted as Cs .
f)多出数据字段个数：一个数据集中实际数据字段在数据字典定义数据字段中没有出现的个数，记做Ca。f) The number of extra data fields: the number of actual data fields in a data set that do not appear in the data fields defined in the data dictionary, denoted as Ca .
g)冗余记录总数：记录数据完全重复的记录数，记做Rr。g) The total number of redundant records: the number of records whose data is completely repeated, denoted as Rr .
h)实际冗余记录数：记录数据完全重复的不同数据内容个数，记做Rk。h) Actual number of redundant records: the number of different data contents with completely repeated record data, denoted as Rk .
2)数据表级质量评价计算方法2) Data sheet-level quality evaluation calculation method
a)及时性：设Th为某次数据发生时间，Tf为数据形成时间，Ta为数据到达时间，业务及时性约束规则定义的滞后时间记做Td1，传输及时性约束规则定义的滞后时间记做Td2，业务滞后时间为：ΔTbus＝Tf-Th；业务及时性延迟为：传输滞后时间为：ΔTdata＝Ta-Tf；传输及时性延迟为：a) Timeliness: Let Th be the time when a certain data occurs, Tf is the time when data is formed, Ta is the time when data arrives, and the lag time defined by the business timeliness constraint rules is recorded as Td1 , and the delay time defined by the transmission timeliness constraint rules is The lag time is recorded as Td2 , and the business lag time is: ΔTbus = Tf -Th ; the business timeliness delay is: Transmission lag time is: ΔTdata =Ta -Tf ; transmission timeliness delay is:
b)完整性：完整性评估指标包括问题数据个数(Dq-int)，问题记录数(Rq-int)，缺少记录数(Rl-int)，冗余记录数(Rr-int)，实际冗余记录数(Rk-int)，缺少数据字段个数(Cq-int)，多出数据字段个数(Ca-int)，设Cr为数据集的记录数，Cqd为(一个数据集中)数据字典定义字段个数，Rqc-int为完整性问题记录覆盖率，记录完整性为：Scr＝(1-Rl-int/(Cr-Rr-int+Rk-int+Rl-int))×100％；结构完整性为：Scs＝(1-Cs-int/Cqd)×100％；数据完整性为：Scomp＝(1-Dq-int/Cr(Cqd-Cs-int+Ca-int))×100％b) Integrity: Integrity evaluation indicators include the number of problem data (Dq-int ), the number of problem records (Rq-int ), the number of missing records (Rl-int ), the number of redundant records (Rr-int ), the number of actual redundant records (Rk-int ), the number of missing data fields (Cq-int ), the number of extra data fields (Ca-int ), let Cr be the number of records in the data set, Cqd is the number of fields defined in the data dictionary (in a data set), Rqc-int is the record coverage rate of the integrity problem, and the record integrity is: Scr ＝(1-Rl-int /(Cr -Rr-int +Rk-int +Rl-int ))×100%; structural integrity is: Scs =(1-Cs-int /Cqd )×100%; data integrity is: Scomp =(1- Dq-int /Cr (Cqd -Cs-int +Ca-int ))×100％
c)准确性：c) Accuracy:
●合规性：合规性的评估指标包括：问题分类数(Sq-com)，问题数据字段个数(Cq-com)，问题数据个数(Dq-com)，问题记录数(Rq-com)，缺少数据字段个数(Cs-com)，多出数据字段个数(Ca-com)，设Cr为数据集的记录数，Cqd为(一个数据集中)数据字典定义字段个数，Rqc-com为合规性问题记录覆盖率，那么数据合规性为：Sacc＝(1-Dq-com/Cr(Cqd-Cs-com+Ca-com))×100％●Compliance: The evaluation indicators of compliance include: the number of question categories (Sq-com ), the number of question data fields (Cq-com ), the number of question data (Dq-com ), and the number of question records ( Rq-com ), the number of missing data fields (Cs-com ), the number of excess data fields (Ca-com ), let Cr be the number of records in the data set, and Cqd be the data (in a data set) The dictionary defines the number of fields, and Rqc-com is the record coverage of compliance issues, then the data compliance is: Sacc ＝(1-Dq-com /Cr (Cqd -Cs-com +Ca -com ))×100%
●一致性：一致性评估指标包括问题数据字段数(Cq-cons)，问题分类数(Sq-cons)，问题数据个数(Dq-cons)，问题记录数(Rq-cons)，缺少数据字段个数(Cs-cons)，多出数据字段个数(Ca-cons)，设Cr为数据集的记录数，Cqd为(一个数据集中)数据字典定义数据字段个数，Rqc为问题记录覆盖率，那么数据一致性为：Scons＝(1-Dq-cons/Cr(Cqd-Cs-cons+Ca-cons))×100％● Consistency: Consistency evaluation indicators include the number of question data fields (Cq-cons ), the number of question classifications (Sq-cons ), the number of question data (Dq-cons ), and the number of question records (Rq-cons ) , the number of missing data fields (Cs-cons ), the number of excess data fields (Ca-cons ), let Cr be the number of records in the data set, and Cqd be the number of data fields defined by the data dictionary (in a data set) number, Rqc is the problem record coverage, then the data consistency is: Scons =(1-Dq-cons /Cr (Cqd -Cs-cons +Ca-cons ))×100%
B.任务表级评估量化计算B. Quantitative Calculation of Task Sheet Level Evaluation
任务级评估指标方法是对预先定义的评估任务进行的汇总计算。一个评估任务针对包含一个或多个数据表(或数据表的子表)，且一个数据集可进行多个指标的评估。任务级评估首先按照数据表级评估指标方法进行计算，获得所有数据表(或子表)的参数后，按照任务级评估指标方法再完成针对任务的指标量化。Task-level evaluation metrics methods are aggregate calculations performed on pre-defined evaluation tasks. An evaluation task is aimed at including one or more data tables (or sub-tables of data tables), and a data set can be evaluated with multiple indicators. The task-level evaluation is first calculated according to the data table-level evaluation index method. After obtaining the parameters of all data tables (or sub-tables), the task-specific index quantification is completed according to the task-level evaluation index method.
1)任务表级质量评估结果数据1) Task table-level quality assessment result data
首先，给出以下反映任务级质量评估实际情况的结果数据：First, the following result data reflecting the actual situation of task-level quality assessment are given:
a)问题分类数：是多个数据集的问题分类数之和，记做其中n为数据集的个数。a) Question classification number: It is the sum of the question classification numbers of multiple data sets, recorded as where n is the number of datasets.
b)问题记录数：是多个数据集的问题记录数之和，记做其中n为数据集的个数。b) Number of problem records: It is the sum of the number of problem records in multiple data sets, recorded as where n is the number of datasets.
c)问题数据个数：是多个数据集的问题数据数之和，记做其中n为数据集的个数。c) Number of problem data: It is the sum of the number of problem data in multiple data sets, recorded as where n is the number of datasets.
d)问题数据字段个数：是多个数据集的问题数据字段个数之和，记做d) The number of question data fields: it is the sum of the number of question data fields in multiple data sets, recorded as
e)缺少数据字段个数：是多个数据集的缺少数据字段个数之和，记做e) The number of missing data fields: it is the sum of the number of missing data fields in multiple data sets, recorded as
f)多出数据字段个数：是多个数据集的缺少数据字段个数之和，记做f) The number of extra data fields: it is the sum of the number of missing data fields in multiple data sets, recorded as
g)冗余记录总数：是多个数据集的冗余记录个数之和，记做其中n为数据集的个数。g) The total number of redundant records: it is the sum of the redundant records of multiple data sets, recorded as where n is the number of datasets.
h)实际冗余记录数：是多个数据集的实际冗余记录个数之和，记做h) The number of actual redundant records: it is the sum of the actual redundant records of multiple data sets, recorded as
其中n为数据集的个数where n is the number of data sets
2)任务表级质量评估量化计算方法2) Task table-level quality assessment quantitative calculation method
a)及时性：设为相应数据发生时间，为相应数据形成时间，为相应数据到达时间，业务及时性约束规则定义的对应滞后时间记做传输及时性约束规则定义的对应滞后时间记做业务滞后时间为：ΔTbus＝Tf-Th；业务及时性延迟为：传输滞后时间为：ΔTdata＝Ta-Tf；传输及时性延迟为：其中n为数据集的个数。a) Timeliness: set is the occurrence time of the corresponding data, is the corresponding data formation time, For the corresponding data arrival time, record the corresponding lag time defined by the business timeliness constraint rules as The corresponding lag time defined by the transmission timeliness constraint rule is denoted as The business lag time is: ΔTbus = Tf -Th ; the business timeliness delay is: Transmission lag time is: ΔTdata =Ta -Tf ; transmission timeliness delay is: where n is the number of datasets.
b)完整性：完整性评估指标包括问题数据个数问题记录数冗余记录数实际冗余记录数缺少记录数缺少数据字段个数多出数据字段个数设Cr为一个数据集的记录数，Cqd为一个数据集数据字典定义数据字段个数，Cs为本数据集缺少数据字段个数，Rqc-int为本数据集的完整性问题记录覆盖率，记录完整性为：b) Integrity: Integrity evaluation indicators include the number of problem data number of problem records Number of redundant records Actual number of redundant records number of missing records Number of missing data fields The number of extra data fields Let Cr be the number of records in a data set, Cqd be the number of data fields defined in the data dictionary of a data set, Cs be the number of missing data fields in this data set, and Rqc-int be the integrity problem records of this data set Coverage, record completeness is:
结构完整性为： Structural integrity is:
数据完整性为： Data integrity is:
其中n为数据集的个数。 where n is the number of datasets.
c)准确性：c) Accuracy:
●合规性：合规性的评估指标包括：问题分类数问题数据字段个数问题数据个数问题记录数缺少数据字段个数多出数据字段个数设Cr为一个数据集的记录数，Cqd为本数据集数据字典定义数据字段个数，那么数据准确性为：●Compliance: The evaluation indicators of compliance include: the number of problem classifications Number of question data fields Number of problem data number of problem records Number of missing data fields The number of extra data fields Let Cr be the number of records in a data set, and Cqd be the number of data fields defined in the data dictionary of this data set, then the data accuracy is:
其中n为数据集的个数 where n is the number of data sets
●一致性：一致性评估指标包括问题数据字段数问题分类数问题数据个数问题记录数缺少数据字段个数多出数据字段个数设Cr为一个数据集的记录数，Cqd为本数据集中数据字典定义数据字段个数，Rqc为本数据集一致性问题记录覆盖率，那么数据一致性为：其中n为数据集的个数。● Consistency: Consistency evaluation indicators include the number of question data fields Number of question categories Number of problem data number of problem records Number of missing data fields The number of extra data fields Suppose Cr is the number of records in a data set, Cqd is the number of data fields defined by the data dictionary in this data set, and Rqc is the record coverage rate of consistency problems in this data set, then the data consistency is: where n is the number of datasets.
C.数据质量扩展指标评价方法C. Data quality extended index evaluation method
1)扩展级评估结果数据1) Extended-level evaluation result data
首先，给出以下反映扩展级质量评估实际情况的结果数据：First, the following result data reflecting the actual situation of the extended-level quality assessment are given:
a)问题分类数：是全部数据集的问题分类数之和，记做a) Number of question classifications: It is the sum of the number of question classifications in all datasets, denoted as
b)问题数据字段个数：全部数据集的问题数据字段个数之和，记做b) Number of question data fields: the sum of the number of question data fields in all data sets, recorded as
c)缺少数据字段个数：全部数据集的缺少数据字段个数之和，记做c) The number of missing data fields: the sum of the number of missing data fields in all data sets, recorded as
d)多出数据字段个数：全部数据集的多出数据字段个数之和，记做d) The number of extra data fields: the sum of the number of extra data fields in all data sets, recorded as
e)冗余记录总数：多个数据集的冗余记录个数之和，记做e) The total number of redundant records: the sum of the redundant records of multiple data sets, denoted as
f)实际冗余记录数：多个数据集的实际冗余记录个数之和，记做f) The number of actual redundant records: the sum of the actual redundant records of multiple data sets, recorded as
其中n为数据字典定义的数据表数量。Among them, n is the number of data tables defined by the data dictionary.
2)扩展质量评估计算方法2) Extended quality assessment calculation method
a)数据字典完备性：完备性评估指标包括违背数据字段完备性约束规则的问题分类数问题域个数问题表个数问题数据字段个数设Cfield,Ctable,Cqd,Crela为分别为(一个数据集中)数据字典定义的域数量、表数量、数据字段数量、关系数量，那么域完备性为表完备性为字段完备性为其中n为数据字典定义的数据表数量。a) Completeness of the data dictionary: the completeness evaluation index includes the number of problem categories that violate the completeness constraint rules of the data field Number of problem domains number of questions Number of question data fields Let Cfield , Ctable , Cqd , and Crela be the number of fields, tables, data fields, and relations defined by the data dictionary (in a data set), then the domain completeness is Table completeness is Field completeness is Among them, n is the number of data tables defined by the data dictionary.
b)设计实现一致性：完备性评估指标包括违背设计实现一致性性约束规则的问题分类数问题域个数问题表个数多出数据字段个数设Cfield,Ctable,Cqd,Crela为分别为(一个数据集中)数据字典定义的域数量、表数量、数据字段数量、关系数量，那么域设计实现一致性影响因子为表设计实现一致性影响因子为字段设计实现一致性影响因子为其中n为数据字典定义的数据表数量。b) Consistency of design and implementation: the completeness evaluation index includes the number of problem classifications that violate the constraint rules of design and implementation consistency Number of problem domains number of questions The number of extra data fields Let Cfield , Ctable , Cqd , and Crela be the number of fields, tables, data fields, and relationships defined by the data dictionary (in a data set), then the domain design and implementation consistency impact factor is The impact factor of table design and implementation consistency is Field design and implementation consistency impact factor is Among them, n is the number of data tables defined by the data dictionary.
以上实施例仅用以说明本发明的技术方案而非对其限制，尽管参照上述实施例对本发明进行了详细的说明，所属领域的普通技术人员依然可以对本发明的具体实施方式进行修改或者等同替换，而这些未脱离本发明精神和范围的任何修改或者等同替换，其均在申请待批的本发明的权利要求保护范围之内。The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art can still modify or equivalently replace the specific embodiments of the present invention. , and any modifications or equivalent replacements that do not deviate from the spirit and scope of the present invention are all within the protection scope of the claims of the pending application of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610326404.9ACN105976120A (en) | 2016-05-17 | 2016-05-17 | Electric power operation monitoring data quality assessment system and method |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610326404.9ACN105976120A (en) | 2016-05-17 | 2016-05-17 | Electric power operation monitoring data quality assessment system and method |
| Publication Number | Publication Date |
|---|---|
| CN105976120Atrue CN105976120A (en) | 2016-09-28 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610326404.9APendingCN105976120A (en) | 2016-05-17 | 2016-05-17 | Electric power operation monitoring data quality assessment system and method |
| Country | Link |
|---|---|
| CN (1) | CN105976120A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530121A (en)* | 2016-10-24 | 2017-03-22 | 南方电网科学研究院有限责任公司 | Method and system for detecting safety protection compliance of power monitoring system |
| CN106528828A (en)* | 2016-11-22 | 2017-03-22 | 山东浪潮云服务信息科技有限公司 | Multi-dimensional checking rule-based data quality detection method |
| CN106649840A (en)* | 2016-12-30 | 2017-05-10 | 国网江西省电力公司经济技术研究院 | Method suitable for power data quality assessment and rule check |
| CN107358334A (en)* | 2017-05-25 | 2017-11-17 | 全球能源互联网研究院 | Data accuracy decision method, device, terminal and computer-readable recording medium |
| CN107657544A (en)* | 2017-09-14 | 2018-02-02 | 国网辽宁省电力有限公司 | A kind of automatic paying method of the improved electricity charge and system |
| CN107748775A (en)* | 2017-10-17 | 2018-03-02 | 上海计算机软件技术开发中心 | A kind of data governing system based on the quality of data |
| CN107895003A (en)* | 2017-10-31 | 2018-04-10 | 山东浪潮云服务信息科技有限公司 | A kind of data quality checking method and apparatus |
| CN108197280A (en)* | 2018-01-10 | 2018-06-22 | 上海电气集团股份有限公司 | Mining ability evaluation method based on industrial equipment data |
| CN108229784A (en)* | 2017-11-09 | 2018-06-29 | 中国电力科学研究院有限公司 | The multidimensional data quality evaluating method and system of a kind of intelligent distribution network |
| CN108876146A (en)* | 2018-06-14 | 2018-11-23 | 国网山东省电力公司聊城供电公司 | A kind of electric network data class assets management method and system |
| CN109254959A (en)* | 2018-08-17 | 2019-01-22 | 广东技术师范学院 | A kind of data evaluation method, apparatus, terminal device and readable storage medium storing program for executing |
| CN109783553A (en)* | 2018-11-28 | 2019-05-21 | 中国电力科学研究院有限公司 | A kind of power distribution network mass data increased quality system |
| CN110147966A (en)* | 2019-05-28 | 2019-08-20 | 国网经济技术研究院有限公司 | Enterprise operation data quality management method |
| CN110210719A (en)* | 2019-05-10 | 2019-09-06 | 中国电力科学研究院有限公司 | A kind of power equipment static data method for evaluating quality and system |
| CN110309131A (en)* | 2019-04-12 | 2019-10-08 | 北京星网锐捷网络技术有限公司 | The method for evaluating quality and device of massive structured data |
| CN110347741A (en)* | 2019-07-18 | 2019-10-18 | 普元信息技术股份有限公司 | The system and its control method of the outputting result quality of data are effectively promoted in big data treatment process |
| CN110472109A (en)* | 2019-07-30 | 2019-11-19 | 深圳中科保泰科技有限公司 | Mobilism Data Quality Analysis method and plateform system |
| CN110599056A (en)* | 2019-09-19 | 2019-12-20 | 国网福建省电力有限公司 | Power data quality management method and system |
| CN110611576A (en)* | 2018-06-14 | 2019-12-24 | 亿阳信通股份有限公司 | Data quality monitoring method, device, equipment and storage medium |
| CN111190073A (en)* | 2019-12-31 | 2020-05-22 | 中国电力科学研究院有限公司 | Power grid wide area measurement interaction and search service system |
| CN111257668A (en)* | 2020-01-17 | 2020-06-09 | 国网湖南省电力有限公司 | Data inspection method and system for power supply data system |
| CN111400365A (en)* | 2020-02-26 | 2020-07-10 | 杭州美创科技有限公司 | Business system data quality detection method based on standard SQ L |
| CN111639850A (en)* | 2020-05-27 | 2020-09-08 | 中国电力科学研究院有限公司 | Quality evaluation method and system for multi-source heterogeneous data |
| CN111797081A (en)* | 2020-05-21 | 2020-10-20 | 国网江西省电力有限公司吉安供电分公司 | A system and method for evaluating the quality of power operation monitoring data |
| CN111858646A (en)* | 2020-07-21 | 2020-10-30 | 国网浙江省电力有限公司营销服务中心 | A method and system for checking the quality data format of an electric energy meter |
| CN111897889A (en)* | 2020-08-17 | 2020-11-06 | 国网辽宁省电力有限公司信息通信分公司 | A Data Table Integrity Evaluation Method for Power Industry Business System |
| CN111949644A (en)* | 2020-08-17 | 2020-11-17 | 沈阳京华博瑞科技有限公司 | Data quality evaluation method and equipment for power industry service system |
| CN112488528A (en)* | 2020-12-01 | 2021-03-12 | 东莞中国科学院云计算产业技术创新与育成中心 | Data set processing method, device, equipment and storage medium |
| CN112529677A (en)* | 2020-12-22 | 2021-03-19 | 四川新网银行股份有限公司 | Automatic data quality evaluation method and readable storage medium |
| CN111177134B (en)* | 2019-12-26 | 2021-04-02 | 上海科技发展有限公司 | Data quality analysis method, device, terminal and medium applicable to massive data |
| CN112700157A (en)* | 2021-01-07 | 2021-04-23 | 杭州数梦工场科技有限公司 | Data asset generation method and device and electronic equipment |
| CN113282588A (en)* | 2021-06-11 | 2021-08-20 | 亿景智联(北京)科技有限公司 | Method and device for evaluating quality of spatio-temporal data |
| CN113469559A (en)* | 2021-07-19 | 2021-10-01 | 广东电网有限责任公司 | Quality bit design and display method and system based on data quality inspection |
| CN113886373A (en)* | 2021-09-09 | 2022-01-04 | 浙江大华技术股份有限公司 | Data processing method and device and electronic equipment |
| CN114064618A (en)* | 2020-07-31 | 2022-02-18 | 中国电信股份有限公司 | Data quality evaluation method and system |
| CN114124743A (en)* | 2021-11-16 | 2022-03-01 | 广东电网有限责任公司 | Method and system for executing data application full link check rule |
| CN114490615A (en)* | 2022-02-10 | 2022-05-13 | 红有软件股份有限公司 | Data quality inspection system and method based on EPDM data model |
| CN114610700A (en)* | 2022-03-07 | 2022-06-10 | 中国标准化研究院 | Standardized management and quality assurance methods for industrial data |
| CN115001953A (en)* | 2022-05-30 | 2022-09-02 | 中国第一汽车股份有限公司 | Electric vehicle data quality evaluation method, device, terminal and storage medium |
| CN117421328A (en)* | 2023-11-23 | 2024-01-19 | 上海信投数字科技有限公司 | Data management method, system, equipment and readable medium based on artificial intelligence |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040236723A1 (en)* | 2001-08-30 | 2004-11-25 | Reymond Marc Andre | Method and system for data evaluation, corresponding computer program product, and corresponding computer-readable storage medium |
| CN102708149A (en)* | 2012-04-01 | 2012-10-03 | 河海大学 | Data quality management method and system |
| CN103247008A (en)* | 2013-05-07 | 2013-08-14 | 国家电网公司 | Quality evaluation method of electricity statistical index data |
| CN105389649A (en)* | 2015-10-21 | 2016-03-09 | 中国电力科学研究院 | Electric energy quality abnormal data detection and processing method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040236723A1 (en)* | 2001-08-30 | 2004-11-25 | Reymond Marc Andre | Method and system for data evaluation, corresponding computer program product, and corresponding computer-readable storage medium |
| CN102708149A (en)* | 2012-04-01 | 2012-10-03 | 河海大学 | Data quality management method and system |
| CN103247008A (en)* | 2013-05-07 | 2013-08-14 | 国家电网公司 | Quality evaluation method of electricity statistical index data |
| CN105389649A (en)* | 2015-10-21 | 2016-03-09 | 中国电力科学研究院 | Electric energy quality abnormal data detection and processing method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530121A (en)* | 2016-10-24 | 2017-03-22 | 南方电网科学研究院有限责任公司 | Method and system for detecting safety protection compliance of power monitoring system |
| CN106530121B (en)* | 2016-10-24 | 2020-03-24 | 南方电网科学研究院有限责任公司 | Method and system for detecting safety protection compliance of power monitoring system |
| CN106528828A (en)* | 2016-11-22 | 2017-03-22 | 山东浪潮云服务信息科技有限公司 | Multi-dimensional checking rule-based data quality detection method |
| CN106649840A (en)* | 2016-12-30 | 2017-05-10 | 国网江西省电力公司经济技术研究院 | Method suitable for power data quality assessment and rule check |
| CN107358334A (en)* | 2017-05-25 | 2017-11-17 | 全球能源互联网研究院 | Data accuracy decision method, device, terminal and computer-readable recording medium |
| CN107657544A (en)* | 2017-09-14 | 2018-02-02 | 国网辽宁省电力有限公司 | A kind of automatic paying method of the improved electricity charge and system |
| CN107748775A (en)* | 2017-10-17 | 2018-03-02 | 上海计算机软件技术开发中心 | A kind of data governing system based on the quality of data |
| CN107895003A (en)* | 2017-10-31 | 2018-04-10 | 山东浪潮云服务信息科技有限公司 | A kind of data quality checking method and apparatus |
| CN108229784A (en)* | 2017-11-09 | 2018-06-29 | 中国电力科学研究院有限公司 | The multidimensional data quality evaluating method and system of a kind of intelligent distribution network |
| CN108197280B (en)* | 2018-01-10 | 2022-05-13 | 上海电气集团股份有限公司 | Mining ability evaluation method based on industrial equipment data |
| CN108197280A (en)* | 2018-01-10 | 2018-06-22 | 上海电气集团股份有限公司 | Mining ability evaluation method based on industrial equipment data |
| CN110611576A (en)* | 2018-06-14 | 2019-12-24 | 亿阳信通股份有限公司 | Data quality monitoring method, device, equipment and storage medium |
| CN110611576B (en)* | 2018-06-14 | 2023-05-09 | 亿阳信通股份有限公司 | Data quality monitoring method, device, equipment and storage medium |
| CN108876146A (en)* | 2018-06-14 | 2018-11-23 | 国网山东省电力公司聊城供电公司 | A kind of electric network data class assets management method and system |
| CN109254959A (en)* | 2018-08-17 | 2019-01-22 | 广东技术师范学院 | A kind of data evaluation method, apparatus, terminal device and readable storage medium storing program for executing |
| CN109783553A (en)* | 2018-11-28 | 2019-05-21 | 中国电力科学研究院有限公司 | A kind of power distribution network mass data increased quality system |
| CN110309131A (en)* | 2019-04-12 | 2019-10-08 | 北京星网锐捷网络技术有限公司 | The method for evaluating quality and device of massive structured data |
| CN110210719A (en)* | 2019-05-10 | 2019-09-06 | 中国电力科学研究院有限公司 | A kind of power equipment static data method for evaluating quality and system |
| CN110210719B (en)* | 2019-05-10 | 2024-08-09 | 中国电力科学研究院有限公司 | A method and system for evaluating static data quality of power equipment |
| CN110147966A (en)* | 2019-05-28 | 2019-08-20 | 国网经济技术研究院有限公司 | Enterprise operation data quality management method |
| CN110347741A (en)* | 2019-07-18 | 2019-10-18 | 普元信息技术股份有限公司 | The system and its control method of the outputting result quality of data are effectively promoted in big data treatment process |
| CN110347741B (en)* | 2019-07-18 | 2023-05-05 | 普元信息技术股份有限公司 | System for effectively improving output result data quality in big data processing process and control method thereof |
| CN110472109B (en)* | 2019-07-30 | 2022-06-21 | 深圳中科保泰空天技术有限公司 | Dynamic data quality analysis method and platform system |
| CN110472109A (en)* | 2019-07-30 | 2019-11-19 | 深圳中科保泰科技有限公司 | Mobilism Data Quality Analysis method and plateform system |
| CN110599056A (en)* | 2019-09-19 | 2019-12-20 | 国网福建省电力有限公司 | Power data quality management method and system |
| CN111177134B (en)* | 2019-12-26 | 2021-04-02 | 上海科技发展有限公司 | Data quality analysis method, device, terminal and medium applicable to massive data |
| CN111190073B (en)* | 2019-12-31 | 2024-04-16 | 中国电力科学研究院有限公司 | A power grid wide-area measurement interaction and search service system |
| CN111190073A (en)* | 2019-12-31 | 2020-05-22 | 中国电力科学研究院有限公司 | Power grid wide area measurement interaction and search service system |
| CN111257668A (en)* | 2020-01-17 | 2020-06-09 | 国网湖南省电力有限公司 | Data inspection method and system for power supply data system |
| CN111400365B (en)* | 2020-02-26 | 2023-09-19 | 杭州美创科技股份有限公司 | Service system data quality detection method based on standard SQL |
| CN111400365A (en)* | 2020-02-26 | 2020-07-10 | 杭州美创科技有限公司 | Business system data quality detection method based on standard SQ L |
| CN111797081A (en)* | 2020-05-21 | 2020-10-20 | 国网江西省电力有限公司吉安供电分公司 | A system and method for evaluating the quality of power operation monitoring data |
| CN111639850A (en)* | 2020-05-27 | 2020-09-08 | 中国电力科学研究院有限公司 | Quality evaluation method and system for multi-source heterogeneous data |
| CN111858646A (en)* | 2020-07-21 | 2020-10-30 | 国网浙江省电力有限公司营销服务中心 | A method and system for checking the quality data format of an electric energy meter |
| CN111858646B (en)* | 2020-07-21 | 2023-11-07 | 国网浙江省电力有限公司营销服务中心 | A method and system for checking quality data format of electric energy meter |
| CN114064618A (en)* | 2020-07-31 | 2022-02-18 | 中国电信股份有限公司 | Data quality evaluation method and system |
| CN111897889A (en)* | 2020-08-17 | 2020-11-06 | 国网辽宁省电力有限公司信息通信分公司 | A Data Table Integrity Evaluation Method for Power Industry Business System |
| CN111949644A (en)* | 2020-08-17 | 2020-11-17 | 沈阳京华博瑞科技有限公司 | Data quality evaluation method and equipment for power industry service system |
| CN111949644B (en)* | 2020-08-17 | 2022-04-29 | 沈阳京华博瑞科技有限公司 | Data quality evaluation method and equipment for power industry service system |
| CN111897889B (en)* | 2020-08-17 | 2023-10-20 | 国网辽宁省电力有限公司信息通信分公司 | Data table integrity evaluation method for power industry service system |
| CN112488528A (en)* | 2020-12-01 | 2021-03-12 | 东莞中国科学院云计算产业技术创新与育成中心 | Data set processing method, device, equipment and storage medium |
| CN112529677A (en)* | 2020-12-22 | 2021-03-19 | 四川新网银行股份有限公司 | Automatic data quality evaluation method and readable storage medium |
| CN112700157A (en)* | 2021-01-07 | 2021-04-23 | 杭州数梦工场科技有限公司 | Data asset generation method and device and electronic equipment |
| CN113282588A (en)* | 2021-06-11 | 2021-08-20 | 亿景智联(北京)科技有限公司 | Method and device for evaluating quality of spatio-temporal data |
| CN113469559A (en)* | 2021-07-19 | 2021-10-01 | 广东电网有限责任公司 | Quality bit design and display method and system based on data quality inspection |
| CN113886373A (en)* | 2021-09-09 | 2022-01-04 | 浙江大华技术股份有限公司 | Data processing method and device and electronic equipment |
| CN114124743B (en)* | 2021-11-16 | 2023-10-20 | 广东电网有限责任公司 | Method and system for executing full-link inspection rule of data application |
| CN114124743A (en)* | 2021-11-16 | 2022-03-01 | 广东电网有限责任公司 | Method and system for executing data application full link check rule |
| CN114490615A (en)* | 2022-02-10 | 2022-05-13 | 红有软件股份有限公司 | Data quality inspection system and method based on EPDM data model |
| CN114610700A (en)* | 2022-03-07 | 2022-06-10 | 中国标准化研究院 | Standardized management and quality assurance methods for industrial data |
| CN115001953A (en)* | 2022-05-30 | 2022-09-02 | 中国第一汽车股份有限公司 | Electric vehicle data quality evaluation method, device, terminal and storage medium |
| CN115001953B (en)* | 2022-05-30 | 2023-11-14 | 中国第一汽车股份有限公司 | Electric automobile data quality evaluation method, device, terminal and storage medium |
| CN117421328A (en)* | 2023-11-23 | 2024-01-19 | 上海信投数字科技有限公司 | Data management method, system, equipment and readable medium based on artificial intelligence |
| Publication | Publication Date | Title |
|---|---|---|
| CN105976120A (en) | Electric power operation monitoring data quality assessment system and method | |
| US10120912B2 (en) | System and method for combination-based data analysis | |
| US8600709B2 (en) | Adaptive analytics multidimensional processing system | |
| CN118761745B (en) | OA collaborative workflow optimization method applied to enterprise | |
| CA2708911C (en) | Marketing model determination system | |
| US20080189632A1 (en) | Severity Assessment For Performance Metrics Using Quantitative Model | |
| CN107545349A (en) | A kind of Data Quality Analysis evaluation model towards electric power big data | |
| US10089409B2 (en) | Event-triggered data quality verification | |
| CN108052542B (en) | Multidimensional data analysis method based on presto data | |
| Guo et al. | Research on key problems of data quality in large industrial data environment | |
| US8688499B1 (en) | System and method for generating business process models from mapped time sequenced operational and transaction data | |
| US7992126B2 (en) | Apparatus and method for quantitatively measuring the balance within a balanced scorecard | |
| CN114943424B (en) | A method and system for generating enterprise management index relationship | |
| Zhang et al. | A grey measurement of product complexity | |
| CN117314200A (en) | Scientific and technological achievement evaluation method and system based on enterprise demands | |
| CN117786223A (en) | Audit data analysis method and system based on big data | |
| CN117453805A (en) | Visual analysis method for uncertainty data | |
| CN116166643A (en) | A data quality management method | |
| CN117910934A (en) | Material replaceability assessment method, device, computer equipment and storage medium | |
| Liu et al. | Application of requirement-oriented data quality evaluation method | |
| CN114254871A (en) | A data asset management method based on business data model | |
| Man et al. | A noval data quality controlling and assessing model based on rules | |
| CN113469559A (en) | Quality bit design and display method and system based on data quality inspection | |
| Sandeep et al. | Performance analyzer: An approach for performance analysis of enterprise servers | |
| CN116010399A (en) | Method and apparatus for macro data production and evaluation |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right | Effective date of registration:20170811 Address after:102209 Beijing City, Changping District science and Technology Park in the future smart grid research institute hospital Applicant after:GLOBAL ENERGY INTERCONNECTION RESEARCH INSTITUTE Applicant after:State Grid Shanghai Municipal Electric Power Company Applicant after:State Grid Corporation of China Address before:102209 Beijing City, Changping District science and Technology Park in the future smart grid research institute hospital Applicant before:GLOBAL ENERGY INTERCONNECTION RESEARCH INSTITUTE Applicant before:State Grid Corporation of China | |
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information | Address after:102209 Beijing City, the future of science and Technology City Binhe Road, No. 18, No. Applicant after:GLOBAL ENERGY INTERCONNECTION RESEARCH INSTITUTE Co.,Ltd. Applicant after:STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER Co. Applicant after:STATE GRID CORPORATION OF CHINA Address before:102209 Beijing City, Changping District science and Technology Park in the future smart grid research institute hospital Applicant before:GLOBAL ENERGY INTERCONNECTION RESEARCH INSTITUTE Co.,Ltd. Applicant before:STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER Co. Applicant before:State Grid Corporation of China |