CN114495497A - Method and system for distinguishing and interpolating traffic abnormal data - Google Patents
Method and system for distinguishing and interpolating traffic abnormal dataDownload PDFInfo
- Publication number
- CN114495497A CN114495497ACN202210059461.0ACN202210059461ACN114495497ACN 114495497 ACN114495497 ACN 114495497ACN 202210059461 ACN202210059461 ACN 202210059461ACN 114495497 ACN114495497 ACN 114495497A
- Authority
- CN
- China
- Prior art keywords
- data
- traffic
- abnormal
- flow
- speed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Chemical & Material Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Analytical Chemistry (AREA)
- Traffic Control Systems (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明属于异常数据处理领域，尤其涉及一种交通异常数据的判别和插补方法及系统。The invention belongs to the field of abnormal data processing, and in particular relates to a method and system for discriminating and interpolating traffic abnormal data.
背景技术Background technique
本部分的陈述仅仅是提供了与本发明相关的背景技术信息，不必然构成在先技术。The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
目前对错误交通数据判别方法多为阈值区间判别法，阈值区间主要基于该路段的相关信息，其中阈值参数分别是速度，流量，占有率三者的有效取值范围。通过交通数据与阈值区间进行比较，从而判断数据是否异常，这种判别方法对错误数据的判别不够全面，有可能出现漏判的情况。填补方法多为求异常数据前后临近数据的加权平均值。这种交通异常数据判别方法较为简单，填补正确率不高，且需要异常数据下一时刻的数据，不能及时填补数据。At present, most of the methods for judging erroneous traffic data are the threshold interval discrimination method. The threshold interval is mainly based on the relevant information of the road section. The threshold parameters are the effective value ranges of speed, flow and occupancy rate. By comparing the traffic data with the threshold interval, it can be judged whether the data is abnormal or not. This judgment method is not comprehensive enough to judge the erroneous data, and there may be a situation of missed judgment. Most of the filling methods are to obtain the weighted average of the adjacent data before and after the abnormal data. This method of discriminating traffic abnormal data is relatively simple, the filling accuracy rate is not high, and the data of the next moment of abnormal data is required, and the data cannot be filled in time.
因此现有方法存在对交通数据漏判的可能性。发现异常交通数据后，存在交通异常数据填补正确率不高、填补不及时等问题。因此，现有的交通异常数据判别和填补方法有一定的局限性，需要进行优化改进。Therefore, the existing methods have the possibility of missing the judgment of the traffic data. After abnormal traffic data is found, there are problems such as low filling accuracy and untimely filling of abnormal traffic data. Therefore, the existing traffic anomaly data discrimination and filling methods have certain limitations and need to be optimized and improved.
发明内容SUMMARY OF THE INVENTION
为了解决上述背景技术中存在的技术问题，本发明提供一种交通异常数据的判别和插补方法及系统，其通过判断相应数据字段空缺状态判断交通数据是否缺失，通过阈值区间以及交通数据逻辑关系判断交通数据是否错误，通过基于时间空间序列的牛顿插值法在线对交通异常数据填补；具有识别精度高、填补正确率高、填补及时的优点。In order to solve the technical problems existing in the above-mentioned background art, the present invention provides a method and system for discriminating and interpolating abnormal traffic data, which judges whether the traffic data is missing by judging the vacancy state of the corresponding data field, and determines whether the traffic data is missing through the threshold interval and the logical relationship of the traffic data. To judge whether the traffic data is wrong or not, the abnormal traffic data is filled online by the Newton interpolation method based on the time-space sequence; it has the advantages of high recognition accuracy, high filling accuracy and timely filling.
为了实现上述目的，本发明采用如下技术方案：In order to achieve the above object, the present invention adopts the following technical solutions:
本发明的第一个方面提供一种交通异常数据的判别和插补方法。A first aspect of the present invention provides a method for discriminating and interpolating traffic abnormal data.
一种交通异常数据的判别和插补方法，包括：A discriminant and imputation method for abnormal traffic data, comprising:
获取一定路段内的交通异常数据；Obtain traffic anomaly data within a certain road section;
确定所述交通异常数据的类型；所述交通异常数据的类型包括缺失数据和错误数据；其中，错误数据的判别过程包括：判断此路段的速度、流量和占有率是否在设定的阈值区间，若否，则为错误数据；判断速度、流量与占有率之间的关系是否符合逻辑，若否，则为错误数据；Determine the type of the abnormal traffic data; the type of the abnormal traffic data includes missing data and wrong data; wherein, the process of judging the wrong data includes: judging whether the speed, flow and occupancy rate of this road section are within the set threshold range, If not, it is wrong data; judge whether the relationship between speed, flow and occupancy rate is logical, if not, it is wrong data;
对异常数据进行填补，得到修正后该路段内的交通数据。The abnormal data is filled to obtain the corrected traffic data in the road section.
本发明的第二个方面提供一种交通异常数据的判别和插补系统。A second aspect of the present invention provides a system for discriminating and interpolating traffic abnormal data.
一种交通异常数据的判别和插补系统，包括：A discriminant and imputation system for abnormal traffic data, including:
数据获取模块，其被配置为：获取一定路段内的交通异常数据；a data acquisition module, which is configured to: acquire abnormal traffic data in a certain road section;
异常数据确定模块，其被配置为：确定所述交通异常数据的类型；所述交通异常数据的类型包括缺失数据和错误数据；其中，错误数据的判别过程包括：判断此路段的速度、流量和占有率是否在设定的阈值区间，若否，则为错误数据；判断速度、流量与占有率之间的关系是否符合逻辑，若否，则为错误数据；An abnormal data determination module, which is configured to: determine the type of the abnormal traffic data; the type of the abnormal traffic data includes missing data and wrong data; wherein, the process of judging the wrong data includes: judging the speed, flow and Whether the occupancy rate is in the set threshold range, if not, it is wrong data; judge whether the relationship between speed, flow and occupancy rate is logical, if not, it is wrong data;
填补数据模块，其被配置为：对异常数据进行填补，得到修正后该路段内的交通数据。The filling data module is configured to: fill in the abnormal data to obtain the corrected traffic data in the road section.
本发明的第三个方面提供一种计算机可读存储介质。A third aspect of the present invention provides a computer-readable storage medium.
一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如上述第一个方面所述的交通异常数据的判别和插补方法中的步骤。A computer-readable storage medium having a computer program stored thereon, the program implements the steps in the method for discriminating and interpolating traffic abnormal data as described in the first aspect above when the program is executed by a processor.
本发明的第四个方面提供一种计算机设备。A fourth aspect of the present invention provides a computer apparatus.
一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如上述第一个方面所述的交通异常数据的判别和插补方法中的步骤。A computer device, comprising a memory, a processor and a computer program stored in the memory and running on the processor, the processor implements the discrimination of traffic abnormality data as described in the first aspect above when the processor executes the program and steps in the imputation method.
与现有技术相比，本发明的有益效果是：Compared with the prior art, the beneficial effects of the present invention are:
本发明提出了一种根据交通数据中的速度、流量、占有率的阈值区间以及三者之间的逻辑关系对错误数据判别的新方法，该方法降低了对交通数据的漏判，提高了异常交通数据的识别率，相比与单一的阈值区间更具有科学性和可靠性。The invention proposes a new method for judging erroneous data according to the threshold interval of speed, flow and occupancy rate in traffic data and the logical relationship between the three, which reduces the missed judgment of traffic data and improves abnormality The recognition rate of traffic data is more scientific and reliable than a single threshold interval.
本发明提出了一种基于时间空间序列的牛顿插补方法，通过计算正常数据附近若干点的时间序列和空间序列牛顿插值，然后比较两种序列牛顿插值的平均绝对误差大小，选择误差较小的序列值重新构建牛顿插值多项式，将交通异常数据时刻带入插值多项式，即可计算出预测值。这种方法比历史数据求均值的方法提高了异常交通数据填补的正确率。The invention proposes a Newton interpolation method based on time-space sequence. By calculating the time sequence and space sequence Newton interpolation of several points near normal data, and then comparing the average absolute error of the two sequences of Newton interpolation, the one with the smaller error is selected. The sequence value reconstructs the Newton interpolation polynomial, and the traffic anomaly data moment is brought into the interpolation polynomial to calculate the predicted value. This method improves the correct rate of abnormal traffic data filling than the method of averaging historical data.
本发明的交通异常数据检测、填补方法能够实时检测异常交通数据，无需异常数据的下一时刻数据，能及时在线对异常数据进行插补。The traffic abnormal data detection and filling method of the present invention can detect abnormal traffic data in real time, does not need the next moment data of abnormal data, and can perform online interpolation of abnormal data in time.
附图说明Description of drawings
构成本发明的一部分的说明书附图用来提供对本发明的进一步理解，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。The accompanying drawings forming a part of the present invention are used to provide further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation of the present invention.
图1是本发明示出的交通异常数据的判别和插补方法的流程图；Fig. 1 is the flow chart of the discrimination and interpolation method of abnormal traffic data shown in the present invention;
图2是本发明示出的缺失数据判别流程图；Fig. 2 is the missing data discrimination flow chart shown in the present invention;
图3是本发明示出的错误数据判别流程图；Fig. 3 is the wrong data discrimination flow chart shown in the present invention;
图4是本发明示出的异常数据插补流程图；Fig. 4 is the abnormal data interpolation flow chart shown in the present invention;
图5是本发明示出的交通数据缺失识别图；Fig. 5 is the traffic data missing identification diagram shown in the present invention;
图6是本发明示出的交通逻辑判断第一大类错误情况的识别图；Fig. 6 is the identification diagram of the traffic logic judgment first category error situation shown in the present invention;
图7是本发明示出的交通逻辑判断第三大类情况被进一步判别为错误数据的识别图；Fig. 7 is the identification diagram of the traffic logic judgment shown in the present invention being further judged as wrong data;
图8是本发明示出的缺失数据的填补图；Fig. 8 is the filling figure of missing data shown in the present invention;
图9是本发明示出的错误数据的填补图。FIG. 9 is a fill-in diagram of erroneous data shown in the present invention.
具体实施方式Detailed ways
下面结合附图与实施例对本发明作进一步说明。The present invention will be further described below with reference to the accompanying drawings and embodiments.
应该指出，以下详细说明都是例示性的，旨在对本发明提供进一步的说明。除非另有指明，本文使用的所有技术和科学术语具有与本发明所属技术领域的普通技术人员通常理解的相同含义。It should be noted that the following detailed description is exemplary and intended to provide further explanation of the invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
需要注意的是，这里所使用的术语仅是为了描述具体实施方式，而非意图限制根据本发明的示例性实施方式。如在这里所使用的，除非上下文另外明确指出，否则单数形式也意图包括复数形式，此外，还应当理解的是，当在本说明书中使用术语“包含”和/或“包括”时，其指明存在特征、步骤、操作、器件、组件和/或它们的组合。It should be noted that the terminology used herein is for the purpose of describing specific embodiments only, and is not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural as well, furthermore, it is to be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates that There are features, steps, operations, devices, components and/or combinations thereof.
需要注意的是，附图中的流程图和框图示出了根据本公开的各种实施例的方法和系统的可能实现的体系架构、功能和操作。应当注意，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，所述模块、程序段、或代码的一部分可以包括一个或多个用于实现各个实施例中所规定的逻辑功能的可执行指令。也应当注意，在有些作为备选的实现中，方框中所标注的功能也可以按照不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，或者它们有时也可以按照相反的顺序执行，这取决于所涉及的功能。同样应当注意的是，流程图和/或框图中的每个方框、以及流程图和/或框图中的方框的组合，可以使用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以使用专用硬件与计算机指令的组合来实现。It is noted that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of the present disclosure. It should be noted that each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which may include one or more components used in implementing various embodiments Executable instructions for the specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented using dedicated hardware-based systems that perform the specified functions or operations , or can be implemented using a combination of dedicated hardware and computer instructions.
实施例一Example 1
如图1所示，本实施例提供了一种交通异常数据的判别和插补方法，本实施例以该方法应用于服务器进行举例说明，可以理解的是，该方法也可以应用于终端，还可以应用于包括终端和服务器和系统，并通过终端和服务器的交互实现。服务器可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务器、云通信、中间件服务、域名服务、安全服务CDN、以及大数据和人工智能平台等基础云计算服务的云服务器。终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表等，但并不局限于此。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接，本申请在此不做限制。本实施例中，该方法包括以下步骤：As shown in FIG. 1 , this embodiment provides a method for discriminating and interpolating traffic abnormal data. This embodiment uses the method applied to the server as an example. It can be understood that the method can also be applied to the terminal. It can be applied to include terminals, servers and systems, and is realized through the interaction of terminals and servers. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud service, cloud database, cloud computing, cloud function, cloud storage, network server, cloud communication, intermediate Cloud servers for basic cloud computing services such as software services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:
获取一定路段内的交通异常数据；Obtain traffic anomaly data within a certain road section;
确定所述交通异常数据的类型；所述交通异常数据的类型包括缺失数据和错误数据；其中，错误数据的判别过程包括：判断此路段的速度、流量和占有率是否在设定的阈值区间，若否，则为错误数据；判断速度、流量与占有率之间的关系是否符合逻辑，若否，则为错误数据；Determine the type of the abnormal traffic data; the type of the abnormal traffic data includes missing data and wrong data; wherein, the process of judging the wrong data includes: judging whether the speed, flow and occupancy rate of this road section are within the set threshold range, If not, it is wrong data; judge whether the relationship between speed, flow and occupancy rate is logical, if not, it is wrong data;
对异常数据进行填补，得到修正后该路段内的交通数据。The abnormal data is filled to obtain the corrected traffic data in the road section.
本实施例提出的方法包括对异常交通数据的识别以及异常数据的填补两个步骤。The method proposed in this embodiment includes two steps of identifying abnormal traffic data and filling in abnormal data.
其中，异常交通数据的判别包括缺失数据的识别和错误数据的识别。Among them, the identification of abnormal traffic data includes the identification of missing data and the identification of wrong data.
1、针对缺失数据的识别：1. Identifying missing data:
缺失数据主要包括三种类型，单个数据缺失，包括速度缺失，流量缺失，占有率缺失。两个数据缺失，包括速度和流量缺失，速度和占有率缺失，流量和占有率缺失。三个数据都缺失。There are mainly three types of missing data. Single data is missing, including speed missing, traffic missing, and occupancy rate missing. Two data are missing, including missing speed and traffic, missing speed and occupancy, and missing traffic and occupancy. All three data are missing.
缺失数据识别原理为通过对速度、流量、占有率这三个字段的数据进行逐条访问判断数据缺失情况。若访问到某一条数据的速度字段缺失，且该条数据对应的其它字段数据不缺失，则为单个数据中的速度缺失，其它单个数据缺失识别原理相同。若访问到某一条数据的速度、流量两个字段缺失，该条数据对应的占有率字段数据不缺失，则为两个数据缺失，其它两个数据缺失识别原理相同。若访问到某一条数据的速度、流量、占有率三个字段数据同时缺失，则为三个数据缺失。缺失数据识别完成后，显示出数据缺失类型以及缺失条数，判别流程如图2所示。The principle of missing data identification is to determine the missing data by accessing the data of the three fields of speed, traffic and occupancy one by one. If the speed field of a certain piece of data is missing, and the data of other fields corresponding to this piece of data are not missing, it means that the speed of a single data is missing, and the identification principle of other single data missing is the same. If the speed and traffic fields of a certain piece of data are missing, and the data in the occupancy field corresponding to the piece of data is not missing, the two data are missing, and the identification principles for the other two missing data are the same. If the speed, traffic, and occupancy rate of a certain piece of data are missing at the same time, then three data are missing. After the identification of missing data is completed, the type of missing data and the number of missing data are displayed, and the identification process is shown in Figure 2.
2、针对错误数据的识别：2. Identification of wrong data:
错误数据主要通过交通数据与阈值区间比较以及交通数据逻辑判断方法进行判别。对交通数据进行识别判断时，不同的道路等级、控制类型所设定阈值范围是不一样的。The error data is mainly judged by comparing the traffic data with the threshold interval and the logical judgment method of the traffic data. When identifying and judging traffic data, the threshold ranges set by different road grades and control types are different.
(1)阈值区间法(1) threshold interval method
交通流量：交通检测器是在一个相对较短的时间内完成交通流量数据的采集，在相对较短的时间内，可能没有车辆经过，所以，交流流量最小值取值可以为零，最大值与交通道路的通行能力和采样间隔有关。每条道路通行能力不同、采集设备的采样间隔也可能不同，因此，每条道路的最大交通流量根据道路历史通行能力、采集间隔结合实际情况赋值，可以根据经验值稍加修正。Traffic flow: The traffic detector completes the collection of traffic flow data in a relatively short period of time. In a relatively short period of time, there may be no vehicles passing by. Therefore, the minimum value of the AC flow can be zero, and the maximum value is equal to The capacity of the traffic road is related to the sampling interval. The traffic capacity of each road is different, and the sampling interval of the collection equipment may also be different. Therefore, the maximum traffic flow of each road is assigned according to the historical traffic capacity of the road, the collection interval and the actual situation, and can be slightly revised according to the empirical value.
速度：车辆的速度检测是在相当短的时间间隔内完成的。结合现实情况，每条道路都会限速，且速度限制各不相同，而道路车辆超速也有可能发生，还可能存在随机误差，所以速度最小值可以为零，最大值可以通过道路限速及经验值修正得到。Speed: The speed detection of the vehicle is done in fairly short time intervals. Combined with the actual situation, each road will have a speed limit, and the speed limit is different, and the road vehicle speeding may also occur, and there may be random errors, so the minimum speed can be zero, and the maximum speed can be determined by the road speed limit and experience value. Correction is obtained.
占有率：这里提出的占有率是指时间占有率，时间占有率是指检测出道路有车辆的时间与交通检测器检测总时间的比值，在交通检测器工作时间内，有可能出现极端情况，道路一直有车辆出现或者道路一直没有车辆出现，故定义时间占有率的最小值为0，最大为100％。Occupancy rate: The occupancy rate proposed here refers to the time occupancy rate, and the time occupancy rate refers to the ratio of the time when there are vehicles on the road detected to the total detection time of the traffic detector. During the working time of the traffic detector, there may be extreme situations. There are always vehicles on the road or there are no vehicles on the road, so the minimum value of the defined time occupancy rate is 0 and the maximum is 100%.
(2)交通逻辑判断(2) Traffic logic judgment
采集到的数据可根据速度、流量与占有率三者之间的逻辑关系进行分类。三者之间的逻辑关系可分为三大类，第一大类：三者中的一个不为零，两个为零；第二大类：三者中速度为零或流量为零，其它两者不为零；第三大类：三者均为零，三者均不为零及占有率为零，速度、流量不为零。第一大类和第二大类不符合逻辑的情况直接视为错误数据。第三大类需要进行进一步判别。The collected data can be classified according to the logical relationship between speed, traffic and occupancy. The logical relationship between the three can be divided into three categories, the first category: one of the three is not zero, and the two are zero; the second category: the speed or flow of the three is zero, and the other The two are not zero; the third category: the three are all zero, the three are not zero and the occupancy rate is zero, and the speed and flow are not zero. Cases that are not logical in the first and second categories are directly treated as erroneous data. The third category requires further discrimination.
第三大类：The third category:
1、交通数据中三者都为零有两种可能性，一种情况是在交通检测器工作时间内流量较少，在这个检测时间段内没有车经过，此时视为正确数据，这种情况一般的特征是前后临近数据都为零或都较小。另一种是因各种原因，数据上传失败导致的数据缺失。判别方法：根据交通流知识，在低流量时，车辆的到达是随机的，服从特定的概率分布。经过概率估算，可以确定在一定采样间隔内经过车辆最小数量。计算出临近采样间隔的流量平均值，若临近采样间隔的平均值大于该最小车辆数，则采样间隔内出现流量为零的可能性接近零，这种情况下的数据视为错误数据。1. There are two possibilities for all three of the traffic data to be zero. One is that there is less traffic during the working hours of the traffic detector, and no vehicles pass by during this detection time period. At this time, it is regarded as correct data. The general characteristic of the situation is that the adjacent data before and after are all zero or small. The other is data missing due to various reasons, data upload failure. Discrimination method: According to the knowledge of traffic flow, the arrival of vehicles is random at low traffic and obeys a specific probability distribution. Through probability estimation, the minimum number of vehicles passing by within a certain sampling interval can be determined. Calculate the traffic average value of the adjacent sampling interval. If the average value of the adjacent sampling interval is greater than the minimum number of vehicles, the possibility of zero traffic in the sampling interval is close to zero, and the data in this case is regarded as wrong data.
2、占有率为零但流量和车速不为零，分为两种情况。第一种情况是当流量较小时，占有率不到百分之一，此时，由于格式显示原因，导致占有率为零而流量和车速不为零，视为正确数据；第二种情况是由于某种原因导致的数据错误。进一步判别：设置一个最小流量限值，占有率只可能在流量小于限值时才会显示为零。通过道路交通流量、车速及占有率的关系，可以计算最小流量限制，如果实际流量大于其值，则认为数据是错误的。为保证正确数据保留，错误数据剔除，最小流量的限值需要设置的比较大。为满足条件，采集区间车辆速度可以取参考路段的限速；平均有效车身长度可以取最小有效车身长度值；占有率取值尽可能小。2. The occupancy rate is zero but the flow and speed are not zero. There are two cases. The first case is that when the traffic is small, the occupancy rate is less than 1%. At this time, due to the format display, the occupancy rate is zero but the traffic and vehicle speed are not zero, which is regarded as correct data; the second case is Data error for some reason. Further judgment: set a minimum flow limit, the occupancy rate will only show zero when the flow is less than the limit. Through the relationship between road traffic flow, vehicle speed and occupancy rate, the minimum flow limit can be calculated. If the actual flow rate is greater than its value, the data is considered to be wrong. In order to ensure correct data retention and erroneous data rejection, the minimum flow limit needs to be set relatively large. In order to meet the conditions, the vehicle speed in the collection interval can take the speed limit of the reference road section; the average effective vehicle length can take the minimum effective vehicle length value; the occupancy rate can take the value as small as possible.
3、三者都不为零，分为两种情况。当出现三者都不为零时，通过相邻车道历史数据或本车道历史数据计算实际的平均有效车身长度，通过道路交通流量、车速及占有率的关系可以得到其平均有效车长。第一种情况是通过三者关系计算出来的值在实际的平均有效车身长度区间之内，认为数据正确，第二种情况是通过三者关系计算出来的值不在实际的平均有效车身长度区间之内，视为错误数据。3, the three are not zero, divided into two cases. When the three are not zero, the actual average effective vehicle length is calculated from the historical data of adjacent lanes or the historical data of the current lane, and the average effective vehicle length can be obtained from the relationship between road traffic flow, vehicle speed and occupancy rate. The first case is that the value calculated by the relationship between the three is within the actual average effective body length range, and the data is considered correct. The second case is that the value calculated by the relationship between the three is not within the actual average effective body length range. is considered as erroneous data.
交通异常数据判别流程如图3所示。The flow of traffic anomaly data discrimination is shown in Figure 3.
2、异常数据的填补2. Filling of abnormal data
(1)对单个数据的缺失，利用速度、流量、占有率三者之间的逻辑公式可以计算缺失值，其中有效平均车长可根据单个缺失数据的上几个时刻或者相邻车道的数据求出，将求出的数据填补即可。(1) For the missing of a single data, the missing value can be calculated by using the logical formula between the speed, traffic and occupancy rate, in which the effective average vehicle length can be calculated according to the last few moments of a single missing data or the data of adjacent lanes It is enough to fill in the obtained data.
(2)对于两个数据或者三个数据缺失以及错误数据的处理，可以通过基于时间空间序列数据的牛顿插值法进行填补。填补步骤如下：(2) For the processing of missing two data or three data and wrong data, it can be filled by Newton interpolation method based on time-space series data. The filling steps are as follows:
1)在异常交通数据的附近选取若干正常的数据节点，利用时间序列和空间序列的n个数据点构建插值函数N(x)，要求构造的插值函数穿过选取的点。1) Select several normal data nodes near the abnormal traffic data, and construct an interpolation function N(x) by using n data points of time series and space series, and the constructed interpolation function is required to pass through the selected points.
2)选取正常数据节点假定为异常交通数据，将基于时间序列的牛顿和基于空间序列的牛顿插值法求出的预测值分别与正常数据求平均绝对百分比误差，然后比较两个平均绝对百分比误差。比较结果较小的一方，插值效果更好，则选用基于该序列的牛顿插值法对异常交通数据进行填补。平均绝对百分比误差的定义为真实值减去预测值除以真实值的绝对值乘以百分之百。2) Select the normal data node to assume abnormal traffic data, calculate the average absolute percentage error between the predicted value based on the Newton of time series and the Newton interpolation method based on space series and the normal data respectively, and then compare the two average absolute percentage errors. If the comparison result is smaller, the interpolation effect is better, and the Newton interpolation method based on this sequence is used to fill the abnormal traffic data. The mean absolute percentage error is defined as the true value minus the predicted value divided by the absolute value of the true value multiplied by 100 percent.
3)如果选用的是基于时间序列的牛顿插值法，利用异常交通数据时刻的前n个数据重新构建拉牛顿插值多项式，将异常交通数据的时刻带入插值多项式，即可计算出预测值；如果选用的是基于空间序列的牛顿插值法，利用异常交通数据同一时刻的相邻车道前n个数据重新构建牛顿插值多项式，将异常交通数据的时刻带入插值多项式，即可计算出预测值。3) If the Newton interpolation method based on time series is selected, the LaNewton interpolation polynomial is reconstructed using the first n data at the time of abnormal traffic data, and the time of abnormal traffic data is brought into the interpolation polynomial to calculate the predicted value; if The Newton interpolation method based on the spatial sequence is selected, and the Newton interpolation polynomial is reconstructed by using the first n data of the adjacent lanes at the same time of the abnormal traffic data, and the time of the abnormal traffic data is brought into the interpolation polynomial to calculate the predicted value.
异常数据插补流程如图4所示。The abnormal data interpolation process is shown in Figure 4.
为了验证本实施例的所提方法的准确性，做如下说明：In order to verify the accuracy of the proposed method of the present embodiment, the following descriptions are made:
图5为利用异常交通数据判别方法检测缺失数据，通过逐条访问速度、流量、占有率三者的字段，通过三者字段缺失情况，判断出数据缺失类型并统计缺失条数，反馈给数据质量检测，从而判断数据质量好坏情况，缺失的交通数据将不绘制在图形中。Figure 5 shows the detection of missing data using the abnormal traffic data discrimination method. By accessing the three fields of speed, traffic and occupancy rate one by one, and through the absence of the three fields, the type of data missing and the number of missing data are determined and fed back to the data quality inspection. , so as to judge the quality of the data, and the missing traffic data will not be drawn in the graph.
图6、图7为异常交通数据判别方法检测交通错误数据，检测出的错误数据在图中用区别于正常数据点的颜色绘制，并累加错误条数，反馈给数据质量检测，从而判断数据质量好坏情况。其中交通逻辑第一大类和第三大类绘制的数据在图中用方框框出，这两种错误情况根据阈值区间法进行判别，将得出相反的结果。交通逻辑第一大类和第三大类数据可能在阈值区间内，但三者之间存在错误的逻辑关系，所以本方法判别交通异常数据更全面，不容易漏判。Figure 6 and Figure 7 are the abnormal traffic data discrimination method to detect traffic error data. The detected error data is drawn in a color different from normal data points in the figure, and the number of errors is accumulated and fed back to the data quality inspection to judge the data quality. Good and bad situations. Among them, the data drawn by the first category and the third category of traffic logic are framed in the figure. These two error conditions are judged according to the threshold interval method, and the opposite results will be obtained. The first and third categories of traffic logic data may be within the threshold range, but there is an erroneous logical relationship between the three, so this method is more comprehensive in judging traffic anomalies and is not easy to miss.
图8、图9分别为缺失数据和错误数据的实时插补，通过上述提到的基于时间空间序列的牛顿插补法计算出预测值。此时，将缺失和错误数据的预测值绘制在图中，图中错误数据点的颜色变为正常数据点的颜色，缺失数据和错误数据条数清零。Figures 8 and 9 show the real-time interpolation of missing data and wrong data, respectively. The predicted value is calculated by the Newton interpolation method based on the time-space sequence mentioned above. At this point, the predicted values of missing and incorrect data are plotted in the graph, the color of the incorrect data points in the graph changes to the color of normal data points, and the number of missing data and incorrect data is cleared to zero.
实施例二Embodiment 2
本实施例提供了一种交通异常数据的判别和插补系统。This embodiment provides a system for discriminating and interpolating traffic abnormal data.
一种交通异常数据的判别和插补系统，包括：A discriminant and imputation system for abnormal traffic data, including:
数据获取模块，其被配置为：获取一定路段内的交通异常数据；a data acquisition module, which is configured to: acquire abnormal traffic data in a certain road section;
异常数据确定模块，其被配置为：确定所述交通异常数据的类型；所述交通异常数据的类型包括缺失数据和错误数据；其中，错误数据的判别过程包括：判断此路段的速度、流量和占有率是否在设定的阈值区间，若否，则为错误数据；判断速度、流量与占有率之间的关系是否符合逻辑，若否，则为错误数据；An abnormal data determination module, which is configured to: determine the type of the abnormal traffic data; the type of the abnormal traffic data includes missing data and wrong data; wherein, the process of judging the wrong data includes: judging the speed, flow and Whether the occupancy rate is in the set threshold range, if not, it is wrong data; judge whether the relationship between speed, flow and occupancy rate is logical, if not, it is wrong data;
填补数据模块，其被配置为：对异常数据进行填补，得到修正后该路段内的交通数据。The filling data module is configured to: fill in the abnormal data to obtain the corrected traffic data in the road section.
此处需要说明的是，上述数据获取模块、异常数据确定模块和填补数据模块与实施例一中的步骤所实现的示例和应用场景相同，但不限于上述实施例一所公开的内容。需要说明的是，上述模块作为系统的一部分可以在诸如一组计算机可执行指令的计算机系统中执行。It should be noted here that the above data acquisition module, abnormal data determination module and filling data module have the same examples and application scenarios as the steps in Embodiment 1, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules may be executed in a computer system such as a set of computer-executable instructions as part of the system.
实施例三Embodiment 3
本实施例提供了一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如上述实施例一所述的交通异常数据的判别和插补方法中的步骤。This embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps in the method for discriminating and interpolating traffic abnormality data as described in the first embodiment above.
实施例四Embodiment 4
本实施例提供了一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如上述实施例一所述的交通异常数据的判别和插补方法中的步骤。This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the traffic described in the first embodiment when the processor executes the program. Steps in Discrimination and Imputation Methods for Outlier Data.
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用硬件实施例、软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(RandomAccessMemory，RAM)等。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. During execution, the processes of the embodiments of the above-mentioned methods may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims (10)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210059461.0ACN114495497B (en) | 2022-01-19 | 2022-01-19 | Method and system for judging and interpolating traffic abnormal data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210059461.0ACN114495497B (en) | 2022-01-19 | 2022-01-19 | Method and system for judging and interpolating traffic abnormal data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114495497Atrue CN114495497A (en) | 2022-05-13 |
| CN114495497B CN114495497B (en) | 2024-01-12 |
Family
ID=81472998
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210059461.0AActiveCN114495497B (en) | 2022-01-19 | 2022-01-19 | Method and system for judging and interpolating traffic abnormal data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114495497B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114757589A (en)* | 2022-06-14 | 2022-07-15 | 深圳市拓安信计控仪表有限公司 | Data processing method, server and storage medium |
| CN115188186A (en)* | 2022-06-28 | 2022-10-14 | 公安部交通管理科学研究所 | A method for monitoring traffic flow in an area |
| CN116168471A (en)* | 2023-02-28 | 2023-05-26 | 广东高标电子科技有限公司 | Method and system for monitoring tire pressure of an electric vehicle |
| CN116311930A (en)* | 2023-03-15 | 2023-06-23 | 河北省交通规划设计研究院有限公司 | Traffic flow state data time gathering method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104240499A (en)* | 2014-06-23 | 2014-12-24 | 银江股份有限公司 | Abnormal congestion point judgment method based on microwave data |
| CN104318772A (en)* | 2014-10-31 | 2015-01-28 | 重庆大学 | Traffic flow data quality detection method for highway |
| CN109360415A (en)* | 2018-09-30 | 2019-02-19 | 北京工业大学 | A method for identifying abnormal data of road traffic flow |
| CN112364910A (en)* | 2020-11-05 | 2021-02-12 | 长安大学 | Expressway toll data abnormal event detection method and device based on peak clustering |
- 2022
- 2022-01-19CNCN202210059461.0Apatent/CN114495497B/enactiveActive
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104240499A (en)* | 2014-06-23 | 2014-12-24 | 银江股份有限公司 | Abnormal congestion point judgment method based on microwave data |
| CN104318772A (en)* | 2014-10-31 | 2015-01-28 | 重庆大学 | Traffic flow data quality detection method for highway |
| CN109360415A (en)* | 2018-09-30 | 2019-02-19 | 北京工业大学 | A method for identifying abnormal data of road traffic flow |
| CN112364910A (en)* | 2020-11-05 | 2021-02-12 | 长安大学 | Expressway toll data abnormal event detection method and device based on peak clustering |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114757589A (en)* | 2022-06-14 | 2022-07-15 | 深圳市拓安信计控仪表有限公司 | Data processing method, server and storage medium |
| CN114757589B (en)* | 2022-06-14 | 2022-10-11 | 深圳市拓安信计控仪表有限公司 | Data processing method, server and storage medium |
| CN115188186A (en)* | 2022-06-28 | 2022-10-14 | 公安部交通管理科学研究所 | A method for monitoring traffic flow in an area |
| CN115188186B (en)* | 2022-06-28 | 2024-02-20 | 公安部交通管理科学研究所 | A method for monitoring traffic flow in a region |
| CN116168471A (en)* | 2023-02-28 | 2023-05-26 | 广东高标电子科技有限公司 | Method and system for monitoring tire pressure of an electric vehicle |
| CN116311930A (en)* | 2023-03-15 | 2023-06-23 | 河北省交通规划设计研究院有限公司 | Traffic flow state data time gathering method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114495497B (en) | 2024-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114495497A (en) | Method and system for distinguishing and interpolating traffic abnormal data | |
| WO2021179574A1 (en) | Root cause localization method, device, computer apparatus, and storage medium | |
| CN112508243B (en) | Training method and device for multi-fault prediction network model of power information system | |
| CN111526063B (en) | Method, device, terminal and storage medium for monitoring breakpoint of link based on whole service | |
| CN113672600B (en) | Abnormality detection method and system | |
| CN112188534B (en) | Abnormality detection method and device | |
| CN116049146B (en) | Database fault processing method, device, equipment and storage medium | |
| CN111176953A (en) | Anomaly detection and model training method thereof, computer equipment and storage medium | |
| CN112883929A (en) | Online video abnormal behavior detection model training and abnormal detection method and system | |
| CN110474862B (en) | A kind of network traffic abnormal detection method and device | |
| CN115906544B (en) | Reliability qualification test method based on reliability growth process | |
| CN111949496A (en) | A data detection method and device | |
| CN116127270A (en) | Abnormal detection method, device, equipment, medium and product | |
| CN113132160A (en) | Method and system for detecting network sub-health state of client node | |
| CN112799911A (en) | A node health state detection method, device, equipment and storage medium | |
| US20220222545A1 (en) | Generation method, non-transitory computer-readable storage medium, and information processing device | |
| CN116258991A (en) | An anomaly detection device, system, method, and computer program product | |
| CN114492531B (en) | Elevator fault detection method, device and computer readable storage medium | |
| CN113112043B (en) | A method, device and apparatus for determining abnormal resource transfer situation | |
| CN117035433B (en) | Illegal funds transfer customer identification method and device | |
| CN118346364A (en) | Abnormal data identification method for mine electrical monitoring system | |
| CN116107859B (en) | Container fault prediction method and device, electronic equipment and storage medium | |
| US20220222579A1 (en) | Deterioration detection method, non-transitory computer-readable storage medium, and information processing device | |
| CN115510998A (en) | Transaction abnormal value detection method and device | |
| US20220222580A1 (en) | Deterioration detection method, non-transitory computer-readable storage medium, and information processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |