CN111858680B - System and method for rapidly detecting satellite telemetry time sequence data abnormity in real time - Google Patents
System and method for rapidly detecting satellite telemetry time sequence data abnormity in real timeDownload PDFInfo
- Publication number
- CN111858680B CN111858680BCN202010764018.4ACN202010764018ACN111858680BCN 111858680 BCN111858680 BCN 111858680BCN 202010764018 ACN202010764018 ACN 202010764018ACN 111858680 BCN111858680 BCN 111858680B
- Authority
- CN
- China
- Prior art keywords
- data
- abnormal
- statistic
- windows
- current window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2216/00—Indexing scheme relating to additional aspects of information retrieval not explicitly covered by G06F16/00 and subgroups
- G06F2216/03—Data mining
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Radio Relay Systems (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明涉及卫星检测领域和计算机数据挖掘技术领域，特别涉及一种快速实时检测卫星遥测时序数据异常的方法。The invention relates to the field of satellite detection and the technical field of computer data mining, in particular to a method for rapidly and real-timely detecting abnormality of satellite telemetry time series data.
背景技术Background technique
由于卫星运行环境及其复杂，受多种因素影响，卫星在轨运行期间会出现各种异常。及时检测发现数据异常是确保卫星安全健康运行的关键之一。Due to the complexity of the satellite operating environment and the influence of various factors, various anomalies will occur during the satellite's orbital operation. Timely detection of data anomalies is one of the keys to ensuring the safe and healthy operation of satellites.
卫星在轨运行过程中的遥测时序数据流是卫星各个分系统下各部件参数的直接观测量，能够反映各部件的工作状态。发现数据异常变化是进行卫星故障诊断的一个重要步骤。发现卫星遥测时序数据的异常变化，就像给人看病时首先通过X光、彩超等手段发现患病部位一样。卫星在轨运行时不断向地面传输各种监控数据。其中绝大部分时间的数据都是正常数据。所以若用人工来判读所有数据，找出异常变化点，需要花费时间特别长，显然工作效率十分低下。所以只有借助于计算机用数据挖掘的方法才能高效处理海量卫星时序数据，及时、准确地挖掘出卫星异常变化。这是进行卫星故障诊断、故障原因分析以及卫星健康管理的一个首要过程。显然，如果不能快速准确地找出卫星异常变化点，就像医生不能找到病人患病部位一样，根本无法开展后续的故障诊断活动。The telemetry time series data stream during the satellite in-orbit operation is the direct observation of the parameters of each component under each sub-system of the satellite, which can reflect the working status of each component. Finding abnormal data changes is an important step in satellite fault diagnosis. Finding abnormal changes in satellite telemetry time-series data is like finding out the diseased part by means of X-rays and color Doppler ultrasounds when seeing a doctor. When the satellite is in orbit, it continuously transmits various monitoring data to the ground. Most of the time data are normal data. Therefore, it will take a long time to interpret all the data manually and find out the abnormal change points, and obviously the work efficiency is very low. Therefore, only with the help of computer-based data mining methods can the massive satellite time series data be efficiently processed, and the abnormal changes of satellites can be dug out in a timely and accurate manner. This is a primary process for satellite fault diagnosis, fault cause analysis and satellite health management. Obviously, if the abnormal change point of the satellite cannot be found quickly and accurately, just like the doctor cannot find the diseased part of the patient, the follow-up fault diagnosis activities cannot be carried out at all.
异常检测通常有以下几类方法：1.基于模型的技术，首先建立一个数据模型，异常是那些同模型不能完美拟合的对象。例如，数据分布的模型可以通过估计概率分布的参数来创建。如果一个对象不服从该分布，则认为他是一个异常。2.基于邻近度的技术，在对象之间定义邻近性度量，异常对象是那些远离大部分其他对象的对象。当数据能够以二维或者三维散布图呈现时，可以从视觉上检测出基于距离的离群点。3.基于密度的技术，对象的密度估计可以相对直接计算，特别是当对象之间存在邻近性度量。低密度区域中的对象相对远离近邻，可能被看做为异常。以上这些常规的异常检测的方法每次检测时需要全部的数据进行计算，因此很难实时的处理大量的数据。Anomaly detection usually has the following types of methods: 1. Model-based technology, first establish a data model, anomalies are those objects that cannot be perfectly fitted with the model. For example, a model of a data distribution can be created by estimating the parameters of a probability distribution. An object is considered an anomaly if it does not obey the distribution. 2. Proximity-based techniques that define a proximity measure between objects, outlier objects are those objects that are far away from most other objects. Distance-based outliers can be detected visually when the data can be presented in a 2D or 3D scatter plot. 3. Density-based techniques, density estimates for objects can be relatively straightforward to compute, especially when there is a proximity measure between objects. Objects in low-density regions are relatively far from their neighbors and may be seen as anomalies. The above conventional anomaly detection methods require all data to be calculated each time they are detected, so it is difficult to process a large amount of data in real time.
而现有针对实时检测卫星遥测时序数据异常的处理技术中，中国专利CN201510319857.4提出一种在轨卫星推力器温度异常实时诊断的方法，中国专利CN201910749737.6提出一种多探头星铭感器在在轨自主故障诊断与修复方法，中国专利CN201610648303.3，提供了一种基于遥测数据小波变换的卫星异常检测方法，采用基本小波对遥测数据进行小波分解，得到高频分量和低频分量。通过对遥测数据高频小波系数重构的信号进行基于窗口的平稳性分析，并将窗口内数据的均方差作为数据平稳性的评价函数检测卫星的异常。中国专利CN201811144187.7，提供了一种数据驱动的卫星分系统异常预测方法，该方法利用小波分析将滑动窗口尺寸设置为选择属性的第一主周期，并用此窗口划分卫星遥测数据流。使用双向遍历从分段后的窗口数据中挖掘出最小稀有模式，提高挖掘效率。针对挖掘出来的最小稀有模式，计算其异常识别因子，检测卫星遥测数据中的异常，预测卫星分系统中可能出现的异常情况。Among the existing processing technologies for real-time detection of satellite telemetry timing data anomalies, Chinese patent CN201510319857.4 proposes a real-time diagnosis method for abnormal temperature of on-orbit satellite thrusters, and Chinese patent CN201910749737.6 proposes a multi-probe star sensor in the On-orbit autonomous fault diagnosis and repair method, Chinese patent CN201610648303.3, provides a satellite anomaly detection method based on wavelet transform of telemetry data, using basic wavelet to decompose telemetry data to obtain high-frequency components and low-frequency components. Through the window-based stationarity analysis of the signal reconstructed by the high-frequency wavelet coefficients of the telemetry data, the mean square error of the data in the window is used as the evaluation function of the data stationarity to detect satellite anomalies. Chinese patent CN201811144187.7 provides a data-driven satellite subsystem anomaly prediction method, which uses wavelet analysis to set the sliding window size as the first main period of the selected attribute, and uses this window to divide the satellite telemetry data stream. Mining minimum rare patterns from segmented window data using bidirectional traversal to improve mining efficiency. According to the mined rare pattern, its anomaly identification factor is calculated, anomalies in satellite telemetry data are detected, and possible anomalies in satellite subsystems are predicted.
现有方法要么只能针对特定的在轨卫星参数数据进行异常检测，具有很大局限性。要么只从单一的角度分析数据，不具备足够可靠性。Existing methods either can only perform anomaly detection for specific in-orbit satellite parameter data, which has great limitations. Either only analyze the data from a single point of view, which is not reliable enough.
发明内容SUMMARY OF THE INVENTION
为了克服上述现有技术的缺点，解决利用计算机对在轨卫星的遥测时序数据进行实时异常检测问题，本发明的目的在于提供一种快速实时检测卫星遥测时序数据异常的方法，适用于卫星部件参数的遥测时序数据实时检测。In order to overcome the above-mentioned shortcomings of the prior art and solve the problem of using a computer to perform real-time anomaly detection on the telemetry time series data of an in-orbit satellite, the purpose of the present invention is to provide a method for fast real-time detection of satellite telemetry time series data anomalies, which is suitable for satellite component parameters Real-time detection of telemetry time series data.
为了实现上述目的，本发明采用的技术方案是：In order to achieve the above object, the technical scheme adopted in the present invention is:
一种快速实时检测卫星遥测时序数据异常的系统，包括：A system for rapid real-time detection of satellite telemetry time series data anomalies, including:
数据选择模块，获取实时的卫星遥测时序数据流，为接下来的异常检测准备数据；The data selection module obtains the real-time satellite telemetry time series data stream and prepares the data for the next anomaly detection;
模型选择模块，从时域统计量异常检测模型、时域一阶导数异常检测模型、频域相似性异常检测模型和频域统计量异常检测模型中任选一种或者几种组合进行异常检测；The model selection module selects one or several combinations of anomaly detection models for time-domain statistics, first-order derivative anomaly detection models in time-domain, frequency-domain similarity anomaly detection models, and frequency-domain statistics anomaly detection models for anomaly detection;
模型参数设置模块，完成滑动窗口大小和阈值的配置；The model parameter setting module completes the configuration of the sliding window size and threshold;
实时异常检测模块，在完成模型参数设置之后，启动所选择的异常检测模块或异常检测模块组合，载入遥测时序数据流，利用滑动窗口划分遥测时序数据流，自适应比较当前窗口与前k个窗口得到异常分数，其中时域统计量异常检测模型、时域一阶导数异常检测模型和频域统计量异常检测模型以当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数，而频域相似性异常检测模型以当前窗口与前k个窗口分别计算欧式距离作为相似性度量，选取其中最大的距离作为异常分数，如果异常分数超过阈值，则判定卫星部件参数对应的遥测数据异常，定位异常发生的时间，并报告检测到的异常；The real-time anomaly detection module, after completing the model parameter setting, starts the selected anomaly detection module or combination of anomaly detection modules, loads the telemetry time series data stream, uses the sliding window to divide the telemetry time series data stream, and adaptively compares the current window with the first k ones The window gets the anomaly score, in which the time domain statistics anomaly detection model, the time domain first derivative anomaly detection model and the frequency domain statistics anomaly detection model take the difference between each statistic of the current window and the overall corresponding statistic of the previous k windows to account for the top k The percentage of the overall statistics of each window is used as the anomaly score, while the frequency domain similarity anomaly detection model calculates the Euclidean distance between the current window and the first k windows as the similarity measure, and selects the largest distance as the anomaly score. If the anomaly score exceeds the threshold , then determine the abnormality of the telemetry data corresponding to the parameters of the satellite components, locate the time when the abnormality occurs, and report the detected abnormality;
检测结果展示模块，对异常检测报告结果进行存储和解析，图形化展示检测结果。The detection result display module stores and parses the abnormal detection report results, and graphically displays the detection results.
所述数据选择模块完成包括卫星类型、数据实体、数据清洗状态、分系统、部件以及参数的用户设置，获取实时的卫星遥测时序数据流。The data selection module completes user settings including satellite types, data entities, data cleaning status, subsystems, components and parameters, and acquires real-time satellite telemetry time series data streams.
所述时域统计量异常检测模型、时域一阶导数异常检测模型、频域相似性异常检测模型和频域统计量异常检测模型分别从时域和频域两个角度分析数据，利用滑动窗口对卫星遥测时序数据流进行分段处理。The time-domain statistics anomaly detection model, the time-domain first derivative anomaly detection model, the frequency-domain similarity anomaly detection model, and the frequency-domain statistics anomaly detection model analyze data from two perspectives, the time domain and the frequency domain, respectively, and use a sliding window. Segment the satellite telemetry time series data stream.
所述时域统计量异常检测模型进行异常点判定的步骤为：The steps of the time-domain statistic anomaly detection model for determining an anomaly point are:
(1)提取当前窗口内数据的统计量，包括最大值、最小值、均值、方差；(1) Extract the statistics of the data in the current window, including the maximum value, the minimum value, the mean value, and the variance;
(2)提取前k个窗口内数据的总体统计量，取K个窗口统计量中最大值中的最大值，所有最小值中的最小值，所有均值的均值，所有方差的均值；(2) Extract the overall statistics of the data in the first k windows, and take the maximum value among the maximum values in the K window statistics, the minimum value among all the minimum values, the mean value of all means, and the mean value of all variances;
(3)计算当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数；(3) Calculate the percentage of the difference between each statistic of the current window and the total corresponding statistic of the first k windows to the total statistic of the first k windows as the abnormal score;
(4)将异常分数与给定的阈值做比较，如果大于某一统计量的阈值，则当前窗口数据存在异常，由于滑动窗口的步长为1，可判定出当前窗口的异常由最后一个数据导致即该数据为异常点，则报告该异常点对应统计量的异常；(4) Compare the abnormal score with the given threshold. If it is greater than the threshold of a certain statistic, the current window data is abnormal. Since the step size of the sliding window is 1, it can be determined that the abnormality of the current window is determined by the last data. As a result, the data is an abnormal point, and the abnormality of the corresponding statistic of the abnormal point is reported;
所述时域一阶导数异常检测模型进行异常点判定的步骤为：The steps of the time-domain first-order derivative anomaly detection model for determining an anomaly point are:
(1)对当前窗口内数据求一阶导数值；(1) Calculate the first derivative value of the data in the current window;
(2)提取当前窗口内数据的统计量，包括最大值、最小值、均值、方差；(2) Extract the statistics of the data in the current window, including the maximum value, the minimum value, the mean value, and the variance;
(3)提取前k个窗口内数据的总体统计量，取K个窗口统计量中最大值中的最大值，所有最小值中的最小值，所有均值的均值，所有方差的均值；(3) Extract the overall statistics of the data in the first k windows, and take the maximum value among the maximum values in the K window statistics, the minimum value among all the minimum values, the mean value of all means, and the mean value of all variances;
(4)计算当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数；(4) Calculate the percentage of the difference between each statistic of the current window and the overall corresponding statistic of the first k windows to the overall statistic of the first k windows as the abnormal score;
(5)将异常分数与给定的阈值做比较，如果大于某一统计量的阈值，则当前窗口数据存在异常，由于滑动窗口的步长为1，可判定出当前窗口的异常由最后一个数据导致即该数据为异常点，则报告该异常点对应统计量的异常；(5) Compare the abnormal score with the given threshold. If it is greater than the threshold of a certain statistic, the current window data is abnormal. Since the step size of the sliding window is 1, it can be determined that the abnormality of the current window is determined by the last data. As a result, the data is an abnormal point, and the abnormality of the corresponding statistic of the abnormal point is reported;
所述频域统计量异常检测模型进行异常点判定的步骤为：The steps of the frequency domain statistic anomaly detection model for determining an anomaly point are:
(1)对当前窗口内数据做傅里叶变换，进一步计算得到其能量谱；(1) Fourier transform is performed on the data in the current window, and its energy spectrum is obtained by further calculation;
(2)提取当前窗口内数据的统计量，包括能量最大值对应的频率、频率的加权平均、频率的方差；(2) Extract the statistics of the data in the current window, including the frequency corresponding to the maximum energy value, the weighted average of the frequency, and the variance of the frequency;
(3)提取前k个窗口内数据的总体统计量，取K个窗口中能量最大值中的最大值对应的频率、所有频率的加权平均的均值、所有频率的方差的均值；(3) Extract the overall statistics of the data in the first k windows, and take the frequency corresponding to the maximum value of the energy maximum values in the K windows, the mean value of the weighted average of all frequencies, and the mean value of the variance of all frequencies;
(4)计算当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数；(4) Calculate the percentage of the difference between each statistic of the current window and the overall corresponding statistic of the first k windows to the overall statistic of the first k windows as the abnormal score;
(5)将异常分数与给定的相对阈值做比较，如果大于某一统计量的阈值，则当前窗口数据存在异常，由于滑动窗口的步长为1，可判定出当前窗口的异常由最后一个数据导致即该数据为异常点，则报告该异常点对应统计量的异常；(5) Compare the abnormal score with the given relative threshold. If it is greater than the threshold of a certain statistic, the current window data is abnormal. Since the step size of the sliding window is 1, it can be determined that the abnormality of the current window is determined by the last one. If the data causes the data to be an abnormal point, report the abnormality of the statistics corresponding to the abnormal point;
所述频域相似性异常检测模型进行异常点判定的步骤为：The steps of the frequency domain similarity anomaly detection model for determining an anomaly point are:
(1)对当前窗口内数据做傅里叶变换，进一步计算得到其能量谱；(1) Fourier transform is performed on the data in the current window, and its energy spectrum is obtained by further calculation;
(2)当前窗口与前k个窗口分别计算欧式距离作为相似性度量，选取其中最大的距离作为异常分数。(2) Calculate the Euclidean distance between the current window and the first k windows as the similarity measure, and select the largest distance as the anomaly score.
(3)将异常分数与给定的阈值做比较，如果大于该阈值，则当前窗口数据存在异常，由于滑动窗口的步长为1，可判定出当前窗口的异常由最后一个数据导致即该数据为异常点，则报告该异常点频域相似性的异常。(3) Compare the abnormal score with the given threshold. If it is greater than the threshold, the current window data is abnormal. Since the step size of the sliding window is 1, it can be determined that the abnormality of the current window is caused by the last data, that is, the data If it is an abnormal point, report the abnormality of the frequency domain similarity of the abnormal point.
本发明还提供了一种快速实时检测卫星遥测时序数据异常的方法，包括：The present invention also provides a method for fast real-time detection of abnormal satellite telemetry time series data, including:
获取实时的卫星遥测时序数据流，为接下来的异常检测准备数据；Obtain real-time satellite telemetry time series data streams to prepare data for subsequent anomaly detection;
配置滑动窗口大小和阈值；Configure sliding window size and threshold;
采用时域统计量异常检测、时域一阶导数异常检测、频域相似性异常检测和频域统计量异常检测中任选一种或者几种组合的方式，分别从时域和频域两个角度分析数据，进行异常检测，其方法为：载入遥测时序数据流，利用滑动窗口划分遥测时序数据流，自适应比较当前窗口与前k个窗口得到异常分数，其中时域统计量异常检测模型、时域一阶导数异常检测模型和频域统计量异常检测模型以当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数，而频域相似性异常检测模型以当前窗口与前k个窗口分别计算欧式距离作为相似性度量，选取其中最大的距离作为异常分数，根据阈值对遥测时序数据进行异常检测，如果异常分数超过阈值，则判定卫星部件参数对应的遥测数据异常，定位异常发生的时间，并报告检测到的异常；Select one or several combinations of time-domain statistic anomaly detection, time-domain first-order derivative anomaly detection, frequency-domain similarity anomaly detection, and frequency-domain statistic anomaly detection, respectively. Analyze the data from an angle and perform anomaly detection. The method is: load the telemetry time series data stream, divide the telemetry time series data stream with a sliding window, and adaptively compare the current window and the previous k windows to obtain anomaly scores. The time domain statistics anomaly detection model , Time-domain first-order derivative anomaly detection model and frequency-domain statistics anomaly detection model take the difference between each statistic of the current window and the overall corresponding statistic of the previous k windows as the percentage of the overall statistics of the first k windows as the anomaly score, while the frequency The domain similarity anomaly detection model uses the current window and the first k windows to calculate the Euclidean distance as the similarity measure, selects the largest distance as the anomaly score, and performs anomaly detection on the telemetry time series data according to the threshold. If the anomaly score exceeds the threshold, it is judged The abnormal telemetry data corresponding to the parameters of the satellite components, locate the time when the abnormality occurs, and report the detected abnormality;
对异常检测报告结果进行存储和解析，并图形化展示检测结果。Store and parse the results of anomaly detection reports, and graphically display the detection results.
具体地，本发明通过设置卫星类型、数据实体、数据清洗状态、分系统、部件以及参数，获取实时的卫星遥测时序数据流。Specifically, the present invention acquires the real-time satellite telemetry time series data stream by setting the satellite type, data entity, data cleaning state, subsystem, components and parameters.
本发明所得异常的卫星遥测时序数据与相应的卫星异常变化对应，因此可根据异常数据判断卫星异常变化。具体地，卫星在轨运行过程中的遥测时序数据流是卫星各个分系统下各部件参数的直接观测量，能够反映各部件的工作状态。卫星在轨运行时不断向地面传输各部件参数的遥测时序数据流，利用数据挖掘的方法高效的对这些海量的数据进行实时异常检测处理，能够及时、准确地挖掘出卫星异常变化。这是进行卫星故障诊断、故障原因分析以及卫星健康管理的一个首要过程。The abnormal satellite telemetry time series data obtained by the present invention corresponds to the corresponding abnormal satellite changes, so the abnormal changes of the satellites can be judged according to the abnormal data. Specifically, the telemetry time series data stream during the satellite in-orbit operation is the direct observation of the parameters of each component under each sub-system of the satellite, and can reflect the working status of each component. When the satellite is in orbit, it continuously transmits the telemetry time series data stream of the parameters of each component to the ground. The method of data mining is used to efficiently perform real-time abnormal detection and processing on these massive data, which can timely and accurately mine the abnormal changes of the satellite. This is a primary process for satellite fault diagnosis, fault cause analysis and satellite health management.
与现有技术相比，本发明利用滑动窗口对遥测数据流进行分段处理，每个窗口数据与前k个窗口比较，具有自适应性，无需人工设置固定的参数上下限。本发明从时域和频域多重角度分析数据，综合了四种实时异常检测技术，包括时域统计量异常检测、时域一阶导数异常检测、频域相似性异常检测、频域统计量异常检测，可以将多种异常检测技术同时使用或者组合使用，从而降低了漏检率和误报率，能够有效利用遥测时序数据流快速实时检测卫星遥测时序数据异常，帮助专家实时监测卫星运行状态，确保卫星健康安全运行。Compared with the prior art, the present invention uses sliding windows to process the telemetry data stream in segments, and the data of each window is compared with the first k windows, which is adaptive, and does not need to manually set fixed upper and lower limits of parameters. The invention analyzes data from multiple angles of time domain and frequency domain, and integrates four real-time abnormal detection technologies, including time domain statistics abnormal detection, time domain first derivative abnormal detection, frequency domain similarity abnormal detection, and frequency domain statistical abnormal detection. Detection, a variety of abnormal detection technologies can be used at the same time or in combination, thereby reducing the missed detection rate and false alarm rate, and can effectively use the telemetry time series data stream to quickly detect satellite telemetry time series data anomalies in real time, helping experts monitor satellite operation status in real time, Ensure the healthy and safe operation of the satellite.
附图说明Description of drawings
图1是本发明方法的模块框架图。FIG. 1 is a module frame diagram of the method of the present invention.
图2是本发明方法的总体流程图。Figure 2 is a general flow chart of the method of the present invention.
图3是本发明方法使用“南分流调节器温度”和“+Y能源_IN5”的具体参数实例，经过四种实时异常检测方法检测异常点的结果曲线。3 is an example of the specific parameters of the method of the present invention using "south split regulator temperature" and "+Y energy_IN5", and the result curve of detecting abnormal points through four real-time abnormal detection methods.
具体实施方式Detailed ways
下面结合附图和实施例详细说明本发明的实施方式。The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.
如图1所示，本发明一种快速实时检测卫星遥测时序数据异常的系统，利用滑动窗口技术对遥测数据流进行分段处理，结合卫星遥测时序数据固有的特征，进行异常点自动检测，其包括：As shown in Figure 1, a system of the present invention for fast real-time detection of abnormal satellite telemetry time series data, utilizes sliding window technology to carry out segmentation processing on telemetry data stream, and combines the inherent characteristics of satellite telemetry time series data to automatically detect abnormal points. include:
数据选择模块1-1，完成包括卫星类型、数据实体、数据清洗状态、分系统、部件以及参数的设置，为接下来的异常检测准备数据。The data selection module 1-1 completes the settings including satellite type, data entity, data cleaning status, subsystems, components and parameters, and prepares data for the next anomaly detection.
卫星类型，即选择卫星的所属类型，可选的卫星类型有高轨卫星和低轨卫星；数据实体，选择相应卫星类型下的具体卫星实体；数据清洗状态，即选择检测数据清洗的状态，可选的有原始数据和归一化数据；分系统，即选择相应卫星实体下分系统，可选的分系统有控制推进分系统和电源分系统；部件，即选择相应分系统下的部件，控制推进分系统下可选的部件有发动机、推力器等，电源分系统下可选择的部件有北母线、北蓄电池组等；参数，即选择相应部件下的具体参数，例如，部件选择了北电池组，该部件可选参数有北蓄电池组充电电流、北蓄电池组放电电流、北蓄电池组放电容量等。Satellite type, that is, select the type of satellite. The optional satellite types include high-orbit satellites and low-orbit satellites; data entity, select the specific satellite entity under the corresponding satellite type; data cleaning status, that is, select the status of detection data cleaning, you can The original data and normalized data are selected; the sub-system, that is, the sub-system under the corresponding satellite entity is selected, and the optional sub-system includes the control propulsion sub-system and the power sub-system; the component, that is, the component under the corresponding sub-system is selected to control the sub-system. The optional components under the propulsion sub-system include engines, thrusters, etc., and the optional components under the power sub-system include the north busbar, the north battery pack, etc.; parameters, that is, select the specific parameters under the corresponding components, for example, the north battery is selected for the component The optional parameters of this component include the charging current of the battery pack, the discharge current of the battery pack, and the discharge capacity of the battery pack.
模型选择模块1-2，完成异常检测方法的设置，从时域统计量异常检测模型、时域一阶导数异常检测模型、频域相似性异常检测模型和频域统计量异常检测模型中任选一种或者几种组合进行异常检测；这些异常检测模块分别从时域和频域两个不同角度分析数据，是快速实时检测卫星遥测时序数据异常的核心部分。The model selection module 1-2 completes the setting of the anomaly detection method, which can be selected from the anomaly detection model of time domain statistics, the first derivative of time domain anomaly detection model, the similarity anomaly detection model of frequency domain, and the anomaly detection model of frequency domain statistics. One or several combinations are used for anomaly detection; these anomaly detection modules analyze data from two different perspectives, time domain and frequency domain, respectively, and are the core part of rapid real-time detection of satellite telemetry time series data anomalies.
模型参数设置模块1-3，完成滑动窗口大小和阈值的配置。Model parameter setting modules 1-3 complete the configuration of the sliding window size and threshold.
实时异常检测模块1-4，在完成模型参数设置之后，启动所选择的异常检测模块或异常检测模块组合，载入遥测时序数据流，利用滑动窗口划分遥测时序数据流，对遥测时序数据流进行分段处理，当前窗口与前k个窗口进行比较得到异常分数，其中时域统计量异常检测模型、时域一阶导数异常检测模型和频域统计量异常检测模型以当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数，而频域相似性异常检测模型以当前窗口与前k个窗口分别计算欧式距离作为相似性度量，选取其中最大的距离作为异常分数，具有自适应性，无需人工设置固定的参数上下限，具体地，前k个窗口作为一个“基准”与当前窗口提取到的数据特征来比较，而这个“基准”不是固定的，而是根据当前窗口的前k个窗口来实时更新，实现自适应性。进而根据阈值对卫星遥测时序数据进行异常检测，如果异常分数超过阈值，则判定卫星部件参数对应的遥测数据为异常，定位异常发生的时间，并报告检测到的异常。The real-time anomaly detection modules 1-4, after completing the model parameter settings, start the selected anomaly detection module or anomaly detection module combination, load the telemetry time series data stream, divide the telemetry time series data stream by sliding windows, and perform the telemetry time series data stream. Segmentation processing, the current window is compared with the previous k windows to obtain anomaly scores, among which the time domain statistics anomaly detection model, the time domain first derivative anomaly detection model and the frequency domain statistics anomaly detection model are based on the statistics of the current window and the previous ones. The percentage of the difference between the corresponding statistics of the k windows in the total statistics of the first k windows is taken as the anomaly score, and the frequency domain similarity anomaly detection model calculates the Euclidean distance between the current window and the first k windows as the similarity measure, and selects which The maximum distance is used as the anomaly score, which is adaptive and does not need to manually set fixed upper and lower limits of parameters. Specifically, the first k windows are used as a "benchmark" to compare with the data features extracted from the current window, and this "benchmark" is not Fixed, but updated in real time according to the first k windows of the current window to achieve adaptability. Then, anomaly detection is performed on the satellite telemetry time series data according to the threshold. If the anomaly score exceeds the threshold, the telemetry data corresponding to the satellite component parameters is determined to be anomalous, the time when the anomaly occurs is located, and the detected anomaly is reported.
检测结果展示模块1-5，对异常检测报告结果进行持久化存储和解析，通过解析返回的结果，为用户图形化展示检测结果。The detection result display modules 1-5 perform persistent storage and parsing of the abnormal detection report results, and graphically display the detection results for the user by parsing the returned results.
本发明的系统还包括数据分析结果图形试图组件、用户交互组件等必须组件。The system of the present invention also includes necessary components such as a data analysis result graphic attempt component, a user interaction component, and the like.
根据图1和图2，本发明的实时检测方法流程为：According to Fig. 1 and Fig. 2, the real-time detection method flow process of the present invention is:
步骤1，数据选择，获取实时的卫星遥测时序数据流，为接下来的异常检测准备数据；Step 1, data selection, obtain real-time satellite telemetry time series data stream, and prepare data for the next anomaly detection;
步骤2，模型选择，从时域统计量异常检测模型、时域一阶导数异常检测模型、频域相似性异常检测模型和频域统计量异常检测模型中任选一种或者几种组合进行异常检测；Step 2, model selection, select one or several combinations from the time domain statistics anomaly detection model, the time domain first derivative anomaly detection model, the frequency domain similarity anomaly detection model and the frequency domain statistics anomaly detection model to perform anomaly. detection;
步骤3，模型参数，根据上一步选择的模型完成相应模型参数设置，即完成滑动窗口大小和阈值的配置；Step 3, model parameters, complete the corresponding model parameter settings according to the model selected in the previous step, that is, complete the configuration of the sliding window size and threshold;
步骤4，实时异常检测，根据之前步骤的选择，初始化模型参数，流入卫星遥测数据流，执行异常检测方法，进行自动异常点识别；首先初始化模型参数包括滑动窗口大小和阈值，利用滑动窗口技术对遥测时序数据流进行分段处理；比较当前窗口与前k个窗口得到异常分数，如果异常分数超过阈值，则判定卫星部件参数对应的遥测数据为异常，定位异常发生的时间，具体地：Step 4, real-time anomaly detection, according to the selection of the previous steps, initialize the model parameters, flow into the satellite telemetry data stream, execute the anomaly detection method, and perform automatic anomaly point identification; first, initialize the model parameters including the sliding window size and threshold, and use the sliding window technology to detect the abnormal points. The telemetry time series data stream is segmented; compare the current window and the first k windows to obtain an anomaly score, if the anomaly score exceeds the threshold, it is determined that the telemetry data corresponding to the satellite component parameters is abnormal, and the time when the anomaly occurs is located, specifically:
对于时域统计量异常检测，提取当前窗口内数据的统计量，包括最大值、最小值、均值、方差。然后取K个窗口统计量中最大值中的最大值，所有最小值中的最小值，所有均值的均值，所有方差的均值。计算当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数；对于时域一阶导数异常检测，对当前窗口内数据求一阶导数值。提取当前窗口内数据的统计量，包括最大值、最小值、均值、方差。然后提取前k个窗口内数据的总体统计量，取K个窗口统计量中最大值中的最大值，所有最小值中的最小值，所有均值的均值，所有方差的均值。计算当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数；对于频域统计量异常检测，对当前窗口内数据做傅里叶变换，进一步计算得到其能量谱。然后提取当前窗口内数据的统计量，包括能量最大值对应的频率、频率的加权平均、频率的方差。接着提取前k个窗口内数据的总体统计量，取K个窗口中能量最大值中的最大值对应的频率、所有频率的加权平均的均值、所有频率的方差的均值。计算当前窗口各个统计量与前k个窗口总体对应统计量之差占前k个窗口总体统计量的百分比作为异常分数；对于频域相似性异常检测，对当前窗口内数据做傅里叶变换，进一步计算得到其能量谱。当前窗口与前k个窗口分别计算欧式距离作为相似性度量，选取其中最大的距离作为异常分数。得到异常分数以后接着进行异常判定，将异常分数与给定的阈值做比较，如果超过阈值，则当前窗口数据存在异常，由于滑动窗口的步长为1，可判定出当前窗口的异常由最后一个数据导致即该数据为异常点，则报告该异常点，可视化展示结果。如果没有异常，滑动窗口滑动，步长为1，接着检测下一个窗口。For anomaly detection of time-domain statistics, extract the statistics of the data in the current window, including the maximum value, minimum value, mean value, and variance. Then take the maximum of the maximum of the K window statistics, the minimum of all the minimums, the mean of all the means, and the mean of all the variances. Calculate the percentage of the difference between each statistic of the current window and the corresponding statistic of the first k windows to the total statistic of the first k windows as the anomaly score; for the first-order derivative anomaly detection in the time domain, the first-order derivative value is calculated for the data in the current window. . Extract the statistics of the data in the current window, including the maximum, minimum, mean, and variance. Then extract the overall statistics of the data in the first k windows, take the maximum value among the maximum values in the K window statistics, the minimum value among all the minimum values, the mean value of all means, and the mean value of all variances. Calculate the percentage of the difference between each statistic of the current window and the corresponding statistic of the first k windows as a percentage of the total statistic of the first k windows as the anomaly score; The energy spectrum is obtained by further calculation. Then extract the statistics of the data in the current window, including the frequency corresponding to the maximum energy value, the weighted average of the frequency, and the variance of the frequency. Then, the overall statistics of the data in the first k windows are extracted, and the frequency corresponding to the maximum energy maximum value in the K windows, the mean of the weighted average of all frequencies, and the mean of the variance of all frequencies are taken. Calculate the percentage of the difference between each statistic of the current window and the total corresponding statistic of the first k windows to the total statistic of the first k windows as anomaly score; for frequency domain similarity anomaly detection, perform Fourier transform on the data in the current window, The energy spectrum is obtained by further calculation. The Euclidean distance is calculated separately for the current window and the first k windows as the similarity measure, and the largest distance is selected as the anomaly score. After the abnormal score is obtained, the abnormality judgment is performed, and the abnormal score is compared with the given threshold. If it exceeds the threshold, the current window data is abnormal. Since the step size of the sliding window is 1, it can be determined that the abnormality of the current window is determined by the last one. If the data causes the data to be an abnormal point, the abnormal point is reported and the result is displayed visually. If there is no abnormality, the sliding window slides with a step size of 1, and then detects the next window.
步骤5，检测结果展示，通过解析异常检测方法返回的结果，可视化展示检测结果。
下面结合具体数据，展示本发明方法的具体流程。参照图1，首先进行数据选择，选择“南分流调节器温度”和“+Y能源_IN5”两个参数，获得对应的卫星遥测时序数据流；其次进行模型选择，同时选择时域统计量异常检测，时域一阶导数异常检测，频域相似性异常检测和频域统计量异常检测四种实时异常检测模型；第三进行模型参数设置，具体包括窗口大小和阈值设置。窗口大小定义为10，各统计量阈值设置为50，相似性阈值设置为2；然后初始化模型参数，流入卫星遥测数据流，执行异常检测方法进行自动的异常点识别；最后进行检测结果展示，两个参数的异常检测结果参照图3。从图中可以看出本发明方法可有效检测卫星遥测时序数据中的异常点，通过该异常点，即可初步判断卫星的相应异常变化，为进一步作出详尽的故障判断提供基础。The specific process of the method of the present invention is shown below in conjunction with specific data. Referring to Figure 1, first perform data selection, select the two parameters of "south shunt regulator temperature" and "+Y energy_IN5", and obtain the corresponding satellite telemetry time series data stream; secondly, perform model selection, and select time domain statistics abnormality at the same time Detection, time domain first derivative anomaly detection, frequency domain similarity anomaly detection and frequency domain statistics anomaly detection four real-time anomaly detection models; thirdly, model parameter settings, including window size and threshold settings. The window size is defined as 10, the threshold of each statistic is set to 50, and the similarity threshold is set to 2; then initialize the model parameters, flow into the satellite telemetry data stream, and execute the anomaly detection method for automatic outlier identification; finally, the detection results are displayed, two Refer to Figure 3 for the abnormal detection results of each parameter. It can be seen from the figure that the method of the present invention can effectively detect abnormal points in the satellite telemetry time series data, and through the abnormal points, the corresponding abnormal changes of the satellite can be preliminarily judged, which provides a basis for further detailed fault judgment.
综上，本发明提供了一种快速实时检测卫星遥测时序数据异常的方法，适用于卫星部件参数的遥测时序数据。本发明方法利用滑动窗口对遥测数据流进行分段处理，每个窗口数据与前k个窗口比较，具有自适应性，无需人工设置固定的参数上下限。本发明方法从时域和频域多重角度分析数据，综合了四种实时异常检测技术，包括时域统计量异常检测、时域一阶导数异常检测、频域相似性异常检测、频域统计量异常检测。本发明方法可以将多种异常检测技术同时使用或者组合使用，从而降低了漏检率和误报率，能够有效利用遥测时序数据流快速实时检测卫星遥测时序数据异常，帮助专家实时监测卫星运行状态，确保卫星健康安全运行。To sum up, the present invention provides a method for quickly and real-time detection of abnormality of satellite telemetry time series data, which is suitable for telemetry time series data of satellite component parameters. The method of the invention uses the sliding window to process the telemetry data stream in segments, and the data of each window is compared with the first k windows, which has self-adaptation, and does not need to manually set fixed upper and lower limits of parameters. The method of the invention analyzes data from multiple perspectives of time domain and frequency domain, and integrates four real-time abnormal detection technologies, including time domain statistics abnormal detection, time domain first derivative abnormal detection, frequency domain similarity abnormal detection, and frequency domain statistics. abnormal detection. The method of the invention can use a variety of abnormal detection technologies at the same time or in combination, thereby reducing the missed detection rate and the false alarm rate, effectively using the telemetry time series data stream to quickly detect the abnormality of satellite telemetry time series data in real time, and helping experts to monitor the satellite operation state in real time. , to ensure the healthy and safe operation of the satellite.
Claims (6)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010764018.4ACN111858680B (en) | 2020-08-01 | 2020-08-01 | System and method for rapidly detecting satellite telemetry time sequence data abnormity in real time |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010764018.4ACN111858680B (en) | 2020-08-01 | 2020-08-01 | System and method for rapidly detecting satellite telemetry time sequence data abnormity in real time |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111858680A CN111858680A (en) | 2020-10-30 |
| CN111858680Btrue CN111858680B (en) | 2022-10-25 |
Family
ID=72954257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010764018.4AActiveCN111858680B (en) | 2020-08-01 | 2020-08-01 | System and method for rapidly detecting satellite telemetry time sequence data abnormity in real time |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111858680B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112286951B (en)* | 2020-11-26 | 2025-01-10 | 杭州数梦工场科技有限公司 | Data detection method and device |
| CN112948755B (en)* | 2021-04-01 | 2023-06-23 | 中国空空导弹研究院 | Telemetry sine parameter interpretation method |
| CN113852440B (en)* | 2021-09-29 | 2023-12-29 | 应急管理部国家自然灾害防治研究院 | Method and system for precise time calibration of electromagnetic field observation data of Zhangheng first satellite |
| CN115062649A (en)* | 2022-06-02 | 2022-09-16 | 南京凯奥思数据技术有限公司 | Method and system for identifying gearbox vibration signal outliers using spatiotemporal correlations |
| CN116304613A (en)* | 2022-12-26 | 2023-06-23 | 华润数字科技有限公司 | Evaluation method, system, electronic equipment and storage medium for industrial process operation state |
| CN116720062A (en)* | 2023-05-11 | 2023-09-08 | 北京扬铭科技发展有限责任公司 | Intelligent satellite signal monitoring and identifying method and device |
| CN116933028A (en)* | 2023-07-20 | 2023-10-24 | 西安理工大学 | Telemetry data anomaly detection method based on B-spline envelope analysis |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102664006A (en)* | 2012-04-14 | 2012-09-12 | 中国人民解放军国防科学技术大学 | Abnormal voice detecting method based on time-domain and frequency-domain analysis |
| WO2015107268A1 (en)* | 2014-01-16 | 2015-07-23 | Aboa Legis Oy | Method and device for the detection of respiratory rate |
| CN106055919A (en)* | 2016-08-09 | 2016-10-26 | 航天东方红卫星有限公司 | Satellite abnormity detection method based on telemetry data wavelet transformation |
| CN106650297A (en)* | 2017-01-06 | 2017-05-10 | 南京航空航天大学 | Satellite subsystem anomaly detection method without domain knowledge |
| CN108063698A (en)* | 2017-12-15 | 2018-05-22 | 东软集团股份有限公司 | Unit exception detection method and device, program product and storage medium |
| CN110187275A (en)* | 2019-06-06 | 2019-08-30 | 中车株洲电力机车研究所有限公司 | A kind of magneto method for detecting health status and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10317275B2 (en)* | 2016-01-20 | 2019-06-11 | Simmonds Precision Products, Inc. | Vibration monitoring systems |
- 2020
- 2020-08-01CNCN202010764018.4Apatent/CN111858680B/enactiveActive
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102664006A (en)* | 2012-04-14 | 2012-09-12 | 中国人民解放军国防科学技术大学 | Abnormal voice detecting method based on time-domain and frequency-domain analysis |
| WO2015107268A1 (en)* | 2014-01-16 | 2015-07-23 | Aboa Legis Oy | Method and device for the detection of respiratory rate |
| CN106055919A (en)* | 2016-08-09 | 2016-10-26 | 航天东方红卫星有限公司 | Satellite abnormity detection method based on telemetry data wavelet transformation |
| CN106650297A (en)* | 2017-01-06 | 2017-05-10 | 南京航空航天大学 | Satellite subsystem anomaly detection method without domain knowledge |
| CN108063698A (en)* | 2017-12-15 | 2018-05-22 | 东软集团股份有限公司 | Unit exception detection method and device, program product and storage medium |
| CN110187275A (en)* | 2019-06-06 | 2019-08-30 | 中车株洲电力机车研究所有限公司 | A kind of magneto method for detecting health status and system |
Non-Patent Citations (2)
| Title |
|---|
| "Abnormal Data Classification Using Time-Frequency Temporal Logic";Luan Viet Nguyen et al.;《ACM：Proceedings of the 20th International Conference on Hybrid Systems: Computation and Control》;20170413;237-242页* |
| "一种基于时序分析异常数据的跌倒行为监测方法";王忠民 等;《计算机应用研究》;20180331;839-843页* |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111858680A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111858680B (en) | System and method for rapidly detecting satellite telemetry time sequence data abnormity in real time | |
| EP3279819B1 (en) | Method, system and computer device for capacity prediction based on kalman filter | |
| CN108008332B (en) | New energy remote testing equipment fault diagnosis method based on data mining | |
| CN110414154B (en) | Fan component temperature abnormity detection and alarm method with double measuring points | |
| EP2989705B1 (en) | Method and apparatus for defect pre-warning of power device | |
| WO2021179572A1 (en) | Operation and maintenance system anomaly index detection model optimization method and apparatus, and storage medium | |
| EP3105644B1 (en) | Method of identifying anomalies | |
| US12039045B2 (en) | Event analysis in an electric power system | |
| CN106055919B (en) | A kind of satellite method for detecting abnormality based on telemetry wavelet transformation | |
| CN108760302A (en) | A kind of on-line monitoring and fault diagnosis system of wind power generating set bearing | |
| CN105003453A (en) | Online monitoring and fault diagnosis system of mine fan | |
| CN101713983B (en) | Semiconductor process monitoring method based on independent component analysis and Bayesian inference | |
| CN111322082B (en) | A TBM hob condition monitoring and fault diagnosis method and system | |
| CN204113701U (en) | A kind of mine fan on-line monitoring and fault diagnosis system | |
| CN117074852A (en) | Power distribution network electric energy monitoring and early warning management method and system | |
| CN115290286A (en) | Integrated learning evaluation system for flow field stability of sub-span supersonic wind tunnel | |
| CN110469461B (en) | Fracture estimation method and device for fan toothed belt and readable storage medium | |
| CN104713730B (en) | Method for determining degeneration rate of aircraft engine according to vibration signal | |
| CN119939456A (en) | A method for identifying abnormalities in multi-modal data of main transformer operation | |
| Ren et al. | Online anomaly detection using machine learning and HPC for power system synchrophasor measurements | |
| CN108377209A (en) | Equipment fault detecting system based on SCADA and detection method | |
| CN105302476B (en) | A kind of reliability data online acquisition for nuclear power plant equipment analyzes storage system and its storage method | |
| CN113625133A (en) | An online monitoring and feedback system and method for partial discharge of power distribution equipment | |
| JP2023109769A (en) | Learning device, abnormal sign detection device, abnormal sign detection system, learning method, and program | |
| CN115106615B (en) | Welding deviation real-time detection method and system based on intelligent working condition identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |