CN113515889B - Dynamic wind speed prediction model building method - Google Patents
Dynamic wind speed prediction model building methodDownload PDFInfo
- Publication number
- CN113515889B CN113515889BCN202110557310.3ACN202110557310ACN113515889BCN 113515889 BCN113515889 BCN 113515889BCN 202110557310 ACN202110557310 ACN 202110557310ACN 113515889 BCN113515889 BCN 113515889B
- Authority
- CN
- China
- Prior art keywords
- wind speed
- model
- prediction
- speed prediction
- error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Development Economics (AREA)
- General Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Software Systems (AREA)
- Entrepreneurship & Innovation (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Molecular Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Primary Health Care (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Feedback Control In General (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域Technical Field
本发明涉及风速预测技术领域，特别是涉及一种动态风速预测模型建立方法。The present invention relates to the technical field of wind speed prediction, and in particular to a method for establishing a dynamic wind speed prediction model.
背景技术Background Art
近年来，能源结构朝低碳式方向不断发展，以风电为代表的可再生能源电网渗透率逐年攀升。随着以风力发电为代表的可再生能源大规模并网，电网调度经济性逐渐提高，但由于风速具有波动性、间接性及低能量密度等特点，因而严重降低了电力系统运行的可靠性。因此，为了更好地利用风力发电，同时兼顾电力系统的稳定性，需要对风速进行短期精准预测，但是应建立怎样的预测模型，以及如何对预测模型进行优化来提高预测精度和泛化能力，目前还尚未给出明确定义。In recent years, the energy structure has been developing in a low-carbon direction, and the penetration rate of renewable energy grids represented by wind power has been rising year by year. With the large-scale grid connection of renewable energy represented by wind power, the economic efficiency of grid dispatch has gradually improved, but due to the volatility, indirectness and low energy density of wind speed, the reliability of power system operation has been seriously reduced. Therefore, in order to better utilize wind power generation while taking into account the stability of the power system, it is necessary to make short-term accurate predictions of wind speed. However, what kind of prediction model should be established and how to optimize the prediction model to improve the prediction accuracy and generalization ability have not yet been clearly defined.
发明内容Summary of the invention
本发明的目的是提供一种动态风速预测模型建立方法，采用两次Q强化学习算法构建了动态风速预测模型，具有泛化能力强、鲁棒性好、预测精度高的特点。The purpose of the present invention is to provide a method for establishing a dynamic wind speed prediction model. A twice Q reinforcement learning algorithm is used to construct a dynamic wind speed prediction model, which has the characteristics of strong generalization ability, good robustness and high prediction accuracy.
为实现上述目的，本发明提供了如下方案：To achieve the above object, the present invention provides the following solutions:
一种动态风速预测模型建立方法，包括以下步骤：A method for establishing a dynamic wind speed prediction model comprises the following steps:
S1)获取目标区域的实测风速数据，并对实测风速数据进行预处理；S1) obtaining the measured wind speed data of the target area and preprocessing the measured wind speed data;
S2)将预处理后的实测风速数据分为风速训练集、风速测试集和风速检验集，利用多种预测算法对风速训练集进行训练，并对风速测试集进行预测，得到多种风速预测模型，构成Q学习模型集；S2) dividing the pre-processed measured wind speed data into a wind speed training set, a wind speed test set and a wind speed inspection set, training the wind speed training set using a variety of prediction algorithms, and predicting the wind speed test set to obtain a variety of wind speed prediction models to form a Q learning model set;
S3)在Q学习模型集中添加风速波动情况和属性因素，通过Q强化学习算法选出每时段的最佳风速预测模型，得到初步的风速预测数据，并根据初步的风速预测数据以及对应的实测风速数据计算出风速预测误差；S3) adding wind speed fluctuation and attribute factors to the Q learning model set, selecting the best wind speed prediction model for each time period through the Q reinforcement learning algorithm, obtaining preliminary wind speed prediction data, and calculating the wind speed prediction error based on the preliminary wind speed prediction data and the corresponding measured wind speed data;
S4)基于风速预测误差构建误差Q学习模型库，通过Q强化学习算法在误差Q学习模型库中选出最佳风速预测误差模型以修正初步的风速预测值，得到最终的风速预测数据。S4) constructing an error Q learning model library based on the wind speed prediction error, and selecting the best wind speed prediction error model in the error Q learning model library through the Q reinforcement learning algorithm to correct the preliminary wind speed prediction value to obtain the final wind speed prediction data.
可选的，步骤S1)中所述对实测风速数据进行预处理是指采用相邻数据互补法替换实测风速数据中缺失及异常的数值。Optionally, the preprocessing of the measured wind speed data in step S1) refers to replacing missing and abnormal values in the measured wind speed data with an adjacent data complementation method.
可选的，所述方法在步骤S4)之后还包括：Optionally, the method further comprises, after step S4):
S5)利用所述风速检验集验证所述最佳风速预测误差模型的有效性。S5) Using the wind speed test set to verify the effectiveness of the optimal wind speed prediction error model.
可选的，验证所述最佳风速预测误差模型的有效性选取了均方误差ε1、相对误差ε2和决定系数R2三种评价指标对最终的风速预测数据进行评价，计算公式分别如下：Optionally, to verify the effectiveness of the optimal wind speed prediction error model, three evaluation indicators, namely, mean square error ε1 , relative error ε2 and determination coefficient R2, are selected to evaluate the final wind speed prediction data, and the calculation formulas are as follows:
其中：xt、yt、
可选的，步骤S2)中所述Q学习模型集中采用的多种预测算法为5种，包括LSTM、XGBoost、SVR、BP神经网络和KRR的学习算法。Optionally, the multiple prediction algorithms used in the Q learning model in step S2) are 5, including LSTM, XGBoost, SVR, BP neural network and KRR learning algorithms.
可选的，步骤S4)中所述误差Q学习模型库中采用的多种预测算法为5种，包括SVR、BP神经网络、GKRR、PKRR和MHKRR的学习算法。Optionally, the multiple prediction algorithms used in the error Q learning model library in step S4) are 5, including learning algorithms of SVR, BP neural network, GKRR, PKRR and MHKRR.
可选的，步骤S3)中和步骤S4)中所述风速预测误差的计算公式如下：Optionally, the calculation formulas for the wind speed prediction error in step S3) and step S4) are as follows:
式中：
可选的，步骤S4)中所述最终的风速预测数据的计算公式如下：Optionally, the calculation formula for the final wind speed prediction data in step S4) is as follows:
式中：y为最终的风速预测值，
可选的，步骤S3)中和步骤S4)中所述Q强化学习算法采用误差和模型排名混合的奖励函数，奖励函数的计算公式如下：Optionally, the Q reinforcement learning algorithm in step S3) and step S4) adopts a reward function that is a mixture of error and model ranking, and the calculation formula of the reward function is as follows:
式中：R(s,a)为奖励函数；S为状态空间，S＝{s1,…,sI,…,sN}，sI为当前风速预测模型，N为风速预测模型的数量；A为动作空间，A＝{a1,…,aJ,…,aN}，aJ为在下一个预测时间步长从当前风速预测模型切换到下一风速预测模型的动作；RANK(MI,t)和RANK(MI,t+1)分别为第t时刻、第t+1时刻风速预测模型MI的排名；TIME(MI,t)和TIME(MI,t+1)为第t时刻、第t+1时刻风速预测模型MI的计算时间；α、β为权重系数，且满足α+β＝1。In the formula: R(s,a) is the reward function; S is the state space, S = {s1 ,…,sI ,…,sN }, sI is the current wind speed prediction model, and N is the number of wind speed prediction models; A is the action space, A = {a1 ,…,aJ ,…,aN }, aJ is the action of switching from the current wind speed prediction model to the next wind speed prediction model at the next prediction time step; RANK(MI,t ) and RANK(MI,t+1 ) are the rankings of the wind speed prediction modelMI at the tth time and the t+1th time respectively; TIME(MI,t ) and TIME(MI,t+1 ) are the calculation times of the wind speed prediction modelMI at the tth time and the t+1th time; α and β are weight coefficients, and satisfy α+β＝1.
根据本发明提供的具体实施例，本发明公开了以下技术效果：本发明提供的动态风速预测模型建立方法，采用两次Q强化学习算法构建了动态风速预测模型，其中一个Q学习代理负责选出最佳风速预测模型进行初步的风速预测，另一个Q学习代理负责通过计算误差将其输入到误差校正部分，从中选出最佳风速预测误差模型，得到最优预测策略；并且Q学习在风速预测部分和误差修正部分都有效选择了最佳预测模型；本发明的误差校正使预测平均相对误差减少了50％，误差校正环节对成熟预测模型具有有效性；本发明通过构建风速预测模型对不同季节的典型月进行了预测，结果表明其泛化能力强、鲁棒性好、预测精度高，解决了由于风速波动性、间接性及低能量密度等特点导致的电力系统运行可靠性降低问题，可显著提高含可再生能源并网的电网调度经济性和风电场的运行安全性。According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the method for establishing a dynamic wind speed prediction model provided by the present invention adopts a two-time Q reinforcement learning algorithm to construct a dynamic wind speed prediction model, wherein one Q learning agent is responsible for selecting the best wind speed prediction model for preliminary wind speed prediction, and the other Q learning agent is responsible for inputting the calculated error into the error correction part, and selecting the best wind speed prediction error model therefrom to obtain the optimal prediction strategy; and Q learning effectively selects the best prediction model in both the wind speed prediction part and the error correction part; the error correction of the present invention reduces the average relative error of the prediction by 50%, and the error correction link is effective for mature prediction models; the present invention predicts typical months of different seasons by constructing a wind speed prediction model, and the results show that it has strong generalization ability, good robustness and high prediction accuracy, and solves the problem of reduced reliability of power system operation caused by the characteristics of wind speed volatility, indirectness and low energy density, and can significantly improve the dispatch economy of power grids with renewable energy connected to the grid and the operation safety of wind farms.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
图1为本发明实施例动态风速预测模型建立方法的流程图；FIG1 is a flow chart of a method for establishing a dynamic wind speed prediction model according to an embodiment of the present invention;
图2为本发明实施例动态风速预测模型建立方法的原理图；FIG2 is a schematic diagram of a method for establishing a dynamic wind speed prediction model according to an embodiment of the present invention;
图3为本发明实施例奖励函数的收敛图；FIG3 is a convergence diagram of a reward function according to an embodiment of the present invention;
图4为本发明实施例东北某实际风场2019年风速波动图；FIG4 is a wind speed fluctuation diagram of an actual wind farm in Northeast China in 2019 according to an embodiment of the present invention;
图5a为本发明实施例春季典型月(3月份)QWSP、LSTM、BP神经网络风速预测数据图；FIG5 a is a graph showing wind speed prediction data of QWSP, LSTM and BP neural networks in a typical spring month (March) according to an embodiment of the present invention;
图5b为本发明实施例夏季典型月(6月份)QWSP、LSTM、BP神经网络风速预测数据图；FIG5 b is a graph showing wind speed prediction data of QWSP, LSTM and BP neural networks in a typical summer month (June) according to an embodiment of the present invention;
图5c为本发明实施例秋季典型月(9月份)QWSP、LSTM、BP神经网络风速预测数据图；FIG5 c is a diagram of wind speed prediction data of QWSP, LSTM, and BP neural networks in a typical autumn month (September) according to an embodiment of the present invention;
图5d为本发明实施例冬季典型月(12月份)QWSP、LSTM、BP神经网络风速预测数据图；FIG5 d is a diagram of wind speed prediction data of QWSP, LSTM, and BP neural networks in a typical winter month (December) according to an embodiment of the present invention;
图6a为本发明实施例春季典型月(3月份)DPDQ风速预测数据图；FIG6 a is a DPDQ wind speed prediction data diagram for a typical spring month (March) according to an embodiment of the present invention;
图6b为本发明实施例夏季典型月(6月份)DPDQ风速预测数据图；FIG6 b is a DPDQ wind speed prediction data diagram for a typical summer month (June) according to an embodiment of the present invention;
图6c为本发明实施例秋季典型月(9月份)DPDQ风速预测数据图；FIG6 c is a DPDQ wind speed prediction data diagram for a typical autumn month (September) according to an embodiment of the present invention;
图6d为本发明实施例冬季典型月(12月份)DPDQ风速预测数据图。FIG. 6 d is a graph showing DPDQ wind speed prediction data for a typical winter month (December) according to an embodiment of the present invention.
具体实施方式DETAILED DESCRIPTION
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
本发明的目的是提供一种动态风速预测模型建立方法，采用两次Q强化学习算法构建了动态风速的预测模型，具有泛化能力强、鲁棒性好、预测精度高的特点。The purpose of the present invention is to provide a method for establishing a dynamic wind speed prediction model. A twice Q reinforcement learning algorithm is used to construct a dynamic wind speed prediction model, which has the characteristics of strong generalization ability, good robustness and high prediction accuracy.
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本发明作进一步详细的说明。In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
如图1至图2所示，本发明实施例提供的动态风速预测模型建立方法，包括以下步骤：As shown in FIG. 1 and FIG. 2 , the method for establishing a dynamic wind speed prediction model provided by an embodiment of the present invention includes the following steps:
S1)获取目标区域的实测风速数据，并对实测风速数据进行预处理；S1) obtaining the measured wind speed data of the target area and preprocessing the measured wind speed data;
S2)将预处理后的实测风速数据分为风速训练集、风速测试集和风速检验集，利用多种预测算法对风速训练集进行训练，并对风速测试集进行预测，得到多种风速预测模型，构成Q学习模型集；S2) dividing the pre-processed measured wind speed data into a wind speed training set, a wind speed test set and a wind speed inspection set, training the wind speed training set using a variety of prediction algorithms, and predicting the wind speed test set to obtain a variety of wind speed prediction models to form a Q learning model set;
S3)在Q学习模型集中添加风速波动情况和属性因素(风速、温度、湿度、风向和湍流速度)，通过Q强化学习算法选出每时段的最佳风速预测模型，得到初步的风速预测数据，并根据初步的风速预测数据以及对应的实测风速数据计算出风速预测误差；S3) adding wind speed fluctuations and attribute factors (wind speed, temperature, humidity, wind direction and turbulence speed) to the Q learning model set, selecting the best wind speed prediction model for each time period through the Q reinforcement learning algorithm, obtaining preliminary wind speed prediction data, and calculating the wind speed prediction error based on the preliminary wind speed prediction data and the corresponding measured wind speed data;
S4)基于风速预测误差构建误差Q学习模型库，通过Q强化学习算法在误差Q学习模型库中选出最佳风速预测误差模型以修正初步的风速预测值，对误差进行校正，得到最终的风速预测数据。S4) constructing an error Q learning model library based on the wind speed prediction error, selecting the best wind speed prediction error model in the error Q learning model library through the Q reinforcement learning algorithm to correct the preliminary wind speed prediction value, correcting the error, and obtaining the final wind speed prediction data.
步骤S2)中所述Q学习模型集中采用的多种预测算法为5种，考虑到风速序列的高变异性，深度学习可对实测风速数据进行深层特征挖掘，在风速波动剧烈时能更好的预测风速变换趋势，但在细节上可能存在过拟合的现象，而SVR和BP神经网络往往在风速波动较平缓时段，拥有更高的预测精度，在风速波动剧烈时具有较大的预测误差，故选取深度学习算法LSTM、集成学习算法XGBoost及浅层学习算法SVR、BP神经网络、KRR五种算法作为风速预测模型集中的基础模型，以期针对不同波动情况，Q学习能在其中选取到更适合的预测模型，其中KRR选择基于多项式核函数的PKRR。There are five kinds of prediction algorithms used in the Q learning model set described in step S2). Considering the high variability of wind speed sequences, deep learning can perform deep feature mining on the measured wind speed data, and can better predict the wind speed change trend when the wind speed fluctuates violently, but there may be overfitting in the details. SVR and BP neural networks often have higher prediction accuracy during periods of relatively gentle wind speed fluctuations, and have larger prediction errors when the wind speed fluctuates violently. Therefore, five algorithms, including deep learning algorithm LSTM, ensemble learning algorithm XGBoost, shallow learning algorithm SVR, BP neural network, and KRR, are selected as the basic models in the wind speed prediction model set, so that Q learning can select a more suitable prediction model for different fluctuation conditions, among which KRR selects PKRR based on polynomial kernel function.
Q强化学习算法的原理简述如下：The principle of the Q reinforcement learning algorithm is briefly described as follows:
为了训练Q学习代表，首先在马尔可夫决策过程中定义了基于强化学习的动态模型选择的数学框架，通常Q学习代理根据状态-作用值矩阵(Q矩阵)在一系列状态下采取顺序操作，直到达到最终目标，通过评估当前状态空间的预测效果得到奖励更新Q矩阵，状态空间S由当前预测模型组成：To train a Q-learning agent, we first define a mathematical framework for dynamic model selection based on reinforcement learning in a Markov decision process. Typically, a Q-learning agent takes sequential actions in a series of states according to a state-action value matrix (Q matrix) until the final goal is reached. The Q matrix is updated by evaluating the prediction effect of the current state space. The state space S consists of the current prediction model:
S＝{s1,…,sI,…,sN}S＝{s1 ,…,sI ,…,sN }
式中：sI表示当前风速预测模型；N为风速预测模型的数量。同样，动作空间A由下一步骤的风速预测模型组成：Where: sI represents the current wind speed prediction model; N is the number of wind speed prediction models. Similarly, the action space A consists of the wind speed prediction models of the next step:
A＝{a1,…,aJ,…,aN}A＝{a1 ,…,aJ ,…,aN }
其中aJ表示在下一个预测时间步长从当前风速预测模型切换到下一风速预测模型的动作。为了使用Q学习成功地解决马尔可夫决策过程，最核心的部分是通过适当的奖励函数R(s,a)得到奖励矩阵R，本发明实施例定义误差和模型排名混合奖励函数如下：Where aJ represents the action of switching from the current wind speed prediction model to the next wind speed prediction model at the next prediction time step. In order to successfully solve the Markov decision process using Q learning, the most important part is to obtain the reward matrix R through an appropriate reward function R(s,a). The embodiment of the present invention defines the error and model ranking mixed reward function as follows:
Rt(sI,aJ)＝α[RANK(MI,t)-RANK(MJ,t+1)]+β[TIME(MI,t)-TIME(MJ,t+1)]Rt (sI ,aJ )=α[RANK(MI,t )-RANK(MJ,t+1 )]+β[TIME(MI,t )-TIME(MJ,t+1 ) ]
式中：RANK(MI,t)和RANK(MI,t+1)分别为第t时刻、第t+1时刻风速预测模型MI的排名；TIME(MI,t)和TIME(MI,t+1)分别为第t时刻、第t+1时刻风速预测模型MI的计算时间；α、β为权重系数，且满足α+β＝1。由于Q学习为无模型的动态模型选择框架，往往会择优选取模型进行预测，故当两模型均排名为1时，该项为0，则失去奖惩作用。故选择加权两个Q学习框架，使奖励函数更具普适性。定义状态空间、行动空间和奖励函数后，通过使用Q学习训练数据集Tt训练Q学习动态预测模型。Where: RANK(MI,t ) and RANK(MI,t+1 ) are the rankings of the wind speed prediction modelMI at the tth moment and the t+1th moment respectively; TIME(MI,t ) and TIME(MI,t+1 ) are the calculation times of the wind speed prediction modelMI at the tth moment and the t+1th moment respectively; α and β are weight coefficients, and α+β＝1 is satisfied. Since Q learning is a model-free dynamic model selection framework, it often selects the best model for prediction. Therefore, when both models are ranked 1, this item is 0, and the reward and punishment function is lost. Therefore, the weighted two Q learning frameworks are selected to make the reward function more universal. After defining the state space, action space and reward function, the Q learning dynamic prediction model is trained by using the Q learning training data setTt .
采用衰变t贪婪方法的Q学习代理从一开始就采取完全随机的动作，同时在学习过程中通过衰减来降低随机性，在Ne次训练之后，Q学习算法最终将收敛到最优策略Q*，该策略用于在Q学习过程中找到最优动作a*。具体步骤如下：The Q-learning agent using the decaying t-greedy method takes completely random actions from the beginning, and reduces the randomness through decay during the learning process. AfterNe training times, the Q-learning algorithm will eventually converge to the optimal strategy Q*, which is used to find the optimal action a* during the Q-learning process. The specific steps are as follows:
(1)定义模型步长k，预测尺度n，模型库尺寸NM，Q学习数据集Tt，动态预测模型数据集Tc，控制学习的积极性的学习率κ，权衡未来回报的折现因子γ，训练次数Ne，确保在Tc的每个步骤中，从N个模型中选择最佳模型；(1) Define the model step size k, prediction scale n, model library size NM , Q learning dataset Tt , dynamic prediction model dataset Tc , learning rate κ to control the learning enthusiasm, discount factor γ to weigh future returns, and number of training timesNe to ensure that the best model is selected from N models in each step of Tc ;
(2)初始化Q(s,a)，ω＝1，开始训练；以ω的概率选择随机动作ae，否则选择(2) Initialize Q(s,a), ω = 1, and start training; select a random action ae with probability ω, otherwise select
(3)根据奖励函数计算公式计算更新奖励矩阵R；(3) Calculate and update the reward matrix R according to the reward function calculation formula;
(4)通过下式更新Q(s,a)：(4) Update Q(s,a) by the following formula:
(5)重复(2)～(4)k次，找到每次的最优动作
步骤S3)中和步骤S4)中所述风速预测误差的计算公式如下：The calculation formula for the wind speed prediction error in step S3) and step S4) is as follows:
式中：
步骤S4)中所述误差Q学习模型库中采用的多种预测算法为5种；对于误差校正模型集(即风速预测误差模型构成的误差Q学习模型库)的选取，由于预测误差的波动性和变异性远没有实测风速序列剧烈，更多的是需要对误差序列进行细节上的预测，因此，本发明实施例选取效率较高的SVR、BP神经网络、GKRR、PKRR、MHKRR五种模型构成误差校正模型集，其中GKRR、PKRR和MHKRR模型为采用不同核函数的KRR模型。There are five kinds of prediction algorithms used in the error Q learning model library in step S4); for the selection of the error correction model set (i.e., the error Q learning model library composed of the wind speed prediction error model), since the volatility and variability of the prediction error are far less severe than the measured wind speed sequence, more detailed predictions of the error sequence are required. Therefore, the embodiment of the present invention selects five models with higher efficiency, namely SVR, BP neural network, GKRR, PKRR, and MHKRR, to form the error correction model set, among which the GKRR, PKRR and MHKRR models are KRR models using different kernel functions.
所述方法在步骤S4)之后还包括：S5)利用所述风速检验集验证所述最佳风速预测误差模型的有效性；验证所述最佳风速预测误差模型的有效性，选取了均方误差ε1、相对误差ε2和决定系数R2三种评价指标对最终的风速预测数据进行评价，其中ε1、ε2最优期望为0，R2最优期望为1，计算公式分别如下：The method further comprises, after step S4), S5) using the wind speed test set to verify the validity of the optimal wind speed prediction error model; to verify the validity of the optimal wind speed prediction error model, three evaluation indicators, namely, mean square error ε1 , relative error ε2 and determination coefficient R2, are selected to evaluate the final wind speed prediction data, wherein the optimal expectations of ε1 and ε2 are 0, and the optimal expectation of R2 is 1, and the calculation formulas are as follows:
其中：xt、yt、
选取东北某实际风场2019年风速及相关属性数据开展研究，对该风场各季度典型月(3月、6月、9月和12月)进行短期风速预测，并对实测风速数据进行预处理，采用相邻数据互补法替换缺失及异常数据值，实测风速数据的采样间隔为10min，一个月共计4320个点，取每个月前20天数据为风速训练集，21-25日数据为风速测试集，26-30日为风速检验集(用以检验模型参数设定是否合适)The wind speed and related attribute data of an actual wind farm in Northeast China in 2019 were selected for research. Short-term wind speed forecasts were made for typical months in each quarter (March, June, September and December) of the wind farm, and the measured wind speed data were preprocessed. The adjacent data complementary method was used to replace missing and abnormal data values. The sampling interval of the measured wind speed data was 10 minutes, with a total of 4320 points in a month. The data from the first 20 days of each month were taken as the wind speed training set, the data from the 21st to the 25th were taken as the wind speed test set, and the data from the 26th to the 30th were taken as the wind speed test set (to test whether the model parameter settings are appropriate).
模型超参数设置如下：The model hyperparameters are set as follows:
本发明实施例Q学习框架具体参数设置为κ＝0.1，γ＝0.8，以确保动态模型选择的学习速度，Ne＝100，并充分考虑奖励函数的未来奖惩，选取α＝0.9，β＝0.1；根据风场实际运行情况，选择进行步长为6的日前预测(k＝6，n＝144)，即根据风速训练集数据的训练结果，采用最佳策略为下一个k步做出模型选择决策，基础模型的超参数设置见表1。The specific parameters of the Q learning framework of the embodiment of the present invention are set to κ=0.1, γ=0.8 to ensure the learning speed of dynamic model selection,Ne =100, and fully consider the future rewards and penalties of the reward function, and select α=0.9, β=0.1; according to the actual operation of the wind farm, a day-ahead prediction with a step size of 6 (k=6, n=144) is selected, that is, according to the training results of the wind speed training set data, the best strategy is used to make a model selection decision for the next k steps. The hyperparameter settings of the basic model are shown in Table 1.
表1不同算法的超参数设置Table 1 Hyperparameter settings of different algorithms
关于Q学习奖励函数的设置，目前在人工智能算法中比较常用的自适应误差函数，但在训练过程中发现采用自适应误差函数为奖励函数进行的Q学习未能收敛，这是因为预测评估指标的大小不仅取决于预测模型，而且会随着时间而变化，采取从糟糕的模型切换到最佳模型的操作可能仍会收到负回报(由于预测评估指标的下降)。同时，一个预测模型的成熟与否不仅与预测精度有关，还与其所付出的时间成本相关，由此，提出了另一种奖励函数对模型效果进行评价，即综合考虑模型排名改进和模型预测时间。两种方法的训练结果如图3所示，可以看出，该奖励函数成功收敛，有效避免了时间序列效应。Regarding the setting of the Q-learning reward function, the adaptive error function is currently more commonly used in artificial intelligence algorithms. However, during the training process, it was found that the Q-learning using the adaptive error function as the reward function failed to converge. This is because the size of the prediction evaluation index depends not only on the prediction model, but also changes over time. Switching from a bad model to the best model may still receive a negative return (due to the decline in the prediction evaluation index). At the same time, the maturity of a prediction model is not only related to the prediction accuracy, but also to the time cost. Therefore, another reward function is proposed to evaluate the model effect, that is, comprehensively consider the model ranking improvement and model prediction time. The training results of the two methods are shown in Figure 3. It can be seen that the reward function successfully converges and effectively avoids the time series effect.
东北风场2019年一整年的风速波动情况如图4所示，可以看出该风场风能量密度较大，其中春冬两季风速波动较为剧烈，不同时刻风速差较大，最高风速超过25m/s。而夏秋两季风速多为低于10m/s，波动较为平缓，风能量密度明显低于春冬季节。The wind speed fluctuations of the Northeast Wind Farm in 2019 are shown in Figure 4. It can be seen that the wind energy density of the wind farm is relatively large. The wind speed fluctuations in spring and winter are more drastic, with large wind speed differences at different times, and the highest wind speed exceeds 25m/s. In summer and autumn, the wind speed is mostly below 10m/s, with relatively gentle fluctuations, and the wind energy density is significantly lower than that in spring and winter.
为检验和说明基于Q学习的动态模型选择的有效性，选取单预测模型LSTM和BP神经网络两种不同预测原理的人工智能算法与基于双重Q学习的动态预测(Dynamicprediction based on double Q learning，DPDQ)模型中的风速预测部分(Q learningwind speed prediction，QWSP)进行仿真对比分析，对各季度典型月风速进行滑动步长为6的日前预测，具体预测结果如图5a至图5d所示。In order to test and illustrate the effectiveness of the dynamic model selection based on Q learning, two artificial intelligence algorithms with different prediction principles, the single prediction model LSTM and the BP neural network, were selected for simulation and comparative analysis with the wind speed prediction part (Q learning wind speed prediction, QWSP) in the dynamic prediction (Dynamic prediction based on double Q learning, DPDQ) model. The typical monthly wind speed of each quarter was predicted with a sliding step of 6. The specific prediction results are shown in Figures 5a to 5d.
由图5a至图5d可以看出，QWSP在应对各季节不同的风速波动情况下均能得到良好表现，整体的预测效果优于单一预测模型；而从细节上看，夏秋两个季节的风速波动较为平缓，风速也较低，春冬季节的风速相对较高。展开图5a中121-126部分和图5b中115-120部分可以看出，各模型均能得到较好的预测效果；图5c中61-66部分，BP神经网络和LSTM由于没有进行动态选择，导致其不能应对所有的风速变化情况，预测结果与实际值偏差较大；而图(d)中的细节预测结果则不然，从右图可以看出，QWSP模型的预测结果偏差大于BP神经网络，这主要是Q学习模型选择失误而导致的预测偏差较大，由于奖励函数机制的设置，本次所选模型的排名应较为靠后，导致本次奖励为负值，从而在下一次模型选择中进行修正，以期得到更好的预测结果。在Q学习的动态选择正确时，预测精度普遍偏高，模型具有较好的稳健性；而当单预测模型过拟合时，将会出现预测误差较大的情况，由于奖励函数的机制，使模型能够及时在下一时段得到修正。由此，基于Q学习的模型选择策略，可使风速预测模型性能得到整体提高。各季度典型月风速预测误差见表2。As can be seen from Figure 5a to Figure 5d, QWSP can achieve good performance in dealing with different wind speed fluctuations in each season, and the overall prediction effect is better than a single prediction model; and from the details, the wind speed fluctuations in summer and autumn are relatively gentle, and the wind speed is also low, while the wind speed in spring and winter is relatively high. Expanding the 121-126 part in Figure 5a and the 115-120 part in Figure 5b shows that each model can achieve good prediction results; in the 61-66 part in Figure 5c, the BP neural network and LSTM cannot cope with all wind speed changes due to the lack of dynamic selection, and the prediction results deviate greatly from the actual value; however, the detailed prediction results in Figure (d) are not the case. As can be seen from the right figure, the prediction result deviation of the QWSP model is greater than that of the BP neural network. This is mainly due to the large prediction deviation caused by the wrong selection of the Q learning model. Due to the setting of the reward function mechanism, the ranking of the selected model should be relatively low, resulting in a negative reward for this time, so that it will be corrected in the next model selection in order to obtain better prediction results. When the dynamic selection of Q learning is correct, the prediction accuracy is generally high and the model has good robustness; when the single prediction model is overfitted, the prediction error will be large. Due to the mechanism of the reward function, the model can be corrected in time in the next period. Therefore, the model selection strategy based on Q learning can improve the overall performance of the wind speed prediction model. The typical monthly wind speed prediction errors in each quarter are shown in Table 2.
表2不同方法短期预测误差Table 2 Short-term prediction errors of different methods
由表2中数据可知，由于夏季风速波动最小，各模型R2都达到了0.9左右，而冬季风速波动大，给模型带来一定的预测难度，导致预测误差ε1也随之增大。本发明实施例所提模型基于强化学习的动态模型选择使其R2结果在各季度均最接近于1，ε1误差也为三个模型中最小。From the data in Table 2, it can be seen that since the wind speed fluctuation is the smallest in summer, the R2 of each model reaches about 0.9, while the wind speed fluctuation is large in winter, which brings certain prediction difficulties to the model, resulting in an increase in the prediction error ε1. The dynamic model selection based on reinforcement learning of the model proposed in the embodiment of the present invention makes its R2 result closest to 1 in each season, and the ε1 error is also the smallest among the three models.
为验证本发明实施例误差校正环节的有效性，利用DPDQ对各季节进行日前风速预测，结果如图6a至图6d所示,可以看出DPDQ模型在各季节的预测值均能达到较好的效果，但对于某些风速差过大的极端风况，如图6c中的最高风速点仍存在一定的预测误差，这是不可避免的。In order to verify the effectiveness of the error correction link of the embodiment of the present invention, DPDQ is used to predict the day-ahead wind speed in each season. The results are shown in Figures 6a to 6d. It can be seen that the prediction values of the DPDQ model in each season can achieve good results. However, for some extreme wind conditions with excessive wind speed differences, such as the highest wind speed point in Figure 6c, there is still a certain prediction error, which is inevitable.
本发明提供的动态风速预测模型建立方法，采用两次Q强化学习算法构建了动态风速预测模型，其中一个Q学习代理负责选出最佳风速预测模型进行初步的风速预测，另一个Q学习代理负责通过计算误差将其输入到误差校正部分，从中选出最佳风速预测误差模型，得到最优预测策略；并且Q学习在风速预测部分和误差修正部分都有效选择了最佳预测模型；本发明的误差校正使预测平均相对误差减少了50％，误差校正环节对成熟预测模型具有有效性；本发明通过构建风速预测模型对不同季节的典型月进行了预测，结果表明其泛化能力强、鲁棒性好、预测精度高，解决了由于风速波动性、间接性及低能量密度等特点导致的电力系统运行可靠性降低问题，可显著提高含可再生能源并网的电网调度经济性和风电场的运行安全性。The method for establishing a dynamic wind speed prediction model provided by the present invention adopts a two-time Q reinforcement learning algorithm to construct a dynamic wind speed prediction model, wherein one Q learning agent is responsible for selecting the best wind speed prediction model for preliminary wind speed prediction, and the other Q learning agent is responsible for inputting the calculated error into the error correction part, thereby selecting the best wind speed prediction error model and obtaining the optimal prediction strategy; and the Q learning effectively selects the best prediction model in both the wind speed prediction part and the error correction part; the error correction of the present invention reduces the average relative error of the prediction by 50%, and the error correction link is effective for mature prediction models; the present invention predicts typical months of different seasons by constructing a wind speed prediction model, and the results show that the model has strong generalization ability, good robustness and high prediction accuracy, solves the problem of reduced reliability of power system operation caused by the characteristics of wind speed volatility, indirectness and low energy density, and can significantly improve the dispatch economy of power grids containing renewable energy and the operation safety of wind farms.
本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处。综上所述，本说明书内容不应理解为对本发明的限制。This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110557310.3ACN113515889B (en) | 2021-05-21 | 2021-05-21 | Dynamic wind speed prediction model building method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110557310.3ACN113515889B (en) | 2021-05-21 | 2021-05-21 | Dynamic wind speed prediction model building method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113515889A CN113515889A (en) | 2021-10-19 |
| CN113515889Btrue CN113515889B (en) | 2023-06-13 |
Family
ID=78064601
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110557310.3AExpired - Fee RelatedCN113515889B (en) | 2021-05-21 | 2021-05-21 | Dynamic wind speed prediction model building method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113515889B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114118537B (en)* | 2021-11-09 | 2024-10-22 | 南京航空航天大学 | Airspace flight carbon emission combined prediction method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104899446A (en)* | 2015-06-05 | 2015-09-09 | 上海大学 | Method for simulating fluctuating wind speeds on basis of data drive |
| CN108229732A (en)* | 2017-12-20 | 2018-06-29 | 上海电机学院 | ExtremeLearningMachine wind speed ultra-short term prediction method based on error correction |
| CN110474339A (en)* | 2019-08-07 | 2019-11-19 | 国网福建省电力有限公司 | A kind of electric network reactive-load control method based on the prediction of depth generation load |
| CN111064229A (en)* | 2019-12-18 | 2020-04-24 | 广东工业大学 | Wind-light-gas-storage joint dynamic economic dispatch optimization method based on Q-learning |
| CN111985712A (en)* | 2020-08-19 | 2020-11-24 | 华北电力大学（保定） | Multi-step wind speed combined prediction model building method |
- 2021
- 2021-05-21CNCN202110557310.3Apatent/CN113515889B/ennot_activeExpired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104899446A (en)* | 2015-06-05 | 2015-09-09 | 上海大学 | Method for simulating fluctuating wind speeds on basis of data drive |
| CN108229732A (en)* | 2017-12-20 | 2018-06-29 | 上海电机学院 | ExtremeLearningMachine wind speed ultra-short term prediction method based on error correction |
| CN110474339A (en)* | 2019-08-07 | 2019-11-19 | 国网福建省电力有限公司 | A kind of electric network reactive-load control method based on the prediction of depth generation load |
| CN111064229A (en)* | 2019-12-18 | 2020-04-24 | 广东工业大学 | Wind-light-gas-storage joint dynamic economic dispatch optimization method based on Q-learning |
| CN111985712A (en)* | 2020-08-19 | 2020-11-24 | 华北电力大学（保定） | Multi-step wind speed combined prediction model building method |
Non-Patent Citations (5)
| Title |
|---|
| Yonggang Li 等.Short-Term Direct Probability Prediction Model of Wind Power Based on Improved Natural Gradient Boosting.《energies》.2020,第1-15页.* |
| 刘永强 ; 续毅 ; 贺永辉 ; 柳文斌 ; .基于双向长短期记忆神经网络的风电预测方法.天津理工大学学报.2020,(第05期),第51-56+61页.* |
| 李永刚 ; 王月 ; 刘丰瑞 ; 吴滨源 ; .基于Stacking融合的短期风速预测组合模型.电网技术.2020,(第08期),第85-92页.* |
| 李永刚 等.基于双重 Q 学习的动态风速预测模型.《电工技术学报》.2022,第37卷(第7期),第1810-1819页.* |
| 王毅 ; 于明 ; 李永刚 ; .基于模型预测控制方法的风电直流微网集散控制.电工技术学报.2016,(第21期),第 61-70页.* |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113515889A (en) | 2021-10-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11581740B2 (en) | Method, system and storage medium for load dispatch optimization for residential microgrid | |
| CN116544934B (en) | Power scheduling method and system based on power load prediction | |
| CN112734128A (en) | 7-day power load peak value prediction method based on optimized RBF | |
| CN110837915B (en) | Low-voltage load point prediction and probability prediction method for power system based on hybrid integrated deep learning | |
| CN118350513B (en) | ISWO-LSTM model-based carbon emission prediction method and system | |
| CN113592144B (en) | Medium-long term runoff probability forecasting method and system | |
| CN113344288B (en) | Water level prediction method, device and computer-readable storage medium for cascade hydropower station group | |
| CN110598929A (en) | Wind power nonparametric probability interval ultrashort term prediction method | |
| CN115496290A (en) | Medium-and-long-term runoff time-varying probability prediction method based on 'input-structure-parameter' full-factor hierarchical combination optimization | |
| CN115764870A (en) | Multivariable photovoltaic power generation power prediction method and device based on automatic machine learning | |
| CN115409645A (en) | An Energy Management Method for Integrated Energy Systems Based on Improved Deep Reinforcement Learning | |
| CN120389441B (en) | Operation optimization method and system of source-grid-load-storage system based on digital twin | |
| CN115663914A (en) | An Aggregate Scheduling Method for Virtual Power Plants Containing Wind Power Based on Deep Reinforcement Learning | |
| CN113515889B (en) | Dynamic wind speed prediction model building method | |
| CN118054400A (en) | Wind power prediction method and system based on interpretability and model fusion | |
| CN117526294A (en) | Short-term building load prediction method | |
| CN116822360A (en) | Power system frequency trajectory prediction method, device, medium and equipment | |
| CN118630745A (en) | Power System Net Load Forecasting Method Based on Neural Network and Gaussian Regression Model | |
| Huang et al. | Probabilistic prediction intervals of wind speed based on explainable neural network | |
| CN117117858B (en) | Wind turbine generator power prediction method, device and storage medium | |
| Zhou et al. | Double-tank liquid level control based on genetic algorithm | |
| CN115409291A (en) | Wind power prediction method and system with wind speed correction | |
| CN119134338B (en) | Method for controlling load frequency of power system by deep reinforcement learning | |
| Zheng et al. | Long Term Electricity Consumption Forecast Based on STL-IDBO-LSSVM | |
| CN118982161A (en) | A decision-making method and system for intraday forward-looking dispatch of power grid based on SAC |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20230613 | |
| CF01 | Termination of patent right due to non-payment of annual fee |