大连海事大学学报 >
2017 , Vol. 43 >Issue 4: 97 - 103
DOI: https://doi.org/10.16411/j.cnki.issn1006-7736.2017.04.014
基于强化学习的非线性时间序列智能预测模型
收稿日期: 2017-06-16
修回日期: 2017-08-16
网络出版日期: 2017-08-21
基金资助
国家社会科学基金重大项目（14ZDB133）; 国家自然科学基金资助项目（61572079）;北京市科委重大项目科技计划课题（D151100004215003）.
Intelligent prediction model for nonlinear time series based on reinforcement learning
Received date: 2017-06-16
Revised date: 2017-08-16
Online published: 2017-08-21
针对时间序列预测中非线性、噪声高等特点，提出时间序列向前多步混合智能预测模型. 首先，在模型训练过程中，提出一种将强化学习与隐马尔可夫模型相结合的新方法，强化学习运用TD(λ)方法，采用历史观测数据作为报酬回报，强调远近期历史数据的不同影响程度并用以迭代增强历史观测数据在模型中的作用；进一步，在向前多步预测过程中，提出一种以强化学习为桥梁、将神经网络与隐马尔可夫模型相结合的方法，用以充分发挥神经网络数据拟合优势和隐马尔可夫模型减小系统随机误差方面的优势.利用稀土期货交易数据进行预测实验，结果表明：智能预测模型显著降低了预测的平均绝对误差、百分比绝对平均误差、均方根误差，提高了预测的准确性和效果.
孙若莹 , 范厚明 , 赵刚 . 基于强化学习的非线性时间序列智能预测模型[J]. 大连海事大学学报, 2017 , 43(4) : 97 -103 . DOI: 10.16411/j.cnki.issn1006-7736.2017.04.014
Aiming at the nonlinear and high noise issues in time series prediction, an hybrid intelligent model for time series forward multistep prediction was proposed. Firstly, for training the model, a method of combining reinforcement learning and hidden Markov model was proposed, TD(λ) approach was used to reinforce learning, and historical observed data was adopted as reward returns and emphasized the different degree of influence in far more recent to iteratively improve the roles of historical observed data played in model training processes. Further, in the process of forward multi-step prediction, a method of combining reinforcement learning with neural network and hidden Markov model was proposed, which took full advantages of data fitting by neural network and reducing systematic random error in predicting processes by hidden Markov model. Experimental results of rare earth futures trade data predictions show that the intelligent hybrid model obviously reduces mean absolute error, mean absolute percentage error and root mean square error, which improves the prediction precision and effect.
［1］ 张大斌,李红燕,刘肖,等. 非线性时间序列的小波–模糊神经网络集成预测方法［J］. 中国管理科学,2013,21(专辑):647-651.
ZHANG D B,LI H Y, LIU X, et al. A integrated prediction method of wavelet-fuzzy neural network for nonlinear time series［J］. Chinese Journal of Management Science, 2013,21(Special Issue):647-651.(in Chinese)
［2］ 章强,王学锋,殷明. 超大型矿砂船运营背景下的国际铁矿石海运运价的波动性［J］. 上海海事大学学报,2016,37(3): 40-46.
ZHANG Q,WANG X F,YIN M. Volatility of international shipping freight rate for iron ore under background of VLOC operation［J］. Journal of Shanghai Maritime University, 2016,37(3): 40-46.(in Chinese)
［3］ 林屹,严洪森,周博. 基于多维泰勒网的非线性时间序列预测方法及其应用［J］. 控制与决策,2014,29(5):795-801.
LIN Y,YAN H S,ZHOU B. Nonlinear time series prediction method based on multi-dimensional Taylor network and its applications［J］. Control and Decision, 2014,29(5):795-801.(in Chinese)
［4］ 赛英,张凤廷,张涛. 基于支持向量机的中国股指期货回归预测研究［J］. 中国管理科学,2013,21(3):35-39.
SAI Y,ZHANG F T,ZHANG T. Research of Chinese stock index futures regression prediction based on support vector machines［J］. Chinese Journal of Management Science,2013,21(3):35-39.(in Chinese)
［5］ PATEL J,SHAH S,THAKKAR P, et al. Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques［J］.Expert Systems with Applications,2015,42(1): 259-268.
［6］ 王新迎,韩敏. 基于极端学习机的多变量混沌时间序列预测［J］. 物理学报,2012,61(8): 97-105.
WANG X Y,HAN M. Multivariate chaotic time series prediction based on extreme learning machine［J］. Acta Physica Sinica,2012,61(8): 97-105.(in Chinese)
［7］ ZHANG H J,SUN R Y.Parameter analysis of hybrid intelligent model for the prediction of rare earth stock futures［C］// Proceedings of the 12th International Conference on Natural Computation,Fuzzy Systems and Knowledge Discovery(ICNC-FSKD).［S.l.］: IEEE Press,2016.
［8］ 鲍漪澜.基于支持向量机的金融时间序列分析预测算法研究［D］.大连:大连海事大学,2013.
BAO Y L.Research on the analysis and predictive algorithm of financial time series based on support vector machine［D］.Dalian:Dalian Maritime University,2013.(in Chinese)
［9］ 余文利,廖建平,马文龙. 一种新的基于隐马尔可夫模型的股票价格时间序列预测方法［J］. 计算机应用与软件,2010,27(6):186-190.
YU W L,LIAO J P, MA W L. A novel hidden Markov model-based stock price time series forecasting method［J］. Computer Applications and Software, 2010,27(6):186-190.(in Chinese)［10］ 刘震, 王厚军, 龙兵,等.一种基于加权隐马尔可夫的自回归状态预测模型［J］.电子学报,2009,37(10):2113-2118.
LIU Z,WANG H J,LONG B, et al. Research on condition trend prediction based on weighed hidden Markov and autoregressive model［J］. Acta Physica Sinica,2009,37(10):2113-2118.(in Chinese)
［11］孙若莹,赵刚.多主体强化学习协作策略研究［M］.北京:清华大学出版社,2014: 72-78.
［12］ HASSAN M R, NATH B,KIRLEY M. A fusion model of HMM, ANN and GA for stock market forecasting［J］. Expert Systems with Applications, 2007, 33(1): 171-180.
/
| 〈 |
|
〉 |