427Accesses
7Citations
Abstract
Deep Boltzmann machine (DBM) has been successfully applied in classification, regression and time series modeling. For nonlinear system modelling, DBM should also have many advantages over the other neural networks, such as input features extraction and noise tolerance. In this paper, we use DBM to model nonlinear systems by calculating the probability distributions of the input and output. Two novel weight updating algorithms are proposed to obtain these distributions. We use binary encoding and conditional probability transformation methods. The proposed methods are validated with two benchmark nonlinear systems.
This is a preview of subscription content,log in via an institution to check access.
Access this article
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
Price includes VAT (Japan)
Instant access to the full article PDF.







Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
LeCun Y, Bengio Y, Hinton GE (2015) Deep learning. Nature 521(7553):436–444
Wang Z, Wang X (2018) A deep stochastic weight assignment network and its application to chess playing. J Parall Distrib Comput 117:205–211
Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Salakhutdinov R, Hinton GE (2009) Deep boltzmann machines. In: 12th International conference on artificial intelligence and statistics (AISTATS). Clearwater Beach, Florida, USA
Salakhutdinov R, Hinton GE (2012) An efficient learning procedure for deep boltzmann machines. Neural Comput 24:1967–2006
Larochelle H, Bengio Y (2008) Classification using discriminative restricted Boltzmann machines. In: 25th International conference on machine learning, Helsinki, Finland, pp 536–543
Le Roux N, Bengio Y (2008) Representational power of restricted Boltzmann machines and deep belief networks. Neural Comput 20:1631–1649
Erhan D, Bengio Y, Courville A, Manzagol P-A, Vincent P (2010) Why does unsupervised pre-training help deep learning? J Mach Learn Res 11:625–660
Wang X, Zhang T, Wang R (2017) Non-iterative deep learning: incorporating restricted boltzmann machine into multilayer random weight neural networks. IEEE Trans Syst Man Cybern Syst.https://doi.org/10.1109/TSMC.2017.2701419
Hinton GE, Sejnowski TJ (1986) Learning and relearning in Boltzmann machines. In: Parallel distributed processing: explorations in the microstructure of cognition, vol 1: foundations, pp 282–317, MIT Press, Cambridge, MA
Qiu L, Zhang L, Ren Y, Suganthan PN, Amaratunga G (2014) Ensemble deep learning for regression and time series forecasting. In: 2014 IEEE symposium on computational intelligence in ensemble learning (CIEL), pp 1–6, Orlando, FL, USA
Busseti E, Osband I, Wong S (2012) Deep learning for time series modeling. Technical report, Stanford University
Längkvist M, Karlsson L, Loutfi A (2014) A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recogn Lett 42:11–24
Romeu P et al (2013) Time-series forecasting time-series, of indoor temperature using pre-trained deep neural networks. Artificial Neural Networks and Machine Learning-ICANN. Springer, Berlin Heidelberg, pp 451–458
Ljung L (1987) System identification-theory for user. Prentice Hall, Englewood Cliffs
Hintonand GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313:504–507
Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Advances in neural information processing systems (NIPS’06). MIT Press, Cambridge, pp 153–160
de la Rosa E, Yu W (2016) Randomized algorithms for nonlinear system identification with deep learning modification. Inform Sci 364:197–212
Narendra KS, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4-2
Jagannathan S, Lewis FL (1996) Identification of nonlinear dynamical systems using multilayered neural networks. Automatica 32(12):1707–1712
Busseti E, Osband I, Wong S (2012) Deep learning for time series modeling. CS 229 Technical Report, Stanford University
de la Rosa E, Yu W (2015) Restricted Boltzmann machine for nonlinear system modeling. In: 14th IEEE international conference on machine learning and applications (ICMLA15), Miami, USA
de la Rosa E, Yu W, Li X (2016) Nonlinear system modeling with deep neural networks and autoencoders algorithm. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC16), Budapest, Hungary, pp 2157–2162
de la Rosa E, Yu W (2015) Nonlinear system identification using deep learning and randomized algorithms. In: 2015 IEEE international conference on information and automation, Lijing, China, pp 274–279
Chen S, Billings SA (1992) Neural networks for nonlinear system modelling and identification. Int J Control 56(2):319–346
Cybenko G (1989) Approximation by superposition of sigmoidal activation function. Math Control Sig Syst 2:303–314
Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw. 17(4):879–92
Ackley DH, Hinton GE, Sejnowski TJ (1985) A learning algorithm for boltzmann machines. Cogn Sci 9:147–169
Bengio Y, Delalleau O (2009) Justifying and generalizing contrastive divergence. Neural Comput 21(6):1601–1621
Box G, Jenkins G, Reinsel G (2008) Time series analysis: forecasting and control, 4th edn. Wiley, New York
Bergstra J, Bengio Y (2011) Random search for hyper-parameter optimization. J Machine Learn Res 13:281–305
Schoukens J, Suykens J, Ljung L (2009) Wiener-Hammerstein benchmark, 15th IFAC symposiumon system identification. Saint-Malo, France
Erhan D, Manzagol PA, Bengio Y, Bengio S, Vincent P (2009) The difficulty of training deep architectures and the effect of unsupervised pretraining. In: 12th International conference on artificial intelligence and statistics (AISTATS’09), pp 153–160
Bartlett PL (1997) For valid generalization, the size of the weights is more important than the size of the network. In: Mozer M, Jordan M, Petsche T (eds) Advances in neural information processing systems’ 1996, vol 9. MIT Press, Cambridge, pp 134–140
Jang JS (1993) ANFIS: Adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23:665–685
Mitra S, Hayashi Y (2000) Neuro-fuzzy rule generation: survey in soft computing framework. IEEE Trans Neural Netw 11(3):748–769
Rivals I, Personnaz L (2003) Neural-network construction and selection in nonlinear modeling. IEEE Trans Neural Netw 14(4):804–820
Wang X, Cao W (2018) Non-iterative approaches in training feed-forward neural networks and their applications. Soft Comput.https://doi.org/10.1007/s00500-018-3203-0
Leung FHF, Lam HK, Ling SH, Tam PKS (2003) Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Transact Neural Netw 14:79–88
Author information
Authors and Affiliations
Departamento de Control Automatico, CINVESTAV-IPN (National Polytechnic Institute), Av. IPN 2508, Mexico City, 07360, Mexico
Wen Yu & Erick de la Rosa
- Wen Yu
You can also search for this author inPubMed Google Scholar
- Erick de la Rosa
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toWen Yu.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yu, W., de la Rosa, E. Deep Boltzmann machine for nonlinear system modelling.Int. J. Mach. Learn. & Cyber.10, 1705–1716 (2019). https://doi.org/10.1007/s13042-018-0847-0
Received:
Accepted:
Published:
Issue Date:
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative