451Accesses
3Altmetric
Abstract
The issue of human motion prediction aimed to predict sequences of joint positions or joint rotations of human skeleton has recently grown in importance. The Recurrent Neural Network is widely applied on the sequence prediction problems which has been proved effective. However it is difficult to train the model with human skeleton data of multi-dimensional as input, which would do naive forecasting to produce motionless sequence. To address the problem, it is a consensus that additional information will help to improve the accuracy, thus the angular velocities are extracted from the joint rotations as the input to enhance the prediction. Further more, this work adopts proper strategies on the basis of a stacked Gated Recurrent Unit network and verify them on the human motion prediction task. The experimental results show that our network outperforms the state-of-art on the short-term prediction task, and generates plausible action sequences in a relatively long period of time.
This is a preview of subscription content,log in via an institution to check access.
Access this article
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
Price includes VAT (Japan)
Instant access to the full article PDF.







Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Barsoum E, Kender J, Liu Z (2018) HP-GAN: Probabilistic 3D human motion prediction via GAN. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1418–1427
Boulic R, Thalmann NM, Thalmann D (1990) A global human walking model with real-time kinematic personification. Vis Comput 6(6):344–358
Brownlee J (2017) Long short-term memory networks with python: develop sequence prediction models with deep learning. Machine Learning Mastery, Jason Brownlee
Butepage J, Black M.J., Kragic D., Kjellstrom H (2017) Deep representation learning for human motion prediction and classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6158–6166
Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078
Dong M, Xu C (2019) On retrospecting human dynamics with attention. In: Proceedings of the 28th international joint conference on artificial intelligence. AAAI Press, pp 708–714
Euler L (1758) Novi commentarii academiae scientiarum petropolitanae, American
Fragkiadaki K, Levine S, Felsen P, Malik J (2015) Recurrent network models for human dynamics. In: Proceedings of the IEEE international conference on computer vision, pp 4346–4354
Gopalakrishnan A, Mali A, Kifer D, Giles L, Ororbia AG (2019) A neural temporal model for human motion prediction. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 12116–12125
Gui LY, Wang YX, Liang X, Moura JM (2018) Adversarial geometry-aware human motion prediction. In: Proceedings of the european conference on computer vision (ECCV), pp 786– 803
Gui LY, Zhang K, Wang YX, Liang X, Moura JM, Veloso M (2018) Teaching robots to predict human motion. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 562–567
Gurbuz SZ, Amin MG (2019) Radar-based human-motion recognition with deep learning: Promising applications for indoor monitoring. IEEE Sig Process Mag 36(4):16–28
Hazewinkel M, Gubareni N, Kirichenko VV (2004) Algebras rings and modules. USA: Springer, New York
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Ionescu C, Papava D, Olaru V, Sminchisescu C (2013) Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Trans Pattern Anal Mach Intell 36 (7):1325–1339
Jain A, Zamir AR, Savarese S, Saxena A (2016) Structural-rnn: Deep learning on spatio-temporal graphs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5308–5317
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:14126980
Kundu JN, Gor M, Babu RV (2019) Bihmp-gan: Bidirectional 3d human motion prediction gan. In: Proceedings of the AAAI conference on artificial intelligence no. 01, vol 33, pp 8553– 8560
Lamb AM, Goyal AGAP, Zhang Y, Zhang S, Courville AC, Bengio Y (2016) Professor forcing: A new algorithm for training recurrent networks. In: Advances in neural information processing systems, pp 4601–4609
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
Li C, Zhang Z, Sun Lee W, Hee Lee G (2018) Convolutional sequence to sequence model for human dynamics. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5226–5234
Li H, Shen Y, Zhu Y (2018) Stock price prediction using attention-based multiinput lstm. In: Asian conference on machine learning, pp 454–469
Liu J, Shahroudy A, Wang G, Duan LY, Chichung AK (2019) Skeleton-based online action prediction using scale selection network. IEEE Trans Pattern Anal Mach Intell 42(6):1453–1467
Liu Z, Wu S, Jin S, Liu Q, Lu S, Zimmermann R, Cheng L (2019) Towards natural and accurate future motion prediction of humans and animals. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 10004–10012
Martinez J, Black MJ, Romero J (2017) On human motion prediction using recurrent neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2891–2900
Pavllo D, Feichtenhofer C, Auli M, Grangier D (2019) Modeling human motion with quaternion-based neural networks. arXiv:190107677
Plappert M, Mandery C, Asfour T (2018) Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks. Robot Auton Syst 109:13–26
Pourpanah F, Lim CP, Hao Q (2019) A reinforced fuzzy artmap model for data classification. Int J Mach Learn Cybern 10(7):1643–1655
Qi Z, Shu X, Tang J (2018) Dotanet: Two-stream match-recurrent neural networks for predicting social game result. In: 2018 IEEE fourth international conference on multimedia big data (BigMM), pp 1–5
Sang HF, Chen ZZ, He DK (2020) Human motion prediction based on attention mechanism. Multimed Tools Appl 79(9):5529–5544
Song Y, Demirdjian D, Davis R (2012) Continuous body and hand gesture recognition for natural human-computer interaction. ACM Trans Interact Intell Syst (TiiS) 2(1):5
Strickland J (2008) What is a gimbal–and what does it have to do with nasa
Sugiartawan P, Pulungan R, Sari AK (2017) Prediction by a hybrid of wavelet transform and long-short-term-memory neural network. Int J Adv Comput Sci Appl 8(2):326–332
Tang Y, Ma L, Liu W, Zheng W (2018) Long-term human motion prediction by modeling motion context and enhancing motion dynamic. arXiv:180502513
Tanisaro P, Heidemann G (2018) An empirical study on bidirectional recurrent neural networks for human motion recognition. In: 25th international symposium on temporal representation and reasoning (TIME 2018) no. 21, vol 120, pp 1–19
Tong L, Song Q, Ge Y, Liu M (2013) Hmm-based human fall detection and prediction method using tri-axial accelerometer. IEEE Sensors J 13(5):1849–1856
Toshev A, Szegedy C (2014) Deeppose: Human pose estimation via deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1653–1660
Xu YT, Li Y, Meger D (2019) Human motion prediction via pattern completion in latent representation space. In: 2019 16th conference on computer and robot vision (CRV), pp 57–64
Yan S, Xiong Y, Lin D (2018) Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Thirty-second AAAI conference on artificial intelligence, pp 12026–12035
Ding C, Liu K et al (2021) Spatio-temporal attention on manifold space for 3D human action recognition. In: Applied intelligence, pp 560–570
Men Q, Ho ES, Shum HP, Leung H (2020) A quadruple diffusion convolutional recurrent network for human motion prediction. In: IEEE transactions on circuits and systems for video technology
Acknowledgements
This work is supported by National Natural Science Foundation of China (61807002).
Author information
Authors and Affiliations
School of Computer Science and Technology, Beijing Institute of Technology, No.5, South Zhongguancun Road, Haidian District, Beijing, 100081, Beijing, People’s Republic of China
Yue Yu, Niehao Tian & XiangYu Hao
State Key Laboratory of Smart Manufacturing for Special Vehicles and Transmission System, Inner Mongolia No. 2 Mailbox, 014030, Baotou City, People’s Republic of China
Tao Ma & Chunguang Yang
- Yue Yu
You can also search for this author inPubMed Google Scholar
- Niehao Tian
You can also search for this author inPubMed Google Scholar
- XiangYu Hao
You can also search for this author inPubMed Google Scholar
- Tao Ma
You can also search for this author inPubMed Google Scholar
- Chunguang Yang
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toYue Yu.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yu, Y., Tian, N., Hao, X.et al. Human motion prediction with gated recurrent unit model of multi-dimensional input.Appl Intell52, 6769–6781 (2022). https://doi.org/10.1007/s10489-021-02764-x
Accepted:
Published:
Issue Date:
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative