1041Accesses
13Citations
Abstract
Pedestrian trajectory prediction has many real-world applications, such as crowd video surveillance and self-driving cars. However, this is a challenging problem due to the complexity of modeling social interactions between road agents (pedestrians and vehicles). Previous studies that addressed trajectory prediction applied traditional deep learning approaches, including CNN and RNN. Meanwhile, the applications of graph neural networks have drawn increasing interest recently, and significant progress has been made in extracting features from complex and unstructured data. In this paper, we propose STM-GCN, a spatiotemporal multi-graph convolutional network for pedestrian trajectory prediction. Our approach consists of collecting information about the social interactions between pedestrians in a crowd that are position-based and velocity-based interactions integrated into a multi-graph. Then we apply Graph Neural Network learning on the obtained multi-graph for predictions. Through experiments on the ETH and UCY benchmarking datasets, our proposed method outperforms the state of the art by 6% average displacement error (ADE) and 10% final displacement error (FDE). Our implementation is available athttps://github.com/YoussefTaki/STMGCN.
This is a preview of subscription content,log in via an institution to check access.
Access this article
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
Price includes VAT (Japan)
Instant access to the full article PDF.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.Availability of data and materials
In this work, we used two publicly available datasets widely used in the literature.
Code Availability
Applicable.
References
Houenou A, Bonnifait P, Cherfaoui V, Yao W (2013) Vehicle trajectory prediction based on motion model and maneuver recognition. In: IEEE International Conference on Intelligent Robots and Systems, pp 4363–4369
Zhou B, Tang X, Wang X (2015) Learning collective crowd behaviors with dynamic pedestrian-agents. Int J Comput Vis 111(1):50–68
Rasmussen CE, Williams CKI (2005) Gaussian processes for machine learning (adaptive computation and machine learning). The MIT Press, Cambridge
Alahi A, Goel K, Ramanathan V, Robicquet A, Fei-Fei L, Savarese S (2016) Social lstm: human trajectory prediction in crowded spaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 961–971
Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprintarXiv:1803.01271
Mohamed A, Qian K, Elhoseiny M, Claudel C (2020) Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 14424–14432
Pellegrini S, Ess A, Schindler K, van Gool L (2009) You’ll never walk alone: modeling social behavior for multi-target tracking. In: 2009 IEEE 12th International Conference on Computer Vision, pp 261–268
Lerner A, Chrysanthou Y, Lischinski D (2007) Crowds by example. Comput Graph Forum 26(3):655–664
Lefèvre S, Laugier C, Ibañez-Guzmán J (2011) Exploiting map information for driver intention estimation at road intersections. In: Intelligent Vehicles Symposium (IV), 2011 IEEE. IEEE, pp 583–588
Danielsson S, Petersson L, Eidehall A (2007) Monte carlo based threat assessment: analysis and improvements. In: Intelligent Vehicles Symposium, 2007 IEEE. IEEE, pp 233–238
Firl J, Stübing H, Huss SA, Stiller C (2012) Predictive maneuver evaluation for enhancement of car-to-x mobility data. In: Intelligent Vehicles Symposium (IV), 2012 IEEE. IEEE, pp 558–564
Kalman RE (1960) A new approach to linear filtering and prediction problems. Trans ASME-J Basic Eng 82(Series D):35–45
Helbing D, Molnr P (1995) Social force model for pedestrian dynamics. Phys Rev E 51:4282–4286
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780.https://doi.org/10.1162/neco.1997.9.8.1735
Gupta A, Johnson J, Fei-Fei L, Savarese S, Alahi A (2018) Social gan: socially acceptable trajectories with generative adversarial networks. IEEE/CVF Confer Comput Vis Pattern Recogn 2018:2255–2264.https://doi.org/10.1109/CVPR.2018.00240
Bi H, Fang Z, Mao T, Wang Z, Deng Z (2019) Joint prediction for kinematic trajectories in vehicle-pedestrian-mixed scenes. In: The IEEE International Conference on Computer Vision (ICCV)
Xu Y, Piao Z, Gao S (2018) Encoding crowd interaction with deep neural network for pedestrian trajectory prediction. In: The IEEE Conference on Computer Vision, and Pattern Recognition (CVPR)
Li K, Eiffert S, Shan M, Gomez-Donoso2 F (2021) Attentional-GCNN: adaptive pedestrian trajectory prediction towards generic autonomous vehicle use cases. In: 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE
Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprintarXiv:1609.02907
Vemula A, Muelling K, Oh J (2018) Social attention: modeling attention in human crowds. In: IEEE International Conference on Robotics and Automation (ICRA) 2017, pp 1–7
Zhang L, She Q, Guo P (2019) Stochastic trajectory prediction with social graph network. arXiv preprintarXiv:1907.10233
Zhang P, Ouyang W, Zhang P, Xue J, Zheng N (2019) Sr-lstm: state refinement for lstm towards pedestrian trajectory prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 12085–12094
Nikhil N, Morris BT (2018) Convolutional neural networks for trajectory prediction. In: Proceedings of the European Conference on Computer Vision (ECCV)
Zamboni S, Tilahun Kefato Z (2022) Pedestrian trajectory prediction with convolutional neural networks. Pattern Recogn 121:108252
Chandra R, Bhattacharya U, Bera A (2021) TraPHic: trajectory prediction in dense and heterogeneous traffic using weighted interactions.arXiv:1812.04767v4 [cs.RO]
Huang Y, Bi H, Li Z, Mao T, Wang Z (2019) Stgat: modeling spatial-temporal interactions for human trajectory prediction. In: The IEEE International Conference on Computer Vision (ICCV)
Rainbow BA, Men Q, Shum HP (2021) Semantics-STGCNN: a semantics-guided spatial-temporal graph convolutional network for multi-class trajectory prediction.arXiv:2108.04740v1 [cs.CV]
Kipf TN, Welling M, Semi-supervised classification with graph convolutional networks. arXiv preprintarXiv:1609.02907
Eiffert S, Li K, Shan M, Worrall S, Sukkarieh S, Nebot E (2020) Probabilistic crowd gan: multimodal pedestrian trajectory prediction using a graph vehicle-pedestrian attention network. IEEE Robot Autom Lett 5(4):5026–5033
Pellegrini S, Ess A, Schindler K, Van Gool L (2009) You’ll never walk alone: modeling social behavior for multi-target tracking. In: 2009 IEEE 12th International Conference on Computer Vision. IEEE, pp 261–268
He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1026–1034
Sadeghian A, Kosaraju V, Sadeghian A, Hirose N, Rezatofighi H, Savarese S (2019) Sophie: an attentive gan for predicting paths compliant to social and physical constraints. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1349–1358
Liang J, Jiang L, Niebles JC, Hauptmann AG, Fei- Fei L (2019) Peeking into the future: predicting future person activities and locations in videos. In: CVPR
Li J, Ma H, Tomizuka M (2019) Conditional generative neural system for probabilistic trajectory prediction. arXiv preprintarXiv:1905.01631
Zhou H, Ren D, Xia H, Fan M, Yang X, Huang H (2021) AST-GNN: an attention-based spatio-temporal graph neural network for Interaction-aware pedestrian trajectory prediction. Neurocomputing 445:298–308
Yao M, Tang J (2021) ma2021deep, deep learning on graphs. Cambridge University Press
Funding
Not applicable.
Author information
Authors and Affiliations
Moulay Ismail University, ENSAM, Meknes, Morocco
Taki Youssef, Elmoukhtar Zemmouri & Anas Bouzid
- Taki Youssef
Search author on:PubMed Google Scholar
- Elmoukhtar Zemmouri
Search author on:PubMed Google Scholar
- Anas Bouzid
Search author on:PubMed Google Scholar
Contributions
The authors confirm their contribution to the paper as follows: study conception and design: Taki Youssef and Elmoukhtar Zemmouri; implementation: Taki Youssef1, Elmoukhtar Zemmouri, and Anas Bouzid; analysis and interpretation of results: Taki Youssef and Elmoukhtar Zemmouri; draft manuscript preparation: Taki Youssef and Elmoukhtar Zemmouri;. All authors reviewed the results and approved the final version of the manuscript.
Corresponding author
Correspondence toTaki Youssef.
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Ethics approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Youssef, T., Zemmouri, E. & Bouzid, A. STM-GCN: a spatiotemporal multi-graph convolutional network for pedestrian trajectory prediction.J Supercomput79, 20923–20937 (2023). https://doi.org/10.1007/s11227-023-05467-x
Accepted:
Published:
Issue Date:
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative