Movatterモバイル変換


[0]ホーム

URL:


Skip to main content

Advertisement

Springer Nature Link
Log in

Online learning for 3D LiDAR-based human detection: experimental analysis of point cloud clustering and classification methods

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

This paper presents a system for online learning of human classifiers by mobile service robots using 3D LiDAR sensors, and its experimental evaluation in a large indoor public space. The learning framework requires a minimal set of labelled samples (e.g. one or several samples) to initialise a classifier. The classifier is then retrained iteratively during operation of the robot. New training samples are generated automatically using multi-target tracking and a pair of “experts” to estimate false negatives and false positives. Both classification and tracking utilise an efficient real-time clustering algorithm for segmentation of 3D point cloud data. We also introduce a new feature to improve human classification in sparse, long-range point clouds. We provide an extensive evaluation of our the framework using a 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments demonstrate the influence of the system components and improved classification of humans compared to the state-of-the-art.

This is a preview of subscription content,log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. Due to privacy issues, currently panoramic images are not included in the dataset.

  2. Although the “group” samples are not used in this paper, these annotations are included in our dataset to be used by other researchers for group tracking or other applications.

  3. Dataset collection is also in progress at University of Lincoln, Czech Technical University in Prague, and University of Technology of Belfort-Montbéliard.

References

  • Arras, K. O., Mozos, O. M., & Burgard, W. (2007). Using boosted features for the detection of people in 2d range data. InProceedings of the 2007 IEEE international conference on robotics and automation (ICRA) (pp. 3402–3407).

  • Bellotto, N., & Hu, H. (2009). Multisensor-based human detection and tracking for mobile service robots.IEEE Transactions on Systems, Man, and Cybernetics Part B,39(1), 167–181.

    Article  Google Scholar 

  • Bellotto, N., & Hu, H. (2010). Computationally efficient solutions for tracking people with a mobile robot: An experimental evaluation of bayesian filters.Autonomous Robots,28, 425–438.

    Article  Google Scholar 

  • Beyer, L., Hermans, A., Linder, T., Arras, K. O., & Leibe, B. (2018). Deep person detection in two-dimensional range data.IEEE Robotics and Automation Letters,3(3), 2726–2733.

    Article  Google Scholar 

  • Bianco, S., Ciocca, G., Napoletano, P., & Schettini, R. (2015). An interactive tool for manual, semi-automatic and automatic video annotation.Computer Vision and Image Understanding,131, 88–99.

    Article  Google Scholar 

  • Bogoslavskyi, I., & Stachniss, C. (2016) Fast range image-based segmentation of sparse 3d laser scans for online operation. InProceedings of the 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 163–169).

  • Chang, C. C., & Lin, C. J. (2011). LIBSVM: A library for support vector machines.ACM Transactions on Intelligent Systems and Technology,2, 1–27.

    Article  Google Scholar 

  • Cortes, C., & Vapnik, V. (1995). Support-vector networks.Machine Learning,20(3), 273–297.

    MATH  Google Scholar 

  • Dequaire, J., Ondruska, P., Rao, D., Wang, D., & Posner, I. (2017). Deep tracking in the wild: End-to-end tracking using recurrent neural networks.International Journal of Robotics Research (IJRR),37, 1–21.

    Google Scholar 

  • Deuge, M.D., Quadros, A., Hung, C., & Douillard, B. (2013). Unsupervised feature learning for classification of outdoor 3d scans. InProceedings of the Australasian conference on robotics and automation (ACRA).

  • Dewan, A., Caselitz, T., Tipaldi, G.D., & Burgard, W. (2016). Motion-based detection and tracking in 3D LiDAR scans. InProceedings of the 2016 IEEE international conference on robotics and automation (ICRA) (pp. 4508–4513).

  • Dondrup, C., Bellotto, N., Jovan, F., & Hanheide, M. (2015). Real-time multisensor people tracking for human-robot spatial interaction. InProceedings of the 2015 IEEE international conference on robotics and automation (ICRA), workshop on machine learning for social robotics.

  • Douillard, B., Underwood, J. P., Kuntz, N., Vlaskine, V., Quadros, A. J., Morton, P., & Frenkel, A. (2011). On the segmentation of 3d LIDAR point clouds. InProceedings of the 2011 IEEE international conference on robotics and automation (ICRA) (pp. 2798–2805).

  • Everingham, M., Gool, L. J. V., Williams, C. K. I., Winn, J. M., & Zisserman, A. (2010). The pascal visual object classes (VOC) challenge.International Journal of Computer Vision,88, 303–338.

    Article  Google Scholar 

  • Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? the KITTI vision benchmark suite. InProceedings of the 2012 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 3354–3361).

  • González, A., Villalonga, G., Xu, J., Vázquez, D., Amores, J., & López, A.M. (2015). Multiview random forest of local experts combining RGB and LIDAR data for pedestrian detection. InProceedings of the 2015 IEEE intelligent vehicles symposium (IV) (pp. 356–361).

  • Häselich, M., Jöbgen, B., Wojke, N., Hedrich, J., & Paulus, D. (2014). Confidence-based pedestrian tracking in unstructured environments using 3d laser distance measurements. InProceedings of the 2014 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 4118–4123).

  • Jafari, O. H., Mitzel, D., & Leibe, B. (2014). Real-time RGB-D based people detection and tracking for mobile robots and head-worn cameras. InProceedings of the 2014 IEEE international conference on robotics and automation (ICRA) (pp. 5636–5643).

  • Julier, S. J., & Uhlmann, J. K. (2004). Unscented filtering and nonlinear estimation.Proceedings of the IEEE,92(3), 401–422.

    Article  Google Scholar 

  • Kalal, Z., Mikolajczyk, K., & Matas, J. (2012). Tracking-learning-detection.IEEE Transactions on Pattern Analysis and Machine Intelligence,34, 1409–1422.

    Article  Google Scholar 

  • Kavasidis, I., Palazzo, S., Salvo, R. D., Giordano, D., & Spampinato, C. (2012). A semi-automatic tool for detection and tracking ground truth generation in videos. InProceedings of the 1st international workshop on visual interfaces for ground truth collection in computer vision applications.

  • Keerthi, S. S., & Lin, C. J. (2003). Asymptotic behaviors of support vector machines with gaussian kernel.Neural Computation,15(7), 1667–1689.

    Article  Google Scholar 

  • Kidono, K., Miyasaka, T., Watanabe, A., Naito, T., & Miura, J. (2011). Pedestrian recognition using high-definition LIDAR. InProceedings of the 2011 IEEE intelligent vehicles symposium (IV) (pp. 405–410).

  • Krajnik, T., Fentanes, J. P., Santos, J. M., & Duckett, T. (2017). Fremen: Frequency map enhancement for long-term mobile robot autonomy in changing environments.IEEE Transactions on Robotics,33, 964–977.

    Article  Google Scholar 

  • Leigh, A., Pineau, J., Olmedo, N. A., & Zhang, H. (2015). Person tracking and following with 2d laser scanners. InProceedings of the 2015 IEEE international conference on robotics and automation (ICRA) (pp. 726–733).

  • Li, K., Wang, X., Xu, Y., & Wang, J. (2016). Density enhancement-based long-range pedestrian detection using 3-d range data.IEEE Transactions on Intelligent Transportation Systems,17, 1368–1380.

    Article  Google Scholar 

  • Li, X. R., & Jilkov, V. P. (2003). Survey of maneuvering target tracking. part i: Dynamic models.IEEE Transactions on Aerospace and Electronic Systems,39(4), 1333–1364.

    Article  Google Scholar 

  • Linder, T., Breuers, S., Leibe, B., & Arras, K. O. (2016). On multi-modal people tracking from mobile platforms in very crowded and dynamic environments. InProceedings of the 2016 IEEE international conference on robotics and automation (ICRA) (pp. 5512–5519).

  • Luber, M., & Arras, K. O. (2013). Multi-hypothesis social grouping and tracking for mobile robots. InProceedings of Robotics: Science and Systems.

  • Mohler, B. J., Thompson, W. B., Creem-Regehr, S. H., Pick, H. L., & Warren, W. H. (2007). Visual flow influences gait transition speed and preferred walking speed.Experimental Brain Research,181(2), 221–228.

    Article  Google Scholar 

  • Moosmann, F., Pink, O., & Stiller, C. (2009). Segmentation of 3d lidar data in non-flat urban environments using a local convexity criterion. InProceedings of the 2009 IEEE intelligent vehicles symposium (IV) (pp. 1931–0587).

  • Munaro, M., & Menegatti, E. (2014). Fast RGB-D people tracking for service robots.Autonomous Robots,37, 227–242.

    Article  Google Scholar 

  • Navarro-Serment, L. E., Mertz, C., & Hebert, M. (2009). Pedestrian detection and tracking using three-dimensional ladar data. InProceedings of the 7th conference on field and service robotics (FSR) (pp. 103–112).

  • Premebida, C., Carreira, J., Batista, J., & Nunes, U. (2014). Pedestrian detection combining RGB and dense LIDAR data. InProceedings of the 2014 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 4112–4117).

  • Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A.Y. (2009). ROS: An open-source robot operating system. InDICRA workshop on open source software.

  • Read, J., Bifet, A., Pfahringer, B., & Holmes, G. (2012). Batch-incremental versus instance-incremental learning in dynamic and evolving data. InProceedings of the eleventh international symposium on intelligent data analysis (IDA 2012) (pp. 313–323).

  • Rusu, R.B. (2009).Semantic 3D object maps for everyday manipulation in human living environments. Ph.D. thesis, Computer Science department, Technische Universitaet Muenchen, Germany.

  • Rusu, R.B., & Cousins, S. (2011). 3D is here: Point Cloud Library (PCL). InProceedings of the 2011 IEEE international conference on robotics and automation (ICRA).

  • Shackleton, J., Voorst, B. V., & Hesch, J. A. (2010). Tracking people with a 360-degree lidar. InProceedings of the Seventh IEEE international conference on advanced video and signal based surveillance (AVSS) (pp. 420–426).

  • Spinello, L., & Arras, K.O. (2011). People detection in RGB-D data. InProceedings of the 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3838–3843).

  • Spinello, L., Luber, M., & Arras, K.O. (2011). Tracking people in 3d using a bottom-up top-down detector. InProceedings of the 2011 IEEE international conference on robotics and automation (ICRA) (pp. 1304–1310).

  • Sun, L., Yan, Z., Mellado, S.M., Hanheide, M., & Duckett, T. (2018). 3DOF pedestrian trajectory prediction learned from long-term autonomous mobile robot deployment data. InIn proceedings of the 2018 IEEE international conference on robotics and automation (ICRA). Brisbane, Australia.

  • Sun, L., Yan, Z., Zaganidis, A., Zhao, C., & Duckett, T. (2018). Recurrent-octomap: Learning state-based map refinement for long-term semantic mapping with 3-d-lidar data.IEEE Robotics and Automation Letters,3(4), 3749–3756.

    Article  Google Scholar 

  • Teichman, A., Levinson, J., & Thrun, S. (2011). Towards 3d object recognition via classification of arbitrary object tracks. InProceedings of the 2011 IEEE international conference on robotics and automation (ICRA) (pp. 4034–4041).

  • Teichman, A., & Thrun, S. (2012). Tracking-based semi-supervised learning.International Journal of Robotics Research (IJRR),31(7), 804–818.

    Article  Google Scholar 

  • Vintr, T., Yan, Z., Duckett, T., & Krajnik, T. (2019). Spatio-temporal representation for long-term anticipation of human presence in service robotics. InProceedings of the 2018 IEEE international conference on robotics and automation (ICRA). Montreal, Canada.

  • Wang, D. Z., & Posner, I. (2015). Voting for voting in online point cloud object detection. InProceedings of Robotics: Science and Systems.

  • Yan, Z., Duckett, T., & Bellotto, N. (2017). Online learning for human classification in 3D LiDAR-based tracking. InIn Proceedings of the 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). Vancouver, Canada.

  • Yan, Z., Sun, L., Duckett, T., & Bellotto, N. (2018). Multisensor online transfer learning for 3d lidar-based human detection with a mobile robot. InProceedings of the 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). Madrid, Spain.

  • Zermas, D., Izzat, I., & Papanikolopoulos, N. (2017). Fast segmentation of 3d point clouds: A paradigm on lidar data for autonomous vehicle applications. InProceedings of the 2018 IEEE international conference on robotics and automation (ICRA). Singapore.

  • Zhou, Y., & Tuzel, O. (2018). Voxelnet: End-to-end learning for point cloud based 3d object detection. InProceedings of the 2018 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 4490–4499).

Download references

Author information

Authors and Affiliations

  1. Distributed Artificial Intelligence and Knowledge Laboratory (CIAD), University of Technology of Belfort-Montbéliard (UTBM), Belfort, France

    Zhi Yan

  2. Lincoln Centre for Autonomous Systems (L-CAS), University of Lincoln, Lincoln, UK

    Tom Duckett & Nicola Bellotto

Authors
  1. Zhi Yan

    You can also search for this author inPubMed Google Scholar

  2. Tom Duckett

    You can also search for this author inPubMed Google Scholar

  3. Nicola Bellotto

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toZhi Yan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant agreement No 645376 (FLOBOT).

Rights and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, Z., Duckett, T. & Bellotto, N. Online learning for 3D LiDAR-based human detection: experimental analysis of point cloud clustering and classification methods.Auton Robot44, 147–164 (2020). https://doi.org/10.1007/s10514-019-09883-y

Download citation

Keywords

Mathematics Subject Classification

Access this article

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Advertisement


[8]ページ先頭

©2009-2025 Movatter.jp