Movatterモバイル変換

[0]ホーム
URL:

Skip to main content

Advertisement

Springer Nature Link
Log in

Automated patient localization in challenging hospital environments

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Telehealth adoption accelerated in the past few years. Telehealth can be offered via simple video consultations or in more complex environments with doctors remotely assisting an operation or remote intensive care unit (ICU) support through a video feed. In these complex settings, it is essential for the doctors joining remotely to understand the remote environment quickly. The most critical information is identifying the patient’s location, which may not be immediately apparent in an operating room or ICU busy with equipment and other clinicians. In this paper, we trained object detection models to find the bounding box of patients in such complex hospital environments using a relatively small number of images (for an object detection task) from standard video cameras without depth or other information beyond a regular image. The best model achieves a high precision-recall area under the curve for this task, even when trained with a small dataset. We describe the process to identify the best model, considerations to deploy the model, and suggest improvements for future work.

This is a preview of subscription content,log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availibility

The dataset used in this study is publicly available athttps://github.com/CAMMA-public/mvor.

References

  1. Alsaidi M, Altaher AS, Jan MT, Altaher A, Salekshahrezaee Z (2022) COVID-19 Classification Using Deep Learning Two-Stage Approach. [cs, eess].https://doi.org/10.48550/arXiv.2211.15817 .http://arxiv.org/abs/2211.15817. Accessed 07 Dec 2023

  2. Shaver J (2022) The State of Telehealth Before and After the COVID-19 Pandemic. Prim Care 49(4):517–530.https://doi.org/10.1016/j.pop.2022.04.002. Accessed 23 Jan 2023

  3. Health Resources and Services Administration (2022) What is telehealth? | Telehealth.HHS.gov.https://telehealth.hhs.gov/patients/understanding-telehealth/. Accessed 31 Jan 2023

  4. Ofoma UR, Maddox TM, Perera C, Waken RJ, Drewry AM, Liu L, Boyle W, Kollef M, Joynt Maddox KE (2021) Characteristics of U.S. Acute Care Hospitals That Have Implemented Telemedicine Critical Care. Crit Care Explor 3(7):0468.https://doi.org/10.1097/CCE.0000000000000468. Accessed 23 Jan 2023

  5. Hata, J.S.: Telemedicine/Virtual ICU: Where Are We and Where Are We Going?14(2), 126 (2018)https://doi.org/10.14797/mdcj-14-2-126 . Number: 2 Publisher: Houston Methodist DeBakey Heart & Vascular Center. Accessed 31 Jan 2023

  6. U.S. Department of Veteran Affairs (2022) Bay Pines VA Implements TeleCritical Care in ICU | VA Bay Pines health care.https://www.va.gov/bay-pines-health-care/news-releases/bay-pines-va-implements-telecritical-care-in-icu/. Accessed 24 Jan 2023

  7. Madani A, Hirpara D, Chadi SA, Dhar P, Okrainec A (2021) Leveraging Videoconferencing Technology to Augment Surgical Training During a Pandemic. Ann Surg Open 2(2):035.https://doi.org/10.1097/AS9.0000000000000035. Accessed 01 July 2023

  8. Defense Visual Information Distribution Service (2015) The online doc is in... Lt. Cmdr. Josephine Nguyen.https://nara.getarchive.net/media/the-online-doc-is-in-lt-cmdr-josephine-nguyen-fc81a2. Accessed 29 Jan 2023

  9. Srivastav V, Issenhuth T, Kadkhodamohammadi A, Mathelin M, Gangi A, Padoy N (2021) MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimation. [cs].https://doi.org/10.48550/arXiv.1808.08180 .http://arxiv.org/abs/1808.08180. Accessed 01 Feb 2023

  10. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol 1.https://doi.org/10.1109/CVPR.2001.990517 . ISSN: 1063-6919

  11. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE Computer society conference on computer vision and pattern recognition (CVPR’05), vol 1. pp 886–8931.https://doi.org/10.1109/CVPR.2005.177 . ISSN: 1063-6919

  12. Zou Z, Chen K, Shi Z, Guo Y, Ye J (2023) Object Detection in 20 Years: a Survey. Proc IEEE 111(3):257–276.https://doi.org/10.1109/JPROC.2023.3238524. Conference Name: Proceedings of the IEEE

  13. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. [cs].https://doi.org/10.48550/arXiv.1311.2524 .http://arxiv.org/abs/1311.2524. Accessed 13 Feb 2023

  14. Girshick R (2015) Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV). pp 1440–1448.https://doi.org/10.1109/ICCV.2015.169 . ISSN: 2380-7504

  15. Ren S, He K, Girshick R, Sun, J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol 28. Curran Associates, Inc.https://papers.nips.cc/paper/2015/hash/14bfa6bb14875e45bba028a21ed38046-Abstract.html. Accessed 20 Mar 2023

  16. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection, pp 779–788 (2016).https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Redmon_You_Only_Look_CVPR_2016_paper.html. Accessed 13 Feb 2023

  17. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: Single Shot MultiBox Detector, vol 9905. pp 21–37.https://doi.org/10.1007/978-3-319-46448-0_2. arXiv:1512.02325 [cs].http://arxiv.org/abs/1512.02325. Accessed 13 Feb 2023

  18. Sharma V, Mir RN (2020) A comprehensive and systematic look up into deep learning based object detection techniques: a review. Comput Sci Rev 38:100301.https://doi.org/10.1016/j.cosrev.2020.100301. Accessed 15 Feb 2023

  19. Wong K-Y (2023) YOLOv7 hyperparameters from scratch. original-date: 2022-07-06T15:14:06Z.https://github.com/WongKinYiu/yolov7/blob/2fdc7f14395f6532ad05fb3e6970150a6a83d290/data/hyp.scratch.custom.yaml. Accessed 28 Feb 2023

  20. Redmon J, Farhadi A (2017) YOLO9000: Better, Faster, Stronger. pp 7263–7271.https://openaccess.thecvf.com/content_cvpr_2017/html/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.html. Accessed 13 Feb 2023

  21. Redmon J, Farhadi A (2018) YOLOv3: An Incremental Improvement. arXiv. arXiv:1804.02767 [cs].https://doi.org/10.48550/arXiv.1804.02767 .http://arxiv.org/abs/1804.02767. Accessed 13 Feb 2023

  22. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) YOLOv4: Optimal Speed and Accuracy of Object Detection. [cs, eess].https://doi.org/10.48550/arXiv.2004.10934 .http://arxiv.org/abs/2004.10934. Accessed 13 Feb 2023

  23. Jocher G, Chaurasia A, Stoken A, Borovec J, NanoCode012, Kwon Y, Michael K, TaoXie, Fang J, imyhxy, Lorna, Yifu Z, Wong C, Abhiram V, Montes D, Wang Z, Fati C, Nadar J, Laughing, UnglvKitDe, Sonck V, tkianai, yxNONG, Skalski P, Hogan A, Nair D, Strobel M, Jain M (2022) ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation. Zenodo.https://doi.org/10.5281/zenodo.7347926 .https://zenodo.org/record/7347926. Accessed 13 Feb 2023

  24. Jocher G, Chaurasia A, Qiu J (2023) YOLO by Ultralytics. original-date: 2022-09-11T16:39:45Z.https://github.com/ultralytics/ultralytics. Accessed 15 Feb 2023

  25. Jan MT, Hashemi A, Jang J, Yang K, Zhai J, Newman D, Tappen R, Furht B (2013) Non-intrusive drowsiness detection techniques and their application in detecting early dementia in older drivers. In: Arai K (ed) Proceedings of the future technologies conference (FTC) 2022, vol 2. Lecture Notes in Networks and Systems. Springer, Cham, pp 776–796.https://doi.org/10.1007/978-3-031-18458-1_53

  26. Jan MT, Moshfeghi S, Conniff JW, Jang J, Yang K, Zhai J, Rosselli M, Newman D, Tappen R, Furht B (2022) Methods and Tools for Monitoring Driver’s Behavior. In: 2022 International conference on computational science and computational intelligence (CSCI). IEEE, Las Vegas, pp 1269–1273.https://doi.org/10.1109/CSCI58124.2022.00228 .https://ieeexplore.ieee.org/document/10216710/. Accessed 07 Dec 2023

  27. Moshfeghi S, Jan MT, Conniff J, Ghoreishi, SGA, Jang J, Furht B, Yang K, Rosselli, M, Newman, D., Tappen, R, Smith, D (2023) In: Vehicle sensing and data analysis for older drivers with mild cognitive impairment. [cs].https://doi.org/10.48550/arXiv.2311.09273 .http://arxiv.org/abs/2311.09273. Accessed 07 Dec 2023

  28. Ghoreishi SGA, Moshfeghi S, Jan MT, Conniff J, Yang K, Jang J, Furht B, Tappen R, Newman D, Rosselli M, Zhai J (2023) Anomalous Behavior Detection in Trajectory Data of Older Drivers. [cs].https://doi.org/10.48550/arXiv.2311.17822 .http://arxiv.org/abs/2311.17822. Accessed 07 Dec 2023

  29. Li L, Huang W, Gu IYH, Tian Q (2003) Foreground object detection from videos containing complex background. In: Proceedings of the eleventh acm international conference on multimedia. MULTIMEDIA ’03. pp 2–10. Association for Computing Machinery, New York.https://doi.org/10.1145/957013.957017 . Accessed 02 July 2023

  30. Chen Q, Liu T, Shang Y, Shao Z, Ding H (2019) Salient object detection: integrate salient features in the deep learning framework. IEEE Access 7:152483–152492.https://doi.org/10.1109/ACCESS.2019.2948062. Conference Name: IEEE Access

  31. Zhu D, Xu G, Zhou J, Di E, Li M (2021) Object detection in complex road scenarios: improved YOLOv4-tiny algorithm. In: 2021 2nd Information communication technologies conference (ICTC). pp 75–80.https://doi.org/10.1109/ICTC51749.2021.9441643

  32. Tao T, Dong D, Huang S, Chen W, Yang L (2020) Object detection-based license plate localization and recognition in complex environments. Transp Res Rec 2674(12):212–223.https://doi.org/10.1177/0361198120954202. Publisher: SAGE Publications Inc. Accessed 02 July 2023

  33. Zhang C, Kang F, Wang Y (2022) An improved apple object detection method based on lightweight YOLOv4 in complex backgrounds. Remote Sens 14(17):4150.https://doi.org/10.3390/rs14174150. Number: 17 Publisher: Multidisciplinary Digital Publishing Institute. Accessed 02 July 2023

  34. Chun LZ, Dian L, Zhi JY, Jing W, Zhang C (2020) YOLOv3: face detection in complex environments. Int J Comput Intell Syst 13(1):1153–1160.https://doi.org/10.2991/ijcis.d.200805.002. Publisher: Atlantis Press. Accessed 02 July 2023

  35. Ma AJ, Rawat N, Reiter A, Shrock C, Zhan A, Stone A, Rabiee A, Griffin S, Needham DM, Saria S (2017) Measuring patient mobility in the icu using a novel noninvasive sensor. Crit Care Med 45(4):630.https://doi.org/10.1097/CCM.0000000000002265

  36. Yeung S, Rinaldo F, Jopling J, Liu B, Mehra R, Downing NL, Guo M, Bianconi GM, Alahi A, Lee J, Campbell B, Deru K, Beninati W, Fei-Fei L, Milstein A (2019) A computer vision system for deep learning-based detection of patient mobilization activities in the ICU. NPJ Digit Med 2(1), 1–5 (2019)https://doi.org/10.1038/s41746-019-0087-z. Number: 1 Publisher: Nature Publishing Group. Accessed 23 Jan 2023

  37. Reiter A, Ma A, Rawat N, Shrock C, Saria S (2016) Process Monitoring in the Intensive Care Unit: Assessing Patient Mobility Through Activity Analysis with a Non-Invasive Mobility Sensor. In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W (eds) Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016. Lecture Notes in Computer Science. Springer, Cham, pp 482–490.https://doi.org/10.1007/978-3-319-46720-7_56 .https://link.springer.com/chapter/10.1007/978-3-319-46720-7_56

  38. European Union (2016) Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance). Legislative Body: EP, CONSIL.http://data.europa.eu/eli/reg/2016/679/oj/eng. Accessed 20 Mar 2023

  39. Sen. Kassebaum, Nancy Landon [R-KS] (1996) S.1028 - 104th Congress (1995-1996): Health Insurance Reform Act of 1995. Archive Location: 04/23/1996.http://www.congress.gov/. Accessed 20 Mar 2023

  40. Alsaidi M, Jan MT, Altaher A, Zhuang H, Zhu X (2023) Tackling the class imbalanced dermoscopic image classification using data augmentation and GAN. Multimed Tools Appl.https://doi.org/10.1007/s11042-023-17067-1. Accessed 24 Nov 2023

  41. Lin T-Y, Maire M, Belongie S, Bourdev L, Girshick R, Hays J, Perona P, Ramanan D, Zitnick CL, Dollár P (2015) Microsoft COCO: Common Objects in Context. [cs].https://doi.org/10.48550/arXiv.1405.0312 .http://arxiv.org/abs/1405.0312. Accessed 24 Feb 2023

  42. Padilla R, Passos WL, Dias TLB, Netto SL, Silva EAB (2021) A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit. Electronics 10(3):279.https://doi.org/10.3390/electronics10030279. Number: 3 Publisher: Multidisciplinary Digital Publishing Institute. Accessed 14 Feb 2023

  43. Kasper-Eulaers M, Hahn N, Berger S, Sebulonsen T, Myrland O, Kummervold PE (2021) Short Communication: Detecting Heavy Goods Vehicles in Rest Areas in Winter Conditions Using YOLOv5. Algorithms 14(4):114.https://doi.org/10.3390/a14040114. Number: 4 Publisher: Multidisciplinary Digital Publishing Institute. Accessed 10 Feb 2023

  44. Wu B, Pang C, Zeng X, Hu X (2022) ME-YOLO: Improved YOLOv5 for detecting medical personal protective equipment. Appl Sci 12(23):11978.https://doi.org/10.3390/app122311978. Number: 23 Publisher: Multidisciplinary Digital Publishing Institute. Accessed 10 Feb 2023

  45. Wang C-Y, Bochkovskiy A, Liao H-YM (2021) Scaled-YOLOv4: scaling cross stage partial network. In: 2021 IEEE/CVF Conference on computer vision and pattern recognition (CVPR). pp 13024–13033.https://doi.org/10.1109/CVPR46437.2021.01283 . ISSN: 2575-7075

  46. Karpathy A (2019) A recipe for training neural networks.http://karpathy.github.io/2019/04/25/recipe/. Accessed 22 Feb 2023

  47. Godbole V, Dahl GE, Gilmer J, Shallue CJ, Nado Z (2023) Deep Learning Tuning Playbook. Google Research. original-date: 2023-01-18T23:32:32Z.https://github.com/google-research/tuning_playbook. Accessed 22 Feb 2023

  48. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? | Proceedings of the 27th international conference on neural information processing systems - vol 2.https://dl.acm.org/doi/10.5555/2969033.2969197. Accessed 24 Feb 2023

  49. Jocher G (2021) YOLOv5 Study: mAP vs Batch-Size\(\cdot \) Discussion #2452\(\cdot \) ultralytics/yolov5.https://github.com/ultralytics/yolov5/discussions/2452. Accessed 23 Feb 2023

  50. Wong K-Y (2023) Official YOLOv7. original-date: 2022-07-06T15:14:06Z.https://github.com/WongKinYiu/yolov7. Accessed 22 Feb 2023

  51. Wong K-Y (2023) YOLOv7 hyperparameters P5 model. original-date: 2022-07-06T15:14:06Z.https://github.com/WongKinYiu/yolov7/blob/2fdc7f14395f6532ad05fb3e6970150a6a83d290/data/hyp.scratch.p5.yaml. Accessed 28 Feb 2023

  52. Zhang H, Cisse M, Dauphin YN, Lopez-Paz D (2018) mixup: Beyond Empirical Risk Minimization. [cs, stat].https://doi.org/10.48550/arXiv.1710.09412.http://arxiv.org/abs/1710.09412. Accessed 28 Feb 2023

  53. Raghu M, Zhang C, Kleinberg J, Bengio S (2019) Transfusion: Understanding Transfer Learning for Medical Imaging. [cs, stat].https://doi.org/10.48550/arXiv.1902.07208.http://arxiv.org/abs/1902.07208. Accessed 26 Feb 2023

  54. Glenn J (2022) Multi-GPU Training\(\cdot \) Issue #475\(\cdot \) ultralytics/yolov5.https://github.com/ultralytics/yolov5/issues/475. Accessed 27 Feb 2023

  55. Ultralytics (2022) yolov7/metrics.py at 55b90e111984dd85e7eed327e9ff271222aa8b82\(\cdot \) WongKinYiu/yolov7.https://github.com/WongKinYiu/yolov7/blob/55b90e111984dd85e7eed327e9ff271222aa8b82/utils/metrics.py. Accessed 27 Feb 2023

  56. Glenn J (2020) Hyperparameter Evolution\(\cdot \) Issue #607\(\cdot \) ultralytics/yolov5.https://github.com/ultralytics/yolov5/issues/607. Accessed 27 Feb 2023

  57. Chen JYC, Thropp JE (2007) Review of low frame rate effects on human performance. IEEE Trans Syst Man Cybern Part A: Syst Hum 37(6):1063–1076.https://doi.org/10.1109/TSMCA.2007.904779. Conference Name: IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans

  58. Mokari H, Firouzmand E, Sharifi I, Doustmohammadi A (2023) Resilient control strategy and attack detection on platooning of smart vehicles under DoS attack. ISA Trans.https://doi.org/10.1016/j.isatra.2023.11.019. Accessed 24 Nov 2023

  59. Mokari H, Firouzmand E, Sharifi I, Doustmohammadi A (2022) Deception attack detection and resilient control in platoon of smart vehicles. In: 2022 30th International conference on electrical engineering (ICEE). pp 29–35.https://doi.org/10.1109/ICEE55646.2022.9827376 . ISSN: 2642-9527.https://ieeexplore.ieee.org/abstract/document/9827376?casa_token=cPr3BZs34EEAAAAA:uxKVNf--S2iR-6xCCaEtDigZX0ueATmcFCmijPpUCdZmcpJacRI3KjXbld088rIxick5i6yHaw. Accessed 24 Nov 2023

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Florida Atlantic University, 777 Glades Road, Boca Raton, 33431, FL, USA

    Muhammad Tanveer Jan, Christian Garbin, Oge Marques & Hari Kalva

  2. Atos Unify, Boca Raton, 33431, FL, USA

    Christian Garbin & Johannes Ruetschi

Authors
  1. Muhammad Tanveer Jan

    You can also search for this author inPubMed Google Scholar

  2. Christian Garbin

    You can also search for this author inPubMed Google Scholar

  3. Johannes Ruetschi

    You can also search for this author inPubMed Google Scholar

  4. Oge Marques

    You can also search for this author inPubMed Google Scholar

  5. Hari Kalva

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toMuhammad Tanveer Jan.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jan, M., Garbin, C., Ruetschi, J.et al. Automated patient localization in challenging hospital environments.Multimed Tools Appl83, 63439–63457 (2024). https://doi.org/10.1007/s11042-024-18118-x

Download citation

Keywords

Access this article

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Advertisement

[8]ページ先頭
©2009-2025 Movatter.jp