Movatterモバイル変換

[0]ホーム
URL:

Skip to main content

Advertisement

Springer Nature Link
Log in

Detecting Inner Emotions from Video Based Heart Rate Sensing

  • Conference paper
  • First Online:

Part of the book series:Lecture Notes in Computer Science ((LNAI,volume 10363))

Included in the following conference series:

Abstract

Recognizing human emotion by computer vision is an interesting and challenging problem. In particular, the reading of inner emotions, has received limited attention. In this paper, we use a remote video-based heart rate sensing technique to obtain physiological data that provides an indication of a person’s inner emotions. This method allows for contactless estimates of heart rate data while the subject is watching emotionally stimulating video clips. We also compare against a wearable heart rate sensor to validate the usefulness of the proposed remote heart rate reading framework. We found that the reading of heart rates of a subject effectively detects the inner emotional reactions of human subjects while they were watching funny and horror videos—despite little to no facial expressions at times. These findings are validated from the reading of heart rates for 40 subjects with our vision-based method compared against conventional wearable sensors. We also find that the change in heart rate along with emotionally stimulating content is statistically significant and our remote sensor is well correlated with the wearable contact sensor.

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 5719
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 7149
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

References

  1. Cohen, I., Sebe, N., Garg, A., Chen, L.S., Huang, T.S.: Facial expression recognition from video sequences. In: Proceedings of 2002 IEEE International Conference on Multimedia and Expo, ICME 2002 (2002)

    Google Scholar 

  2. Zhang, S., Zhao, X., Lei, B.: Facial expression recognition based on local binary patters and local fisher discriminant analysis. In: PMC (2011)

    Google Scholar 

  3. Stern, R.M., Ray, W.J., Quigley, K.S.: Psychophysiological Recording, 2nd edn. Oxford University Press, New York (2001)

    Google Scholar 

  4. Stern, J.A.: Toward a Definition of Psychophysiology, vol. 1, no. 1. Version of Record online: 30 January 2007.http://onlinelibrary.wiley.com/doi/10.1111/j.1469-8986.1964.tb02626.x/abstract

  5. Kim, K.H., Bang, S.W., Kim, S.R.: Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput.42, 419–427 (2004)

    Article  Google Scholar 

  6. Zong, C., Chetouani, M.: Hilbert-huang transform based physiological signals analysis for emotion recognition. In: IEEE International Symposium on Signal Processing and Information Technology, pp. 334–339 (2009)

    Google Scholar 

  7. Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell.23(10), 1176–1189 (2001)

    Article  Google Scholar 

  8. Chanel, G., Kierkels, J.J.M., Soleymani, M., Pun, T.: Short-term emotion assessment in a recall paradigm. Int. J. Hum.-Comput. Stud.67, 607–627 (2009)

    Article  Google Scholar 

  9. Li, X., Chen, J., Zhao, G., Pietkainen, M.: Remote heart rate measurement from face videos under realistic situations. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, pp. 4264–4271 (2014)

    Google Scholar 

  10. Monkaresi, H., Hussain, M.S., Calvo, R.A.: Using remote heart rate measurement for affect detection. In: Florida Artificial Intelligence Research Society Conference, the Twenty-Seventh International Flairs Conference, pp. 119–123 (2014)

    Google Scholar 

  11. Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., Freeman, W.T.: Eulerian video magnification for revealing subtle changes in the world. ACM Trans. (2012)

    Google Scholar 

  12. Kwon, S., Kim, H., Park, K.S.: Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2174–2177, August 2012

    Google Scholar 

  13. Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Expr.16, 21434–21445 (2008)

    Article  Google Scholar 

  14. Lam, A., Kuno, Y.: Robust heart rate measurement from video using select random patches. In: ICCV 2015, pp. 3640–3648 (2015)

    Google Scholar 

  15. Poh, M.-Z., McDuff, D., Picard, R.: Advancements in noncontact, multiparameter physio-logical measurements using a webcam. IEEE Trans. Biomed. Eng.58(1), 7–11 (2011)

    Article  Google Scholar 

  16. Li, X., Chen, J., Zhao, G., Pietikainen, M.: Remote heart rate measurement from face videos under realistic situations. In: IEEE Computer Vision and Pattern Recognition (CVPR), pp. 4264–4271 (2014)

    Google Scholar 

  17. Tulyakov, S., Alameda-Pineda, X., Ricci, E., Yin, L., Cohn, J.F., Sebe, N.: Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  18. Kaewkannate, K., Kim, S.: A comparison of wearable fitness devices. BMC Public Health16(1), 433 (2016)

    Article  Google Scholar 

  19. Evenson, K.R., Goto, M.M., Furberg, R.D.: Systematic review of the validity and reliability of consumer-wearable activity trackers. Int. J. Behav. Nutr. Phys. Act.12(1), 159 (2015)

    Article  Google Scholar 

  20. Balakrishnan, G., Durand, F., Guttag, J.: Detecting pulse from head motions in video. In: IEEE Computer Vision and Pattern Recognition (CVPR) (2013)

    Google Scholar 

Download references

Acknowledgments

This work was supported by JSPS KAKENHI Grant Number 26540131 and the Tateishi Science and Technology Foundation.

Author information

Authors and Affiliations

  1. Graduate School of Science and Engineering, Saitama University, Saitama, Japan

    Keya Das, Sarwar Ali, Koyo Otsu, Hisato Fukuda, Antony Lam, Yoshinori Kobayashi & Yoshinori Kuno

Authors
  1. Keya Das

    You can also search for this author inPubMed Google Scholar

  2. Sarwar Ali

    You can also search for this author inPubMed Google Scholar

  3. Koyo Otsu

    You can also search for this author inPubMed Google Scholar

  4. Hisato Fukuda

    You can also search for this author inPubMed Google Scholar

  5. Antony Lam

    You can also search for this author inPubMed Google Scholar

  6. Yoshinori Kobayashi

    You can also search for this author inPubMed Google Scholar

  7. Yoshinori Kuno

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toKeya Das.

Editor information

Editors and Affiliations

  1. Tongji University, Shanghai, China

    De-Shuang Huang

  2. Liverpool John Moores University, Liverpool, United Kingdom

    Abir Hussain

  3. Inha University, Incheon, Korea (Republic of)

    Kyungsook Han

  4. Indian Institute of Technology Madras, Chennai, India

    M. Michael Gromiha

Rights and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Das, K.et al. (2017). Detecting Inner Emotions from Video Based Heart Rate Sensing. In: Huang, DS., Hussain, A., Han, K., Gromiha, M. (eds) Intelligent Computing Methodologies. ICIC 2017. Lecture Notes in Computer Science(), vol 10363. Springer, Cham. https://doi.org/10.1007/978-3-319-63315-2_5

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 5719
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 7149
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

[8]ページ先頭
©2009-2025 Movatter.jp