Movatterモバイル変換


[0]ホーム

URL:


Skip to main content

Advertisement

Springer Nature Link
Log in

Analysing the Safety of Decision-Making in Autonomous Systems

  • Conference paper
  • First Online:

Abstract

We characterise an autonomous system as one that has the capability to take decisions independently from human control. This independent and autonomous decision making could give rise to new hazards or hazard causes not present in an equivalent human-controlled system, e.g. through lack of human real-world understanding. Despite the increased adoption of autonomous systems there has been a dearth of research in the area of safety analysis and assurance of decision-making for autonomous systems. This paper is intended to be a first step to fill this gap. We compare and contrast the differing causal models of autonomous and non-autonomous systems, and build on existing safety engineering techniques in order to define a process (Decision Safety Analysis) for the analysis of autonomous decision-making. We show, using a real-world example, how this process supports the development of safety requirements to mitigate hazardous scenarios.

This work is funded by the Assuring Autonomy International Programmehttps://www.york.ac.uk/assuring-autonomy.

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 7435
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 9294
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Similar content being viewed by others

References

  1. Safety and ethics of autonomous systems project overview. Technical report, Royal Academy of Engineering (2020)

    Google Scholar 

  2. Adler, R., Feth, P., Schneider, D.: Safety engineering for autonomous vehicles. In: 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshop (DSN-W), pp. 200–205. IEEE (2016)

    Google Scholar 

  3. Burton, S., Habli, I., Lawton, T., McDermid, J., Morgan, P., Porter, Z.: Mind the gaps: assuring the safety of autonomous systems from an engineering, ethical, and legal perspective. Artif. Intell.279, 103201 (2020)

    Article MathSciNet  Google Scholar 

  4. International Electrotechnical Commission: IEC 61882 (2016)

    Google Scholar 

  5. Ericson, C.A., et al.: Hazard Analysis Techniques for System Safety. Wiley, Hoboken (2015)

    Google Scholar 

  6. Gleirscher, M., Johnson, N., Karachristou, P., Calinescu, R., Law, J., Clark, J.: Challenges in the safety-security co-assurance of collaborative industrial robots. arXiv preprintarXiv:2007.11099 (2020)

  7. Guiochet, J.: Hazard analysis of human-robot interactions with HAZOP-UML. Saf. Sci.84, 225–237 (2016)

    Article  Google Scholar 

  8. Harper, C., Caleb-Solly, P.: Towards an ontological framework for environmental survey hazard analysis of autonomous systems. In: SafeAI@ AAAI (2021)

    Google Scholar 

  9. Ishimatsu, T., Leveson, N.G., Thomas, J., Katahira, M., Miyamoto, Y., Nakao, H.: Modeling and hazard analysis using STPA (2010)

    Google Scholar 

  10. Kahneman, D.: Thinking, Fast and Slow. Macmillan (2011)

    Google Scholar 

  11. Klein, G.A., Orasanu, J., Calderwood, R., Zsambok, C.E., et al.: Decision Making in Action: Models and Methods. Ablex Norwood, New Jersey (1993)

    Google Scholar 

  12. Koehler, J.J.: The influence of prior beliefs on scientific judgments of evidence quality. Organ. Behav. Hum. Decis. Process.56(1), 28–55 (1993)

    Article  Google Scholar 

  13. Koopman, P., Fratrik, F.: How many operational design domains, objects, and events? In: SafeAI@AAAI (2019)

    Google Scholar 

  14. Lesage, B.M.J.R., Alexander, R.: SASSI: safety analysis using simulation-based situation coverage for cobot systems. In: Proceedings of SafeComp 2021, York (2021)

    Google Scholar 

  15. Marcus, G., Davis, E.: Rebooting AI: Building Artificial Intelligence We Can Trust. Vintage (2019)

    Google Scholar 

  16. Medrano-Berumen, C., İlhan Akbaş, M.: Validation of decision-making in artificial intelligence-based autonomous vehicles. J. Inf. Telecommun5(1), 83–103 (2021)

    Google Scholar 

  17. Oberheid, H., Hasselberg, A., Söffker, D.: Know your options-analysing human decision making in dynamic task environments with state space methods. Hum. Centred Autom. 285–300 (2011)

    Google Scholar 

  18. Osborne, M.: ISA Robot Safety of Decision Making.https://www-users.cs.york.ac.uk/mo705/isarobot.html

  19. Stansbury, R.S., Agah, A.: A robot decision making framework using constraint programming. Artif. Intell. Rev.38(1), 67–83 (2012)

    Article  Google Scholar 

  20. Walker, G., et al.: Modelling driver decision-making at railway level crossings using the abstraction decomposition space. Cogn. Technol. Work23(2), 225–237 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Assuring Autonomy International Programme, Department of Computer Science, University of York, Deramore Lane, York, YO10 5GH, UK

    Matt Osborne, Richard Hawkins & John McDermid

Authors
  1. Matt Osborne

    You can also search for this author inPubMed Google Scholar

  2. Richard Hawkins

    You can also search for this author inPubMed Google Scholar

  3. John McDermid

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toMatt Osborne.

Editor information

Editors and Affiliations

  1. Fraunhofer IKS, Munich, Germany

    Mario Trapp

  2. University of Erlangen-Nuremberg, Erlangen, Germany

    Francesca Saglietti

  3. University of Erlangen-Nuremberg, Erlangen, Germany

    Marc Spisländer

  4. Thales Deutschland GmbH, Ditzingen, Germany

    Friedemann Bitsch

Rights and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Osborne, M., Hawkins, R., McDermid, J. (2022). Analysing the Safety of Decision-Making in Autonomous Systems. In: Trapp, M., Saglietti, F., Spisländer, M., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2022. Lecture Notes in Computer Science, vol 13414. Springer, Cham. https://doi.org/10.1007/978-3-031-14835-4_1

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 7435
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 9294
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only


[8]ページ先頭

©2009-2025 Movatter.jp