- Matt Osborne ORCID:orcid.org/0000-0002-9941-453111,
- Richard Hawkins ORCID:orcid.org/0000-0001-7347-341311 &
- John McDermid ORCID:orcid.org/0000-0003-4745-427211
Part of the book series:Lecture Notes in Computer Science ((LNCS,volume 13414))
Included in the following conference series:
1399Accesses
Abstract
We characterise an autonomous system as one that has the capability to take decisions independently from human control. This independent and autonomous decision making could give rise to new hazards or hazard causes not present in an equivalent human-controlled system, e.g. through lack of human real-world understanding. Despite the increased adoption of autonomous systems there has been a dearth of research in the area of safety analysis and assurance of decision-making for autonomous systems. This paper is intended to be a first step to fill this gap. We compare and contrast the differing causal models of autonomous and non-autonomous systems, and build on existing safety engineering techniques in order to define a process (Decision Safety Analysis) for the analysis of autonomous decision-making. We show, using a real-world example, how this process supports the development of safety requirements to mitigate hazardous scenarios.
This work is funded by the Assuring Autonomy International Programmehttps://www.york.ac.uk/assuring-autonomy.
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 7435
- Price includes VAT (Japan)
- Softcover Book
- JPY 9294
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Safety and ethics of autonomous systems project overview. Technical report, Royal Academy of Engineering (2020)
Adler, R., Feth, P., Schneider, D.: Safety engineering for autonomous vehicles. In: 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshop (DSN-W), pp. 200–205. IEEE (2016)
Burton, S., Habli, I., Lawton, T., McDermid, J., Morgan, P., Porter, Z.: Mind the gaps: assuring the safety of autonomous systems from an engineering, ethical, and legal perspective. Artif. Intell.279, 103201 (2020)
International Electrotechnical Commission: IEC 61882 (2016)
Ericson, C.A., et al.: Hazard Analysis Techniques for System Safety. Wiley, Hoboken (2015)
Gleirscher, M., Johnson, N., Karachristou, P., Calinescu, R., Law, J., Clark, J.: Challenges in the safety-security co-assurance of collaborative industrial robots. arXiv preprintarXiv:2007.11099 (2020)
Guiochet, J.: Hazard analysis of human-robot interactions with HAZOP-UML. Saf. Sci.84, 225–237 (2016)
Harper, C., Caleb-Solly, P.: Towards an ontological framework for environmental survey hazard analysis of autonomous systems. In: SafeAI@ AAAI (2021)
Ishimatsu, T., Leveson, N.G., Thomas, J., Katahira, M., Miyamoto, Y., Nakao, H.: Modeling and hazard analysis using STPA (2010)
Kahneman, D.: Thinking, Fast and Slow. Macmillan (2011)
Klein, G.A., Orasanu, J., Calderwood, R., Zsambok, C.E., et al.: Decision Making in Action: Models and Methods. Ablex Norwood, New Jersey (1993)
Koehler, J.J.: The influence of prior beliefs on scientific judgments of evidence quality. Organ. Behav. Hum. Decis. Process.56(1), 28–55 (1993)
Koopman, P., Fratrik, F.: How many operational design domains, objects, and events? In: SafeAI@AAAI (2019)
Lesage, B.M.J.R., Alexander, R.: SASSI: safety analysis using simulation-based situation coverage for cobot systems. In: Proceedings of SafeComp 2021, York (2021)
Marcus, G., Davis, E.: Rebooting AI: Building Artificial Intelligence We Can Trust. Vintage (2019)
Medrano-Berumen, C., İlhan Akbaş, M.: Validation of decision-making in artificial intelligence-based autonomous vehicles. J. Inf. Telecommun5(1), 83–103 (2021)
Oberheid, H., Hasselberg, A., Söffker, D.: Know your options-analysing human decision making in dynamic task environments with state space methods. Hum. Centred Autom. 285–300 (2011)
Osborne, M.: ISA Robot Safety of Decision Making.https://www-users.cs.york.ac.uk/mo705/isarobot.html
Stansbury, R.S., Agah, A.: A robot decision making framework using constraint programming. Artif. Intell. Rev.38(1), 67–83 (2012)
Walker, G., et al.: Modelling driver decision-making at railway level crossings using the abstraction decomposition space. Cogn. Technol. Work23(2), 225–237 (2021)
Author information
Authors and Affiliations
Assuring Autonomy International Programme, Department of Computer Science, University of York, Deramore Lane, York, YO10 5GH, UK
Matt Osborne, Richard Hawkins & John McDermid
- Matt Osborne
You can also search for this author inPubMed Google Scholar
- Richard Hawkins
You can also search for this author inPubMed Google Scholar
- John McDermid
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toMatt Osborne.
Editor information
Editors and Affiliations
Fraunhofer IKS, Munich, Germany
Mario Trapp
University of Erlangen-Nuremberg, Erlangen, Germany
Francesca Saglietti
University of Erlangen-Nuremberg, Erlangen, Germany
Marc Spisländer
Thales Deutschland GmbH, Ditzingen, Germany
Friedemann Bitsch
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Osborne, M., Hawkins, R., McDermid, J. (2022). Analysing the Safety of Decision-Making in Autonomous Systems. In: Trapp, M., Saglietti, F., Spisländer, M., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2022. Lecture Notes in Computer Science, vol 13414. Springer, Cham. https://doi.org/10.1007/978-3-031-14835-4_1
Download citation
Published:
Publisher Name:Springer, Cham
Print ISBN:978-3-031-14834-7
Online ISBN:978-3-031-14835-4
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative