Accepted author manuscript, 170 KB, PDF document
Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Chapter
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Chapter
}
TY - CHAP
T1 - Situational awareness and adherence to the principle of distinction as a necessary condition for lawful autonomy
AU - Suchman, Lucy Alice
PY - 2017
Y1 - 2017
N2 - As a contribution to the CCW’s third informal meeting of experts on lethal autonomous weapon systems (LAWS), this briefing paper focuses on the implications of the requirement of situational awareness for autonomous action – whether by humans, machines or complex human-machine systems. For the purposes of this paper, ‘autonomy’ refers to self-directed action, and more specifically the action-according-to-rule that comprises military discipline. Unlike the algorithmic sense of a rule as that term is used in Artificial Intelligence (AI), military rules always require interpretation in relation to a specific situation, or situational awareness. Focusing on the principle of distinction, I argue that International Humanitarian Law (IHL) presupposes capacities of situational awareness that it does not, and cannot, fully specify. At the same time, autonomy or ‘self-direction’ in the case of machines requires the adequate specification (by human designers) of the conditions under which associated actions should be taken. This requirement for unambiguous specification of condition/action rules marks a crucial difference between autonomy as a legally accountable human capacity, and machine autonomy. The requirement for situational awareness in the context of combat, as a prerequisite for action that adheres to IHL, raises serious doubts regarding the feasibility of lawful autonomy in weapon systems.
AB - As a contribution to the CCW’s third informal meeting of experts on lethal autonomous weapon systems (LAWS), this briefing paper focuses on the implications of the requirement of situational awareness for autonomous action – whether by humans, machines or complex human-machine systems. For the purposes of this paper, ‘autonomy’ refers to self-directed action, and more specifically the action-according-to-rule that comprises military discipline. Unlike the algorithmic sense of a rule as that term is used in Artificial Intelligence (AI), military rules always require interpretation in relation to a specific situation, or situational awareness. Focusing on the principle of distinction, I argue that International Humanitarian Law (IHL) presupposes capacities of situational awareness that it does not, and cannot, fully specify. At the same time, autonomy or ‘self-direction’ in the case of machines requires the adequate specification (by human designers) of the conditions under which associated actions should be taken. This requirement for unambiguous specification of condition/action rules marks a crucial difference between autonomy as a legally accountable human capacity, and machine autonomy. The requirement for situational awareness in the context of combat, as a prerequisite for action that adheres to IHL, raises serious doubts regarding the feasibility of lawful autonomy in weapon systems.
KW - arms control
KW - autonomous weapons
M3 - Chapter
SP - 273
EP - 283
BT - Lethal Autonomous Weapons Systems
A2 - Geiß, Robin
PB - German Federal Foreign Office
CY - Berlin
ER -