Steven Umbrello

Autonomous weapons systems, often referred to as ‘killer robots’, have been a hallmark of popular imagination for decades. However, with the inexorable advance of artificial intelligence systems (AI) and robotics, killer robots are quickly becoming a reality. These lethal technologies can learn, adapt, and potentially make life and death decisions on the battlefield with little-to-no human involvement. This naturally leads to not only legal but ethical concerns as to whether we can meaningful control such machines, and if so, then how. Such concerns are made even more poignant by the ever-present fear that something may go wrong, and the machine may carry out some action(s) violating the ethics or laws of war. Researchers, policymakers, and designers are caught in the quagmire of how to approach these highly controversial systems and to figure out what exactly it means to have meaningful human control over them, if at all. In Designed for Death, Dr Steven Umbrello aims to not only produce a realistic but also an optimistic guide for how, with human values in mind, we can begin to design killer robots. Drawing on the value sensitive design (VSD) approach to technology innovation, Umbrello argues that context is king and that a middle path for designing killer robots is possible if we consider both ethics and design as fundamentally linked. Umbrello moves beyond the binary debates of whether or not to prohibit killer robots and instead offers a more nuanced perspective of which types of killer robots may be both legally and ethically acceptable, when they would be acceptable, and how to design for them.

This Open Access book shows how value sensitive design (VSD), responsible innovation, and comprehensive engineering can guide the rapid development of technological responses to the COVID-19 crisis. Responding to the ethical challenges of data-driven technologies and other tools requires thinking about values in the context of a pandemic as well as in a post-COVID world. Instilling values must be prioritized from the beginning, not only in the emergency response to the pandemic, but in how to proceed with new societal precedents materializing, new norms of health surveillance, and new public health requirements. The contributors with expertise in VSD bridge the gap between ethical acceptability and social acceptance. By addressing ethical acceptability and societal acceptance together, VSD guides COVID-technologies in a way that strengthens their ability to fight the virus, and outlines pathways for the resolution of moral dilemmas. This volume provides diachronic reflections on the crisis response to address long-term moral consequences in light of the post-pandemic future. Both contact-tracing apps and immunity passports must work in a multi-system environment, and will be required to succeed alongside institutions, incentive structures, regulatory bodies, and current legislation. This text appeals to students, researchers and importantly, professionals in the field.

Most engineers Fwork within social structures governing and governed by a set of values that primarily emphasise economic concerns. The majority of innovations derive from these loci. Given the effects of these innovations on various communities, it is imperative that the values they embody are aligned with those societies. Like other transformative technologies, artificial intelligence systems can be designed by a single organisation but be diffused globally, demonstrating impacts over time. This paper argues that in order to design for this broad stakeholder group, engineers must adopt a systems thinking approach that allows them to understand the sociotechnicity of artificial intelligence systems across sociocultural domains. It claims that value sensitive design, and envisioning cards in particular, provides a solid first step towards helping designers harmonise human values, understood across spatiotemporal boundaries, with economic values, rather than the former coming at the opportunity cost of the latter.

Although artificial intelligence has been given an unprecedented amount of attention in both the public and academic domains in the last few years, its convergence with other transformative technologies like cloud computing, robotics, and augmented/virtual reality is predicted to exacerbate its impacts on society. The adoption and integration of these technologies within industry and manufacturing spaces is a fundamental part of what is called Industry 4.0, or the Fourth Industrial Revolution. The impacts of this paradigm shift on the human operators who continue to work alongside and symbiotically with these technologies in the industry bring with it novel ethical issues. Therefore, how to design these technologies for human values becomes the critical area of intervention. This paper takes up the case study of robotic AI-based assistance systems to explore the potential value implications that emerge due to current design practices and use. The design methodology known as Value Sensitive Design (VSD) is proposed as a sufficient starting point for designing these technologies for human values to address these issues.

There is a growing body of scholarship that is addressing the ethics, in particular, the bioethics of space travel and colonisation. Naturally, a variety of perspectives concerning the ethical issues and moral permissibility of different technological strategies for confronting the rigours of space travel and colonisation have emerged in the debate. Approaches ranging from genetically enhancing human astronauts to modifying the environments of planets to make them hospitable have been proposed as methods. This paper takes a look at a critique of human bioenhancement proposed by Mirko Garasic who argues that the bioenhancement of human astronauts is not only functional but necessary and thus morally permissible. However, he further claims that the bioethical arguments proposed for the context of space do not apply to the context of Earth. This paper forwards three arguments for how Garasic’s views are philosophically dubious: (1) when he examines our responsibility towards future generations he refers to a moral principle (which we will call the principle of mere survival) which, besides being vague, is not morally acceptable; (2) the idea that human bioenhancement is not natural is not only debatable but morally irrelevant; and (3) it is not true that the situations that may arise in space travel cannot occur on Earth. We conclude that not only is the (bio)enhancement of humans on Earth permissible but perhaps even necessary in certain circumstances.

The impacts that AI and robotics systems can and will have on our everyday lives are already making themselves manifest. However, there is a lack of research on the ethical impacts and means for amelioration regarding AI and robotics within tourism and hospitality. Given the importance of designing technologies that cross national boundaries, and given that the tourism and hospitality industry is fundamentally predicated on multicultural interactions, this is an area of research and application that requires particular attention. Specifically, tourism and hospitality have a range of context-unique stakeholders that need to be accounted for in the salient design of AI systems is to be achieved. This paper adopts a stakeholder approach to develop the conceptual framework to centralize human values in designing and deploying AI and robotics systems in tourism and hospitality. The conceptual framework includes several layers – ‘Human-human-AI’ interaction level, direct and indirect stakeholders, and the macroenvironment. The ethical issues on each layer are outlined as well as some possible solutions to them. Additionally, the paper develops a research agenda on the topic.

The pervasiveness of AI-empowered technologies across multiple sectors has led to drastic changes concerning traditional social practices and how we relate to one another. Moreover, market-driven Big Tech corporations are now entering public domains, and concerns have been raised that they may even influence public agenda and research. Therefore, this chapter focuses on assessing and evaluating what kind of business model is desirable to incentivise the AI for Social Good (AI4SG) factors. In particular, the chapter explores the implications of this discourse for SDG #17 (global partnership) and how this goal may encourage Big Tech corporations to strengthen multi-stakeholder partnerships that promote effective public-private and civil society partnerships and the meaningful co-presence of non-market and market values. In doing so, the chapter proposes an analysis of the sociological notion of "social license to operate" (SLO) elaborated in the mining and extractive industry literature and introduces it into the discourse on sustainable digital business models and responsible management of risks in the digital age. This serves to explore how such a social license can be adopted as a practice by digital business models to foster trust, collaboration and coordination among different actors - AI researchers and initiatives, institutions and civil society at large - for the support of SDGs interrelated targets and goals.

Traditional medical practices and relationships are changing given the widespread adoption of AI-driven technologies across the various domains of health and healthcare. In many cases, these new technologies are not specific to the field of healthcare. Still, they are existent, ubiquitous, and commercially available systems upskilled to integrate these novel care practices. Given the widespread adoption, coupled with the dramatic changes in practices, new ethical and social issues emerge due to how these systems nudge users into making decisions and changing behaviours. This article discusses how these AI-driven systems pose particular ethical challenges with regards to nudging. To confront these issues, the value sensitive design approach is adopted as a principled methodology that designers can adopt to design these systems to avoid harming and contribute to the social good. The AI for Social Good factors are adopted as the norms constraining maleficence. In contrast, higher-order values specific to AI, such as those from the EU High-Level Expert Group on AI and the United Nations Sustainable Development Goals, are adopted as the values to be promoted as much as possible in design. The use case of Amazon Alexa's Healthcare Skills is used to illustrate this design approach. It provides an exemplar of how designers and engineers can begin to orientate their design programs of these technologies towards the social good.

Since the early 1990's the value sensitive design (VSD) approach has been a continually burgeoning design methodology for technological innovation. VSD is commonly described as a "principled approach" to technology design, given that it is explicitly orientated towards designing technologies for human values, rather than sidelining them to ad hoc and/or ex post facto design. However, in much of its near three-decades-long development, the VSD approach has mostly been adopted as a conceptual framework to assess existing technologies and to explore how the consequences of the technology's development could have been influenced positively if VSD was adopted early on and throughout the design process (Friedman et al. 2002). Similarly, VSD has been adopted as a conceptual framework for analysing how future speculative technologies like molecular manufacturing (Umbrello 2019) and advanced artificial intelligence systems can be directed towards positive futures (Umbrello and van de Poel 2021). Yet, despite the benefits of these endeavours and VSD being an approach that can be adapted to any given design domain, it has yet to be widely appropriated in existing design spaces as a primary design approach. In this short paper, we aim to fill this lacuna by framing VSD as a toolkit that can be adopted by existing design teams rather than a wholesale approach. In particular, we describe how VSD maps seamlessly onto agile workflows, an approach to project management that is widely adopted globally. By doing so, we hope that the VSD approach will likewise be more attractive and thus adopted more broadly. As a consequence, a more explicit orientation towards designing technologies for human values will follow.

The international debate on the ethics and legality of autonomous weapon systems (AWS) as well as the call for a ban are primarily focused on the nebulous concept of fully autonomous AWS. More specifically, on AWS that are capable of target selection and engagement without human supervision or control. This thesis argues that such a conception of autonomy is divorced both from military planning and decision-making operations as well as the design requirements that govern AWS engineering and subsequently the tracking and tracing of moral responsibility. To do this, this thesis marries two different levels of meaningful human control (MHC), termed levels of abstraction, to couple military operations with design ethics. In doing so, this thesis argues that the contentious notion of ‘full’ autonomy is not problematic under this two-tiered understanding of MHC. It proceeds to propose the value sensitive design (VSD) approach as a means for designing for MHC.

Since its inception, the trolley problem has sparked a rich debate both within and beyond moral philosophy. Often used as a primer for students to begin thinking about moral intuitions as well as how to distinguish between different forms of moral reasoning, the trolley problem is not without its uses in very practical, applied field like engineering. Often thought of as unrealistic by technically-oriented engineers, trolley cases in fact, help us to think about moral responsibility in a high tech world. This chapter explores the usefulness of trolley-like thinking within the realm of responsible innovation and discusses how despite the inherent issues with trolley scenarios, they remain nonetheless an indispensable tool for helping us to explore ways to maximize our moral responsibility in innovation.

Technological developments involving robotics and artificial intelligence devices are being employed evermore in elderly care and the healthcare sector more generally, raising ethical issues and practical questions warranting closer considerations of what we mean by “care” and, subsequently, how to design such software coherently with the chosen definition. This paper starts by critically examining the existing approaches to the ethical design of care robots provided by Aimee van Wynsberghe, who relies on the work on the ethics of care by Joan Tronto. In doing so, it suggests an alternative to their non-principled approach, an alternative suited to tackling some of the issues raised by Tronto and van Wynsberghe, while allowing for the inclusion of two orientative principles. Our proposal centres on the principles of autonomy and vulnerability, whose joint adoption we deem able to constitute an original revision of a bottom-up approach in care ethics. Conclusively, the ethical framework introduced here integrates more traditional approaches in care ethics in view of enhancing the debate regarding the ethical design of care robots under a new lens.

The ethics of technology has primarily focused on what values are and how they can be embedded in technologies through design. In this context, some work has been done to show the efficacy of a number of design approaches. However, existing studies have not clearly pointed out the ways in which design-for-values approaches can be used by design team managers to properly organize and use technologies in practice. This chapter attempts to fill this gap by discussing the value sensitive design (VSD) approach as a useful means of co-designing technologies as a toolkit for existing workflow management, in this case, Agile. It will be demonstrated that VSD shows promise as a way of democratically designing technologies as well as fostering democratic technology policy innovation.

Healthcare is becoming increasingly automated with the development and deployment of care robots. There are many benefits to care robots but they also pose many challenging ethical issues. This paper takes care robots for the elderly as the subject of analysis, building on previous literature in the domain of the ethics and design of care robots. Using the value sensitive design approach to technology design, this paper extends its application to care robots by integrating the values of care, values that are specific to AI, and higher-scale values such as the United Nations Sustainable Development Goals. The ethical issues specific to care robots for the elderly are discussed at length alongside examples of specific design requirements that work to ameliorate these ethical concerns.

Autonomous weapons systems (AWS), sometimes referred to as “killer robots”, are receiving evermore attention, both in public discourse as well as by scholars and policymakers. Much of this interest is connected with emerging ethical and legal problems linked to increasing autonomy in weapons systems, but there is a general underappreciation for the ways in which existing law might impact on these new technologies. In this paper, we argue that as AWS become more sophisticated and increasingly more capable than flesh-and-blood soldiers, it will increasingly be the case that such soldiers are “in the power” of those AWS which fight against them. This implies that such soldiers ought to be considered hors de combat, and not targeted. In arguing for this point, we draw out a broader conclusion regarding hors de combat status, namely that it must be viewed contextually, with close reference to the capabilities of combatants on both sides of any discreet engagement. Given this point, and the fact that AWS may come in many shapes and sizes, and can be made for many different missions, we argue that each particular AWS will likely need its own standard for when enemy soldiers are deemed hors de combat. We conclude by examining how these nuanced views of hors de combat status might impact on meaningful human control of AWS.

As an idea, transhumanism has received increasing attention in recent years and across numerous domains. Despite presidential candidates such as Zoltan Istvan, who ran on an explicitly Transhumanist platform in 2016 but later dropped out to endorse Hillary Clinton, transhumanism has taken root more recently in the conspiratorial imaginations of the dark web. Given the philosophy’s central emphasis on technology as an inherent good, imaginations in supposed alt-right internet circles have criticised it as an ideological gateway to global, fully-automated Communism. This is not to say that such discussions on transhumanism are exclusively siloed and on the margins of society. Related discussions are happening at various well-known institutions and research centres such as the Institute for Ethics and Emerging Technologies, a non-profit think tank dedicated to techno-progressivism where I have been managing director for half a decade. What I mean to say here is that transhumanism is not monolithic. It is best described as multi-faceted and existing in different instantiations across multiple domains. James Michael MacFarlane’s recent book, Transhumanism as a New Social Movement: The Techno- Imagination, is an attempt to trace the history, meaning, and practices that characterise this variegated term.

A new book by Klaus Schwab and Thierry Malleret, "COVID-19: The Great Reset", is reviewed. COVID-19 not only exacerbates social, economic, environmental, cultural and technological trends that already existed pre-pandemic, but also come with emergent and novel ones that may need to be pushed in certain directions to ensure that the fragilities of the pre-pandemic world do not manifest themselves in the future. This book is intended to provide a map of the various arguments and trends of how COVID-19 has ‘reset’ most, if not all, sectors. Building robust and resilient systems of governance and living while avoiding the pitfalls of returning to how things were requires imagining how to direct these resets to more sustainable and green futures. The authors suggest radical, yet ambiguous ways to achieve these goals that may ultimately undermine them.

The last decade has witnessed the mass distribution and adoption of smart home systems and devices powered by artificial intelligence systems ranging from household appliances like fridges and toasters to more background systems such as air and water quality controllers. The pervasiveness of these sociotechnical systems makes analyzing their ethical implications necessary during the design phases of these devices to ensure not only sociotechnical resilience, but to design them for human values in mind and thus preserve meaningful human control over them. This paper engages in a conceptual investigations of how meaningful human control over smart home devices can be attained through design. The value sensitive design (VSD) approach is proposed as a way of attaining this level of control. In the proposed framework, values are identified and defined, stakeholder groups are investigated and brought into the design process and the technical constraints of the technologies in question are considered. The paper concludes with some initial examples that illustrate a more adoptable way forward for both ethicists and engineers of smart home devices.

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention model for autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles.

The last decade has witnessed the mass distribution and adoption of smart home systems and devices powered by artificial intelligence systems ranging from household appliances like fridges and toasters to more background systems such as air and water quality controllers. The pervasiveness of these sociotechnical systems makes analyzing their ethical implications necessary during the design phases of these devices to ensure not only sociotechnical resilience, but to design them for human values in mind and thus preserve meaningful human control over them. This paper engages in a conceptual investigations of how meaningful human control over smart home devices can be attained through design. The value sensitive design (VSD) approach is proposed as a way of attaining this level of control. In the proposed framework, values are identified and defined, stakeholder groups are investigated and brought into the design process and the technical constraints of the technologies in question are considered. The paper concludes with some initial examples that illustrate a more adoptable way forward for both ethicists and engineers of smart home devices.

The purpose of this paper is to review and critique Wessel Reijers and Bert Gordijn’s paper Moving from value sensitive design to virtuous practice design. In doing so, it draws on recent literature on developing value sensitive design (VSD) to show how the authors’ virtuous practice design (VPD), at minimum, is not mutually exclusive to VSD. This paper argues that virtuous practice is not exclusive to the basic methodological underpinnings of VSD. This can therefore strengthen, rather than exclude the VSD approach. Likewise, this paper presents not only a critique of what was offered as a “potentially fruitful alternative to VSD” but further clarifies and contributes to the VSD scholarship in extending its potential methodological practices and scope. It is concluded that VPD does not appear to offer any original contribution that more recent instantiations of VSD have not already proposed and implemented.

The international debate on the ethics and legality of autonomous weapon systems (AWS), along with the call for a ban, primarily focus on the nebulous concept of fully autonomous AWS. These are AWS capable of target selection and engagement absent human supervision or control. This paper argues that such a conception of autonomy is divorced from both military planning and decision-making operations; it also ignores the design requirements that govern AWS engineering and the subsequent tracking and tracing of moral responsibility. To show how military operations can be coupled with design ethics, this paper marries two different kinds of meaningful human control (MHC) termed levels of abstraction. Under this two-tiered understanding of MHC, the contentious notion of ‘full’ autonomy becomes unproblematic.

Manufacturing and industry practices are undergoing an unprecedented revolution as a consequence of the convergence of emerging technologies such as artificial intelligence, robotics, cloud computing, virtual and augmented reality, among others. This fourth industrial revolution is similarly changing the practices and capabilities of operators in their industrial environments. This paper introduces and explores the notion of the Operator 4.0 as well as how this novel way of conceptualizing the human operator necessarily implicates human values in the technologies that constitute it. The design approach known as value sensitive design (VSD) is used to explore how these Operator 4.0 technologies can be designed for human values. Expert elicitation surveys were used to determine the values of industry stakeholders and examples of how the VSD methodology can be adopted by engineers in order to design for these values is illustrated. The results provide preliminary adoption strategies that industrial teams can take to Operator 4.0 technology for human values.

A new book by Maurizio Balistreri, "Sex robot. L’amore al tempo delle macchine", is reviewed. Sex robots not only exacerbate social, ethical and cultural issues that already exist, but also come with emergent and novel ones. This book is intended to build on the recent research on both robotics and the growing scholarship on sex robots more generally, however with greater attention to the developments of the philosophical issues of how to deal with these new artefacts and steps for living among these types of systems into the future.

Emerging technologies such as cloud computing, augmented and virtual reality, artificial intelligence and robotics, among others, are transforming the field of manufacturing and industry as a whole in unprecedent ways. This fourth industrial revolution is consequentially changing how operators that have been crucial to industry success go about their practices in industrial environments. This short paper briefly introduces the notion of the Operator 4.0 as well as how this novel way of conceptualizing the human operator necessarily implicates human values in the technologies that constitute it. Similarly, the design methodology known as value sensitive design (VSD) is drawn upon to discuss how these Operator 4.0 technologies can be design for human values and, conversely, how a potential value-sensitive Operator 4.0 can be used to strengthen the VSD methodology in developing novel technologies.

Value Sensitive Design (VSD) is an established method for integrating values into technical design. It has been applied to different technologies and, more recently, to artificial intelligence (AI). We argue that AI poses a number of challenges specific to VSD that require a somewhat modified VSD approach. Machine learning (ML), in particular, poses two challenges. First, humans may not understand how an AI system learns certain things. This requires paying attention to values such as transparency, explicability, and accountability. Second, ML may lead to AI systems adapting in ways that ‘disembody’ the values embedded in them. To address this, we propose a threefold modified VSD approach: 1) integrating a known set of VSD principles (AI4SG) as design norms from which more specific design requirements can be derived; 2) distinguishing between values that are promoted and respected by the design to ensure outcomes that not only do no harm but also contribute to good; and 3) extending the VSD process to encompass the whole life cycle of an AI technology in order to monitor unintended value consequences and redesign as needed. We illustrate our VSD for AI approach with an example use case of a SARS-CoV-2 contact tracing app.

The value sensitive design (VSD) approach to designing transformative technologies for human values is taken as the object of study in this chapter. VSD has traditionally been conceptualized as another type of technology or instrumentally as a tool. The various parts of VSD’s principled approach would then aim to discern the various policy requirements that any given technological artifact under consideration would implicate. Yet, little to no consideration has been given to how laws, regulations, policies and social norms engage within VSD practices. Similarly, how the interactive nature of the VSD approach can, in turn, influence those directives. This is exacerbated when we consider machine ethics policy that have global consequences outside their development spheres. What constructs and models will position AI designers to engage in policy concerns? How can the design of AI policy be integrated with technical design? How might VSD be used to develop AI policy? How might law, regulations, social norms, and other kinds of policy regarding AI systems be engaged within value sensitive design? This chapter takes the VSD as its starting point and aims to determine how laws, regulations and policies come to influence how value trade-offs can be managed within VSD practices. It shows that the iterative and interactional nature of VSD both permits and encourages existing policies to be integrated both early on and throughout the design process. The chapter concludes with some potential future research programs.