Astrid Schomäcker

Modern computer systems are ubiquitous in contemporary life yet many of them remain opaque. This poses significant challenges in domains where desiderata such as fairness or accountability are crucial. We suggest that the best strategy for achieving system transparency varies depending on the specific source of opacity prevalent in a given context. Synthesizing and extending existing discussions, we propose a taxonomy consisting of eight sources of opacity that fall into three main categories: architectural, analytical, and socio-technical. For each source, we provide initial suggestions as to how to address the resulting opacity in practice. The taxonomy provides a starting point for requirements engineers and other practitioners to understand contextually prevalent sources of opacity, and to select or develop appropriate strategies for overcoming them.

A central goal of research in explainable artificial intelligence (XAI) is to facilitate human understanding. However, understanding is an elusive concept that is difficult to target. In this paper, we argue that a useful way to conceptualize understanding within the realm of XAI is via certain human abilities. We present four criteria for a useful conceptualization of understanding in XAI and show that these are fulfilled by an abilities-based approach: First, thinking about understanding in terms of specific abilities is motivated by research from numerous disciplines involved in XAI. Second, an abilities-based approach is highly versatile and can capture different forms of understanding important in XAI application contexts. Third, abilities can be operationalized for empirical studies. Fourth, abilities can be used to clarify the link between explainability, understanding, and societal desiderata concerning AI, like fairness and trustworthiness. Conceptualizing understanding as abilities can therefore support interdisciplinary collaboration among XAI researchers, provide practical benefit across diverse XAI application contexts, facilitate the development and evaluation of explainability approaches, and contribute to satisfying the societal desiderata of different stakeholders concerning AI systems.

In the literature on Explainable Artificial Intelligence (XAI), it is often claimed that explainability can ensure the fairness of AI-based decision-making [5, 25, 37]. However, fairness of AI-based decision-making is a complex and multi-dimensional issue, ranging from attempts at statistical definitions of fairness [11] via legal definitions of discrimination [4] to psychological research into perceived fairness [55]. Given this complexity, it is not always clear how explainability can help improve the fairness of AI decision-making. This paper reviews the concepts and approaches to fairness in AI decision-making in philosophy, psychology, law, and computer science and discusses the potential relationship with explainability. It suggests three ways in which explanations might be relevant for fairness: Explanations could be a conceptual requirement of fairness, a (potential) direct cause of fairness, or they could remove a barrier that hinders the use of other means to increase fairness. With the help of these three categories the paper clarifies how explainability might be used to improve the fairness of AI decision-making.