Nick Schuster

AI systems are increasingly in a position to have deep and systemic impacts on human wellbeing. Projects in value alignment, a critical area of AI safety research, must ultimately aim to ensure that all those who stand to be affected by such systems have good reason to accept their outputs. This is especially challenging where AI systems are involved in making morally controversial decisions. In this paper, we consider three current approaches to value alignment: crowdsourcing, reinforcement learning from human feedback, and constitutional AI. We argue that all three fail to accommodate reasonable moral disagreement, since they provide neither good epistemic reasons nor good political reasons for accepting AI systems’ morally controversial outputs. Since these appear to be the most promising approaches to value alignment currently on offer, we conclude that accommodating reasonable moral disagreement remains an open problem for AI safety, and we offer guidance for future research.

Recommender systems are artificial intelligence technologies, deployed by online platforms, that model our individual preferences and direct our attention to content we’re likely to engage with. As the digital world has become increasingly saturated with information, we’ve become ever more reliant on these tools to efficiently allocate our attention. And our reliance on algorithmic recommendation may, in turn, reshape us as moral agents. While recommender systems could in principle enhance our moral agency by enabling us to cut through the information saturation of the internet and focus on things that matter, as they’re currently designed and implemented they’re apt to interfere with our ability to attend appropriately to morally relevant factors. In order to analyze the distinctive moral problems algorithmic recommendation poses, we develop a framework for the ethics of attention and an account of judicious attention allocation as a moral skill. We then discuss empirical evidence suggesting that attentional moral skill can be thwarted and undermined in various ways by algorithmic recommendation and related affordances of online platforms, as well as economic and technical considerations that support this concern. Finally, we consider how emerging technologies might overcome the problems we identify.

According to agent-centered virtue ethics, acting well is not a matter of conforming to agent-independent moral standards, like acting so as to respect humanity or maximize utility. Instead, virtuous agents determine what is called for in their circumstances through good practical reason. This is an attractive view, but it requires a plausible account of how good practical reason works. To that end, some theorists invoke the skill model of virtue, according to which virtue involves essentially the same kind of practical reason as ordinary skills. I contend, however, that ordinary skills provide a plausible and informative model of good practical reason only insofar as they are assessed by agent-independent standards. And so virtue, likewise, must be assessed primarily by agent-independent moral standards, if the skill model is to serve its purpose. I consider how agent-centered virtue ethics might avert this dilemma. But I ultimately suggest proceeding a different way.

In the Aristotelian tradition, the psychological difference between virtue and continence is commonly understood in terms of inner harmony versus inner conflict. Virtuous agents experience inner harmony between feeling and action because they do not care to do other than what their circumstances call for, whereas continent agents feel conflicted about doing what is called for because of competing concerns. Critics of this view argue, however, that when the circumstances require sacrificing something of genuine value, virtuous agents can indeed feel conflicted about acting well. But if this is so, what differentiates virtuous from merely continent agency? This essay argues that the traditional distinction conflates two aspects of virtue as well as two species of continence. And distinguishing between them provides resources for making sense of the complex relationship between inner conflict and good moral agency.