Daniel Kilov

As Large Language Models (LLMs) are increasingly deployed as Artificial Moral Advisors and autonomous agents making ethical decisions, evaluating their moral competence has become critical. However, existing evaluations may inadequately assess the moral reasoning capabilities needed for real-world deployment, focusing primarily on whether models can match human judgments on carefully curated ethical scenarios. We surveyed 69 papers evaluating LLM ethical competence (2020-2025) and developed a taxonomy categorising evaluations across datasets, behaviors, and metrics. Our comprehensive analysis maps the methodological landscape of this rapidly growing field and reveals several critical limitations. Most significantly, the vast majority of studies rely on pre-packaged scenarios that highlight morally relevant features, failing to test models' ability to identify ethical considerations in noisy, realistic contexts-what we term "moral sensitivity''. Additionally, evaluations overemphasize verdict accuracy rather than assessing moral reasoning quality and steerability, with few studies testing whether models can be appropriately guided toward different ethical frameworks. Most studies rely on "ground truth'' comparisons despite philosophical arguments that reasonable moral pluralism precludes definitive moral ground truth. We argue for a significant methodological shift: moving from curated scenarios to unfiltered information streams, from verdict accuracy to reasoning quality and steerability, and from ground truth metrics to assessments of reasonableness and consistency. This reorientation is essential for developing AI systems that can navigate moral complexity in real-world deployment scenarios.

AI systems are increasingly in a position to have deep and systemic impacts on human wellbeing. Projects in value alignment, a critical area of AI safety research, must ultimately aim to ensure that all those who stand to be affected by such systems have good reason to accept their outputs. This is especially challenging where AI systems are involved in making morally controversial decisions. In this paper, we consider three current approaches to value alignment: crowdsourcing, reinforcement learning from human feedback, and constitutional AI. We argue that all three fail to accommodate reasonable moral disagreement, since they provide neither good epistemic reasons nor good political reasons for accepting AI systems’ morally controversial outputs. Since these appear to be the most promising approaches to value alignment currently on offer, we conclude that accommodating reasonable moral disagreement remains an open problem for AI safety, and we offer guidance for future research.

ABSTRACT Wilfrid Sellars [1962: 1] described philosophy as an attempt to ‘understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term’. But it is distinctive of philosophy that many of us are interested not only in how the world is but in ways that it could be. That is, philosophy is concerned with facts about modality. Some of the most important arguments in philosophy hinge on modal premises, and philosophers have typically assumed special expertise in evaluating these modal premises. Replicating Goldvarg and Johnson-Laird’s [2000] study of modal illusions, we show that training in mathematics and not philosophy predicts success in overcoming such illusions (n = 395). This study is the first to test the modal expertise of professional philosophers directly. Our findings undermine claims to modal expertise. Philosophical training does not inoculate expert philosophers against basic mistakes in modal judgment.

Buoyed by research in philosophy and moral psychology, virtue ethics has become increasingly influential in the literature. This renewed attention has also led to the development of the situationist challenge: empirical studies undermine the idea that we possess character traits that allow us to act virtuously across contexts. A promising reply to the situationist challenge is that we should not conceive of virtues as traits. Instead, we should conceive of them as expert skills. Here, I raise a new challenge for those who go in for the (s)kill against situationism. Expert skills are brittle, rather than flexible or robust. They typically develop in narrow ways and can falter in novel situations. Experts also sometimes perform worse than novices in novel situations. Such findings present a problem for the virtue-as-skill thesis. Someone who will help others only in narrow contexts or who causes more harm than good does not exemplify virtue. The brittleness of expert skills means that virtue theorists must either (a) abandon the idea that virtues explain behaviour across contexts or (b) abandon the idea that virtues are expert skills. It’s brittle virtue or bust – you can’t teach an old dogma new skills.

Wilfrid Sellars [1962: 1] described philosophy as an attempt to ‘understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term’. But it is distinctive of philosophy that many of us are interested not only in how the world is but in ways that it could be. That is, philosophy is concerned with facts about modality. Some of the most important arguments in philosophy hinge on modal premises, and philosophers have typically assumed special expertise in evaluating these modal premises. Replicating Goldvarg and Johnson-Laird’s [2000] study of modal illusions, we show that training in mathematics and not philosophy predicts success in overcoming such illusions (n = 395). This study is the first to test the modal expertise of professional philosophers directly. Our findings undermine claims to modal expertise. Philosophical training does not inoculate expert philosophers against basic mistakes in modal judgment.

Expertise has become a topic of increased interest to philosophers. Fascinating in its own right, expertise also plays a crucial role in several philosophical debates. My aim in this paper is to draw attention to an important, and hitherto unappreciated feature of expertise: its brittleness. Experts are often unable to transfer their proficiency in one domain to other, even intuitively similar domains. Experts are often unable to flexibly respond to changes within their domains. And, even more surprisingly, experts will occasionally be outperformed by novices when confronted with novel circumstances within their domains of expertise. In section 1, I marshal the evidence in favour of brittleness. In section 2, I argue that appeals to brittleness can advance the dialectic in debates on skilled action and provide reasons to reject a powerful recent argument offered by Christensen et al. : 693–719, 2019). In section 3, I appeal to brittleness to argue against a common conception of philosophical expertise, according to which philosophers possess a domain-general set of reasoning skills. Although my argument in this section is largely negative, there is a twist. Recalibrating our understanding of philosophical expertise opens new avenues of research for defenders of the so-called ‘expertise defence’ against the findings of experimental philosophy.