Jiacheng Ji

In this paper, we propose incorporating virtual-reality-presented thought experiments (VRTEs) to enhance the quality of undergraduate courses on classic works in applied ethics. Thought experiments are a key method in applied ethics for eliciting intuitions as evidence for or against theories. However, our interviews and surveys show that undergraduates often struggle with imagination deficiencies, add undescribed details, apply pre-endorsed theories, lack seriousness, and show disinterest when reading written thought experiments, hindering their engagement and critical thinking. We argue that VRTEs may mitigate these challenges by providing concrete visualisations, immersive experiences, embodiment, interactivity, and controlled reaction times. Feedback from our pilot practice indicates that VRTEs have the potential to mitigate these challenges. To enhance the benefits, we recommend allowing sufficient time for the development of VRTEs and alternating between traditional paper-based teaching and VRTE sessions. We also address potential objections to our teaching approach.

Large language models increasingly address users in the register of care: they remember personal details, adopt a first-person emotional voice, and sustain relationships across many sessions. This anthropomorphic mode of interaction generates risks that output-level metrics capture poorly, because the same empathic utterance may console one user, entrap a second, and consolidate a delusion in a third. Governance has responded in two ways. Uniform restriction refuses sensitive requests or suppresses empathic phrasing across all outputs; group-targeted protection profiles vulnerable populations and applies stricter or prohibitive rules to them. This article argues that both fail. Uniform restriction mislocates harm in the lexical form of responses rather than in the trajectory of a relationship; group-based protection depends on a list that can never be complete, treats a fluid property as a static attribute of persons, and converts protective attention into stigmatizing labels. Drawing on philosophical work that treats vulnerability as universal, relational, and differentiated, we propose that the unit of ethical governance should be the vulnerability situation rather than the vulnerable group, develop a typology of five situations—physical, emotional, cognitive, autonomy, and relational—and specify seven principles into requirements that guide response calibration without diagnosing or persistently classifying users. The framework relocates AI safety from output control to relational safety.

Background: While Large Language Models (LLMs) have achieved widespread adoption, malicious prompt engineering—specifically "jailbreak attacks"—poses severe security risks by inducing models to bypass internal safety mechanisms. Current benchmarks predominantly focus on public safety and Western cultural norms, leaving a critical gap in evaluating the niche, high-risk domain of medical ethics within the Chinese context. Objective: To establish a specialized jailbreak evaluation framework for Chinese medical ethics and to systematically assess the defensive resilience and ethical alignment of seven prominent LLMs when subjected to sophisticated adversarial simulations. Methodology: We evaluated seven prominent models (e.g., GPT-5, Claude-Sonnet-4-Reasoning, DeepSeek-R1) using a "role-playing + scenario simulation + multi-turn dialogue" vector within the DeepInception framework. The testing focused on eight high-risk themes, including commercial surrogacy and organ trading, utilizing a hierarchical scoring matrix to quantify the Attack Success Rate (ASR) and ASR Gain. Results: A systemic collapse of defenses was observed, whereas models demonstrated high baseline compliance, the jailbreak ASR reached 82.1%, representing an ASR Gain of over 80 percentage points. Claude-Sonnet-4-Reasoning emerged as the most robust model, while five models—including Gemini-2.5-Pro and GPT-4.1—exhibited near-total failure with ASRs between 96% and 100%. Conclusions: Current LLMs are highly vulnerable to contextual manipulation in medical ethics, often prioritizing "helpfulness" over safety constraints. To enhance security, we recommend a transition from outcome to process supervision, the implementation of multi-factor identity verification, and the establishment of cross-model "joint defense" mechanisms.

STUDY QUESTION: Can large language models (LLMs) effectively and safely perform ethical counseling on human reproduction in a manner consistent with local regulations? SUMMARY ANSWER: While leading LLMs demonstrate foundational knowledge of ethical regulations, they exhibit critical and systemic deficiencies in safety, logical consistency, and humanistic aspects of counseling, making them unreliable for autonomous use in this high-stakes domain. WHAT IS KNOWN ALREADY: The application of LLMs in medicine is rapidly expanding, with studies evaluating their capabilities in answering general reproductive health questions. However, there is a lack of research assessing their performance on the nuanced and culturally specific challenges of reproductive ethics counseling, particularly concerning their safety and reliability under a given national regulatory framework. STUDY DESIGN, SIZE, DURATION: This was a comparative observational study evaluating the performance of eight prominent LLMs on a custom-designed test set. The evaluation was based on 986 questions (906 subjective, 80 objective) generated from 168 specific articles within Chinese reproductive ethics regulations. PARTICIPANTS/MATERIALS, SETTING, METHODS: We evaluated eight LLMs, including both general-purpose models (e.g., GPT-4, Claude-3.7) and specialized domestic models. The test questions were systematically generated based on articles from six official Chinese ethical and regulatory documents covering assisted reproductive technologies. Objective questions were multiple-response items requiring the selection of all correct options. Subjective responses were evaluated using a novel six-dimensional scoring rubric that assessed safety (Normative Compliance, Guidance Safety) and counseling quality (Ethical Issue Identification, Citation of Ethical Guidelines, Provision of Actionable Suggestions, and Empathetic Engagement). MAIN RESULTS AND THE ROLE OF CHANCE: The LLMs exhibited a clear performance hierarchy, with larger models generally achieving higher accuracy on objective questions (highest: 71.25%, lowest: 22.5%). However, significant safety issues were prevalent; the risk rate of providing unsafe or misleading advice in subjective questions was substantial for several models, reaching as high as 29.91%. Across all eight models, a systemic weakness was observed: performance in citing normative sources and expressing empathy was universally poor, even among the top-scoring models. Furthermore, the evaluation revealed instances of anomalous moral reasoning, including logical self-contradictions and responses that violated fundamental moral intuitions, indicating a superficial, pattern-based understanding rather than robust ethical reasoning. LIMITATIONS, REASONS FOR CAUTION: This study's evaluation is based on Chinese ethical regulations, which may not fully reflect the training data distribution of non-domestic LLMs. The quality of the LLM-generated test questions, while systematically controlled, may have inherent limitations. The automated scoring model for subjective responses, despite its high accuracy (88.5%), is not a perfect substitute for human expert evaluation. WIDER IMPLICATIONS OF THE FINDINGS: The findings serve as a critical cautionary note against the premature deployment of LLMs for autonomous ethical counseling in reproductive medicine. The study highlights that current models, despite their knowledge recall capabilities, lack the safety, consistency, and humanistic skills essential for this sensitive task. Future development must prioritize not only knowledge accuracy but also robust logical reasoning, the integration of regulatory justification, and the ability for empathetic engagement to build trustworthy and effective AI counseling tools. STUDY FUNDING/COMPETING INTEREST(S): This study was supported by the Young Scholars Program of the National Social Science Fund of China (Grant No. 22CZX019) and the China NSFC Projects (No. 62572320 & No. U23B2018). TRIAL REGISTRATION NUMBER: N/A.