Computational Social Choice and Machine Learning for Ethical Decision Making

The problem of ethical decision making presents a grand challenge for modern AI research. Arguably, the main obstacle to automating ethical decisions is the lack of a formal specification of ground-truth ethical principles, which have been the subject of debate for centuries among philosophers (e.g., trolley problem). Indeed, a major approach to the ethics of AI is to design it to act according to the aggregate views of society – the so-called social choice. However, the normative basis of this approach appears weak due to the fact that there is no one single aggregate ethical view of society. Instead, we face three sets of decisions: standing, concerning whose ethics views are included; measurement, concerning how their views are identified; and aggregation, concerning how individual views are combined to a single view that will guide AI behaviour. These decisions pose difficult ethical dilemmas with major consequences for AI behaviour and must be made up front in the initial AI design.

Against this background, recent research proposes to combine machine learning (ML) techniques and the formal methods of computational social choice (COMSOC) to learn a model of societal preferences, and, when faced with a particular ethical dilemma at run time, efficiently aggregate those preferences to identify a desirable choice. Specifically, a voting-based algorithm inspired by a theory of swap-dominance efficient voting rules, has been implemented and successfully evaluated in the autonomous vehicle domain. However, the systematic study of voting systems and their applicability in automated ethical decision making is still lacking.

This PhD project will explore how the above approach extends to other application domains and social choice mechanisms, and analyse different combination of those, in order to create successful automated ethical decision-making systems.

A Voting-Based System for Ethical Decision Making
Ritesh NoothigaVu, Snehalkumar ‘Neil’ S. Gaikwad, Edmond Awad, Sohan Dsouza, Iyad Rahwan, Pradeep Ravikumar, Ariel D. Procaccia AAAI 2018

Social choice ethics in artificial intelligence Seth D. Baum
AI & SOCIETY volume 35, pages165–176(2020)

Project ID



Maria Polukarov


AI Planning, Argumentation, Norms, Reasoning