Lerning ethics to guide AI decisions

Abstract:
The research in value alignment and AI is centred in ensuring that autonomous AI agents behave in a way that is aligned with human moral values and value preferences, commonly referred to as value systems. But which value system ought the AI align with? While there has been extensive research on AI value alignment, the literature usually assumes a known target value system to align with. Nonetheless, due to the complex and abstract nature of moral values, the problem of constructing a model to capture human value systems is far from resolved. To start with, different humans understand moral values differently in terms of behaviour, hence these intricacies have to be part of the model. Also, value preferences are highly dependent on the context, which the model has to also capture. Finally, the aggregation of individual value systems into a collective value system has to take into account these particularities. This means that mechanisms used to obtain and aggregate preferences in other domains cannot readily and satisfactorily be applied to capture value systems. Unfortunately, until this problem is resolved, AI value alignment will not be possible. This project aims at providing a novel model to capture individual and collective value systems. Our model will capture the aforementioned intricacies and provide value system knowledge from individual to collective levels. Importantly, we will formally study each step of the process, researching for desirable properties that should be maintained, hence not merely providing a computational solution but one that is solidly founded.We will provide an innovative application and validation of our model in the area of policymaking.

Reflections:
The goal for this project is to develop AI tools to understand the human values of society so that we can make AI respect them (e.g. when making automated decisions). Working with moral values is a complex task, values are abstract in nature, but for AI to use them we have to model them mathematically. However, can we make a formula to represent freedom? what about fariness? or security? Two people might not even agree in a definition of these terms. The AI values community usually resorts to text inputs, for example, to detect the relevant values in a situation, however the problem of building a model of society's values exceeds current capabilities. Our aim is to exploit the power of large language models to address this problem.