Responsible Autonomy for Hybrid Intelligence

Abstract

In hybrid intelligence (HI) systems, artificial intelligence (AI) agents and humans work together to solve complex tasks. In these interactions, each agent is expected to work autonomously and be responsible for their actions. By capturing consent as a regulation of actions in a normative environment (such as a HI system), an agent can determine an appropriate action within the normative environment, and reason on the moral and ethical requirements and effects of the action. Current consent representations do not allow agents to reason on normative actions, which limit agent autonomy. We are developing a representation of consent that captures the nuances of consent from human-human interaction, and expresses them computationally to allow the AI agent to responsibly practice autonomy in a HI system. In future work, the proposed representation will be evaluated against human intuitions about consent, and compared to current consent representations to ensure a robust and domain-agnostic formalisation. Further research includes developing a consent representation that can manage multi-party consent and shared resources, specifying accounts for consent violations to determine culpability, and exploring a developmental approach to norm representation and management for greater perceived agent responsibility and autonomy in a HI system.