Sparse communication in multi-agent deep reinforcement learning

Abstract

Learning to communicate efficiently is central to multi-agent deep reinforcement learning (MADRL). Existing methods often require agents to exchange messages intensively, which abuses communication channels and leads to high communication overhead. Only a few methods target on learning sparse communication, but they allow limited information to be shared, which affects the efficiency of policy learning. In this work, we propose a multi-agent deep reinforcement learning framework with a decentralized communication scheduling process. The proposed framework, which we call Model-Based Communication (MBC), employs supervised learning to build a message estimation model. This model is used by individual agents to decide if they have to communicate their local information to other agents: agents do not communicate their local information if the intended messages can be properly estimated by others. The MBC framework enables multiple agents to make decisions with sparse communication. We evaluate our framework in a variety of mixed cooperative-competitive environments in both homogeneous and heterogeneous domains. The experimental results show that the MBC improves the performance the state-of-art baselines in both domains and leads to a lower communication overhead compared to the baselines.