Multi-Agent Reinforcement Learning for Cyber Defence

Practice-project I: The Effect of Varying Network Size During MARL Training on Agent Performance with applications to changing network topologies

This group project investigated the effect of varying network sizes during Multi-Agent Reinforcement Learning (MARL) training as part of the TTCP CAGE Challenge using the CybORG simulation environment. The study focused on dynamic network topologies, where the number of edge devices changes over time, reflecting the evolving nature of modern enterprise, IoT, and mobile networks.

We trained agents using Proximal Policy Optimisation (PPO) and examined how changes in network size influenced learning performance. As additional edge nodes were introduced, the observation and action spaces expanded, increasing the complexity of the learning task and presenting challenges for scalability. Through experimentation, we analysed how these dynamic environments affected the agent’s ability to learn effective cyber defence strategies.

Our findings highlighted the difficulties of applying MARL to environments with evolving network topologies, where increasing action spaces can significantly impact training efficiency and agent performance. The project also explored potential approaches to improve scalability and adaptability.