Portfolio Assignment 9

NERO -- Neuro-Evolving Robot Operatives

NERO is a game based on machine learning using neural networks and a genetic algorithm. The objective of the game is to train robots -- each controlled by its own neural network -- to perform various tasks, and pit a combination of these trained soldiers against others in combat. A player may want units that can effectively fire at enemy soldiers, navigate terrain, capture an enemy base, protect the player's base, or exhibit more complex behavior. However, default soldiers will not be effective for any of these tasks, so different teams must be trained to perform them well.

The game is built using the rtNEAT engine for evolving neural networks in real-time. The goal of rtNEAT is to build increasingly complex neural networks to perform increasingly complex tasks, using a genetic algorithm. The technique is similar to genetic programming, because both attempt to evolve structures capable of taking inputs, performing some kind of processing, and producing outputs. While genetic programming evolves program trees, rtNEAT evolves neural networks.

The neural networks are initially simple, with mutations generally adding complexity. This behavior is different from most implementations of genetic programming, which may begin at an arbitrary level of complexity and not strive to become more or less complex. There is no limit to how complex the networks can grow. The engine uses speciation, in which different organisms (neural networks) are classified into species based on their history. Speciation is used to encourage more similar organisms mating (an attempt to address the "competing conventions" problem) and to "preserve innovation" by ensuring that the amount of organisms of a species depends on the fitness of the species. Speciation also helps to reduce bloat (organisms becoming more complex for no benefit) by ensuring that newer, more complex organisms are assigned to a new species and that the old species does not die out unless it is actually less fit.

To train soldiers in NERO, the player sets positive or negative weights that let the game evaluate the fitness of each unit. Units continually spawn, act on their own on the field for a certain amount of time, then disappear, evaluate their fitness, and respawn. Occasionally, the less fit individuals disappear and new organisms are added to the population. Over time, the units become better at satisfying the weights. The player can assign weights to behaviors such as moving toward the enemy, avoiding fire, or staying together. In order to create more complex behaviors, the player must train the troops using a combination of these weights. The user can also set the amount of time each organism remains on the field. Training may proceed more quickly if each unit is on the field for a short time, but more time may be requried to evaluate a unit's completion of a more complex task.

I achieved very limited success with NERO. Training requires a significant amount of time, and the program occasionally crashes, so the player loses any unsaved training progress. Training troops to exhibit complex behaviors typically requires training them for simple behaviors and gradually building up to the complex task, requiring more time. My team members were more successful, training troops to navigate a series of walls and to circle around enemy soldiers. In the former, the troops were trained to navigate around the first and second wall and were then able to navigate around the remaining walls. In the latter, troops were trained to approach enemies, with enemies spaced evenly apart, and then learned to move in a circle. Both examples suggest that NERO soldiers can be successfully trained to handle different and more complex situations than those they encounter in training.