Why does NEAT use a bias node instead of having a bias parameter in each node?
Mainly because not all nodes need a bias. Thus, it would unnecessarily enlarge the search space to be searching for a proper bias for every node in the system. Instead, we let evolution decide which nodes need biases by connecting the bias node to those nodes. This issue is not a major concern; it could work either way. You can easily code a bias into every node and try that as well. • Have you tried using non-sigmoid activation functions? Yes, that is what CPPNs do. See Compositional Pattern Producing Networks: A Novel Abstraction of Development . The kinds of activation functions available to NEAT biases the kinds of functions it is likely to evolve. As in all machine learning, the right bias (if you can find it) provides and advantage. • How does NEAT avoid bloat? For those who don’t know, bloat is a major problem in Genetic Programming (GP) that results from genomes getting bigger and bigger without any corresponding increase in fitness, leading to unwieldy programs that cannot be
