What is that 2nd network that seems to lie on top of the main network?
A. This is the backpropagation network. Every Axon and Synapse has a corresponding BackAxon and BackSynapse which attaches to the upper right corner of the corresponding forward component. Data flows forward from the input to the output through the forward propagation network. The criterion compares the output with the desired response, and computes the error. The error is then injected into the backpropagation network, and the data flows through this network, back towards the original input.
