What are OLS and subset/stepwise regression?
If you are statistician, “OLS” means “ordinary least squares” (as opposed to weighted or generalized least squares), which is what the NN literature often calls “LMS” (least mean squares). If you are a neural networker, “OLS” means “orthogonal least squares”, which is an algorithm for forward stepwise regression proposed by Chen et al. (1991) for training RBF networks. OLS is a variety of supervised training. But whereas backprop and other commonly-used supervised methods are forms of continuous optimization, OLS is a form of combinatorial optimization. Rather than treating the RBF centers as continuous values to be adjusted to reduce the training error, OLS starts with a large set of candidate centers and selects a subset that usually provides good training error. For small training sets, the candidates can include all of the training cases. For large training sets, it is more efficient to use a random subset of the training cases or to do a cluster analysis and use the cluster means
