Is the architecture of the cluster peer to peer, or does one server act as a scheduler and control point?
A. The head node hosts the scheduler, which dispatches tasks to the compute nodes. A parallel application (as opposed to multiple serial applications running in a parametric sweep) runs as a distributed application with a process on each of the nodes—one of them a master process that spawns the others and handles input and output files. When a massively parallel application has been started on all nodes, it can effectively be running peer to peer with its other instances on the other nodes.