What happens if $PBS_NODEFILE is modified?
Bad Things will happen. We’ve had reports from some sites that system administrators modify the $PBS_NODEFILE in each job according to local policies. This will currently cause Open MPI to behave in an unpredictable fashion. As long as no new hosts are added to the hostfile, it usually means that Open MPI will incorrectly map processes to hosts, but in some cases it can cause Open MPI to fail to launch processes altogether. The best course of action is to not modify the $PBS_NODEFILE. 5. Can I specify a hostfile or use the –host option to mpirun when running in a Torque / PBS environment? As of version v1.2.1, no. Open MPI will fail to launch processes properly when a hostfile is specifed on the mpirun command line, or if the mpirun [–host] option is used. We’re working on correcting the error. A future version of Open MPI will likely launch on the hosts specified either in the hostfile or via the –host option as long as they are a proper subset of the hosts allocated to the Torque
