How does Open MPI get the list of hosts from Torque / PBS Pro?
Open MPI has changed how it obtains hosts from Torque / PBS Pro over time: • v1.0 and v1.1 series: The list of hosts allocated to a Torque / PBS Pro job is obtained directly from the scheduler using the internal TM API. • v1.2 series: Due to scalability limitations in how the TM API was used in the v1.0 and v1.1 series, Open MPI was modified to read the $PBS_NODEFILE to obtain hostnames. Specifically, reading the $PBS_NODEFILE is much faster at scale than how the v1.0 and v1.1 series used the TM API. It is possible that future versions of Open MPI may switch back to using the TM API in a more scalable fashion, but there isn’t currently a huge demand for it (reading the $PBS_NODEFILE works just fine). Note that the TM API is used to launch processes in all versions of Open MPI; the only thing that has changed over time is how Open MPI obtains hostnames. 4. What happens if $PBS_NODEFILE is modified? Bad Things will happen. We’ve had reports from some sites that system administrators modi
