How much registered memory is used by Open MPI? Is there a way to limit it?
Open MPI uses registered memory in several places, and therefore the total amount used is calculated by a somewhat-complex formula that is directly influenced by MCA parameter values. It can be desirable to enforce a hard limit on how much registered memory is consumed by MPI applications. For example, some platforms have limited amounts of registered memory available; setting limits on a per-process level can ensure fairness between MPI processes on the same host. Another reason is that registered memory is not swappable; as more memory is registered, less memory is available for (non-registered) process code and data. When little unregistered memory is available, swap thrashing of unregistered memory can occur. Each instance of the openib BTL module in an MPI process (i.e., one per HCA port and LID) will use up to a maximum of the sum of the following quantities: Description Amount Explanation User memory mpool_rdma_rcache_size_limit By default Open MPI will register as much user mem
