Does Open MPI support XRC?
Yes. XRC (eXtended Reliable Connection) decreases the memory consumption of Open MPI and improves its scalability by significantly decreasing number of QPs per machine. XRC is available on Mellanox ConnectX family HCAs with OFED 1.4 and later. See this FAQ entry for instructions how to tell Open MPI to use XRC receive queues. 43. How do I specify the type of receive queues that I want OMPI to use? You can use the btl_openib_receive_queues MCA parameter to specify the exact type of the receive queues for the OMPI to use. This can be advantageous, for example, when you know the exact sizes of messages that your MPI application will use — Open MPI can internally pre-post receive buffers of exactly the right size. See this paper for more details. The btl_openib_receive_queues parameter takes a colon-delimited string listing one or more receive queues of specific sizes and characteristics. For now, all processes in the job must use the same string. You can specify three kinds of receive qu
