What is “registered” (or “pinned”) memory?
“Registered” memory means two things: • The memory has been “pinned” by the operating system such that the virtual memory subsystem will not relocate the buffer (until it has been unpinned). • The network adapter has been notified of the virtual-to-physical address mapping. These two factors allow network adapters to move data between the network fabric and physical RAM without involvement of the main CPU or operating system. Note that many people say “pinned” memory when they actually mean “registered” memory. However, a host can only support so much registered memory, so it is treated as a precious resource. Additionally, the cost of registering (and unregistering) memory is fairly high. Open MPI takes aggressive steps to use as little registered memory as possible (balanced against performance implications, of course) and mitigate the cost of registering and unregistering memory. 14. I’m getting errors about “error registering openib memory”; what do I do? With OpenFabrics (and ther
