FG-MPI: Fine-Grain MPI

  • Current implementations of MPI are coarse-grained, with a single MPI process per OS-process, however, there is nothing in the MPI specification precluding a finer-grain interpretation of the standard. During my PhD work, I have designed and implemented Fine-Grain MPI (FG-MPI), a system that allows execution of hundreds and thousands of MPI processes on one node or communicating between nodes inside a cluster. FG-MPI was implemented by integrating into the MPICH2 middleware, a popular open source implementation of the MPI library from the Argonne National Laboratory. Our goal was to take advantage of the type of runtime systems used by fine-grain languages and to integrate that into MPI to obtain the best of both these programming models; the ability to have fine-grain parallelism, while maintaining MPI’s rich support for communication between machines.

    FG-MPI adds a new dimension to mapping processes onto nodes, which can be used to better match the granularity of processes to better fit the cache and improve the performance of existing algorithms. For communication efficiency, we exploit the locality of MPI processes in the system and implement optimized communication between concurrent processes in the same OS-process. On a multicore machine we have shown that FG-MPI achieves good performance (equal to or better) than other multicore parallel languages and runtime systems. FG-MPI provides a vehicle to investigate issues related to scalability of the MPI middleware without requiring the corresponding number of cores or machines. We have investigated scalability issues related to MPI groups and communicators and defined new efficient algorithms for communicator creation and storage of process maps. FG-MPI's light-weight design and ability to expose massive concurrency enables a task-oriented programming approach that can be used to simplify MPI programming and avoid some of the non-blocking communication. The fine-grain nature of FG-MPI makes it suitable for chips with a large number of cores. As well, it is based on message-passing and it will be portable to multicore chips with or without support for cache-coherence.
  • Papers and talks related to FG-MPI are here

    SCTP User-land Stack

  • (SCTP User-land Stack) I have worked on the design and implementation of user-land SCTP stack in a group project. It is based on the FreeBSD kernel code and makes the functionality available as a user-space library to run on different platforms. This is an open source project and was released on November 14, 2011.
  • SCTP based Middleware for MPI

    During my Masters, I worked on a group project on the design, implementation and evaluation of Stream Control Transmission Protocol (SCTP) based middleware for Message Passing Interface (MPI). Our research group was the first to introduce the use of SCTP for MPI to the High Performance Computing (HPC) community. We identified significant similarities in SCTP features and the semantics of message passing interface. This led to a design that alleviated the shortcomings of previous implementations, especially in the case of network loss, and had built-in security and resilience to network failures. During our research we closely collaborated with Randall Stewart, Cisco Distinguished Engineer and Dr. Michael Tüxen of Münster University of Applied Science, who are the primary designers of SCTP. During our work we were able to uncover problems in the implementation of the protocol that led to improvements in the FreeBSD kernel stack by the SCTP developers. We were awarded the Cisco University Research Program (URP) grant for our work.