Experiences with adding SYCL support to GROMACS
KEYNOTE
Experiences with adding SYCL support to GROMACS
GROMACS is an open-source, high-performance molecular dynamics (MD) package primarily used for biomolecular simulations, accounting for ~5% of HPC utilization worldwide. Due to the extreme computing needs of MD, significant efforts are invested in improving the performance and scalability of simulations. Target hardware ranges from supercomputers to laptops of individual researchers and volunteers of distributed computing projects such as Folding@Home. The code has been designed both for portability and performance by explicitly adapting algorithms to SIMD and data-parallel processors. A SIMD intrinsic abstraction layer provides high CPU performance. Explicit GPU acceleration has long used CUDA to target NVIDIA devices and OpenCL for AMD/Intel devices.
In this talk, we discuss the experiences and challenges of adding support for the SYCL platform into the established GROMACS codebase and share experiences and considerations in porting and optimization. While OpenCL offers the benefits of using the same code to target different hardware, it suffers from several drawbacks that add significant development friction. Its separate-source model leads to code duplication and makes changes complicated. The need to use C99 for kernels, while the rest of the codebase uses C++17, exacerbates these issues. Another problem is that OpenCL, while supported by most GPU vendors, is never the main framework and thus is not getting the primary support or tuning efforts. SYCL alleviates many of these issues, employing a single-source model based on the modern C++ standard. In addition to being the primary platform for Intel GPUs, the possibility to target AMD and NVIDIA GPUs through other implementations (e.g., hipSYCL) might make it possible to reduce the number of separate GPU ports that have to be maintained.
Some design differences from OpenCL, such as flow directed acyclic graphs (DAGs) instead of in-order queues, made it necessary to reconsider the GROMACS’s task scheduling approach and architectural choices in the GPU backend. Additionally, supporting multiple GPU platforms presents a challenge of balancing performance (low-level and hardware-specific code) and maintainability (more generalization and code-reuse). We will discuss the limitations of the existing codebase and interoperability layers with regards to adding the new platform; the compute performance and latency comparisons; code quality considerations; and the issues we encountered with SYCL implementations tested. Finally, we will discuss our goals for the next release cycle for the SYCL backend and the overall architecture of GPU acceleration code in GROMACS.
Speaker(s)
Erik Lindahl
Erik Lindahl received a PhD from the KTH Royal Institute of Technology in 2001, and performed postdoctoral research at Groningen University, Stanford University and the Pasteur Institute. He is currently professor of Biophysics at Stockholm University, with a second appointment as professor of Theoretical Biophysics at the Royal Institute of Technology. Lindahl’s research is focused on understanding the molecular mechanisms of membrane proteins, in particular ion channels, through a combination of molecular simulations and experimental work involving cryo-EM and electrophysiology. He has authored some 130 scientific publications and is the recipient of an ERC starting grant. Lindahl heads the international GROMACS molecular simulation project, which is one of the leading scientific codes to exploit parallelism on all levels from accelerators and assembly code to supercomputers and distributed computing. He is co-director of the Swedish e-Science Research Center as well as the Swedish National Bioinformatics Infrastructure, and lead scientist of the BioExcel Center-of-Excellence for Computational Biomolecular Research. His research work has been awarded with the Prix Jeune Chercheur Blaise Pascal, the Sven and Ebba-Christian Högberg prize, and the Wallenberg Consortium North prize. Lindahl is currently the chair of the PRACE Scientific Steering Committee.
