The International Conference for High Performance Computing, Networking, Storage and Analysis
Large-Scale Granular Simulations Using Dynamic Load Balance on a GPU Supercomputer.
Authors: Satori Tsuzuki (Tokyo Institute of Technology), Takayuki Aoki (Tokyo Institute of Technology)
Abstract: Billion particles are required to describe granular phenomena by using particle simulations based on Distinct Element Method (DEM), which computes contact interactions among particles. Multiple GPUs on TSUBAME 2.5 in Tokyo Tech are used to boost the simulation and are assigned to each domain decomposed in space. Since the particle distribution changes in time and space, dynamic domain decomposition is required for large-scale DEM particle simulations. We have introduced a two-dimensional slice-grid method to keep the same number of particles for each domain. Due to particles across the sub-domain boundary, the memory used for living particles is fragmented and the optimum frequency of de-fragmentation on CPU is studied by taking account for data transfer cost through the PCI-Express bus.
Our dynamic load balance works well with good scalability in proportion to the GPU number. We demonstrate several DEM simulations running on TSUBAME 2.5.