Load Balancing

The dynamic load-balancing (LB) capability allows the nanoFluidX code to redistribute the work load among the GPUs, which generally improves the overall performance.

The LB capability is more prominent when dealing with irregular geometries in which large portions of empty space are among the components. The LB capability will be less pronounced for a highly-compact geometry, such as a fully filled box. Either way, having the LB capability turned on generally results in faster run time. The LB capability was tested on a highly irregular test case (dam break) and has shown excellent scaling up to 100 GPUs. The main parameters are as follows:

loadbalancingParameters
{
    type                               DYNAMIC
    frequency                          100
    imbalance_tol                      1.1
}

type: Type of load balancing.; Options: NOLB / DYNAMIC; Default: NOLB
frequency: This parameter specifies the frequency at which the workload will be redistributed among the GPUs. The frequency is expressed in time steps, so a number of 100 sets the redistribution every 100 time steps. This value can cause computational overhead if set to very low values. Keep the default value of 100 time steps, or at least that order of magnitude.; Default: 100
imbalance_tol: This parameter specifies the workload tolerance discrepancy between the GPUs. The default value of 1.1 implies that the work load redistribution will be carried out among two GPUs only if the work load that they are handling differs by more than 10 percent.; Default: 1.1

Overall, the parameters for the LB are best kept at their default values, once the LB is turned on, as poor choice of the above parameters can actually detriment the run time of the simulation.