Running the Parallel Version

Run the parallel version of Feko with optional parameters for an efficient solution.

The parallel version of Feko may be used on any system that is licensed to run multiple Feko processes concurrently. If a system has a multi-core CPU (for example a quad-core CPU) then a sequential Feko licence will allow a parallel solution with up to 4 parallel processes to be launched. For systems with multiple CPUs (for example, a system with 2 separate dual-core CPUs) a 2-CPU parallel Feko licence will be required in order to run parallel solutions using all 4 of the available cores.

In order to use the parallel version of Feko from the GUI, it is required to configure the host names and number of processes that will be used for each node. (This is initially be set up during installation of Feko, meaning that reconfiguration is only necessary if changes are made.)

On the Solve/Run tab, in the Run/Launch group, click the dialog launcher.

On the Feko Solver tab, under Parallel execution, click the Configure button. In the dialog (see Figure 1), the host names and number of processes to be started on each host must be entered. Usually one process per available core on each machine should be chosen, for example 4 processes for a quad-core machine. It is also possible to use this option to implement a crude load balancing system: running more processes on hosts with faster CPUs or more memory. Nodes may be added or removed from the current cluster setup using the Add and Remove buttons respectively.

Note: For parallel Windows PC clusters, Feko must be installed at the same location on each host.

It is recommended that the parallel job is started from a PC that forms part of the cluster and that this host is listed first.¹

After clicking OK the hosts are saved to a machines.feko file in the directory specified by the environment variable FEKO_USER_HOME. This file is then used in the actual parallel process launching.

On the Feko tab of the Component launch options dialog (shown in Figure 2), further settings with regards to the Feko solution can be made.

Tip: Use the Output MFLOPS rate . . . and Network latency and bandwidth options to ensure an optimum configuration for the nodes.

Feko will print a table giving the performance of the various nodes. These checks are repeated each time Feko calculates the solution.

CAUTION:

A significant amount of time may be required if the test file contains multiple frequencies. These options should therefore not be kept selected after the initial setup, except for debugging purposes.

The target priority of the Feko run may also be set on this tab. Setting the priority below normal will allow other interactive work on the same computer. However, all machines in the cluster operate at the speed of the slowest node - starting other CPU-intensive jobs on one of the nodes in a cluster is generally not recommended.

In order to use parallel solving after it has been set up, do the following: On the Solve/Run tab, in the Run/Launch group, click the Parallel icon. A check mark will be displayed next to the menu option. Any Feko solver runs that are launched while this option is checked will use the parallel version of Feko.

From the command line (for example on a UNIX workstation), the parallel Feko version is started as follows:

runfeko example1 -np x

where the parameter x following -np indicates the required number of processes to be used in the parallel solution. In addition to the arguments listed in Running the Sequential Version, the parallel version accepts the following optional parameters:

-np x: Start the parallel Feko version with x processes. The -np all option is also supported when all available processors in the machines file should be used.

--machines-file machname: The file machname is the machines file with the node names and the number of CPUs (see below).
--mpi-options ...: Unless another --xxx-options parameter is used, all options following this are passed to the MPI launcher (for example mpirun or mpiexec).
--parallel-authenticate <method>: Sets the authentication method to be used for parallel Feko runs. The following authentication methods are available:; default: Platform dependent default (same as if option not specified).; localonly: Run the parallel job on local host only and thus no authentication is required; sspi: Windows Active Directory (SSPI) authentication is used. This option is available on Windows only.; registry: Encrypt the credentials (username and password) into the registry. This option is available on Windows only.

The number of processes to launch on each available host is specified in a machines file with the following general syntax:

Hostname:Number of processes

For example assume that host1 has 4 processors and host2 has 8, then the machines file will be as follows:

host1:4
host2:8

With this machines file, if 6 parallel processes are requested then Fekowill use 4 processes on host1 and 2 processes on host2.

If only one process is to be started on any host, then instead of the entry host3:1 in the machines file, the shorter form host3 may be used.

The machines file (machines.feko) is located in %FEKO_HOME%\shared\mpi and is automatically created during the installation of the parallel version of Feko. This file is the default machines file used by Feko. If a different distribution of the processes is required, this file can be manually edited (however this action is strongly discouraged).

Tip: Create a separate machines file with the syntax described above if a different number of parallel processes are required than what was specified during the installation.

The environment variable FEKO_MACHFILE can be used to force RUNFEKO to use this file instead of the default. The required commands assuming the desired machines file is, for example machname, are as follows (for the sh shell):

FEKO_MACHFILE=./machname
export FEKO_MACHFILE
runfeko example1 -np 6

Alternatively the name of the machines file can be passed as an argument to RUNFEKO on the command line as follows:

runfeko example1 -np 6 --machines-file ../../machname

Using RUNFEKO is independent of the respective platforms and MPI implementations (the discussion of the environment variable FEKO_WHICH_MPI contains more information). For certain applications or experienced users it may be necessary to pass additional options to MPI³. These options are added after the argument --mpi-options. For example on a ScaMPI cluster (assuming FEKO_WHICH_MPI=6), the call

runfeko example_08 -np 6 --mpi-options -immediate_handling \
threaded -smtrace 5-6

(all on one line) is interpreted internally and Feko is executed with the command

/opt/scali/bin/mpimon -export env -immediate_handling threaded \
-smtrace 5-6 /opt/feko/bin/feko.csv example_08 -- host1 4 \
host2 2

Note: host1 and host2 are examples only—the actual information is taken from the machines file.

In addition to using the --mpi-options command line option, the MPI environment can be controlled by setting certain environment variables. For instance, when using Intel MPI the environment variable I_MPI_DEVICE⁴ is quite important to control which device should be used (sockets or shared memory, or RDMA device). Such environment variables is set up internally by means of Lua initialisation scripts. See the Feko Installation Guide for more information.

¹ It is possible to launch the job without including the local machine. The .fek input file must be located on the first PC in the list and the .out and .bof output files are created on this PC in the same directory as the project directory on the local machine. It is the user’s responsibility to transfer the files between the local machine and the first machine in the list if these are not the same. Alternately remote parallel launching can be used where Feko does this copying explicitly.

For more details refer to the Intel MPI and MPICH documentation in mpi\<platform>\<mpi-version>\doc directory of the Feko installation.

³ Note that additional (one time) configuration settings might be required by the domain administrator to prepare the Windows domain for this kind of authentication requests.

⁴ For more details refer to the Intel MPI and MPICH documentation in mpi\<platform>\<mpi-version>\doc directory of the Feko installation.