Implement Placement Sets

Placement sets are the sets of vnodes within which PBS will try to place a job. PBS tries to group vnodes into the most useful sets, according to how well connected the vnodes are, or the values of resources available at the vnodes. Placement sets are used to improve task placement (optimizing to provide a “good fit”) by exposing information on system configuration and topology. A scheduler tries to put a job in the smallest appropriate placement set.

Parameters

Requirements for Placement Sets

Placement sets are enabled by enabling the Use Placement Sets parameter.

Server-level placement sets are defined by setting the Placement Set Keys parameter to a list of vnode-level string array resources.

At least one vnode-level string array resource must exist on vnodes and be set to values that can be used to assign the vnodes to placement sets.

What Defines a Placement Set, Series, or Pool

Placement sets are defined by the values of vnode-level string array resources. You define placement sets by specifying the names of these resources for the Placement Set Keys parameter for the server. Each value of each resource defines a different placement set. A placement set is all of the vnodes that have the same value for a specified defining resource. For example, if the defining resource is a vnode-level string array named “switch”, which has the values “S1”, “S2”, and “S3”, the set of vnodes where switch has the value “S2” is a placement set. If some vnodes have more than one substring, and one of those substrings is the same in each vnode, those vnodes make up a placement set. For example, if the resource is “router”, and vnode V0 has resources_available.router set to “r1i0,r1”, and vnode V1 has resources_available.router set to “r1i1,r1”, V0 and V1 are in the placement set defined by resources_available.router = “r1”. If the resource has N distinct values across the vnodes, including the value zero and being unset, there can be N-1 or N placement sets defined by that resource. If the only_explicit_psets scheduler attribute is False, there are N placement sets. If the only_explicit_psets scheduler attribute is True, there are N-1 placement sets.

Each placement set can have a different number of vnodes; the number of vnodes is determined only by how many vnodes share that resource value.

Each placement set series is defined by the values of a single resource across all the vnodes. For example, if there are three switches, S1, S2 and S3, and there are vnodes with resources_available.switch that take on one or more of these three values, then there will be three placement sets in the series.

Whenever you define any placement sets, you are defining a placement pool. Placement pools can be defined for the server. You define a server-level placement pool by setting the server’s Placement Set Keys to a list of one or more vnode-level string array resources.

How Placement Sets Are Used

You use placement sets to group vnodes according to the value of one or more resources. Placement sets allow you to group vnodes into useful sets.

You can run multi-vnode jobs in one placement set. For example, it makes the most sense to run a multi-vnode job on vnodes that are all connected to the same high-speed switch.

PBS will attempt to place each job in the smallest possible set that is appropriate for the job.

Determining Whether Job Can Run

For more information about placement sets see Placement Sets in the PBS Professional Administrator's Guide.