Implement Fairshare

Implement fairshare to ensure that jobs are run in the order of how deserving they are.

Fairshare Scheduling Policy

Overview of the fairshare scheduling policy, associated parameters and the fairshare tree.

Fairshare is a scheduling policy that prioritizes jobs according to how much of a specified resource is being used by, and has recently been used by, job submitters. Job submitters can be organized into groups and subgroups, so that jobs can also be prioritized according to those groups’ resource usage. Users and groups can each be allotted a percentage of total resource usage.

Fairshare ensures that jobs are run in the order of how deserving they are. A scheduler performs the fairshare calculations each scheduling cycle. If fairshare is enabled, all jobs have fairshare applied to them and there is no exemption from fairshare.

Fairshare can be configured for either primetime or non-primetime, or both. You cannot configure different behaviors for fairshare during primetime and non-primetime.

Fairshare Parameters

Enable Fairshare
Enables the fairshare algorithm, and turns on usage collecting. Jobs will be selected based on a function of their recent usage and priority (shares). Disabled by default.
Fairshare Entity
Specifies the entity for which fairshare usage data will be collected. Can be one of “euser”, “egroup”, “Account_Name”, “queue”, or “egroup:euser”. Not a prime option. Defaults to "euser".
Fairshare Decay Factor
Decay multiplier for fairshare usage reduction. Each decay period, the usage is multiplied by this value. Valid values: between 0 and 1, not inclusive. Not a prime option. Defaults to 0.5.
Fairshare Decay Interval
Time between fairshare usage decay operations. Not a prime option. Defaults to 24 hours.
Fairshare Formula
Specifies the mathematical formula to use in Fairshare calculations. It is composed of PBS resources as well as mathematical operators that are standard Python operators and/or those in the Python math module. When using a PBS resource, if resources_used.RESOURCE_NAME exists, that value is used. Otherwise, the value is taken from Resource_List.RESOURCE_NAME. Not a prime option. Defaults to "cput" (CPU time).
Fairshare Shares for Unknown Entities
The number of shares for the unknown group. These shares determine the portion of a resource to be allotted to that group via fairshare.
Fairshare Enforce No Shares
If this option is enabled, jobs whose entity has zero shares will never run. Requires fairshare to to be enabled. Not a prime option.

Fairshare Tree

Fairshare uses a tree structure, where each vertex in the tree represents some set of job owners and is assigned usage shares. Shares are used to apportion the partition’s or site’s resources. The default tree always has a root vertex and an unknown vertex. The default behavior of fairshare is to give all users the same amount of the resource being tracked. In order to apportion a partition’s or site's resources according to a policy other than equal shares for each user, you create a fairshare tree to reflect that policy.

To configure non-default fairshare, a hierarchical tree structure is set up made up of interior vertices and leaves. Interior vertices are departments, which can contain both departments and leaves. Leaves are for fairshare entities, defined by setting the Fairshare Entity parameter to one of the following:
Table 1. Fairshaire Entities
Keyword Fairshare Entities Purpose
euser Username Individual users are allotted shares of the resource being tracked.
egroup OS group name Groups as a whole are allotted shares of the resource being tracked.
egroup:euser Combinations of username and group name Useful when a user is a member of more than one group, and needs to use a different allotment in each group.
Account_Name Account IDs Shares are allotted by account string (Account_Name job attribute).
queue Queues Shares are allotted between queues.
Apportioning of resources for the partition or site is among these entities. These entities' usage of the designated resource is used in determining the start times of the jobs associated with them. If you wish to have a user appear in more than one department, you can use egroup:euser to distinguish between that user's different resource allotments.

Allotting Shares in the Tree

You assign shares to each vertex in the tree. The actual number of shares given to a vertex or assigned in the tree is not important. What is important is the ratio of shares among each set of sibling vertices. Competition for resources is between siblings only. The sibling with the most shares gets the most resources.

Shares Among Unknown Entities

The root vertex always has a child called "unknown". Any entity not listed in the scheduler’s resource_group file will be made a child of "unknown", designating the entity as unknown. The shares used by unknown entities are controlled by two scheduling parameters Fairshare Shares for Unknown Entities and Fairshare Enforce No Shares.

The parameter Fairshare Shares for Unknown Entities controls how many shares are assigned to the "unknown" vertex. By default this parameter is unset and the "unknown" vertex has 0 shares. The children of the unknown vertex have equal amounts of the shares assigned to the "unknown" vertex.

The parameter Fairshare Enforce No Shares controls whether an entity without any shares can run jobs. If enabled, entities without shares cannot run jobs. If disabled, entities without any shares can run jobs, but only when no other entities’ jobs are available to run.

For a complete description about how to implement fairshare see Using Fairshare in the PBS Professional Administrator's Guide.

Use Fairshare

Configure fairshare scheduling.

Important: The fairshare scheduling parameters can only be updated if the cluster was added by a user with passwordless sudo permissions and therefore may not appear as Scheduling parameters.
  1. Click the Configure tab.
  2. Choose the HPC to configure.

    Choose an HPC
    Figure 1. Choose an HPC
  3. Click Scheduling from the PBS Professional menu located on the left-hand side of the web page.
  4. Scroll down to the second list of Scheduling parameters.
  5. Click located to the right of Enable Fairshare.
  6. Choose one of the following options to enable or disable fairshare scheduling:
    • To enable fairshare scheduling for both prime and non-prime time, enable Enable Fairshare and enable both Prime and Non-Prime.
    • To enable fairshare scheduling for only prime time, enable Enable Fairshare and enable Prime and disable Non-Prime.
    • To enable fairshare scheduling for only non-prime time, enable Enable Fairshare and enable Non-Prime and disable Prime.
    • To disable fairshare scheduling disable Enable Fairshare.
  7. Click to save the changes.
  8. Click located in the upper right-hand corner of the web page.
  9. Enable the Display Advanced Settings check box.
  10. Scroll up to the first list of Scheduling parameters.
  11. Click located to the right of Fairshare Decay Factor.
  12. For Fairshare Decay Factor, specify the decay multiplier for fairshare usage reduction.
    Each decay period, the usage is multiplied by this value.
  13. Click to save the change.
  14. Click located to the right of Fairshare Decay Interval.
  15. For Fairshare Decay Interval, specify the time between fairshare usage decay operations.
    The interval can be entered as an integer in seconds or in the format: [[HH:]MM:]SS[.milliseconds]. Default is 24:00:00.
  16. Click to save the change.
  17. Click located to the right of Fairshare Enforce No Shares.
  18. For Fairshare Enforce No Shares, choose one of the following options:
    • Enable this option so that entities without shares cannot run jobs.
    • Disable this option so that entities without any shares can run jobs, but only when no other entities’ jobs are available to run.
  19. Click to save the change.
  20. Click located to the right of Fairshare Entity.
  21. For Fairshare Entity, specify the entity for which fairshare usage data will be collected by choosing one of the following options:
    • euser
    • egroup
    • Account_Name
    • queue
    • egroup:euser
  22. Click to save the change.
  23. Click located to the right of Fairshare Formula.
  24. For Fairshare Formula, specify the mathematical formula to use in fairshare calculations.
    The calculation is composed of PBS resources as well as mathematical operators that are standard Python operators and/or those in the Python math module. When using a PBS resource, if resources_used.RESOURCE_NAME exists, that value is used. Otherwise, the value is taken from Resource_List.RESOURCE_NAME.
  25. Click to save the change.
  26. Click located to the right of Fairshare Shares for Unknown Entitites.
  27. For Fairshare Shares for Unknown Entities, enter the number of shares for the "unknown" group.
    These shares determine the portion of a resource to be allotted to that group via Fairshare.
  28. Click to save the change.