Common Node Attribute Specification
BC Project: FY14-01
Date of Policy: 13 Mar 2014
Last Updated: 08 Dec 2020 (see Revision Log)
This project defines a common method for specifying the type of nodes required on all allocated HPC systems with heterogeneous node configurations.
Definition of a common method will facilitate commonality across HPC Centers and systems.
Currently, the centers use Workload Management System directives to specify node attributes. The Altair PBS Professional #PBS –l select directive is used here. Examples of node attributes are memory size, or presence of an accelerator such as an Intel Many Integrated Core (MIC), also known as XEON Phi or a Graphics Processing Unit (GPU). The following generic names will be used to specify the listed attributes:
| Attribute | Description |
|---|---|
| bigmem | A node which has more memory than other nodes. |
| nmics | Number of Intel MIC (phi) devices per node |
| ngpus | Number of Nvidia GPU devices per node |
The list of attributes may increase as the use of accelerators in high performance computing increases.
Request for 4 nodes with 48 CPUs (no special devices):
#PBS -l select=4:ncpus=48:mpiprocs=1
Request for 2 bigmem nodes with 44 CPUs:
#PBS -l select=2:ncpus=44:bigmem=1:mpiprocs=1
Request for 8 nodes with 64 CPUs and 1 nmic:
#PBS -l select=8:ncpus=64:nmics=1:mpiprocs=1
Request for 16 nodes with 48 CPUs and 1 GPU:
#PBS -l select=16:ncpus=48:ngpus=1:mpiprocs=1
Requests for 1 node with 44 CPUs (no special devices) and 32 nodes with 22 CPUs and 1 GPU:
#PBS -l select=1:ncpus=44:mpiprocs=1+32:ncpus=22:ngpus=1:mpiprocs=1
A summary of usage for each system for the participating HPCMP centers can be found at the HPC Centers website in the "Guide to the Batch Queuing System."
Notice: Future HPCMP allocated HPC systems may make use of a diversity of HPC job schedulers (PBSpro, SLURM, LSF) to submit batch jobs. Accordingly, hereafter revisions of this policy will include examples of how other job schedulers make use of directives to specify node attributes supported by the job schedulers.
| Date | Revision |
|---|---|
| 08 Dec 2020 | Replaced PBS scheduler by a generic HPC job scheduler |
| 26 Apr 2018 | BC Team Audit |
| 16 Jun 2016 | BC Team Audit |
| 13 Mar 2014 | First release of policy |