Baseline Configuration: FY05-04 - Common Set of Environment Variables

When to Contact the HPC Help Desk	Users should contact the HPC Help Desk when assistance is needed for unclassified problems, issues, or questions.
Hours of Operation	8:00 a.m. - 8:00 p.m. Eastern, Monday - Friday (excluding Federal holidays).
HPC Centers Home Page	https://centers.hpc.mil/
Help Desk Video Tutorial	Getting Help: the HPC Help Desk
Phone Number	1-877-222-2039 or (937) 255-0679
Help E-mail	help@helpdesk.hpc.mil
Accounts E-mail	accounts@helpdesk.hpc.mil
HPC Help Desk Manager E-mail	manager@helpdesk.hpc.mil
After Hours	Calls, e-mails and tickets received after normal operating hours will be addressed the following business day.
Mailing Address	DoD HPCMP HPC Help Desk AFRL/RCM 2435 Fifth Street Wright-Patterson Air Force Base, OH - 45433-7802
Ticket Requests	Active users may submit tickets from the Active User Help Ticket. Inactive users may submit tickets using the Inactive User Help Ticket.

Common Set of Environment Variables

BC Project: FY05-04
Date of Policy: 14 Jul 2005
Last Updated: 27 Sep 2019 (see Revision Log)

This policy defines a core set of environment variables that represent the same entity at each of the HPCMP allocated centers participating in the Baseline Configuration (BC) Initiative. These variables shall be predefined in the user login, batch and compute environments, making them automatically available to users at each center.

This list of environment variables is subject to change, and will be reviewed on a periodic basis for potential additions or deletions.

The current list of variables is as follows:

$ARCHIVE_HOME

This is an individual user's directory on the archival file system that serves a given compute platform. $ARCHIVE_HOME is intended to be used as a long-term file storage area by a user. It is not intended to be used by executing programs as this can significantly slow the I/O portion of a program.

$ARCHIVE_HOST

This is the hostname of the archival system serving a particular compute platform. In cases where the archival system is not network mounted to the compute platform, the combination of $ARCHIVE_HOST and $ARCHIVE_HOME will give the exact location of a user's long-term archival storage.

$BC_NODE_TYPE

This variable contains the type of node on which the variable is queried. Values of $BC_NODE_TYPE will be: LOGIN, STANDARD, PHI, BIGMEM, BATCH or ACCELERATOR. Note that BATCH is a node type used by some architectures such as Cray to launch jobs onto other nodes.

$BC_CORES_PER_NODE

This variable contains the number of CPU cores per node for the node type on which the variable is queried.

$BC_STANDARD_NODE_CORES

This variable contains the number of CPU cores per node for a standard compute node and will be set on all nodes including login nodes.

$BC_ACCELERATOR_NODE_CORES

This variable contains the number of CPU cores per node for a compute node which features CPUs and a hosted accelerator processor such as GPU or non-self-hosted Phi. The variable will be set on all nodes including login nodes if the system has any ACCELERATOR nodes. It will be set to 0 on systems without ACCELERATOR nodes.

$BC_BIGMEM_NODE_CORES

This variable contains the number of cores per node for a big memory (BIGMEM) compute node and will be set on all nodes including login nodes if the system has any BIGMEM nodes. It will be set to 0 on systems without BIGMEM nodes.

$BC_PHI_NODE_CORES

This variable contains the number of Phi cores per node, and will be set on all nodes including login nodes if the system has any Phi nodes. It will be set to 0 on systems without Phi nodes.

Example of Thunder Node Configuration
	Login Nodes	Compute Nodes
	Login Nodes	Standard Memory	Large Memory	GPU Accelerated	Phi Accelerated
Cores	28	36	36	2 x 2880	2 x 61

Example of Onyx Node Configuration
	Login Nodes	Compute Nodes
	Login Nodes	Standard Memory	Large Memory	GPU Accelerated	KNL
Cores	22	44	44	1 x 3584	64

Example of Thunder and Onyx Values
Variable	Thunder	Onyx
$BC_STANDARD_NODE_CORES	36	44
$BC_ACCELERATOR_NODE_CORES	28	22
$BC_BIGMEM_NODE_CORES	36	44
$BC_PHI_NODE_CORES	122	64

$BC_HOST

This variable is defined as the generic (not node specific) hostname of the system in lowercase format. Examples include centennial, lightning, onyx and armstrong.

$BC_MEM_PER_NODE

This variable contains the approximate maximum memory per node available to an end user program (in integer MBytes) for the compute node type to which a job is being submitted.

$BC_MPI_TASKS_ALLOC

This variable, intended to be referenced from inside a job script, contains the number of MPI tasks that are allocated for a particular job.

$BC_NODE_ALLOC

This variable, intended to be referenced from inside a job script, contains the number of nodes allocated for a particular job.

$CC

This variable is set by any system supporting a C compiler. If a default compiler is provided, this variable is set to the default C compiler. Any subsequent change to the C compiling environment should reset this variable to the C compiler for that environment. The variable refers to the compiler required for program execution on the system's compute nodes.

$CENTER

This variable contains the path for a user's directory on the Center Wide File System (CWFS).

$COST_HOME

This variable contains the path to the base directory of the default installation of the Common Open Source Tools (COST) installed on a particular compute platform. (See BC policy FY13-01 for COST details.)

$CXX

This variable is set by any system supporting a C++ compiler. If a default compiler is provided, this variable is set to the default C++ compiler. Any subsequent change to the C++ compiling environment should reset this variable to the C++ compiler for that environment. The variable refers to the compiler required for program execution on the system's compute nodes.

$DAAC_HOME

This variable points to the top level directory for the DAAC (Data Analysis and Assessment Center) supported tools, named daac. For more information, please see BC policy FY08-01 (DAAC Supported Visualization Tools)."

$F77

This variable is set by any system supporting a Fortran 77 compiler. If a default compiler is provided, this variable is set to the default Fortran 77 compiler. Any subsequent change to the Fortran 77 compiling environment should reset this variable to the Fortran 77 compiler for that environment. The variable refers to the compiler required for program execution on the system's compute nodes.

$F90

This variable is set by any system supporting a Fortran 90 compiler. If a default compiler is provided, this variable is set to the default Fortran 90 compiler. Any subsequent change to the Fortran 90 compiling environment should reset this variable to the Fortran 90 compiler for that environment. The variable refers to the compiler required for program execution on the system's compute nodes.

$JAVA_HOME

This variable contains the path to the base directory of the default installation of JAVA on a particular compute platform.

$KRB5_HOME

This variable is set to point to the installation directory of the Kerberos utilities.

$PET_HOME

This variable contains the path to the system-wide accessible directory containing the tools formerly installed and maintained by the PET staff. Software in $PET_HOME are considered as experimental or are being evaluated for widespread use. Certain packages in $PET_HOME will be migrated to $COST_HOME, as appropriate

$PROJECTS_HOME

This variable contains the path to the base directory that contains subdirectories for user-supported applications and codes under BC policy FY10-07 (Common Location to Maintain Codes)"

$SAMPLES_HOME

This variable contains the path to the base directory for the sample codes and scripts installed by a center's staff under BC policy FY06-15 (Sample Code Repository)."

$WORKDIR

This is an individual user's directory on the local temporary file system (i.e., local high-speed disk) that is available on all HPCMP high performance computing (HPC) systems. $WORKDIR is intended to be used by executing programs to perform file I/O that is local to that system in order to avoid slower file I/O across a network mounted file system, such as a user's home or archive directories. It is not intended to be used as a permanent file storage area by users. Accordingly, this file system is NOT backed up or exported to any other system. In the event of file or directory structure deletion or a catastrophic disk failure, such files and directory structures are lost. Thus, it is the user's responsibility to transfer files that need to be saved to a location that allows for permanent file storage, such as the user's archival or home directory locations.

Revision Log
Date	Revision
27 Sep 2019	Changed "PETTT" to "PET"
18 Dec 2018	The policy is modified to explicitly state that all environment variables shall be predefined in the user login, batch and compute environments
20 Sep 2018	BC Team Audit - Deprecated $CSI_HOME, re-added $PET_HOME as an evaluation area
10 Oct 2017	Revised definition of $BC_CORES_PER_NODE and introduced the following five environment variables: $BC_NODE_TYPE, $BC_STANDARD_NODE_CORES, $BC_ACCELERATOR_NODE_CORES, $BC_BIGMEM_NODE_CORES and $BC_PHI_NODE_CORES
22 May 2014	Added $COST_HOME, deprecated $PET_HOME, removed SLM language from $ARCHIVE_HOME and $ARCHIVE_HOST, and stipulated lowercase for hostnames in $BC_HOST
12 Dec 2013	$DAAC_HOME re-defined
31 May 2012	$HOST and $KRB5HOME replaced by $BC_HOST and $KRB5_HOME respectively
16 Mar 2012	BC Team Audit
02 Sep 2011	$CENTER and $PROJECTS_HOME added to the list
14 Apr 2009	BC_ Variables added to the list
15 Jan 2009	$DAAC_HOME added to the list
10 Jul 2008	$CSI_HOME and $SAMPLES_HOME added to the list

HPC Help Desk