Common Compute Node Local WorkDir Addressing

BC Project: FY20-01
Date of Policy: 28 Aug 2020
Last Updated: 16 Dec 2021 (see Revision Log)

This policy defines a common method by which users can access a unique local work directory for each batch job, regardless of how it is implemented (e.g. SSD, NVMe, Optane memory, or high speed HDD devices) on each allocated HPC system. The path of this local work directory is defined in an environment variable ($LOCALWORKDIR) for each batch job on the nodes assigned to that job.

This local disk space will provide "near high-speed memory" performance characteristics to enhance application performance. It will be deleted (by the system) after each batch job so that full read/write access to the local workspace can be guaranteed to the next job/user on the node.

Since the workspace defined by $LOCALWORKDIR is only available for the duration of a batch job, the user is responsible for copying any files that need to be retained beyond the length of the job run time to another space such as $WORKDIR, $HOME, or $ARCHIVE_HOME. Likewise, at the beginning of a job, files that require high bandwidth can be copied to $LOCALWORKDIR from these other spaces.

Each site/system can define how best to implement the $LOCALWORKDIR deletion policy.

Shared compute nodes are exempt from this policy.

Revision Log
Date Revision
16 Dec 2021BC Team Audit
28 Aug 2020Initial version