Work Space Management

BC Project: FY05-02
Date of Policy: 28 Mar 2006
Last Updated: 03 Apr 2023 (see Revision Log)

A scratch space is a directory on the local temporary file system (i.e. local high-speed disk) which can be accessed by use of the environment variables $WORKDIR, $WORKDIR2, $WORKDIR3, etc. defined in BC policy FY05-04 (Common Set of Environment Variables).

In order to provide sufficient free scratch space to our users, the following scratch space maintenance policy will be implemented on all HPC systems: All user files and directory structures in scratch space that have been accessed (i.e. opened, read, or modified) any time within the previous 21 days are exempt from deletion. Please note that the 21 days retention time is a minimum. Centers may institute policies that have longer retention times whenever possible.

For all systems, the minimum scratch space retention period was selected so that it would be long enough to allow users to retain temporary files in scratch space but short enough to prevent the need to delete files less than 21 days old. In extreme emergency situations system administrators may be forced to delete files and directory structures in scratch space less than 21 days old until sufficient disk space is freed up. In the event of such extreme circumstances, center staff will make every effort to preserve user data as well as contact affected users to aid in the recovery of their valuable data.

To avoid early deletion of files and directory structures in scratch space users are encouraged to use their scratch space efficiently and economically by removing files and directories that are no longer required for current or near-term processing. Users are responsible for saving their files for long-term storage to either archival space ($ARCHIVE_HOME) or home directory space ($HOME), copying their files to the Center-Wide File System ($CENTER) for further processing, or transferring the files to a system at their local facility. Note that the amount of available space in the home directory is much smaller than the space in the scratch directory and is generally configured with a quota limit on individual usage. See BC policy FY12-01 (Minimum Home Directory Size and Backup Schedule) for more information on home directory size and quota limits.

Revision Log
Date Revision
03 Apr 2023Added language for multiple WORKDIR file systems.
30 Sep 2021BC Team Audit
24 May 2018BC Team Audit
19 May 2016BC Team Audit: 15 days retention time increased to 21 days
08 Jun 2015Changed title of policy from "Minimum Scratch Space Retention Time" to the present title. Policy was also re-worded to bring uniformity in the calculations of the retention times across all participating centers
08 Apr 2014BC Team Audit
16 Mar 2012BC Team Audit
12 Jan 201210 days retention time increased to 15 days
11 Jan 20095 days retention time increased to 10 days