Work Space Management
BC Project: FY05-02
Date of Policy: 28 Mar 2006
Last Updated: 03 Apr 2023 (see Revision Log)
A scratch space is a directory on the local temporary file system (i.e. local
high-speed disk) which can be accessed by use of the environment variables
$WORKDIR, $WORKDIR2, $WORKDIR3, etc. defined in BC policy
FY05-04 (Common Set of Environment Variables).
In order to provide sufficient free scratch space to our users, the following scratch space maintenance policy will be implemented on all HPC systems: All user files and directory structures in scratch space that have been accessed (i.e. opened, read, or modified) any time within the previous 21 days are exempt from deletion. Please note that the 21 days retention time is a minimum. Centers may institute policies that have longer retention times whenever possible.
For all systems, the minimum scratch space retention period was selected so that it would be long enough to allow users to retain temporary files in scratch space but short enough to prevent the need to delete files less than 21 days old. In extreme emergency situations system administrators may be forced to delete files and directory structures in scratch space less than 21 days old until sufficient disk space is freed up. In the event of such extreme circumstances, center staff will make every effort to preserve user data as well as contact affected users to aid in the recovery of their valuable data.
To avoid early deletion of files and directory structures in scratch space users are
encouraged to use their scratch space efficiently and economically by removing files and directories
that are no longer required for current or near-term processing. Users are responsible for saving
their files for long-term storage to either archival space ($ARCHIVE_HOME) or home
directory space ($HOME), copying their files to the Center-Wide File System
($CENTER) for further processing, or transferring the files to a system at their local
facility. Note that the amount of available space in the home directory is much smaller than the space
in the scratch directory and is generally configured with a quota limit on individual usage.
See BC policy FY12-01 (Minimum Home Directory Size
and Backup Schedule) for more information on home directory size and quota limits.
| Date | Revision |
|---|---|
| 03 Apr 2023 | Added language for multiple WORKDIR file systems. |
| 30 Sep 2021 | BC Team Audit |
| 24 May 2018 | BC Team Audit |
| 19 May 2016 | BC Team Audit: 15 days retention time increased to 21 days |
| 08 Jun 2015 | Changed title of policy from "Minimum Scratch Space Retention Time" to the present title. Policy was also re-worded to bring uniformity in the calculations of the retention times across all participating centers |
| 08 Apr 2014 | BC Team Audit |
| 16 Mar 2012 | BC Team Audit |
| 12 Jan 2012 | 10 days retention time increased to 15 days |
| 11 Jan 2009 | 5 days retention time increased to 10 days |