Transfer Queue

BC Project: FY14-02
Date of Policy: 08 Sep 2014
Last Updated: 03 Apr 2023 (see Revision Log)

The transfer queue is a batch queue that is configured for the sole purpose of allowing users to transfer data into and out of the High Performance Computing (HPC) platforms at each center. Users are able to submit batch scripts to move data between the various storage areas/file systems within a given center using a common set of commands. The transfer queue environment supports all environment variables defined by BC policy FY05-04 (Environment Variables), especially those referring to storage locations. The transfer queue has a wall clock limit of at least 48 hours. User allocations will not be charged for transfer queue jobs.

The following storage areas are accessible from the transfer queue:

1. Workspace (The temporary work file system of the machine on which the job is running).
2. Center Wide File System (CWFS).
3. The Mass Storage Archival System (MSAS) of the site.
4. The Home directory File system.

For transferring data between the MSAS and either the Workspace or CWFS, one of the following must be supported:

1. The archive command and its associated subcommands put, get, ls, stat, and mkdir, defined in BC policy FY06-09 (Common Archival Retrieval),
2. The standard Posix commands, ls, cp, mv, rm, cd, mkdir, rmdir, chmod, and chgrp.

For sample script(s), please see BC policy FY06-15 (Sample Code Repository).

A summary of queue information for the participating centers can be found at the HPC Centers website.

---

Revision Log

Date	Revision
03 Apr 2023	Added that user allocations will not be charged
07 Oct 2021	BC Team Audit, added Home Directory File system
25 Mar 2019	Transfer queue wall clock limit increased to at least 48 hours
20 Sep 2018	BC Team Audit