Carpenter Quick Start Guide
1. Introduction
This document provides information you'll need to quickly begin working on Carpenter. For more detailed information, see the Carpenter User Guide.
2. Get a Kerberos Ticket
For security purposes, you must have a current Kerberos ticket on your computer before attempting to connect to Carpenter. A Kerberos client kit must be installed on your desktop to get a Kerberos ticket. Information about installing Kerberos clients on your Windows desktop can be found on the Kerberos & Authentication page.
3. Connect to Carpenter
Carpenter can be accessed via Kerberized ssh
as follows:
% ssh user@carpenter.erdc.hpc.mil
4. Home, Working, Global, and Center-wide Directories
Each user has file space in the $HOME, $WORKDIR, $GLOBAL, and $CENTER directories. The $HOME, $WORKDIR, $GLOBAL, and $CENTER environment variables are predefined for you and point to the appropriate locations in the file systems. You are strongly encouraged to use these variables in your scripts.
Note: $WORKDIR and $GLOBAL are "scratch" file systems, and $CENTER is a center-wide file system accessible to all ERDC DSRC production systems. Neither of these file systems is backed up. You are responsible for managing files in your $WORKDIR, $GLOBAL and $CENTER directories by backing up files to the archive system and deleting unneeded files. Currently, $WORKDIR files older than 21 days, $GLOBAL files older than 60 days, and $CENTER files older than 120 days are subject to being purged.
If it is determined as part of the normal purge cycle that files in your $WORKDIR, $GLOBAL, or $CENTER directories must be deleted, you WILL NOT be notified prior to deletion. You are responsible to monitor your workspace to prevent data loss.
5. Transfer Files and Data to Carpenter
File transfers to DSRC systems must be performed using Kerberized versions
of the following tools: scp and sftp.
For example, the command below
uses secure copy (scp) to copy a local file into a destination
directory on a Carpenter login node.
% scp local_file user@carpenter.erdc.hpc.mil:/target_dir
For additional information on file transfers to and from Carpenter, see the File Transfers section of the Carpenter User Guide.
6. Submit Jobs to the Batch Queue
The Portable Batch System (PBS) is the workload management system
for Carpenter. To submit a batch job, use the following command:
qsub [ options ] my_job_script
where my_job_script is the name of the file containing your batch script.
For more information on using PBS or job scripts, see the
Carpenter User Guide, the
Carpenter PBS Guide, or the
sample script examples in the $SAMPLES_HOME directory on
Carpenter.
7. Batch Queues
The following table describes the queues available on Carpenter:
Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Max Queued Per User | Max Running Per User | Description |
---|---|---|---|---|---|---|
Highest | urgent | 24 Hours | 9,408 | N/A | N/A | Jobs belonging to DoD HPCMP Urgent Projects |
debug* | 1 Hour | 13,824 | N/A | 2 | Time/resource-limited for user testing and debug purposes | |
HIE | 24 Hours | 192 | 1 | 1 | Rapid response for interactive work. For more information see the HPC Interactive Environment (HIE) User Guide. | |
high_lw | 168 Hours | 7,488 | N/A | 3 | Long-walltime jobs belonging to DoD HPCMP High Priority Projects | |
high_lg | 24 Hours | 100,032 | N/A | 2 | Large jobs belonging to DoD HPCMP High Priority Projects | |
high_sm | 24 Hours | 9,408 | N/A | 17 | Small jobs belonging to DoD HPCMP High Priority Projects | |
frontier_lw | 168 Hours | 7,488 | N/A | 3 | Long-walltime jobs belonging to DoD HPCMP Frontier Projects | |
frontier_lg | 24 Hours | 100,032 | N/A | 2 | Large jobs belonging to DoD HPCMP Frontier Projects | |
frontier_sm | 24 Hours | 9,408 | N/A | 17 | Small jobs belonging to DoD HPCMP Frontier Projects | |
standard_lw | 168 Hours | 7,488 | N/A | 3 | Long-walltime standard jobs | |
standard_lg | 24 Hours | 100,032 | N/A | 2 | Large standard jobs | |
standard_sm | 24 Hours | 9,408 | N/A | 17 | Small standard jobs | |
serial | 168 Hours | 1 | N/A | 10 | Single-core serial jobs. 1 core per hour charged to project allocation. | |
transfer | 48 Hours | 1 | N/A | 10 | Data transfer for user jobs. Not charged against project allocation. See the ERDC DSRC Archive Guide, section 5.2. | |
Lowest | background** | 4 Hours | 9,408 | N/A | 3 | User jobs that are not charged against the project allocation |
* The running job limit on the debug queue per user is 2.
** The running job limit on the background queue per user is 3.
8. Monitoring Your Job
You can monitor your batch jobs on Carpenter using the cqstat, qpeek, qview, or qstat commands.
The qstat command lists all jobs in the queue. The -u username option shows only jobs owned by the given user, as follows:
% qstat -u user1 Req'd Req'd Elap Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time -------- -------- -------- --------- ------ --- --- ------ ----- - ----- 1570853 user1 background tw2 -- 1 1 -- 12:00 R 01:12 1570854 user1 frontier inspect -- 16 1 -- 06:00 Q -- 1570939 user1 standard 45dh8 -- 2 1 -- 02:00 R 00:22
Notice the output contains the JobID for each job. This ID can be used with the qpeek, qview, qstat, and qdel commands.
To delete a job, use the command qdel jobID.
To view a partially completed output file, use the qpeek jobID command.
9. Archiving Your Work
When your job is finished, you should archive any important data to prevent automatic deletion by the purge scripts.
Copy one or more files to the archive system
archive put [-C path ] [-s] file1 [file2 ...]
Copy one or more files from the archive system
archive get [-C path ] [-s] file1 [file2 ...]
For more information on archiving your files, see the ERDC DSRC Archive Guide.
10. Modules
Software modules are a very convenient way to set needed environment variables and include necessary directories in your path so commands for particular applications can be found. Carpenter uses modules to initialize your environment with COTS application software, system commands and libraries, compiler suites, environment variables, and PBS batch system commands.
Several modules are loaded automatically as soon as you log in. To view the currently loaded modules, use module list. To see the entire list of available modules, use module avail. You can modify the configuration of your environment by loading and unloading modules. For complete information on how to do this, see the ERDC DSRC Modules Guide.
11. Available Software
A list of software on Carpenter is available on the Software page.
12. Advance Reservation Service (ARS)
A subset of Carpenter's nodes has been set aside for use as part of the ARS. The ARS allows users to reserve a user-designated number of nodes for a specified number of hours starting at a specific date/time. This service enables users to execute interactive or other time-critical jobs within the batch system environment. The ARS is accessible at https://reservation.hpc.mil. Authenticated access is required. For more information on ARS, see the ARS User Guide.