Non-Allocated Systems
The HPCMP provides a limited number of HPC systems for non-allocated use. These systems are typically experimental or unique in nature, and do not include a full complement of application software, however they provide users an opportunity to explore nontraditional architectures while operating free of the constraints of allocated use.
Builder is a single-node Aspen Systems Linux Gigabyte server intended primarily to provide a platform for building and sharing Singularity software containers. The system is populated with AMD 7742 processors and Nvidia V100 graphics processing units. Builder uses Intel Gigabit Ethernet as its high-speed network for I/O traffic. Builder uses AVAGO MegaRAID to manage its local file system that targets 130 TB of disk storage. Builder is equipped with two 64-core processors (128 total cores) running the RHEL 8 operating system, two GPUs, 1,024 gigabytes of memory, with no user-accessible swap space.
Available Documentation
| Date / Time | Details |
|---|---|
| 2026 Nov 18 07:30 - 17:00 HI | System Maintenance |
| Login/GPU | |
|---|---|
| Total Nodes | 1 |
| Processor | AMD 7742 Rome |
| Processor Speed | 2.25 GHz |
| Sockets / Node | 2 |
| Cores / Node | 128 |
| Total CPU Cores | 128 |
| Usable Memory / Node | 1 TB |
| Accelerators / Node | 2 |
| Accelerator | NVIDIA V100 |
| Memory / Accelerator | 32 GB |
| Storage on Node | 130 TB SAS HDD Array |
| Interconnect | XL710 for 40GbE QSFP+ to cwfs, I350 Gigabit Network Connection to DREN |
| Operating System | RHEL 8.5 |
Queue Information
Builder does not employ a job scheduler, so there are no queues.
Coral is an Aspen Systems Linux Cluster with x86_64 and ARM nodes, enabling architecture comparisons. It features 8-way A100 GPUs, AMD MI100/MI250 GPUs, and DPUs for accelerated computing. With standard and large-memory configurations, InfiniBand interconnect, and a Weka parallel file system, Coral supports diverse HPC tasks requiring batch scheduling and parallel processing. It serves as a testbed for ARM-based HPC evaluation.
Available Documentation
| Date / Time | Details |
|---|---|
| 2026 Nov 02 07:30 - Nov 03 00:00 HI (Completed) | System Maintenance |
| Login | Compute Nodes | |||||||
|---|---|---|---|---|---|---|---|---|
| A100 GPU | AMD M100 GPU | AMD MI250 GPU | ARM + A100 + DPU | ARM | DPU | |||
| Total Nodes | 2 | 8 | 2 | 2 | 4 | 4 | 2 | |
| Processor | Intel 6326 Ice Lake | Intel 6338 Ice Lake | AMD 7513 Milan | AMD 7513 Milan | Ampere Altra Q80 | Ampere Altra Q64 | Intel 6338 Ice Lake | |
| Processor Speed | 2.9 GHz | 2.0 GHz | 2.6 GHz | 2.6 GHz | 3.0 GHz | 3.0 GHz | 2.0 GHz | |
| Sockets / Node | 2 | 2 | 2 | 2 | 1 | 2 | 2 | |
| Cores / Node | 32 | 64 | 64 | 64 | 80 | 128 | 64 | |
| Total CPU Cores | 64 | 512 | 128 | 128 | 320 | 512 | 128 | |
| Usable Memory / Node | 480 GB | 2 TB | 1 TB | 1 TB | 480 GB | 480 GB | 480 GB | |
| Accelerators / Node | 1 | 8 | 8 | 4 | 2 | N/A | N/A | |
| Accelerator | NVIDIA A40 | NVIDIA A100 SXM4 (NVLINK) | NVIDIA A100 PCIe | AMD MI100 | AMD MI250 | NVIDIA A100 | N/A | N/A |
| Memory / Accelerator | 48 GB | 80 GB | 80 GB | 32 GB | 128 GB | 80 GB | N/A | N/A |
| Storage on Node | 3.84 TB NVMe SSD | 3.84 TB NVMe SSD | 3.84 TB NVMe SSD | 3.84 TB NVMe SSD | 3.84 TB NVMe SSD | 6 TB SAS | 3.84 TB NVMe SSD | 3.84 TB NVMe SSD |
| Interconnect | HDR InfiniBand | HDR InfiniBand | HDR InfiniBand | HDR InfiniBand | HDR InfiniBand | HDR InfiniBand | HDR InfiniBand | HDR InfiniBand |
| Operating System | RHEL8 | RHEL8 | RHEL8 | RHEL8 | RHEL8 | RHEL8 | RHEL8 | RHEL8 |
| Workspace | 240 TBytes WekaIO (all NVMe) + 200 TBytes Object HDD Tier | |||||||
| Node Names | coral1 coral2 |
cr2cn1 cr2cn2 cr2cn3 cr2cn4 |
cr3cn5 cr3cn6 cr3cn7 cr3cn8 |
cr4mi1 cr4mi2 |
cr4mi3 cr4mi4 |
cr2arm1 cr2arm2 cr2arm3 cr2arm4 |
cr4arm5 cr4arm6 cr4arm7 cr4arm8 |
cr1dpu1 cr1dpu2 |
| L3 Cache (MB) | N/A | 48 | 48 | 128 | 128 | 32 | 32 | 48 |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Max Queued Per User | Max Running Per User | Description |
|---|---|---|---|---|---|---|
| Highest | ||||||
|
debug | 30 Minutes | N/A | N/A | N/A | Time/resource-limited for user testing and debug purposes |
| standard | 7 Days | N/A | N/A | N/A | Standard jobs | |
| transfer | 2 Days | N/A | N/A | N/A | Data transfer for user jobs. Not charged against project allocation. See the AFRL DSRC Archive Guide, section 5.2. | |
| Lowest |
Jean is a Liqid system located at the ARL DSRC. It has 488 standard compute nodes, 20 large-memory nodes, 42 AI/ML nodes, and 26 visualization nodes.
| Login | Standard | Large-Memory | Visualization | AI/ML | Transfer | |
|---|---|---|---|---|---|---|
| Total Nodes | 9 | 494 | 24 | 24 | 49 | 2 |
| Processor | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake |
| Processor Speed | 2.3 GHz | 2.3 GHz | 2.3 GHz | 2.3 GHz | 2.3 GHz | 2.3 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 | 2 | 2 |
| Cores / Node | 96^ | 96^ | 96^ | 96^ | 96^ | 96^ |
| Total CPU Cores | 864 | 47,242 | 2,304 | 2,304 | 2,784 | 192 |
| Usable Memory / Node | 748 GB | 385 GB or 757 GB | 1546 GB | 772 GB | 772 GB | 772 GB |
| Accelerators / Node | 1 or 4 | None | None | 1 | 4 | None |
| Accelerator | NVIDIA A100 PCIe4 | N/A | N/A | NVIDIA A100 PCIe 4 | NVIDIA A100 PCIe 4 | N/A |
| Memory / Accelerator | 40 GB | N/A | N/A | 40 GB | 40 GB | N/A |
| Storage on Node | 7 TB NVMe SSD | None | 14 TB NVMe SSD | 7 TB NVMe SSD | 7 TB NVMe SSD | N/A |
| Interconnect | HDR InfiniBand | HDR InfiniBand | HDR InfiniBand | HDR InfiniBand | HDR Infiniband | HDR InfiniBand |
| Operating System | RHEL 8 | RHEL 8 | RHEL 8 | RHEL 8 | RHEL 8 | RHEL 8 |
^ There are 96 cores/node. However, WEKA consumes 4 cores/node so only 92 cores/node are available. To ensure your job does not run slowly or encounter issues, 92 available cores/nodes should be used in your batch scripts. For example, if you want to run a 4-node job with all available cores (92 cores/node not 96 cores/node), the following Slurm directives should be utilized: --nodes=4, --ntasks=368 (not 384 cores) and --ntasks-per-node=92 (not 96 cores).
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | urgent | 96 Hours | 12,000*^ | Jobs belonging to DoD HPCMP Urgent Projects |
|
HIE | 24 Hours | 576*^ | Rapid response for interactive work. For more information see the HPC Interactive Environment (HIE) User Guide. |
| debug** | 1 Hour | 13,632*^ | Time/resource-limited for user testing and debug purposes | |
| high | 96 Hours | 12,000*^ | Jobs belonging to DoD HPCMP High Priority Projects | |
| frontier | 168 Hours | 12,000*^ | Jobs belonging to DoD HPCMP Frontier Projects | |
| cots | 96 Hours | 12,000*^ | Abaqus and Fluent jobs | |
| interactive | 12 Hours | 12,000*^ | Interactive Jobs. | |
| standard | 168 Hours | 12,000*^ | Standard jobs | |
| standard-long | 336 Hours | 12,000*^ | Standard jobs needing additional wall time | |
| AIML | 168 Hours | 960*^ | Dedicated queue for jobs utilizing GPUs | |
| Lowest | serial | 96 Hours | 192*^ | User jobs that are not charged against the project allocation |
^ Important! Jean is a specialized Mission system for Modeling, Simulation & Analysis and AI/ML work. This system utilizes a flexible access policy to facilitate access for critical, high priority work. All core counts will be adjusted on a "at need" basis determined by the needs of the priority projects running in the priority batch access queues.
* There are 96 cores/node as reflected in the Max Cores per Job column in the table above. However, WEKA consumes 4 cores/node so only 92 cores/node are available. To ensure your job does not run slowly or encounter issues, 92 available cores/nodes should be used in your batch scripts. For example, if you want to run a 4-node job with all available cores (92 cores/node not 96 cores/node), the following Slurm directives should be utilized: --nodes=4, --ntasks=368 (not 384 cores) and --ntasks-per-node=92 (not 96 cores).
** The running job limit on the debug queue per user is 4.
Wheat is a Liqid system located at the ERDC DSRC. It has 754 standard compute nodes, 7 large-memory nodes, 16 1-GPU visualization nodes, 83 4-GPU MLA nodes, and 21 6-GPU MLA nodes (a total of 81,052 compute cores). It has 376 TB of useable memory and is rated at 4.2 PFLOPS.
| Login | Standard | Large-Memory | Visualization | 4-GPU MLA | 6-GPU MLA | |
|---|---|---|---|---|---|---|
| Total Nodes | 6 | 754 | 7 | 16 | 83 | 21 |
| Processor | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake | Intel 9242 Cascade Lake |
| Processor Speed | 2.3 GHz | 2.3 GHz | 2.3 GHz | 2.3 GHz | 2.3 GHz | 2.3 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 | 2 | 2 |
| Cores / Node | 92 | 92 | 92 | 92 | 92 | 92 |
| Total CPU Cores | 552 | 69,368 | 644 | 1,472 | 7,636 | 1,932 |
| Usable Memory / Node | 8 GB | 356 GB | 3 TB | 731 GB | 731 GB | 731 GB |
| Accelerators / Node | 1 | None | 4 | 1 | 4 | 6 |
| Accelerator | NVIDIA A100 | N/A | NVIDIA A100 | NVIDIA A100 | NVIDIA A100 | NVIDIA A100 |
| Memory / Accelerator | 40 GB | N/A | 40 GB | 40 GB | 40 GB | 40 GB |
| Storage on Node | None | None | None | None | None | None |
| Interconnect | Ethernet | InfiniBand | InfiniBand | InfiniBand | InfiniBand | InfiniBand |
| Operating System | RHEL 8 | RHEL 8 | RHEL 8 | RHEL 8 | RHEL 8 | RHEL 8 |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | urgent | 168 Hours | 2,392 | Jobs belonging to DoD HPCMP Urgent Projects |
|
debug^ | 1 Hour | 3,680 | Time/resource-limited for user testing and debug purposes |
| HIE | 24 Hours | 92 | Rapid response for interactive work. For more information see the HPC Interactive Environment (HIE) User Guide. | |
| high_lw | 168 Hours | 1,932 | Long-walltime jobs belonging to DoD HPCMP High Priority Projects | |
| high_lg | 24 Hours | 35,420 | Large jobs belonging to DoD HPCMP High Priority Projects | |
| frontier_lw | 168 Hours | 1,932 | Long-walltime jobs belonging to DoD HPCMP Frontier Projects | |
| frontier_lg | 24 Hours | 35,420 | Large jobs belonging to DoD HPCMP Frontier Projects | |
| standard_lw | 168 Hours | 1,932 | Long-walltime standard jobs | |
| standard_lg | 24 Hours | 35,420 | Large standard jobs | |
| transfer | 48 Hours | 1 | Data transfer for user jobs. Not charged against project allocation. See the ERDC DSRC Archive Guide, section 5.2. | |
| Lowest | background^^ | 4 Hours | 48 | User jobs that are not charged against the project allocation |
^ The running job limit on the debug queue per user is 2.
^^ The running job limit on the background queue per user is 3.
Important! Wheat is a Specialized Mission System for Modeling, Simulation & Analysis and AI/ML work. This system utilizes a flexible access policy to facilitate access for critical, high priority work. The service levels are zero-allocated (non-allocated) work running in background queues – available to all users with a zero allocation – while approved priority projects can request priority batch access queues such as standard, high, frontier and urgent.