Unclassified Systems
Centennial is an SGI ICE XA system located at the ARL DSRC. It has 1,784 standard compute nodes, 32 large-memory compute nodes, and 32 GPU compute nodes (a total of 1,848 compute nodes or 73,920 compute cores). It has 253 TB of memory and is rated at 2.6 peak PFLOPS.
Available Documentation
| Date / Time | Details |
|---|---|
| 2022 Aug 09 08:00 - 10:00 ET | Network Maintenance |
| Login | Standard | Large-Memory | GPU | |
|---|---|---|---|---|
| Total Nodes | 24 | 1,784 | 32 | 32 |
| Processor | Intel E5-2698v4 Broadwell | Intel E5-2698v4 Broadwell | Intel E5-2698v4 Broadwell | Intel E5-2698v4 Broadwell |
| Processor Speed | 2.2 GHz | 2.2 GHz | 2.6 GHz | 2.2 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 |
| Cores / Node | 40 | 40 | 40 | 40 |
| Total CPU Cores | 960 | 71,360 | 1,280 | 1,280 |
| Useable Memory / Node | 252 GB | 124 GB | 508 GB | 252 GB |
| Accelerators / Node | None | None | None | 1 |
| Accelerator | n/a | n/a | n/a | NVIDIA K40P PCIe |
| Memory / Accelerator | n/a | n/a | n/a | 12 GB |
| Storage on Node | None | None | None | None |
| Interconnect | InfiniBand EDR | InfiniBand EDR | InfiniBand EDR | InfiniBand EDR |
| OS | RHEL | RHEL | RHEL | RHEL |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | debug | 1 Hour | N/A | User diagnostic jobs |
|
transfer | 48 Hours | 1 | Data transfer jobs |
| urgent | 96 Hours | N/A | Designated urgent jobs by DoD HPCMP | |
| high | 96 Hours | N/A | Designated high-priority projects by service/agency | |
| frontier | 168 Hours | N/A | Frontier projects only | |
| cots | 96 Hours | N/A | Abaqus and Fluent jobs | |
| interactive | 12 Hours | N/A | Interactive jobs | |
| standard-long | 200 Hours | N/A | ARL DSRC permission required | |
| standard | 168 Hours | N/A | Normal user jobs | |
| Lowest | background | 24 Hours | 4,096 | User jobs that will not be charged against the project allocation |
Gaffney is an HPE SGI 8600 system located at the Navy DSRC. It has 704 standard compute nodes, 16 large-memory nodes, and 32 GPU-accelerated nodes (a total of 752 compute nodes or 36,096 compute cores). It has 154 TB of memory and is rated at 3.05 peak PFLOPS.
Available Documentation
| Login | Standard | Large-Memory | GPU | |
|---|---|---|---|---|
| Total Nodes | 8 | 704 | 16 | 32 |
| Processor | Intel 8168 Skylake | Intel 8168 Skylake | Intel 8168 Skylake | Intel 8168 Skylake |
| Processor Speed | 2.7 GHz | 2.7 GHz | 2.7 GHz | 2.7 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 |
| Cores / Node | 48 | 48 | 48 | 48 |
| Total CPU Cores | 384 | 33,792 | 768 | 1,536 |
| Useable Memory / Node | 320 GB | 170 GB | 742 GB | 367 GB |
| Accelerators / Node | None | None | None | 1 |
| Accelerator | n/a | n/a | n/a | NVIDIA P100 PCIe |
| Memory / Accelerator | n/a | n/a | n/a | 16 GB |
| Storage on Node | None | None | 3.2 TB SSD | None |
| Interconnect | Intel Omni-Path | Intel Omni-Path | Intel Omni-Path | Intel Omni-Path |
| OS | RHEL | RHEL | RHEL | RHEL |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | urgent | 24 Hours | 768 | Designated urgent projects by DoD HPCMP |
|
|
frontier | 168 Hours | 14,400 | Designated frontier projects by DoD HPCMP |
| high | 168 Hours | 15,840 | Designated high-priority jobs by Service/Agency | |
| debug | 30 Minutes | 2,400 | User diagnostic jobs | |
| HIE | 24 Hours | 384 | Rapid response for interactive work | |
| standard | 168 Hours | 8,160 | Normal priority user jobs | |
| serial | 168 Hours | 1 | Serial user jobs | |
| gpu | 24 Hours | 384 | GPU-accelerated jobs | |
| transfer | 48 Hours | N/A | Data transfer jobs | |
| bigmem | 96 Hours | 720 | Large-memory jobs | |
| Lowest | background | 4 Hours | 1,200 | User jobs that will not be charged against the project allocation |
Koehr is an HPE SGI 8600 system located at the Navy DSRC. It has 704 standard compute nodes, 16 large-memory nodes, and 32 GPU-accelerated nodes (a total of 752 compute nodes or 36,096 compute cores). It has 154 TB of memory and is rated at 3.05 peak PFLOPS.
Available Documentation
| Login | Standard | Large-Memory | GPU | |
|---|---|---|---|---|
| Total Nodes | 8 | 704 | 16 | 32 |
| Processor | Intel 8168 Skylake | Intel 8168 Skylake | Intel 8168 Skylake | Intel 8168 Skylake |
| Processor Speed | 2.7 GHz | 2.7 GHz | 2.7 GHz | 2.7 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 |
| Cores / Node | 48 | 48 | 48 | 48 |
| Total CPU Cores | 384 | 33,792 | 768 | 1,536 |
| Useable Memory / Node | 320 GB | 170 GB | 742 GB | 367 GB |
| Accelerators / Node | None | None | None | 1 |
| Accelerator | n/a | n/a | n/a | NVIDIA P100 PCIe |
| Memory / Accelerator | n/a | n/a | n/a | 16 GB |
| Storage on Node | None | None | 3.2 TB SSD | None |
| Interconnect | Intel Omni-Path | Intel Omni-Path | Intel Omni-Path | Intel Omni-Path |
| OS | RHEL | RHEL | RHEL | RHEL |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | urgent | 24 Hours | 768 | Designated urgent projects by DoD HPCMP |
|
|
frontier | 168 Hours | 14,400 | Designated frontier projects by DoD HPCMP |
| high | 168 Hours | 14,304 | Designated high-priority jobs by Service/Agency | |
| debug | 30 Minutes | 2,400 | User diagnostic jobs | |
| HIE | 24 Hours | 384 | Rapid response for interactive work | |
| standard | 168 Hours | 8,160 | Normal priority user jobs | |
| serial | 168 Hours | 1 | Serial user jobs | |
| gpu | 24 Hours | 384 | GPU-accelerated jobs | |
| transfer | 48 Hours | N/A | Data transfer jobs | |
| bigmem | 96 Hours | 720 | Large-memory jobs | |
| Lowest | background | 4 Hours | 1,200 | User jobs that will not be charged against the project allocation |
Mustang is an HPE SGI 8600 system located at the AFRL DSRC. It has 1,128 standard compute nodes, 24 large-memory compute nodes, and 24 GPU compute nodes (a total of 1,176 compute nodes or 56,448 compute cores). It has 244 TB of memory and is rated at 4.88 peak PFLOPS.
Available Documentation
| Date / Time | Details |
|---|---|
| 2022 Jul 28 14:00 ET - TBD (In Progress) | Archive Maintenance |
| Login | Standard | Large-Memory | GPU | |
|---|---|---|---|---|
| Total Nodes | 12 | 1,128 | 24 | 24 |
| Processor | Intel 8168 Skylake | Intel 8168 Skylake | Intel 8168 Skylake | Intel 8168 Skylake |
| Processor Speed | 2.7 GHz | 2.7 GHz | 2.7 GHz | 2.7 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 |
| Cores / Node | 48 | 48 | 48 | 48 |
| Total CPU Cores | 576 | 54,144 | 1,152 | 1,152 |
| Useable Memory / Node | 380 GB | 180 GB | 744 GB | 372 GB |
| Accelerators / Node | None | None | None | 1 |
| Accelerator | n/a | n/a | n/a | NVIDIA P100 PCIe |
| Memory / Accelerator | n/a | n/a | n/a | 16 GB |
| Storage on Node | None | None | None | None |
| Interconnect | Intel Omni-Path | Intel Omni-Path | Intel Omni-Path | Intel Omni-Path |
| OS | RHEL | RHEL | RHEL | RHEL |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | urgent | 168 Hours | 28,224 | Jobs belonging to DoD HPCMP Urgent Projects |
|
|
debug | 1 Hour | 1,152 | User testing |
| high | 168 Hours | 28,224 | Jobs belonging to DoD HPCMP High Priority Projects | |
| frontier | 168 Hours | 28,224 | Jobs belonging to DoD HPCMP Frontier Projects | |
| standard | 168 Hours | 28,224 | Standard jobs | |
| HIE | 24 Hours | 96 | Rapid response for interactive work | |
| transfer | 48 Hours | 1 | Data transfer for user jobs | |
| Lowest | background | 120 Hours | 48 | Unrestricted access - no allocation charge |
Narwhal is an HPE Cray EX system located at the Navy DSRC. It has 2,176 standard compute nodes, 12 large-memory nodes, 16 visualization accelerated nodes, 32 1-MLA accelerated nodes, and 32 2-MLA accelerated nodes (a total of 2,268 compute nodes or 290,304 compute cores). It has 590 TB of memory and is rated at 12.8 peak PFLOPS.
Available Documentation
| Login | Standard | Large-Memory | Visualization | MLA 1-GPU | MLA 2-GPU | |
|---|---|---|---|---|---|---|
| Total Nodes | 11 | 2,176 | 12 | 16 | 32 | 32 |
| Processor | AMD 7H12 Rome | AMD 7H12 Rome | AMD 7H12 Rome | AMD 7H12 Rome | AMD 7H12 Rome | AMD 7H12 Rome |
| Processor Speed | 2.6 GHz | 2.6 GHz | 2.6 GHz | 2.6 GHz | 2.6 GHz | 2.6 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 | 2 | 2 |
| Cores / Node | 128 | 128 | 128 | 128 | 128 | 128 |
| Total CPU Cores | 1,408 | 278,528 | 1,536 | 2,048 | 4,096 | 4,096 |
| Useable Memory / Node | 226 GB | 238 GB | 995 GB | 234 GB | 239 GB | 229 GB |
| Accelerators / Node | None | None | None | 1 | 1 | 2 |
| Accelerator | n/a | n/a | n/a | NVIDIA V100 PCIe | NVIDIA V100 PCIe | NVIDIA V100 PCIe |
| Memory / Accelerator | n/a | n/a | n/a | 32 GB | 32 GB | 32 GB |
| Storage on Node | 880 GB SSD | None | 1.8 TB SSD | None | 880 GB SSD | 880 GB SSD |
| Interconnect | HPE Slingshot | HPE Slingshot | HPE Slingshot | HPE Slingshot | HPE Slingshot | HPE Slingshot |
| OS | SLES | SLES | SLES | SLES | SLES | SLES |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | urgent | 24 Hours | 16,384 | Designated urgent projects by DoD HPCMP |
|
|
frontier | 168 Hours | 65,536 | Designated frontier projects by DoD HPCMP |
| high | 168 Hours | 16,384 | Designated high-priority projects by Service/Agency | |
| debug | 30 Minutes | 8,192 | User diagnostic jobs | |
| HIE | 24 Hours | 3,072 | Rapid response for interactive work | |
| viz | 24 Hours | 128 | Visualization jobs | |
| standard | 168 Hours | 32,768 | Normal priority user jobs | |
| mla | 24 Hours | 128 | Machine Learning Accelerated jobs | |
| smla | 24 Hours | 128 | Machine Learning Accelerated jobs | |
| dmla | 24 Hours | 128 | Machine Learning Accelerated jobs | |
| serial | 168 Hours | 1 | Serial User jobs | |
| bigmem | 96 Hours | 1,280 | Large-memory jobs | |
| transfer | 24 Hours | N/A | Data transfer jobs | |
| Lowest | background | 4 Hours | 1,024 | User jobs that will not be charged against the project allocation. |
Onyx is a Cray XC40/50 system. It has 4,810 standard compute nodes, 4 large-memory compute nodes, 32 GPU compute nodes, 32 Knights Landing (Phi) compute nodes, and 64 Machine Learning Accelerator (MLA) multi-GPGPU nodes (a total of 4,942 compute nodes or 217,128 compute cores). It is rated at 6.06 peak PFLOPS.
Available Documentation
| Login | Standard | Large-Memory | KNL | GPU | 2-MLA | 10-MLA | |
|---|---|---|---|---|---|---|---|
| Total Nodes | 12 | 4,810 | 4 | 32 | 32 | 60 | 4 |
| Processor | Intel E5-2699v4 Broadwell | Intel E5-2699v4 Broadwell | Intel E5-2699v4 Broadwell | Intel 7230 Knights Landing | Intel E5-2699v4 Broadwell | Intel 6148 Skylake | Intel 6148 Skylake |
| Processor Speed | 2.8 GHz | 2.8 GHz | 2.8 GHz | 1.3 GHz | 2.8 GHz | 2.4 GHz | 2.4 GHz |
| Sockets / Node | 2 | 2 | 2 | 1 | 1 | 2 | 2 |
| Cores / Node | 44 | 44 | 44 | 64 | 22 | 40 | 40 |
| Total CPU Cores | 264 | 211,640 | 176 | 2,048 | 704 | 2,400 | 160 |
| Useable Memory / Node | 247 GB | 121 GB | 1 TB | 90 GB | 247 GB | 172 GB | 735 GB |
| Accelerators / Node | None | None | None | None | 1 | 2 | 10 |
| Accelerator | n/a | n/a | n/a | n/a | NVIDIA P100 PCIe | NVIDIA V100 SXM2 | NVIDIA V100 PCIe |
| Memory / Accelerator | n/a | n/a | n/a | n/a | 16 GB | 16 GB | 32 GB |
| Storage on Node | None | None | None | None | None | None | None |
| Interconnect | Ethernet | Cray Aries | Cray Aries | Cray Aries | Cray Aries | InfiniBand | InfiniBand |
| OS | SLES | CLE | CLE | CLE | CLE | CentOS | CentOS |
| Priority | Queue Name | Max Wall Clock Time | Max Jobs | Min Cores Per Job | Max Cores Per Job | Description |
|---|---|---|---|---|---|---|
| Highest | urgent | 24 Hours | N/A | 22 | 7,260 | Designated urgent jobs by DoD HPCMP |
|
|
debug | 1 Hour | 4 | 22 | 11,484 | User testing |
| HIE | 24 Hours | 2 | 22 | 110 | Rapid response for interactive work | |
| frontier_lg | 24 Hours | 2 | 7,261 | 143,968 | Frontier projects only (large jobs) | |
| frontier_lw | 168 Hours | 15 | 22 | 15,708 | Frontier projects only (long walltime) | |
| frontier_sm | 48 Hours | 70 | 22 | 7,260 | Frontier projects only (small jobs) | |
| high_lg | 24 Hours | 2 | 8,449 | 105,820 | Designated high-priority jobs by Service/Agency (large jobs) | |
| high_lw | 168 Hours | 15 | 22 | 10,824 | Designated high-priority jobs by Service/Agency (long walltime) | |
| high_sm | 24 Hours | 70 | 22 | 8,448 | Designated high-priority jobs by Service/Agency (small jobs) | |
| frontier_md | 96 Hours | 2 | 15,709 | 34,540 | Frontier projects only (medium sized, long walltime) | |
| standard_lg | 24 Hours | 2 | 7,261 | 105,820 | Normal priority jobs (large jobs) | |
| standard_lw | 168 Hours | 3 | 22 | 5,808 | Normal priority jobs (long walltime) | |
| standard_sm | 24 Hours | 70 | 22 | 7,260 | Normal priority jobs (small jobs) | |
| transfer | 48 Hours | 6 | 1 | 1 | Data transfer jobs. Access to the long-term storage | |
| Lowest | background | 4 Hours | 6 | 22 | 7,260 | Unrestricted access - no allocation charge |
SCOUT is an IBM Power9 system located at the ARL DSRC. It has 22 Training nodes, each with 6 nVidia V100 GPUs, 128 Inference nodes, each with 4 nVidia T4 GPUs, and 2 Visualization nodes, each with 2 nVidia GPUs (a total of 152 compute nodes or 6,080 cores). It has 45 TB of memory.
Available Documentation
| Date / Time | Details |
|---|---|
| 2022 Aug 09 08:00 - 10:00 ET | Network Maintenance |
| 2022 Aug 16 07:30 - Aug 17 18:00 ET | System Maintenance |
| Login | Training | Inference | Visualization | |
|---|---|---|---|---|
| Total Nodes | 4 | 22 | 128 | 2 |
| Processor | IBM POWER9 | IBM POWER9 | IBM POWER9 | IBM POWER9 |
| Processor Speed | 2.55 GHz | 2.55 GHz | 2.55 GHz | 2.55 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 |
| Cores / Node | 40 | 40 | 40 | 40 |
| Total CPU Cores | 160 | 880 | 5,120 | 80 |
| Useable Memory / Node | 502 GB | 690 GB | 246 GB | 502 GB |
| Accelerators / Node | None | 6 | 4 | 2 |
| Accelerator | n/a | NVIDIA V100 PCIe | NVIDIA T4 PCIe | NVIDIA V100 PCIe |
| Memory / Accelerator | n/a | 32 GB | 16 GB | 16 GB |
| Storage on Node | 1.5 TB PCIe | 12 TB PCIe | 3.3 TB PCIe | 3.3 TB PCIe |
| Interconnect | InfiniBand EDR | InfiniBand EDR | InfiniBand EDR | InfiniBand EDR |
| OS | RHEL | RHEL | RHEL | RHEL |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | transfer | 48 Hours | N/A | Data transfer jobs |
|
|
urgent | 96 Hours | N/A | Designated urgent jobs by DoD HPCMP |
| debug | 1 Hour | N/A | User diagnostic jobs | |
| high | 168 Hours | N/A | Designated high-priority projects by service/agency | |
| frontier | 168 Hours | N/A | Frontier projects only | |
| HIE | 24 Hours | N/A | Rapid response for interactive work | |
| interactive | 12 Hours | N/A | Interactive jobs | |
| standard | 168 Hours | N/A | Normal user jobs | |
| Lowest | background | 24 Hours | N/A | User jobs that will not be charged against the project allocation |
Warhawk is an HPE Cray EX system located at the AFRL DSRC. It has 1,024 standard compute nodes, 4 large-memory nodes, 24 1-GPU visualization nodes, and 40 2-GPU Machine-Learning nodes (a total of 1,092 compute nodes or 139,776 compute cores). It has 564 TB of memory and is rated at 6.86 peak PFLOPS.
Available Documentation
| Date / Time | Details |
|---|---|
| 2022 Jul 28 14:00 ET - TBD (In Progress) | Archive Maintenance |
| Login | Standard | Large-Memory | Visualization | Machine-Learning | |
|---|---|---|---|---|---|
| Total Nodes | 7 | 1,024 | 4 | 24 | 40 |
| Processor | AMD 7H12 Rome | AMD 7H12 Rome | AMD 7H12 Rome | AMD 7H12 Rome | AMD 7H12 Rome |
| Processor Speed | 2.6 GHz | 2.6 GHz | 2.6 GHz | 2.6 GHz | 2.6 GHz |
| Sockets / Node | 2 | 2 | 2 | 2 | 2 |
| Cores / Node | 128 | 128 | 128 | 128 | 128 |
| Total CPU Cores | 896 | 131,072 | 512 | 3,072 | 5,120 |
| Useable Memory / Node | 995 GB | 503 GB | 995 GB | 503 GB | 503 GB |
| Accelerators / Node | None | None | None | 1 | 2 |
| Accelerator | n/a | n/a | n/a | NVIDIA V100 PCIe | NVIDIA V100 PCIe |
| Memory / Accelerator | n/a | n/a | n/a | 32 GB | 32 GB |
| Storage on Node | None | None | None | None | None |
| Interconnect | Cray Slingshot | Cray Slingshot | Cray Slingshot | Cray Slingshot | Cray Slingshot |
| OS | SLES | SLES | SLES | SLES | SLES |
| Priority | Queue Name | Max Wall Clock Time | Max Cores Per Job | Description |
|---|---|---|---|---|
| Highest | urgent | 168 Hours | 69,888 | Jobs belonging to DoD HPCMP Urgent Projects |
|
|
debug | 1 Hour | 2,816 | User testing |
| high | 168 Hours | 69,888 | Jobs belonging to DoD HPCMP High Priority Projects | |
| frontier | 168 Hours | 69,888 | Jobs belonging to DoD HPCMP Frontier Projects | |
| standard | 168 Hours | 69,888 | Standard jobs | |
| HIE | 24 Hours | 256 | Rapid response for interactive work | |
| transfer | 48 Hours | 1 | Data transfer for user jobs | |
| Lowest | background | 120 Hours | 2,816 | Unrestricted access - no allocation charge |