Unclassified Systems

Centennial is currently Up.

Centennial is an SGI ICE XA system located at the ARL DSRC. It has 1,784 standard compute nodes, 32 large-memory compute nodes, and 32 GPU compute nodes (a total of 1,848 compute nodes or 73,920 compute cores). It has 253 TBytes of memory and is rated at 2.6 peak PFLOPS.

More Info

Node Configuration
Login Nodes Compute Nodes
Standard Memory Large Memory GPU Accelerated
Total Cores | Nodes 960 | 24 71,360 | 1,784 1,280 | 32 1,280 | 32
Operating System RHEL
Cores/Node 40 40 + 1 GPU
(1 x 2,880 GPU cores)
Core Type Intel Xeon
E5-2698v4 Broadwell
Intel Xeon
E5-2698v4 Broadwell
+NVIDIA Tesla K40P
Core Speed 2.2 GHz 2.6 GHz 2.2 GHz
Memory/Node 256 GBytes 128 GBytes 512 GBytes 256 GBytes
Accessible Memory/Node 252 GBytes 124 GBytes 508 GBytes 252 GBytes
Memory Model Shared on node. Shared on node.
Distributed across cluster.
Interconnect Type Ethernet / InfiniBand
Queue Descriptions and Limits
Priority Queue
Name
Max Wall
Clock Time
Max Cores
Per Job
Comments
Highest debug 1 Hour N/A User diagnostic jobs
Down arrow for decreasing priority transfer 48 Hours 1 Data transfer jobs
urgent 96 Hours N/A Designated urgent jobs by DoD HPCMP
staff 368 Hours N/A ARL DSRC staff testing only. System testing and user support
high 96 Hours N/A Designated high-priority projects by service/agency
frontier 168 Hours N/A Frontier projects only
cots 96 Hours N/A Abaqus, Fluent, and Cobalt jobs
interactive 12 Hours N/A Interactive jobs
standard-long 200 Hours N/A ARL DSRC permission required
standard 168 Hours N/A Normal user jobs
Lowest background 24 Hours User jobs that will not be charged against the project allocation
Gaffney is currently Up.

Gaffney is an HPE SGI 8600 system located at the Navy DSRC. It has 704 standard compute nodes, 16 large-memory nodes, and 32 GPU-accelerated nodes (a total of 752 compute nodes or 36,096 compute cores). It has 154 TBytes of memory and is rated at 3.05 peak PFLOPS.

More Info

Maintenance
Date / TimeDetails
2021 Sep 18 14:00 - Sep 19 10:21 CT (Completed) System Maintenance
2021 Sep 30 08:00 - 18:00 CT System Maintenance
Node Configuration
Login Nodes Compute Nodes
Standard Memory Large Memory GPU Accelerated
Total Cores | Nodes 384 | 8 33,792 | 704 768 | 16 1,536 | 32
Operating System RHEL
Cores/Node 48 48 + 1 GPU
(1 x 3,584 GPU cores)
Core Type Intel Xeon Platinum 8168 Intel Xeon Platinum 8168
+NVIDIA Tesla P100
Core Speed 2.7 GHz
Memory/Node 384 GBytes 192 GBytes 768 GBytes 384 GBytes
+16 GBytes
Accessible Memory/Node 380 GBytes 180 GBytes 744 GBytes 372 GBytes
Memory Model Shared on node. Shared on node.
Distributed across cluster.
Interconnect Type Intel Omni-Path
Queue Descriptions and Limits
Priority Queue
Name
Max Wall
Clock Time
Max Cores
Per Job
Comments
Highest urgent 24 Hours 768 Designated urgent projects by DoD HPCMP
Down arrow for decreasing priority frontier 168 Hours 14,400 Designated frontier projects by DoD HPCMP
high 168 Hours 15,840 Designated high-priority jobs by Service/Agency
debug 30 Minutes 2,400 User diagnostic jobs
standard 168 Hours 8,160 Normal priority user jobs
serial 168 Hours 1 Serial user jobs
gpu 24 Hours 48 GPU-accelerated jobs
transfer 48 Hours N/A Data transfer jobs
bigmem 96 Hours 288 Large-memory jobs
Lowest background 4 Hours 1,200 User jobs that will not be charged against the project allocation
Koehr is currently Up.

Koehr is an HPE SGI 8600 system located at the Navy DSRC. It has 704 standard compute nodes, 16 large-memory nodes, and 32 GPU-accelerated nodes (a total of 752 compute nodes or 36,096 compute cores). It has 154 TBytes of memory and is rated at 3.05 peak PFLOPS.

More Info

Maintenance
Date / TimeDetails
2021 Sep 18 14:00 - Sep 19 10:21 CT (Completed) System Maintenance
Node Configuration
Login Nodes Compute Nodes
Standard Memory Large Memory GPU Accelerated
Total Cores | Nodes 384 | 8 33,792 | 704 768 | 16 1,536 | 32
Operating System RHEL
Cores/Node 48 48 + 1 GPU
(1 x 3,584 GPU cores)
Core Type Intel Xeon Platinum 8168 Intel Xeon Platinum 8168
+NVIDIA Tesla P100
Core Speed 2.7 GHz
Memory/Node 384 GBytes 192 GBytes 768 GBytes 384 GBytes
+16 GBytes
Accessible Memory/Node 380 GBytes 180 GBytes 744 GBytes 372 GBytes
Memory Model Shared on node. Shared on node.
Distributed across cluster.
Interconnect Type Intel Omni-Path
Queue Descriptions and Limits
Priority Queue
Name
Max Wall
Clock Time
Max Cores
Per Job
Comments
Highest urgent 24 Hours 768 Designated urgent projects by DoD HPCMP
Down arrow for decreasing priority frontier 168 Hours 14,400 Designated frontier projects by DoD HPCMP
high 168 Hours 15,840 Designated high-priority jobs by Service/Agency
debug 30 Minutes 2,400 User diagnostic jobs
standard 168 Hours 8,160 Normal priority user jobs
serial 168 Hours 1 Serial user jobs
gpu 24 Hours 48 GPU-accelerated jobs
transfer 48 Hours N/A Data transfer jobs
bigmem 96 Hours 288 Large-memory jobs
Lowest background 4 Hours 1,200 User jobs that will not be charged against the project allocation
Mustang is currently running in a degraded state.

Mustang is an HPE SGI 8600 system located at the AFRL DSRC. It has 1,128 standard compute nodes, 24 large-memory compute nodes, and 24 GPU compute nodes (a total of 1,176 compute nodes or 56,448 compute cores). It has 244 TBytes of memory and is rated at 4.88 peak PFLOPS.

More Info

Status: Degraded

Mustang is currently running in a degraded state. Jobs may be negatively affected.

Node Configuration
Login Nodes Compute Nodes
Standard Memory Large Memory GPU Accelerated
Total Cores | Nodes 576 | 12 54,144 | 1,128 1,152 | 24 1,152 | 24
Operating System RHEL
Cores/Node 48 48 + 1 GPU
(1 x 3,584 GPU cores)
Core Type Intel Xeon Platinum 8168 Intel Xeon Platinum 8168
+NVIDIA Tesla P100
Core Speed 2.7 GHz
Memory/Node 384 GBytes 192 GBytes 768 GBytes 384 GBytes
+16 GBytes
Accessible Memory/Node 380 GBytes 180 GBytes 744 GBytes 372 GBytes
Memory Model Shared on node. Shared on node.
Distributed across cluster.
Interconnect Type Intel Omni-Path
Queue Descriptions and Limits
Priority Queue
Name
Max Wall
Clock Time
Max Cores
Per Job
Comments
Highest urgent 168 Hours 28,224 Jobs belonging to DoD HPCMP Urgent Projects
Down arrow for decreasing priority debug 1 Hour 1,152 User testing
high 168 Hours 28,224 Jobs belonging to DoD HPCMP High Priority Projects
frontier 168 Hours 28,224 Jobs belonging to DoD HPCMP Frontier Projects
standard 168 Hours 28,224 Standard jobs
HIE 24 Hours 96 Rapid response for interactive work
transfer 48 Hours 1 Data transfer for user jobs
Lowest background 120 Hours 48 Unrestricted access - no allocation charge
Narwhal is currently Up.

Narwhal is an HPE Cray EX system located at the Navy DSRC. It has 2,176 standard compute nodes, 12 large-memory nodes, 16 visualization accelerated nodes, 32 1-MLA accelerated nodes, and 32 2-MLA accelerated nodes (a total of 2,268 compute nodes or 290,304 compute cores). It has 590 TBytes of memory and is rated at 12.8 peak PFLOPS.

More Info

Maintenance
Date / TimeDetails
2021 Sep 18 14:00 - Sep 19 10:21 CT (Completed) System Maintenance
2021 Sep 30 09:00 - 18:00 CT System Maintenance
Node Configuration
Login Nodes Compute Nodes
Standard Memory Large Memory Visualization Accelerated 1-MLA Accelerated 2-MLA Accelerated
Total Cores | Nodes 1,408 | 11 278,528 | 2,176 1,536 | 12 2,048 | 16 4,096 | 32
Operating System SLES
Cores/Node 128 128 + 1 GPU
(1 x 5,120 CUDA cores,
1 x 640 Tensor cores)
128 + 2 GPUs
(2 x 5,120 CUDA cores,
2 x 640 Tensor cores)
Core Type AMD Epyc 7H12 (64 cores/socket) AMD Epyc 7H12
+NVIDIA Tesla V100-PCIE
Core Speed 2.6 GHz
Memory/Node 256 GBytes 1 TByte 256 GBytes
+ 1 x 32 GBytes
256 GBytes
+ 2 x 32 GBytes
Accessible Memory/Node 246 GBytes 990 GBytes 246 GBytes
+ 1 x 32 GBytes
246 GBytes
+ 2 x 32 GBytes
Memory Model Shared on node. Shared on node.
Distributed across cluster.
Interconnect Type HPE Slingshot
Queue Descriptions and Limits
Priority Queue
Name
Max Wall
Clock Time
Max Cores
Per Job
Comments
Highest urgent 24 Hours 16,384 Designated urgent projects by DoD HPCMP
Down arrow for decreasing priority frontier 168 Hours 32,768 Designated frontier projects by DoD HPCMP
high 168 Hours 16,384 Designated high-priority projects by Service/Agency
debug 30 Minutes 8,192 User diagnostic jobs
HIE 24 Hours 1,024 Rapid response for interactive work
viz 24 Hours 128 Visualization jobs
standard 168 Hours 16,384 Normal priority user jobs
mla 24 Hours 128 Machine Learning Accelerated jobs
smla 24 Hours 128 Machine Learning Accelerated jobs
dmla 24 Hours 128 Machine Learning Accelerated jobs
bigmem 24 Hours 224 Large-memory jobs
transfer 24 Hours N/A Data transfer jobs
Lowest background 4 Hours 1,024 User jobs that will not be charged against the project allocation.
Onyx is currently Up.

Onyx is a Cray XC40/50 system. It has 4,810 standard compute nodes, 4 large-memory compute nodes, 32 GPU compute nodes, 32 Knights Landing (Phi) compute nodes, and 64 Machine Learning Accelerator (MLA) multi-GPGPU nodes (a total of 4,942 compute nodes or 217,128 compute cores). It is rated at 6.06 peak PFLOPS.

More Info

Node Configuration
Login Nodes Compute Nodes
Standard Memory Large Memory KNL GPU Accelerated 2-MLA Accelerated 10-MLA Accelerated
Total Cores | Nodes 264 | 12 211,640 | 4,810 176 | 4 2,048 | 32 704 | 32 2,400 | 60 160 | 4
Operating System SLES Cray Linux Environment CentOS
Cores/Node 22 44 64 22 + 1 GPU
(1 x 3,584 CUDA cores)
40 + 2 GPUs
(2 x 5,120 CUDA cores)
40 + 10 GPUs
(10 x 5,120 CUDA cores)
Core Type Dual Intel E5-2699v4
Broadwell
(22 cores/socket)
Intel Phi 7230
Knights Landing
Intel E5-2699v4
Broadwell
+NVIDIA P100
Dual Intel 6148
Skylake
(20 cores/socket)
+NVIDIA V100 SXM2
Dual Intel 6148
Skylake
(20 cores/socket)
+NVIDIA V100 PCIe
Core Speed 2.8 GHz 1.3 GHz 2.8 GHz 2.4 GHz
Memory/Node 256 GBytes 128 GBytes 1 TByte 96 GBytes 256 GBytes
+16 GBytes
192 GBytes
+2 x 16 GBytes
768 GBytes
+10 x 32 GBytes
Accessible Memory/Node 8 GBytes 121 GBytes 991 GBytes 90 GBytes 247 GBytes
+16 GBytes
172 GBytes
+2 x 16 GBytes
735 GBytes
+10 x 32 GBytes
Memory Model Shared on node. Shared on node.
Distributed across cluster.
Interconnect Type Ethernet Cray Aries InfiniBand
Queue Descriptions and Limits
Priority Queue
Name
Max Wall
Clock Time
Max
Jobs
Min Cores
Per Job
Max Cores
Per Job
Comments
Highest urgent 24 Hours N/A 22 7,260 Designated urgent jobs by DoD HPCMP
Down arrow for decreasing priority test 24 Hours N/A 22 N/A Staff-only testing
debug 1 Hour 4 22 11,484 User testing
HIE 24 Hours 2 22 110 Rapid response for interactive work
frontier_lg 24 Hours 2 7,261 143,968 Frontier projects only (large jobs)
frontier_lw 168 Hours 15 22 15,708 Frontier projects only (long walltime)
frontier_sm 48 Hours 70 22 7,260 Frontier projects only (small jobs)
high_lg 24 Hours 2 8,449 105,820 Designated high-priority jobs by Service/Agency (large jobs)
high_lw 168 Hours 15 22 10,824 Designated high-priority jobs by Service/Agency (long walltime)
high_sm 24 Hours 70 22 8,448 Designated high-priority jobs by Service/Agency (small jobs)
frontier_md 96 Hours 2 15,709 34,540 Frontier projects only (medium sized, long walltime)
standard_lg 24 Hours 2 7,261 105,820 Normal priority jobs (large jobs)
standard_lw 168 Hours 3 22 5,808 Normal priority jobs (long walltime)
standard_sm 24 Hours 70 22 7,260 Normal priority jobs (small jobs)
transfer 48 Hours 6 1 1 Data transfer jobs. Access to the long-term storage
Lowest background 4 Hours 6 22 7,260 Unrestricted access - no allocation charge
SCOUT is currently Up.

SCOUT is an IBM Power9 system located at the ARL DSRC. It has 22 Training nodes, each with 6 nVidia V100 GPUs, 128 Inference nodes, each with 4 nVidia T4 GPUs, and 2 Visualization nodes, each with 2 nVidia GPUs (a total of 152 compute nodes or 6,080 cores). It has 45 TBytes of memory.

More Info

Node Configuration
Login Nodes GPU Accelerated Compute Nodes
Training Inference Visualization
Total Cores | Nodes 160 | 4 880 | 22 5,120 | 128 80 | 2
Operating System RHEL
Cores/Node 40 40 + 6 GPUs
(6 x 5,120 CUDA cores,
6 x 640 Tensor cores)
40 + 4 GPUs
(4 x 2,500 CUDA cores,
4 x 320 Tensor cores)
40 + 2 GPUs
(2 x 5,120 CUDA cores,
2 x 640 Tensor cores)
Core Type IBM Power9 IBM Power9
+NVIDIA Volta V100
IBM Power9
+NVIDIA T4 GPUs
IBM Power9
+NVIDIA Volta V100
Core Speed 2.55 GHz
Memory/Node 512 GBytes 700 GBytes 256 GBytes 512 GBytes
Accessible Memory/Node 502 GBytes 690 GBytes 246 GBytes 502 GBytes
Memory Model Shared on node. Shared on node.
Distributed across cluster.
Interconnect Type Ethernet / EDR InfiniBand
Queue Descriptions and Limits
Priority Queue
Name
Max Wall
Clock Time
Max Cores
Per Job
Comments
Highest transfer 48 Hours N/A Data transfer jobs
Down arrow for decreasing priority urgent 96 Hours N/A Designated urgent jobs by DoD HPCMP
debug 1 Hour N/A User diagnostic jobs
high 168 Hours N/A Designated high-priority projects by service/agency
frontier 168 Hours N/A Frontier projects only
HIE 24 Hours N/A Rapid response for interactive work
interactive 12 Hours N/A Interactive jobs
standard 168 Hours N/A Normal user jobs
Lowest background 24 Hours N/A User jobs that will not be charged against the project allocation