Shared License Buffer (SLB) User Guide

Table of Contents

1. SLB Basics

1.1. Overview

The Software License Buffer (SLB) regulates shared license usage across all HPC systems by granting and enforcing license reservations for Abaqus, ANSYS, LS-DYNA, MATLAB, STAR-CCM+, and US3D.

A license reservation is granted for a certain number of licenses, for a specific time window, and optionally, for certain licensed features of an application. Licenses cannot be checked out without a license reservation, and batch jobs are started only when a license reservation request has been approved.

Users may access the SLB for license reservations in three ways:

  1. Via the scheduler (PBS or Slurm), by designating the needed licenses in the job script
  2. Interactively by running an SLB script on any HPC system
  3. Via the Advance Reservation Service (ARS)

1.2. Reservable Licensed Features

The following table shows all licensed application features controlled by SLB.

Application Licenses and Features Controlled by SLB
Application Main License Licensed Features
Abaqus abaqus N/A
ANSYS cfd_solve_level1 / ansys anshpc anshpc_pack
LS-DYNA lsdyna / mppdyna N/A
STAR-CCM+ ccmppowerplus N/A

1.3. Requestable Licensed Features

The following licensed features may be requested in scheduler jobs if the user wants the scheduler to hold the job until the features are available. These features are not reservable through the SLB, so their availability is not guaranteed.

US3D: us3d

MATLAB: Communication_Blocks, Communication_Toolbox, Compiler, Control_Toolbox, Curve_Fitting_Toolbox, Distrib_Computing_Toolbox, Fixed-Point_Blocks, Fixed_Point_Toolbox, GADS_Toolbox, Identification_Toolbox, Image_Toolbox, MATLAB_Coder, MATLAB_Report_Gen, Neural_Network_Toolbox, Optimization_Toolbox, PDE_Toolbox, Real-Time_Workshop, SIMULINK, SIMULINK_Report_Gen, Signal_Blocks, Signal_Toolbox, Simulink_Control_Design, Simulink_Design_Optim, Statistics_Toolbox, Symbolic_Toolbox, Wavelet_Toolbox

1.4. Requesting Assistance

The HPC Help Desk is available to help users with unclassified problems, issues, or questions. Analysts are on duty 8:00 a.m. - 8:00 p.m. Eastern, Monday - Friday (excluding Federal holidays).

2. Using the SLB via the Scheduler

To use shared licenses in a batch job, all you must do is specify which licenses the scheduler needs to access. After that, you don’t need to do anything else since the scheduler and the SLB internally handle the license reservation processing.

To get the main license for an application, use one of the following:

SLB License Request Specifications
Application PBS/Slurm Sample Request Shortcut
Abaqus #PBS -l abaqus=N
#SBATCH –L abaqus:N
Ansys CFD
(formerly Fluent)		 #PBS -l cfd_solve_level1=1
#PBS -l ansys=1
#PBS -l anshpc=N
#PBS -l anshpc_pack=N
#SBATCH -L cfd_solve_level1:1
#SBATCH -L ansys:1
#SBATCH -L ansys:N
#SBATCH -L anshpc_pack:N
LS-DYNA #PBS -l mppdyna=N
#PBS -l lsdyna=N
#SBATCH -L mppdyna:N
#SBATCH -L lsdyna:N
Matlab #PBS -l matlab=1
#PBS -l Matlab_Distrib_Comp_Engine=N
#SBATCH -L matlab:1
#SBATCH -L Matlab_Distrib_Comp_Engine:N		 -l matlab=N
(where N>1)
Star-CCM+ #PBS -l ccmppowerplus=1
#SBATCH -L ccmppowerplus:1		 -l starccm=1
-l starccm:1
us3d #PBS -l us3d=1
#SBATCH -L us3d:1

3. Using the SLB Interactively

To make a reservation interactively (limited by application and restricted to a 2-hour max), run the utility SLB_make_resv.pl and specify the application and number of licenses. The SLB determines whether the requested number of licenses is available. If so, the license reservation request is approved, and you are notified that you have a license reservation. If the SLB determines an insufficient number of licenses are available, the SLB denies the license request and notifies you. The user may then make a different request for either fewer licenses or for a different time window.

3.1. Command-Line Scripts

Using the SLB interactively via the command line interface is a simple matter of picking the appropriate script and running it with the correct arguments. Some arguments are optional, some are required, and some have defaults. If a script is run with a required argument missing, it will output a "usage" message prompting the user to supply the needed argument.

The command-line interface enables the functions shown below, which can be performed on the HPC systems. Additional information about each of the commands can be found in section 3.2, below.

Command-Line References
Function Command
Make a license reservation SLB_make_resv.pl -app app -nlic N -host hostname -user username -st start_datetime -et end_datetime
Terminate a license reservation SLB_term_resv.pl -app app -id resv_id
Display licenses currently available SLB_all_avail.pl app
Display all license reservations SLB_disp_all_resv.pl -app app
Display ARS license reservations SLB_disp_ARS_resv.pl
Display job license reservations SLB_disp_job_resv.pl
Display interactive license reservations SLB_disp_inter_resv.pl
Display reservable and requestable features SLB_disp_licensed_features.pl app
Display reservable features SLB_disp_reservable_features.pl
Query number of licenses avail in time window SLB_query_avail_block.pl -app app -st start_datetime -et end_datetime -tok feature1=N [feature2=M] (Example: -tok solve_level1)
Query number of licenses avail at time "t" SLB_query_avail_t.pl -app app -st start_datetime -tok feature1=N [feature2=M]
Query a reservation SLB_query_resv.pl -app app -nlic N -st start_datetime -et end_datetime
Query when N licenses will be available SLB_query_when.pl -app app -tok feature1=N [feature2=M]
Compute number of Abaqus licenses needed SLB_abaqus_map.pl -ncpu1 N

Several of the scripts above have a "time window," for making a license reservation, querying about license availability, or displaying license reservations. Each of these scripts has a default time window but allows specification of an arbitrary time window using input arguments. A time window can be specified in any of the four following ways:

The start and end times can be specified in month/day_hour:minute format: -st MM/DD_HH:mm -et MM/DD_HH:mm

The start and end times can be specified in epoch time (the time elapsed in seconds since Jan. 1, 1970): -ts epoch_start -te epoch_end

The start time can be specified as -st MM/DD_HH:mm, and the duration can be specified in hours: -st MM/DD_HH:mm -dur hours

The start time can be specified in epoch time and the duration can be specified in hours: -ts epoch_start -dur hours

The following section describes the commands in the Command-Line References Table above.

3.2. Script Explanations

3.2.1. SLB_make_resv.pl

Use SLB_make_resv.pl to request a license reservation. The user is notified if the request is granted or denied. If granted, the reservation ID is displayed.

Required inputs include application -app app and number of licenses: -nlic N or -tok feature1=N [feature2=M]. A time window of two (2) hours is provided.

The following is an example of how to use SLB_make_resv.pl: SLB_make_resv.pl -app matlab -nlic 1 -host narwhal

After finishing use of the reserved licenses, use SLB_term_resv.pl to release the reservation. Alternatively, the reservation will automatically terminate after 2 hours. Please note that not all SLB controlled applications are eligible for interactive reservations, and the SLB will indicate as such if a reservation is requested for a restricted application.

3.2.2. SLB_term_resv.pl

This script terminates a reservation. If a reservation is not terminated explicitly (i.e., by calling SLB_term_resv.pl or by the batch queuing system job ending), the reservation expires at the end of its reservation window. However, by calling SLB_term_resv.pl, a reservation is terminated immediately, freeing up the licenses in it. The two required inputs for SLB_term_resv.pl are application (-app app) and ID (-id ID). The SLB assigns the ID when the user creates the reservation. The user can find the ID by running SLB_disp_resv.pl.

3.2.3. SLB_disp_all_resv.pl

This script displays license reservations. It first shows the time window displaying reservations and then displays the reservations per application in chronological order. The easiest way to run this command is with no arguments, but input arguments can be used to specify the desired time window as described above, the type of reservations displayed (the batch queuing system, ARS, or interactive), or just one application.

The default time window starts at the present time and has a duration of 365 days.

If only one type of reservation is desired (type=job, ARS, or interactive), then it may be specified using -type J, -type A, or -type I. "J" is used for "job" reservations. If no type is specified, reservations of all three types are shown.

If the user specifies an application (e.g., via -app abaqus), the SLB returns the license reservations for that application. If the user does not specify an application, reservations for all applications are shown.

3.2.4. SLB_disp_ARS_resv.pl

This script displays ARS reservations for all applications. Its time window starts at the present and has a 365-day duration.

3.2.5. SLB_disp_job_resv.pl

This script displays batch job reservations for all applications. Its time window starts at the present and has a 365-day duration.

3.2.6. SLB_disp_inter_resv.pl

This script displays interactive reservations for all applications. Its time window starts at the present (by default but can be specified as later) and has a 365-day duration.

3.2.7. SLB_disp_licensed_features.pl

This script shows reservable and requestable licensed features of each SLB application. For an application, it first shows those licensed features that can be reserved and then the requestable licenses. A requestable licensed feature is one that when requested prompts the batch queuing system to throttle the job until a license is available. There is no guarantee of availability and no capability to reserve the license in advance. For each feature, the result shows the current number of licenses available. This script optionally accepts one application as an argument. If no argument is given, it generates output for all supported applications.

3.2.8. SLB_disp_reservable_features.pl

This script shows the reservable features of each application and the number of licenses under the control of the SLB, which excludes licenses dedicated for groups or individuals. This script takes no arguments.

3.2.9. SLB_query_avail_block.pl

This script queries the number of licenses available for a given period (block) of time. The block of time is specifiable as described above using -st and -et, or -ts and -te. If not specified, the time period defaults to a window starting at the present and lasting 4 hours.

3.2.10. SLB_query_avail_t.pl

This script queries the number of licenses available at a given time. The time is specifiable as described above using -st or -ts. If not specified, the time defaults to the present time. The output shows not only how many licenses are available at the specified time, but how long after that they will remain available.

3.2.11. SLB_query_resv.pl

This script works virtually identically to SLB_make_resv.pl (described above), except it does not make a reservation. It just verifies that the requested license reservation could be made at the present time. A user does not need to specify username or host, but all other inputs are like those of SLB_make_resv.pl. One difference is that this script will allow use of the -st and -et, or -ts and -te inputs.

3.2.12. SLB_query_when.pl

This query returns the time at which the specified number of licenses will first be available and how long they will be available.

3.2.13. SLB_abaqus_map.pl

This utility computes the number of Abaqus licenses needed for an N core job. The formula it computes is P*N^(0.422), where P=3 (for Foundation functionality), P=4 (for Foundation and Design or Foundation and Direct Coupling Interface), P=5 (for Standard functionality), or P=6 (for Aqua functionality). This script accepts either a single value for N (-ncpu N) or a range N1->N2 (-ncpu1 N1 -ncpu2 N2).

4. Advance Reservation Service

Reservations for a future time can be made through ARS located at https://reservation.hpc.mil/reserve/

Follow the online prompts to complete your reservation.

5. Application Specific Features

5.1. Abaqus

Abaqus users can run: SLB_abaqus_map.pl -ncpu N and select the number of licenses based on P. P values are:

P Values Related to Functionality
P Value Functionality
3 Foundation
4 Foundation & Design or
Foundation & Direct Coupling Interface
5 Standard
6 Aqua

5.2. ANSYS_CFD (acfd, formerly Fluent)

The licenses are named: cfd_solve_level1, ansys, anshpc, and anshpc_pack.

5.2.1. Overview

cfd_solve_level1: CFD solver license for CFX/Fluent
ansys: Multiphysics solver license
anshpc: Workgroup core license
anshpc_pack: Large-scale pack core (=>512 cores) license

5.2.2. anshpc and solve_level Licenses

The cfd_solve_level1 license provides Ansys CFD solver access at up to four CPU cores.

Therefore, a job running on more than 4 cores requires 1 cfd_solve_level1 license and N minus 4 anshpc licenses.

Example: 128 core-jobs: One cfd-solve_level1 license and 124 anshpc licenses.

5.2.3. anshpc_pack Licenses

Pack licenses are for large-scale computation at 512 cores or higher.

Pack licenses have a multiplying effect – each additional anshpc_pack license multiplies the existing total pack core count by 4, as shown in the table below.

Example: 512 cores x 4 = 2,048 2,048 cores x 4 = 8,192

Remember that the cfd_solve_level1 license provides 4 cores. So, your total number of cores is the total pack core count + the 4 cores provided by the solver license.

anshpc_pack Core Calculations
anshpc_pack Pack Cores
4 512
5 2,048
6 8,192
7 32,728
8 131,072

Why Is the Lowest Amount of Pack Licenses Requestable 4?

Due to the multiplying effect mentioned earlier, it is a waste of resources to use Pack licenses at any number less than four packs (512 cores). Three Packs would only grant 128 cores, two Packs would only grant 32 cores, etc. At these numbers, it is HPCMP policy to direct users to leverage the anshpc workgroup licenses.

Can I Run a Job That Requests Both Pack and Standard anshpc Licenses?

No. Attempting to reserve both pack and standard anshpc licenses will cause job failures. The two are not mixable; only one should be used. Even when running separate jobs, it can be problematic to have mixed use-cases of these license types, due to the application license server lacking awareness of which license types to allocate to which specific jobs. It is highly recommended to only use one type of license at any given timeframe, across jobs.

ansys Licenses

These are the Multiphysics licenses, and similarly to cfd_solve_level1 provide 4 cores. They, however, operate independently and similarly to the CFX/Fluent cfd_solve_level1 licenses. ansys licenses will work for CFX, Fluent, LFEMAG, and structural simulations. They will not work for HFSS jobs.