Shared License Buffer (SLB) User Guide
Table of Contents
1. Introduction
1.1. The Purpose of the SLB
The Software License Buffer (SLB) regulates shared license usage across all HPC systems by granting and enforcing license reservations for Abaqus, ANSYS, LS-DYNA, MATLAB, and STAR-CCM+.
The purpose of an SLB license reservation is to enable a job to check out its licenses when needed. To this end, checkouts are denied to jobs lacking a reservation. A license reservation is for a certain number of licenses, for one or more specific users, and for a specific time frame. Certain licenses cannot be checked out without a license reservation, and batch jobs are allowed to start only if their license reservation request has been approved.
In addition to "reservable" licenses, the SLB supports "requestable" licenses. MATLAB is supported this way. When a requestable license is specified, the batch queuing system will throttle the job until the license is available, just as it does for reservable licenses, except no reservation is created or required. This throttling of the job until the licenses are available greatly improves the chances of the licenses being available a minute or so later when the job's executable needs to check them out. But the lack of a reservation means there is no guarantee, as there is a small chance another job will check out the licenses first. Nor is there a capability to reserve a requestable license in advance. So, a requestable license is less robust than a reservable license but is nevertheless helpful because the batch queuing system will withhold a job until the requested licenses are available. In the future, the licenses for MATLAB could be changed based upon which policy is considered most helpful to users.
Users may access SLB for license reservations in three ways:
- Via the batch queuing system, by listing the needed licenses in the job script or job submission command
- Via the Advance Reservation System (ARS)
- Interactively by running an SLB script on any HPC system
This last option allows the user to make a license reservation for any time frame (present or future) without going through the batch queuing system or ARS. The intent is to allow users to reserve licenses for interactive work. This is described in detail later.
In addition to this SLB User's Guide, documentation is available in the SLB directory on each HPC in the "docs" subdirectory.
1.2. Requesting Assistance
The HPC Help Desk is available to help users with unclassified problems, issues, or questions. Analysts are on duty 8:00 a.m. - 8:00 p.m. Eastern, Monday - Friday (excluding Federal holidays).
E-mail: help@helpdesk.hpc.mil
Phone: 1-877-222-2039 or (937) 255-0679
2. Using the SLB via PBS or Slurm
To use shared licenses in a batch queuing system job, a user only needs to specify which licenses the batch job needs. This specification is most easily done in the job script but can be done on the job submission command using the job resource option. The queueing system and the SLB handle the license reservation processing. SLB processing notifies the license manager of which users are allowed to check out licenses and how many.
Before starting a job, the batch queueing system makes a license reservation request to the SLB for the licenses requested and the job's time frame. The SLB determines if enough licenses are available for the job's time frame (from the current time until the end of the job's wall-clock time), considering the number of licenses requested for the job, and the existing license reservations for the application. If sufficient licenses are available for the entire duration of the job, the SLB creates a license reservation for it and notifies the batch queuing system that it can start the job. If there are not enough licenses to make a reservation, then the SLB tells the batch queuing system to not start the job, and the batch queuing system will try again later. After granting a reservation, the SLB coordinates with the license manager before giving final approval for the job to start. Proper checkout of the licenses is dependent on the license manager following the instructions given to it by the SLB.
To get a license for an application, use one of the following batch jobs:
| Application | PBS | Slurm |
|---|---|---|
| Abaqus | #PBS -l abaqus=N | #SBATCH -L abaqus:N |
| ANSYS_CFD | #PBS -l cdf_solve_level1=1
#PBS -l ansys=1 #PBS -l anshpc=N #PBS -l anshpc_pack=N |
#SBATCH -L cfd_solve_level1:1
#SBATCH -L anshpc_pack:N #SBATCH -L anshpc:N #SBATCH -L ansys:1 |
| LS-DYNA | #PBS -l lsdyna=N
#PBS -l mppdyna=N |
#SBATCH -L lsdyna:N
#SBATCH -L mppdyna:N |
| MATLAB | #PBS -l MATLAB=1 (or matlab=1)
#PBS -l MATLAB_Distrib_Comp_Engine=N |
#SBATCH -L MATLAB:1
#SBATCH -L MATLAB_Distrib_Comp_Engine:N |
| STAR-CCM+ | #PBS -l ccmppowerplus=1 | #SBATCH -L ccmppowerplus:1 |
3. Using the SLB Interactively
There are SLB utilities that display license reservations, licensed features, and current license availability. There are also several "query" scripts that answer specific questions about when and how many licenses will be available in the future.
To make a reservation interactively, run the utility SLB_make_resv.pl and specify the application, number of licenses, and time window. The SLB determines whether the requested number of licenses is available throughout the specified time window. If so, the license reservation request is approved, and the user is notified of the license reservation. If the SLB determines an insufficient number of licenses are available, the SLB denies the license request and notifies the user. The user may then make a different request for either fewer licenses or for a different time window.
If one wants to obtain information on license availability beforehand to make a more informed reservation request (perhaps through ARS), one can run SLB queries (listed in the chart below) to answer the following questions:
- How many licenses are available now?
- When will N licenses be available?
- How many licenses will be available at a given time?
- How many licenses will be available throughout a certain time window?
- Could a reservation for N licenses be made for a certain time window?
The last query is the same as making a reservation request except a reservation is not made; instead, the SLB merely informs the user as to whether the request would have been approved or denied. And as mentioned earlier, the user can run a utility to display all reservations. All these queries inform the users about license availability so they may make the best license reservation(s) appropriate for their needs.
3.1. Command-Line Scripts
Using the SLB interactively is a simple matter of picking the right script and running it with the appropriate arguments. Some arguments are optional, some are required, and some have defaults. If a script is run with a required argument missing, it outputs a "usage" message prompting the user to supply the needed argument.
The command line interface includes the scripts below, which are available on all the HPC systems. The SLB directory on each HPC has additional scripts as well. Additional information about each of the commands can be found in the sections following the chart.
| Function | Example |
|---|---|
| Make a license reservation | SLB_make_resv.pl -app matlab -nlic 1 -user username -st 02/09_08:45 -et 02/09_16:00 |
| Terminate a license reservation | SLB_term_resv.pl -app matlab -id CarI54321 |
| Display current availability | SLB_all_avail.pl abaqus |
| Display all license reservations | SLB_disp_all_resv.pl -app matlab |
| Display ARS license reservations | SLB_disp_ARS_resv.pl |
| Display job license reservations | SLB_disp_job_resv.pl |
| Display interactive license reservations | SLB_disp_inter_resv.pl |
| Display reservable and requestable features | SLB_disp_licensed_features.pl abaqus |
| Display reservable features | SLB_disp_reservable_features.pl |
| Query number of licenses avail in time window | SLB_query_avail_block.pl -app acfd -st 12/23_08:00 -et 12/26_15:00 -tok cfd_solve_level1 |
| Query number of licenses avail at time t | SLB_query_avail_t.pl -app acfd -st 12/23_08:00 -tok cfd_solve_level1 |
| Query a reservation | SLB_query_resv.pl -app acfd -nlic 1 -st 02/09_06:40 -et 02/09_11:00 |
| Query when N licenses will be available | SLB_query_when.pl -app acfd -tok anshpc=128 |
| Compute number of Abaqus licenses needed | SLB_abaqus_map.pl -ncpu1 78 -ncpu2 104 |
| See internal SLB limitations | SLB_disp_limits.pl |
| See projection of future license availability | SLB_disp_timeline.pl |
| See meaning of SLB return codes | SLB_show_rc.pl |
Several of the scripts above require a time window. Each of them has a default time window but allows specification of an arbitrary time window in any of the following ways:
The start and end times can be specified in month/day_hour:minute format:
-st MM/DD_HH:mm -et MM/DD_HH:mm
The start and end times can be specified in epoch time (number of seconds since January 1, 1970 00:00:00 UT):
-ts epoch_start -te epoch_end
The start time can be specified as -st MM/DD_HH:mm, and the duration can be specified in hours:
-st MM/DD_HH:mm -dur hours
The start time can be specified in epoch time, and the duration can be specified in hours:
-ts epoch_start -dur hours
The following section describes the commands in the Command-Line References table, above.
3.1.1. SLB_make_resv.pl
Use SLB_make_resv.pl to request a license reservation. The user is notified if the request is granted or denied. If granted, the reservation ID is displayed.
Required inputs include application -app app and number of licenses: -nlic N or -tok "feature1=N feature2=M". A time window is also needed and can be specified as described above with two exceptions:
(1) the duration is specified in minutes, and
(2) the duration default is 120 (2 hours).
Unless the default time window of two hours is desired, the time window must be specified. Users should specify the user with -user username, but if omitted the script attempts to identify the user.
The following is an example of how to use SLB_make_resv.pl:
SLB_make_resv.pl -app matlab -nlic 1 -host narwhal -user username -st 02/09_08:45 -et 02/09_16:00
After making a reservation, a user should run SLB_disp_resv.pl to ensure the reservation is correct. If not, cancel any faulty reservation by running SLB_term_resv.pl and remake the reservation with SLB_make_resv.pl.
3.1.2. SLB_term_resv.pl
This script terminates a reservation. Unless a reservation is terminated explicitly by SLB_term_resv.pl (either by the user or by the queueing system when the batch job ends), the reservation will expire at the end of its time window. As soon as the licenses are no longer needed, SLB_term_resv.pl should be called to free up the licenses for other users. The two required inputs for SLB_term_resv.pl are application (-app app) and ID (-id ID). The SLB assigns the ID when the user creates the reservation. The user can find the ID by running SLB_disp_resv.pl.
3.1.3. SLB_disp_all_resv.pl
This script displays license reservations. It first shows the overall time window and then displays the reservations per application in chronological order. The easiest way to run this command is with no arguments, but input arguments can be used to specify the desired time window as described above, the type of reservations displayed (the batch queuing system, ARS, or interactive), or just one application.
The default time window starts at the current time and has a duration of 365 days. If the user specifies an application (e.g., -app abaqus), the SLB returns the license reservations for that application only. If the user does not specify an application, reservations for all applications are shown.
If only one type of reservation is desired (type=job, ARS, or interactive), then it may be specified using -type J, -type A, or -type I, for job, ARS, or interactive reservations respectively. If no type is specified, reservations of all three types are shown. But it is easier to use one of the next three SLB scripts to display reservations for just one type.
3.1.4. SLB_disp_ARS_resv.pl
This script displays ARS reservations for all applications. Its time window starts at the present and has a 365-day duration.
3.1.5. SLB_disp_job_resv.pl
This script displays batch job reservations for all applications. Its time window starts at the present and has a 365-day duration.
3.1.6. SLB_disp_inter_resv.pl
This script displays interactive reservations for all applications. By default, its time window starts at the present (it can be specified to start later) and has a 365-day duration.
3.1.7. SLB_disp_licensed_features.pl
This script shows reservable and requestable licensed features of each SLB application. For an application, it first shows the licensed features that can be reserved and then the requestable licenses. For each feature, the output shows the current number of licenses available. This script optionally accepts one application as an argument. If no argument is given, it generates output for all supported applications.
3.1.8. SLB_disp_reservable_features.pl
This script shows the reservable features of each application and the number of licenses under the control of the SLB, which excludes licenses dedicated for groups or individuals. This script takes no arguments.
3.1.9. SLB_query_avail_block.pl
This script queries the number of licenses available for a given period (block) of time. The block of time is specifiable as described above using -st and -et, or -ts and -te. By default, its time window starts at the present and lasts four hours.
3.1.10. SLB_query_avail_t.pl
This script queries the number of licenses available at a given time. The time is specifiable as described above using -st or -ts. If not specified, the time defaults to the current time. The output shows not only how many licenses are available at the specified time, but how long after that they will remain available.
3.1.11. SLB_query_resv.pl
This script works virtually identically to SLB_make_resv.pl (described above), except it does not make a reservation. A user does not need to specify username or host, but all other inputs are like those of SLB_make_resv.pl. Instead of making a reservation, this script gives a return code according to whether the requested license reservation could be made at the present time.
3.1.12. SLB_query_when.pl
This query returns the time at which the specified number of licenses will first be available and how long they will be available.
3.1.13. SLB_abaqus_map.pl
This utility computes the number of Abaqus licenses needed for an N core job. The formula it computes is P*N^(0.422), where P=3 for jobs using Abaqus Foundation functionality, P=4 for jobs using Foundation and Design or Foundation and Direct Coupling Interface, P=5 for jobs using Standard functionality, or P=6 for jobs using Aqua functionality. This script accepts either a single value for N (-ncpu N) or a range N1 -> N2 (-ncpu1 N1 -ncpu2 N2). This script also displays a set of directives that the user may use to have the queueing system compute the number of licenses for the job.
4. Licensed Features
The SLB controls the licenses of some (but not all) licensed features. The advantage of the SLB controlling licenses for a feature is that users may reserve the license and be assured of being able to use that feature during the reservation period.
Some licensed features have so many licenses that they would never run out, in which case there is no advantage in making the feature reservable. Having the SLB control the licenses of the feature burdens the user with making a reservation but accomplishes nothing since the licenses would rarely be exhausted. Therefore, certain features are deliberately not controlled by the SLB, which relieves users from the need to make a reservation.
4.1. Licenses and Licensed Features
The main application licenses and licensed features controlled by the SLB are as follows:
| Application | Main License | Licensed Features |
|---|---|---|
| Abaqus | abaqus | N/A |
| ANSYS_CFD | cfd_solve_level1 | ansys, anshpc, anshpc_pack |
| LS-DYNA | lsdyna | mppdyna |
| MATLAB | MATLAB | MATLAB_Distrib_Comp_Engine |
| STAR-CCM+ | ccmppowerplus | N/A |
4.2. ANSYS_CFD (acfd, formerly Fluent)
The Ansys licenses are named cfd_solve_level1, ansys, anshpc, and anshpc_pack. A parallel job running on N cores requires one cfd_solve_level1 license and N anshpc licenses.
Ansys pack licenses, labeled as anshpc_pack, allow for greatly expanded Ansys job core usage when paired with Ansys CFX or multiphysics solvers. The Ansys pack licenses have a multiplier effect, wherein the Pack licenses scale by 4X cores for each Pack used. Due to this scaling effect, Pack license usage is not viable below four Packs, so no jobs should be run requesting fewer than four Pack licenses. In this case, use "anshpc" licenses.
Please refer to the table below to clarify how many Ansys Packs are required for specific job core counts:
| HPC Packs | HPC Pack Cores | Solver Cores | Total Cores |
|---|---|---|---|
| 0 | 0 | 4 | 4 |
| 1 | 8 | 4 | 12 |
| 2 | 32 | 4 | 36 |
| 3 | 128 | 4 | 132 |
| 4 | 512 | 4 | 516 |
| 5 | 2048 | 4 | 2052 |
| 6 | 8192 | 4 | 8196 |
| 7 | 32728 | 4 | 32732 |
| 8 | 131072 | 4 | 131076 |