Check Status of Batch Jobs
The cqstat command is available on all user-accessible HPC systems at the participating centers. It is able to produce one of two reports: A Default Report, and a Priority Report. In both reports, information is displayed in two sections: Active Jobs, and Waiting Jobs. The two sections are separated by a summary that shows the number of cores and nodes in use for each type of node configured on the system. The Priority Report is identical to the Default Report with the addition of a priority column in the Waiting Jobs section.
Sample Output
$ cqstat --prio ---------------------------------------------------------ACTIVE JOBS------------------------------------------------------- JOBID JOBNAME USERNAME SUBPROJECT ST CORES NODES QUEUE REMAINING START TIME --------------------------------------------------------------------------------------------------------------------------- 1082416 Job_1234 useraaaa PROJECT012345 R 18432 512 background 0:03:59 Thu Jun 8 07:52:19 1082445 TestingAAAA userbbbb PROJECTAAAAAA R 64 2 debug 0:13:22 Thu Jun 8 08:05:35 1081154 prodrun03 usercccc PROJECT987654 R 252 9 standard 11:58:22 Wed Jun 7 20:35:50 1082185 my_projX userdddd PROJECT000000 R 560 20 high 14:03:05 Thu Jun 8 00:40:06 4 Active Jobs 18432 of 124416 Standard Compute Cores Active ( 14.81%) 512 of 3456 Standard Compute Nodes Active ( 14.81%) 252 of 896 GPU Co-Processors Active ( 28.13%) 9 of 32 GPU Nodes Active ( 28.13%) 64 of 128 Large Memory Cores Active ( 50.00%) 2 of 4 Large Memory Nodes Active ( 50.00%) 560 of 4984 Phi-Attached Standard Cores Active ( 11.24%) 20 of 178 Phi Nodes Active ( 11.24%) --------------------------------------------------------WAITING JOBS--------------------------------------------------------------------------------------------------------------- JOBID JOBNAME USERNAME SUBPROJECT ST CORES NODES QUEUE WALLCLOCK QUEUE TIME WAIT TIME EST START TIME PRIORITY ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1076944 run usereeee PROJECTZYXWVU Q 4608 128 standard 24:00:00 Thu Jun 1 10:08:55 6:22:28:35 Thu Jun 8 09:29:14 2017 86.8373 1080013 my_projY userdddd PROJECT000000 Q 560 20 high 72:00:00 Tue Jun 6 12:11:46 1:20:25:44 Sat Jun 10 20:35:51 2017 61.6933 1073628 Data_Coll_Main userffff PROJECT135790 Q 1080 30 standard 96:00:00 Tue May 30 18:54:02 8:13:43:28 Tue Jun 20 04:34:49 2017 46.5995 1082404 Job_5678 useraaaa PROJECT012345 Q 28 1 background 00:40:00 Thu Jun 8 07:48:24 0:49:06 Thu Jun 8 10:28:53 2017 -999.8600 Total Jobs: 8 Active jobs: 4 Waiting Jobs: 4
Description of Report Columns
Columns included in all reports:
- JOBID - ID number given to the job by the batch queuing system
- JOBNAME - Name given to the job by either the user or the batch queuing system
- USERNAME - Username of the person who submitted the job
- SUBPROJECT - Subproject whose allocation will be charged for time spent running this job
- ST - Job status: Q(ueued), H(eld), W(aiting), R(unning), etc.
- CORES - Number of computational cores of all types requested by the job
- NODES - Number of computational nodes of all types requested by the job
- QUEUE - Queue in which the job has been placed
Additional columns included in the Active Jobs list:
- REMAINING - Time remaining for the job to complete execution (HHH:MM:SS)
- START TIME - Start time of job execution (DAY MON DD HH:MM:SS)
Additional columns included in the Waiting Jobs list:
- WALLCLOCK - Requested maximum wall time for job execution (HHH:MM:SS)
- QUEUE TIME - Time when the job entered the batch queue (DAY MON DD HH:MM:SS)
- WAIT TIME - Time that the job has been waiting to run (DDD:HH:MM:SS)
- EST START TIME - Estimated start time for the job, based on current queue conditions (DAY MON DD HH:MM:SS)
Additional columns included in the Priority Report:
- PRIORITY - Calculated relative priority for each job, based on current queue conditions
*** NOTE: The values for Estimated Start Time and Priority change quickly.
Back to BC Policy FY10-06.