Tip:
Highlight text to annotate it
X
In this video, you will learn how to submit simple batch jobs to the scheduler.
We will use the "qsub" command to submit all of our jobs. We submit a script via qsub which
is then sent to the scheduling system. The scheduler will then find free resources and
run the script on an available node.
qsub needs to know a few things in order to run jobs. First, it needs to know how many
resources the job requires. We have other videos dedicated to creating proper resource
requests, so you are encouraged to watch those videos as well. The tutorial on using the
Script Generator should be especially instructive.
The resource request is specified by using the "-l" parameter in either the script or
from the command line. To start off, we will run qsub using command line parameters.
Here, we are typing out a resource request to submit as part of our job. We are requesting
one node, one processor on that node, 100 MB of memory per process, a walltime of five
minutes, and to run in the "test" QOS.
The second thing that qsub needs to know is what script you want to run. You can specify
the filename of a script, or qsub can read it via standard in. In this example, we will
send qsub the "hostname" command to run. We'll add "echo hostname |" at the beginning of
the line. Then press enter. The scheduler then gives us the job ID. This is a number
that uniquely identifies the job that was just submitted. Let's check on that job using
the job ID. It can take a minute or two for the job to be visible to all commands, so
we may need to wait a little while. Here, "qstat" shows us that the job was queued successfully.
checkjob -v also shows us useful information about the job, including that the job is already
finished.
By default, when a job finishes it will put its output in two files inside of the directory
from which we ran qsub. The files will have the job name (STDIN in this case) and either
.o or .e, followed by the job ID. The standard output will be placed in the .o file. As we
can see here, the hostname command produced the output "m6-1-1.local" which is a compute
node on the system. If the program produced any warnings or errors through standard error,
it will show up in the .e file. This job produced no error output.
The other way to submit scripts to qsub is to pass a filename to it. Let's create a quick
script using vim. We will use bash to run it. We will also place the same resource request
in the script as we did on the command line. Inside of a script, the scheduler treats "#PBS"
specially. A "#PBS" line contains arguments that you could also use on the command line.
In this case, the "#PBS -l" line is the resource request. We can also put in a "-N" line that
gives the job a name. There are several other arguments that can be passed to qsub, but
they will not be covered in this tutorial. See the Script Generator for more examples.
We can now specify what commands to run just like in a normal bash shell script. We can
put in the same hostname command like before. We could also call something from /fslapps.
Several commands can be used, just like in a normal script.
Let's save the file and submit it. To submit a file, we just simply pass the filename to
qsub on the command line. It can be either an absolute or relative path. The job is now
submitted to qsub. Instead of using the "-l" line on the command line to specify a resource
request, qsub will parse the file we submitted for "#PBS" lines and use those instead.
This has been a brief demonstration on how to use qsub. Please see the other documentation
and video tutorials on our website for more information about proper resource requests
and how to manage queued jobs.