clusterjob.backends package¶
Package for default backends
-
class
clusterjob.backends.
ClusterjobBackend
[source]¶ Bases:
object
Abstract base class for all clusterjob backends. All backends must inherit from this and implement the interface specified below.
Attributes: - name (str) – (default) name of the backend
- extension (str) – extension to be used for job scripts
-
common_keys
= ['name', 'queue', 'time', 'nodes', 'ppn', 'threads', 'mem', 'stdout', 'stderr']¶
-
cmd_submit
(jobscript)[source]¶ Given a
JobScript
instance, return a command that submits the job to the scheduler. The returned command must be be a sequence of program arguments or a string, see args argument ofsubprocess.Popen
.
-
get_job_id
(response)[source]¶ Given the stdout from the command returned by
cmd_submit()
, return a job ID as a str, or None if the job ID cannot be determined
-
cmd_status
(run, finished=False)[source]¶ Given a
AsyncResult
instance, return a command (cf.cmd_submit()
) that queries the scheduler for the job status. Iffinished=True
, the command should be appropriate for a run that has already finished.
-
get_status
(respone, finished=False)[source]¶ Given the stdout from the command returned by
cmd_status()
, return one of the status code defined inclusterjob.status
, or None if the status cannot be determined.
-
cmd_cancel
(run)[source]¶ Given a
AsyncResult
instance, return a command (cf.cmd_submit()
) that cancels the run.
-
resource_headers
(jobscript)[source]¶ Given a
JobScript
instance, return a list of lines (no trailing newlines) that encode the resource requirements, to be added at the top of the rendered job script, between the shbang and the script body. At the very least, keys in the jobscript resources dict that are in the list ofcommon_keys
must be handled, or aResourcesNotSupportedError
must be raised.
-
replace_body_vars
(body)[source]¶ Given a multiline string that is the body of the job script, replace the placeholders for environment variables with backend-specific realizations, and return the modified body
At a minimum the following environment variables should be handled:
Core Environment Variables
$CLUSTERJOB_ID
- The job ID assigned by the scheduler after submission
$CLUSTERJOB_WORKDIR
- The directory on the cluster from which the job script was submitted.
$CLUSTERJOB_SUBMIT_HOST
- The hostname on which the job script was submitted.
$CLUSTERJOB_NAME
- The name of the job.
$CLUSTERJOB_NODELIST
- The hostname(s) on which the job script is running.