ganga.GangaCore.GPIDev.Lib.Job.Job module

Job is an interface for submision, killing and querying the jobs :-).

Basic configuration:

The “application” attribute defines what should be run. Applications may be generic arbitrary executable scripts or complex, predefined objects.

The “backend” attribute defines where and how to run. Backend object represents a resource or a batch system with various configuration parameters.

Available applications, backends and other job components may be listed using the plugins() function. See help on plugins() function.

The “status” attribute represents the state of Ganga job object. It is automatically updated by the monitoring loop. Note that typically at the backends the jobs have their own, more detailed status. This information is typically available via “job.backend.status” attribute.

Bookkeeping and persistency:

Job objects contain basic book-keeping information: “id”, “status” and “name”. Job objects are automatically saved in a job repository which may be a special directory on a local filesystem or a remote database.

Input/output and file workspace:

There is an input/output directory called file workspace associated with each job (“inputdir” and “outputdir” properties). When a job is submitted, all input files are copied to the file workspace to keep consistency of the input while the job is running. Ganga then ships all files in the input workspace to the backend systems in a sandbox.

The list of input files is defined by the application (implicitly). Additional files may be explicitly specified in the “inputsandbox” attribute.

Job splitting:

The “splitter” attributes defines how a large job may be divided into smaller subjobs. The subjobs are automatically created when the main (master) job is submitted. The “subjobs” attribute gives access to individual subjobs. The “master” attribute of a subjob points back to the master job.

Postprocessors:

The “postprocessors” attribute is a list of actions to perform once the job has completed. This includes how the output of the subjobs may be merged, user defined checks which may fail the job, and an email notification.

Datasets: PENDING Datasets are highly application and virtual organisation specific.

ganga.GangaCore.GPIDev.Lib.Job.Job.const_lock

This is a context manager which acquires the const write lock on the object’s root object.

This lock acquires exclusive access over an object tree to prevent it from changing. Reading schema attributes on the object is still allowed but changing them is not. Only one thread can hold this lock at once.