Using Different Backends

One of the main benefits of Ganga is that you can submit to different clusters/systems (in Ganga, these are termed backends) by only changing one or two lines in your scripts. Though there are often very different ways of submission for each backend, Ganga tries to hide this as much as possible. We’ll cover the main options in Ganga Core below but to get a full list, use:

plugins("backends")

Local Backend

This is the default and refers to the machine that Ganga is running on. The job will be spawned as a separate process, independent of Ganga. Typical usage is:

j = Job()
j.backend = Local()
j.submit()

There are no editable options for the object itself but there are two config options that you can view with config.Local.

Batch Backends

Ganga supplies backend objects for most of the major batch systems around - LSF, PBS and Condor. You should obviously use the one that is relevant to the system you are running on. Typical usage is detailed below though as with all these, you can get more help using help(<backend>) and config.<backend>:

LSF

The default setup will work for LSF set up at CERN but you can change the patterns searched for in the output of the LSF commands if these are different in your particular setup.

j = Job()
j.backend = LSF()
j.backend.queue = '1nh'

PBS

Very similar to the LSF backend, this is setup by default to submit to a typical Torque/Maui installation but again, can easily be changed to reflect your specific setup:

j = Job()
j.backend = PBS()
j.submit()

Condor

Condor is a little different than the other backends but should still submit to most typical installations. There is also a requirements object that can be used to specify memory, architecture, etc.

j = Job()
j.backend = Condor()
j.backend.getenv = "True"  # send the environment to the host
j.backend.requirements.memory = 1200
j.submit()

Also note that the getenv option is defined as a string so in your .gangarc, you would need to set it to:

[Condor]
getenv = 'True'

To avoid Ganga attempting to assign a boolean instead.

Dirac Backend

To submit to a Dirac instance, you will need to have the Dirac client installed and a Dirac proxy available.

Using GridPP Dirac on CVMFS

There is an installed version of Dirac configured to use the GridPP Dirac instance available on the Ganga CVMFS area. To run with this, simply do the following:

  • If you are not running with on the GridPP VO, change the following in your ~/.gangarc file:
[defaults_DiracProxy]
group = <dirac user group>
  • After this, you can run Ganga with the Dirac plugins setup and pointing to the GridPP Dirac instance by simply running:
/cvmfs/ganga.cern.ch/runGanga-dirac.sh

Installing and Configuring the Dirac Client

If don’t have access to CVMFS then you will need to install and configure the Dirac client. How this is done will depend somewhat on your Dirac instance but for the one hosted by GridPP, first follow the installation instructions here

After successfully doing this, do the following steps to configure Ganga:

  • Edit your .gangarc file and set the following options:
[Configuration]
RUNTIME_PATH = GangaDirac

[LCG]
GLITE_SETUP = /home/<username>/dirac_ui/bashrc

[DIRAC]
DiracEnvSource = /home/<username>/dirac_ui/bashrc

[defaults_DiracProxy]
group = <dirac user group>

Testing Dirac Submission Through Ganga

To test everything is working, run Ganga (either using the CMVFS helper script runGanga-dirac.sh or as normal). It should ask you to generate a proxy if you haven’t already and then leave you at the IPython prompt.

To test that all is working, try to submit a basic job to the local machine you’re running and then to DIRAC:

j = Job()
j.backend = Dirac()
j.submit()

If you go to the Dirac portal (for GridPP, go here), you should now see your job! On completion, you can view the output of the job using:

j.peek('Ganga_Executable.log')

LCG Backends

There are several ways to submit ‘directly’ to the LCG grid through Ganga that are detailed below. However, all these require that you have a Grid UI installed and a Grid Proxy available. Ganga will check if you have a proxy when it needs it and prompt to create one if it’s not available. To set your config correctly, use the following:

#### LCG backend

#### ARC backend

#### CREAM backend

## Remote Backend

This backend allows you to submit a job ‘through’ a gateway host. IT IS CURRENTLY BEING FIXED!!!