QueuesΒΆ

Many tasks in Ganga can take a lot of time from job submission to output download. Several things are already handled in the background by the Monitoring System, but you may have user tasks that you want to also push into the background that can run in parallel. This is where queues can be used.

To start with, you can view the state of the background threads by just typing queues:

Ganga In [38]: queues
Ganga Out [38]:
                        Ganga user threads:                         |             Ganga monitoring threads:
                        ------------------                          |              ------------------------
Name                       Command                    Timeout       | Name                       Command                      Timeout
----                       -------                    -------       | ----                       -------                      -------
User_Worker_0            idle                       N/A             | Ganga_Worker_0           idle                         N/A
User_Worker_1            idle                       N/A             | Ganga_Worker_1           idle                         N/A
User_Worker_2            idle                       N/A             | Ganga_Worker_2           idle                         N/A

Ganga user queue:
----------------
[]
Ganga monitoring queue:
----------------------
[]

[12:57:37]
Ganga In [39]:

To add a function call to the queue such as a submit call, do the following:

for i in range(1, 10):
    j = Job()
    queues.add(j.submit)

You can also supply your own functions as well as provide arguments to these functions:

def f(x):
    print(x)

queues.add(f, args=(123,))

Queues can also be used to submit subjobs in parallel:

j = Job()
j.splitter = GenericSplitter()
j.splitter.attribute = 'application.args'
j.splitter.values = [i for i in range(0, 10)]
j.parallel_submit = True
j.submit()