ganga.GangaCore.Lib.Splitters.GenericSplitter module

Split job by changing arbitrary job attribute.

This splitter allows the creation of a series of subjobs where the only difference between different jobs can be defined by giving the “attribute” and “values” of the splitter object.

For example, to split a job according to the given application arguments:

s = GenericSplitter() s.attribute = ‘application.args’ s.values = [[“hello”,”1”],[“hello”,”2”]] … … j = Job(splitter=s) j.submit()

To split a job into two LCG jobs running on two different CEs:

s = GenericSplitter() s.attribute = ‘backend.CE’ s.value = [“quanta.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-atlas”,”lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-atlas”] … … j = Job(backend=LCG(),splitter=s) j.submit()

to split over mulitple attributes, use the multi_args option:

j = Job() j.splitter = GenericSplitter() j.splitter.multi_attrs = { “application.args”:[“hello1”, “hello2”], “application.env”:[{“MYENV”:”test1”}, {“MYENV”:”test2”}] }

this will result in two subjobs, one with args set to ‘hello1’ and the MYENV set to ‘test1’, the other with args set to ‘hello2’ and the MYENV set to ‘test2’.

Known issues of this generic splitter:

• it will not work if specifying different backends for the subjobs

ganga.GangaCore.Lib.Splitters.GenericSplitter.const_lock

This is a context manager which acquires the const write lock on the object’s root object.

This lock acquires exclusive access over an object tree to prevent it from changing. Reading schema attributes on the object is still allowed but changing them is not. Only one thread can hold this lock at once.