KaliVeda
1.12/06
Heavy-Ion Analysis Toolkit
|
Base class for interface to a batch job management system.
The different "batch" systems available are described in the $KVROOT/KVFiles/.kvrootrc or the user's $HOME/.kvrootrc, as in the following example:
BatchSystem: Xterm Xterm.BatchSystem.Title: Execute task in an X-terminal window Xterm.BatchSystem.DefaultJobOptions: -T #JobName# -e Xterm.BatchSystem.JobSubCmd: xterm Xterm.BatchSystem.JobScript: "$KVROOT/bin/KaliVedaAnalysis -b -n"&
For each named batch system, there may be a corresponding plugin which defines the actual class to use, if it is necessary to extend the functionality of the KVBatchSystem base class, as for example:
Plugin.KVBatchSystem: Xterm KVRootBatch KVMultiDet "KVRootBatch(const Char_t*)"
The "default" batch system is defined as follows:
#Default batch system Default.BatchSystem: Xterm
It can be accessed via the batch system manager (see KVBatchSystemManager):
Global pointer gBatchSystem is 0 if analysis task is not run in batch mode If it is non-zero, it gives access to the batch system used to run the task. Therefore, if the user puts
in her analysis code, she can then use gBatchSystem as follows:
Definition at line 77 of file KVBatchSystem.h.
Public Member Functions | |
KVBatchSystem (const Char_t *name) | |
virtual | ~ KVBatchSystem () |
virtual void | AlterJobs (TGWindow *, TList *) |
void | cd () |
Make this the default batch system. More... | |
virtual Bool_t | CheckJobParameters () |
Checks the job and ask for the job name if needed. More... | |
virtual void | Clear (Option_t *opt="") |
virtual void | GetBatchSystemParameterList (KVNameValueList &) |
virtual const Char_t * | GetDefaultJobOptions () const |
virtual const Char_t * | GetJobName () const |
virtual const Char_t * | GetJobSubCmd () const |
virtual const Char_t * | GetJobSubCmdLine () |
virtual KVList * | GetListOfJobs () |
KVNameValueList & | GetParameters () |
virtual void | Print (Option_t *="") const |
virtual void | PrintJobs (Option_t *="") |
virtual void | ReadBatchEnvFile (TEnv *) |
virtual void | Run () |
virtual void | SanitizeJobName () const |
virtual void | SetAnalyser (KVDataAnalyser *da) |
virtual void | SetBatchSystemParameters (const KVNameValueList &) |
Use the parameters in the list to set all relevant parameters for batch system. More... | |
virtual void | SetDefaultJobOptions (const Char_t *opt) |
virtual void | SetJobName (const Char_t *name) |
virtual void | SetJobScript (const Char_t *path) |
virtual void | SubmitJob () |
virtual void | SubmitTask (KVDataAnalyser *da) |
virtual void | WriteBatchEnvFile (TEnv *) |
![]() | |
KVBase () | |
Default constructor. More... | |
KVBase (const Char_t *name, const Char_t *title="") | |
Ctor for object with given name and type. More... | |
KVBase (const KVBase &) | |
copy ctor More... | |
virtual | ~ KVBase () |
virtual void | Copy (TObject &) const |
Make a copy of this object. More... | |
const Char_t * | GetLabel () const |
UInt_t | GetNumber () const |
UInt_t | GetNumberOfObjects () const |
virtual TObject * | GetObject () const |
const Char_t * | GetType () const |
Bool_t | HasLabel () const |
virtual Bool_t | IsCalled (const Char_t *name) const |
Bool_t | IsLabelled (const Char_t *l) const |
virtual Bool_t | IsType (const Char_t *typ) const |
virtual void | List () |
KVBase & | operator= (const KVBase &) |
copy assignment operator More... | |
Double_t | ProtectedGetX (const TF1 *func, Double_t val, int &status, Double_t xmin=0.0, Double_t xmax=0.0) const |
void | SetLabel (const Char_t *lab) |
virtual void | SetNumber (UInt_t num) |
virtual void | SetType (const Char_t *str) |
![]() | |
TNamed () | |
TNamed (const char *name, const char *title) | |
TNamed (const TNamed &named) | |
TNamed (const TString &name, const TString &title) | |
virtual | ~TNamed () |
virtual TObject * | Clone (const char *newname="") const |
virtual Int_t | Compare (const TObject *obj) const |
virtual void | FillBuffer (char *&buffer) |
virtual const char * | GetName () const |
virtual const char * | GetTitle () const |
virtual ULong_t | Hash () const |
virtual Bool_t | IsSortable () const |
virtual void | ls (Option_t *option="") const |
TNamed & | operator= (const TNamed &rhs) |
virtual void | SetName (const char *name) |
virtual void | SetNameTitle (const char *name, const char *title) |
virtual void | SetTitle (const char *title="") |
virtual Int_t | Sizeof () const |
![]() | |
TObject () | |
TObject (const TObject &object) | |
virtual | ~TObject () |
void | AbstractMethod (const char *method) const |
virtual void | AppendPad (Option_t *option="") |
virtual void | Browse (TBrowser *b) |
ULong_t | CheckedHash () |
virtual const char * | ClassName () const |
virtual void | Delete (Option_t *option="") |
virtual Int_t | DistancetoPrimitive (Int_t px, Int_t py) |
virtual void | Draw (Option_t *option="") |
virtual void | DrawClass () const |
virtual TObject * | DrawClone (Option_t *option="") const |
virtual void | Dump () const |
virtual void | Error (const char *method, const char *msgfmt,...) const |
virtual void | Execute (const char *method, const char *params, Int_t *error=0) |
virtual void | Execute (TMethod *method, TObjArray *params, Int_t *error=0) |
virtual void | ExecuteEvent (Int_t event, Int_t px, Int_t py) |
virtual void | Fatal (const char *method, const char *msgfmt,...) const |
virtual TObject * | FindObject (const char *name) const |
virtual TObject * | FindObject (const TObject *obj) const |
virtual Option_t * | GetDrawOption () const |
virtual const char * | GetIconName () const |
virtual char * | GetObjectInfo (Int_t px, Int_t py) const |
virtual Option_t * | GetOption () const |
virtual UInt_t | GetUniqueID () const |
virtual Bool_t | HandleTimer (TTimer *timer) |
Bool_t | HasInconsistentHash () const |
virtual void | Info (const char *method, const char *msgfmt,...) const |
virtual Bool_t | InheritsFrom (const char *classname) const |
virtual Bool_t | InheritsFrom (const TClass *cl) const |
virtual void | Inspect () const |
void | InvertBit (UInt_t f) |
virtual Bool_t | IsEqual (const TObject *obj) const |
virtual Bool_t | IsFolder () const |
R__ALWAYS_INLINE Bool_t | IsOnHeap () const |
R__ALWAYS_INLINE Bool_t | IsZombie () const |
void | MayNotUse (const char *method) const |
virtual Bool_t | Notify () |
void | Obsolete (const char *method, const char *asOfVers, const char *removedFromVers) const |
void | operator delete (void *ptr) |
void | operator delete[] (void *ptr) |
void * | operator new (size_t sz) |
void * | operator new (size_t sz, void *vp) |
void * | operator new[] (size_t sz) |
void * | operator new[] (size_t sz, void *vp) |
TObject & | operator= (const TObject &rhs) |
virtual void | Paint (Option_t *option="") |
virtual void | Pop () |
virtual Int_t | Read (const char *name) |
virtual void | RecursiveRemove (TObject *obj) |
void | ResetBit (UInt_t f) |
virtual void | SaveAs (const char *filename="", Option_t *option="") const |
virtual void | SavePrimitive (std::ostream &out, Option_t *option="") |
void | SetBit (UInt_t f) |
void | SetBit (UInt_t f, Bool_t set) |
virtual void | SetDrawOption (Option_t *option="") |
virtual void | SetUniqueID (UInt_t uid) |
virtual void | SysError (const char *method, const char *msgfmt,...) const |
R__ALWAYS_INLINE Bool_t | TestBit (UInt_t f) const |
Int_t | TestBits (UInt_t f) const |
virtual void | UseCurrentStyle () |
virtual void | Warning (const char *method, const char *msgfmt,...) const |
virtual Int_t | Write (const char *name=0, Int_t option=0, Int_t bufsize=0) |
virtual Int_t | Write (const char *name=0, Int_t option=0, Int_t bufsize=0) const |
Static Public Member Functions | |
static KVBatchSystem * | GetBatchSystem (const Char_t *plugin) |
![]() | |
static Bool_t | AreEqual (Double_t x, Double_t y, Long64_t maxdif=1) |
Comparison between two 64-bit floating-point values. More... | |
static void | BackupFileWithDate (const Char_t *path) |
static void | CombineFiles (const Char_t *file1, const Char_t *file2, const Char_t *newfilename, Bool_t keep=kTRUE) |
static void | Deprecated (const char *method, const char *advice) |
static Bool_t | FindClassSourceFiles (const Char_t *class_name, KVString &imp_file, KVString &dec_file, const Char_t *dir_name=".") |
static Bool_t | FindExecutable (TString &exec, const Char_t *path="$(PATH)") |
static const Char_t * | FindFile (const Char_t *search, TString &wfil) |
static const Char_t * | GetBINDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetDATABASEFilePath () |
static const Char_t * | GetDATADIRFilePath (const Char_t *namefile="") |
static Bool_t | GetDataSetEnv (const Char_t *dataset, const Char_t *type, Bool_t defval) |
static const Char_t * | GetDataSetEnv (const Char_t *dataset, const Char_t *type, const Char_t *defval) |
static Double_t | GetDataSetEnv (const Char_t *dataset, const Char_t *type, Double_t defval) |
static const Char_t * | GetETCDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetExampleFilePath (const Char_t *library, const Char_t *namefile) |
Return full path to example file for given library (="KVMultiDet", "BackTrack", etc.) More... | |
static const Char_t * | GetINCDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetKVBuildDate () |
Returns KaliVeda build date. More... | |
static const Char_t * | GetKVBuildDir () |
Returns top-level directory used for build. More... | |
static const Char_t * | GetKVBuildTime () |
Returns KaliVeda build date. More... | |
static const Char_t * | GetKVBuildType () |
Returns KaliVeda build type (cmake build: Release, Debug, RelWithDebInfo, ...) More... | |
static const Char_t * | GetKVBuildUser () |
Returns username of person who performed build. More... | |
static const Char_t * | GetKVSourceDir () |
Returns top-level directory of source tree used for build. More... | |
static const Char_t * | GetKVVersion () |
Returns KaliVeda version string. More... | |
static const Char_t * | GetLIBDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetListOfPlugins (const Char_t *base) |
static const Char_t * | GetListOfPluginURIs (const Char_t *base) |
static const Char_t * | GetPluginURI (const Char_t *base, const Char_t *plugin) |
static void | GetTempFileName (TString &base) |
static const Char_t * | GetTEMPLATEDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetWORKDIRFilePath (const Char_t *namefile="") |
static const Char_t * | gitBranch () |
Returns git branch of sources. More... | |
static const Char_t * | gitCommit () |
Returns last git commit of sources. More... | |
static void | InitEnvironment () |
static bool | is_gnuinstall () |
static Bool_t | IsThisAPlugin (const TString &uri, TString &base) |
static TPluginHandler * | LoadPlugin (const Char_t *base, const Char_t *uri="0") |
static Bool_t | OpenContextMenu (const char *method, TObject *obj, const char *alt_method_name="") |
static void | OpenTempFile (TString &base, std::ofstream &fp) |
static void | PrintSplashScreen () |
Prints welcome message and infos on version etc. More... | |
static Bool_t | SearchAndOpenKVFile (const Char_t *name, std::ifstream &file, const Char_t *kvsubdir="", KVLockfile *locks=0) |
static Bool_t | SearchAndOpenKVFile (const Char_t *name, std::ofstream &file, const Char_t *kvsubdir="", KVLockfile *locks=0) |
static Bool_t | SearchKVFile (const Char_t *name, TString &fullpath, const Char_t *kvsubdir="") |
static const Char_t * | WorkingDirectory () |
![]() | |
static Longptr_t | GetDtorOnly () |
static Bool_t | GetObjectStat () |
static void | SetDtorOnly (void *obj) |
static void | SetObjectStat (Bool_t stat) |
Protected Member Functions | |
virtual void | ChangeDefJobOpt (KVDataAnalyser *da) |
![]() | |
virtual void | DoError (int level, const char *location, const char *fmt, va_list va) const |
void | MakeZombie () |
Protected Attributes | |
KVDataAnalyser * | fAnalyser |
the analyser object which requests job submission, it has all details on the job More... | |
KVString | fCurrJobName |
name of current job being submitted More... | |
KVString | fDefOpt |
default options for job submission command More... | |
KVString | fJobName |
base job name More... | |
KVString | fJobScript |
full path of shell script to be executed by batch system More... | |
KVString | fJobSubCmd |
shell command for submitting job More... | |
KVNameValueList | fParList |
list of parameters/switches to be passed on job submission command line More... | |
![]() | |
TString | fLabel |
label for the object More... | |
![]() | |
TString | fName |
TString | fTitle |
![]() | |
kOnlyPrepStep | |
Additional Inherited Members | |
![]() | |
enum | EKaliVedaBits { kIsKaliVedaObject = BIT(23) } |
![]() | |
enum | EDeprecatedStatusBits |
enum | EStatusBits |
![]() | |
kBitMask | |
kCanDelete | |
kCannotPick | |
kHasUUID | |
kInconsistent | |
kInvalidObject | |
kIsOnHeap | |
kIsReferenced | |
kMustCleanup | |
kNoContextMenu | |
kNotDeleted | |
kObjInCanvas | |
kOverwrite | |
kSingleKey | |
kWriteDelete | |
kZombie | |
#include <KVBatchSystem.h>
KVBatchSystem::KVBatchSystem | ( | const Char_t * | name | ) |
Constructor with name of batch system. Name will be used to retrieve resources from configuration file, e.g. batch system title, job submission command, name of batch script, default job options, runs per job in multijob mode:
[batch system name].BatchSystem.Title: [batch system name].BatchSystem.JobSubCmd: [batch system name].BatchSystem.JobScript: [batch system name].BatchSystem.DefaultJobOptions: set title of batch system
Definition at line 36 of file KVBatchSystem.cpp.
|
virtual |
Definition at line 156 of file KVBatchSystem.h.
void KVBatchSystem::cd | ( | ) |
Make this the default batch system.
Definition at line 165 of file KVBatchSystem.cpp.
|
protectedvirtual |
PRIVATE method called by SubmitTask() at moment of job submission. Depending on the current environment, the default job submission options may be changed by this method. For example, default options may be changed depending on the analysis task to be performed. We look for an environment variable of the form:
[batch system name].BatchSystem.DefaultJobOptions.[analysis task name]
and if found we use the options defined by it. If not, we use the default job options defined by [batch system name].BatchSystem.DefaultJobOptions
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 281 of file KVBatchSystem.cpp.
|
virtual |
Checks the job and ask for the job name if needed.
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 346 of file KVBatchSystem.cpp.
Clear previously set parameters in order to create a new job submission command (default options are not affected: use SetDefaultJobOptions to change them)
Reimplemented from KVBase.
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 149 of file KVBatchSystem.cpp.
|
static |
Create batch system object defined as a plugin in .kvrootrc If no plugin is found, we create a new KVBatchSystem base object which will be initialised from resources defined in .kvrootrc using its name. check and load plugin library
Definition at line 227 of file KVBatchSystem.cpp.
|
virtual |
Fill the list with all relevant parameters for batch system, set to their default values.
Parameters defined here are: JobName [string] AutoJobName [bool] AutoJobNameFormat [string]
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 445 of file KVBatchSystem.cpp.
|
inlinevirtual |
Definition at line 125 of file KVBatchSystem.h.
|
virtual |
Returns name of batch job, either during submission of batch jobs or when an analysis task is running in batch mode (access through gBatchSystem global pointer).
Depending on the batch system, some sanitization of the jobname may be required e.g. to remove "illegal" characters from the jobname. This is done by SanitizeJobName() before the jobname is returned.
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 319 of file KVBatchSystem.cpp.
|
inlinevirtual |
Definition at line 101 of file KVBatchSystem.h.
|
virtual |
Builds and returns static string containing full command line necessary to submit a batch job. This is constructed as follows:
[JobSubCmd] [default options] [par1 val1] [par2 val2] ... [JobScript]
The 'default options' can be set with SetDefaultJobOptions().
The 'par1,val1' pairs are the named parameters held in the fParList KVParameterList. For each parameter in the list, the name of the parameter is used for 'par1' and the value of the parameter is used for 'val1' etc. etc. e.g. if the job submission command requires an option to be passed with the following syntax [JobSubCmd] -Noption one of the parameters in the list should be called "-N" and its value should be 'option'
In order to add a simple flag such as [JobSubCmd] -V add a parameter called "-V" with value "".
The special variable #JobName# can be used anywhere and will be replaced by the jobname returned by GetJobName() at the moment of job submission.
Definition at line 100 of file KVBatchSystem.cpp.
|
virtual |
Create and fill list with KVBatchJob objects, one for each job currently handled by the batch system.
Needs to be implemented for specific systems in child classes. This method returns 0x0.
Reimplemented in KV_CCIN2P3_GE.
Definition at line 424 of file KVBatchSystem.cpp.
|
inline |
Definition at line 96 of file KVBatchSystem.h.
if option="log", print infos for batch log file if option="all", print detailed info on batch system
Reimplemented from KVBase.
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 396 of file KVBatchSystem.cpp.
Reimplemented in KVRootBatch, KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 107 of file KVBatchSystem.h.
Read any useful information on batch system from the TEnv (this method is used by KVDataAnalyser::ReadBatchEnvFile)
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 382 of file KVBatchSystem.cpp.
|
virtual |
Processes the job requests for the batch system. In normal mode, this submits one job for the data analyser fAnalyser In multijobs mode, this submits one job for each run in the runlist associated to fAnalyser
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 207 of file KVBatchSystem.cpp.
|
inlinevirtual |
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 152 of file KVBatchSystem.h.
|
inlinevirtual |
Definition at line 146 of file KVBatchSystem.h.
|
virtual |
Use the parameters in the list to set all relevant parameters for batch system.
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 467 of file KVBatchSystem.cpp.
fDefOpt.Form(opt,gSystem->GetGroupInfo()->fGroup.Data());
Definition at line 120 of file KVBatchSystem.h.
Set the job name. In MultiJobsMode this will be used as the base name for all jobs; each individual job will have the name 'basejobname_Rxxxx", with xxxx=run number for job.
The job name can be generated automatically by replacing certain special symbols in the name given here depending on the characteristics of the job. See KVDataAnalyser::ExpandAutoBatchName for allowed symbols.
Definition at line 489 of file KVBatchSystem.cpp.
Definition at line 115 of file KVBatchSystem.h.
|
virtual |
Submits a job to batch system, i.e. executes the string formed by GetJobSubCmdLine, if all necessary parameters have been given (any missing ones will be asked for). Parameters specific to a given batch system can be added my modifying the CheckJobParameters() method for the associated child class. set environment variables required by KaliVedaAnalysis batch executable
Definition at line 181 of file KVBatchSystem.cpp.
|
virtual |
Submit data analysis task described by KVDataAnalyser object to the batch system.
Note that the default options for this batch system may be changed just before job submission depending on the current environment, see ChangeDefJobOpt().
Reimplemented in KVPROOFLiteBatch.
Definition at line 251 of file KVBatchSystem.cpp.
Store any useful information on batch system in the TEnv (this method is used by KVDataAnalyser::WriteBatchEnvFile)
Reimplemented in KV_CCIN2P3_Slurm, and KV_CCIN2P3_GE.
Definition at line 368 of file KVBatchSystem.cpp.
|
protected |
the analyser object which requests job submission, it has all details on the job
Definition at line 81 of file KVBatchSystem.h.
|
mutableprotected |
name of current job being submitted
Definition at line 87 of file KVBatchSystem.h.
|
protected |
default options for job submission command
Definition at line 86 of file KVBatchSystem.h.
|
protected |
base job name
Definition at line 83 of file KVBatchSystem.h.
|
protected |
full path of shell script to be executed by batch system
Definition at line 85 of file KVBatchSystem.h.
|
protected |
shell command for submitting job
Definition at line 84 of file KVBatchSystem.h.
|
protected |
list of parameters/switches to be passed on job submission command line
Definition at line 82 of file KVBatchSystem.h.