Base class for transferring data between data repositories.

Created by KVClassFactory on Mon Sep 18 14:27:21 2006 Author: franklan

If the user has defined two or more repositories in her $HOME/.kvrootrc file, then it is possible to transfer files from one to the other by running one of the following commands on the local machine:

KVDataTransfer::NewTransfer("ccali", "default")->Run() // transfer from ccali to local machine

KVDataTransfer::NewTransfer("default", "ccali")->Run() // transfer from local machine to ccali

KVDataTransfer::Run

virtual void Run()

Definition: KVDataTransfer.cpp:161

KVDataTransfer::NewTransfer

static KVDataTransfer * NewTransfer(const Char_t *source_rep, const Char_t *target_rep)

Definition: KVDataTransfer.cpp:72

The same menu-driven approach as for submitting analysis tasks is used to allow the user to choose among the available datasets, systems and runs to transfer. The transferred files are copied in to the target repository, creating any missing directories which may be needed.

It is also possible to use a KVDataTransfer object interactively, in this case you should use:

KVDataTransfer* trans = KVDataTransfer::NewTransfer("ccali", "default");
trans->SetDataSet(...);
trans->SetDataType(...);
trans->SetRuns(...);
trans->TransferRuns();

To use the BBFTP transfer facility, a bbftp client must be installed on the user's machine If the client is not located in the user's PATH, he should give the full path to the executable in the configuration, e.g. on WinXP with cygwin version of bbftp client:

ccali.DataRepository.FileTransfer.type: c:\cygwin\bin\bbftp.exe

c

#define c(i)

Definition at line 50 of file KVDataTransfer.h.

Public Member Functions
	KVDataTransfer ()

virtual	~ KVDataTransfer ()

virtual void	Run ()

virtual void	TransferRuns ()

Public Member Functions inherited from KVDataSetAnalyser
	KVDataSetAnalyser ()
	Default constructor. More...

virtual	~KVDataSetAnalyser ()
	Destructor. More...

virtual void	AddJobDescriptionList (TList *)

void	ChooseDataSet ()

void	ChooseDataType ()

void	ChooseRuns (KVDBSystem system=nullptr, const Char_t data_type="")

void	ChooseSystem (const Char_t *data_type="")

void	ClearRunList ()

TString	ExpandAutoBatchName (const Char_t *format) const

KVDBRun *	GetAnalysedRun () const

KVDBSystem *	GetAnalysedSystem () const

const KVDataSet *	GetDataSet () const

const KVNumberList &	GetFullRunList () const

const KV2Body *	GetKinematics () const
	Return pointer to kinematics of analysed system if defined. More...

Int_t	GetNumberOfFilesToAnalyse () const

const Char_t *	GetRecognisedAutoBatchNameKeywords () const

KVString	GetRootDirectoryOfDataToAnalyse () const
	Returns path to data to be analysed. More...

const KVNumberList &	GetRunList () const

Int_t	GetRunNumberFromFileName (const Char_t *)
	Get the run number from the filename. More...

const KVDBSystem *	GetSystem () const

Bool_t	IsChooseDataSet () const

Bool_t	IsChooseRuns () const

Bool_t	IsChooseSystem () const

Bool_t	IsChooseTask () const

Bool_t	ReadBatchEnvFile (const Char_t *filename)

void	Reset ()

void	SetChooseDataSet (Bool_t yes=kTRUE)

void	SetChooseRuns (Bool_t yes=kTRUE)

void	SetChooseSystem (Bool_t yes=kTRUE)

void	SetChooseTask (Bool_t yes=kTRUE)

void	SetDataSet (const Char_t *name)

void	SetDataSet (KVDataSet *ds)

void	SetFullRunList (const KVNumberList &nl)

void	SetRuns (const KVNumberList &nl, Bool_t check=kTRUE)

void	SetSystem (KVDBSystem *syst)
	Set the System used in the analysis. More...

TString	SystemBatchName () const

void	WriteBatchEnvFile (const Char_t *jobname, Bool_t save=kTRUE)

Public Member Functions inherited from KVDataAnalyser
	KVDataAnalyser ()
	Default constructor. More...

virtual	~ KVDataAnalyser ()

Bool_t	BatchMode () const

virtual Bool_t	CheckStatusUpdateInterval (Long64_t nevents) const

virtual Bool_t	CheckTaskVariables ()

void	ChooseNbEventToRead ()
	Ask user to set number of events to read. More...

void	ChooseRunningMode ()

void	ChooseUserClass ()
	Choose the user's analysis class. More...

void	DeleteBatchStatusFile () const
	Delete batch status file (and backup - '.bak') for batch job. More...

void	DoStatusUpdate (Long64_t nevents) const
	Print infos on events treated, disk usage, memory usage. More...

Bool_t	DoUserClassFilesExist ()

virtual TTree *	GetAnalysedTree () const

KVDataAnalysisTask *	GetAnalysisTask () const

const Char_t *	GetBatchName ()

const Char_t *	GetBatchStatusFileName () const
	Returns full path to file used to store status of running batch jobs. More...

virtual const KVBatchSystem *	GetBatchSystem ()

const KVString &	GetDataType () const

const Char_t *	GetLaunchDirectory () const

Long64_t	GetNbEventToRead (void) const

KVDataAnalyser *	GetParent () const

TString	GetPathToFileInLaunchDirectory (const TString &) const

EProofMode	GetProofMode () const

Double_t	GetStatusUpdateInterval () const

virtual Long64_t	GetTotalEntriesToRead () const

const Char_t *	GetUserClass ()

const KVString &	GetUserClassDec () const

const KVString &	GetUserClassImp () const

const KVString &	GetUserClassOptions () const

const KVString &	GetUserIncludes () const

const KVString &	GetUserLibraries () const

Bool_t	IsMenus () const

Bool_t	IsQuit () const

Bool_t	IsSubmit () const

Bool_t	IsUserClassValid () const

virtual void	postAnalysis ()

virtual void	postEndAnalysis ()

virtual void	postEndRun ()

virtual void	postInitAnalysis ()

virtual void	postInitRun ()

virtual void	preAnalysis ()

virtual void	preEndAnalysis ()

virtual void	preEndRun ()

virtual void	preInitAnalysis ()

virtual void	preInitRun ()

virtual void	RegisterUserClass (TObject *)

virtual void	RunMenus ()
	Run data analyser in menu-driven mode. More...

Bool_t	RunningInLaunchDirectory ()

virtual void	SetAnalysedTree (TTree *)

void	SetAnalysisTask (KVDataAnalysisTask *at)

virtual void	SetAuxFileList (TList *)

void	SetBatchMode (Bool_t on=kTRUE)

void	SetBatchName (const Char_t *batchname)

void	SetBatchSystem (KVBatchSystem *bs)

void	SetDataType (const Char_t *name)

virtual void	SetFileList (TList *)

void	SetMenus (Bool_t on=kTRUE)

void	SetNbEventToRead (Long64_t nb=0)

void	SetParent (KVDataAnalyser *da)

void	SetProofMode (EProofMode e)

void	SetQuit (Bool_t yes=kTRUE)

void	SetStatusUpdateInterval (double t)

void	SetSubmit (Bool_t yes=kTRUE)

virtual void	SetTriggerConditionsForRun (int)

void	SetUserClass (const Char_t *kvs, Bool_t check=kTRUE)

void	SetUserClassOptions (const Char_t *o="")

void	SetUserIncludes (const Char_t *incDirs=0)

void	SetUserLibraries (const Char_t *libs=0)

void	UpdateBatchStatusFile (Int_t totev, Int_t evread, TString disk) const
	Update infos in batch status file. More...

void	UseBaseClassSubmitTask ()

void	WriteBatchInfo (TTree *)

Public Member Functions inherited from TObject
	TObject ()

	TObject (const TObject &object)

virtual	~TObject ()

void	AbstractMethod (const char *method) const

virtual void	AppendPad (Option_t *option="")

virtual void	Browse (TBrowser *b)

ULong_t	CheckedHash ()

virtual const char *	ClassName () const

virtual void	Clear (Option_t *="")

virtual TObject *	Clone (const char *newname="") const

virtual Int_t	Compare (const TObject *obj) const

virtual void	Copy (TObject &object) const

virtual void	Delete (Option_t *option="")

virtual Int_t	DistancetoPrimitive (Int_t px, Int_t py)

virtual void	Draw (Option_t *option="")

virtual void	DrawClass () const

virtual TObject *	DrawClone (Option_t *option="") const

virtual void	Dump () const

virtual void	Error (const char method, const char msgfmt,...) const

virtual void	Execute (const char method, const char params, Int_t *error=0)

virtual void	Execute (TMethod method, TObjArray params, Int_t *error=0)

virtual void	ExecuteEvent (Int_t event, Int_t px, Int_t py)

virtual void	Fatal (const char method, const char msgfmt,...) const

virtual TObject *	FindObject (const char *name) const

virtual TObject *	FindObject (const TObject *obj) const

virtual Option_t *	GetDrawOption () const

virtual const char *	GetIconName () const

virtual const char *	GetName () const

virtual char *	GetObjectInfo (Int_t px, Int_t py) const

virtual Option_t *	GetOption () const

virtual const char *	GetTitle () const

virtual UInt_t	GetUniqueID () const

virtual Bool_t	HandleTimer (TTimer *timer)

virtual ULong_t	Hash () const

Bool_t	HasInconsistentHash () const

virtual void	Info (const char method, const char msgfmt,...) const

virtual Bool_t	InheritsFrom (const char *classname) const

virtual Bool_t	InheritsFrom (const TClass *cl) const

virtual void	Inspect () const

void	InvertBit (UInt_t f)

virtual Bool_t	IsEqual (const TObject *obj) const

virtual Bool_t	IsFolder () const

R__ALWAYS_INLINE Bool_t	IsOnHeap () const

virtual Bool_t	IsSortable () const

R__ALWAYS_INLINE Bool_t	IsZombie () const

virtual void	ls (Option_t *option="") const

void	MayNotUse (const char *method) const

virtual Bool_t	Notify ()

void	Obsolete (const char method, const char asOfVers, const char *removedFromVers) const

void	operator delete (void *ptr)

void	operator delete[] (void *ptr)

void *	operator new (size_t sz)

void *	operator new (size_t sz, void *vp)

void *	operator new[] (size_t sz)

void *	operator new[] (size_t sz, void *vp)

TObject &	operator= (const TObject &rhs)

virtual void	Paint (Option_t *option="")

virtual void	Pop ()

virtual void	Print (Option_t *option="") const

virtual Int_t	Read (const char *name)

virtual void	RecursiveRemove (TObject *obj)

void	ResetBit (UInt_t f)

virtual void	SaveAs (const char filename="", Option_t option="") const

virtual void	SavePrimitive (std::ostream &out, Option_t *option="")

void	SetBit (UInt_t f)

void	SetBit (UInt_t f, Bool_t set)

virtual void	SetDrawOption (Option_t *option="")

virtual void	SetUniqueID (UInt_t uid)

virtual void	SysError (const char method, const char msgfmt,...) const

R__ALWAYS_INLINE Bool_t	TestBit (UInt_t f) const

Int_t	TestBits (UInt_t f) const

virtual void	UseCurrentStyle ()

virtual void	Warning (const char method, const char msgfmt,...) const

virtual Int_t	Write (const char *name=0, Int_t option=0, Int_t bufsize=0)

virtual Int_t	Write (const char *name=0, Int_t option=0, Int_t bufsize=0) const

Static Public Member Functions
static KVDataTransfer *	NewTransfer (const Char_t source_rep, const Char_t target_rep)

Static Public Member Functions inherited from KVDataAnalyser
static Bool_t	AbortProcessingLoop ()

static KVDataAnalyser *	GetAnalyser (const Char_t *plugin)

static Bool_t	IsRunningBatchAnalysis ()

static void	RunAnalyser (const Char_t *plugin="")

static void	SetAbortProcessingLoop (Bool_t now=kTRUE)

Static Public Member Functions inherited from TObject
static Longptr_t	GetDtorOnly ()

static Bool_t	GetObjectStat ()

static void	SetDtorOnly (void *obj)

static void	SetObjectStat (Bool_t stat)

Protected Member Functions
virtual void	CheckTargetRepository ()

virtual void	ChooseAnalysisTask ()

virtual void	ExecuteCommand ()=0

virtual void	init ()
	Initialisation of data transfer. More...

virtual void	set_dataset_name (const Char_t *name)

virtual void	set_dataset_pointer (KVDataSet *ds)

virtual void	SetTransferExec (const Char_t *path)

virtual void	SubmitTask ()

virtual void	WriteTransferScript ()=0

Protected Member Functions inherited from KVDataSetAnalyser
void	_set_dataset_pointer (KVDataSet *ds)

Bool_t	CheckWhatToAnalyseAndHow ()

void	ChooseWhatToAnalyse ()
	TO IMPLEMENT ? More...

void	ChooseWhatToDo ()
	TO IMPLEMENT ? More...

Bool_t	NeedToChooseWhatToAnalyse () const

Bool_t	NeedToChooseWhatToDo () const

virtual void	PostRunReset ()
	Called at end of Run() after execution of analysis task. More...

virtual Bool_t	PreSubmitCheck ()

virtual KVNumberList	PrintAvailableRuns (KVString &datatype)

virtual void	set_up_analyser_for_task (KVDataAnalyser *the_analyser)

void	SetCurrentRun (KVDBRun *r)

Protected Member Functions inherited from KVDataAnalyser
virtual Bool_t	CheckIfUserClassIsValid (const KVString &alternative_base_class="")

void	CopyAnalysisResultsToLaunchDirectory ()

const Char_t *	GetACliCMode ()

TEnv *	GetBatchInfoFile () const

TObject *	GetInstanceOfUserClass (const KVString &alternative_base_class="")

void	ScanWorkingDirectory (TList **)

Protected Member Functions inherited from TObject
virtual void	DoError (int level, const char location, const char fmt, va_list va) const

void	MakeZombie ()

Protected Attributes
TString	fCmdFile
	name of command file given to transfer agent More...

Bool_t	fOK
	may be set to kFALSE by init(), in which case Run() will abort More...

KVDataRepository *	fSourceRep
	repository containing source files More...

KVDataRepository *	fTargetRep
	repository where files will be copied More...

TString	fTransferExec
	full path to transfer client executable More...

Protected Attributes inherited from KVDataAnalyser
TList *	fWorkDirInit
	list of files in working directory before task runs More...

Protected Attributes inherited from TObject
	kOnlyPrepStep

Additional Inherited Members
Public Types inherited from KVDataAnalyser
enum	EProofMode { None , Lite , Proof }

Public Types inherited from TObject
enum	EDeprecatedStatusBits

enum	EStatusBits

Public Attributes inherited from TObject
	kBitMask

	kCanDelete

	kCannotPick

	kHasUUID

	kInconsistent

	kInvalidObject

	kIsOnHeap

	kIsReferenced

	kMustCleanup

	kNoContextMenu

	kNotDeleted

	kObjInCanvas

	kOverwrite

	kSingleKey

	kWriteDelete

	kZombie

#include <KVDataTransfer.h>

Inheritance diagram for KVDataTransfer:

[legend]

Constructor & Destructor Documentation

◆ KVDataTransfer()

KVDataTransfer::KVDataTransfer ( )

Default constructor. Use KVDataTransfer::NewTransfer to create a new data transfer object with the correct properties to transfer files between two repositories.

Definition at line 27 of file KVDataTransfer.cpp.

◆ ~ KVDataTransfer()

virtual KVDataTransfer::~ KVDataTransfer ( )

virtual

Member Function Documentation

◆ CheckTargetRepository()

void KVDataTransfer::CheckTargetRepository ( )

protectedvirtual

Make sure that the target repository has the necessary dataset/subdirectory to receive the transferred files. If not, we create the new dataset/subdir.

Definition at line 269 of file KVDataTransfer.cpp.

◆ ChooseAnalysisTask()

virtual void KVDataTransfer::ChooseAnalysisTask ( )

inlineprotectedvirtual

Print list of all possible data analysis tasks for the chosen data set and invite the user to choose one of them. If only one task available, it is automatically selected not possible in batch mode

Reimplemented from KVDataSetAnalyser.

Definition at line 61 of file KVDataTransfer.h.

◆ ExecuteCommand()

virtual void KVDataTransfer::ExecuteCommand ( )

protectedpure virtual

Implemented in KVDataTransferXRD, and KVDataTransferDMS.

◆ init()

void KVDataTransfer::init ( void )

protectedvirtual

Initialisation of data transfer.

Definition at line 298 of file KVDataTransfer.cpp.

◆ NewTransfer()

KVDataTransfer * KVDataTransfer::NewTransfer	(	const Char_t *	source_rep,
		const Char_t *	target_rep
	)

static

Creates a new data transfer object to transfer files between the two named data repositories.

The type of the created object depends on the values of the environment variables (defined in .kvrootrc file):

source_rep.DataRepository.FileTransfer.type

target_rep.DataRepository.FileTransfer.type

If either of these = bbftp, we create a KVDataTransferBBFTP object.

If source_rep.DataRepository.FileTransfer.type = xrd, we create a KVDataTransferXRD object.

By default (no type given), we use a KVDataTransferSFTP object. In fact, the type of object created is defined in .kvrootrc by the following plugins:

Plugins for data transfer between repositories

Used by KVDataTransfer::NewTransfer

Plugin.KVDataTransfer: sftp KVDataTransferSFTP KVMultiDet "KVDataTransferSFTP()" +Plugin.KVDataTransfer: bbftp KVDataTransferBBFTP KVMultiDet "KVDataTransferBBFTP()" +Plugin.KVDataTransfer: xrd KVDataTransferXRD KVMultiDet "KVDataTransferXRD()"

Definition at line 72 of file KVDataTransfer.cpp.

◆ Run()

void KVDataTransfer::Run ( )

virtual

Perform file transfer The user selects runs which are available in the source repository in the same way as when performing data analysis Note that this will make the 'source' repository the current active repository (i.e. gDataRepository and gDataSetManager will correspond to source repository after execution of this command).

Reimplemented from KVDataAnalyser.

Definition at line 161 of file KVDataTransfer.cpp.

◆ set_dataset_name()

void KVDataTransfer::set_dataset_name ( const Char_t * name )

protectedvirtual

Set dataset to be analysed. If 'name' is not the name of a valid and available dataset in the 'source' data repository an error message is printed.

Reimplemented from KVDataSetAnalyser.

Definition at line 311 of file KVDataTransfer.cpp.

◆ set_dataset_pointer()

void KVDataTransfer::set_dataset_pointer ( KVDataSet * ds )

protectedvirtual

Set dataset to be used for transfer. If the chosen dataset is not available, an error message is printed Only datasets which are available in the source repository can be transferred If the pointer actually corresponds to a dataset in the target repository, we replace it with a pointer to the dataset with the same name in the source repository. allow user to reset dataset pointer to 0

Reimplemented from KVDataSetAnalyser.

Definition at line 339 of file KVDataTransfer.cpp.

◆ SetTransferExec()

virtual void KVDataTransfer::SetTransferExec ( const Char_t * path )

inlineprotectedvirtual

Definition at line 57 of file KVDataTransfer.h.

◆ SubmitTask()

virtual void KVDataTransfer::SubmitTask ( )

inlineprotectedvirtual

In interactive mode, the data analysis task is performed by instanciating and initialising the KVDataAnalyser child class specified by the task, and then calling its Run() method. In batch mode, the job is submitted to the chosen batch system.

Reimplemented from KVDataAnalyser.

Definition at line 64 of file KVDataTransfer.h.

◆ TransferRuns()

void KVDataTransfer::TransferRuns ( )

virtual

Based on information gathered from user (see Run()), perform the transfer of files. Any missing directories in the target repository are created beforehand. After transfer, the available runlist for the target repository is updated

Definition at line 217 of file KVDataTransfer.cpp.

◆ WriteTransferScript()

virtual void KVDataTransfer::WriteTransferScript ( )

protectedpure virtual

Implemented in KVDataTransferXRD, and KVDataTransferDMS.

Member Data Documentation

◆ fCmdFile

TString KVDataTransfer::fCmdFile

protected

name of command file given to transfer agent

Definition at line 54 of file KVDataTransfer.h.

◆ fOK

Bool_t KVDataTransfer::fOK

protected

may be set to kFALSE by init(), in which case Run() will abort

Definition at line 74 of file KVDataTransfer.h.

◆ fSourceRep

KVDataRepository* KVDataTransfer::fSourceRep

protected

repository containing source files

Definition at line 52 of file KVDataTransfer.h.

◆ fTargetRep

KVDataRepository* KVDataTransfer::fTargetRep

protected

repository where files will be copied

Definition at line 53 of file KVDataTransfer.h.

◆ fTransferExec

TString KVDataTransfer::fTransferExec

protected

full path to transfer client executable

Definition at line 55 of file KVDataTransfer.h.

The documentation for this class was generated from the following files:

kaliveda.doxygen/KVMultiDet/analysis/KVDataTransfer.h
kaliveda.doxygen/KVMultiDet/analysis/KVDataTransfer.cpp

Public Member Functions

Static Public Member Functions

Protected Member Functions

Protected Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ KVDataTransfer()

◆ ~ KVDataTransfer()

Member Function Documentation

◆ CheckTargetRepository()

◆ ChooseAnalysisTask()

◆ ExecuteCommand()

◆ init()

◆ NewTransfer()

Plugins for data transfer between repositories

Used by KVDataTransfer::NewTransfer

◆ Run()

◆ set_dataset_name()

◆ set_dataset_pointer()

◆ SetTransferExec()

◆ SubmitTask()

◆ TransferRuns()

◆ WriteTransferScript()

Member Data Documentation

◆ fCmdFile

◆ fOK

◆ fSourceRep

◆ fTargetRep

◆ fTransferExec