Package qb :: Package backend :: Module pythonBackEnd :: Class PythonQubeBackEnd

[frames] | no frames]

Class PythonQubeBackEnd

source code

The base class for PipelineFX's python back-end modules.

Instance Methods

[hide private]

__init__(self, job)
Function to initialize class members.

source code

printClassInfo(self)

source code

jobSetup(self)
Perform any steps necessary to initialize the working enviroment prior to beginning any agendaItem-specific steps.

source code

executeWork(self)
Request an agendaItem (work) from the supervisor and do any steps necessary to perform the work.

source code

jobTeardown(self)
Perform any steps necessary to clean up the working enviroment prior to shutting down the job instance.

source code

hasJobPhaseCmds(self, phase)
Test for the existence of job setup/teardown cmds in the job package

source code

tuple

getLogData(self, logpath, iostream, offset=0)
seek fwd in a file, read to EOF, then record the file position, which will be the offset the next time we enter this function

source code

tuple

logHandler(self, work, qbTokens=None)
Parse a job log for regex matches (progress, errors, outputPaths, etc...), update the job or work packages with the logmatches, set the job status accordingly, and report back to the supervisor.

source code

updateResultPackage(self, work, resultDict)
Update the Qube qb.Work object's resultpackage dictionary.

source code

parseLogData(self, data, *args)
Find any matches to the job's regular expressions.

source code

bool

validateOutputFileSize(self, work)
Check the work/jobInstance's resultpackage for any files in the outputPaths, and ensure that their file size exceeds the job's minimum file size

source code

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Class Variables

[hide private]

QB_WAITING_TIMEOUT = 5

int LOGREAD_TIME_THRESHOLD = 30
how often to parse the job log data for regex matches

Properties

[hide private]

Inherited from object: __class__

Method Details

[hide private]

init(self, job)
(Constructor)

source code

Function to initialize class members.

Parameters:

job (qb.Job) - The qube job object passed to the worker, this represents a job instance.

Overrides: object.__init__

executeWork(self)

source code

Request an agendaItem (work) from the supervisor and do any steps necessary to perform the work.

Raises:

NotImplementedError - Raised when this method is not overridden in a derived class.

getLogData(self, logpath, iostream, offset=0)

source code

seek fwd in a file, read to EOF, then record the file position, which will be the offset the next time we enter this function

Parameters:

logpath (str) - full path to a job log file to scan
iostream (str) - name of the iostream, usually "stderr" or "stdout"
offset (int) - position in file to begin to read data from
fPos (int)
data (str)

Returns: tuple

a tuple containing the log data and the end position of the file

logHandler(self, work, qbTokens=None)

source code

Parse a job log for regex matches (progress, errors, outputPaths, etc...), update the job or work packages with the logmatches, set the job status accordingly, and report back to the supervisor.

Parameters:

qbTokens (dict) - a dictionary containing the various QB_FRAME* cmdrange tokens evaluated in the context of the current running work item to aid in calculating the in-chunk progress.

Returns: tuple

Return booleans for each error condition checked; error regex found a match, and whether a size check failed

updateResultPackage(self, work, resultDict)

source code

Update the Qube qb.Work object's resultpackage dictionary.

The resultpackage is a dictionary that the Qube supervisor retrieves from the worker after processing work. It can be used to update the job object with data that is determined at execution time.

Any key/value pairs can be passed into the resultpackage as a dictionary; it's up to the developer to pass key/values that are meaningful to Qube.

The most common use is to set the 'outputpaths' value for an agenda item. This value is used by the Qube GUI to display the image in the "Output" tab of the UI.

Parameters:

work (qb.Work) - The Qube qb.Work object whose resultpackage dictionary is to be updated
resultDict - a dictionary containing key/value pairs to be inserted into the work's resultpackage dictionary.

parseLogData(self, data, *args)

source code

Find any matches to the job's regular expressions.

Parameters:

data (string) - a portion of a job instance's job log

validateOutputFileSize(self, work)

source code

Check the work/jobInstance's resultpackage for any files in the outputPaths, and ensure that their file size exceeds the job's minimum file size

Parameters:

work (qb.Work or qb.Subjob) - The work/job object whose resultpackage dictionary is to be scanned

Returns: bool

return False if the contents of the outputPaths contain a file path whose size does not exceed the job's validate_fileMinSize, otherwise return True for all other cases.

Class PythonQubeBackEnd

__init__(self, job) (Constructor)

executeWork(self)

getLogData(self, logpath, iostream, offset=0)

logHandler(self, work, qbTokens=None)

updateResultPackage(self, work, resultDict)

parseLogData(self, data, *args)

validateOutputFileSize(self, work)

init(self, job)
(Constructor)