PYME.ParallelTasks.HTTPTaskPusher module

Created on Thu Nov 26 23:59:45 2015

@author: david

class PYME.ParallelTasks.HTTPTaskPusher.HTTPRecipePusher(recipe=None, recipeURI=None)

Bases: object

Methods

fileTasksForInputs(**kwargs)
fileTasksForInputs(**kwargs)
class PYME.ParallelTasks.HTTPTaskPusher.HTTPTaskPusher(dataSourceID, metadata, resultsFilename, queueName=None, startAt=10, dataSourceModule=None, serverfilter='')

Bases: object

Create a pusher and push tasks for each frame in a series. For use with the new cluster distribution architecture

Parameters:

dataSourceID : str

The URI of the data source - e.g. PYME-CLUSTER://serverfilter/path/to/data

metadata : PYME.IO.MetaDataHandler object

The acquisition and analysis metadata

resultsFilename : str

The cluster relative path to the results file. e.g. “<username>/analysis/<date>/seriesname.h5r”

queueName : str

a name to give the queue. The results filename is used if no name is given.

startAt : int

which frame to start at. TODO - read from metadata instead of taking as a parameter.

dataSourceModule : str [optional]

The name of the module to use for reading the raw data. If not given, it will be inferred from the dataSourceID

serverfilter : str

A cluster filter, for use when multiple PYME clusters are visible on the same network segment.

Methods

cleanup()
fileTasksForFrames()
cleanup()
fileTasksForFrames()
PYME.ParallelTasks.HTTPTaskPusher.launch_localize(analysisMDH, seriesName)

Pushes an analysis task for a given series to the distributor

Parameters:

analysisMDH : dictionary-like

MetaDataHandler describing the analysis tasks to launch

seriesName : str

cluster path, e.g. pyme-cluster:///example_folder/series

Returns

——-

PYME.ParallelTasks.HTTPTaskPusher.verify_cluster_results_filename(resultsFilename)

Checks whether a results file already exists on the cluster, and returns an available version of the results filename. Should be called before writing a new results file.

Parameters:

resultsFilename : str

cluster path, e.g. pyme-cluster:///example_folder/name.h5r

Returns

——-

resultsFilename : str

cluster path which may have _# appended to it if the input resultsFileName is already in use, e.g. pyme-cluster:///example_folder/name_1.h5r