PYME.cluster.HTTPTaskPusher module¶
Created on Thu Nov 26 23:59:45 2015
@author: david
- class PYME.cluster.HTTPTaskPusher.HTTPTaskPusher(dataSourceID, metadata, resultsFilename, queueName=None, startAt=10, dataSourceModule=None, serverfilter='')¶
Bases:
objectCreate a pusher and push tasks for each frame in a series. For use with the new cluster distribution architecture
- Parameters
- dataSourceIDstr
The URI of the data source - e.g. PYME-CLUSTER://serverfilter/path/to/data
- metadataPYME.IO.MetaDataHandler object
The acquisition and analysis metadata
- resultsFilenamestr
The cluster relative path to the results file. e.g. “<username>/analysis/<date>/seriesname.h5r”
- queueNamestr
a name to give the queue. The results filename is used if no name is given.
- startAtint
which frame to start at. TODO - read from metadata instead of taking as a parameter.
- dataSourceModulestr [optional]
The name of the module to use for reading the raw data. If not given, it will be inferred from the dataSourceID
- serverfilterstr
A cluster filter, for use when multiple PYME clusters are visible on the same network segment.
- cleanup()¶
- fileTasksForFrames()¶
- class PYME.cluster.HTTPTaskPusher.RecipePusher(recipe=None, recipeURI=None)¶
Bases:
object- fileTasksForInputs(**kwargs)¶
- PYME.cluster.HTTPTaskPusher.launch_localize(analysisMDH, seriesName)¶
Pushes an analysis task for a given series to the distributor
- Parameters
- analysisMDHdictionary-like
MetaDataHandler describing the analysis tasks to launch
- seriesNamestr
cluster path, e.g. pyme-cluster:///example_folder/series
- Returns
- ——-
- PYME.cluster.HTTPTaskPusher.verify_cluster_results_filename(resultsFilename)¶
Checks whether a results file already exists on the cluster, and returns an available version of the results filename. Should be called before writing a new results file.
- Parameters
- resultsFilenamestr
cluster path, e.g. pyme-cluster:///example_folder/name.h5r
- Returns
- ——-
- resultsFilenamestr
cluster path which may have _# appended to it if the input resultsFileName is already in use, e.g. pyme-cluster:///example_folder/name_1.h5r