PYME.IO.unifiedIO module

PYME.IO.unifiedIO.dirname(url)
PYME.IO.unifiedIO.local_or_temp_filename(*args, **kwds)

Gets a local filename for a given url.

The purpose is to let us load files from the cluster using IO libraries (e.g. pytables, tifffile) which need a filename, not a file handle.

Does the following (in order).

  • checks to see if the url refers to a local file, if so return the filename
  • if the url points to a cluster file, check to see if it happens to be stored locally. If so, return the local path
  • download file to a temporary file and return the filename of that temporary file.

NB: This should be used as a context manager (ie in a with statement) so that the temporary file gets cleaned up properly

Parameters:url : basestring
Returns:filename : basestring

Notes

TODO - there is potential for optimization by using memmaps rather than files on disk.

PYME.IO.unifiedIO.openFile(filename, mode='rb')
PYME.IO.unifiedIO.read(filename)
PYME.IO.unifiedIO.split_cluster_url(url)