PYME.IO.unifiedIO module¶
- PYME.IO.unifiedIO.assert_name_ok(name)¶
Raise if name contains reserved/invalid characters for use with, e.g. clusterIO.
- Parameters
- name: str or bytes
The filename or url to query
- PYME.IO.unifiedIO.assert_path_ok(name)¶
- PYME.IO.unifiedIO.assert_uri_ok(name)¶
Raise if name contains reserved/invalid characters for use with, e.g. clusterIO.
- Parameters
- name: str or bytes
The filename or url to query
- PYME.IO.unifiedIO.assert_uri_path_ok(name)¶
combination of assert_path_ok and assert_uri_ok - runs assert_uri_ok if we have a URI, assert_name_ok if we have a name
- PYME.IO.unifiedIO.check_name(name, win=False)¶
Check if filename / url is OK to use with, e.g. clusterIO.
- Parameters
- name: str
The filename or url to query
- win: bool
is name a windows path
- Returns
- ok: bool
True if filename/url is fully compatible
- PYME.IO.unifiedIO.check_path(name)¶
- PYME.IO.unifiedIO.check_uri(name)¶
- PYME.IO.unifiedIO.dirname(url)¶
- PYME.IO.unifiedIO.fix_name(name)¶
Cleans filename / url for use with, e.g. clusterIO, by replacing spaces with underscores and removing all percent-encoded characters other than ‘:’ and ‘/’.
- Parameters
- name: str
The filename or url to query
- Returns
- fixed_name: str
The cleaned file name
- PYME.IO.unifiedIO.is_cluster_uri(url)¶
Checks whether the supplied uri/filename is a pyme-cluster uri
- Parameters
- url
- Returns
- bool
- PYME.IO.unifiedIO.local_or_temp_filename(url)¶
Gets a local filename for a given url.
The purpose is to let us load files from the cluster using IO libraries (e.g. pytables, tifffile) which need a filename, not a file handle.
Does the following (in order).
checks to see if the url refers to a local file, if so return the filename
if the url points to a cluster file, check to see if it happens to be stored locally. If so, return the local path
download file to a temporary file and return the filename of that temporary file.
NB: This should be used as a context manager (ie in a with statement) so that the temporary file gets cleaned up properly
- Parameters
- urlbasestring
- Returns
- filenamebasestring
Notes
TODO - there is potential for optimization by using memmaps rather than files on disk.
- PYME.IO.unifiedIO.openFile(filename, mode='rb')¶
- PYME.IO.unifiedIO.read(filename)¶
Read a file from disk or the cluster.
NOTE: filename is expected to be sanitized / trusted, this should not be called with user data from a web endpoint.
- Parameters
- filename
- Returns
- PYME.IO.unifiedIO.safe_read(filename)¶
Read in a web-endpoint safe manner (i.e. disallow local file access)
Currently just a wrapper around clusterIO.get_file, but also as a forwards-compatible stub if we expand unifiedIO to support more general HTTP or, e.g. OMERO URIs.
- PYME.IO.unifiedIO.split_cluster_url(url)¶
- PYME.IO.unifiedIO.verbose_fix_name(name)¶
Wrapper for fix_name which sacrifices performance in order to complain. Parameters ———- name: str
The filename or url to query
- Returns
- fixed_name: str
The cleaned file name
- PYME.IO.unifiedIO.write(filename, data)¶