PYME.cluster.HTTPDataServer module

This is a very simple HTTP server which allows data to be saved to the server using PUT

Warning

Security warning

The code as it stands lets any client write arbitrary data to the server. The only concessions we make to security are:

1. Paths are vetted to make sure that they reside under our server root (i.e. no escape through .. etc)

2. We enforce a write-once scheme (if a file exists it can’t be overridden)

This protects against people accidentally or maliciously altering data, or discovering server settings, but leaves us open to denial of service type attacks in which a malicious client could fill up our storage.

THE CODE AS IT STANDS SHOULD ONLY BE USED ON A TRUSTED NETWORK

TODO: Add some form of authentication. Needs to be low overhead (e.g. digest based)

class PYME.cluster.HTTPDataServer.PYMEHTTPRequestHandler(*args, **kwargs)

Bases: SimpleHTTPRequestHandler

bandwidthTesting = False

do_GET()

Serve a GET request.

do_PUT()

get_glob()

get_h5_part(path)

get_status()

get_tabular_part(path)

Parameters

path: str

OS-translated path to an hdf or h5r file on the dataserver computer. Append the part of the file to read after the file extension, e.g. .h5r/Events. Return format (for arrays) can additionally be specified, as can slices using the following syntax: test.h5r/FitResults.json?from=0&to=100. Supported array formats include json and npy.

Returns

f: BytesIO

Requested part of the file encoded as bytes

list_directory(path)

Helper to produce a directory listing (absent index.html).

Return value is either a file object, or None (indicating an error). In either case, the headers are sent, making the interface the same as for send_head().

list_h5(path)

log_error(format, *args)

Log an error.

This is called when a request cannot be fulfilled. By default it passes the message on to log_message().

Arguments are the same as for log_message().

XXX This should go to the separate error log.

log_message(format, *args)

Log an arbitrary message.

This is used by all other logging functions. Override it if you have specific logging wishes.

The first argument, FORMAT, is a format string for the message to be logged. If the format string contains any % escapes requiring parameters, they should be specified as subsequent arguments (it’s just like printf!).

The client ip address and current date/time are prefixed to every message.

log_request(code='-', size='-')

Log an accepted request.

This is called by send_response().

logrequests = False

protocol_version = 'HTTP/1.0'

send_error(code, message=None)

Send and log an error reply.

Arguments are the error code, and a detailed message. The detailed message defaults to the short entry matching the response code.

This sends an error response (so it must be called before any output has been generated), logs the error, and finally sends a piece of HTML explaining the error to the user.

send_head()

Common code for GET and HEAD commands.

This sends the response code and MIME headers.

Return value is either a file object (which has to be copied to the outputfile by the caller unless the command was HEAD, and must be closed by the caller under all circumstances), or None, in which case the caller has nothing further to do.

timeout = None

timeoutTesting = 0

translate_path(path)

Translate a /-separated PATH to the local filename syntax.

Components that mean special things to the local file system (e.g. drive or directory names) are ignored. (XXX They should probably be diagnosed.)

class PYME.cluster.HTTPDataServer.ThreadedHTTPServer(server_address, RequestHandlerClass, bind_and_activate=True)

Bases: ThreadingMixIn, HTTPServer

Handle requests in a separate thread.

Constructor. May be extended, do not override.

PYME.cluster.HTTPDataServer.getTextFileLock(filename)

PYME.cluster.HTTPDataServer.main(protocol='HTTP/1.0')

PYME.cluster.HTTPDataServer.makedirs_safe(dir)

A safe wrapper for makedirs, which won’t throw an error if the directory already exists. This replicates the functionality of the exists_ok flag in the python 3 version of os.makedirs, but should work with both pytion 2 and python 3.

Parameters

dirstr, directory to be created

Returns

class PYME.cluster.HTTPDataServer.statusPoller(*args, **kwargs)

Bases: Thread

This constructor should always be called with keyword arguments. Arguments are:

group should be None; reserved for future extension when a ThreadGroup class is implemented.

target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.

name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.

args is the argument tuple for the target invocation. Defaults to ().

kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.

If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

stop()

PYME.cluster.HTTPDataServer.updateStatus()