Analysing Localisation Microscopy data¶
Starting distributed analysis infrastructure¶
To improve performance, PYME distributes localization analysis over multiple worker processes, with a server process used for communication. These can either all be on the same machine, or be distributed across a network/cluster. The server and worker processes need to be running before starting the analysis. This is achieved with the
command. On OSX or linux, enter this in a terminal window. On Windows either enter it in an “Anaconda prompt” (found
under the “Anaconda” group on the start menu, or navigate to the directory you installed Anaconda to (most likely
C:\Users\<username>\Miniconda2) and make a shortcut to
somewhere convenient for you.
launchWorkers script starts the server and the same number
of workers as there are cores on the current machine . The server and worker processes should then find
each other automatically  and communicate using a package called Pyro. On Windows you may be prompted to allow
python through the firewall, which you should accept.
We are currently in the process of migrating all our analysis to use the new, high-throughput infrastructure which is
python3 compatible, faster, and more robust. This will mean that you need to run
PYMEClusterOfOne (or similar),
launchWorkers but will leave the rest of the process largely unchanged.
Once the server and worker processes are running, the data should be opened using dh5view.
Data acquired using PYMEAcquire¶
Data spooled to a
.h5 file can be opened by running
dh5view filename.h5, by double clicking
Anaconda\Scripts directory (or a shortcut) and using the file open dialog, or by ascociating
files. For data saved directly to a queue, the easiest way is probably to click the Analyse button on the
Spooling panel in PYMEAcquire. Many protocols will do this
automatically after a the intial pre-bleaching phase has been performed.
Data not acquired using PYMEAcquire¶
For data not originating from PYMEAcquire the process is a little more complex as
dh5view will not detect that it
should launch in “Localization Mode” and some metadata will probably be missing.
(see Analysing data not generated by PYMEAcquire for details). In short, either use
dh5view -m LM filename or use the file open dialog,
complete any missing metadata entries and then choose
LMAnalysis from the
Special attention is also needed for analysing simultaeneous multi-colour data (see Using an image splitting device for multi-colour ratiometric imaging).
With the data loaded in dh5view, one should see something like:
The Analysis and Point Finding panes in the left hand panel control the analysis settings.
PYME offers a large number of different fit types, and is easily extensible to support more. The most useful ones are given below. If in doubt, you usually want LatGaussFitFR.
|Threshold||The threshold for event detection. This is not a strict threshold, but rather a scaling factor applied to a local threhold computed from the estimated pixel SNR. Will generally not need to be altered (except for some of the interpolation based fits)|
|Type||The type of fit to perform|
|Interp||The type of interpolation to perform (only methods with Interp in their name)|
|Start at||The frame # should to start our analysis at|
|Background||What range of frames should be used for background estimation. Uses python slice notation.|
|Subtract background||Should we subtract the background before fitting (rather than just for event detection)|
|Debounce r||The radius (in pixels) within which 2 events cannot be reliably distinguished|
|Shifts||The shift field to use for chromatic shift correction (only methods with Splitter in their name.|
|PSF||The PSF measurement to use (only Interp methods)|
|Track Fiducials||Whether to estimate drift using fiduciaries|
|Variance Map||An image containing a pixel by pixel map of camera variance (read noise) for sCMOS correction|
|Dark Map||An image containing a pixel by pixel map of dark values for sCMOS correction|
|Flatfield Map:||An image used for flatfielding (gain correction) with sCMOS cameras|
Some fit modules will also display custom settings.
Starting the fitting¶
Testing the object detection¶
Whilst the threhold factor is fairly robust, it is generally worth testing the
detection by clicking the Test button. This performs the object finding step
on a selection of frames spaced throughout the sequence. If this fails one should
Camera.ADOffset setting in the metadata (accessible through the Metadata tab)
to see if this is reasonable before attempting to tweak the detection threshold. (The ADOffset is estimated by
taking a number of dark frames before the acquisition starts, and can be fooled if
the room lights are on and/or the laser shutters are misbehaving). Metadata parameters
can be edited by right clicking the appropriate field in the Metadata tab.
Launching the analysis tasks¶
Once satisfied with the event detection, the analysis proper can be started by
clicking the Go button. The results will automatically be saved as
.h5r files, either under the
PYMEDATADIR directory (if the environment variable was set earlier), or in a directory
PYMEData in the users home directory (
c:\\Users\\<username>\\ under windows).
.h5r files can be opened in VisGUI.
A crude form of distributing the analysis over multiple computers (a small ad-hoc cluster) can be achieved by:
- Make sure that the same version of PYME is installed on all machines
launchWorkerson each machine you want to use.
A much better approach, however, is to use the ‘new-style’ distributed analysis which is both significantly faster and more robust. TODO - write docs.
|||If you’re running the analysis on the data acquisition computer, and have a decent number of
cores available (e.g. 8), better performance can be achieved by reserving a core for
each of the Acqusition and Server processes (ie limiting the number of workers to 6 in our
8 core case). This can be done by explicitly specifying the number of workers to launch as
an argument eg: |
|||Using the zeroconf/MDNS protocol.|