Manage Memory Issues in iSMS

About memory handling in iSMS

iSMS is intentionally designed to process raw image data interactively. While this makes data analysis more fun it also imposes a requirement to the amount of RAM on your computer.

How iSMS stores raw image data

iSMS stores all loaded image arrays as unsigned 16-bit integers (uint16). This data type forces a constraint on the range of pixel values that the raw data can have, namely integers in the range 0-65535. This is also the typical raw data range for cameras, however, this constraint means that all negative values in the raw data, and after camera background subtraction, is set to 0.

For uint16 data types, each pixel takes up 2 bytes of RAM. For large movies, the raw image data thus takes up a lot of memory. For example, a single 512x512 movie with 2000 frames takes up 512*512*2000*2 = 1048576000 bytes = 1.049 Gb of RAM.

In addition to the full raw movie, iSMS cuts and stores in RAM a raw movie of each emission channel ('ROI movies'). This is in order to process calculations faster and to save memory since the ROI movies require less RAM than the full raw movie.

When using drift-correction, iSMS stores the drift-corrected movie in addition to the uncorrected movie ('Drift corrected ROI movies'). This is in order to be able restore the original data if an attempt at drift compensation fails for some reason.

Solution to the memory problem

In iSMS, raw image data can be removed from RAM without loss of any processed data, such as FRET-pairs. This effectively solves memory issues since the memory required by the processed data is negligible compared to the raw image stack.

The raw image data can, and should, be removed from RAM once all intensity traces have been calculated. On a machine with 8 Gb RAM you are typically able to analyse 4-5 raw movies of size 512x512 x2000 frames at a time - twice as much on a 16 Gb machine.

The workflow in iSMS is as follows:

  1. Load and process a number of movies (4-10 depending on RAM and movie size).
  2. Remove the raw image data from memory, as described below.
  3. Repeat step 1 and 2 on a new set of movies.
  4. When intensity traces need to be recalculated, e.g. when settings have been changed, reload the raw image data into RAM, as described below.

When saving a session, iSMS only saves the processed data and not the raw image stack. Instead, the raw image data can be reloaded from file after opening the session. In this way, raw image data is loaded, removed, and reloaded into memory over and over during the data analysis.

Monitoring memory in iSMS

Memory is monitored in the lower right corner or, for a more detailed report, by navigating to 'Performance->Memory->Memory profile'. Monitoring memory is not supported for mac-platforms.

Clearing raw image data from RAM

To clear raw image data from RAM, press the '-' button left of the files list box or navigate to 'Performance->Memory->Clear raw data from RAM'.
In the dialog select files to clear in the list box. The bracket suffix of each file denotes which type of raw image data the file currently holds in RAM. Select which type of raw data to clear in the lower check boxes - the data types are described above. The 'raw movies' are needed to create the 'ROI movies'. The 'ROI movies' are needed to calculate the intensity traces. Thus, clear the raw data when emission channels have been aligned and clear the ROI data when FRET-pairs have been found and the time-traces have been calculated.
When raw image data has been cleared, all FRET-pairs and traces are still available. Files missing the raw data are indicated in the files list box as a '(-raw)' suffix to the filename. Without the raw data, moving the ROIs in the raw image to the left has no effect. Whenever you attempt an action that requires the raw image data, a dialog will inform you about the required action.

Reloading raw image data into RAM

To reload the raw image data back into iSMS press the 'o' button left of the files list box or navigate to 'Performance->Memory->Reload raw data into RAM'.
Select which files to reload in the dialog. The data will be loaded from the original file path by default. If the file path is broken, a file selection dialog will be opened to specify the path to the raw file. Check the check box in the lower left corner to force a file path selection dialog.
When the raw image data is reloaded the '(-raw)' suffix is removed from the file.
Once the raw data has been reloaded, you can once again move the emission channel ROIs and calculate new intensity traces.