CIAtah

Note: in general use the individual pages in the navigation pane on the left, as those will be most up-to-date.

Software package for calcium imaging analysis of one- and two-photon imaging datasets.

calciumImagingAnalysis_logo

  • Includes a GUI to allow users to do large-scale batch analysis, accessed via the repository's CIAtah class.
  • The underlying functions can also be used to create GUI-less, command line-ready analysis pipelines. Functions located in ciapkg and +ciapkg sub-folders.
  • Includes all major calcium imaging analysis steps: pre-processing (motion correction, spatiotemporal downsampling, spatial filtering, relative fluorescence calculation, etc.), support for multiple cell-extraction methods, automated cell classification (coming soon!), cross-session cell alignment, and more.
  • Has several example calcium imaging datasets that it will automatically download to help users test out the package.
  • Includes code for determining animal position (e.g. in open-field assay).
  • Supports Neurodata Without Borders data standard (see calcium imaging tutorial) for reading/writing cell-extraction (e.g. outputs of PCA-ICA, CELLMax, CNMF, CNMF-E, etc.). Supports reading NWB movie files (write support coming soon).
  • Requires MATLAB.

Contact: Biafra Ahanonu, PhD (bahanonu [at] alum [dot] mit [dot] edu).

Made in USA.
USA

Contents

Detailed README

Quick start guide

Below are steps needed to quickly get started using the CIAtah software package in MATLAB.

  • Clone the CIAtah repository (using GitHub desktop or command line) or download the repository zip and unzip.
  • Point the MATLAB path to the CIAtah root folder (NOT @ciatah sub-folder in the repository).
  • Run the below MATLAB commands.
  • Afterwards, likely want to run modelAddNewFolders module first in order to add folders containing imaging data to the current class object.
  • [Optional] Users on Windows systems should download Everything (https://www.voidtools.com/). It is a very useful and extremely fast search engine for files and folders on a computer that can allow users to quickly get lists of folders then need to analyze in CIAtah.
  • [Optional] Users who want to run analysis via the command line can run edit ciapkg.demo.cmdLinePipeline and run each segment of code there to see what commands are needed to perform each step. It assumes you have already run example_downloadTestData.
% Run these commands in MATLAB to get started.

% Loads the class into an object for use in this session
obj = ciatah;

% Runs routines to check dependencies and help user get setup.
obj.setup;

% Open the class menu (always type `obj` then enter load the class/modules menu)
obj % then hit enter, no semicolon!

CIAtah main GUI notes

  • All main decisions for choosing a method/procedure to run, cell-extraction algorithm, and which folders to analyze are in a single window.
  • The GUI will real-time update the selected folders based on the selections in the subject, assay, and folder filter areas.
  • Sections not relevant for a specific method are grayed out.
  • Tab to cycle through selection areas. Green background is the currently selected area, dark gray background is area that had previously been selected but is not the active area, and white background is for areas that have not been selected yet.
  • Hover mouse over method names for tooltip that gives additional information.

For example, selecting middle two assays automatically changes selection in Loaded folders section.

image image

Certain sections become available when user selects the appropriate method (e.g. cell-extraction method available when selecting modelExtractSignalsFromMovie).

image

Additional quick start notes

  • See additional details in Processing calcium imaging data for running the full processing pipeline.
  • To force load all directories, including most external software packages (in _external_programs folder), type ciapkg.loadAllDirs; into MATLAB command line. This is most relevant when you need to access specific functions in an outside repository that are normally hidden until needed.
  • When issues are encountered, first check the *Common issues and fixes Wiki page to see if a solution is there. Else, submit a new issue or email Biafra (bahanonu [at] alum.mit.edu).
  • Notes:
    • There are two sets of test data that are downloaded:
      • Single session analysis: data\2014_04_01_p203_m19_check01_raw can be used to test the pipeline until the cross-session alignment step.
      • Batch analysis: data\batch contains three imaging sessions that should be processed and can then be used for the cross-session alignment step. Users should try these sessions to get used to batched analysis.
    • For Fiji dependency, when path to Miji.m (e.g. \Fiji.app\scripts folder) is requested, likely in \_external_programs\FIJI_FOLDER\Fiji.app\scripts where FIJI_FOLDER varies depending on OS, unless the user requested a custom path or on OSX (in which case, find Fiji the install directory).
      • If you run into Java heap space memory errors when Miji tries to load Fiji in MATLAB, make sure "java.opts" file is in MATLAB start-up folder or that CIAtah folder is the MATLAB start-up folder (instructions on changing).
    • CIAtah often uses regular expressions to find relevant movie and other files in folders to analyze.
      • For example, by default it looks for any movie files in a folder containing concat, e.g. concat_recording_20140401_180333.h5 (test data). If you have a file called rawData_2019_01_01_myInterestingExperiment.avi and all your raw data files start with rawData_ then change the regular expression to rawData_ when requested by the repository. See setMovieInfo module to change after adding new folders.
    • CIAtah generally assumes users have imaging data associated with one imaging session and animal in a given folder. Follow folder naming conventions in Data for best experience.
    • External software packages are downloaded into _external_programs folder and should be placed there if done manually.

Users can alternatively run setup as below. 

% Run these commands in MATLAB to get started.

% Loads all directories
loadBatchFxns;

% Loads the class into an object for use in this session
obj = ciatah;

% Download and load dependent software packages into "_external_programs" folder.
% Also download test data into "data" folder.
% Normally only need to one once after first downloading CIAtah package.
obj.loadDependencies;

% Add folders containing imaging data.
obj.modelAddNewFolders;

% [optional] Set the names CIAtah will look for in each folder
obj.setMovieInfo;

% Open class menu to pick module to run.
obj.runPipeline; % then hit enter!

Acknowledgments

Thanks to Jones G. Parker, PhD (https://parker-laboratory.com/) for providing extensive user feedback during development of the CIAtah software package.

Additional thanks to Drs. Jesse Marshall, Jérôme Lecoq, Tony H. Kim, Hakan Inan, Lacey Kitch, Maggie Larkin, Elizabeth Otto Hamel, Laurie Burns, and Claudia Schmuckermair for providing feedback, specific functions, or helping develop aspects of the code used in the CIAtah software package.

References

Please cite Corder, Ahanonu, et al. 2019 Science publication or the Ahanonu, 2018 Zenodo release if you used the software package or code from this repository to advance/help your research:

@article{corderahanonu2019amygdalar,
  title={An amygdalar neural ensemble that encodes the unpleasantness of pain},
  author={Corder, Gregory and Ahanonu, Biafra and Grewe, Benjamin F and Wang, Dong and Schnitzer, Mark J and Scherrer, Gr{\'e}gory},
  journal={Science},
  volume={363},
  number={6424},
  pages={276--281},
  year={2019},
  publisher={American Association for the Advancement of Science}
}

@misc{biafra_ahanonu_2018_2222295,
  author       = {Biafra Ahanonu},
  title        = {{ CIAtah : a software package for
                   analyzing one- and two-photon calcium imaging
                   datasets.}},
  month        = December,
  year         = 2018,
  doi          = {10.5281/zenodo.2222295},
  url          = {https://doi.org/10.5281/zenodo.2222295}
}

Questions?

Please email any additional questions not covered in the repository to bahanonu [at] alum.mit.edu or open an issue.

Detailed README

All the remaining sections encompass a more detailed README that provide a step-by-step instructions for running the entire analysis pipeline along with notes, tips, data organization, and other information.

Repository notes

  • Covers preprocessing of calcium imaging videos, cell and activity trace extraction (supports the following methods: PCA-ICA, CELLMax, EXTRACT, CNMF, CNMF-E, and ROI), manual and automated sorting of cell extraction outputs, cross-session alignment of cells, and more.
  • Supports PCA-ICA, CNMF, CNMF-E, and ROI cell extraction methods publicly along with CELLMax and EXTRACT for Schnitzer Lab collaborators. Additional methods can be integrated upon request.
  • Most extensively tested on Windows MATLAB 2018b and 2019a. Moderate testing on Windows MATLAB 2015b, 2017a, 2017b, and 2018b along with OSX (10.10.5) 2017b and 2018b. Individual functions and ciatah class should work on other MATLAB versions after 2015b, but submit an issue if errors occur. Newer MATLAB version preferred.
  • This repository consists of code used in and released with
    • G. Corder, __B. Ahanonu__, B. F. Grewe, D. Wang, M. J. Schnitzer, and G. Scherrer (2019). An amygdalar neural ensemble encoding the unpleasantness of painful experiences. Science, 363, 276-281. http://science.sciencemag.org/content/363/6424/276.
    • and similar code helped process imaging or behavioral data in:
      • J.G. Parker, J.D. Marshall, B. Ahanonu, Y.W. Wu, T.H. Kim, B.F. Grewe, Y. Zhang, J.Z. Li, J.B. Ding, M.D. Ehlers, and M.J. Schnitzer (2018). Diametric neural ensemble dynamics in parkinsonian and dyskinetic states. Nature, 557, 177–182. https://doi.org/10.1038/s41586-018-0090-6.
      • Y. Li, A. Mathis, B.F. Grewe, J.A. Osterhout, B. Ahanonu, M.J. Schnitzer, V.N. Murthy, and C. Dulac (2017). Neuronal representation of social information in the medial amygdala of awake behaving mice. Cell, 171(5), 1176-1190. https://doi.org/10.1016/j.cell.2017.10.015.
  • Code mostly developed while in Prof. Mark Schnitzer's lab at Stanford University. Credit to those who helped in Acknowledgments.
  • Please check the 'Wiki' for further instructions on specific processing/analysis steps and additional information of software used by this package.
  • When issues are encountered, first check the Common issues and fixes Wiki page to see if a solution is there. Else, submit a new issue.

image

Installation

Clone the CIAtah repository or download the repository zip and unzip.

  • Point the MATLAB path to the CIAtah folder.
  • Run loadBatchFxns.m before using functions in the directory. This adds all needed directories and sub-directories to the MATLAB path.
  • Type obj = ciatah; into MATLAB command window and follow instructions that appear after to add data and run analysis.
  • Run the ciatah class method loadDependencies or type obj.loadDependencies after initializing a ciatah object into the command window to download and add Fiji to path, download CNMF/CNMF-E repositories, download/setup CVX (for CNMF/CNMF-E), and download example data.

Note

  • Place CIAtah in a folder where MATLAB will have write permissions, as it also creates a private subdirectory to store some user information along with downloading required external software packages.
  • file_exchange folder contains File Exchange functions used by CIAtah.
  • In general, it is best to set the MATLAB startup directory to the CIAtah folder. This allows java.opts and startup.m to set the correct Java memory requirements and load the correct folders into the MATLAB path.
  • If CIAtah IS NOT the startup folder, place java.opts wherever the MATLAB startup folder is so the correct Java memory requirements are set (important for using ImageJ/Miji in MATLAB).
  • If it appears an old CIAtah repository is loaded after pulling a new version, run restoredefaultpath and check that old CIAtah folders are not in the MATLAB path.

Test data

To download test data, run loadDependencies module (e.g. obj.loadDependencies) and select Download test one-photon data. option to download example one-photon miniature microscope test data to use for testing CIAtah preprocessing, cell extraction, and cell classification code. The data will be located in the data folder within the repository root directory.

Else run example_downloadTestData.m if haven't started an instance of CIAtah.

Dependencies

By default external MATLAB-based software packages are stored in _external_programs.

MATLAB Toolbox dependencies

  • Primary toolboxes
    • distrib_computing_toolbox
    • image_toolbox
    • signal_toolbox
    • statistics_toolbox
  • Secondary toolboxes (not required for main pre-processing pipeline)
    • video_and_image_blockset
    • bioinformatics_toolbox
    • financial_toolbox
    • neural_network_toolbox

Parallel Computing Toolbox (PCT)

By default both CIAtah and PCT auto-start a parallel pool for functions that use parallelization (e.g. or calls to parfor). For some users this may not be desired, in that case go to MATLAB preferences and uncheck the below.

image

Or enter the following commands into the MATLAB command window:

parSet = parallel.Settings;
parSet.Pool.AutoCreate = false;

ImageJ

  • Run downloadMiji from downloads\downloadMiji.m or obj.loadDependencies (when class initialized) to download Fiji version appropriate to your platform.
  • Else download Fiji (preferably 2015 December 22 version): https://imagej.net/Fiji/Downloads.
  • Make sure have Miji in Fiji installation: http://bigwww.epfl.ch/sage/soft/mij/.
  • This is used as an alternative to the CIAtah playMovie.m function for viewing movies and is needed for some movie modification steps.

Saleae

CNMF and CNMF-E

Neurodata Without Borders

Neurodata Without Borders (NWB) file support requires the following GitHub repositories be present in the _external_programs folder. These are downloaded automatically when running obj.setup.

Repository organization

Below are a list of the top-level directories and what types of functions or files are within.

  • @ciatah - Contains ciatah class and associated methods for calcium imaging analysis.
  • externalprograms_ - External software packages (e.g. CNMF, CELLMax, and others) are stored here.
  • overloaded - Functions that overload core MATLAB functions to add functionality or fix display issues.
  • +ciapkg - Package containing CIAtah functions. All functions will eventually be moved here to create a clean namespace.
  • ciapkg
    • behavior - Processing of behavior files (e.g. accelerometer data, Saleae files, etc.).
    • classification - Classification of cells, e.g. manual classification of cell extraction outputs or cross-session grouping of cells.
    • data - Location of test data.
    • download - Functions that help download external code packages or data.
    • file_exchange - Contains any outside code from MATLAB's File Exchange that are dependencies in repository functions.
    • hdf5 - Functions concerned with HDF5 input/output.
    • image - Functions concerned with processing images (or [x y] matrices).
    • inscopix - Functions concerned with Inscopix-specific data processing (e.g. using the ISX MATLAB API).
    • io - Contains functions concerned with file or function input-output.
    • motion_correction - Functions concerned with motion correction.
    • movie_processing - Functions concerned with preprocessing calcium imaging videos, e.g. spatial filtering, downsampling, etc.
    • neighbor - Detection and display of neighboring cell information.
    • private - This directory contains various user settings, output pictures/data/logs from ciatah modules, and more. This directory is NOT included in the MATLAB path, hence is good for storing related scripts without interfering with CIAtah.
    • python - Python code, e.g. for processing Saleae data.
    • serial - Code for saving and processing serial port data, e.g. Arduino streaming data.
    • settings - Functions concerned with settings for other functions.
    • signal_extraction - Functions related to cell extraction, e.g. running PCA-ICA.
    • signal_processing - Functions to process cell activity traces.
    • tracking - ImageJ and MATLAB functions to track animal location in behavior movies.
    • unit_tests [optional] - Functions to validate specific repository functions.
    • video - Functions to manipulate or process videos, e.g. making movie montages or adding dropped frames.
    • view - Functions concerned with displaying data or information to the user, normally do not process data.

Data

The class generally operates on the principal that a single imaging session is contained within a single folder or directory. Thus, even if a single imaging session contains multiple trials (e.g. the imaging data is split across multiple movies) this is fine as the class will concatenate them during the preprocessing step.

The naming convention in general is below. Both TIF and AVI raw files are converted to HDF5 after processing since that format offers more flexibility during cell extraction and other steps.

Input and output files

  • Default raw imaging data filename: concat_.*.(h5|tif).
  • Default raw processed data filename: folderName_(processing steps).h5, where folderName is the directory name where the calcium imaging movies are located.
  • Main files output by CIAtah. Below, .* normally indicates the folder name prefixed to the filename.
    • .*_pcaicaAnalysis.mat: Where PCA-ICA outputs are stored.
    • .*_ICdecisions_.*.mat: Where decisions for cell (=1) and not cell (=0) are stored in a valid variable.
    • .*_regionModSelectUser.mat: A mask of the region (=1) to include in further analyses.
    • .*_turboreg_crop_dfof_1.h5: Processed movie, in this case motion corrected, cropped, and Δ_F/F_.
    • processing_info: a folder containing preprocessing information.

NWB Support

CIAtah supports NWB format and by default will output cell-extraction analysis as CIAtah format unless user specifies otherwise. NWB files are by default stored in the nwbFiles sub-folder. This can be changed by setting the obj.nwbFileFolder property to a different folder name.

  • Default image mask HDF5 dataset name: '/processing/ophys/ImageSegmentation/PlaneSegmentation'.
  • Default fluorescence activity HDF5 dataset name: '/processing/ophys/Fluorescence/RoiResponseSeries'.

Preferred folder naming format

Folders should following the format YYYY_MM_DD_pXXX_mXXX_assayXX_trialXX where:

  • YYYY_MM_DD = normal year/month/day scheme.
  • pXXX = protocol number, e.g. p162, for the set of experiments performed for the same set of animals.
  • mXXX = subject ID/number, e.g. m805 or animal ID.
  • assayXX = assay ID and session number, e.g. vonfrey01 is the 1st von Frey assay session.
  • trialXX = the trial number of the current assay session, only applicable if multiple trials in the same assay session.

Videos

  • HDF5:
    • Saved as a [x y t] 3D matrix where x and y are the height and width of video while t is number of frames.
    • /1 as the name for directory containing movie data.
    • HDF can be read in using Fiji, see http://lmb.informatik.uni-freiburg.de/resources/opensource/imagej_plugins/hdf5.html.
    • Each HDF5 file should contain imaging data in a dataset name, e.g. /1 is the default datasetname for [x y frames] 2D calcium imaging movies in this repository.
    • Most functions have a inputDatasetName option to specify the dataset name if different from /1.
  • TIF
    • Normal [x y frames] tif.
  • AVI
    • Raw uncompressed grayscale [x y frames] avi.

Cell images

  • IC filters from PCA-ICA and images from CNMF(-E).
    • [x y n] matrix
    • x and y being height/width of video and n is number of ICs output.

Cell traces

  • IC traces from PCA-ICA and images from CNMF(-E).
    • [n f] matrix.
    • n is number of ICs output and f is number of movie frames.

Processing calcium imaging data

The general pipeline for processing calcium imaging data is below. This repository includes code to do nearly every step.

ciapkg_pipeline.png

To start using the CIAtah software package, enter the following into the MATLAB command window.

% Loads all directories
loadBatchFxns;

% Loads the class into an object.
obj = ciatah;

% Open the class menu
obj % then hit enter, no semicolon!
% Alternatively
obj.runPipeline; % then hit enter!

The general order of functions that users should run is ([optional] are those not critical for most datasets):

  • loadDependencies
    • If user is running CIAtah for the first time, this module has several options to download and load CNMF/CNMF-E code for cell extraction, Fiji for viewing/modifying videos (using Miji), and test data from a miniature microscope experiment.
  • modelDownsampleRawMovies [optional]
    • If users have raw calcium imaging data that needs to be spatially downsampled, e.g. raw data from Inscopix nVista software.
  • modelAddNewFolders
    • Users should always use this method first, used to add folders to the current class object.
    • For example, if users ran example_downloadTestData.m, then add the folder [githubRepoPath]\data\2014_04_01_p203_m19_check01_raw where githubRepoPath is the absolute path to the current CIAtah repository.
  • viewMovie
    • Users should check that CIAtah loads their movies correctly and that Miji is working.
    • Remember to check that Imaging movie regexp: (regular expression class uses to find user movies within given folders) setting matches name of movies currently in repository.
  • viewMovieRegistrationTest [optional]
    • Users can check different spatial filtering and registration settings.
    • tregRunX folders (where X is a number) contain details of each run setting. Delete from analysis folder if don't need outputs later.
    • Remember to adjust contrast in resulting montage movies since different filtering will change the absolute pixel values.
  • modelPreprocessMovie
    • Main processing method for CIAtah. Performs motion correction, spatial filtering, cropping, down-sampling, and relative fluorescence calculations. If using Inscopix nVista 1.0 or 2.0, also will correct for dropped frames.
  • modelModifyMovies
    • GUI that allows users to remove movie regions not relevant to cell extraction.
  • modelExtractSignalsFromMovie
    • Performs cell extraction, currently PCA-ICA with the ability to run more recent algorithms (e.g. CNMF) upon request.
  • modelVarsFromFiles
    • Run after modelExtractSignalsFromMovie to load cell image and trace information into the current class object.
  • viewCellExtractionOnMovie [optional]
    • This function overlays the cell extraction outputs on snippets of the processed video, allowing users to check that cell extraction correctly identified all the cells.
  • computeManualSortSignals
    • A GUI to allow users to classify cells and not cells in cell extraction outputs.
  • modelModifyRegionAnalysis [optional]
    • Users are able to select specific cells from cell extraction manual sorting to include in further analyses.
  • computeMatchObjBtwnTrials
    • Method to register cells across imaging sessions. Also includes visual check GUI in viewMatchObjBtwnSessions method.
    • Note: it is heavily advised that throughout a particular animal's imaging sessions, that you keep the acquisition frame dimensions identical. This makes cross-session registration easier. Else you will have to crop all sessions for that animal to the same size ensuring that the area of interest is present in each.

Spatially downsample raw movies or convert to HDF5 with modelDownsampleRawMovies

Users have the ability to spatially downsample raw movies, often necessary to denoise the data, save storage space, and improve runtimes of later processing steps. For most data, users can downsample 2 or 4 times in each spatial dimension while still retaining sufficient pixels per cell to facilitate cell-extraction.

To run, either select modelDownsampleRawMovies in the GUI menu or type the below command after initializing a CIAtah obj.

obj.modelDownsampleRawMovies;

This will pop-up the following screen. Users can

  • input several folders where ISXD files are by separating each folder path with a comma (Folder(s) where raw HDF5s are located),
  • specify a common root folder to save files to (Folder to save downsampled HDF5s to:),
  • and input a root directory that contains the sub-folders with the raw data (Decompression source root folder(s)). The function will automatically put each file in its corresponding folder, make sure folder names are unique (this should be done anyways for data analysis reasons).

image

Converting Inscopix ISXD files to HDF5

To convert from Inscopix ISXD file format (output by nVista v3+ and nVoke) to HDF5 run modelDownsampleRawMovies without changing the regular expression or make sure it looks for .*.isxd or similar. Users will need the latest version of the Inscopix Data Processing Software as these functions take advantage of their API. If CIAtah cannot automatically find the API, it will ask the user to direct it to the root location of the Inscopix Data Processing Software (see below).

image

Check movie registration before pre-processing with viewMovieRegistrationTest

Users should spatially filter one-photon or other data with background noise (e.g. neuropil). To get a feel for how the different spatial filtering affects SNR/movie data before running the full processing pipeline, run viewMovieRegistrationTest module. Then select either matlab divide by lowpass before registering or matlab bandpass before registering then change filterBeforeRegFreqLow and filterBeforeRegFreqHigh settings, see below.

Within each folder will be a sub-folder called preprocRunTest inside of which is a series of sub-folders called preprocRun## that will contain a file called settings.mat that can be loaded into modelPreprocessMovie so the same settings that worked during the test can be used during the actual pre-processing run.

image

  • You'll get an output like the below:
    • A: The top left is without any filtering while the other 3 are with different bandpass filtering options.
    • B: Cell ΔF/F intensity profile from the raw movie. Obtain by selecting Analyze->Plot profile from Fiji menu after selecting a square segment running through a cell.
    • C: Same cell ΔF/F intensity profile from the bottom/left movie (note the y-axis is the same as above). Obtained in same manner as B.

image

Preprocessing calcium imaging movies with modelPreprocessMovie

After users instantiate an object of the ciatah class and enter a folder, they can start preprocessing of their calcium imaging data with modelPreprocessMovie.

  • See below for a series of windows to get started, the options for motion correction, cropping unneeded regions, Δ_F/F_, and temporal downsampling were selected for use in the study associated with this repository.
  • If users have not specified the path to Miji, a window appears asking them to select the path to Miji's scripts folder.
  • If users are using the test dataset, it is recommended that they do not use temporal downsampling.
  • Vertical and horizontal stripes in movies (e.g. CMOS camera artifacts) can be removed via stripeRemoval step. Remember to select correct stripOrientationRemove,stripSize, and stripfreqLowExclude options in the preprocessing options menu.

image

Next the user is presented with a series of options for motion correction, image registration, and cropping.:

  • The options highlighted in green are those that should be considered by users.
  • Users can over their mouse over each option to get tips on what they mean.
  • In particular, make sure that inputDatasetName is correct for HDF5 files and that fileFilterRegexp matches the form of the calcium imaging movie files to be analyzed.
  • After this, the user is asked to let the algorithm know how many frames of the movie to analyze (defaults to all frames).
  • Then the user is asked to select a region to use for motion correction. In general, it is best to select areas with high contrast and static markers such as blood vessels. Stay away from the edge of the movie or areas outside the brain (e.g. the edge of microendoscope GRIN lens in one-photon miniature microscope movies).

image

Save/load preprocessing settings

Users can also enable saving and loading of previously selected pre-processing settings by changing the red option below.

image

Settings loaded from previous run (e.g. of modelPreprocessMovie) or file (e.g. from viewMovieRegistrationTest runs) are highlighted in orange. Settings that user has just changed are still highlighted in green.

image

The algorithm will then run all the requested preprocessing steps and presented the user with the option of viewing a slice of the processed file. Users have now completed pre-processing.

image

Manual movie cropping with modelModifyMovies

If users need to eliminate specific regions of their movie before running cell extraction, that option is provided. Users select a region using an ImageJ interface and select done when they want to move onto the next movie or start the cropping. Movies have NaNs or 0s added in the cropped region rather than changing the dimensions of the movie.

image

Extracting cells with modelExtractSignalsFromMovie

Users can run PCA-ICA, CNMF, CNMF-E, and ROI cell extraction by following the below set of option screens. Details on running the new Schnitzer lab cell-extraction methods will be added here after they are released.

We normally estimate the number of PCs and ICs on the high end, manually sort to get an estimate of the number of cells, then run PCA-ICA again with IC 1.5-3x the number of cells and PCs 1-1.5x number of ICs.

To run CNMF or CNMF-E, run loadDependencies module (e.g. obj.loadDependencies) after ciatah class is loaded. CVX (a CNMF dependency) will also be downloaded and cvx_setup run to automatically set it up.

image

The resulting output (on Figure 45+) at the end should look something like:

image

Loading cell-extraction output data with modelVarsFromFiles

In general, after running cell-extraction (modelExtractSignalsFromMovie) on a dataset, run the modelVarsFromFiles module. This allows CIAtah to load/pre-load information about that cell-extraction run.

If you had to restart MATLAB or are just loading CIAtah fresh but have previously run cell extraction, run this method before doing anything else with that cell-extraction data.

A menu will pop-up like below when modelVarsFromFiles is loaded, you can normally just leave the defaults as is.

image

Validating cell extraction with viewCellExtractionOnMovie

After users have run cell extraction, they should check that cells are not being missed during the process. Running the method viewCellExtractionOnMovie will create a movie with outlines of cell extraction outputs overlaid on the movie.

Below is an example, with black outlines indicating location of cell extraction outputs. If users see active cells (red flashes) that are not outlined, that indicates either exclusion or other parameters should be altered in the previous modelExtractSignalsFromMovie cell extraction step.

2014_04_01_p203_m19_check01_raw_viewCellExtractionOnMovie_ezgif-4-57913bcfdf3f_2

Sorting cell extraction outputs with computeManualSortSignals

Outputs from PCA-ICA (and most other common cell extraction algorithms like CNMF, etc.) output signal sources that are not cells and thus must be manually removed from the output. The repository contains a GUI for sorting cells from not cells. Below users can see a list of options that are given before running the code, those highlighted in green

image

GUI usage on large imaging datasets

  • To manually sort on large movies that will not fit into RAM, select the below options (highlighted in green). This will load only chunks of the movie asynchronously into the GUI as you sort cell extraction outputs. image

Usage

Below are two examples of the interface and code to run if not using the CIAtah GUI.

Usage instructions below for signalSorter.m:

Main inputs

  • inputImages - [x y N] matrix where N = number of images, x/y are dimensions.
  • inputSignals - [N frames] double matrix where N = number of signals (traces).
  • inputMovie - [x y frames] matrix

Main outputs

  • choices - [N 1] vector of 1 = cell, 0 = not a cell
  • inputImagesSorted - [x y N] filtered by choices
  • inputSignalsSorted - [N frames] filtered by choice
iopts.inputMovie = inputMovie; % movie associated with traces
iopts.valid = 'neutralStart'; % all choices start out gray or neutral to not bias user
iopts.cropSizeLength = 20; % region, in px, around a signal source for transient cut movies (subplot 2)
iopts.cropSize = 20; % see above
iopts.medianFilterTrace = 0; % whether to subtract a rolling median from trace
iopts.subtractMean = 0; % whether to subtract the trace mean
iopts.movieMin = -0.01; % helps set contrast for subplot 2, preset movie min here or it is calculated
iopts.movieMax = 0.05; % helps set contrast for subplot 2, preset movie max here or it is calculated
iopts.backgroundGood = [208,229,180]/255;
iopts.backgroundBad = [244,166,166]/255;
iopts.backgroundNeutral = repmat(230,[1 3])/255;
[inputImagesSorted, inputSignalsSorted, choices] = signalSorter(inputImages, inputSignals, 'options',iopts);

BLA one-photon imaging data signal sorting GUI

out-1

mPFC one-photon imaging data signal sorting GUI (from example_downloadTestData.m)

image

Removing cells not within brain region with modelModifyRegionAnalysis

If the imaging field-of-view includes cells from other brain regions, they can be removed using modelModifyRegionAnalysis

image

Cross-session cell alignment with computeMatchObjBtwnTrials

This step allows users to align cells across imaging sessions (e.g. those taken on different days). See the Cross session cell alignment wiki page for more details and notes on cross-session alignment.

  • Users run computeMatchObjBtwnTrials to do cross-day alignment (first row in pictures below).
  • Users then run viewMatchObjBtwnSessions to get a sense for how well the alignment ran.
  • computeCellDistances and computeCrossDayDistancesAlignment allow users to compute the within session pairwise Euclidean centroid distance for all cells and the cross-session pairwise distance for all global matched cells, respectively.

image

Users can then get the matrix that gives the session IDs

% Global IDs is a matrix of [globalID sessionID]
% Each (globalID, sessionID) pair gives the within session ID for that particular global ID
globalIDs = alignmentStruct.globalIDs;

View cross-session cell alignment with viewMatchObjBtwnSessions

To evaluate how well cross-session alignment works, computeMatchObjBtwnTrials will automatically run viewMatchObjBtwnSessions at the end, but users can also run it separately after alignment. The left are raw dorsal striatum cell maps from a single animal. The right shows after cross-session alignment; color is used to indicate a global ID cell (e.g. the same cell matched across multiple days). Thus, same color cell = same cell across sessions.

2017_05_02_p545_m121_p215_raw 2017_05_02_p545_m121_p215_corrected_biafraalgorithm2

Save cross-session cell alignment with modelSaveMatchObjBtwnTrials

Users can save out the alignment structure by running modelSaveMatchObjBtwnTrials. This will allow users to select a folder where CIAtah will save a MAT-file with the alignment structure information for each animal.

ImageJ+MATLAB based mouse location tracking

Functions needed (have entire CIAtah loaded anyways):

  • mm_tracking.ijm is the tracking function for use in ImageJ, place in plugins folder. If already had CIAtah download Fiji, place in the _external_programs/[Fiji directory]/Fiji.app/plugins folder.
  • removeIncorrectObjs.m is a function to clean-up the ImageJ output.
  • createTrackingOverlayVideo is a way to check the output from the tracking by overlaying mouse tracker onto the video.

Instructions for ImageJ and Matlab

Example screen after running mm_tracking within ImageJ, click to expand.

image

After the above screen, there will be multiple other screens culminating in one where a threshold is chosen that is used to remove non-animal pixels from analysis. The threshold matters quite a bit and the script ignores anything that isn't red (i.e. larger than threshold) OR not within the range specified by the parameters below.

image

The script opens the AVI as a virtual stack and asks for the threshold is so that I can quickly scan through the entire movie to make sure the set threshold works even with slight/major changes in illumination, e.g. the below threshold will work across many frames

image

If the threshold is set to low, certain frames will not have the animal detected, e.g. if the lighting changes. image

Once ImageJ is finished, within Matlab run the following code (cleans up the ImageJ tracking by removing small objects and adding NaNs for missing frames along with making a movie to check output). Modify to point toward paths specific for your data.

% CSV file from imageJ and AVI movie path used in ImageJ
moviePath = 'PATH_TO_AVI_USED_IN_IMAEJ';
csvPath = 'PATH_TO_CSV_OUTPUT_BY_IMAGEJ';
% clean up tracking
[trackingTableFilteredCell] = removeIncorrectObjs(csvPath,'inputMovie',{moviePath});
% make tracking video
% frames to use as example check
nFrames=1500:2500;
inputMovie = loadMovieList(moviePath,'frameList',nFrames);
[inputTrackingVideo] = createTrackingOverlayVideo(inputMovie,movmean(trackingTableFilteredCell.XM(nFrames),5),movmean(trackingTableFilteredCell.YM(nFrames),5));
playMovie(inputTrackingVideo);

Example output from 2017_09_11_p540_m381_openfield01_091112017

image

Using createTrackingOverlayVideo to verify tracking matches animal position on a per frame basis.

image

License

Copyright (C) 2013-2020 Biafra Ahanonu

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.