labcompass_preprocessing

Preprocessing for the LabCompass spectral flow cytometry screen: from one .fcs file per well to one annotated AnnData object per experimental loop, carrying logicle-transformed marker intensities, a UMAP embedding and Leiden clusters.

The screen measures a 20-antibody haematopoietic panel on cord blood cells grown under many combinations of cytokines and small molecules. Each combination is one experiment; each round of the screen is one loop.

What is in here

unmixing/                  the R notebooks that unmix and filter the wells, before stage 00
notebooks/helper_func.py   the few helpers too long to repeat in every notebook
notebooks/<Loop>/          one folder per loop: notebooks, lookup tables, README
env/                       the pinned Python environment that produced the published files
docs/PREPROCESSING.md      what each step does, with the package and version that does it
docs/AUDIT.md              what was found in the original notebooks, and what changed

There is nothing to install. helper_func.py sits beside the loop folders and a notebook reaches it with sys.path.append(".."). It holds only the pieces that are long and shared: the interactive gating class, gate application, downsampling, the cleaning steps, the logicle loop, the embedding and the four plotting functions. Everything else a stage does is written out in the notebook cell that does it, so you can read a stage without opening a second file.

Each loop folder has its own README.md giving that loop's file-dependency graph: which notebook reads which file and which notebook writes it. Read that first when you want to rerun a loop.

The loops

loop	what it covers	experiments	raw data
Loop0	every plate acquired up to that point; draws the gates and builds the calibration	122 after removals	all of for_Juan_Licyel/
Loop1	LPHO013 plate 1	216 to 227	LPHO013/plate1
Loop1p5	LPHO013 plate 2	228 to 238	LPHO013/plate2
Loop2	LPHO014, first block	239 to 247	LPHO014
Loop2p5	LPHO014, second block	248 to 251	LPHO014
Loop3	LPHO014, third block	252 to 258	LPHO014
Loop4	LPHO014, fourth block	259	LPHO014
Loop4p5	LPHO014, fifth block	260	LPHO014
Fig6	LPHO015, the experiments behind Figure 6	261 to 280	LPHO015

Loop0 is the reference loop. It is the only one that builds the fluorescence-minus-one calibration, and every other loop carries a copy of the two files it produces, marker_cutoff.pkl and logicle_parameters.json, and uses them unchanged. The intensity scale is therefore the same across loops.

Two other things vary. Loop0 and Loop1 are the only loops that draw gates by hand; the rest apply gates drawn in Loop1. Loop0 and Fig6 are the only loops that go on to cluster; the other seven stop after the UMAP.

Getting started

conda env create -f env/environment.yml
conda activate labcompass
jupyter lab

Then open the loop you want under notebooks/ and run its notebooks in numerical order.

Where the raw data lives

/rds/project/rds-SDzz0CATGms/unsorted/for_Juan_Licyel

One unmixed .fcs file per well. The raw data is far too large to commit, so the repository records only the path. If your copy sits elsewhere, set the environment variable before starting Jupyter:

export LABCOMPASS_FCS_ROOT=/your/path/to/for_Juan_Licyel

Every notebook reads that variable and falls back to the path above.

Before stage 00: unmixing and cell filtering

The .fcs files under for_Juan_Licyel/ are not the files that the cytometer wrote. Each one went through three steps in R first:

1. Spectral unmixing with the AutoSpectral package, with autofluorescence spectra fitted separately for each culture condition from its own unstained well.
2. Quality control with PeacoQC, which removes the events recorded during unstable periods of the acquisition.
3. Renaming of the channels to the names that stage 00 expects, and writing of the result to a filtered/ folder.

The notebooks that did this are in unmixing/, with a README that gives each step, its inputs and outputs, and the event counts before and after PeacoQC. So far they cover LPHO014 plate 1 only, which feeds Loop2 and Loop2p5. The notebooks for the other plates and datasets have not been received.

These notebooks are a record of what was run. They are committed as received with the outputs stripped, their paths are absolute, and they need their own R environment, which is not in env/.

Relation to the original notebooks

This repository is a cleaned rewrite. docs/AUDIT.md lists what was found in the original notebooks, what was fixed, and what was deliberately left alone. It also gives the mapping from the original working directories and file names to the ones used here.