Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 25 additions & 14 deletions .github/workflows/pkgdown.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# .github/workflows/pkgdown.yaml
#
# Builds the website and publishes it to the gh-pages branch on every push to
# main. The tutorials really run here, so each build installs SeuratData and
# downloads both public test datasets. A failed build publishes nothing, and the
# main. The tutorials really run here. They download their data with
# download_NICHESv2_data() from this repository's data-v1 release, so they need
# nothing outside NICHESv2. Only the "Starting from a Seurat object" page uses
# SeuratData; its install step may fail without stopping the build, and the page
# then shows its code without output. A failed build publishes nothing, and the
# live site stays as it was.
#
# To test a build without publishing it, start this workflow by hand from the
Expand Down Expand Up @@ -36,24 +39,32 @@ jobs:
with:
use-public-rspm: true

# Step 3: install this package, everything in DESCRIPTION, and the tools
# the website needs
# Step 3: install this package, everything in DESCRIPTION, and pkgdown
- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: |
any::pkgdown
any::remotes
any::Seurat
local::.

# Step 4: install SeuratData and download the test datasets
# SeuratData isn't on CRAN, so it comes from GitHub. It gets its own step
# because Step 3 would also require Azimuth, an optional SeuratData
# package that isn't on CRAN or Bioconductor.
- name: Install SeuratData and test datasets
run: |
remotes::install_github("satijalab/seurat-data", upgrade = "never")
SeuratData::InstallData(c("stxBrain", "ifnb"))
# Step 4 (optional): install Seurat and SeuratData for the Seurat page
# continue-on-error: true → a failure here doesn't stop the build
# dependencies: "hard" → skips SeuratData's optional Azimuth
# package, which isn't on CRAN
- name: Install Seurat and SeuratData (optional)
id: seuratdata
continue-on-error: true
uses: r-lib/actions/setup-r-dependencies@v2
with:
packages: |
any::Seurat
github::satijalab/seurat-data
dependencies: '"hard"'
cache: false

- name: Download the SeuratData datasets (optional)
if: steps.seuratdata.outcome == 'success'
continue-on-error: true
run: SeuratData::InstallData(c("stxBrain", "ifnb"))
shell: Rscript {0}

# Step 5: build the site
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,6 @@ docs
# Local Claude Code settings and OmniPath download logs
.claude/settings.local.json
omnipathr-log/

# Tutorial data built by data-raw/build_tutorial_data.R; uploaded to a GitHub release
data-raw/tutorial-data/
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ Custom data frames are also accepted; must have `ligand` and `receptor` columns

- `extract_NICHESInputs_Seurat()` — supports Seurat V4 and V5; detects assay class automatically; **spatial coordinates must be extracted manually** and added to the returned metadata before calling `create_NICHESObject()`
- `extract_NICHESInputs_AnnData()` — reads `.h5ad` via `rhdf5` only; no Python required
- `download_NICHESv2_data()` — downloads the two public tutorial datasets (`"stxBrain"`, `"ifnb"`) from the GitHub release `data-v1`, checks their MD5 sums and caches them in `tools::R_user_dir("NICHESv2", "cache")`. On a failed download or a wrong sum it returns `NULL` with a message instead of an error, as CRAN asks of internet resources. `data-raw/build_tutorial_data.R` builds the release files; rebuilding them changes the MD5 sums in `.niches_datasets()`. The `NICHESv2.data.url` option points the function at another folder, for tests and local site builds.

## Repository layout notes

Expand All @@ -119,7 +120,7 @@ Custom data frames are also accepted; must have `ligand` and `receptor` columns
- `data-raw/` — scripts that build the bundled data files; excluded from package build
- `tests/testthat/` — one test file per source file; `helper-testutils.R` provides shared fixtures
- `vignettes/` — one vignette, `NICHESv2.Rmd`, which runs on bundled data; `vignettes/articles/` holds the website pages and is excluded from the package build
- `.github/workflows/` — `R-CMD-check.yaml` runs the checks; `pkgdown.yaml` builds the website and publishes it from `main`, removing this file first because pkgdown turns top-level Markdown files into pages
- `.github/workflows/` — `R-CMD-check.yaml` runs the checks; `pkgdown.yaml` builds the website and publishes it from `main`, removing this file first because pkgdown turns top-level Markdown files into pages. Only the Seurat page uses SeuratData; its install steps may fail without stopping the build
- `demo-scripts/` — standalone R scripts demonstrating real-dataset usage (not part of the package build)
- `working-scripts-MSBR/` and `working-scripts-NW/` — exploratory and prototyping scripts; not part of the package
- `package management/` — design documents
1 change: 1 addition & 0 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Imports:
ggplot2 (>= 3.4.0),
parallel,
stats,
tools,
utils
Suggests:
SeuratObject,
Expand Down
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export(compute_CellToCell)
export(create_Edgelist_SamplingByType)
export(create_Edgelist_Spatial)
export(create_NICHESObject)
export(download_NICHESv2_data)
export(export_to_TissuePlex)
export(extract_NICHESInputs_AnnData)
export(extract_NICHESInputs_Seurat)
Expand Down
3 changes: 3 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ First public release.
Python.
- `export_to_TissuePlex()` writes edges to a Parquet file for interactive
viewing.
- `download_NICHESv2_data()` downloads the two public datasets used in the
website tutorials: a 10x Visium mouse brain section and interferon-beta
stimulated PBMCs.

## Changes for users of the 0.0.9000 development version

Expand Down
167 changes: 167 additions & 0 deletions R/download_NICHESv2_data.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
# download_NICHESv2_data.R ------------------------------------------------
# Download one of the public datasets used by the website tutorials from a
# release of the NICHESv2 GitHub repository, check it, cache it, and return
# inputs ready for create_NICHESObject(). The files are built by
# data-raw/build_tutorial_data.R.
#
# FUNCTIONS:
# download_NICHESv2_data() -- main function
# .read_inputs() -- read inputs.rds with Matrix loaded
# .niches_data_url() -- release folder the files are attached to
# .niches_datasets() -- file name, MD5 sum and size of each dataset
#
# DEPS: tools, utils


# ==============================================================================
#' Download a Public Tutorial Dataset
#'
#' @description
#' Downloads one of the two public datasets used in the website tutorials,
#' checks its MD5 sum, and returns a count matrix and metadata ready for
#' \code{create_NICHESObject()}. The files are attached to a release of the
#' NICHESv2 GitHub repository, so no other package is needed. Each dataset is
#' downloaded once and then read from \code{cache.dir}.
#'
#' @section Datasets:
#' \describe{
#' \item{\code{"stxBrain"}}{10x Genomics Visium mouse brain, sagittal-anterior
#' section 1: 2,696 spots and 31,053 genes, with spot coordinates in
#' \code{x} and \code{y}. About 14 MB. CC BY 4.0, credit 10x Genomics.}
#' \item{\code{"ifnb"}}{Human PBMCs, untreated or stimulated with
#' interferon-beta (Kang et al., 2018): 13,999 cells and 14,053 genes, with
#' the condition in \code{stim} and cell-type labels from the Satija Lab in
#' \code{seurat_annotations}. About 11 MB. CC BY 4.0.}
#' }
#' Each dataset's folder in \code{cache.dir} also holds a \code{PROVENANCE.md}
#' file stating its source, license and the changes made.
#'
#' @param dataset \code{"stxBrain"} or \code{"ifnb"}.
#' @param cache.dir Folder to keep downloaded files in. Default: the NICHESv2
#' user cache folder, \code{tools::R_user_dir("NICHESv2", "cache")}. Delete
#' the folder to free the space.
#' @param force Download again even if the dataset is already cached.
#' Default \code{FALSE}.
#' @param verbose Print progress messages. Default \code{TRUE}.
#'
#' @return A list with \code{count.mtx} (a sparse genes x cells matrix of raw
#' counts) and \code{meta.data} (a data frame with one row per cell), the
#' same shape that \code{extract_NICHESInputs_Seurat()} returns. If the
#' download fails, or the file doesn't match its MD5 sum, a message explains
#' why and \code{NULL} is returned invisibly.
#'
#' @references
#' Kang HM, Subramaniam M, Targ S, et al. (2018). Multiplexed droplet
#' single-cell RNA-sequencing using natural genetic variation.
#' \emph{Nature Biotechnology}, 36(1), 89--94. \doi{10.1038/nbt.4042}
#'
#' @examples
#' \donttest{
#' # Downloads about 11 MB, kept in a temporary folder here.
#' inputs <- download_NICHESv2_data("ifnb", cache.dir = tempdir())
#' if (!is.null(inputs)) dim(inputs$count.mtx)
#' }
#'
#' @export
download_NICHESv2_data <- function(dataset,
cache.dir = tools::R_user_dir("NICHESv2", "cache"),
force = FALSE,
verbose = TRUE) {

datasets <- .niches_datasets()
if (missing(dataset) || !is.character(dataset) || length(dataset) != 1L ||
!dataset %in% names(datasets))
stop(
"[download_NICHESv2_data] 'dataset' must be one of: ",
paste0("'", names(datasets), "'", collapse = ", "), ".",
call. = FALSE
)

info <- datasets[[dataset]]
rds.file <- file.path(cache.dir, dataset, "inputs.rds")

if (!force && file.exists(rds.file)) {
if (verbose)
message("[download_NICHESv2_data] Reading cached '", dataset, "' from ",
dirname(rds.file))
return(.read_inputs(rds.file))
}

# -- Download and check the zip ---------------------------------------------
dir.create(cache.dir, recursive = TRUE, showWarnings = FALSE)
zip.file <- file.path(cache.dir, info$file)
url <- paste(.niches_data_url(), info$file, sep = "/")

if (verbose)
message("[download_NICHESv2_data] Downloading '", dataset, "' (",
info$size, ") ...")
old.options <- options(timeout = max(600, getOption("timeout")))
on.exit(options(old.options), add = TRUE)

downloaded <- tryCatch(
utils::download.file(url, zip.file, mode = "wb", quiet = TRUE) == 0L,
error = function(e) FALSE,
warning = function(w) FALSE
)
if (!downloaded || !file.exists(zip.file)) {
unlink(zip.file)
message(
"[download_NICHESv2_data] Could not download '", dataset, "' from\n ",
url, "\n Check your internet connection and try again."
)
return(invisible(NULL))
}

if (!identical(unname(tools::md5sum(zip.file)), info$md5)) {
unlink(zip.file)
message(
"[download_NICHESv2_data] The downloaded file for '", dataset,
"' doesn't match its expected MD5 sum, so it was deleted.\n ",
"Try again with force = TRUE. If this keeps happening, please open an ",
"issue at https://github.com/RaredonLab/NICHESv2/issues"
)
return(invisible(NULL))
}

# -- Unpack into cache.dir/<dataset>/ ---------------------------------------
unlink(file.path(cache.dir, dataset), recursive = TRUE)
utils::unzip(zip.file, exdir = cache.dir)
unlink(zip.file)

.read_inputs(rds.file)
}


## INTERNAL HELPERS ----

#' Read an inputs.rds file. Matrix is loaded first: otherwise readRDS() loads it
#' while unserializing, and the first dim() call on the matrix returns NULL.
#' @noRd
.read_inputs <- function(rds.file) {
loadNamespace("Matrix")
readRDS(rds.file)
}

#' Release folder that holds the dataset files. The NICHESv2.data.url option
#' points elsewhere for tests and for building the site before an upload.
#' @noRd
.niches_data_url <- function() {
getOption(
"NICHESv2.data.url",
"https://github.com/RaredonLab/NICHESv2/releases/download/data-v1"
)
}

#' File name, MD5 sum and download size of each dataset. The sums come from
#' data-raw/build_tutorial_data.R and must match the uploaded files.
#' @noRd
.niches_datasets <- function() {
list(
stxBrain = list(file = "stxBrain.zip",
md5 = "f2085cbb105198f29e94456eeebdf261",
size = "14 MB"),
ifnb = list(file = "ifnb.zip",
md5 = "bd4d47cac9dc110aa253f4c6e95fcc43",
size = "11 MB")
)
}
4 changes: 4 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ navbar:
href: articles/spatial-tutorial.html
- text: "Non-spatial: stimulated PBMCs"
href: articles/nonspatial-tutorial.html
- text: Starting from a Seurat object
href: articles/seurat-objects.html
guides:
text: Guides
menu:
Expand All @@ -40,6 +42,7 @@ articles:
contents:
- articles/spatial-tutorial
- articles/nonspatial-tutorial
- articles/seurat-objects
- title: Guides
navbar: ~
contents:
Expand Down Expand Up @@ -86,6 +89,7 @@ reference:
- create_Edgelist_SamplingByType
- title: Example data
contents:
- download_NICHESv2_data
- NICHESv2_inputs
- NICHESv2_example
- title: LR databases
Expand Down
Loading
Loading