Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
dd7edae
Fix package-name casing and add CRAN submission files
DougManuel Jul 3, 2026
05bddc1
Add post-v0.4.0 development plan
DougManuel Jul 3, 2026
1ab2603
Pin edge-case input contract with characterization tests
DougManuel Jul 3, 2026
122e398
Suppress deterministic fixture warnings in sanity-anchor test
DougManuel Jul 3, 2026
1ecbeff
Allow n = 0 in create_mock_data() and align n validation with the spe…
DougManuel Jul 3, 2026
127abd0
Suppress deterministic fixture warnings in n = 0 contract test
DougManuel Jul 3, 2026
f9c3601
Check databaseStart upfront in create_mock_data()
DougManuel Jul 3, 2026
bb969e4
Add exponential distribution to the native backend
DougManuel Jul 3, 2026
ff12d2d
Add recodeflow integration test for native exponential
DougManuel Jul 3, 2026
b749b11
Move rate after rtype in continuous constructors and polish exponenti…
DougManuel Jul 3, 2026
361e269
Address PR review findings: guards, case normalization, contextual er…
DougManuel Jul 3, 2026
6099176
Merge pull request #45 from Big-Life-Lab/v04x-hardening
DougManuel Jul 3, 2026
f59197f
Add accepted v0.5 seed-contract ADR (#38)
DougManuel Jul 6, 2026
1ad2cb7
Add #38 seed-contract implementation plan
DougManuel Jul 6, 2026
fd4cf47
Adopt L'Ecuyer-CMRG per-stage seed sub-streams (#38)
DougManuel Jul 6, 2026
e3d38da
Silence RNGkind-quality caution in the seed-contract test
DougManuel Jul 6, 2026
c50c70c
Unify legacy path on the seed contract and announce fallbacks (#38)
DougManuel Jul 6, 2026
577ec9a
Complete seed-contract tests and docs (#38)
DougManuel Jul 6, 2026
5bc0b97
Pin normal.kind and sample.kind in the seed primitive (#38)
DougManuel Jul 6, 2026
280916d
Silence deprecated sample.kind caution in the RNGkind-independence test
DougManuel Jul 6, 2026
0f9bda8
Address PR #48 review: warning leak, stage-index pin, doc accuracy (#38)
DougManuel Jul 13, 2026
c4c0527
Merge pull request #48 from Big-Life-Lab/v05-seed-contract
DougManuel Jul 16, 2026
e4586c6
Add accepted v0.5 formula-evaluator ADR (#39)
DougManuel Jul 7, 2026
3d2549c
Add #39 formula-evaluator implementation plan
DougManuel Jul 16, 2026
9dba1c0
Add formula variable type to the mock_spec layer (#39)
DougManuel Jul 16, 2026
0d91853
Defer formula parse errors to validate_mock_spec (#39)
DougManuel Jul 16, 2026
7e5a8f4
Add evaluate_mock_formulas() with sandboxed post-baseline evaluation …
DougManuel Jul 16, 2026
612810c
Strict-validate specs at evaluator entry and mirror the formula skip …
DougManuel Jul 16, 2026
b2ff0e0
Wire mockFormula through the recodeflow adapter and orchestrator (#39)
DougManuel Jul 16, 2026
af1fb0f
Scope the D6 mockFormula carve-out to the requested databaseStart (#39)
DougManuel Jul 16, 2026
a969417
Document formula-derived variables and refresh deferred-work claims (…
DougManuel Jul 17, 2026
779463d
Add defensive warnings for factor-to-numeric coercion and conflicting…
DougManuel Jul 17, 2026
58bdfc7
Reconcile plan and ADR bookkeeping with the shipped #38/#39 work
DougManuel Sep 18, 2026
43db46b
Add R CMD check workflow and capture simstudy in the lockfile
DougManuel Sep 18, 2026
3d63ba9
Merge branch 'v05-ci-lockfile' into v05-formula-evaluator
DougManuel Sep 18, 2026
f1ed15d
Prepare v0.5.0 release: version, NEWS structure, README status, cran-…
DougManuel Sep 18, 2026
99d1fd2
Add the formula-variable exports to the pkgdown reference index (#39)
DougManuel Sep 18, 2026
ffff4cc
Return the native zero-row schema from the simstudy backend for n = 0…
DougManuel Sep 18, 2026
f1164f2
Merge branch 'v05-ci-lockfile' into v05-formula-evaluator
DougManuel Sep 18, 2026
e75b05b
Merge branch 'v05-ci-lockfile' into v05-release-prep
DougManuel Sep 18, 2026
3135d15
Merge branch 'v05-formula-evaluator' into v05-release-prep
DougManuel Sep 18, 2026
682768b
Add proposed v0.5 survival-dates ADR (#40)
DougManuel Sep 23, 2026
c9b5eba
Accept the v0.5 survival-dates ADR with D5 and D8 ratified (#40)
DougManuel Sep 24, 2026
097841c
Correct the survival-dates ADR from plan preparation (#40)
DougManuel Sep 24, 2026
01e5607
Expand range-notation missing codes on the v0.4 path (#58)
DougManuel Sep 24, 2026
2548b51
Merge branch 'v05-range-missing-codes' into v05-survival-dates
DougManuel Sep 24, 2026
0da65f6
Add the #40 survival-dates implementation plan
DougManuel Sep 24, 2026
7532dca
Merge pull request #51 from Big-Life-Lab/v05-ci-lockfile
DougManuel Sep 24, 2026
2fdf29b
Merge pull request #52 from Big-Life-Lab/v05-formula-evaluator
DougManuel Sep 24, 2026
976c175
Merge pull request #59 from Big-Life-Lab/v05-range-missing-codes
DougManuel Sep 24, 2026
9ccc041
Amend the survival ADR and plan after external design review (#40)
DougManuel Sep 24, 2026
4290de6
Characterize the legacy survival engine before the port (#40)
DougManuel Sep 24, 2026
8406b17
Add the survival variable type to the mock_spec layer (#40)
DougManuel Sep 24, 2026
181c250
Add generate_survival_dates() as a post-baseline stage (#40)
DougManuel Sep 24, 2026
192425f
Generate survival dates from recodeflow metadata in create_mock_data(…
DougManuel Sep 24, 2026
7cf2278
Allow is.na in mockFormula to derive survival status (#40)
DougManuel Sep 24, 2026
75f3aed
Deprecate create_wide_survival_data() in favour of anchor metadata (#40)
DougManuel Sep 24, 2026
90d9a07
Document metadata-driven survival dates (#40)
DougManuel Sep 24, 2026
b682a15
Stop recommending validate = FALSE for survival metadata errors (#40)
DougManuel Sep 24, 2026
b0a4e97
Restructure survival docs as asserted integration tests (#40)
DougManuel Sep 26, 2026
5c57b2a
Tighten survival doc claims after review (#40)
DougManuel Sep 26, 2026
e0930df
Correct the survival stream claim: survival dates share a stream too …
DougManuel Sep 26, 2026
c160e02
Merge pull request #61 from Big-Life-Lab/v05-survival-dates
DougManuel Sep 27, 2026
1b3c201
Merge dev into v05-release-prep and fold the #40 and #58 entries into…
DougManuel Sep 27, 2026
0a6ddf4
Record v0.5 release assembly as complete in the post-v0.4.0 plan
DougManuel Sep 27, 2026
c69d626
Merge pull request #53 from Big-Life-Lab/v05-release-prep
DougManuel Sep 27, 2026
fe4a739
Stop the legacy generator on formula metadata instead of ignoring it
DougManuel Sep 27, 2026
b803718
Fix the minimal example's NaN heights and refresh release docs (#60, …
DougManuel Sep 27, 2026
a2d8225
Commit generated help files so GitHub installs have help pages
DougManuel Sep 27, 2026
f8d012a
Merge pull request #64 from Big-Life-Lab/v05-release-fixes
DougManuel Sep 27, 2026
342f382
Scope legacy variable selection by variable-level databaseStart
DougManuel Sep 27, 2026
b6e2189
Merge pull request #65 from Big-Life-Lab/v05-db-scoping
DougManuel Sep 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
^renv$
^renv\.lock$
^.*\.Rproj$
^.*\.Rcheck$
^.*\.tar\.gz$
^\.Rproj\.user$
^\.DS_Store$
^\.git$
Expand All @@ -23,7 +25,13 @@
^PR_DESCRIPTION_ADDENDUM\.md$
^REVIEWER_QUICKSTART\.md$
^LOCAL_TESTING_GUIDE\.md$
^cran-comments\.md$
^docs$
^_pkgdown\.yml$
^pkgdown$
^development$
^\.claude$
^vignettes/\.quarto$
^vignettes/.*_files$
^vignettes/.*\.html$
^\.superpowers$
53 changes: 53 additions & 0 deletions .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# R CMD check on every push to main/dev and every PR into them.
# Complements pkgdown.yaml, which only builds the site. Dependencies come
# from DESCRIPTION via setup-r-dependencies (Suggests included, so the
# optional simstudy backend is exercised here even when it is absent from
# a developer's local library). See .claude/AI.md for the CI lessons that
# shaped this file.
on:
push:
branches: [main, dev]
pull_request:
branches: [main, dev]
workflow_dispatch:

name: R-CMD-check

permissions:
contents: read

jobs:
R-CMD-check:
runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes
LANG: en_US.UTF-8
LC_ALL: en_US.UTF-8

steps:
- uses: actions/checkout@v4

- uses: r-lib/actions/setup-pandoc@v2

- name: Install Quarto
uses: quarto-dev/quarto-actions/setup@v2

- uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true

- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: any::rcmdcheck, any::roxygen2
needs: check

- name: Generate documentation
run: Rscript -e "roxygen2::roxygenize()"

- uses: r-lib/actions/check-r-package@v2
with:
upload-snapshots: true
error-on: '"warning"'
build_args: 'c("--no-manual")'
args: 'c("--no-manual", "--as-cran")'
9 changes: 7 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,15 @@ CLAUDE.md
# macOS
.DS_Store

# Generated documentation
man/*.Rd
# Generated documentation (man/*.Rd is committed so GitHub installs have help pages)
docs/

# Quarto/HTML output
Generate_mock_data.html
Generate_mock_data_files/
vignettes/.quarto/
vignettes/*_files/
vignettes/*.html

# Superpowers plugin scratch (per-developer)
.superpowers/
24 changes: 24 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,30 @@ covr::package_coverage()
- If adding new test data, document its purpose
- Keep test data small and focused

#### Vignettes are integration tests

The vignettes run MockData's own code on realistic metadata, and `R CMD check` rebuilds every one of them in CI. That makes them integration tests, but only if a vignette fails when the code stops doing what its prose says. A vignette that prints a number without checking it renders happily after a regression, and a number typed into the prose goes stale without anyone noticing.

When you write or change a vignette:

- Compute every factual claim with inline R (`` `r n_deaths` ``) instead of typing the number into the prose.
- Guard the claims with a hidden chunk of named `stopifnot()` conditions, so the render, and therefore `R CMD check` and CI, fails with a message naming the broken claim:

````markdown
```{r}
#| include: false
stopifnot(
"deaths are exactly floor(n * event_prop)" = n_deaths == floor(n * 0.3),
"no event is recorded after a death" = events_after_death == 0
)
```
````

- State known issues as claims too, with the issue number in the message. When the issue is fixed, the render fails until the documentation is updated.
- For a new feature, check that the assertions can fail: break the behaviour in a scratch copy and check that the render stops.

The survival vignettes (`tutorial-survival-data`, `survival-dates-v05`, `survival-design-v05`) follow this pattern.

### Building Documentation

#### Prerequisites for documentation builds
Expand Down
3 changes: 2 additions & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: MockData
Title: Generate Mock Data from Metadata Specifications
Version: 0.4.0
Version: 0.5.0
Authors@R: c(
person("Juan", "Li", role = "aut", email = "juli@ohri.ca"),
person("Douglas", "Manuel", role = c("aut", "cre"), email = "dmanuel@ohri.ca"),
Expand All @@ -25,6 +25,7 @@ VignetteBuilder: quarto
Depends:
R (>= 4.2.0)
Imports:
parallel,
stats
Suggests:
testthat (>= 3.0.0),
Expand Down
5 changes: 5 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@ export(create_date_var)
export(create_mock_data)
export(create_survival_dates)
export(create_wide_survival_data)
export(evaluate_mock_formulas)
export(extract_distribution_params)
export(extract_proportions)
export(generate_garbage_values)
export(generate_mock_data_native)
export(generate_mock_data_simstudy)
export(generate_survival_dates)
export(get_cycle_variables)
export(get_enabled_variables)
export(get_raw_var_dependencies)
Expand All @@ -31,11 +33,14 @@ export(make_garbage)
export(mock_categorical)
export(mock_continuous)
export(mock_date)
export(mock_formula)
export(mock_spec)
export(mock_spec_categorical)
export(mock_spec_continuous)
export(mock_spec_date)
export(mock_spec_formula)
export(mock_spec_from_recodeflow)
export(mock_spec_survival)
export(parse_range_notation)
export(parse_variable_start)
export(postprocess_mock_data)
Expand Down
144 changes: 142 additions & 2 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,143 @@
# MockData 0.5.0 (2026-09-27)

## Breaking changes

- Seeded output changed once for the RNG-stream mechanism in this release.
Within the `create_mock_data()` pipeline (`generate_mock_data_native()`,
`generate_mock_data_simstudy()`, `postprocess_mock_data()`), MockData now
derives all randomness from independent L'Ecuyer-CMRG sub-streams (one per
generation stage) seeded from the single public `seed`, replacing the
previous Mersenne-Twister `seed` / `seed + 1` scheme. The standalone
`create_*` helpers (`create_cat_var()`, `create_con_var()`,
`create_date_var()`, `create_survival_dates()`,
`create_wide_survival_data()`, `sample_with_proportions()`,
`make_garbage()`, `apply_garbage()`) called directly are unaffected and
still seed a single Mersenne-Twister stream per call. For a given seed
**and package version** pipeline output is reproducible and independent of
the session's ambient `RNGkind()`; it is not comparable across the v0.4 →
v0.5 boundary. For seeded calls, generation no longer alters the caller's
RNG state (previously the legacy `validate = FALSE` path reset it). (#38)

**Migrating:** if your tests pin values from seeded v0.4 runs of
`create_mock_data()` or the `mock_spec` pipeline, regenerate those expected
values once under v0.5 and pin them again. The same seed keeps giving
identical output within this version. Standalone `create_*` calls need no
change.

- Migration: add `anchor` (and `censored_by` where a competing risk applies)
to each survival date's row. A date with `followup_min`, `followup_max` or
`event_prop` but no `anchor` now fails with a message naming the fix;
previously it was silently dropped or generated as a plain calendar date.
- Missing codes and garbage are applied to survival dates after the survival
rules. Garbage that places a date before entry is therefore kept, whereas
the legacy engine set such dates to `NA`.
- The packaged minimal example now generates entirely through the v0.4
pipeline, because its Gompertz survival dates no longer force the legacy
generator. Its seeded output differs from v0.4.
- If the legacy generator is selected (`validate = FALSE`,
`variable_details = NULL`, detail-level-only `databaseStart`, or another
unsupported variable) on metadata that sets `anchor`, `create_mock_data()`
stops and names the survival variables rather than dropping them.
- The same applies to formula variables. If the legacy generator is
selected for metadata that sets `mockFormula` on an enabled variable,
`create_mock_data()` stops and names the formula variables. Previously the
legacy generator ignored the formula and returned unrelated random values,
or dropped a `DerivedVar::` variable, without saying so.
- On the legacy path (`validate = FALSE` and the other legacy triggers),
`create_mock_data()` now generates only variables whose variable-level
`databaseStart` includes the requested database, as the v0.4 pipeline
does. Previously it chose variables from the detail rows alone, so a
variable listed only for another database was generated as random values,
and a formula or survival variable belonging to another database stopped
the run.

## New features

- Formula-derived variables (#39, Phase A). Variables can now be generated
from algebraic expressions over other generated variables via a new
`mockFormula` column in `variable_details` (e.g. `weight / (height^2)`),
or the new `mock_formula()` direct API. Formulas are evaluated in
dependency order by `evaluate_mock_formulas()`, in a restricted environment
exposing only the generated columns and a fixed allow-list of base
functions. `DerivedVar::`/`Func::` semantics are unchanged: derived
variables without a `mockFormula` remain excluded from generation, and
`Func::` dispatch is not yet supported.
- `create_mock_data()` now generates survival dates from metadata. A date
whose `variables.csv` row sets `anchor` (its entry-date variable) is a
survival date: `floor(n * event_prop)` rows receive a date between
`followup_min` and `followup_max` days after the anchor, drawn from a
uniform, exponential or Gompertz distribution, and the rest are `NA`. An
optional `censored_by` column names a competing survival date, such as
death, that sets this date to `NA` where it comes first. The statistics and
rules are those of `create_wide_survival_data()`, ported exactly. New
functions: `mock_spec_survival()` and `generate_survival_dates()`. This
resolves the known issue, listed since v0.2.0, that survival data had to be
generated separately.
- Derived columns, from `mockFormula` or survival dates, describe the clean
generated values; missing codes and garbage are then applied to each column
independently. `is.na` is added to the `mockFormula` allow-list so status
and follow-up time can be derived. Derive both from the same observation
window: `as.integer(!is.na(death_date))` only says a death date exists,
not that the death was observed before censoring.
- `create_mock_data()` now accepts `n = 0`, returning a zero-row data frame
with the full generated schema (useful for schema tests), and rejects
fractional, negative, `NA`, and non-finite values of `n` with a clear
message. The validation now matches `generate_mock_data_native()`.
- The native backend now supports `distribution = "exponential"` (parity with
the legacy generator), removing a forced legacy-fallback for exponential
metadata. When the optional simstudy backend is selected, exponential
variables are routed to the native generator (like all non-uniform
continuous distributions); the simstudy package itself is not involved in
their generation. Native exponential values are truncated to the declared
`range` by inverse-CDF sampling, rather than clipped at the range maximum
(with a point mass at the boundary) as the legacy `rexp()` path does.

## Validation and error messages

- Calling `create_mock_data()` without `databaseStart` now fails upfront with
a message naming the argument, instead of a raw missing-argument error.
- With `validate = TRUE` (the default), invalid distribution parameters in
metadata — e.g. `distribution = "exponential"` without a positive `rate` —
now stop generation with a message naming the variable and how to fix it,
instead of warning and substituting a uniform draw. The legacy
warn-and-substitute behaviour remains available via `validate = FALSE`.

- A `censored_by` target may not itself have `censored_by`. Chained censoring
would report some events as observed after observation ended, so it is
rejected in this version with a message naming the fix.

## Bug fixes

- The simstudy backend now returns a typed zero-row data frame for `n = 0`,
identical to the native backend's output, instead of failing inside
`simstudy::genData()` (whose `1:n` id table has two rows when `n = 0`).
(#50)
- The v0.4 pipeline now expands range-notation missing codes such as
`[997,999]` (the usual CCHS and CHMS pattern for don't know, refusal and
not stated) into their individual codes, splitting the row's proportion
equally. Previously a continuous variable with such a code failed in
post-processing, and a categorical variable silently wrote the literal
string `"[997,999]"` into the data. A bracketed missing code that is not
an integer range now fails with a message naming the variable. (#58)
- The packaged minimal example corrects two metadata errors: BMI's low
garbage range had a stray parenthesis (`[-10;15])`), and height's
low-garbage proportion was 1.00 (every row) instead of 0.01; height's
high-garbage range had an infinite upper bound, `(2.1;inf]`, which
produced `NaN` values, and is now `(2.1;2.5]` (#60). With these, #58 and
the survival dates (#40), every variable in the example generates through
the v0.4 pipeline. Seeded output for the example changes.

## Deprecations

- `create_wide_survival_data()` is deprecated and warns once per session.

## Known issues

- Known issues found while porting, reproduced unchanged: #54 (Gompertz
follow-up times are degenerate with the packaged parameters), #55
(administrative censoring is drawn per person), #56 (smaller legacy
discrepancies).

# MockData 0.4.0 (2026-06-10)

## Breaking changes
Expand Down Expand Up @@ -299,8 +439,8 @@ vars_with_garbage <- variables %>%

## Known issues

- Survival variable type must be generated manually with `create_wide_survival_data()`
- Cannot be used in `create_mock_data()` batch generation (requires paired variables)
- Survival variable type must be generated manually with `create_wide_survival_data()` (resolved in 0.5.0, #40)
- Cannot be used in `create_mock_data()` batch generation (requires paired variables) (resolved in 0.5.0, #40)

---

Expand Down
Loading
Loading