Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: MockData
Title: Generate Mock Data from Metadata Specifications
Version: 0.4.0
Version: 0.5.0
Authors@R: c(
person("Juan", "Li", role = "aut", email = "juli@ohri.ca"),
person("Douglas", "Manuel", role = c("aut", "cre"), email = "dmanuel@ohri.ca"),
Expand Down
136 changes: 76 additions & 60 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,55 @@
# MockData (development version)
# MockData 0.5.0 (2026-09-27)

## Formula-derived variables (#39, Phase A)
## Breaking changes

- Variables can now be generated from algebraic expressions over other
generated variables via a new `mockFormula` column in `variable_details`
(e.g. `weight / (height^2)`), or the new `mock_formula()` direct API.
Formulas are evaluated in dependency order by `evaluate_mock_formulas()`,
in a restricted environment exposing only the generated columns and a
fixed allow-list of base functions. `DerivedVar::`/`Func::` semantics are
unchanged: derived variables without a `mockFormula` remain excluded from
generation, and `Func::` dispatch is not yet supported.
- Seeded output changed once for the RNG-stream mechanism in this release.
Within the `create_mock_data()` pipeline (`generate_mock_data_native()`,
`generate_mock_data_simstudy()`, `postprocess_mock_data()`), MockData now
derives all randomness from independent L'Ecuyer-CMRG sub-streams (one per
generation stage) seeded from the single public `seed`, replacing the
previous Mersenne-Twister `seed` / `seed + 1` scheme. The standalone
`create_*` helpers (`create_cat_var()`, `create_con_var()`,
`create_date_var()`, `create_survival_dates()`,
`create_wide_survival_data()`, `sample_with_proportions()`,
`make_garbage()`, `apply_garbage()`) called directly are unaffected and
still seed a single Mersenne-Twister stream per call. For a given seed
**and package version** pipeline output is reproducible and independent of
the session's ambient `RNGkind()`; it is not comparable across the v0.4 →
v0.5 boundary. For seeded calls, generation no longer alters the caller's
RNG state (previously the legacy `validate = FALSE` path reset it). (#38)

## Survival dates in create_mock_data() (#40)
**Migrating:** if your tests pin values from seeded v0.4 runs of
`create_mock_data()` or the `mock_spec` pipeline, regenerate those expected
values once under v0.5 and pin them again. The same seed keeps giving
identical output within this version. Standalone `create_*` calls need no
change.

- Migration: add `anchor` (and `censored_by` where a competing risk applies)
to each survival date's row. A date with `followup_min`, `followup_max` or
`event_prop` but no `anchor` now fails with a message naming the fix;
previously it was silently dropped or generated as a plain calendar date.
- Missing codes and garbage are applied to survival dates after the survival
rules. Garbage that places a date before entry is therefore kept, whereas
the legacy engine set such dates to `NA`.
- The packaged minimal example now generates entirely through the v0.4
pipeline, because its Gompertz survival dates no longer force the legacy
generator. Its seeded output differs from v0.4.
- If the legacy generator is selected (`validate = FALSE`,
`variable_details = NULL`, detail-level-only `databaseStart`, or another
unsupported variable) on metadata that sets `anchor`, `create_mock_data()`
stops and names the survival variables rather than dropping them.

## New features

- Formula-derived variables (#39, Phase A). Variables can now be generated
from algebraic expressions over other generated variables via a new
`mockFormula` column in `variable_details` (e.g. `weight / (height^2)`),
or the new `mock_formula()` direct API. Formulas are evaluated in
dependency order by `evaluate_mock_formulas()`, in a restricted environment
exposing only the generated columns and a fixed allow-list of base
functions. `DerivedVar::`/`Func::` semantics are unchanged: derived
variables without a `mockFormula` remain excluded from generation, and
`Func::` dispatch is not yet supported.
- `create_mock_data()` now generates survival dates from metadata. A date
whose `variables.csv` row sets `anchor` (its entry-date variable) is a
survival date: `floor(n * event_prop)` rows receive a date between
Expand All @@ -24,60 +61,16 @@
functions: `mock_spec_survival()` and `generate_survival_dates()`. This
resolves the known issue, listed since v0.2.0, that survival data had to be
generated separately.
- Migration: add `anchor` (and `censored_by` where a competing risk applies)
to each survival date's row. A date with `followup_min`, `followup_max` or
`event_prop` but no `anchor` now fails with a message naming the fix;
previously it was silently dropped or generated as a plain calendar date.
- Missing codes and garbage are applied to survival dates after the survival
rules. Garbage that places a date before entry is therefore kept, whereas
the legacy engine set such dates to `NA`.
- Derived columns, from `mockFormula` or survival dates, describe the clean
generated values; missing codes and garbage are then applied to each column
independently. `is.na` is added to the `mockFormula` allow-list so status
and follow-up time can be derived. Derive both from the same observation
window: `as.integer(!is.na(death_date))` only says a death date exists,
not that the death was observed before censoring.
- A `censored_by` target may not itself have `censored_by`. Chained censoring
would report some events as observed after observation ended, so it is
rejected in this version with a message naming the fix.
- The packaged minimal example now generates entirely through the v0.4
pipeline, because its Gompertz survival dates no longer force the legacy
generator. Its seeded output differs from v0.4.
- If the legacy generator is selected (`validate = FALSE`,
`variable_details = NULL`, detail-level-only `databaseStart`, or another
unsupported variable) on metadata that sets `anchor`, `create_mock_data()`
stops and names the survival variables rather than dropping them.
- `create_wide_survival_data()` is deprecated and warns once per session.
- Known issues found while porting, reproduced unchanged: #54 (Gompertz
follow-up times are degenerate with the packaged parameters), #55
(administrative censoring is drawn per person), #56 (smaller legacy
discrepancies).

## Reproducibility (breaking change)

- Seeded output changed once for the RNG-stream mechanism in this release.
Within the `create_mock_data()` pipeline (`generate_mock_data_native()`,
`generate_mock_data_simstudy()`, `postprocess_mock_data()`), MockData now
derives all randomness from independent L'Ecuyer-CMRG sub-streams (one per
generation stage) seeded from the single public `seed`, replacing the
previous Mersenne-Twister `seed` / `seed + 1` scheme. The standalone
`create_*` helpers (`create_cat_var()`, `create_con_var()`,
`create_date_var()`, `create_survival_dates()`,
`create_wide_survival_data()`, `sample_with_proportions()`,
`make_garbage()`, `apply_garbage()`) called directly are unaffected and
still seed a single Mersenne-Twister stream per call. For a given seed
**and package version** pipeline output is reproducible and independent of
the session's ambient `RNGkind()`; it is not comparable across the v0.4 →
v0.5 boundary. Pin your own expected values against the version you use.
For seeded calls, generation no longer alters the caller's RNG state
(previously the legacy `validate = FALSE` path reset it).

- `create_mock_data()` now accepts `n = 0`, returning a zero-row data frame
with the full generated schema (useful for schema tests), and rejects
fractional, negative, `NA`, and non-finite values of `n` with a clear
message. The validation now matches `generate_mock_data_native()`.
- Calling `create_mock_data()` without `databaseStart` now fails upfront with
a message naming the argument, instead of a raw missing-argument error.
- The native backend now supports `distribution = "exponential"` (parity with
the legacy generator), removing a forced legacy-fallback for exponential
metadata. When the optional simstudy backend is selected, exponential
Expand All @@ -86,6 +79,23 @@
their generation. Native exponential values are truncated to the declared
`range` by inverse-CDF sampling, rather than clipped at the range maximum
(with a point mass at the boundary) as the legacy `rexp()` path does.

## Validation and error messages

- Calling `create_mock_data()` without `databaseStart` now fails upfront with
a message naming the argument, instead of a raw missing-argument error.
- With `validate = TRUE` (the default), invalid distribution parameters in
metadata — e.g. `distribution = "exponential"` without a positive `rate` —
now stop generation with a message naming the variable and how to fix it,
instead of warning and substituting a uniform draw. The legacy
warn-and-substitute behaviour remains available via `validate = FALSE`.

- A `censored_by` target may not itself have `censored_by`. Chained censoring
would report some events as observed after observation ended, so it is
rejected in this version with a message naming the fix.

## Bug fixes

- The simstudy backend now returns a typed zero-row data frame for `n = 0`,
identical to the native backend's output, instead of failing inside
`simstudy::genData()` (whose `1:n` id table has two rows when `n = 0`).
Expand All @@ -103,11 +113,17 @@
and #58, every variable in the example except its survival dates now
generates through the v0.4 pipeline. Seeded output for the example
changes.
- With `validate = TRUE` (the default), invalid distribution parameters in
metadata — e.g. `distribution = "exponential"` without a positive `rate` —
now stop generation with a message naming the variable and how to fix it,
instead of warning and substituting a uniform draw. The legacy
warn-and-substitute behaviour remains available via `validate = FALSE`.

## Deprecations

- `create_wide_survival_data()` is deprecated and warns once per session.

## Known issues

- Known issues found while porting, reproduced unchanged: #54 (Gompertz
follow-up times are degenerate with the packaged parameters), #55
(administrative censoring is drawn per person), #56 (smaller legacy
discrepancies).

# MockData 0.4.0 (2026-06-10)

Expand Down
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,34 +3,36 @@
<!-- badges: start -->

[![Lifecycle: experimental](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://lifecycle.r-lib.org/articles/stages.html#experimental)
[![Version: 0.4.0](https://img.shields.io/badge/version-0.4.0-blue.svg)](https://github.com/Big-Life-Lab/MockData)
[![Version: 0.5.0](https://img.shields.io/badge/version-0.5.0-blue.svg)](https://github.com/Big-Life-Lab/MockData)
[![pkgdown](https://github.com/Big-Life-Lab/MockData/actions/workflows/pkgdown.yaml/badge.svg)](https://github.com/Big-Life-Lab/MockData/actions/workflows/pkgdown.yaml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

<!-- badges: end -->

**Status: Experimental v0.4.0 release**
**Status: Experimental v0.5.0 release**

MockData is a work-in-progress R package for generating mock testing data from
small metadata specifications. Version 0.4 introduces the
small metadata specifications. Version 0.4 introduced the
`mock_spec` architecture: direct specification helpers, a recodeflow metadata
adapter, native generation, optional `simstudy` generation, and post-processing
diagnostics. It is useful today for development and documentation workflows,
diagnostics. Version 0.5 adds formula-derived variables (`mockFormula`),
survival dates generated from metadata (`anchor`, `censored_by`), and a
per-stage seed contract. It is useful today for development and documentation workflows,
especially when paired with recodeflow-style metadata (see below), but it should
be treated as experimental infrastructure rather than a stable released package.

People are using MockData and reporting that it is helpful. We take that as an
encouraging signal, not as evidence that the package is mature. Please review
the generated data before using it in any workflow that matters.

v0.4 keeps the existing v0.3 public functions available. Several
larger ideas were deliberately deferred rather than rushed into this release:
formula-derived variables, multi-variable correlation, Table 1 bootstrap,
LinkML/schema-first integration, and advanced survival simulation. MockData
v0.5 keeps the existing v0.3 public functions available. Several
larger ideas remain deliberately deferred rather than rushed into a release:
`Func::` dispatch for derived variables, multi-variable correlation, Table 1
bootstrap, LinkML/schema-first integration, and advanced survival simulation. MockData
also remains framed as mock data for development and QA, not synthetic data for
privacy release or inference.

Please try v0.4 with representative `variables.csv` and
Please try v0.5 with representative `variables.csv` and
`variable_details.csv` files and report bugs, surprising output, confusing
diagnostics, or migration issues in
[GitHub Issues](https://github.com/Big-Life-Lab/MockData/issues). Broader design
Expand All @@ -43,7 +45,7 @@ feedback is welcome in
- Use inline metadata for small examples, without creating separate CSV files
- Create categorical, continuous, and date-like fields from simple specifications
- Add intentional invalid or out-of-range "garbage" values for QA testing
- Generate basic time-to-event style mock dates for survival-analysis workflows
- Generate survival dates (entry, events, competing risks, censoring) from metadata
- Support examples and tutorials without exposing real person-level data

**Current development limitations:**
Expand Down
22 changes: 15 additions & 7 deletions cran-comments.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@
Checked with:

- R 4.4.2
- macOS 26.5, aarch64-apple-darwin20
- `R CMD check --as-cran --no-manual`
- macOS 27.0, aarch64-apple-darwin20
- `R CMD check --as-cran --no-manual` on the built tarball
- `_R_CHECK_FORCE_SUGGESTS_=false`, because the local check library does not
include the suggested package `simstudy`

Result:

Expand All @@ -17,17 +19,23 @@ Result:
Notes:

- New submission.
- `simstudy` was not installed in the local check environment, so the local
check used `_R_CHECK_FORCE_SUGGESTS_=false`. `simstudy` is listed in
`Suggests` and is used only for the optional backend.
- `simstudy` (suggested) not available for checking locally. It is installed
in continuous integration, where its tests run.
- The local check reported `unable to verify current time`.

## Continuous integration

The `R-CMD-check` workflow runs the same check on Ubuntu 24.04 with
R 4.6.1, with all suggested packages installed, for every push to
`main`/`dev` and every pull request into them. Latest result: `Status: OK`
(0 errors, 0 warnings, 0 notes).

## Release notes

This is the first CRAN submission of MockData. The package generates mock
testing data from metadata specifications supplied as data frames or CSV files.
It supports categorical, continuous, date, garbage-data, and survival-style
variables for testing data pipelines and documentation examples.
It supports categorical, continuous, date, garbage-data, formula-derived, and
survival-style variables for testing data pipelines and documentation examples.

MockData generates mock data for software testing and workflow validation. It
does not generate privacy-preserving synthetic data for inference or data
Expand Down
8 changes: 4 additions & 4 deletions development/post-v040-development-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -573,10 +573,10 @@ Tasks 5 and 6 **must land in the same minor release**: both change seeded output

### Task 8: v0.5 release assembly

- [ ] Tasks 5 + 6 merged together; one NEWS migration section covering the seeded-output break with before/after guidance.
- [ ] Pinned-value seed regression tests in place (#22 closed).
- [ ] `devtools::check()` clean; `roxygen2::roxygenize()` then `pkgdown::build_site()` clean; `renv::snapshot()` if dependencies moved.
- [ ] Version bump to 0.5.0; downstream heads-up to cchsflow/chmsflow maintainers if any metadata-extension columns were added.
- [x] Tasks 5 + 6 merged together; one NEWS migration section covering the seeded-output break with before/after guidance. (#38, #39 and #40 all ship in 0.5.0; NEWS "Breaking changes" carries the migrating paragraph.)
- [x] Pinned-value seed regression tests in place (#22 closed). (`tests/testthat/test-seed-contract.R`; #22 closed by the release PR.)
- [x] `devtools::check()` clean; `roxygen2::roxygenize()` then `pkgdown::build_site()` clean; `renv::snapshot()` if dependencies moved. (2026-09-27: `R CMD check --as-cran` on the 0.5.0 tarball, 0 errors, 0 warnings, 3 documented notes; `devtools::check()` is blocked by the local toolchain. pkgdown builds; the lockfile was updated in #51.)
- [x] Version bump to 0.5.0; downstream heads-up to cchsflow/chmsflow maintainers if any metadata-extension columns were added. (Maintainer decision 2026-09-22: the extension columns `mockFormula`, `anchor` and `censored_by` are documented in MockData, in `reference-config.qmd`'s placement rule, rather than in a separate notice. They are optional and change nothing for those packages.)

---

Expand Down
Loading