Skip to content

Fully incremental calculate_icers() with NMB and an incremental plot, plus calculate_cers_ref() for comparisons with a reference strategy (#209, #200) - #227

Open
feralaes wants to merge 24 commits into
mainfrom
209-reference-strategy-core
Open

feralaes wants to merge 24 commits into
mainfrom
209-reference-strategy-core

Conversation

@feralaes

@feralaes feralaes commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

This PR delivers the reference-strategy work (#209), as redesigned at the DARTH meeting on Nov 17 and implemented in #261.

What changed from the original design

  • calculate_icers() stays fully incremental. The ref_str options this PR first added to calculate_icers(), print.icers(), plot.icers(), calculate_icers_psa() and summary.psa() have been removed.
  • Comparisons with a reference strategy are now done by a separate function, calculate_cers_ref().

The rest of the description reflects the current branch.

This branch also contains these PRs, which were merged into it: #228 (plots), #229 (PSA summaries, since reworked), #242 (currency), #250 (log axes), #251 (API harmonization) and #261 (redesign).

What it adds

Fully incremental analysis (unchanged frontier)

  • calculate_icers() and calculate_icers_psa():
    • gain wtp, which adds each strategy's net monetary benefit (NMB), and nhb = TRUE, which also adds the net health benefit (NHB);
    • calculate_icers() also gains currency.
    • Without these arguments, the output is the same as before.
  • print.icers() uses the stored currency.
  • plot.icers():
    • incremental = TRUE puts the first non-dominated strategy (the least costly strategy on the frontier) at the origin. This was Eline's original request.
    • wtp draws a willingness-to-pay line through that strategy.

Comparison with a reference strategy (new)

  • calculate_cers_ref(cost, effect, strategies, ref_str, wtp = NULL) compares every strategy with one reference. It gives:
    • Inc_Cost and Inc_Effect;
    • CER, the cost-effectiveness ratio versus the reference;
    • Status: Reference / Dominant / Dominated / More costly, more effective / Less costly, less effective / Equivalent;
    • with wtp, NMB and Inc_NMB.
  • The reference is listed first, then the other strategies by increasing cost.
  • calculate_cers_ref_psa(psa, ref_str, wtp, uncertainty = TRUE) adds 95% intervals and the share of PSA samples in each quadrant, plus the share that is cost-effective versus the reference.
  • print() and plot() methods. The plot is the "star" around the reference, with an optional WTP line.
  • The help pages and a new section of vignette("basic_cea") cite the NICE, CDA-AMC and Zorginstituut Nederland guidelines. All three require a fully incremental analysis; NICE allows pairwise comparisons "when relevant and justified".

PSA plots

Currency and API

Closes

Checks

The examples in #252 are updated to the new design once this branch has #261.

calculate_icers() gains ref_str, wtp, nhb, and currency (added after the
existing arguments, so positional calls keep working). The frontier
(Status, Inc_Cost, Inc_Effect, ICER) is computed exactly as before and is
never changed; the new columns are added afterwards by
add_reference_and_nmb(), and only for the arguments that are given, so
with the defaults the object is identical to before.

- ref_str: Inc_Cost_ref, Inc_Effect_ref, ICER_ref (numeric, NA where a
  ratio is not meaningful), and Status_ref ('Reference', 'Dominant',
  'Dominated', 'More costly, more effective', 'Less costly, less
  effective', or 'Equivalent' for the same cost and effect) (#209).
- wtp: NMB, plus iNMB_ref with a reference (#200); nhb = TRUE also adds
  NHB and iNHB_ref.
- currency: stored on the object and used by print.icers().
- print.icers() gains ref = c('sequential', 'reference', 'both'); the
  default prints the same table as before. Headers name the reference
  ('Incremental Costs vs Standard of care ($)').
- Validation helpers check_ref_str() and check_wtp_nhb() with clear
  errors; a new example in ?calculate_icers.

Tests (test_icers_reference.R) check hand-calculated values for every
Status_ref category, a dominated and a most-expensive reference,
hund_strat, NMB/NHB, one and two strategies, names with spaces, the
errors, and the print views. The default printed tables are identical to
main's print.icers().

Part of #209. Closes #200
…lines

- plot.icers() gains ref_str, wtp, ref_axis, and ref_caption (after the
  existing arguments). ref_str shifts the plot so the reference strategy
  is at the origin (the frontier and statuses are unchanged); wtp draws a
  dotted line with slope wtp through the reference, or without ref_str
  through the least costly frontier strategy. The currency defaults to
  the one stored by calculate_icers() (#209, #202).
- plot.psa() gains wtp, which draws the same line on its incremental
  plane (requires ref_str) (#202).
- plot.psa(ref_str = ) with two strategies: keep a data frame when one
  comparator is left (drop = FALSE); it failed with "'dims' cannot be of
  length 0". ref_str is now checked with check_ref_str().
- Captions name the reference and the WTP threshold; this also removes a
  double space in plot.psa()'s 'Reference Strategy:  X' caption.
- Tests (test_reference_plots.R) check the plotted coordinates, axis
  labels, captions, WTP line slope and intercept, currency, the
  two-strategy case, and the errors. Default plots are unchanged.

Part of #209
…_icers_psa()

- summary.psa() gains ref_str and wtp (after calc_sds). With ref_str it
  adds meanIncCost, meanIncEffect, and the share of PSA samples in each
  quadrant of the incremental CE plane (propNE, propSE, propSW, propNW;
  NA for the reference). With wtp it adds meanNMB, and with ref_str also
  meanIncNMB and propCostEffective (share of samples with a higher NMB
  than the reference) (#209, #202).
- calculate_icers_psa() gains ref_str, wtp, and nhb, passed to
  calculate_icers(). With uncertainty = TRUE and ref_str, it adds 95%
  intervals of the sample-by-sample differences from the reference
  (Lower_95_/Upper_95_ for Inc_Cost_ref, Inc_Effect_ref, iNMB_ref). The
  icers class and attributes are now kept through the uncertainty joins.
- print.icers(): every *_ref column belongs to the reference view.
- Docs: escape % in roxygen. An unescaped % starts an Rd comment, which
  had silently truncated ?calculate_icers_psa ('whether or not 95') and
  ?predict.metamodel.

calculate_icers_psa() output for existing calls is identical to main's
(objects, class, and printed tables, with and without uncertainty).

Part of #209. Closes #202
Incremental CE plane and WTP lines in plot.icers() and plot.psa() (#209, #202)
Reference strategy and quadrant shares in summary.psa() and calculate_icers_psa() (#209, #202)
@feralaes
feralaes requested review from krijkamp and mbkauf November 12, 2026 22:59
…fix entry first in Bug fixes so the open PRs merge in any order
- ceac(), calc_exp_loss(), calc_evpi(), calc_evppi(), calc_evsi(), and
  calculate_icers_psa() store psa$currency on their results.
- Their plot() methods (and print.icers()) use it: currency defaults to
  NULL, resolved by resolve_currency() as the argument if given, else the
  result's currency, else '$'. Existing calls keep working.
- plot.ceac() gains effect_units (the x-axis label hard-coded QALY).
- EVPPI and EVSI computed from net health benefit are in health units:
  calc_evppi() and calc_evsi() store the outcome, and their plots are
  labeled 'EVPPI (QALY)' / 'EVSI (QALY)' instead of a currency.
  calc_evsi() defaults to outcome = 'nhb', so its default plot was
  mislabeled in dollars.
- Tests (test_currency.R) with a euro PSA for every function, overrides,
  effect_units, and dollar defaults; two existing tests now build the
  expected object with the same currency.

Outputs of all vignettes and examples change only by the new currency
attribute; values, plot data, and labels are unchanged.

Closes #203
Carry the currency of a psa object through to its analyses (#203)

@mbkauf mbkauf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

icers.R

  1. Printing with reference strategy has the strategies in the same order as without a reference strategy. So, the example code in the roxygen has the reference strategy is "1", but it ends up being the 55th strategy printed in the table. It should reorder with the reference strategy at the top? Or at least with some other ordering with respect to the reference strategy.
  2. In calculate_icers() lines 117 to 124, there is a chunk of code for if only 1 strategy is provided. It returns a data frame, but should it just return an error that more than 1 strategy must be provided?
  3. Are lines 103, 369-370 necessary?
  4. General naming conventions as it relates to capitalization. Should everything be lower case unless it's for formatting purposes to display.
  5. Line 406, what is everything() doing? Is it necessary?
  6. Add comments to explain code in add_reference_and_nmb() and add_reference_intervals() functions. In general, more commenting with newly written code would be good.
  7. Do we want incremental net monetary and incremental net health benefit to have the lower case "i" in the name (e.g. iNMB and iNHB)?

psa.R

  1. Lines 219-222, why does having a WTP threshold require a reference strategy be supplied?

…ng, wtp without ref_str in plot.psa

- print.icers(ref = "reference") lists the reference first and the other
  strategies by increasing cost; "sequential" and "both" keep the frontier order.
- Rename iNMB_ref/iNHB_ref to Inc_NMB_ref/Inc_NHB_ref, matching Inc_Cost_ref,
  Inc_Effect_ref and summary.psa()'s meanIncNMB. Print headers say "Incremental NMB/NHB".
- plot.psa(wtp = ) no longer requires ref_str: without one, the line goes through the
  mean of the least costly strategy, as plot.icers() already did.
- Comment add_reference_and_nmb(), add_reference_intervals(), the global-variable
  declarations, and the everything() in calculate_icers_psa().
@feralaes

Copy link
Copy Markdown
Collaborator Author

@mbkauf thanks for the careful review. Point by point; the changes are in 308850f.

icers.R

  1. Reference strategy at the top. Agreed. print(x, ref = "reference") now lists the reference first and the other strategies by increasing cost. Only the printed table is reordered, not the object. ref = "sequential" and ref = "both" keep the frontier order, because the sequential ICERs are read down that order. There's a new test using the roxygen example (reference "1", which was row 55 of 100).
  2. One strategy. Kept on purpose. Returning a valid icers object for a single strategy is the fix for Return a valid icers object from single-strategy calculate_icers #183, so code that filters down to one strategy (a subgroup, or a single-strategy PSA) can still print and plot the result instead of failing. A test covers it. If the group would rather have an error, that reverses Return a valid icers object from single-strategy calculate_icers #183, so I'd raise it there.
  3. Lines 103 and 369–370. These are needed. They declare the column names used unquoted in the dplyr code; without them R CMD check reports "no visible binding for global variable". I checked that every name in both lists is used that way. I added a comment saying so. Moving them into one utils::globalVariables() call would be a reasonable cleanup later.
  4. Capitalization. Internal variables follow the DARTH lowercase prefixes. The column names users see (Strategy, Cost, Inc_Cost, ICER, Status, meanCost) were all released in 1.0, so renaming them would break users' scripts. The new columns follow the existing style of each object. I'd put lowercase column names on the list for the 2.0 naming decision rather than change them in 1.1.0.
  5. everything() on line 406. Needed. It keeps the reference and net-benefit columns after the listed ones; select() drops them otherwise (I checked). There's now a comment saying that.
  6. Comments. Added throughout add_reference_and_nmb() and add_reference_intervals().
  7. iNMB / iNHB. Agreed. They're now Inc_NMB_ref and Inc_NHB_ref, which matches Inc_Cost_ref, Inc_Effect_ref, and meanIncNMB in summary.psa(). The printed headers say "Incremental NMB" and "Incremental NHB". Nothing has been released, so the rename costs nothing.

psa.R

  1. WTP requiring a reference strategy. You were right, and it was also inconsistent: plot.icers(wtp = ) already worked without ref_str, drawing the line through the least costly strategy on the frontier. plot.psa() now does the same. Without ref_str the line goes through the mean cost and effect of the least costly strategy, and the docs and NEWS say so. With ref_str it still goes through the origin of the incremental plane. Test added.

Tests: 343 passing, 0 lints, R CMD check --as-cran with only the development-version NOTE. Apart from the renamed column, example and vignette output is unchanged.

…rtainty; drop context() for testthat 3e

- plot.icers() now always reduces x to its seven plot columns as a plain data frame.
  dplyr::select() had dropped the stored currency attribute only when uncertainty
  bounds were present, so the two plots differed under testthat's edition-3 comparison.
- Replace context() in the four test files added by this PR (deprecated in edition 3).
@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.25073% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.92%. Comparing base (a1f3d64) to head (070f5b3).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
R/cers_ref.R 97.64% 4 Missing ⚠️
R/icers.R 98.76% 1 Missing ⚠️
R/psa.R 97.77% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #227      +/-   ##
==========================================
+ Coverage   93.14%   93.92%   +0.77%     
==========================================
  Files          18       19       +1     
  Lines        1794     2073     +279     
==========================================
+ Hits         1671     1947     +276     
- Misses        123      126       +3     
Files with missing lines Coverage Δ
R/add_common_aes.R 90.57% <100.00%> (+0.35%) ⬆️
R/ceac.R 91.53% <100.00%> (+0.55%) ⬆️
R/evpi.R 96.15% <100.00%> (+0.32%) ⬆️
R/evppi.R 95.45% <100.00%> (+0.58%) ⬆️
R/evsi.R 87.50% <100.00%> (+0.50%) ⬆️
R/exp_loss.R 100.00% <100.00%> (ø)
R/metamodel.R 93.26% <ø> (ø)
R/owsa.R 93.02% <100.00%> (+0.13%) ⬆️
R/twsa.R 93.44% <100.00%> (+0.10%) ⬆️
R/icers.R 97.00% <98.76%> (+0.25%) ⬆️
... and 2 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…scale (#201)

The ellipses were computed on the data scale, so on log axes (added with scale_*_log10())
they lost their shape and any part below zero became NaN. plot.psa() now takes xtrans and
ytrans, passes them to its own scales, and computes each ellipse from the transformed
samples before mapping it back. Values a transformation cannot show (zero or negative on a
log scale, common on the incremental plane) give an informative error. With the default
identity transformations the plots are unchanged. The duplicated ellipse code of the two
planes is now one helper, psa_ellipses().
…idth, axis limits, effect_units

- ceac() takes the psa object first, like calc_exp_loss() and calc_evpi(). The old positional
  ceac(wtp, psa) still works with lifecycle::deprecate_soft(); named calls work in any order.
- plot.owsa() and plot.exp_loss() take linewidth; size and lsize still work, deprecated.
- plot.owsa() and plot.twsa() gain xlim/ylim, owsa_opt_strat() gains xlim.
- effect_units defaults to "QALYs" in every plot; per_unit() gives the singular for
  willingness-to-pay labels, so default labels are unchanged ("$/QALY").
- lifecycle (>= 1.0.0) added to Imports.
- API style decision record in .github/CONTRIBUTING.md, including the open 2.0 questions.
feralaes and others added 5 commits November 13, 2026 17:37
plot.psa(): log axes with correctly shaped ellipses (xtrans, ytrans) (#201)
The minimum-dependency CI job could not install ggplot2@3.5.0 with lifecycle@1.0.0.
Current dplyr needs lifecycle (>= 1.0.5) and ggplot2 3.5.0 needs lifecycle (> 1.0.1), so
any lower pin makes the minimum-dependency environment unsolvable. dampack only uses
deprecate_soft(), deprecated() and is_present(), which predate lifecycle 1.0.
Harmonize the API with soft deprecation: ceac(psa, wtp), linewidth, axis limits, effect_units (#214)
feralaes added a commit that referenced this pull request Nov 14, 2026
Resolve R/ceac.R: keep #251's argument order (psa, then wtp) and this PR's @concept in
place of the multi-word @Keywords. Examples now show the documented ceac(psa, wtp) order.
@feralaes

Copy link
Copy Markdown
Collaborator Author

Hi @mbkauf and @krijkamp, Fernando asked me (Claude, working in his session) to send a short guide for reviewing this PR. It's the one most other work is waiting on: three more PRs are built on top of it. GitHub won't let Fernando approve his own PR, so it needs an approval from one of you.

You don't need to read all the code. The most useful review is to run it and check that the results and wording make sense.

1. Try it (about 5 minutes)

In R:

remotes::install_github("DARTH-git/dampack", ref = "209-reference-strategy-core")
library(dampack)
data(cancer_psa_data)
psa <- make_psa_obj(cost = cancer_psa_data$cost,
                    effectiveness = cancer_psa_data$effectiveness,
                    parameters = cancer_psa_data$parameters,
                    strategies = cancer_psa_data$strategies)

icers <- calculate_icers_psa(psa, ref_str = "Radio", wtp = 100000)
print(icers)                    # the usual sequential table (unchanged)
print(icers, ref = "reference") # every strategy vs Radio, Radio listed first
plot(icers, ref_str = "Radio", wtp = 100000)

summary(psa, ref_str = "Radio", wtp = 100000)  # mean differences and quadrant shares
plot(psa, ref_str = "Radio", wtp = 100000)     # incremental CE plane with a WTP line

2. What to look at

Matt:

  • I answered your 8 points in this reply. Five are changed in the code and two are explained. The wtp point turned out to be right, so it's changed too.
  • Please check that you're happy with how each was handled.
  • Then check the plots. You built the original plot.psa(ref_str).

Eline: the methods, as agreed on #209:

  • ICER_ref is NA for the reference and for dominant, dominated and equivalent strategies.
  • Status_ref uses these labels: Reference, Dominant, Dominated, "More costly, more effective", "Less costly, less effective", Equivalent.
  • NMB and Inc_NMB_ref appear only when wtp is given. NHB appears only with nhb = TRUE.
  • In summary.psa(), propNE/propSE/propSW/propNW are the shares of PSA samples in each quadrant of the incremental plane, and propCostEffective is the share with a positive incremental NMB.
  • If anything looks wrong to you as a health economist, that's exactly what we need to hear.

Both:

  • The sequential ICERs (print(icers) without ref =) should look exactly as before. The new features are only added on top.

3. How to leave your review on GitHub

  1. Click the Files changed tab at the top of this PR.
  2. (Optional) To comment on a specific line, hover over it, click the blue +, type your comment and click Start a review.
  3. Click the green Review changes button at the top right, and choose:
    • Approve if it's fine, or fine with small comments;
    • Request changes if something must be fixed before merging;
    • Comment for questions only.
  4. Click Submit review.

A plain comment at the bottom of this page also works. Fernando or I will turn it into changes.

feralaes and others added 2 commits November 17, 2026 15:28
…r comparisons with a reference

As agreed at the DARTH meeting of Nov 17, 2026:
- Remove ref_str from calculate_icers(), print.icers(), plot.icers(),
  calculate_icers_psa() and summary.psa(). They keep wtp/nhb (NMB, NHB)
  and currency.
- plot.icers(incremental = TRUE) puts the first non-dominated strategy
  at the origin.
- New calculate_cers_ref() and calculate_cers_ref_psa(), with print()
  and plot() methods, compare every strategy with a reference strategy,
  citing the NICE, CDA-AMC and Zorginstituut Nederland guidelines.
- basic_cea vignette section, README bullet, pkgdown reference group.

Co-Authored-By: Claude <noreply@anthropic.com>
feralaes and others added 2 commits November 18, 2026 12:39
Co-Authored-By: Claude <noreply@anthropic.com>
Keep calculate_icers() fully incremental; add calculate_cers_ref() for comparisons with a reference strategy (#209)
@feralaes feralaes changed the title Compare strategies with a reference in calculate_icers() and print.icers() (#209, #200) Fully incremental calculate_icers() with NMB and an incremental plot, plus calculate_cers_ref() for comparisons with a reference strategy (#209, #200) Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants