Skip to content

Send OASIS fields before writing history & restart files - #162

Merged
kvrigor merged 7 commits into
masterfrom
dev-refactor-oasis-sendrecv
Nov 10, 2026
Merged

kvrigor merged 7 commits into
masterfrom
dev-refactor-oasis-sendrecv

Conversation

@kvrigor

@kvrigor kvrigor commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Originally, the coupled fields had to wait for eCLM disk writes to finish before being sent to ICON and/or Parflow. Since there's no good reason to do this, it's safe to change the order from write -> oas_send to oas_send -> write:

                             Called    Wallclock        max        min 
/** truncated ouput **/
"CPL:LND_RUN"                   288  1428.060669  15.283545   3.361196  <-- Receive ICON fields
  "l:oas_receive_icon"          289  1095.338989   7.860734   0.001540 
  "l:lc_lnd_import"             289     0.100270   0.000551   0.000301 
  "l:oas_receive_parflow"       289   224.599106   5.027600   0.000266  <-- Receive Parflow fields
  "l:clm_run"                   289   107.667923  11.980367   0.080552 
    "l:shr_orb_decl"            289     0.003025   0.000038   0.000004 
    /** truncated ouput **/
    "l:lnd2atm"                 289     0.625278   0.002752   0.001908 
    "l:oas_send_icon"           289     0.217915   0.001377   0.000457  --> Send eCLM fields to ICON
    "l:oas_send_parflow"        289    13.964260   0.109532   0.004639  --> Send eCLM fields to Parflow
    "l:lnd2glc"                 289     0.053692   0.000249   0.000052 
    "l:wrtdiag"                 289     0.075011   0.000344   0.000094 
    "l:hbuf"                    289     0.461194   0.001816   0.001096 
    "l:clm_drv_io"              289    22.921955  11.824379   0.000003  ### Write eCLM history & restart files
    "l:accum"                   288     0.422128   0.001984   0.001154 
    "l:interpMonthlyVeg"        287     1.566301   1.565523   0.000001 
/** truncated ouput **/

I've manually verified the fixes by running a 3 sim. day, fully-coupled EUR-11 simulation. Simulations completed without issues and took around ~25 minutes. No time savings could be observed, but it remains to be seen how the oas_send -> write order affects the runtime of much longer simulations. For now it's sufficient to simply fix an obvious algorithmic mistake and make sure it does not break existing functionality.

For those interested in the results, see attached LLview reports and the simulation runs archived at /p/data1/slts/rigor1/verification/eCLM_PR162_dev-refactor-oasis-sendrecv

llview_results_after-refactor.pdf
llview_results_before-refactor.pdf

@kvrigor kvrigor linked an issue Nov 4, 2026 that may be closed by this pull request
@kvrigor
kvrigor marked this pull request as ready for review November 6, 2026 08:07
s-poll
s-poll previously approved these changes Nov 8, 2026

@s-poll s-poll left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Paul. Nice PR. The send now happens right after lnd2atm, before history and restart output and I confirmed bit-identity.

Two small requests:

  1. Could you please put the lines, marked in this review, in ifdef statements.
  2. Could you repeat exactly the same simulations as in the ICON-GPU-eCLM scaling study, with and without this PR to quantify the runtime benefit? On JURECA with the default setup there was a runtime benefit of 2%, but there is a larger benefit expected on heterogeneous setups.

Comment thread src/clm5/main/clm_driver.F90 Resolved
Comment thread src/clm5/main/clm_driver.F90 Resolved
@kvrigor

kvrigor commented Nov 9, 2026 •

Copy link
Copy Markdown
Member Author

Thanks for looking into this @s-poll . Re on the ICON-GPU-eCLM scaling study: I'd prefer to do it after this PR is merged since such benchmarking exercises are costly in terms of time and compute resources. This PR also doesn't claim there would be performance improvements, although one could speculate. More importantly, as mentioned in my first post, this PR tries to simply fix an obvious algorithmic mistake and not introduce any regressions.

@kvrigor
kvrigor requested a review from s-poll November 9, 2026 12:40
@kvrigor
kvrigor merged commit 2e36914 into master Nov 10, 2026
6 checks passed
@kvrigor
kvrigor deleted the dev-refactor-oasis-sendrecv branch November 10, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OASIS send/receive calls blocked by eCLM history file saving

2 participants