Skip to content

metatomic: run models through metatomic-core and its plugins - #1510

Draft
EricBoittier wants to merge 5 commits into
plumed:masterfrom
EricBoittier:EricBoittier/metatomic-capi-update
Draft

EricBoittier wants to merge 5 commits into
plumed:masterfrom
EricBoittier:EricBoittier/metatomic-capi-update

Conversation

@EricBoittier

@EricBoittier EricBoittier commented Oct 15, 2026 •

Copy link
Copy Markdown
Description

This moves the METATOMIC action from metatomic-torch (and libtorch) to metatomic-core, the C API of metatomic, which loads models through plugins:

  • TorchScript models exported with metatomic.torch.AtomisticModel are run by the plugin of metatomic-torch, given with PLUGIN_LIBRARY, without starting Python. That plugin is not merged upstream yet, so CI builds metatomic from the head of the pull request adding it, pinned in .ci/install.metatomic (currently e35ed338).
  • Models written as Python scripts (for example subclasses of metatomic.NumpyModel) are run by the Python plugin of metatomic-core, with PLUGIN=python.
  • Without PLUGIN_LIBRARY, PLUMED registers the plugins installed in the same prefix as its libmetatomic, such as the Python plugin of the metatomic Python package.

The module no longer links to libtorch.

Breaking change. metatomic only computes gradients of energies with automatic differentiation. A bias can only act on models that compute the gradients of their feature output with respect to positions and strain themselves. Models without these gradients still work as CVs, and PLUMED stops with an error when a bias acts on them. CHANGES/v2.11.md and the 2.11 migration guide explain how to export such models again.

SPECIES is validated before the atoms are requested: the entries have to cover every atom of the system exactly once, so an atom listed in two SPECIES entries, or an atom that no entry covers, is an error naming that atom. The types are stored by absolute atom index, so without this a gap left atoms with the type 0, and non-contiguous entries wrote past the end of the array. The existing check that the action knows about every atom in the system is kept for SPECIES that stops short of the system.

Tests
  • rt-basic: the TorchScript models compute their gradients explicitly. A model exported without gradients can be printed, and biasing it stops with the error above.
  • rt-core: a model written as a Python script.
  • rt-per-atom (new): per-atom outputs with all atoms, an unordered selection and a single atom, in a triclinic cell, with the model returning its samples in reverse order. The derivatives from --debug-forces are compared with finite differences.
  • rt-soap and rt-perovskite compute their CVs with featomic in Python scripts. featomic-torch 0.7 is built against metatomic-torch 0.1 and can not be loaded with metatomic-torch 0.2. With the projection matrix of the previous TorchScript model, the SOAP model reproduces the previous references exactly. The BaTiO3 model reproduces them as they are.
  • CI: metatomic-core is not on PyPI yet, so .ci/install.metatomic builds metatomic-core and metatomic-torch from a pinned commit. This is done in the -doc-mpi- and -coverage-mpi- Linux jobs, which run with PLUMED_ALL_TESTS=yes: they faskipped, the manual runs the METATOMIC examples, and the module is covered.The other Linux jobs only configure libtorch, for the pytorch module.

All five metatomic regtests pass locally on Linux x86_64 against metatomic including the finite-difference comparison in rt-per-atom.

Target release

I would like my code to appear in release 2.11

Type of contribution
  • changes to code or doc authored by PLUMED developers, or additions of code in the core or within the default modules
  • changes to a module not authored by you
  • new module contribution or edit of a module authored by you
Copyright
  • I agree to transfer the copyright of the code I have written to the PLUMED developers or to the author of the code I am modifying.
  • the module I added or modified contains a COPYRIGHT file with the correct license information. Code should be released under an open source license. I also used the command
    cd src && ./header.sh mymodulename in order to make sure the headers of t
Tests
  • I added a new regtest or modified an existing regtest to validate my
  • I verified that all regtests are passed successfully on [GitHub Actio/plumed2/actions).

@EricBoittier
EricBoittier marked this pull request as draft October 15, 2026 13:30
} // namespace metatomic
} // namespace PLMD

#endif
} // namespace metatomic
} // namespace PLMD

#endif
reinterpret_cast<const double (*)[3]>(&cell(0, 0)),
periodic.data(),
{vesin::VesinCPU, 0},
options,
@carlocamilloni
carlocamilloni marked this pull request as ready for review October 15, 2026 16:52
@carlocamilloni
carlocamilloni marked this pull request as draft October 15, 2026 16:53
Comment thread src/metatomic/MetatomicAction.cpp Fixed
Comment thread src/metatomic/MetatomicAction.cpp Fixed
@EricBoittier
EricBoittier force-pushed the EricBoittier/metatomic-capi-update branch 2 times, most recently from cc4d525 to ebd42f7 Compare October 26, 2026 10:36
@codecov

codecov Bot commented Oct 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.20379% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.21%. Comparing base (c2a5cd1) to head (ebd42f7).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
src/metatomic/metatomic_core.cpp 88.38% 33 Missing ⚠️
src/metatomic/MetatomicAction.cpp 84.78% 21 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1510      +/-   ##
==========================================
- Coverage   83.21%   83.21%   -0.01%     
==========================================
  Files         607      608       +1     
  Lines       76083    76110      +27     
==========================================
+ Hits        63312    63332      +20     
- Misses      12771    12778       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@EricBoittier
EricBoittier force-pushed the EricBoittier/metatomic-capi-update branch from f654919 to f45e347 Compare November 11, 2026 10:34
@EricBoittier EricBoittier changed the title Eric boittier/metatomic capi update metatomic: run models through metatomic-core and its plugins Nov 11, 2026
@EricBoittier
EricBoittier force-pushed the EricBoittier/metatomic-capi-update branch 9 times, most recently from 3611674 to 212e212 Compare November 16, 2026 14:26
EricBoittier and others added 5 commits November 19, 2026 07:17
METATOMIC now uses the C++ API of metatomic-core instead of linking to
libtorch through metatomic-torch. Models are loaded by metatomic plugins:
TorchScript models exported with metatomic.torch.AtomisticModel by the
plugin of metatomic-torch (given with PLUGIN_LIBRARY), and models written
as Python scripts by the Python plugin of metatomic-core (PLUGIN=python).
Without PLUGIN_LIBRARY, PLUMED registers the plugins installed in the same
prefix as its libmetatomic.

metatomic only computes gradients of energies with automatic
differentiation, so a bias can only act on models that compute the
gradients of their feature output explicitly. The TorchScript models of
rt-basic now do, and rt-basic checks that biasing a model without
gradients stops with a clear error. rt-core runs a model defined in a
Python script.

rt-soap and rt-perovskite compute their CVs with featomic in Python
scripts, since featomic-torch 0.7 is built against metatomic-torch 0.1
and can not be loaded with metatomic-torch 0.2. With the projection
matrix of the previous TorchScript model, the SOAP model reproduces the
previous references exactly, and the BaTiO3 model reproduces them as-is.
rt-per-atom runs a per-atom model with all atoms, an unordered selection
and a single atom, in a triclinic cell. The model returns its samples in
reverse order, and the bias gives each per-atom value a different
derivative, so wrong indexing changes the forces. The derivatives from
--debug-forces are compared to finite differences.
metatomic-core is not released on PyPI yet, so .ci/install.metatomic
builds metatomic-core and metatomic-torch from a pinned commit. The
commit is the head of metatensor/metatomic#379, which adds the
metatomic-torch plugin.

This build is only done in the -doc- and -coverage- Linux jobs, which run
with PLUMED_ALL_TESTS=yes: they fail when a metatomic regtest is skipped,
and the manual runs the examples of METATOMIC. The other Linux jobs no
longer install metatomic-torch and featomic-torch, and only configure
libtorch, for the pytorch module.
Nothing upstream bounds the sample indices a model puts in its output.
`check_consistency` validates the names and components of a gradient block
but not the values of its samples: for "positions" gradients the only check
is that the sample names are ('sample', 'system', 'atom') and that the
components are xyz. metatensor range-checks only the leading "sample"
column, when the gradient is added, and skips even that on meta tensors. So
the `atom` column, which indexes a buffer PLUMED sized, is whatever the
model wrote, and it has to be checked here or not at all.

- `derivatives[3 * atom + x]`: bound `atom` against the atom count.
- `forces[sample * n_properties + p]`: bound `sample` against the rows of
  this action's value. In the scalar case that buffer holds one element.
- Both gradient blocks: assert the sample names before reading a column by
  index. The value block's names were already asserted; the gradients' were
  not, and `positions_samples(g, 2)` assumes three columns.
- The per-atom output rows: an atom outside the system, or outside
  SELECTED_ATOMS, now names itself instead of failing a bare
  `plumed_assert`.

Also:

- Symmetrize the virial. A model which is not exactly rotationally
  equivariant returns a slightly non-symmetric strain gradient, and
  metatomic passes it through untouched. The antisymmetric part is not a
  virial and reached the pressure; i-PI's metatomic drivers already keep the
  symmetric part.
- An unsupported dtype is an error again. `float64_ = dtype == Float64` ran
  anything else as float32 without a word, where the released torch-based
  module errors.
- `defaultPlugins()` walks up for `libexec/metatomic` instead of assuming
  `<prefix>/lib/libmetatomic.so`. On a multiarch layout
  (`<prefix>/lib/x86_64-linux-gnu/`) the old two-level walk landed inside
  `lib`, found no plugins, and the user saw "failed to load the model"
  rather than "I looked in the wrong place".
- `apply()` no longer blames the model when PLUMED is the one that did not
  request gradients: `calculate()` only asks for them when
  `doNotCalculateDerivatives()` is false, so the message now covers both.
regtest/metatomic/rt-core-bad-model drives the rt-core distance model through
the python plugin with one of its output indices deliberately out of range,
and requires PLUMED to reject it with a message. Both cases report
"NOT REJECTED PROPERLY" without the previous commit.

- `atom`: a positions gradient with respect to one atom past the end of the
  system. Nothing upstream bounds that column.
- `names`: a positions gradient whose third sample dimension is not "atom",
  which is what makes reading column 2 by index safe.

Two notes recorded while writing it.

The models leave their output alone when the system is empty, because PLUMED
runs a model once on a zero-atom system to discover how many properties it
returns. A model which indexes its own arrays unconditionally fails there
first, with its own error.

There is no case for the `sample` column, because metatensor range-checks
that one itself when the gradient is added:

    invalid value for the 'sample' dimension in gradient samples:
    we got 5, but the values contain 1 samples

so a model cannot hand PLUMED a block with it out of range. The check in
`apply()` stays for the paths where metatensor skips its own -- meta tensors,
and labels built through the unchecked constructor -- and the source now says
so.

rt-core still passes.
@EricBoittier
EricBoittier force-pushed the EricBoittier/metatomic-capi-update branch from bd9b177 to 04b2252 Compare November 19, 2026 06:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants