E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials

E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials Simon Batzner Albert Musaelian Lixin Sun Mario Geiger Jonathan P. Mailoa Mordechai Kornbluth Nicola Molinari Tess Smidt Boris Kozinsky

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials 2022 Nature Communications 10.1038/s41467-022-29939-5 batzner2022-e-3-equivariant-graph-neural-netwo batzner2022-e-3-equivariant-graph-neural-netwo.txt, the project's staged copy 6bb97e4f1133194690da304ee0c796688ba1296c cc-by artificial-intelligence

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them learning the interatomic potential energy and forces of molecules and materials from first-principles calculations, at a training-set size small enough that the reference calculations are affordable Quotation withheld because this record is not cleared for redistribution. whether making a network's internal features equivariant rather than invariant changes how much data it needs, and not merely how accurate it is at a fixed data budget Quotation withheld because this record is not cleared for redistribution. NequIP - E(3)-equivariant convolutions over geometric tensors giving state-of-the-art accuracy on molecules and materials with up to three orders of magnitude less training data, plus an ablation isolating equivariance as the cause and a learning-curve slope that no other method has Quotation withheld because this record is not cleared for redistribution. method Quotation withheld because this record is not cleared for redistribution. sample efficiency in learned interatomic potentials - and the fact that a symmetry built into the representation changes the exponent of the learning curve rather than only its offset Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. learning curves normally follow a power law whose exponent belongs to the data set - methods move the curve up or down and leave its slope alone. Building rotation equivariance into the features changes the slope itself, so the advantage grows with every additional training structure rather than staying constant a symmetry that changes the exponent, not the offset Quotation withheld because this record is not cleared for redistribution. predicting force vectors directly leaves nothing enforcing that they are the gradient of any energy, so long molecular dynamics runs can drift; taking the forces as the gradient of the predicted potential makes energy conservation a property of the construction rather than something the training has to discover conservation by differentiation Quotation withheld because this record is not cleared for redistribution. a reported error is the sum of the model's error and the reference calculation's; running one model on two recomputations of the same molecules separates them, and here the energies move while the forces do not, which says the original benchmark's energy noise was the larger term an instrument reading its own reference Quotation withheld because this record is not cleared for redistribution. each configuration carries one energy and 3N force components, so M structures are M(3N + 1) labels; a data-efficiency claim counted in structures is counted in a unit that hides how much supervision each one supplies, and the authors raise this before making the claim labels per structure as the hidden denominator Quotation withheld because this record is not cleared for redistribution. the data-efficiency claim is counted in structures, and each structure carries 3N + 1 labels - the authors state the alternative denominator themselves Quotation withheld because this record is not cleared for redistribution. the energy comparison against DeepMD is confounded by there being 3N force targets and one energy target per frame, which the authors expect to favour the larger training set on energies Quotation withheld because this record is not cleared for redistribution. the DeepMD version compared against is not the smooth one, and the smooth version does not report results on water and ice, so the comparison could not be made Quotation withheld because this record is not cleared for redistribution. how the many-body character of interactions arises in message-passing potentials has no clear theoretical understanding Quotation withheld because this record is not cleared for redistribution. long-range interactions are not explicitly included, and how much of them message passing captures is unmeasured Quotation withheld because this record is not cleared for redistribution. interpretability is an open challenge - classical potentials assign energy to named interaction types and this model does not Quotation withheld because this record is not cleared for redistribution. the tension is structural: the simple functional form that makes classical force fields interpretable is the same thing that limits their accuracy Quotation withheld because this record is not cleared for redistribution. group representation theory - E(3), tensor rank, parity and spherical harmonics - carried from physics into the design of a network's internal feature space Quotation withheld because this record is not cleared for redistribution. power-law learning curves and their exponents, carried from statistical learning theory into a claim about what a symmetry does to sample complexity Quotation withheld because this record is not cleared for redistribution. a convergence scan over rotation order l in {0, 1, 2, 3}, where l = 0 is the same network with equivariance switched off eliminates: every other part of the architecture as the source of the accuracy - depth, cutoff, radial basis and training setup are identical across the scan, so only the tensor rank differs · outcome: held Quotation withheld because this record is not cleared for redistribution. interactions above l = 0 explicitly turned off to make the network a conventional invariant GNN-IP in a SchNet-style fashion, then run across training-set sizes from 10 to 1000 with identical hyperparameters eliminates: training budget, capacity or hyperparameter tuning as the explanation of the sample efficiency - the comparison is the same network at every data size · outcome: held Quotation withheld because this record is not cleared for redistribution. weight- and feature-controlled versions of the same comparison, reported in the supplementary information eliminates: parameter count as the explanation - a higher-l network has more weights, and without this the ablation would be measuring size rather than symmetry · outcome: held Quotation withheld because this record is not cleared for redistribution. separate models trained and tested on BOTH the original and the revised MD-17 data sets, with energy and force errors reported for each eliminates: the benchmark's own numerical noise being read as model error - the forces stay put while the energies move, which can only be a property of the reference data · outcome: held Quotation withheld because this record is not cleared for redistribution. comparison against kernel methods - sGDML and FCHL19/GPR - and not only against other neural networks eliminates: the small-data claim being tested only against models with the same weakness; kernels are the methods that usually win on small training sets · outcome: held Quotation withheld because this record is not cleared for redistribution. the trained potential used to run molecular dynamics, with the radial and two angular distribution functions compared against the ab-initio trajectory at the same temperature eliminates: a low force error being mistaken for correct dynamics - a potential can score well pointwise and still produce the wrong structure when integrated · outcome: held Quotation withheld because this record is not cleared for redistribution. training exclusively on the 3000 K molten trajectory and evaluating on the 600 K quenched glass eliminates: memorisation of the training phase - the test structures are a different phase of the same material, which the model never saw · outcome: held Quotation withheld because this record is not cleared for redistribution. Li diffusivity measured from NequIP-driven MD started from ten different initial velocity draws and compared against the ab-initio value eliminates: one lucky trajectory - diffusivity from a single run of a superionic conductor is a sample from a distribution, not a number · outcome: held Quotation withheld because this record is not cleared for redistribution. a published Behler-Parrinello network's force RMSE on the same system quoted against NequIP models trained on 100 and 250 structures eliminates: the data-efficiency comparison resting only on this paper's own baselines · outcome: held Quotation withheld because this record is not cleared for redistribution. three different weightings of the energy and force terms in the loss function, reported side by side on the water and ice system eliminates: that the reported accuracy is an artefact of one chosen energy/force weighting in the loss · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. equivariant equivariant Quotation withheld because this record is not cleared for redistribution. invariant invariant Quotation withheld because this record is not cleared for redistribution. rotation order rotation order Quotation withheld because this record is not cleared for redistribution. NequIP NequIP Quotation withheld because this record is not cleared for redistribution. interatomic potentials interatomic potentials Quotation withheld because this record is not cleared for redistribution. log-log slope log-log slope Quotation withheld because this record is not cleared for redistribution. learning curve learning curve Quotation withheld because this record is not cleared for redistribution. noise floor noise floor Quotation withheld because this record is not cleared for redistribution. MD-17 MD-17 Quotation withheld because this record is not cleared for redistribution. DeepMD DeepMD Quotation withheld because this record is not cleared for redistribution. DimeNet DimeNet Quotation withheld because this record is not cleared for redistribution. diffusivity diffusivity Quotation withheld because this record is not cleared for redistribution. NequIP's MD-17 energy error - ON THE ORIGINAL DATA SET: significantly higher, and the paper attributes the difference to that data set's noise floor rather than to the model Quotation withheld because this record is not cleared for redistribution. NequIP's MD-17 energy error - ON THE REVISED DATA SET: significantly lower, and the only version the authors say should be used for comparing methods Quotation withheld because this record is not cleared for redistribution. what raising the rotation order does - CROSSING l = 0 TO l = 1: changes the log-log slope of the learning curve, so the model learns faster from new data Quotation withheld because this record is not cleared for redistribution. what raising the rotation order does - BEYOND l = 1: shifts the curve down without changing its slope, the same thing every other method does to it Quotation withheld because this record is not cleared for redistribution. a training set of M structures - COUNTED AS CONFIGURATIONS: M, which is the number the data-efficiency claim is made in Quotation withheld because this record is not cleared for redistribution. a training set of M structures - COUNTED AS LABELS: M(3N + 1), because each structure carries one energy and 3N force components - the authors raise this themselves before claiming efficiency Quotation withheld because this record is not cleared for redistribution. critiques → the belief that deep neural networks require massive training sets - contradicted here by matching or beating existing models with up to three orders of magnitude fewer reference calculations Quotation withheld because this record is not cleared for redistribution. critiques → invariant graph neural network interatomic potentials, whose features are scalars and which therefore discard the directional information equivariant filters can use Quotation withheld because this record is not cleared for redistribution. critiques → direct prediction of atomic force vectors, which is not guaranteed to conserve energy because the forces are not gradients of a potential Quotation withheld because this record is not cleared for redistribution. critiques → the original MD-17 benchmark, whose energy noise floor the authors say is too high for comparing methods - only the revised recomputation should be used Quotation withheld because this record is not cleared for redistribution. qualifies → the empirical rule that learning-curve exponents belong to the data set and methods only shift the curve - broken here by equivariance, and only by crossing l = 0 to 1 Quotation withheld because this record is not cleared for redistribution. qualifies → its own DeepMD comparison, which used the non-smooth published version because the smooth one reports no results on water and ice Quotation withheld because this record is not cleared for redistribution. equivariant —distinct-from→ invariant equivariant invariant differs in: whether the internal features have a direction that rotates with the molecule · the relation itself is separately evidenced · strength of the relation: 0.95 · differs in (stated by the tagger, no quotation): whether the internal features have a direction that rotates with the molecule · the relation stated: this method carries relative position vectors and higher-order tensors where invariant networks carry only distances Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'In contrast' refers to the rotation-invariant GNN-IPs of the previous sentence. Quotation withheld because this record is not cleared for redistribution. equivariant networks with l >= 1 outperform the identical network at l = 0 at every training-set size from 10 to 1000 equivariant networks with l >= 1 outperform the identical network at l = 0 at every training-set size from 10 to 1000 Quotation withheld because this record is not cleared for redistribution. crossing l = 0 to 1 changes the log-log slope of the learning curve; raising l beyond 1 only shifts it crossing l = 0 to 1 changes the log-log slope of the learning curve; raising l beyond 1 only shifts it Quotation withheld because this record is not cleared for redistribution. identical models give lower energy error on revised MD-17 and similar force error, locating the difference in the benchmark's noise floor rather than the method identical models give lower energy error on revised MD-17 and similar force error, locating the difference in the benchmark's noise floor rather than the method Quotation withheld because this record is not cleared for redistribution. With 1000x fewer training data, NequIP beats DeepMD on force error in all four parts of the data set. With 1000x fewer training data, NequIP beats DeepMD on force error in all four parts of the data set. Quotation withheld because this record is not cleared for redistribution. Trained only on high-temperature molten data, NequIP reproduces the glass's RDF and both ADFs. Trained only on high-temperature molten data, NequIP reproduces the glass's RDF and both ADFs. Quotation withheld because this record is not cleared for redistribution. Li diffusivity from NequIP-driven MD is 1.25e-5 cm2/s against 1.37e-5 from ab-initio MD, a 9% relative error Li diffusivity from NequIP-driven MD is 1.25e-5 cm2/s against 1.37e-5 from ab-initio MD, a 9% relative error Quotation withheld because this record is not cleared for redistribution. the method beats kernel approaches such as sGDML and FCHL19/GPR, which normally win on small training sets the method beats kernel approaches such as sGDML and FCHL19/GPR, which normally win on small training sets Quotation withheld because this record is not cleared for redistribution. Each training configuration of N atoms supplies 3N + 1 labels. Each training configuration of N atoms supplies 3N + 1 labels. Quotation withheld because this record is not cleared for redistribution. the exponent of the learning-curve power law - how fast a learning algorithm improves as new data become available, normally fixed across methods for a given data set log-log slope Quotation withheld because this record is not cleared for redistribution. the reference data set's own numerical error, identified here by training identical models on two recomputations of the same molecules and seeing which error moves noise floor Quotation withheld because this record is not cleared for redistribution. fewer training structures than the DeepMD comparison on water and ice - 133 against 133,500 1000x Quotation withheld because this record is not cleared for redistribution. the relative error of the NequIP Li diffusivity against ab-initio MD 9% Quotation withheld because this record is not cleared for redistribution. structures the Behler-Parrinello baseline needed for an RMSE that NequIP matches with 100 1303 Quotation withheld because this record is not cleared for redistribution. MAE MAE Quotation withheld because this record is not cleared for redistribution. RMSE RMSE Quotation withheld because this record is not cleared for redistribution. rotation order rotation order Quotation withheld because this record is not cleared for redistribution. log-log slope log-log slope Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1038/s41467-022-29939-5.