Improved Elastic Weight Consolidation

This repository contains the implementation and paper-facing experiments for Improved Elastic Weight Consolidation as an Optimization Constraint for Continual Learning.

The revised terminology is deliberate:

• opg_uniform stores outer products of observed-sample loss gradients. For negative log likelihood, this is the observed-label empirical-Fisher algebra, not the model Fisher in general.
• ief_uniform applies the iEF level-set normalization with task-relative damping.
• opg_rdw and ief_rdw cross the same order-dependent residual-diversity weighting rule with the two matrix constructions.
• sampled_fisher samples labels from the frozen predictive distribution and estimates the model Fisher; mas is an external output-sensitivity baseline.

Install and verify

The lockfile is the archival environment specification.

uv sync --frozen --dev
CUDA_VISIBLE_DEVICES="" uv run pytest -q

At the frozen run commit, the expected result is 75 passed with one upstream JAXopt deprecation warning. See REPRODUCIBILITY.md for the manifest, validation, selection, and locked-test workflow, and DATA_PROVENANCE.md for dataset/model sources and split integrity.

Basic usage

from iewc import IEWCConfig, IEWCPlugin

iewc = IEWCConfig(
    lambda_=10000.0,
    tau=1e-2,
    geometry="euclidean",
    sample_weighting="uniform",
)
plugin = IEWCPlugin(config=iewc)

The same configuration object is accepted by the maintained diagonal and low-rank IEWC plugins. Existing keyword arguments such as ewc_lambda, tau, and output_metric remain available for compatibility.

Evidence tiers

The repository retains the rejected submission's broader ImageNet-R, TRACE, forecasting, VOC, DDPM, and CIFAR-100 results as exploratory provenance. The revised paper's confirmatory comparison is separate: its protocol and manifest were frozen before locked-test model evaluation, hyperparameters are selected only on validation data, and locked runs require both the manifest hash and a complete, hashed selection artifact.

Historical paper-table generation remains available through:

uv run python scripts/build_empirical_artifacts.py

Commit 00f9d6d records the pre-cleanup state used for the inherited empirical results. The revised run commit is recorded in every confirmatory result.

Citation

@misc{IEWC,
  title = {Improved Elastic Weight Consolidation as an Optimization Constraint for Continual Learning},
  author = {Wiest, Davide},
  year = {2026},
  publisher = {Zenodo},
  doi = {10.5281/zenodo.20786362},
  url = {https://zenodo.org/records/20786362}
}