Repository navigation
Conversation
ClarkLDF built incremental_fits_ from the LDF-implied ultimate (latest diagonal times the fitted CDF) for both methods. Under Cape Cod the expected ultimate is premium * ELR, so scale_ and norm_resid_ were wrong for Cape Cod fits. On Clark's (2003) genins example the scale was 53,822 instead of the paper's 61,577. Add regression tests against the paper's published LDF and Cape Cod scale parameters, editing the one genins cell that is mistyped in the paper. Fixes casact#1501 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pyright Type CompletenessView the full Project (full
Other symbols referenced but not exported by
Symbols without documentation:
Patch (exported symbols added or changed by this PR): 42.9% fully typed (3 / 7)
Patch symbol details
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1520 +/- ##
==========================================
+ Coverage 94.36% 94.38% +0.01%
==========================================
Files 97 98 +1
Lines 6198 6216 +18
Branches 798 799 +1
==========================================
+ Hits 5849 5867 +18
Misses 225 225
Partials 124 124
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@scks1 thanks for putting in the PR! please address the unittest fails. |
ClarkLDF's optimizer stops at slightly different points across Python versions and numpy/scipy/pandas builds. On Python 3.11 and on Python 3.14 with pandas 3, the fitted scales were 65,030.03 (LDF) and 61,579.31 (Cape Cod), outside the abs=1 tolerance, and a local run that reproduced those values had the ELR inside abs=1e-4 by 0.0000038. Compare all three fitted values with the paper at rel=1e-3. With the Cape Cod fix reverted, the Cape Cod scale is 53,824 and both Cape Cod tests still fail. The LDF test now cites p. 61, where Clark gives the 65,029 scale. Developed with an AI assistant (Árbol, running Muse).
|
Thanks! The failures were in the new tests, not the fix: ClarkLDF's optimizer stops at slightly different points across Python and library versions, so on 3.11 and on 3.14 with pandas 3 the scales came out at 65,030.03 and 61,579.31, just outside the ±1 tolerance. The paper comparisons now use a 0.1% relative tolerance, which the pre-fix Cape Cod scale (about 53,822) still fails, and the LDF test cites p. 61, where Clark gives 65,029. (This follow-up, commit and reply, was made with Árbol, an AI assistant, under my direction.) |
_set_fit_groups already moves a sparse sample_weight to numpy, so the
array_backend check before set_backend('numpy') only mattered for cupy
and left a branch the test suite could not reach. Convert
unconditionally and add a test that a sparse premium triangle gives the
same Cape Cod fit as a dense one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
|
||
| def test_clark_cape_cod_scale(clark_genins, clark_premium): | ||
| """ | ||
| Cape Cod scale parameter matches Clark (2003), p. 68. The fitted |
| assert model.scale_.iloc[0, 0] == pytest.approx(61_577, rel=PAPER_RTOL) | ||
|
|
||
|
|
||
| def test_clark_cape_cod_incremental_fits(clark_genins, clark_premium): |
There was a problem hiding this comment.
is the capecod incrementals available from the paper for us to hardcode into this test?
| if sample_weight: | ||
| # Cape Cod: the expected ultimate is premium * ELR, not the | ||
| # LDF-implied ultimate from the latest diagonal. | ||
| premium = sample_weight.set_backend("numpy").values |
There was a problem hiding this comment.
_set_fit_group already set sample_weight backend to numpy
The Cape Cod scale test now cites p. 67 for the ELR and p. 68 for the scale. test_clark_cape_cod_incremental_fits compares incremental_fits_ with the 55 fitted values Clark prints in Table 2.1 (p. 80), at the same 0.1% relative tolerance as the other paper comparisons. With the Cape Cod fix reverted, every cell is 2.5% to 16.1% away and the test fails. _set_fit_groups already puts sample_weight on the numpy backend, so the Cape Cod ultimate reads sample_weight.values directly, as the solver does. Developed with an AI assistant (Árbol, running Muse).
|
Thanks for the review. All three are addressed in 19115ab:
(The commit and this reply were made with Árbol, an AI assistant, under my direction.) |
Summary of Changes
ClarkLDFfitted the right Cape Cod parameters but computedincremental_fits_from the wrong expected ultimate. For both methods it used the LDF-implied ultimate (latest diagonal × fitted CDF). Under Cape Cod the expected ultimate is premium × ELR. Becausescale_andnorm_resid_are derived fromincremental_fits_, both were wrong for Cape Cod fits.On the reproduction from #1501 (Clark's
geninsexample, with premium 10,000,000 + 400,000·i):The fix is in
chainladder/development/clark.py: whensample_weightis given, the expected ultimate behindincremental_fits_is nowsample_weight × elr_. The LDF method path is unchanged, and its scale (65,029) already matched the paper (p. 61).Related GitHub Issue(s)
Fixes #1501
AI/LLM Usage
This change was written with Claude Opus 5.5 (Claude Code), working under my direction. It localised the bug, wrote the fix and tests, and drafted this description. I reviewed the results before submitting.
Two follow-up commits were made with Árbol, an AI assistant running on Muse, also under my direction, and I reviewed them: relative test tolerances with the p. 61 citation, and the changes from the review (page citations, the comparison with the paper's Table 2.1, and removing a redundant backend conversion).
Additional Context for Reviewers
Typo in Clark's paper. Clark's table prints 2,182,708 for the 1991 origin at age 48, where
genins(the Taylor-Ashe data) has 2,218,270. The sample data is correct, so it is left unchanged. The test fixture edits that one cell so the tests can be checked against the paper's published results, the same way the reproduction in #1501 does.Tests. New file
chainladder/development/tests/test_clark.py:test_clark_ldf_scale: the LDF scale is 65,029 (paper p. 61).test_clark_cape_cod_scale: the Cape Cod ELR is 0.5978 (paper p. 67) and the scale is 61,577 (p. 68), using the assert from [BUG] Incorrect scale parameter computed with ClarkLDF Cape Cod method #1501.test_clark_cape_cod_incremental_fits: the 55 fitted incrementals match the paper's Table 2.1 (p. 80).test_clark_cape_cod_sparse_premium: a premium triangle on the sparse backend gives the same Cape Cod fit.The Cape Cod scale and incremental-fit tests fail on
mainand pass with this change. The LDF and sparse-premium tests pass on both, as expected. The paper comparisons use a 0.1% relative tolerance, because the optimizer's stopping point varies across Python and library versions and moves the scale by a few units; the pre-fix Cape Cod scale is about 13% away. Full suite: 1428 passed, 2 skipped.ruff checkandruff format --checkare clean. I also ran theclrdCape Cod examples from the docstring, with and withoutgroupby="LOB". They give the same ELRs as before and correctly shapedincremental_fits_/scale_.Submitter's Checklist
[FIX],[FEAT],[DOCS],[TST],[CHORE], or[BRK]).Reviewer's Checklist
🤖 Generated with Claude Code