Update the sea ice sensitivity diagnostic for ESMValTool v2.15 - #899
Open
lewisjared wants to merge 79 commits into
Open
Update the sea ice sensitivity diagnostic for ESMValTool v2.15#899lewisjared wants to merge 79 commits into
lewisjared wants to merge 79 commits into
Conversation
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.5.7 to 6.5.8. - [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst) - [Commits](tornadoweb/tornado@v6.5.7...v6.5.8) --- updated-dependencies: - dependency-name: tornado dependency-version: 6.5.8 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
The CMIP7 DRS nests files 12 levels below the archive root, so the limit of 10 silently discovered nothing. Bumping the number would break again whenever the root given sits above MIP-DRS7, so discovery now accepts `depth=None` to walk the whole tree and the CMIP7 adapter uses it.
Adds a test that the CMIP7 adapter reaches files from the bare archive root, which fails if either call site goes back to a fixed depth. A discovery run that matches nothing now warns rather than logging at info, so the streaming path stops failing quietly.
Remove the directory depth limit from dataset discovery
The obs4REF collection was ingested as obs4MIPs, so the catalog could not show which datasets came from the registry and which from the archive. Ingests it as obs4ref instead. The solver folds the obs4REF catalog into the obs4MIPs one before matching, so diagnostics keep asking for obs4MIPs and a dataset held by both is taken from obs4MIPs.
…naliseable The misfiled check flagged any dataset an obs4REF registry carries, so a CERES-EBAF, GPCP, HadISST or TropFlux copy correctly fetched from ESGF was reported as needing a re-ingest. It now looks only at the directory layout the registry actually produces. The merged obs4MIPs catalog dropped its adapter and database, so an unfinalised dataset could no longer be finalised. It now carries them through, and ref doctor solves against the same catalogs the solver would use.
Shares one helper for unwrapping a catalog, hoists the solver imports in the doctor checks now that there is no cycle to dodge, and merges the catalogs once per unsolvable-diagnostics run rather than once per diagnostic explained. Also promotes normalize_requirement_sets, which the doctor checks now use.
Carrying the obs4MIPs adapter through the merge made the catalog reloadable, and a reload would go back to that adapter alone and drop every obs4REF row just merged in. The merge now carries no adapter, so it cannot reload.
The recipe now carries its own model and observational datasets inline, so drop both and supply the solved models per variable instead. The diagnostic script renamed its output to data_values.csv and gave it a period, regression and statistic column header, so read the metric bundle from the new shape.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Author
Regression baseline diff2 test case(s) and 25 native file(s) changed against
esmvaltool/sea-ice-sensitivity/cmip6 -- 13 native file(s)
Committed artefacts changed:
--- old a9fce0fa39fb
+++ new 2119b8fe5751
@@ -10,11 +10,18 @@
"metric"
],
"metric": {
- "annual_siconc_trend": {},
- "annual_tas_trend": {},
- "direct_p_val": {},
- "direct_r_val": {},
- "direct_sensitivity_(notz-style)": {}
+ "gmst_over_time_p_value": {},
+ "gmst_over_time_r_value": {},
+ "gmst_over_time_slope": {},
+ "gmst_over_time_std_err_slope": {},
+ "sia_over_gmst_p_value": {},
+ "sia_over_gmst_r_value": {},
+ "sia_over_gmst_slope": {},
+ "sia_over_gmst_std_err_slope": {},
+ "sia_over_time_p_value": {},
+ "sia_over_time_r_value": {},
+ "sia_over_time_slope": {},
+ "sia_over_time_std_err_slope": {}
},
"region": {
"antarctic": {},
@@ -31,18 +38,32 @@
"historical": {
"CanESM5": {
"antarctic": {
- "annual_siconc_trend": -0.0033150537900789,
- "annual_tas_trend": 0.0327222551618303,
- "direct_p_val": 0.3522661326377206,
- "direct_r_val": -0.1596710307466839,
- "direct_sensitivity_(notz-style)": -0.2176355158771825
+ "gmst_over_time_p_value": 1.1817364161577566e-16,
+ "gmst_over_time_r_value": 0.9329816552013132,
+ "gmst_over_time_slope": 0.0327222551618303,
+ "gmst_over_time_std_err_slope": 0.0021649176384803,
+ "sia_over_gmst_p_value": 0.35226571559906,
+ "sia_over_gmst_r_value": -0.1596710383892059,
+ "sia_over_gmst_slope": -0.2176355123519897,
+ "sia_over_gmst_std_err_slope": 0.2307577282190323,
+ "sia_over_time_p_value": 0.6877781615714882,
+ "sia_over_time_r_value": -0.0693452657693361,
+ "sia_over_time_slope": -0.0033150537900789,
+ "sia_over_time_std_err_slope": 0.00817876208803
},
"arctic": {
- "annual_siconc_trend": -0.0723691365280053,
- "annual_tas_trend": 0.0327222551618303,
- "direct_p_val": 3.949451443123145e-12,
- "direct_r_val": -0.8728090234442475,
- "direct_sensitivity_(notz-style)": -2.066191154341656
+ "gmst_over_time_p_value": 1.1817364161577566e-16,
+ "gmst_over_time_r_value": 0.9329816552013132,
+ "gmst_over_time_slope": 0.0327222551618303,
+ "gmst_over_time_std_err_slope": 0.0021649176384803,
+ "sia_over_gmst_p_value": 3.949456213458991e-12,
+ "sia_over_gmst_r_value": -0.8728089928627014,
+ "sia_over_gmst_slope": -2.0661909580230717,
+ "sia_over_gmst_std_err_slope": 0.1981466561555862,
+ "sia_over_time_p_value": 4.5769362533459706e-12,
+ "sia_over_time_r_value": -0.8716296559629397,
+ "sia_over_time_slope": -0.0723691365280053,
+ "sia_over_time_std_err_slope": 0.0069794989430844
}
}
}
--- old 93a956c8c0c2
+++ new ce0b44849e6f
@@ -222,10 +222,12 @@
">
<figure class="figure">
<a href='plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice sensitivity.png'>
- <img class="figure-img img-fluid" src='plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice sensitivity.png' alt='Sensitivity of sea ice area to annual mean global warming.Mean (dashed), standard deviation (shaded) and plausible values from 1979-2014.'/>
+ <img class="figure-img img-fluid" src='plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice sensitivity.png' alt='Sensitivity of sea ice area to annual mean global warming.
+Mean (dashed), standard deviation (shaded) and plausible (dotted) values from 1979-2014 are shown in grey.'/>
</a>
<figcaption class="figure-caption">
- Sensitivity of sea ice area to annual mean global warming.Mean (dashed), standard deviation (shaded) and plausible values from 1979-2014.
+ Sensitivity of sea ice area to annual mean global warming.
+Mean (dashed), standard deviation (shaded) and plausible (dotted) values from 1979-2014 are shown in grey.
<br>
<br>
<a href='plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice sensitivity.png'>download</a> |
--- old 0e8c755c7b79
+++ new aa316a3d7211
@@ -18,7 +18,7 @@
- parsons_naomi
caption: Decadal trends of sea ice area and global mean temperature.
plot_type: other
-? <OUTPUT_DIR>/executions/recipe/work/antarctic/sea_ice_sensitivity_script/plotted_values.csv
+? <OUTPUT_DIR>/executions/recipe/work/antarctic/sea_ice_sensitivity_script/data_values
: ancestors:
- <OUTPUT_DIR>/executions/recipe/preproc/antarctic/siconc/CMIP6_CanESM5_SImon_historical_r1i1p1f1_siconc_gn_1979-2014.nc
- <OUTPUT_DIR>/executions/recipe/preproc/antarctic/tas/CMIP6_CanESM5_Amon_historical_r1i1p1f1_tas_gn_1979-2014.nc
--- old c12c3034076b
+++ new 452842ae2721
@@ -6,8 +6,10 @@
authors:
- sellar_alistair
- parsons_naomi
- caption: Sensitivity of sea ice area to annual mean global warming.Mean (dashed),
- standard deviation (shaded) and plausible values from 1979-2014.
+ caption: 'Sensitivity of sea ice area to annual mean global warming.
+
+ Mean (dashed), standard deviation (shaded) and plausible (dotted) values from
+ 1979-2014 are shown in grey.'
plot_type: other
? <OUTPUT_DIR>/executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September
Arctic sea ice trends.png
@@ -19,7 +21,7 @@
- parsons_naomi
caption: Decadal trends of sea ice area and global mean temperature.
plot_type: other
-? <OUTPUT_DIR>/executions/recipe/work/arctic/sea_ice_sensitivity_script/plotted_values.csv
+? <OUTPUT_DIR>/executions/recipe/work/arctic/sea_ice_sensitivity_script/data_values
: ancestors:
- <OUTPUT_DIR>/executions/recipe/preproc/arctic/siconc/CMIP6_CanESM5_SImon_historical_r1i1p1f1_siconc_gn_1979-2014.nc
- <OUTPUT_DIR>/executions/recipe/preproc/arctic/tas/CMIP6_CanESM5_Amon_historical_r1i1p1f1_tas_gn_1979-2014.nc
--- old (absent)
+++ new f717d22563f8
@@ -0,0 +1,4 @@
+period,,,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014
+regression,,,gmst_over_time,gmst_over_time,gmst_over_time,gmst_over_time,sia_over_time,sia_over_time,sia_over_time,sia_over_time,sia_over_gmst,sia_over_gmst,sia_over_gmst,sia_over_gmst
+statistic,label,type,slope,r_value,p_value,std_err_slope,slope,r_value,p_value,std_err_slope,slope,r_value,p_value,std_err_slope
+CanESM5,unlabelled,model,0.03272225516183035,0.9329816552013133,1.1817364161577569e-16,0.002164917638480348,-0.003315053790078965,-0.06934526576933618,0.6877781615714882,0.008178762088030088,-0.21763551235198975,-0.15967103838920593,0.35226571559906006,0.2307577282190323
--- old 832fccab6e7e
+++ new (absent)
@@ -1,2 +0,0 @@
-,label,direct_sensitivity_(notz-style),annual_siconc_trend,annual_tas_trend,direct_r_val,direct_p_val
-CanESM5,unlabelled,-0.21763551587718252,-0.003315053790078968,0.03272225516183036,-0.15967103074668393,0.35226613263772066
--- old (absent)
+++ new 2beee01f0310
@@ -0,0 +1,4 @@
+period,,,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014
+regression,,,gmst_over_time,gmst_over_time,gmst_over_time,gmst_over_time,sia_over_time,sia_over_time,sia_over_time,sia_over_time,sia_over_gmst,sia_over_gmst,sia_over_gmst,sia_over_gmst
+statistic,label,type,slope,r_value,p_value,std_err_slope,slope,r_value,p_value,std_err_slope,slope,r_value,p_value,std_err_slope
+CanESM5,unlabelled,model,0.03272225516183035,0.9329816552013133,1.1817364161577569e-16,0.002164917638480348,-0.07236913652800532,-0.8716296559629397,4.5769362533459706e-12,0.006979498943084416,-2.0661909580230713,-0.8728089928627014,3.9494562134589906e-12,0.19814665615558624
--- old c0b13303ee5b
+++ new (absent)
@@ -1,2 +0,0 @@
-,label,direct_sensitivity_(notz-style),annual_siconc_trend,annual_tas_trend,direct_r_val,direct_p_val
-CanESM5,unlabelled,-2.0661911543416562,-0.07236913652800532,0.03272225516183036,-0.8728090234442475,3.949451443123145e-12
--- old 2fc746e657b5
+++ new c6f41e586979
@@ -1,19 +1,19 @@
{
"data": {
- "executions/recipe/work/antarctic/sea_ice_sensitivity_script/plotted_values.csv": {
+ "executions/recipe/work/antarctic/sea_ice_sensitivity_script/data_values.csv": {
"description": "",
"dimensions": {
"region": "antarctic"
},
- "filename": "executions/recipe/work/antarctic/sea_ice_sensitivity_script/plotted_values.csv",
+ "filename": "executions/recipe/work/antarctic/sea_ice_sensitivity_script/data_values.csv",
"long_name": "Annual (not decadal) figures"
},
- "executions/recipe/work/arctic/sea_ice_sensitivity_script/plotted_values.csv": {
+ "executions/recipe/work/arctic/sea_ice_sensitivity_script/data_values.csv": {
"description": "",
"dimensions": {
"region": "arctic"
},
- "filename": "executions/recipe/work/arctic/sea_ice_sensitivity_script/plotted_values.csv",
+ "filename": "executions/recipe/work/arctic/sea_ice_sensitivity_script/data_values.csv",
"long_name": "Annual (not decadal) figures"
}
},
@@ -51,7 +51,7 @@
"region": "arctic"
},
"filename": "executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice sensitivity.png",
- "long_name": "Sensitivity of sea ice area to annual mean global warming.Mean (dashed), standard deviation (shaded) and plausible values from 1979-2014."
+ "long_name": "Sensitivity of sea ice area to annual mean global warming.\nMean (dashed), standard deviation (shaded) and plausible (dotted) values from 1979-2014 are shown in grey."
},
"executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice trends.png": {
"description": "",esmvaltool/sea-ice-sensitivity/cmip7 -- 12 native file(s)
Committed artefacts changed:
--- old (absent)
+++ new e96e3d780172
@@ -0,0 +1,71 @@
+{
+ "DIMENSIONS": {
+ "experiment_id": {
+ "historical": {}
+ },
+ "json_structure": [
+ "experiment_id",
+ "source_id",
+ "region",
+ "metric"
+ ],
+ "metric": {
+ "gmst_over_time_p_value": {},
+ "gmst_over_time_r_value": {},
+ "gmst_over_time_slope": {},
+ "gmst_over_time_std_err_slope": {},
+ "sia_over_gmst_p_value": {},
+ "sia_over_gmst_r_value": {},
+ "sia_over_gmst_slope": {},
+ "sia_over_gmst_std_err_slope": {},
+ "sia_over_time_p_value": {},
+ "sia_over_time_r_value": {},
+ "sia_over_time_slope": {},
+ "sia_over_time_std_err_slope": {}
+ },
+ "region": {
+ "antarctic": {},
+ "arctic": {}
+ },
+ "source_id": {
+ "CanESM5": {}
+ }
+ },
+ "DISCLAIMER": null,
+ "NOTES": null,
+ "PROVENANCE": null,
+ "RESULTS": {
+ "historical": {
+ "CanESM5": {
+ "antarctic": {
+ "gmst_over_time_p_value": 1.169205291004257e-16,
+ "gmst_over_time_r_value": 0.9330249991378756,
+ "gmst_over_time_slope": 0.0327300514800514,
+ "gmst_over_time_std_err_slope": 0.0021646567943726,
+ "sia_over_gmst_p_value": 0.3521364629268646,
+ "sia_over_gmst_r_value": -0.1597133576869964,
+ "sia_over_gmst_slope": -0.2176574617624282,
+ "sia_over_gmst_std_err_slope": 0.2307182401418686,
+ "sia_over_time_p_value": 0.687653233268929,
+ "sia_over_time_r_value": -0.0693744860026367,
+ "sia_over_time_slope": -0.0033165419884169,
+ "sia_over_time_std_err_slope": 0.0081789706465044
+ },
+ "arctic": {
+ "gmst_over_time_p_value": 1.169205291004257e-16,
+ "gmst_over_time_r_value": 0.9330249991378756,
+ "gmst_over_time_slope": 0.0327300514800514,
+ "gmst_over_time_std_err_slope": 0.0021646567943726,
+ "sia_over_gmst_p_value": 3.9593337289312025e-12,
+ "sia_over_gmst_r_value": -0.8727891445159912,
+ "sia_over_gmst_slope": -2.0661001205444336,
+ "sia_over_gmst_std_err_slope": 0.1981568485498428,
+ "sia_over_time_p_value": 4.573249078417207e-12,
+ "sia_over_time_r_value": -0.8716361335971559,
+ "sia_over_time_slope": -0.0723820081241956,
+ "sia_over_time_std_err_slope": 0.0069805243937325
+ }
+ }
+ }
+ }
+}
--- old (absent)
+++ new ce0b44849e6f
@@ -0,0 +1,490 @@
+<!DOCTYPE html>
+<html lang="en">
+
+
+ <head>
+ <meta charset="UTF-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <title>Recipe</title>
+ <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.1/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-4bw+/aepP/YC94hEpVNVgiZdgIC5+VKNBQNGCHeKRQN+PtmoHDEXuppvnDJzQIu9" crossorigin="anonymous">
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.7.0/jquery.min.js"></script>
+ <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.1/dist/js/bootstrap.bundle.min.js" integrity="sha384-HwwvtgBNo3bZJJLYd8oVXjrBZt8cqVSpeBNS5n7C8IVInixGAoxmnlMuBnhbgrkm" crossorigin="anonymous"></script>
+ <style>
+ html {
+ font-size: medium;
+ }
+ body {
+ color: #444;
+ font-family: georgia, times, serif;
+ font-size: 1rem;
+ line-height: 1.4;
+ margin: 2rem auto 8rem;
+ max-width: 1920px;
+ padding: 0 2em;
+ width: auto;
+ }
+ a {
+ text-decoration: none;
+ }
+ a:hover {
+ text-decoration: underline;
+ }
+ h1,h2,h3,h4,h5,h6 {
+ color: #444;
+ font-weight: bold;
+ line-height: 1.2;
+ margin-bottom: 1em;
+ margin-top: 2em;
+ }
+ h1 {
+ font-size: 2.2em;
+ }
+ h2 {
+ font-size: 1.8em;
+ border-bottom: solid 0.1rem #444;
+ }
+ h3 {
+ font-size: 1.5em;
+ }
+ h4 {
+ font-size: 1.3em;
+ }
+ h5 {
+ font-size: 1.2em;
+ font-style: italic;
+ }
+ h6 {
+ font-size: 1.1em;
+ margin-bottom: 0.5rem;
+ }
+ pre,
+ code,
+ xmp {
+ font-family: courier;
+ font-size: 1.1rem;
+ line-height: 1.4;
+ white-space: pre-wrap;
+ }
+ img {
+ width: 500px;
+ }
+ figure {
+ display: inline-block;
+ width: 500px;
+ margin-top: 1em;
+ margin-bottom: 1em;
+ margin-left: 1em;
+ margin-right: 1em;
+ }
+ </style>
+</head>
+
+
+ <body>
+
+ <div class="text-center">
+ <figure class="figure">
+ <img
+ class="figure-img img-fluid"
+ src='https://raw.githubusercontent.com/ESMValGroup/ESMValTool/main/doc/sphinx/source/figures/ESMValTool-logo-2.png'
+ alt='ESMValTool logo.'
+ />
+ </figure>
+ </div>
+
+ <h1>Sea ice sensitivity</h1>
+
+<p>
+ Recipe for quantifying the sensitivity of sea ice to global warming.
+Siconc data is summed for each hemisphere and then compared to the
+change in globally meaned, annually meaned surface air temperature. In
+the northern hemisphere, September sea ice data is used. In the
+southern hemisphere, annual mean sea ice data is used. Two plots are
+produced for each hemisphere, one showing the gradient of the direct
+regression of sea ice area over temperature, and the other showing the
+two separate trends over time.
+</p>
+
+<h3>Authors</h3>
+
+<ul>
+
+ <li>Naomi Parsons (MetOffice, UK; <a href='None'>None</a>)</li>
+
+ <li>Alistair Sellar (MetOffice, UK; <a href='0000-0002-2955-7254'>0000-0002-2955-7254</a>)</li>
+
+ <li>Ed Blockley (MetOffice, UK; <a href='0000-0002-0489-4238'>0000-0002-0489-4238</a>)</li>
+
+</ul>
+
+<h3>Maintainers</h3>
+
+<ul>
+
+ <li>Naomi Parsons (MetOffice, UK; <a href='None'>None</a>)</li>
+
+</ul>
+
+<h3>Projects</h3>
+
+<ul>
+
+</ul>
+
+<h3>References</h3>
+
+<ul>
+
+</ul>
+
+
+<!-- Tab links -->
+<ul class="nav nav-tabs sticky-top bg-light" id="tabDiagnostics" role="tablist">
+ <li class="nav-item">
+ <!-- Filter -->
+ <div class="dropdown" style="position: static">
+ <button type="button" class="btn btn-primary dropdown-toggle" data-bs-toggle="dropdown" aria-expanded="false" data-bs-auto-close="outside">
+ <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-filter" viewBox="0 0 16 16">
+ <path d="M6 10.5a.5.5 0 0 1 .5-.5h3a.5.5 0 0 1 0 1h-3a.5.5 0 0 1-.5-.5zm-2-3a.5.5 0 0 1 .5-.5h7a.5.5 0 0 1 0 1h-7a.5.5 0 0 1-.5-.5zm-2-3a.5.5 0 0 1 .5-.5h11a.5.5 0 0 1 0 1h-11a.5.5 0 0 1-.5-.5z"/>
+ </svg>
+ Filters
+ </button>
+ <div class="dropdown-menu w-100">
+ <div class="container div_filter">
+ <div class="row justify-content-center">
+
+
+ <div class="col-xl-3 col-lg-4 col-sm-6 col-12 filter_category" id="filter_plot_type">
+ <h4>Plot Type</h4>
+
+ <div class="form-check">
+ <input class="form-check-input filter_cb" type="checkbox" value="" id="cb_1_1" rel="f_plot_type_other">
+ <label class="form-check-label" for="cb_1_1">
+ Other
+ </label>
+ </div>
+
+ </div>
+
+ </div>
+ <div class="d-flex justify-content-center align-items-center gap-3">
+ <div class="d-inline-block">
+ <button class="btn btn-primary" id="b_deleteFilters" disabled>Delete Filters</button>
+ </div>
+ <div class="form-check form-switch d-inline-block">
+ <input class="form-check-input" type="checkbox" value="" id="cb_hideEmptyDiagnostics" rel="" checked>
+ <label class="form-check-label" for="cb_hideEmptyDiagnostics">
+ Hide empty diagnostics
+ </label>
+ </div>
+ </div>
+ </div>
+ </div>
+ </div>
+ </li>
+ <li class="nav-item" role="presentation">
+ <button class="nav-link active diagnostics-tab" id="tabAll" data-bs-toggle="tab" data-bs-target="#" type="button" role="tab" aria-controls="" aria-selected="true">All</button>
+ </li>
+
+ <li class="nav-item" role="presentation">
+ <button class="nav-link diagnostics-tab" id="tab_1" data-bs-toggle="tab" data-bs-target="#tabPane_1" type="button" role="tab" aria-controls="tabPane_1" aria-selected="true">Arctic</button>
+ </li>
+
+ <li class="nav-item" role="presentation">
+ <button class="nav-link diagnostics-tab" id="tab_2" data-bs-toggle="tab" data-bs-target="#tabPane_2" type="button" role="tab" aria-controls="tabPane_2" aria-selected="true">Antarctic</button>
+ </li>
+
+</ul>
... 293 more diff line(s) elided
--- old (absent)
+++ new cfa74bdac799
@@ -0,0 +1,29 @@
+? <OUTPUT_DIR>/executions/recipe/plots/antarctic/sea_ice_sensitivity_script/png/Annual
+ Antarctic sea ice sensitivity.png
+: ancestors:
+ - <OUTPUT_DIR>/executions/recipe/preproc/antarctic/siconc/siconc_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ - <OUTPUT_DIR>/executions/recipe/preproc/antarctic/tas/tas_tavg-h2m-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ authors:
+ - sellar_alistair
+ - parsons_naomi
+ caption: Sensitivity of sea ice area to annual mean global warming.
+ plot_type: other
+? <OUTPUT_DIR>/executions/recipe/plots/antarctic/sea_ice_sensitivity_script/png/Annual
+ Antarctic sea ice trends.png
+: ancestors:
+ - <OUTPUT_DIR>/executions/recipe/preproc/antarctic/siconc/siconc_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ - <OUTPUT_DIR>/executions/recipe/preproc/antarctic/tas/tas_tavg-h2m-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ authors:
+ - sellar_alistair
+ - parsons_naomi
+ caption: Decadal trends of sea ice area and global mean temperature.
+ plot_type: other
+? <OUTPUT_DIR>/executions/recipe/work/antarctic/sea_ice_sensitivity_script/data_values
+: ancestors:
+ - <OUTPUT_DIR>/executions/recipe/preproc/antarctic/siconc/siconc_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ - <OUTPUT_DIR>/executions/recipe/preproc/antarctic/tas/tas_tavg-h2m-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ authors:
+ - sellar_alistair
+ - parsons_naomi
+ caption: Annual (not decadal) figures
+ plot_type: other
--- old (absent)
+++ new 61e81ad16557
@@ -0,0 +1,32 @@
+? <OUTPUT_DIR>/executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September
+ Arctic sea ice sensitivity.png
+: ancestors:
+ - <OUTPUT_DIR>/executions/recipe/preproc/arctic/siconc/siconc_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ - <OUTPUT_DIR>/executions/recipe/preproc/arctic/tas/tas_tavg-h2m-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ authors:
+ - sellar_alistair
+ - parsons_naomi
+ caption: 'Sensitivity of sea ice area to annual mean global warming.
+
+ Mean (dashed), standard deviation (shaded) and plausible (dotted) values from
+ 1979-2014 are shown in grey.'
+ plot_type: other
+? <OUTPUT_DIR>/executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September
+ Arctic sea ice trends.png
+: ancestors:
+ - <OUTPUT_DIR>/executions/recipe/preproc/arctic/siconc/siconc_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ - <OUTPUT_DIR>/executions/recipe/preproc/arctic/tas/tas_tavg-h2m-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ authors:
+ - sellar_alistair
+ - parsons_naomi
+ caption: Decadal trends of sea ice area and global mean temperature.
+ plot_type: other
+? <OUTPUT_DIR>/executions/recipe/work/arctic/sea_ice_sensitivity_script/data_values
+: ancestors:
+ - <OUTPUT_DIR>/executions/recipe/preproc/arctic/siconc/siconc_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ - <OUTPUT_DIR>/executions/recipe/preproc/arctic/tas/tas_tavg-h2m-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_1979-2014.nc
+ authors:
+ - sellar_alistair
+ - parsons_naomi
+ caption: Annual (not decadal) figures
+ plot_type: other
--- old (absent)
+++ new d764824f57f8
@@ -0,0 +1,4 @@
+period,,,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014
+regression,,,gmst_over_time,gmst_over_time,gmst_over_time,gmst_over_time,sia_over_time,sia_over_time,sia_over_time,sia_over_time,sia_over_gmst,sia_over_gmst,sia_over_gmst,sia_over_gmst
+statistic,label,type,slope,r_value,p_value,std_err_slope,slope,r_value,p_value,std_err_slope,slope,r_value,p_value,std_err_slope
+CanESM5,unlabelled,model,0.032730051480051474,0.9330249991378756,1.1692052910042573e-16,0.0021646567943726627,-0.003316541988416989,-0.06937448600263676,0.687653233268929,0.008178970646504463,-0.21765746176242828,-0.15971335768699646,0.3521364629268646,0.2307182401418686
--- old (absent)
+++ new aa18e899753a
@@ -0,0 +1,4 @@
+period,,,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014,1979-2014
+regression,,,gmst_over_time,gmst_over_time,gmst_over_time,gmst_over_time,sia_over_time,sia_over_time,sia_over_time,sia_over_time,sia_over_gmst,sia_over_gmst,sia_over_gmst,sia_over_gmst
+statistic,label,type,slope,r_value,p_value,std_err_slope,slope,r_value,p_value,std_err_slope,slope,r_value,p_value,std_err_slope
+CanESM5,unlabelled,model,0.032730051480051474,0.9330249991378756,1.1692052910042573e-16,0.0021646567943726627,-0.07238200812419562,-0.8716361335971559,4.573249078417207e-12,0.0069805243937325805,-2.0661001205444336,-0.8727891445159912,3.9593337289312025e-12,0.19815684854984283
--- old (absent)
+++ new c6f41e586979
@@ -0,0 +1,71 @@
+{
+ "data": {
+ "executions/recipe/work/antarctic/sea_ice_sensitivity_script/data_values.csv": {
+ "description": "",
+ "dimensions": {
+ "region": "antarctic"
+ },
+ "filename": "executions/recipe/work/antarctic/sea_ice_sensitivity_script/data_values.csv",
+ "long_name": "Annual (not decadal) figures"
+ },
+ "executions/recipe/work/arctic/sea_ice_sensitivity_script/data_values.csv": {
+ "description": "",
+ "dimensions": {
+ "region": "arctic"
+ },
+ "filename": "executions/recipe/work/arctic/sea_ice_sensitivity_script/data_values.csv",
+ "long_name": "Annual (not decadal) figures"
+ }
+ },
+ "diagnostics": {},
+ "html": {
+ "<OUTPUT_DIR>/executions/recipe/index.html": {
+ "description": "Page showing the executions of the ESMValTool run.",
+ "dimensions": null,
+ "filename": "<OUTPUT_DIR>/executions/recipe/index.html",
+ "long_name": "Results page"
+ }
+ },
+ "index": "<OUTPUT_DIR>/executions/recipe/index.html",
+ "metrics": null,
+ "plots": {
+ "executions/recipe/plots/antarctic/sea_ice_sensitivity_script/png/Annual Antarctic sea ice sensitivity.png": {
+ "description": "",
+ "dimensions": {
+ "region": "antarctic"
+ },
+ "filename": "executions/recipe/plots/antarctic/sea_ice_sensitivity_script/png/Annual Antarctic sea ice sensitivity.png",
+ "long_name": "Sensitivity of sea ice area to annual mean global warming."
+ },
+ "executions/recipe/plots/antarctic/sea_ice_sensitivity_script/png/Annual Antarctic sea ice trends.png": {
+ "description": "",
+ "dimensions": {
+ "region": "antarctic"
+ },
+ "filename": "executions/recipe/plots/antarctic/sea_ice_sensitivity_script/png/Annual Antarctic sea ice trends.png",
+ "long_name": "Decadal trends of sea ice area and global mean temperature."
+ },
+ "executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice sensitivity.png": {
+ "description": "",
+ "dimensions": {
+ "region": "arctic"
+ },
+ "filename": "executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice sensitivity.png",
+ "long_name": "Sensitivity of sea ice area to annual mean global warming.\nMean (dashed), standard deviation (shaded) and plausible (dotted) values from 1979-2014 are shown in grey."
+ },
+ "executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice trends.png": {
+ "description": "",
+ "dimensions": {
+ "region": "arctic"
+ },
+ "filename": "executions/recipe/plots/arctic/sea_ice_sensitivity_script/png/September Arctic sea ice trends.png",
+ "long_name": "Decadal trends of sea ice area and global mean temperature."
+ }
+ },
+ "provenance": {
+ "environment": {},
+ "log": "<OUTPUT_DIR>/executions/recipe/run/main_log_debug.txt",
+ "modeldata": [],
+ "obsdata": {}
+ }
+}
--- old (absent)
+++ new 4f53cda18c2b
@@ -0,0 +1 @@
+[] |
Tested against a real deployment on Gus, where 81 datasets are affected. One finding listing all of them wrapped into an unreadable blob, and the remedy told the reader to retract 'each of the rows above'. Findings sharing a remedy are already grouped under it once, so one per dataset reads as a list.
Merges the two identical key removal loops, because the split implied a distinction between the groups that does not exist. Prunes the recipe variables by what the solve supplies rather than by the _obs name suffix, so a variable that is neither solved nor named _obs no longer raises a KeyError. Addresses the CSV column index by level name, which drops a dependency on the placeholder names pandas gives unlabelled header levels.
Tested against a real deployment on Gus. pmp/enso_tel groups its reference requirement by activity_id, and the adapter now stamps that from the source type, so the reference data split into an obs4MIPs group and an obs4REF group and every model solved twice against half its references. 892 extra executions across the deployment. The merged rows stand in for obs4MIPs data, so they now carry that activity_id. Their instance_id still names obs4REF, so the provenance is not lost. Also catches the ValueError a diagnostic with no data requirements raises, and adds obs4REF to the aggregate test fixture so it can exercise the fallback.
…alog A deployment that fetched only the obs4REF registry is the ordinary case, and it took the branch that returned the obs4REF catalog untouched. Those rows kept an obs4REF activity_id, so a requirement grouping by it split or rejected them. Normalisation now happens on the one path every added row takes. Also normalises path separators before matching a collection directory.
Mid-upgrade the re-ingested obs4REF row and the old misfiled obs4mips row sit side by side, and every one was reported as superseded. That told the user to retract exactly the row misfiled-obs4ref asks them to keep. Only a genuine obs4MIPs publication counts as one now.
Contributor
It would be great if you could report that at https://github.com/ESMValGroup/ESMValTool/issues |
Bumps [mistune](https://github.com/lepture/mistune) from 3.3.0 to 3.3.3. - [Release notes](https://github.com/lepture/mistune/releases) - [Changelog](https://github.com/lepture/mistune/blob/main/docs/changes.rst) - [Commits](lepture/mistune@v3.3.0...v3.3.3) --- updated-dependencies: - dependency-name: mistune dependency-version: 3.3.3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
The lockfile moved to mypy 2.3.1, but the requirement floor still allowed mypy 1. This pins the floor to 2.3.1 so a fresh resolve cannot disagree with CI.
- The back link is derived from the label, not the removed `slug` field. - The report header now names the same base that `blob_url` builds from, so a local store's `file://` links and the header agree. - Drops the filter count from a docstring, which had already drifted once.
feat: render baseline diffs as a local HTML report
chore(deps): update dependency mypy to v2
A changed `.nc` file previously showed only a size delta and two download links, so a reviewer could not tell a renamed attribute from a numerical regression without opening both blobs by hand. - Adds `netcdf_diff`, which opens both sides with xarray and returns a diff of the ncdump-style header plus one `StatRow` per data variable. - Each row carries min, max, mean and NaN count on both sides, the largest absolute and relative difference, and the number of cells that differ. - NaN counts as equal to NaN, so a masked cell staying masked is not a change. - Rows where something moved are shaded. - Splits the old combined table into a NetCDF section and an "Other files" table. - Declares `xarray`, which the workspace previously only had by accident. - Wraps both tables so a wide one scrolls inside its card, and lets a long file name wrap. Decoding is turned off when opening, so a non-standard calendar or unit cannot fail the report. Stats are whole-array. Per-time or per-level breakdowns are deliberately not here.
- Adds `_variable`, so each side of a data variable is looked up once instead of once for its shape and again for its values. - Extracts `_fetch_pair`, which held the same four lines in `_diff_for` and `_netcdf_for` including the note precedence rule. - Drops the text-flavoured defaults on `_fetch_side`, because `limit` and `oversize` are a pair and a caller that passes one and forgets the other gets a wrong note. - Passes the base side's scale into `_compare` from the reduction `_summarise` already ran, so a large variable is not scanned a third time. - Replaces the three overlapping all-NaN guards in `_compare` with one, and drops the warning filter in `_summarise` that its size check had already made unreachable. - Narrows `_of_kind` to one kind, which is all any call site passes now. - Gives `AnalysedFile.netcdf` the same standing as `text`, with no default. - Adds a `dash` filter so the template no longer decides how an absent shape reads. No behaviour change. The `num` filter is annotated to admit the counts it already formats.
The header was a unified diff, so a reader saw three lines of context around each hunk and nothing else. That names what moved but does not say what the file holds, which is the thing you want when reviewing a baseline you have not seen before. A file whose header did not move showed no header at all. - The header listing now keeps every line and tags the ones that moved, so it doubles as the file's description. The `---`, `+++` and `@@` markers are gone with it. - Adds a Diff and Side by side toggle, so the two headers can be read one next to the other. - `NetcdfDiff` carries `header_old` and `header_new` for that second view, and `header_changed` says whether anything moved. - An unchanged header is marked as such on the summary rather than replaced by a note.
A shaded row said something in the variable changed but not what. Scanning a row of nine old to new pairs to find the one that shifted is work the report can do for the reader. - Adds `Pair`, which holds one statistic on each side and answers whether it moved. - `StatRow` now carries `shape`, `minimum`, `maximum`, `mean` and `nan` as pairs, which drops it from fifteen fields to ten. - The new half of a pair is bold when it differs, and the three diff columns are bold when any cell moved. - A value can be bold while reading the same as its neighbour, because the display rounds to six significant figures and the underlying floats still differ. Also carries an edit to the changelog fragment made outside these changes.
Three findings from the review, all in the statistics. - Keeps the stored dtype when reading a variable. Casting to float first meant an `int64` past 2**53 lost its last digit, so two adjacent values compared equal and a changed cell was reported as unchanged. The mean still accumulates in float64, so a float32 variable reads the same as before. - Reports no maximum difference when a cell moved between NaN and a number. The subtraction gives NaN there, so the finite maximum could read as 0 next to a non-zero cell count. - Bounds what one side may decode to. The fetch cap covers the stored blob, and a compressed file under it can still expand past what the job can hold.
…taset The fold used to take the obs4MIPs copy whatever the versions. A stale published copy would then beat a newer registry one and rerun every diagnostic on the wrong data. The newest version now wins, and obs4MIPs only wins a tie. The superseded-obs4ref doctor check reuses the same merge so the two cannot disagree. Also picks up the review findings: - collect_required_reference_data reads the declared fallback source types when locating a registry. - A primary catalog without an instance_id column is left untouched instead of raising. - The unsolvable-diagnostics finding joins filters with "and" rather than a semicolon. - Fixes a typo and the long lines added to the docs.
feat: show what changed inside a NetCDF baseline
Adds `regression-diff-report.yaml`, which builds the HTML diff of the baselines changed on a branch, uploads it to the reports bucket, and edits one sticky comment on the branch's open pull request. The mint workflow now calls it instead of posting a capped markdown comment itself. This removes `scripts/ci/mint_diff.py`.
Adds `--upload <prefix>` and `--comment-output <path>` to `ref test-cases diff`. The report is pushed to a public object store and the comment links into it, so the comment is one table row per changed case rather than the whole diff. - Adds `ReportStore` in core, a named-key sibling of the content-addressed `NativeStore`. - Keys are validated as safe relative paths, and every remote upload sets a content type. - Adds `ReportStoreConfig`, so the store is routed by `REF_REPORT_STORE_*` and its credentials are read from the environment at upload time only. - Reports live in their own bucket because an R2 token cannot be scoped to a prefix.
- `build_report_store` now validates its own routing, so a blank endpoint or bucket names `REF_REPORT_STORE_*` rather than the native store's variables. - The comment table is driven off the report's kinds, the way the index page already is, so a new file kind gets a column instead of being dropped. - One environment builder serves both templates, since only escaping differs between them.
`preflight` only wrapped botocore's `ClientError`, so `NoCredentialsError` (raised when neither the env vars, the profile nor boto3's default chain resolve anything) escaped the CLI's handler. A CI run with an unset secret is the likeliest way this fails, so it now gets the same operator-facing message as a rejected credential.
Matches the `ref-baselines-` prefix the existing `ref-baselines-public` bucket uses.
- Key validation now delegates to `safe_path`, the containment primitive the rest of the regression package already uses, so a symlink escape and a NUL byte are caught too. - One Jinja environment again. Escaping keys off the template name, so the markdown comment renders unescaped without a second environment that HTML could reach by mistake. - A bad `--upload` prefix now reports the prefix rather than advising a credential check.
A case label outside the key character class would have escaped the handler once the upload had started. It exits 1 with the rest now. The `--upload` prefix still gets its own message, because that one is a typed argument rather than something derived from the repository.
Pins the messages an operator sees when the local root is unwritable, when the bucket denies the request, and when the CLI cannot use the store at all.
The two stores differ only in how a key is chosen: the native store derives it from the content, the report store is handed one. Everything under that was duplicated, so this pulls it into `store.py` and leaves each class holding just its addressing rules. - `_preflight_store` replaces two near-identical `preflight` bodies. - `_write_config_from_env` replaces the credential lookup in both factories. - `S3WriteConfig` carries the env prefix it was resolved from, so it names the right variable in an error and the report factory no longer needs its own pre-checks. - `_StoreConfigProtocol` replaces the two copies, with the native one adding `cache_dir`. The native store picks up the report store's `BotoCoreError` handling as a result, so a missing credential there reports a message instead of a traceback.
chore(deps): bump mistune from 3.3.0 to 3.3.3
feat: publish the baseline diff report and write the PR comment
ci: post baseline diff reports from a dedicated workflow
Ingest obs4REF under its own source type
…ty-recipe * origin/main: (60 commits) refactor: share one transport between the native and report stores test: cover the report store's error paths fix: keep an unexpected key from tracebacking mid-upload refactor: tidy the report upload after review fix: name the reports bucket ref-baselines-reports fix: report an absent credential instead of a traceback fix: address the review of the report upload feat: publish the baseline diff report and write the PR comment chore: clean up ci: post baseline diff reports from a dedicated workflow fix: take the newest version when obs4MIPs and obs4REF both hold a dataset fix: measure netcdf differences without losing precision or missingness feat: emphasise the netcdf values that actually moved feat: show the whole netcdf header, with a side by side view chore: rename the changelog fragment to the PR number refactor: tighten the netcdf analysis after the cleanup reviews feat: add netcdf stats to the baseline diff report chore: add a changelog fragment for the mypy 2 bump docs: correct the stale references left by the refactors chore(deps): raise the mypy requirement to match the lockfile ...
Contributor
Regression baseline diff2 test case(s) changed against
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates the sea ice sensitivity diagnostic for the recipe and script changes in ESMValTool v2.15.
We now need to rewrite the input datasets and obs.
The diagnostic script renamed its output from
plotted_values.csvtodata_values.csvand restructured it.The metric names changed, but the values are equivalent.
annual_tas_trendgmst_over_time_slopeannual_siconc_trendsia_over_time_slopedirect_sensitivity_(notz-style)sia_over_gmst_slopedirect_r_valsia_over_gmst_r_valuedirect_p_valsia_over_gmst_p_valueOne upstream bug is worked around here. The v2.15 script registers its CSV in the provenance without the
.csvsuffix, so the output bundle points at a file that does not exist and the mint fails while collecting outputs.format_resultrestores the suffix.