Repository navigation
MAINT: Add the numpydoc lint configuration - #759
Merged
Merged
Conversation
Add [tool.numpydoc_validation] to pyproject.toml, so that
python -m numpydoc lint $(git ls-files "linearmodels/*.py") --config .
checks the docstrings of the package. It needs numpydoc 1.10 or later, which
added exclude_files. The result is the same with numpydoc 1.10.0 on Python 3.14
and numpydoc 1.11.0 on Python 3.10. This adds no CI step and changes no
docstring: the sweep that fixes the violations is a separate change.
What is checked
- checks = ["all", "ES01", "SA01", "EX01"]: every numpydoc check except the
extended summary, See Also and Examples sections.
- Module docstrings are required (GL08 applies to modules). 43 modules do not
have one, among them iv/__init__.py and iv/common.py.
- linearmodels.test() has no docstring. It is public, so it is not excluded.
What is skipped
- exclude_files: tests, conftest.py, _version.py and _build/. The patterns
accept / and \ and both relative and absolute paths.
- exclude: special methods such as __init__ (the class docstring documents the
constructor), functions nested in functions, and names with a leading
underscore. Three groups of private names are still checked: the public
members of private base classes, such as _IVModelBase.predict and
_LSModelResultsBase.params, because the docs show them on the public
subclasses; _OLS and _Panel, which the reference pages list; and
LinearConstraint, which is defined in the private module system/_utility.py.
Calibration against the docs
scripts/npdoc_validate.py renders 872 objects. 856 of them have a docstring in
the source, and they come from 467 distinct definitions. The lint checks all
467. The other 16 objects are class attributes (ALLOWED_KWARGS, DEFAULT_KERNEL)
and the fields of the PanelModelData namedtuple. Their docstring is the one of
a builtin type, so no change to the source fixes their 40 errors in that report.
The lint also checks 54 module docstrings and 108 public names that the
reference pages do not list, such as the helpers in shared/io.py and
shared/typed_getters.py, the dataset loaders and the warning classes.
Properties
numpydoc lint reads the source and cannot see decorators, so it treats a
property as a function. It asks for a Returns section (RT01) and for a summary
that starts with an infinitive verb (SS05). That adds 228 RT01 and 31 SS05
violations for properties, whose docstring is a summary. override_RT01 and
override_SS05 ignore both checks for docstrings without a Parameters section.
The cost is that RT01 is not reported for methods without parameters. Six are
known: PanelData.copy, HomoskedasticCovariance.deferred_cov and AttrDict.copy,
items, keys and values. numpydoc validate, which imports the object, reports
them.
Declarations that the lint cannot satisfy need an inline comment: 8 property
setters and @overload stubs (# numpydoc ignore=GL08) and the PanelModelData
namedtuple, which documents its fields as parameters (# numpydoc ignore=PR02).
Baseline
1,454 violations in 55 of the 56 linted files:
| Subpackage | Files | Violations |
|---------------|------:|-----------:|
| iv | 9 | 537 |
| panel | 6 | 391 |
| system | 6 | 258 |
| asset_pricing | 4 | 110 |
| shared | 9 | 117 |
| other | 22 | 41 |
"other" is the top-level package, compat, datasets, typing, __future__ and
formula.py. By code: SS03 451, PR09 407, GL08 134, RT05 132, PR01 65, SA04 62,
PR02 37, PR04 30, GL07 27, PR08 25, SS01 24, PR10 16, RT02 14, and 5 or fewer
for each of the other 10 codes.
Add a "Docstring conventions" section to the contributing guide. It has - the nine rules for numpydoc docstrings: ", optional" on every parameter with a default, with the default in the description; unions written "int or str" and braces only for literal values; no subscripted types; what array_like, ndarray and "sequence of X" mean; bool, int, str and callable, and **cov_config and **weight_config with their stars and keys; standard section names in the standard order; concrete types in Returns; parameters that match the signature; and a separate BUG commit for a bug found while editing a docstring. - the command that checks docstrings. It needs numpydoc 1.10 or later. On Windows it needs Python in UTF-8 mode (python -X utf8): numpydoc reads the files with the system encoding, and the lint stops with a UnicodeDecodeError on the curly quotes in datasets/munnell/__init__.py without it. - what is enforced: every numpydoc check except ES01, SA01 and EX01, with the punctuation checks and SA04 named, and the rules that numpydoc cannot check (1 to 4, 9, the keys of rule 5 and the types of rule 7), which are for review. - the scope of the lint and its limits: module docstrings are required; what is skipped and what is not; why a property needs only a summary; and the inline comments for property setters, @overload stubs and NamedTuple classes. The page builds with Sphinx 9.1 with warnings as errors and nitpicky mode.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #759 +/- ##
=======================================
Coverage 99.58% 99.58%
=======================================
Files 110 110
Lines 20328 20328
Branches 1600 1600
=======================================
Hits 20244 20244
Misses 31 31
Partials 53 53
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add [tool.numpydoc_validation] to pyproject.toml, so that
checks the docstrings of the package. It needs numpydoc 1.10 or later, which
added exclude_files. The result is the same with numpydoc 1.10.0 on Python 3.14
and numpydoc 1.11.0 on Python 3.10. This adds no CI step and changes no
docstring: the sweep that fixes the violations is a separate change.
What is checked
extended summary, See Also and Examples sections.
have one, among them iv/init.py and iv/common.py.
What is skipped
accept / and \ and both relative and absolute paths.
constructor), functions nested in functions, and names with a leading
underscore. Three groups of private names are still checked: the public
members of private base classes, such as _IVModelBase.predict and
_LSModelResultsBase.params, because the docs show them on the public
subclasses; _OLS and _Panel, which the reference pages list; and
LinearConstraint, which is defined in the private module system/_utility.py.
Calibration against the docs
scripts/npdoc_validate.py renders 872 objects. 856 of them have a docstring in
the source, and they come from 467 distinct definitions. The lint checks all
467. The other 16 objects are class attributes (ALLOWED_KWARGS, DEFAULT_KERNEL)
and the fields of the PanelModelData namedtuple. Their docstring is the one of
a builtin type, so no change to the source fixes their 40 errors in that report.
The lint also checks 54 module docstrings and 108 public names that the
reference pages do not list, such as the helpers in shared/io.py and
shared/typed_getters.py, the dataset loaders and the warning classes.
Properties
numpydoc lint reads the source and cannot see decorators, so it treats a
property as a function. It asks for a Returns section (RT01) and for a summary
that starts with an infinitive verb (SS05). That adds 228 RT01 and 31 SS05
violations for properties, whose docstring is a summary. override_RT01 and
override_SS05 ignore both checks for docstrings without a Parameters section.
The cost is that RT01 is not reported for methods without parameters. Six are
known: PanelData.copy, HomoskedasticCovariance.deferred_cov and AttrDict.copy,
items, keys and values. numpydoc validate, which imports the object, reports
them.
Declarations that the lint cannot satisfy need an inline comment: 8 property
setters and @overload stubs (# numpydoc ignore=GL08) and the PanelModelData
namedtuple, which documents its fields as parameters (# numpydoc ignore=PR02).
Baseline
1,454 violations in 55 of the 56 linted files:
"other" is the top-level package, compat, datasets, typing, future and
formula.py. By code: SS03 451, PR09 407, GL08 134, RT05 132, PR01 65, SA04 62,
PR02 37, PR04 30, GL07 27, PR08 25, SS01 24, PR10 16, RT02 14, and 5 or fewer
for each of the other 10 codes.