From a74a64ce1300bd545db715f2f0d40833e8c17f8d Mon Sep 17 00:00:00 2001 From: Kevin Sheppard Date: Sun, 11 Oct 2026 20:29:58 +0100 Subject: [PATCH 1/2] MAINT: Add the numpydoc lint configuration Add [tool.numpydoc_validation] to pyproject.toml, so that python -m numpydoc lint $(git ls-files "linearmodels/*.py") --config . checks the docstrings of the package. It needs numpydoc 1.10 or later, which added exclude_files. The result is the same with numpydoc 1.10.0 on Python 3.14 and numpydoc 1.11.0 on Python 3.10. This adds no CI step and changes no docstring: the sweep that fixes the violations is a separate change. What is checked - checks = ["all", "ES01", "SA01", "EX01"]: every numpydoc check except the extended summary, See Also and Examples sections. - Module docstrings are required (GL08 applies to modules). 43 modules do not have one, among them iv/__init__.py and iv/common.py. - linearmodels.test() has no docstring. It is public, so it is not excluded. What is skipped - exclude_files: tests, conftest.py, _version.py and _build/. The patterns accept / and \ and both relative and absolute paths. - exclude: special methods such as __init__ (the class docstring documents the constructor), functions nested in functions, and names with a leading underscore. Three groups of private names are still checked: the public members of private base classes, such as _IVModelBase.predict and _LSModelResultsBase.params, because the docs show them on the public subclasses; _OLS and _Panel, which the reference pages list; and LinearConstraint, which is defined in the private module system/_utility.py. Calibration against the docs scripts/npdoc_validate.py renders 872 objects. 856 of them have a docstring in the source, and they come from 467 distinct definitions. The lint checks all 467. The other 16 objects are class attributes (ALLOWED_KWARGS, DEFAULT_KERNEL) and the fields of the PanelModelData namedtuple. Their docstring is the one of a builtin type, so no change to the source fixes their 40 errors in that report. The lint also checks 54 module docstrings and 108 public names that the reference pages do not list, such as the helpers in shared/io.py and shared/typed_getters.py, the dataset loaders and the warning classes. Properties numpydoc lint reads the source and cannot see decorators, so it treats a property as a function. It asks for a Returns section (RT01) and for a summary that starts with an infinitive verb (SS05). That adds 228 RT01 and 31 SS05 violations for properties, whose docstring is a summary. override_RT01 and override_SS05 ignore both checks for docstrings without a Parameters section. The cost is that RT01 is not reported for methods without parameters. Six are known: PanelData.copy, HomoskedasticCovariance.deferred_cov and AttrDict.copy, items, keys and values. numpydoc validate, which imports the object, reports them. Declarations that the lint cannot satisfy need an inline comment: 8 property setters and @overload stubs (# numpydoc ignore=GL08) and the PanelModelData namedtuple, which documents its fields as parameters (# numpydoc ignore=PR02). Baseline 1,454 violations in 55 of the 56 linted files: | Subpackage | Files | Violations | |---------------|------:|-----------:| | iv | 9 | 537 | | panel | 6 | 391 | | system | 6 | 258 | | asset_pricing | 4 | 110 | | shared | 9 | 117 | | other | 22 | 41 | "other" is the top-level package, compat, datasets, typing, __future__ and formula.py. By code: SS03 451, PR09 407, GL08 134, RT05 132, PR01 65, SA04 62, PR02 37, PR04 30, GL07 27, PR08 25, SS01 24, PR10 16, RT02 14, and 5 or fewer for each of the other 10 codes. --- pyproject.toml | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/pyproject.toml b/pyproject.toml index f7b95e7f42..8525dac6fa 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -318,6 +318,50 @@ force_sort_within_sections=true include_trailing_comma=true use_parentheses=true +[tool.numpydoc_validation] +# Docstring lint. Run it from the repository root with +# python -m numpydoc lint $(git ls-files "linearmodels/*.py") --config . +# The rules are in "Docstring conventions" in doc/source/contributing.rst. +# Needs numpydoc 1.10 or later, which added exclude_files. +# +# Every check except the extended summary (ES01), See Also (SA01) and Examples (EX01). +checks = ["all", "ES01", "SA01", "EX01"] + +# Files that are not linted. The patterns are matched at the start of the path as it +# is given on the command line, and accept both / and \ as the separator. +exclude_files = [ + # Tests + '(?:.*[/\\])?tests[/\\]', + '(?:.*[/\\])?conftest\.py$', + # Generated and build files + '(?:.*[/\\])?_version\.py$', + '(?:.*[/\\])?_build[/\\]', +] + +# Objects that are not linted. The patterns are searched in the dotted name of the +# object, which is the file name without .py, the class and the member. For example, +# the name of IVGMM.fit in linearmodels/iv/model.py is model.IVGMM.fit. +exclude = [ + # Special methods, such as __init__. The class docstring documents the constructor. + '\.__\w+__$', + # Private functions, methods and properties, and everything defined inside them. + '\._[a-z0-9]', + # Private classes, except _OLS and _Panel, which the reference pages list. The public + # members of private base classes, such as _IVModelBase.predict, are linted because + # the documentation shows them on the public subclasses. + '\._(?!OLS$|Panel$)[A-Z][^.]*$', + # Private modules, except LinearConstraint, which the reference pages list. + '^_[a-z0-9][^.]*(\.(?!LinearConstraint(\.|$)).*)?$', + # Functions defined inside other functions or methods. + '^[^.]+\.(?:[^.]+\.)*_*[a-z][^.]*\.[^.]+$', +] + +# numpydoc lint cannot see decorators, so it asks every property for a Returns section and +# for a summary that starts with an infinitive verb. Ignore both checks for docstrings +# without a Parameters section, which are the docstrings of properties. +override_RT01 = ['\A(?![\s\S]*\nParameters\n-+\n)'] +override_SS05 = ['\A(?![\s\S]*\nParameters\n-+\n)'] + [tool.coverage.run] source = [ "linearmodels" ] branch = true From 54d86150e9ab4d567a9b30101faf1954f7389c09 Mon Sep 17 00:00:00 2001 From: Kevin Sheppard Date: Sun, 11 Oct 2026 20:29:58 +0100 Subject: [PATCH 2/2] DOC: Document the docstring conventions Add a "Docstring conventions" section to the contributing guide. It has - the nine rules for numpydoc docstrings: ", optional" on every parameter with a default, with the default in the description; unions written "int or str" and braces only for literal values; no subscripted types; what array_like, ndarray and "sequence of X" mean; bool, int, str and callable, and **cov_config and **weight_config with their stars and keys; standard section names in the standard order; concrete types in Returns; parameters that match the signature; and a separate BUG commit for a bug found while editing a docstring. - the command that checks docstrings. It needs numpydoc 1.10 or later. On Windows it needs Python in UTF-8 mode (python -X utf8): numpydoc reads the files with the system encoding, and the lint stops with a UnicodeDecodeError on the curly quotes in datasets/munnell/__init__.py without it. - what is enforced: every numpydoc check except ES01, SA01 and EX01, with the punctuation checks and SA04 named, and the rules that numpydoc cannot check (1 to 4, 9, the keys of rule 5 and the types of rule 7), which are for review. - the scope of the lint and its limits: module docstrings are required; what is skipped and what is not; why a property needs only a summary; and the inline comments for property setters, @overload stubs and NamedTuple classes. The page builds with Sphinx 9.1 with warnings as errors and nitpicky mode. --- doc/source/contributing.rst | 97 ++++++++++++++++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-) diff --git a/doc/source/contributing.rst b/doc/source/contributing.rst index 09e73b75a6..6ff1d1e9ba 100644 --- a/doc/source/contributing.rst +++ b/doc/source/contributing.rst @@ -22,4 +22,99 @@ linear models are always welcome. **Github** Fork the repo, make a branch with your new feature/documentation/typos -and submit a pull request. \ No newline at end of file +and submit a pull request. + +.. _docstring-conventions: + +Docstring conventions +~~~~~~~~~~~~~~~~~~~~~ + +Docstrings use the `numpydoc `_ +format. Every module, class, function, method and property needs one, unless its name +starts with an underscore. Apply these nine rules to every docstring that you write or +change. + +1. Put ``, optional`` after the type of every parameter that has a default. The default + value goes in the description, never in the type line. +2. Write a union of types as ``int or str``. Use curly braces only for a set of literal + values, written with plain quotes: ``{"robust", "unadjusted"}``. List every value that + the code accepts, aliases included. +3. Do not subscript types. Write ``list of str`` and ``dict of str to DataFrame``, not + ``list[str]`` and ``dict[str, DataFrame]``. +4. ``array_like`` means that the value is converted with ``np.asarray`` and then used as + an array. ``ndarray`` means that it must already be an array. ``sequence of X`` means + that it is only iterated over or indexed. +5. Use ``bool``, ``int``, ``str`` and ``callable``, not ``boolean``, ``integer``, + ``string`` or ``function``. Document ``**cov_config`` and ``**weight_config`` with + their stars, and list their keys. +6. Use the standard section names only, with underlines exactly as long as the header. + Write ``See Also``, not ``See also``. In order, the sections are ``Parameters``, + ``Attributes``, ``Methods``, ``Returns``, ``Yields``, ``Other Parameters``, + ``Raises``, ``Warns``, ``Warnings``, ``See Also``, ``Notes``, ``References`` and + ``Examples``. +7. A ``Returns`` section gives concrete types, such as ``Series`` or ``IVResults``. It + names what the function returns, not what it accepts. +8. Every documented parameter exists, and every parameter is documented, in the order of + the signature. +9. A real bug that you find while editing a docstring goes in its own ``BUG`` commit. + +For example: + +.. code-block:: text + + Parameters + ---------- + cats : DataFrame or array_like + Category codes of pandas categoricals, with shape (nobs, ncats). + output_format : {"csc", "csr", "coo"}, optional + Format of the sparse output. The default is ``"csc"``. + drop : {"first", "last"}, optional + Whether to exclude the first or the last category. The default is ``"first"``. + +Checking docstrings +^^^^^^^^^^^^^^^^^^^ + +``numpydoc lint`` checks docstrings. It needs numpydoc 1.10 or later. Run it from the +root of the repository. It prints one line for each violation, and exits with status 1 if +there is any violation: + +.. code-block:: bash + + python -m pip install "numpydoc>=1.10" + python -m numpydoc lint $(git ls-files "linearmodels/*.py") --config . + +In PowerShell, write ``(git ls-files "linearmodels/*.py")`` for the file list. To check +only some files, pass their paths instead. On Windows, also run Python in UTF-8 mode, for +example with ``python -X utf8``, because numpydoc reads the files with the system +encoding and some docstrings contain non-ASCII characters. + +The ``[tool.numpydoc_validation]`` table in ``pyproject.toml`` sets what is checked. + +* Every numpydoc check is enforced except ``ES01`` (no extended summary), ``SA01`` (no + ``See Also`` section) and ``EX01`` (no ``Examples`` section). The enforced checks + include the punctuation of summaries and of parameter and return descriptions + (``SS03``, ``PR08``, ``PR09``, ``RT04`` and ``RT05``), the description of every + ``See Also`` entry (``SA04``), parameters that match the signature (``PR01``, ``PR02`` + and ``PR03``), and the names and order of the sections (``GL06`` and ``GL07``). + Numpydoc does not check rules 1 to 4 and 9, the keys listed under rule 5 or the types + named under rule 7. Check them in review. +* Module docstrings are required (``GL08`` applies to modules), including the docstrings + of ``__init__.py`` files. One line is enough. +* The linter skips tests, ``conftest.py``, ``_version.py``, special methods such as + ``__init__`` (document the constructor in the class docstring), functions defined + inside other functions, and every name with a leading underscore. The exceptions are + the public members of private base classes, such as ``_IVModelBase.predict``, which the + reference pages show on the public subclasses, and ``_OLS``, ``_Panel`` and + ``LinearConstraint``, which the reference pages list. +* The linter cannot see decorators, so it asks for a ``Returns`` section and an infinitive + verb in the summary of every property. The configuration ignores both checks for a + docstring without a ``Parameters`` section, so a property needs only a summary. A + method without parameters that returns a value, such as ``PanelData.copy``, still needs + a ``Returns`` section, but the linter does not check it. + ``python -m numpydoc validate linearmodels.panel.data.PanelData.copy`` does; ignore + ``ES01``, ``SA01`` and ``EX01`` in its output. +* Some declarations cannot satisfy the linter. A property setter and an ``@overload`` + stub cannot have a docstring that is shown in the documentation: write + ``# numpydoc ignore=GL08`` on its ``def`` line. A ``NamedTuple`` has no ``__init__``, so + the linter reports its fields as unknown parameters: document them under ``Parameters`` + and write ``# numpydoc ignore=PR02`` on its ``class`` line.