Repository navigation
ENH: Rename RandomEffectsResults.wu_hausman to hausman and return a n… - #761
Merged
Merged
Conversation
…amed tuple RandomEffectsResults.wu_hausman is the fixed effects vs random effects Hausman test added in #740. It has not been released, so it is renamed without a deprecation. The old name is easily confused with IVResults.wu_hausman, which is a different test and keeps its name. hausman now returns HausmanTestResults, a typing.NamedTuple with two fields: - statistic: a WaldTestStatistic, or an InvalidTestStatistic - estimates: the DataFrame with the columns b0, b1, b0-b1 and Std. Err. Unpacking still works, stat, estimates = re_res.hausman(fe_res), and the fields can also be read by name. The str and the repr show the statistic (value, p-value, distribution and null hypothesis) followed by the table of estimates. Unlike the repr of the test statistics, they do not end with the id of the object, so the result displays cleanly as the last value in a notebook cell: Hausman specification test H0: No systematic difference in coefficients between models Statistic: 7.1909 P-value: 0.0274 Distributed: chi2(2) Coefficients (b0: consistent under H0 and H1, b1: efficient under H0) b0 b1 b0-b1 Std. Err. exper 150.088403 149.344223 0.744180 2.117138 expersq -7.261883 -7.332176 0.070293 0.193418 HausmanTestResults is added to panel.results.__all__ and to the panel reference. The parameters of hausman are documented as optional. The method is excluded from the attribute access helper in the panel tests under its new name. The existing Hausman tests are renamed. The 9 new tests in test_results_hausman.py check that the old name is gone and that IVResults.wu_hausman is untouched, the type, the fields, indexing and unpacking, the str and repr for valid and invalid statistics, and that the results survive pickling. They fail on main, where neither the method nor the class exists.
…data hausman only checked that the two models have the same number of observations. Probing it on wage_panel with the probe scripts of the 7.1 release plan (probe_newfeat.py and probe_hausman2.py) found that it also accepted models that cannot be compared: - A weighted fixed effects model against an unweighted random effects model returned 118.8. The statistic is 112.1 when neither model is weighted and 109.6 when both are. The models then estimate different quantities, so the difference between their coefficients says nothing about the effects. A ValueError is now raised when the weights differ. The weights are compared after the rescaling to mean 1 that the models apply, so weights that only differ by scale are accepted. - A model of a different dependent variable (hours against lwage) was accepted. A ValueError is now raised when the dependent variables differ. The values are compared, not the names. - Models with the same number of observations but not the same observations, for example one that dropped the first entity and one that dropped the last, were accepted. A ValueError is now raised when the entities and periods of the observations differ. The probes print nan for the different dependent variable and for the random effects model against PooledOLS. That is InvalidTestStatistic.stat, which is always nan: both cases already returned an InvalidTestStatistic on main, with the reason that the statistic is negative. What was missing is the case where the statistic is not finite. A nan statistic is not less than 0, so a WaldTestStatistic with a nan statistic and a nan p-value was returned when a covariance or a coefficient is nan or infinite. hausman now returns an InvalidTestStatistic, with the reason that the coefficients or covariances are not finite or that the statistic is not finite, for these. A random effects model compared with a model that has time effects is still accepted, and this is now documented. RE has no time effects, so the null hypothesis that the test examines is then that the regressors are uncorrelated with the entity effects and with the time effects, and wage_panel gives 215.9 for FE with both effects against RE without time effects. Stata's hausman leaves this to the user. Rejecting it would also rule out the standard way to test the entity effects with year effects: include the year dummies in the random effects model and use the model with entity and time effects as other. The comparison then uses the coefficients that are in both models, and the tests cover this. The tests cover each ValueError, a renamed dependent variable, weights that differ by scale, nan and inf in the covariance and in the coefficients, the random effects model against PooledOLS, which is an InvalidTestStatistic on main and now, and the comparison with a model with time effects. 8 of the 12 tests that are added to test_results_hausman.py by this commit fail without the checks: the three tests of the weights, the different dependent variable, the different observations and the three tests of the nan and inf values. The other 4 describe behavior that does not change.
…is not valid
PanelResults.pesaran_cd returned a statistic for every model. The Pesaran CD
statistic is not valid for the models below, so pesaran_cd now returns an
InapplicableTestStatistic whose reason says why:
- BetweenOLS is fitted to the entity means, so it has no time-varying
residuals, and the series implied by its coefficients is not the model's
error.
- FamaMacBeth: the errors of the model are the residuals of the
period-by-period cross-sectional regressions. The series implied by the
averaged coefficients contains the common, time-varying term
(beta_t - beta_bar)'x_it. With coefficients that vary over time and
independent errors, the statistic of that series rejects in 72.5% to 100%
of the samples (mean statistic 4.6 to 51.8). The period-by-period
residuals sum to zero in every period and have the problem of time effects
below. Fama-MacBeth inference is also already robust to cross-sectional
dependence. With constant coefficients the statistic has the right size
(4.2% to 7.3%), so this exclusion rests on what the series is and not on
its size.
- Models with time effects. They demean the residuals across entities in
every period, which makes the residuals negatively correlated across
entities (Juodis and Reese, 2022, JBES 40(3)). With independent errors the
statistic is centered at about -sqrt(T/2) with a standard deviation of
0.03 to 0.13, so it rejects in 98% to 100% of the samples when T is 10 or
more and never rejects when T is 5.
- Models with other effects that change over time for some entity and have a
group of several entities that is missing from some periods. The panel
example notebook specifies time effects as other_effects=time ids, which
is the same model as time_effects=True. Industry-by-period effects have
the same problem, and so do blocks of periods, also when they are
combined with groups of entities (23% to 28% of rejections at T = 20 with
blocks of 2 periods, and a statistic that is biased downwards for
smaller T). Other effects that do not change over time for any entity,
such as industry or region, do not change the correlations over time. Nor
do effects that are present in every period, such as random groups, which
reject as often as the other models in the simulation, or groups with a
single entity.
The other models were checked by simulation, with independent errors,
y = 1 + 0.5 x + a_i + e_it, N = 20 and 100 entities, T = 5, 10 and 20
periods, 1,000 samples per cell (400 for the models with time effects) and the
5% test. Rejection rate in %, from scripts that are not part of this change:
N=20 N=100
T=5 T=10 T=20 T=5 T=10 T=20
PooledOLS 7.2 5.0 5.1 5.6 5.6 4.2
PanelOLS, entity effects 7.5 4.8 5.2 5.6 5.7 4.2
RandomEffects 7.2 4.9 5.2 5.6 5.6 4.2
FirstDifferenceOLS 7.3 4.8 5.4 5.3 5.8 4.1
PanelOLS, other = groups of
entities 7.3 5.0 5.2 5.7 5.6 4.3
PanelOLS, entity effects and
random groups 7.3 5.0 5.2 5.4 5.6 4.2
BetweenOLS 7.0 5.5 4.9 5.8 5.8 5.8
FamaMacBeth, constant coefs 7.3 5.7 5.4 5.7 5.6 4.2
PanelOLS, time effects 0.0 98.2 100.0 0.0 99.2 100.0
PanelOLS, entity and time 0.0 98.0 100.0 0.0 99.0 100.0
PanelOLS, entity and
industry-by-period effects 0.0 99.5 100.0 0.0 98.2 100.0
PanelOLS, entity effects and
blocks of 2 periods 2.5 4.2 27.5 1.0 4.5 23.5
The applicable models have the same size because the correlation of two
series over time removes anything that is constant over time, so the entity
effects and the constant in the shocks do not matter. The size is 4% to 6%
except for T = 5 and N = 20, where it is 7.2% to 7.5%. For pure noise the
size is 5.9% there, and 6.6%, 7.1% to 7.5% and 8.6% to 11.7% for T = 4, 3
and 2: the variance of the statistic is T / (T - 1) for series that are
demeaned, and the test needs more than a few periods. This is not changed.
Not changed, because the size under independent errors is acceptable, but the
docstring now says what is used:
- FirstDifferenceOLS uses the shocks in levels (idiosyncratic) and not the
residuals of the differenced equation. The size is 4.1% to 7.3% with
independent errors and 6.4% to 8.1% with the differenced residuals. With
random walk errors, for which first differences are efficient, it is 7.5%
to 28.0% for the shocks in levels and 3.8% to 7.8% for the differenced
residuals. (plm::pcdtest cannot be applied to an fd model: it fails with
"argument 1 is not a vector". Run on a pooled regression of the differenced
data it equals the statistic of the differenced residuals.)
- The weights of a weighted model are not applied to the shocks. The size is
3.6% to 7.9% for weights that vary by observation or by entity, and up to
9.8% when the weights vary over time in the same way for all entities.
Multiplying the shocks by the square root of the weights, which is what
plm::pcdtest does for a pooled model, gives 3.4% to 4.8% in that case.
- The statistic needs serially uncorrelated shocks. With random walk errors
every model that uses the shocks in levels rejects in 25% to 28% of the
samples at T = 20.
The docstring explains each case, with the references (Pesaran 2021, Juodis and
Reese 2022), and the memory use: the dense N by N matrices of correlations and
overlapping observations take about 450 MB when N is 4,000.
The tests check that the models above are applicable or not, the reasons, and
that the statistic of an applicable model is the one computed by explicit
loops over pairs of entities. test_pesaran_cd_all_estimators keeps the
estimators where the test applies. 26 of the 39 tests in
test_results_pesaran_cd.py fail on main, where every model that is not
applicable returns a NormalTestStatistic, and the other 13 pass on both.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| def _pesaran_cd_inapplicable(self) -> str | None: | ||
| """Reason why the Pesaran CD test is not valid for the model, if any""" | ||
| from linearmodels.panel.model import BetweenOLS # noqa: PLC0415 |
| from linearmodels.datasets import wage_panel | ||
| from linearmodels.iv.results import IVResults | ||
| from linearmodels.panel import PanelOLS, PooledOLS, RandomEffects | ||
| import linearmodels.panel.results |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #761 +/- ##
==========================================
- Coverage 99.58% 99.58% -0.01%
==========================================
Files 110 112 +2
Lines 20328 20730 +402
Branches 1600 1617 +17
==========================================
+ Hits 20244 20643 +399
- Misses 31 32 +1
- Partials 53 55 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…amed tuple
RandomEffectsResults.wu_hausman is the fixed effects vs random effects
Hausman test added in #740. It has not been released, so it is renamed
without a deprecation. The old name is easily confused with
IVResults.wu_hausman, which is a different test and keeps its name.
hausman now returns HausmanTestResults, a typing.NamedTuple with two fields:
Unpacking still works, stat, estimates = re_res.hausman(fe_res), and the
fields can also be read by name. The str and the repr show the statistic
(value, p-value, distribution and null hypothesis) followed by the table of
estimates. Unlike the repr of the test statistics, they do not end with the
id of the object, so the result displays cleanly as the last value in a
notebook cell:
HausmanTestResults is added to panel.results.all and to the panel
reference. The parameters of hausman are documented as optional. The method
is excluded from the attribute access helper in the panel tests under its
new name.
The existing Hausman tests are renamed. The 9 new tests in
test_results_hausman.py check that the old name is gone and that
IVResults.wu_hausman is untouched, the type, the fields, indexing and
unpacking, the str and repr for valid and invalid statistics, and that the
results survive pickling. They fail on main, where neither the method nor
the class exists.