Compact Guide chapters 5, 6, & 7
It’s a new year and time for a new batch of chapters from A Compact Guide to Classical Inference. These chapters complete the setup for a dramatic development to come in the next release, where the simple formulation of statistical inference which will unify all the different settings found in the introductory course–difference between two means, difference between two proportions, slope of a regression line–into a pair of simple formulas and a test statistic F that directly handles both hypothesis testing and the construction of confidence intervals.
Chapters 1 through 4 brought us to the point where we can measure variation and model variation in the response variable as a function of explanatory variables.
Now, Chapter 5 uses the model function in a very simple way to find model values, the output of the model when the function is applied to the data used to construct the model. Comparing the variation in these model values to the variation in the response variable itself is fundamental to inference.
Computer-age inference wouldn’t use the same training data to fit a model and to evaluate its performance. Instead, separate testing data, held back from the fitting process, would be used. (Techniques like k-fold cross-validation make this efficient both in terms of data and computing.) But this is guide to classical inference and a hallmark of classical inference is to use training data for evaluation but correct for the consequent over-fitting by careful counting and adjustment.
This is where degrees of freedom come in. This is a difficult subject to teach because the concept is so abstract and counting degrees of freedom is far from obvious. In the Compact Guide, we engage in a reframing of the counting process, using degrees of flexibility which can easily be read directly from a simple graph of the model. Chapter 6 describes and motivates the concept.
The last of the new chapters, Chapter 7, completes the set-up for inference by introducing a standard, simple way of summarizing models: effect size. Effect sizes are emphasized in the American Statistical Association’s publications about a “post p < 0.05 world,” and they directly address the substance of the relationships being studied.
With these first seven chapters in hand, you’ll be ready for the inferential calculations and interpretations. But that will have to wait until Chapter 8!
Enjoy the new chapters and get ready for our next release in about two weeks. You can access the chapters at https://dtkaplan.github.io/CompactInference.