ISAT: Understanding p-values

I would like to respond to a recent letter by Mr. Robert Lem regarding Superintendent Critchfield’s characterization of Idaho’s 2025 ISAT results. Mr. Lem writes:

“Using a standard p-value of 0.05, any improvement in scores must show at least a gain of five percentage points in order to be considered statistically significant. So, for example, even the slight decline in 4th grade ISAT scores in Idaho (2024: 67.1%, 2025: 64.6%) is not significant.”

This reflects a fundamental misunderstanding of how p-values are used in statistical analysis. P-values do not operate as fixed percentage point thresholds, nor do they directly translate scores into confidence intervals to be compared in this way.

Moreover, applying inferential statistical methods, such as significance testing, is generally inappropriate when dealing with census data. The ISAT results are not derived from a sample; they represent the performance of every student in the selected grades statewide. When the entire population is measured, there is no sampling uncertainty to estimate, making the application of p-values both unnecessary and misleading.

For a reasonably thorough discussion of hypothesis testing and p-values, see Wikipedia’s entry on the topic. https://en.wikipedia.org/wiki/P-value

MATT K. PETERSEN

Coeur d’Alene