ASSESSMENT
Beyond Average Scores: Looking at COVID Recovery across the Student Achievement Spectrum
Introducing a new interactive display to understand states’ NAEP score changes for higher- and lower-scoring students
By Katherine E. Castellano, Marisol Kevelson, Emily Kerzabi, Michael Fauss, and Anita Sands
Drops in student achievement since the COVID-19 pandemic endanger the advancement of society at large. Thoughtful data visualizations can help reveal how students are faring, and which are in the most need of additional support to be prepared for success after high school.
In April 2025, the National Assessment Governing Board (NAGB) released a NAEP data tool to explore how the nation and states are doing on the path to academic recovery using National Assessment of Educational Progress (NAEP) test scores. Figure 1 provides a snapshot of the tool. It highlights changes in average scores for a grade (grades 4 or 8) and subject (reading or mathematics) with bar charts. The data tool provides a summary of key data points, but as we showed in a recent ETS Research blog post, more complete stories of changes in student performance often hide behind average scores. Or as the mantra goes: “Averages can conceal more than reveal!”
Figure 1. Reproduction of NAGB’s pandemic-era score change bar chart for the nation in Grade 4 reading.
We were inspired to create an interactive display to look at score changes across the full achievement spectrum—because if a picture is worth a thousand words, an interactive display is worth so many more! In this blog post, we highlight what this new tool shows about pre-to-post pandemic change in student performance across all 50 states. Even in states at the top of the NAEP rankings, lower scoring students have more ground to make up than their higher scoring peers. We then review the functionality of our interactive display so you can find the next story in the data.
Figure 2. 2019 and 2024 Grade 4 NAEP reading score distributions for the nation.
In Figure 2, we use the national Grade 4 NAEP reading score distributions for 2019 and 2024 to show how reading scores changed from pre- to post-pandemic. For each year, we display 5 percentile points (available in the NAEP Data Explorer), showing the low (10th and 25th percentiles), middle (50th percentile), and high (75th and 90th percentiles) points of the student achievement spectrum.
Comparing the two score distributions tells us that since the pandemic, scores dropped more among lower scoring students than among higher scoring students. This is seen by the bottom bar representing the 2024 score distribution being wider than the one for 2019. It is shifted to the left for lower scoring students but is almost lined up with the 2019 score distribution for higher scoring students. We also make these observations:
- The lowermost 10 percent of students scored at or below 168 in 2019, but only at or below 158 in 2024—a drop of 10 score points.
- In contrast, the uppermost 10 percent of students scored above 266 in 2019 compared to almost the same score of 265 in 2024—a drop of only 1 score point.
- Using the full score distribution, rather than just looking at average scores, we see that the decline for lower scoring students is about 10 times larger than it is for higher scoring students.
- These differences in score changes across the achievement spectrum translate to the achievement disparity between low- and high-scoring students widening by 9 score points from 2019 to 2024.
Figure 3. Changes in Grade 4 NAEP reading scores for the nation from 2019 to 2024. The horizontal line at 0 distinguishes between positive and negative changes.
To more easily see the variations in score changes across the achievement spectrum, we introduce a variation on a chart type for interpreting trends. As shown in Figure 3, we combine the comparison of two distributions in Figure 2 to a single line.
Unlike typical line plots of assessment results that show assessment year on the horizontal axis, in our chart, we plot the 2019 scores at multiple percentile points on the horizontal axis, and the changes at those points from 2019 to 2024 on the vertical axis. We also include a horizontal line at zero to help identify when scores at certain points along the achievement spectrum increased from 2019 to 2024 (points above the zero line) versus those that declined (points below the zero line). Points that fall on or near the zero line indicate the same or near-same performance in 2024 as pre-pandemic in 2019. In other words, such points indicate that the 2024 grade 4 cohort are at pre-pandemic levels.
To help read the plot, we note that the first point of the line in Figure 3 is positioned:
- at 168 on the horizontal axis—corresponding to the 10th percentile in 2019—and
- at -10 on the vertical axis—corresponding to a drop in 10 points at the 10th percentile from 2019 to 2024.
The other points of the line in Figure 3 are determined similarly for each of the remaining percentile points: 25th, 50th, 75th, and 90th. Together, they form an upward, sloping line that tells the story about change across the achievement spectrum at a quick glance. The upward sloping line reveals that fourth graders in the United States experienced a larger loss in reading at the lower/left end of the student score distribution than at the higher/right end of the distribution. Thus, looking beyond the change in average scores reveals that losses varied across the achievement spectrum and the achievement disparity between low- and high-scoring students grew wider.
Figure 4. Changes in Grade 4 NAEP reading scores for the nation, Louisiana, Mississippi, and Massachusetts from 2019 to 2024. The horizontal line at 0 distinguishes between positive and negative changes.
In our previous ETS Research Blog, we looked at NAEP reading scores for three states: a state that has traditionally done well on NAEP—Massachusetts—and two states that have not but have enacted policies targeted at improving student reading levels—Mississippi and Louisiana. Figure 4 shows our single-line approach for these three states, juxtaposed against the nation overall.
In Figure 4, we see the following:
- The upward-pointing pattern, indicating widening gaps between lower and higher scoring students, is not unique to the nation. It is also apparent, to some extent, for all three states.
- However, the lines for Louisiana and Mississippi are much shallower or less steep, meaning that educational gains and losses were more similar between high and low performers. The line for Louisiana is above 0 and almost flat! This means that Louisiana was able to rebound scores to or beyond pre-pandemic levels for students across the achievement spectrum. Grade 4 students, regardless of academic need or level, are performing better in Louisiana in reading in 2024 than pre-pandemic in 2019! This result may reflect Louisiana’s reforms at work.
Mississippi’s reforms may also be continuing to work to support student needs:
- Mississippi’s line hugs the bold 0 line, indicating relatively comparable scores in 2024 to pre-pandemic 2019 levels across the achievement spectrum.
On the other hand, although Massachusetts had one of the highest average Grade 4 reading NAEP scores in 2024, we see that:
- Massachusetts has the steepest line showing the most distinct change across the achievement spectrum and a widening of the gap between low- and high-scoring students by 15 score points!
Figure 5. 2019 to 2024 changes in Grade 4 NAEP reading results for the nation, all 50 states, the District of Columbia, and the Department of Defense Education Assessment (DoDEA) from our interactive display. A horizontal line at 0 distinguishes between positive and negative changes.
Figure 5 shows the full view of all states from our interactive display for Grade 4 reading. It shows us that Louisiana is unique not just compared to the nation, Mississippi, and Massachusetts, but to all states! Most states have upward sloping lines like Massachusetts and lie completely or almost completely below 0. In other words, most states still have ground to make up to reach pre-pandemic Grade 4 reading score levels, especially for the lowest scoring students.
How do these results look for Grade 8 reading? What about in mathematics? How did your state fare? Try out the interactive display to find your story! Have your own assessment data? Try out this cross-time plotting method with them too!
How to navigate the display:
- Use the left-hand “Controls” panel to toggle between reading or mathematics and Grade 4 or Grade 8 scores.
- Use the left-hand “Controls” menu to toggle between viewing all states or showing selected states of interest. Hold down the “Control” key to select more than one state from the drop-down menu. Hold down the “Shift” key to select consecutively listed states.
- Pause over a point on a line to see a hover box appear with information about the data point, including the state name, selected percentile, and score change at the percentile.
- Take note of the horizontal position of the line to get a sense of baseline, pre-pandemic 2019 performance. For instance, in contrast to Virginia’s line, Puerto Rico’s line in the Grade 4 Mathematics plot is shifted much more to the left, indicating relatively low performance in 2019.
- Take note of the length of the lines to identify states with smaller or larger achievement gaps between lower- and higher scoring students in the baseline 2019 year. For instance, DoDEA has a rather short line for Grade 4 math versus the rather long line for Maryland. Low- and high-scoring students in DoDEA scored more similarly to each other in 2019 than in Maryland.
Katherine E. Castellano, Marisol Kevelson, Emily Kerzabi, and Anita Sands work on large-scale assessment research and Michael Fauss works on innovations in AI use for test security at the ETS Research Institute. Mari is an associate research scientist; Katherine is a principal research scientist; Emily is a research project manager; Michael is a research scientist; and Anita is a lead policy research analyst. The authors also acknowledge and thank Nicholas Munyan-Penney (Assistant Director of P-12 Policy at edtrust) for his thoughtful comments and review of this blog post.