Critical Factors to Understanding the Impact of Covid-19 on Student Learning

As the United States embarks on the long road to recovery from the Covid-19 pandemic, we need to think critically about how we estimate the impact of the pandemic on students’ learning and how we effectively help those negatively impacted.

HumRRO continues to be a steadfast partner to our federal, state, district, and industry colleagues and an advocate for students. We have worked hard to earn a reputation for helping our clients address complicated issues when testing does not go as planned and we stand ready to help education systems come back stronger than before. Here are few key research areas of particular importance for those in the education sector grappling with Covid-19’s impact:

Gauging the impact of the Covid-19 pandemic on the students tested in 2021 is crucial to drawing appropriate inferences from 2021 test scores. Changes in the tested student population create challenges for maintaining trendlines and monitoring student subgroup performance. We need to understand the differences between the characteristics of the tested population of students and the untested population. Then we can use prior performance of both groups (tested and untested in 2021) to draw inferences about the group of students who were not tested in 2021. Such issues must be considered in all test score interpretations and considerations of longitudinal trends. During a recent conversation where a state partner asked us about their trend lines and post-equating data, we helped them consider differences in their tested 2020-2021 student sample versus pre-pandemic samples of students.

Comparability studies, such as between proctored and unproctored assessments in 2021 and between 2021 and 2019 test scores, are vital for ensuring accurate and reliable student ability estimates. When our state partners recently requested advice on their trend lines, we suggested a series of comparability studies to help them better understand their data.

Estimating the effectiveness of mode of instruction (virtual, in-person, hybrid) during Covid-19—overall and for student subgroups—is important as we put the positive lessons learned during the pandemic into practice globally. We know that collecting these data for all schools over time has been challenging. In addition, we expect an interaction between mode of instruction and other factors, such as students’ preferred learning style, home settings, and teachers’ facility with technology.

Estimating how the Covid-19 pandemic impacted special populations of students (e.g., English learners, students with disabilities) differentially from the rest of the student population is one key to creating effective targeted recovery strategies. It is vital that we attend to students who typically receive substantial supports when they attend in-person schooling so that we can gauge the level and effectiveness of supports in other instructional modes.

Setting appropriate goals for “getting back on track” post-pandemic is vital for establishing buy-in from educators, parents, and students. Attainable goals, effective support, and measurable outcomes are all keys to helping schools be better than they were pre-pandemic. HumRRO routinely conducts evaluation studies as well as psychometric analyses, and we advocate for a hybrid approach when considering progress monitoring steps that extend beyond summative testing.

Estimating the non-cognitive impacts of the pandemic on students and educators can help schools, districts, and states remove barriers to learning that might otherwise impede recovery efforts. Measures in the non-cognitive domain include online safety, learning environment, and students’ mental health. HumRRO is currently helping our clients modify indicators of the learning environment and school safety to address varying modes of instruction, which will be key for future recovery efforts.

Estimating the impact of financial hardship on educational performance for students who are newly impoverished, homeless, or otherwise negatively impacted by Covid-19 (or potentially other factors) will help us better understand how economics interacts with opportunity-to-learn and educational outcomes.

Effectively evaluating recovery efforts at the school, district, and state levels is crucial for giving educators and students the tools they need to succeed post-pandemic. Measures must be fair, accurate, reliable, valid, and actionable if we expect improvements in instruction, student support, and student learning.

You can rely on HumRRO for assistance with these and other research demands as you gauge Covid-19’s impact on your students and create policies to address those impacts. Our experience as scientists and as advisors to state education agencies and testing companies allows us to consider the best ways to approach data collection and research in educational settings, while never forgetting the important practical limitations our clients must adhere to. Our goal is to provide actionable advice and research findings that can inform practice, policy, and ultimately benefit students.