In our representative samples of 30,000 professional school applicants and students, scores for critical thinking skills have been dipping over the three years of the COVID-19 pandemic. Those using assessments of critical thinking as a component of admissions may have seen this too. Some people who have experienced COVID are talking about brain fog that continues to keep them from functioning well despite other symptoms subsiding. Whether the cause is long-COVID resulting in brain fog or acute stressors that prevent focused effort, performance has dropped.
Each year, in accordance with our privacy policy, Insight Assessment checks the published comparison percentiles used to benchmark scores for various population groups. Although score distributions vary by discipline, in most years overall critical thinking ability in the undergraduate admissions pool and applicants to professional programs remained remarkably steady. This change since 2020 is cause for concern for student learning. The dip in critical thinking scores across the population signals a significant fall in students’ ability to identify and frame problems, interpret relevant details, eliminate distractors, infer the meaning of change and its threat potential, evaluate potential solutions, and determine likely consequences.
Is it possible that this is just an artifact of some kind? We monitor item completion rate and time on test as surrogates for false low scores, and these values are unchanged. Remote testing is not the cause, as we have been offering online assessments globally for many years and know that scores are not affected by this factor.
Known factors that can cause false low scores on a cognitively demanding assessment include physical illness or fatigue, lack of sleep, and acute anxiety of other factors that create the inability to concentrate. False low scores are typically rare, but in the past few years, these potential threats have been greater due to increased illness and stress in the population at large. Given the known prevalence of COVID on campuses over the time frame where scores have dipped, illness factors may well be an explanation for some portion of this drop in performance.
The data shows people are making an increased number of common reasoning errors since 2020.
Historically there have always been a small number of college applicants who demonstrate weak critical thinking skills. And there have always been others who excel with strong and superior scores. Since 2020, the range of scores for this national aggregate sample is unchanged. But the proportion of scores at the moderate and weak levels has increased. This means that more individuals are making common human reasoning errors, getting the problem wrong, and struggling to draw a warranted inference when they should be able to do so with confidence. After years of stability, such a significant change requires explanation.
Our assessments focus on scale scores to permit educators and trainers to determine how to best design trainings for an observed gap in a particular individual or group. What we see are dipped scores across the various cognitive assessment areas: analysis, interpretation, inference, evaluation, explanation, induction, deduction, and numeracy.
Why suspect COVID is a Cause?
Many COVID sufferers who have been hospitalized have developed severe cognitive issues that persist at discharge and beyond. It is not too surprising to see cognitive impairment while a person is suffering from an acute viral infection, particularly when it is known to create fever and severe headache. What is more surprising are the reports of cognitive symptoms in cases where the infection itself is described as ‘mild’, continuing for months after the infection itself has resolved.
Nearly everyone who describes themselves as having COVID symptoms over several weeks mentions a disturbance in the ability to think well, difficulty with concentration, and in some cases short-term memory loss, some using the term brain fog. As we have had the opportunity to follow these people in the clinic, many deny experiencing mental health issues. Rather they describe their experiences as a difficulty focusing that interferes with their ability to work. They say that they become confused, or they feel unexpectedly disorganized when they try to focus on work or handle problems. Putting their thoughts into words is sometimes more difficult.
Clinical studies have documented forgetfulness, efficiency, and error-prone task execution.¹ These cognitive issues have been seen independent of the severity of symptoms at the time of infection, and to persist for many months or longer in a yet-t0-be-determined percentage of individuals.1,2 Most recently the concern for following the long-term effects of COVID was raised by a population study from the UK. In the group of 95,969 non-hospitalized young people (ages 18-29) a symptom cluster including brain fog was the most common manifestation of long COVID.2 Recurrent infection, in a relatively short time frame, is now a complication that will make the impact of COVID on reasoning capability and learning even more difficult.
Given the impact on learning and work performance, a more systematic investigation seems warranted. In July, the CDC warned that their electronic health data is recording long COVID symptoms in one in five Americans.³ It’s vital to understand and quantify COVID’s neurological footprint on work performance and educational achievement.
Hopeful Signs
Only in the past few months have we begun seeing improved scores in several discipline groups. Most significantly, scores on the HSRT for the health science disciplines have begun to rise significantly in the past 8 months. This is a hopeful sign that the changes we have observed are a fluctuation related to the pandemic and its influence on our collective cognitive ability to focus and think well, rather than some other unknown variable in the population.
What we can be sure of is that the reduced ability to identify problems and develop and monitor needed solutions is a major societal concern related to COVID infection. Medical professionals, first responders, and parents,… none of these can afford to have an impaired cognitive response to impending and existing problems or the failed management of emerging threats.
While neurologists have been leading the investigation into the prevalence of post-viral cognitive issues, they are not the group of professionals who will need to analyze the personal and societal impact of this problem. This work will fall on teachers, trainers, educators, and staff developers of all kinds.
The need to support students who demonstrate focal weaknesses in reasoning skills is not new to student success professionals. Monitoring the effectiveness of our trainees and working personnel is a familiar concern. Now one potential reason for observed failure to meet standards may be post-COVID cognitive issues that need to be acknowledged and addressed.
We will continue to monitor the group performance and provide information about the national population student groups to aid general decision-making for organizations that are assessing critical thinking as a component of admissions, training effectiveness, or hiring potential.
- Hellmuth, J., Barnett, T.A., Asken, B.M. et al. Persistent COVID-19-associated neurocognitive symptoms in non-hospitalized patients. Neurovirol. 27, 191–195 (2021). https://doi.org/10.1007/s13365-021-00954-4
- Subramanian, A., Nirantharakumar, K., Hughes, S. et al. Symptoms and risk factors for long COVID in non-hospitalized adults. Nat Med (2022). https://doi.org/10.1038/s41591-022-01909-w
- CDC. Post–COVID Conditions Among Adult COVID-19 Survivors Aged 18–64 and ≥65 Years — United States, March 2020–November 2021. CDC (2022). Post–COVID Conditions Among Adult COVID-19 Survivors Aged 18–64 and ≥65 Years — United States, March 2020–November 2021 | MMWR (cdc.gov)