Reflections on over 30 years of data
Leaders in business, the health professions, and the military have called for greater attention to critical thinking at all levels of their organizations.
Their focus is practical.
They rightly believe that stronger critical thinking will result in better leadership, decision-making, and problem-solving. And that these mean better business, healthcare, and military outcomes.
This very pragmatic bottom line focus on critical thinking creates demands for effective employee development programs, including assessments.
The accurate assessment of the reasoning skills and thinking mindset of workers in varying sectors required the development of custom instruments (to address their varying levels of decision responsibility). Expertise, developed through several decades in the measurement of critical thinking, and valuable input from professionals and acknowledged leaders in each sector, made it possible to create instruments tailored specifically for people who work in Business, Health Care, Defense, Science and Engineering, Law, and as Educators and First Responders.
This set of empirical assessments, the INSIGHT Series, is used by employers in hiring and for employee development. Today the INSIGHT Series directly targets the reasoning skills and mindset attributes relevant to executives, to professionals in different fields, and to specialized as well as general support staff.
Understanding the specific value of training and evaluating critical thinking has been a gradual process in educational settings. This is also true in business and government settings. It is particularly true in high stakes situations where judgment errors are costly, and where poor thinking skills and an inconsistent thinking mindset directly threaten organizational goals.
Critical thinking assessment & training started in education
As with education, corporate and government leaders have discovered that expecting strong critical thinking of management and staff is inadequate. There must be an explicit organizational focus, and a commitment of resources, to developing this expertise in their workforce. Assessments, coupled with training in critical thinking, are becoming more common as a component of professional development programs seeking to strengthen an existing workforce. The result of effective programs has been a transformation of the decision-making culture throughout an organization.
This is new territory for many businesses and agencies, but once data begins to accumulate, there is no further need to convince leadership that assessing and training critical thinking is worth the investment.
Critical thinking assessment is an important hiring tool
Including explicit assessments as part of the applicant screening process enables organizations to identify candidates for hire who demonstrate strong critical thinking. Having an assessment, in hand, at the time of a candidate interview is the preferred use of tools like the INSIGHT Business Professional.
However, reasoning skills and mindset are not the only components of a candidate’s application. At times, a hiring agent may favor the candidate who interviews well, even if that candidate demonstrates weak reasoning skills or a mindset that was averse to engaging difficult problems (or is lacking in other key mindset attributes). In some cases, these observations are used to set goals in training programs for the new hire.
Once the value of the assessment has become clear to an HR director–perhaps because new hire test scores are seen as consistent with subsequent supervisor ratings and department head observations–the next step is a roll out of a critical thinking initiative for all employee groups. In recent years, US government agency leaders have made strong improvements in their federal workforce groups using this approach.
Assessments of groups of employees can provide an organization with a picture of the critical thinking strengths and weaknesses of their workforce. In the next figure, mid-career professionals show strong and superior overall critical thinking skill in approximately two-thirds of the 3670 assessments. And, yet, that other third, those showing moderate and weak-moderate scores, can lead the organization’s leadership to have serious concerns. What quality of decision-making can the organization expect, if so many of its mid-career professionals display only moderate or weak-moderate reasoning skill?
A deeper dive into the data (produced by INSIGHT Business Professional) shows that the areas needing professional development include problem analysis and reasoning in precise contexts. Getting the problem right in the first place is, of course, a fundamental of strong critical thinking and decision-making. And being able to reason, with precision, is essential where applying regulations, laws, and quantitative information are elements in the job description.
The mindset to engage problems using strong reasoning skills is as important as having the skills themselves.
Consider the distribution scores representing the mindset characteristic of being judicious in this group of employees. The vast majority displayed a positive, or strongly positive inclination toward maturity of judgment and thoughtfulness in their approach to decision-making. But, out of the more than 3800 cases reported, close to 200 of these employed professionals fail to manifest judiciousness. Their responses suggest disagreement with the idea that some problems are ill-structured; the rejection of the notion that there may be more than one reasonable way to resolve problems; and that there are times when a person should revisit, and revise, poor decisions dispassionately.
When it comes to assessing and training critical thinking in a current workforce, it is vitally important to design initiatives that encourage optimal employee buy-in and participation. Being “voluntold” to participate in a program that assesses one’s reasoning will likely raise anxiety in employees. Such an assessment might easily be perceived as a threat to continued employment or a potential impediment to future promotion. Solutions that consider privacy and include focused options for personal development, typically, prove more successful. This is particularly true when the entire organization, including executives, participates. Also, when employees come to appreciate the power of critical thinking to bring benefits, not only at work, but in other areas of their lives, as well.
Compelling evidence that training improves critical thinking skills and mindset, long-term
Evidence of explicit critical thinking training having a long-term benefit is beginning to be found. In one highly notable, longitudinal study of working professionals conducted by the US government in relation to a United States Airforce (USAF) training program for nurses, the effectiveness of a leadership program was studied over an eight-year period. Critical thinking skills were assessed using INSIGHT Health Professional, a form of the academic instrument that is the Health Sciences Reasoning Test (HSRT).
The trainees were assessed at the beginning of the three-month training program (T1) and then again at the end of the training (T2). In this study, the nurses were provided their T1 scores at the beginning of the training to help them focus on areas where growth in their reasoning skill was most needed. The program was held at six training bases, each with a different training team and the same USAF training curriculum.
A matched pairs comparison of the change in scores from T1 to T2 demonstrated that the training significantly improved critical thinking skills in the trainees as a group and many improved exceptionally. These gains were observable at all six training sites.
This was true despite the observation that some portion of the trainees, at each training site, were dropping their scores at T2 due to insufficient focused effort. It was also noted that nurses who did relatively well at T1 were more likely not to fully engage in the retest. Although these false tests at T2 could be eliminated from program evaluation reports, the trainers worked hard in each subsequent annual cohort to stress the importance to seeking a second self-assessment in order to learn whether they were improving due to the importance of critical thinking for military leadership.
This 8-year USAF study is notable because it required a third assessment (T3) of these same personnel after one year of deployment at hospitals located on USAF bases around the world. Of great interest was the observation that scores at T3 continued to improve over T1 and T2. It has long been believed that the benefit of a training program, focused on improving reasoning skills, would have a continued impact as the learner integrates and practices what has been learned.
Said differently, once someone begins to watch their reasoning and their attitudes about reasoning (becomes metacognitive about their reasoning skills and mindset), and they perceive a benefit from this type of reflection, they become instrumental in their own continued development.
Most of us who train critical thinking believe this to be true, and this is why we believe that effective training of critical thinking will have enormous impact for an individual, perhaps, more so, if they had been introduced to reflecting on their reasoning process as a young child.
This post was excerpted and edited from the 2020 white paper by Peter A. Facione, Ph.D., Noreen C. Facione Ph.D., and Carol Ann Gittens, Ph.D., “What the Critical Thinking Data Tells Us.” Click on the white paper title to download the full PDF.