Home / CT Resources / Research on Tool Validation and Assessment Design

Research on Tool Validation and Assessment Design

Research and publications by Insight Assessment senior researchers about tool validation and assessment design includes:

Facione NC, Facione PA. (1996). Assessment design issues for evaluating critical thinking in nursing. Holistic Nursing Practice,10(3):41-53. The assessment of graduating students' critical thinking skills and habits of mind challenges and rewards those who approach the task from a critical thinking perspective. The article identifies and discusses issues in the design of authentic assessments of critical thinking as an educational outcome predictive of competent professional judgment in nursing practice. Authentic assessment implies a multiple methods design that addresses the diverse contexts within which judgments must be made by professional nurses. Most important, it implies a concern for validity and reliability of measurement, selection of appropriate data points, and attention to a number of logistic and practical concerns.

          Abstract: Assessing critical thinking skills and the disposition to use them is crucial in nursing education and research. The CCTDI uses the Delphi Report's consensus definition of Critical thinking as the theoretical basis to measure the disposition toward critical thinking. Item analysis and factor analysis techniques were used to create seven attribute scales that grouped the Delphi descriptive phrases into larger, more unified constructs: Open-mindedness, Analyticity, Cognitive Maturity, Truthseeking, Systematicity, Inquisitiveness, and Self-Confidence. The initial reliability coefficients (Cronbach's alpha .90 overall and .71 -.80 for the seven internal scales) remained relatively stable when the 75-item instrument was administered to 1,019 additional college students (.90 overall, .60 -.78 scales). The instrument has subsequently been used to assess the disposition toward critical thinking in junior high school through the doctoral level. Psychometric research using the CCTDI and related instruments offers the potential of testing a number of interesting hypotheses regarding the attributes of mind which might contribute to improved clinical judgment and critical thinking in nursing.

Facione NC, Facione PA. (1996). Externalizing the critical thinking in knowledge development and clinical judgment. Nursing Outlook. 44(3):129-36. Multiple modalities can successfully measure learning outcomes and gains in critical thinking skills and dispositions, but for many of these measures (rubrics, rating forms, and qualitative measures of a variety of types) to be accurate it is critical to craft assignments and exercises to provide evidence of thinking process and not just a statement of the result of the thinking process.      

          Abstract: The assessment of graduating students' critical thinking skills and habits of mind challenges and rewards those who approach the task from a critical thinking perspective. This paper identifies and discusses issues in the design of authentic assessments of critical thinking as an educational outcome predictive of competent professional judgment in professional practice programs. The paper uses as its running example programs in nursing, but is applicable to programs in business, engineering, social work, teacher preparation, and other areas of professional practice.. Authentic assessment implies a multiple methods design which address the diverse contexts within which judgments must be made by professional nurses. Most important, it implies a concern for validity and reliability of measurement, selection of appropriate data-points, and attention to a number of logistical and practical concerns. Keywords: Outcomes assessment, critical thinking, portfolio assessment, testing, clinical judgment, professional judgment, nursing education.

The User Manual for each Insight Assessment test or measure, authored by the IA senior researchers, provides general information about the development and validation of that tool. For reasons related to the protection of proprietary intellectual and business property and for reasons of test security, detailed information is withheld. But, for example, in developing the CCTDI item analysis and factor analysis techniques were used to create seven stable disposition scales, which grouped the Delphi dispositional descriptions into larger, more unified constructs: open-mindedness, analyticity, cognitive maturity, truth-seeking, systematicity, inquisitiveness, and self-confidence. Cronbach's alphas for scales range.60 to .78 in sufficiently diverse samples and .90 to .92 overall. High CCTDI scores are associated with High scores on a measure of critical thinking skills (CCTST, HSRT, BCTST, TER, etc.) but lower scores on CT skills tend not to correlate strongly with CT disposition scores. Initial publication: Facione NC, Facione PA, Sanchez CA. (1994). Critical thinking disposition as a measure of competent clinical judgment: the development of the California Critical Thinking Disposition Inventory. Journal of Nursing Education, 33(8):345-50.

Facione, NC, Facione, PA, and Giancarlo, CA, (1994). The Disposition Toward Critical Thinking as a Measure of Competent Clinical Judgment: The Development of the California Critical Thinking Disposition Inventory, Journal of Nursing Education. Volume 33, Number 8, pp. 345-350.

          Abstract: Assessing critical thinking skills and the disposition to use them is crucial in nursing education and research. The CCTDI uses the Delphi Report's consensus definition of Critical thinking as the theoretical basis to measure the disposition toward critical thinking. Item analysis and factor analysis techniques were used to create seven attribute scales that grouped the Delphi descriptive phrases into larger, more unified constructs: Open-mindedness, Analyticity, Cognitive Maturity, Truthseeking, Systematicity, Inquisitiveness, and Self-Confidence. The initial reliability coefficients (Cronbach's alpha .90 overall and .71 -.80 for the seven internal scales) remained relatively stable when the 75-item instrument was administered to 1,019 additional college students (.90 overall, .60 -.78 scales). The instrument has subsequently been used to assess the disposition toward critical thinking in junior high school through the doctoral level. Psychometric research using the CCTDI and related instruments offers the potential of testing a number of interesting hypotheses regarding the attributes of mind which might contribute to improved clinical judgment and critical thinking in nursing.

Contact Us

CLIENT CONTACT US FORM

Name*

Email*

Phone

Subject

Message*

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.

IA Services

Insight Assessment recognizes the importance of ongoing independent research projects which will contribute to knowledge about how to improve human reasoning and decision making. We...
Elite Business Testing Service provide complete start-to-finish testing and score reporting. Insight Assessment manages test-taker logins/passwords, test assignments, and testing d...
Many Insight Assessment customers benefit from Learning Management Systems (LMS)  that integrate into their business or institutional processes. These can provide tight sec...
Processes and Policies to Achieve an Authorized TranslationThe California Academic Press (“The Press") supports rigorous, high quality, multi-modal assessment  researc...
We at Insight Assessment extend our thanks to the many scholars, professors, colleagues, and graduate students who have, over the years, collaborated with us to supply authorized tra...

Uses

Excellence in nursing education demands that the best candidates in the applicant pool should be enrolled and educated to be the best professionals possible. Deans of Schools of...
A university in Asia uses the translated language form of the health science test HSRT to assess the critical thinking skills of its nurse graduates. 
Military base uses the Health Science  Reasoning Test (HSRT) to ensure strong critical thinking skills in members of their Nurse Transition Program in which civilian nurses...
Patient simulators have been used as tools for training critical thinking and sound clinical judgment in nurse anesthesology students. The effectiveness of the simulator was document...
Hospitals are seeking funding to improve patient outcomes. An example is the Alemeda County Medical Center initiative to reduce severe sepsis mortality rates, funded by the Gordon an...