Research on Tool Validation and Assessment Design
Research and publications by Insight Assessment senior researchers about tool validation and assessment design includes:
Facione NC, Facione PA. (1996). Assessment design issues for evaluating critical thinking in nursing. Holistic Nursing Practice,10(3):41-53. The assessment of graduating students' critical thinking skills and habits of mind challenges and rewards those who approach the task from a critical thinking perspective. The article identifies and discusses issues in the design of authentic assessments of critical thinking as an educational outcome predictive of competent professional judgment in nursing practice. Authentic assessment implies a multiple methods design that addresses the diverse contexts within which judgments must be made by professional nurses. Most important, it implies a concern for validity and reliability of measurement, selection of appropriate data points, and attention to a number of logistic and practical concerns.
Abstract: Assessing critical thinking skills and the disposition to use them is crucial in nursing education and research. The CCTDI uses the Delphi Report's consensus definition of Critical thinking as the theoretical basis to measure the disposition toward critical thinking. Item analysis and factor analysis techniques were used to create seven attribute scales that grouped the Delphi descriptive phrases into larger, more unified constructs: Open-mindedness, Analyticity, Cognitive Maturity, Truthseeking, Systematicity, Inquisitiveness, and Self-Confidence. The initial reliability coefficients (Cronbach's alpha .90 overall and .71 -.80 for the seven internal scales) remained relatively stable when the 75-item instrument was administered to 1,019 additional college students (.90 overall, .60 -.78 scales). The instrument has subsequently been used to assess the disposition toward critical thinking in junior high school through the doctoral level. Psychometric research using the CCTDI and related instruments offers the potential of testing a number of interesting hypotheses regarding the attributes of mind which might contribute to improved clinical judgment and critical thinking in nursing.
Facione NC, Facione PA. (1996). Externalizing the critical thinking in knowledge development and clinical judgment. Nursing Outlook. 44(3):129-36. Multiple modalities can successfully measure learning outcomes and gains in critical thinking skills and dispositions, but for many of these measures (rubrics, rating forms, and qualitative measures of a variety of types) to be accurate it is critical to craft assignments and exercises to provide evidence of thinking process and not just a statement of the result of the thinking process.
Abstract: The assessment of graduating students' critical thinking skills and habits of mind challenges and rewards those who approach the task from a critical thinking perspective. This paper identifies and discusses issues in the design of authentic assessments of critical thinking as an educational outcome predictive of competent professional judgment in professional practice programs. The paper uses as its running example programs in nursing, but is applicable to programs in business, engineering, social work, teacher preparation, and other areas of professional practice.. Authentic assessment implies a multiple methods design which address the diverse contexts within which judgments must be made by professional nurses. Most important, it implies a concern for validity and reliability of measurement, selection of appropriate data-points, and attention to a number of logistical and practical concerns. Keywords: Outcomes assessment, critical thinking, portfolio assessment, testing, clinical judgment, professional judgment, nursing education.
The User Manual for each Insight Assessment test or measure, authored by the IA senior researchers, provides general information about the development and validation of that tool. For reasons related to the protection of proprietary intellectual and business property and for reasons of test security, detailed information is withheld. But, for example, in developing the CCTDI item analysis and factor analysis techniques were used to create seven stable disposition scales, which grouped the Delphi dispositional descriptions into larger, more unified constructs: open-mindedness, analyticity, cognitive maturity, truth-seeking, systematicity, inquisitiveness, and self-confidence. Cronbach's alphas for scales range.60 to .78 in sufficiently diverse samples and .90 to .92 overall. High CCTDI scores are associated with High scores on a measure of critical thinking skills (CCTST, HSRT, BCTST, TER, etc.) but lower scores on CT skills tend not to correlate strongly with CT disposition scores. Initial publication: Facione NC, Facione PA, Sanchez CA. (1994). Critical thinking disposition as a measure of competent clinical judgment: the development of the California Critical Thinking Disposition Inventory. Journal of Nursing Education, 33(8):345-50.
Facione, NC, Facione, PA, and Giancarlo, CA, (1994). The Disposition Toward Critical Thinking as a Measure of Competent Clinical Judgment: The Development of the California Critical Thinking Disposition Inventory, Journal of Nursing Education. Volume 33, Number 8, pp. 345-350.
Abstract: Assessing critical thinking skills and the disposition to use them is crucial in nursing education and research. The CCTDI uses the Delphi Report's consensus definition of Critical thinking as the theoretical basis to measure the disposition toward critical thinking. Item analysis and factor analysis techniques were used to create seven attribute scales that grouped the Delphi descriptive phrases into larger, more unified constructs: Open-mindedness, Analyticity, Cognitive Maturity, Truthseeking, Systematicity, Inquisitiveness, and Self-Confidence. The initial reliability coefficients (Cronbach's alpha .90 overall and .71 -.80 for the seven internal scales) remained relatively stable when the 75-item instrument was administered to 1,019 additional college students (.90 overall, .60 -.78 scales). The instrument has subsequently been used to assess the disposition toward critical thinking in junior high school through the doctoral level. Psychometric research using the CCTDI and related instruments offers the potential of testing a number of interesting hypotheses regarding the attributes of mind which might contribute to improved clinical judgment and critical thinking in nursing.
Contact Us
* Required fields
Insight Assessment will not share your data with anyone. Click here to view our privacy statement.


