How can I compare pretest to posttest scores in a small sample?

Examining the difference in scores from pretest scores to posttest in a small sample can be very challenging. It’s important to consider what can be validly determined.

When the samples contain the same individuals and there are about 25 or more individuals in the sample one can use the test of significance call a matched pairs t-test.

If there are fewer than 25, determining if there is evidence of a gain due to training is still possible by examining evidence of a change in scores by individual. For each student who took both the pretest and the posttest, calculate a difference score (value of the post test score minus the value of the pretest score) for OVERALL and for each of the scale areas.

This is usually the best was to interpret the effects of a training program in any pretest posttest sample, as it allows you to see the magnitude of the change in scores and display this in graphic format (frequency distribution of the individual difference scores). Individual gains of one point indicate improved reasoning (decreased vulnerability to common reasoning errors) and gains of several points are good evidence that the student benefited greatly from the training program. These gains in the OVERALL score correspond to an improvement of several percentile points when compared to the national norm sample ranking. The change in percentile ranking depends on where the student’s score falls in the curve.  Scores might drop within an individual if the individual is uncertain or inconsistent in how they approach a particular cognitive task, but a drop in scores of several points is usually indicative of poor effort at posttest (in the absence of some other explanation such as cognitive injury, fatigue or testing circumstances).

Insight Assessment testing instruments provide comprehensive group reports including individual and group percentile scores and nationally normed benchmarks.

Contact Us

Let our team of experts help you select the best assessment instrument for your project. You will hear from us within 1 business day.
    By selecting this box, you are agreeing to receive promotional messages from Insight Assessment. Terms and Privacy policy can be found at Insight Assessment does not share or sell personal data.
  • This field is for validation purposes and should be left unchanged.