Should I eliminate outliers from my sample before using my scores in other analyses?

‘Outlier’ is the term used to refer to a data point that is out of scale from the other data in the sample (and therefore possibly less valuable for describing the sample.  In the case of scores on the CCTST or other critical thinking skills tests at Insight Assessment, very high and very low scores are not simply outliers. These critical thinking skills tests are constructed to achieve accurate scores throughout the continuum. It is important to remember that there may be other theoretically justifiable and preferable reasons to eliminate scores from the calculation of the group mean, particularly when your data is being used to justify funding or to evaluate the effectiveness of a particular training program(because they are likely to be false scores). Two justifiable reasons to eliminate scores include 1) tests where less than 60% of the items have been answered (<60% response rate); and 2) time on test of less than 15 minutes (less than the known time needed to read and respond to test items). Either of these would be a justifiable and conservative criterion for determining lack of effort. Information about time on test and % of completed items is available to clients who test online.