|
|
|
|
|
|
|
Critical
thinking. Tests of critical thinking. Critical thinking tests. Critical
thinking skills. IQ. Tests. CT.
Custom
Designed
Dutifulness Scale
Campus
Culture Survey
Custom
Designed
Diversity and Openness Scale
|
Tool
and Response Form Design and Pilot Testing
Designing
an assessment or evaluation tool involves
- (1)
defining as clearly as possible what one wishes to measure,
- (2)
creating test or survey items (typically "agree-disagree"
or "multiple-choice") that responding in one way
would be evidence for the target idea and responding another
way would be evidence against the idea,
- (3)
pilot testing out those items with people who are very similar
to the population from which one wishes to gather data when
the tool is prepared for use,
- (4)
evaluating the performance of each of those items and scales
using appropriate statistical methods and, if possible, by
gathering the impressions and comments of those who took the
pilot test, and
- (5)
creating the final version of the assessment instrument by
eliminating poorly performing items, retaining good ones,
and perhaps making minor adjustments in others that appear
to be worth salvaging.
- (6)
Designing the response is a parallel process, and it includes
pilot testing in order to be sure that test-takers do not
make right or wrong answers only because of flaws in the layout
of the answer form.
Population
Sampling, Tool Production, and Survey Administration
Once a final version of the assessment tool and its
response form have been approved by the client, the instrument and
response forms pages are custom formatted, edited, produced, and
shipped in sufficient number for the intended administration. IA
specialists consult with the client on the conditions for the administration
of the instrument, including, if necessary, the training of proctors
and the means of accessing and surveying a sufficiently large and
representative sample of the target population. Issues of timing,
security, control of the test booklets and response forms, etc.
come into play.
Response
Form Scanning, Scoring, and Statistical Analysis
Response
forms are scanned so that the raw data can be automatically entered
into statistical analysis software. If appropriate, each individual
test-taker's score is recorded and compared to relevant norms or
standards. Group descriptive statistics, such as mean score, high
and low scores, standard deviation, are calculated. And relevant
inferential statistics, such as t-tests or correlations, are calculated.
The results are presented in numerical form and with appropriate
interpretive language and visual aids, such as bar graphs or pie
charts.
IA's
Custom Tool Design Capabilities
Design
of institution-specific tools
Quantitative and qualitative assessment strategies
Service and learning outcomes outcomes assessment
Complete statistical skill base for performing item analysis,
factor analysis, and power analysis
Comprehensive quantitative analysis of all collected data
Statistical analysis of all collected data
Statistical integration with other organization or institutional
data-factors
Assessment project written reports with presentation quality
graphics
Sustained support and collaboration for duration of project
Written project proposal and preliminary cost prospectus
Development and training programs for on-site personnel
Assistance with
the entire instrument development process
Pricing for Custom Tool Design Services
Custom services are negotiated with clients on a
case by case basis. Prices vary depending on the scope of the project
and the client's objectives, requirements, and time frame. Additional
factors include the identification of deliverables and the composition
of the IA consulting team required to respond to those objectives
and requirements. IA custom assessment and evaluation projects have
ranged in scope and cost from a few hundred dollars of consulting
support provided on a hourly fee basis by phone and e-mail for data
analysis and statistical support, to projects costing several thousands
of dollars and requiring several months to move through every phase
of tool design, production, data gathering, and analysis.
|
