Identify outcome measures and instruments

You need outcome measures that reflect the goals of the program. Select your instruments to ensure trustworthy and believable data. The best measures are specific, practical, and directly connected to your program objectives.

Resources

SAMPLE MATERIAL Reliability of Outcome Measures and Data Sources (.pdf 182 KB)

Ensure that appropriate measures with strong validity and reliability are selected for your evaluation.

VIGNETTE Using Direct Measures for Outcomes (.pdf 140.2 KB)

Reflect on this fictitious story to determine whether indirect measures in your evaluation could be replaced by direct measures.

SAMPLE MATERIAL School Climate Survey (.pdf 311.8 KB)

Use the questions from this survey, already field-tested for reliability and validity, to figure out how you might assess changes in school climate.

SAMPLE MATERIAL Student Survey of Noncognitive Outcomes (.pdf 580.4 KB)

Review this example of one evaluator’s approach to measuring noncognitive outcomes in a magnet program.

SAMPLE MATERIAL Checklist: Creating and Evaluating Student Assessment Items (.pdf 182.9 KB)

Use this information to develop student assessment items that meet professional standards and help you evaluate student learning outcomes.

SAMPLE MATERIAL Beyond Reading and Mathematics Test Scores: Arts- and Science-Related Measures (.pdf 450.1 KB)

Identify appropriate assessment measures for magnet themes related to science and the arts.

SAMPLE MATERIAL Evaluation Measures, Purposes, and Schedule (.pdf 249.3 KB)

See how one evaluation team determined measures aligned to program objectives and outlined a schedule for administering instruments.

Extra Resources for MSAP Rigorous Evaluation

Make sure you are familiar with the additional requirements for ensuring valid data in conducting an experimental or quasi-experimental evaluation.

TOOL Getting Key Items Right When You Measure Program Outcomes (.doc 88.5 KB)

Check if your evaluation uses key data collection approaches that produce valid evidence of program impact for a rigorous evaluation.

TIP

Choose outcome measures for assessing student learning that reflect your specific program goals and objectives. Program staff and the evaluator should discuss such items so they are valid (accurately reflect the thematic content) and reliable (provide accurate data).

PITFALL

Shortcuts are tempting, but don’t substitute indirect outcome measure when more direct measures are available. If your goal is to increase the number of students who take advanced science, for example, analyze student transcripts instead of reusing a student attitudes survey. Even if this increases data collection costs, the results will be more valid and useful to your program in the long run.

REMEMBER

Selecting appropriate measures can be difficult to get right. Try to avoid measurement systems that are overly complex (involve too many separate measures), too subjective, or miss key elements of the outcomes you desire.