Which of these is not like the other?
To isolate the impact of your program from other general educational efforts, you must establish and distinguish a unique treatment. Focus your outcome evaluation on aspects of the program you can tie directly to magnet objectives. Keep an eye out for comparison or control schools that may be engaging in magnet-like activities. Just because they aren’t labeled magnet schools doesn’t mean that they won’t be doing similar activities!
Resources
TOOL
Have We Clearly Defined Treatment Elements? (.doc 78 KB)
Use the table to define the essential elements of your magnet school program.
VIGNETTE
Documenting Treatment With Implementation Scales (.pdf 174.1 KB)
Find out how one researcher determined the extent to which a program treatment was unique to a particular school.
Extra Resources for MSAP Rigorous Evaluation
-
In choosing comparison groups for a rigorous evaluation, avoid schools that have programs similar to your magnets. Non-magnet schools may receive funding or reform mandates that result in similar curricular programs, especially in areas like arts or science.
-
For evaluations using random assignment, the assumption is the unique experience of the magnet school is the only major difference between the treatment and control groups. However, it’s still important to know what’s happening in the control group’s schools because if they have similar experiences, this may help account for the lack of differences in outcomes.
TOOL
Selecting Comparison Schools (.doc 86.5 KB)
Assess whether the program elements of a potential comparison school are different enough from your magnet program treatment to be effective for a quasi-experimental evaluation.
VIGNETTE
When a Comparison School Engages in Similar Treatment (.pdf 143.3 KB)
Reflect on common challenges and remedies related to comparison school selection.
