Extra Resources for MSAP Rigorous Evaluation

Conducting an MSAP rigorous evaluation provides a unique opportunity to gauge the impact of your program on student achievement and minority group isolation. There are some extra considerations the evaluation team should think about in setting up a rigorous evaluation. The additional resources provided here and for other practices will help you determine if this type of evaluation is appropriate for your district and, if so, to design and carry out the most rigorous evaluation possible.

Document implementation based on your logic model

• Remember to document implementation activities not just for magnet schools, but for control and comparison schools to ensure they aren’t engaging in similar program activities or treatment. It is important to budget time and resources to conduct this critical step in a rigorous evaluation.

• Make sure your MSAP regular and rigorous evaluators communicate with each other so they do not duplicate or contradict each other’s documentation efforts.

Measure quality of implementation activities

• If program implementation deviates from the original design, then the validity of any outcome or impact evaluation will be compromised. For example, change to dosage (number, type, and duration of activities) or low-quality implementation may have the same effect as having no treatment at all.

Select appropriate and practical instruments

• Choose instruments that help you measure implementation of program elements in comparison sites.

• Watch for new programs that may change conditions in comparison schools. Know ahead of time what changes.