Extra Resources for MSAP Rigorous Evaluation

Conducting an MSAP rigorous evaluation provides a unique opportunity to gauge the impact of your program on student achievement and minority group isolation. There are some extra considerations the evaluation team should think about in setting up a rigorous evaluation. The additional resources provided here and for other practices will help you determine if this type of evaluation is appropriate for your district and, if so, to design and carry out the most rigorous evaluation possible.

Identify what you need from an evaluation

• An MSAP rigorous evaluation is an impact evaluation that uses experimental or quasi-experimental methodology to examine the extent to which changes can be attributed to the magnet program.

• It is especially important to start early when conducting an MSAP rigorous evaluation as it may require additional steps and data sources.

• “Rigorous” refers to the level of scientific rigor in the design and analysis conducted by the evaluators. It does NOT impose more rigorous or intense scrutiny on the district or the program itself.

VIGNETTE Making the Case for Rigorous Evaluation (.pdf 60.4 KB)

Read this fictitious conversation between an evaluator and a program manager to understand the additional information a rigorous evaluation provides about program impact.

---

SAMPLE MATERIAL Overview of MSAP Rigorous Evaluation (.pdf 205.2 KB)

Use this PowerPoint® presentation to provide basic MSAP rigorous evaluation information to stakeholders and staff.

---

VIGNETTE An Administrator Reflects on the Value of Rigorous Evaluation (.pdf 102.3 KB)

Review the ideas in this interview to generate compelling reasons for conducting a rigorous evaluation of your own magnet programs.

---

Partner with an evaluator who will meet your specific needs

• Find an evaluator with expertise in rigorous evaluation of educational programs. This helps to ensure that selection of comparison sites, outcome measures, and other design features are appropriate for the program being evaluated.

• The program director should identify and hire an evaluator at the earliest stage possible when preparing the grant application. This approach enables the evaluator to work with district staff to develop a sound evaluation design for appropriate outcomes using the most effective methods.

VIGNETTE Choosing an Evaluator for Rigorous Evaluation (.pdf 344.7 KB)

Check out the selection criteria and sample application used to determine the MSAP rigorous evaluator for one district's magnet program.

---

Create communication pathways between key staff and evaluators

• It is important to establish communication pathways with your evaluator early, preferably before your grant is submitted, so there is alignment between the planned project, desired outcomes, and the evaluation questions and data collection mechanisms.

• Facilitate communications between your MSAP regular evaluators and rigorous evaluators to ensure efforts aren’t duplicated or developed in contradiction to each other. It will be important to clarify the differences in these two types of evaluations as they affect expectations, roles, and responsibilities.

• Informing families, students, and teachers about what’s involved in a rigorous evaluation is essential. Take care to translate any “evaluator-speak” to explain the research design and its purpose.

SAMPLE MATERIAL Communication With Families About Experimental Design (.pdf 238 KB)

See how specialized communication to families about the use of experimental design can inform, reassure, and enlist families’ support for the evaluation process.

---