Q: How is the Magnet Schools Assistance Program (MSAP) regular evaluation different from a rigorous evaluation?
A: MSAP regular evaluation informs the Department of Education that you have expended your magnet funds as planned and documents the implementation and your progress toward your magnet goals and objectives. There are required reporting forms (ED 524B) that are submitted directly to the DOE. MSAP rigorous evaluation is not required as part of MSAP funding. However, to encourage the use of impact studies of magnet programs, districts receive an invitational priority to include a rigorous (experimental or quasi-experimental) evaluation design in their MSAP grant proposals.
Q: What are common goals for MSAP evaluations?
A: Goals in MSAP evaluations are generally written around six purposes summarized as: 1) eliminating, reducing, or preventing minority group isolation; 2) giving all students an opportunity to achieve challenging academic content and achievement standards; and 3) demonstrating effective and sustainable implementation of the systemic school reforms. For example, a goal may be “75% of students at each magnet school will develop mastery of that curriculum, as determined by multiple performance measures including state assessments, portfolios, teacher checklists, etc.”
Q: What is “minority group isolation” and how does that relate to magnet schools?
A: On the Magnet Schools Assistance Program (MSAP) pages on the Department of Education’s website it states, “The term ‘minority group isolation,’ in reference to a school, means a condition in which minority group children constitute more than 50 percent of the enrollment of the school. See 34 CFR 280.4(b).” There are several objectives for MSAP funding but the primary purpose is “to eliminate, reduce, or prevent minority group isolation in elementary and secondary schools while strengthening students’ knowledge of academic subjects and their grasp of marketable vocational skills.” There are many mechanisms with, or without, MSAP funding to advance this mission.
Q: What is program evaluation and why should we do it?
A: Program evaluation is the systematic collection of data documenting the implementation and potential impact of a program. Program evaluation is undertaken to make judgments about the program’s merits as implemented and to make informed decisions about which aspects of the program should be sustained, revised, or eliminated.
Q: Who should be involved in the evaluation of our magnet program?
A: Representatives from many stakeholder groups can collaborate to support a magnet evaluation: school administrators, teachers, parents, students, community members, evaluators, program director, and key district personnel (e.g., data and accountability, research, or school reform offices). Involving stakeholders generates buy-in for evaluation efforts and will help focus the evaluation on the program’s goals and activities. Stakeholders should have the opportunity to express their goals for the project and understand how program outcomes and decision-making are connected to the evaluation. It may not be feasible or productive to have all stakeholders attend every meeting. As an evaluation team, you will need to determine the critical players and get the right people to the table as you plan meetings and activities.
