Both quality and quantity count. Go beyond logging when or how many activities are completed to determine the quality and reach of your program activities. Make sure you develop clear criteria for high-quality activities and products, and define what you mean by successful implementation.
Resources
TOOL
Measuring Dosage and Quality of Program Implementation (.doc 74 KB)
Learn about challenges and possible strategies for measuring quality and reach of activities.
SAMPLE MATERIAL
Criteria for Evaluating Professional Development (.pdf 155.7 KB)
Use this draft implementation rubric to prompt discussion of how to measure quality of professional development activities.
SAMPLE MATERIAL
Rubrics for Evaluating Teacher Lesson Plans (.pdf 1.3 MB)
Review two models for evaluating implementation quality that match teacher observations against self-reported data.
SAMPLE MATERIAL
Rubric for Magnet Student Classroom Performance (.pdf 167.1 KB)
Use this document to identify the degree to which students demonstrate specific learning and social behaviors related to the magnet curriculum.
SAMPLE MATERIAL
Professional Development Observation Protocol (.pdf 191.3 KB)
Use this observation protocol to document the quality of a professional development session.
SAMPLE MATERIAL
Checklist for Assessing Technology Integration (.pdf 226.6 KB)
Review this framework for assessing the extent to which a school’s staff integrate technology into curriculum and school infrastructure. Apply the ideas in this example to develop checklists for assessing integration of other magnet themes.
VIGNETTE
Documenting Treatment With Implementation Scales (.pdf 174.5 KB)
Reflect on this evaluator’s experience with measuring the degree to which various schools are engaging in treatment activities.
Extra Resources for MSAP Rigorous Evaluation
- If program implementation deviates from the original design, then the validity of any outcome or impact evaluation will be compromised. For example, change to dosage (number, type, and duration of activities) or low-quality implementation may have the same effect as having no treatment at all.
