Even the best-laid plans can fall victim to unanticipated events that might impact the validity of your study. Be aware of common issues that can potentially derail an evaluation, and specify procedures that will minimize negative effects. Build in checkpoints to address changing circumstances and document the reasons for these adjustments to ensure the findings can be interpreted appropriately.
Resources
TOOL
Managing Common Data Issues (.doc 171.5 KB)
Review these preventive strategies and remedies for common data issues.
SAMPLE MATERIAL
Conducting a Pilot Round of Data Collection (.doc 169 KB)
Use this list to identify how pilot testing can and should be incorporated into your data collection plan.
VIGNETTE
How We Handled Emerging Data Issues (.pdf 145.4 KB)
See how one district’s staff addressed common data collection issues at the outset of their evaluation.
Extra Resources for MSAP Rigorous Evaluation
-
By design, rigorous evaluations need data from an additional control or comparison group to determine the impact of a program. This means there is another set of issues to anticipate when planning data collection for these types of evaluations.
-
Make sure you receive data from most (if not all) of the participants in the magnet program, and the control or comparison sites. Uneven attrition from either group compromises the results of a rigorous evaluation. You may want to anticipate higher than average mobility rates for students when figuring out the number of participants you need to make the study work.
TOOL
Anticipating Data Collection Issues for Rigorous Evaluation (.doc 172.5 KB)
Learn how to address specific issues that may negatively impact findings of experimental or quasi-experimental evaluations.
VIGNETTE
Challenges of an MSAP Rigorous Evaluation for an Interdistrict Program (.pdf 175.9 KB)
Reflect on how one interdistrict evaluation team addressed challenges in conducting rigorous evaluation design.
