Use techniques to ensure valid and reliable data

As the saying goes, “garbage in, garbage out.” If you use low-quality data, your final analysis and results will be useless, or even misleading. Remember that the most powerful statistical calculations will still yield poor results if the data being used aren’t valid or reliable in the first place. Safeguard your evaluation by using techniques that help to guarantee you have the right data to generate usable findings.

Resources

TOOL Making Sure Data Are Valid and Reliable (.doc 170.5 KB)

Use the questions and suggestions in this table to ensure the data you collect are valid and reliable.

TOOL Judging the Validity and Reliability of a Survey (.doc 170 KB)

Determine whether surveys proposed for use in your evaluation have been tested for reliability and validity.

TOOL Designing and Implementing a Survey That Generates a High Response Rate (.doc 172 KB)

Legitimize the survey results you collect by using techniques that ensure a healthy response rate.

Extra Resources for MSAP Rigorous Evaluation

Before you begin your data collection process, familiarize yourself with what constitutes valid and reliable data in the context of a rigorous evaluation.

TOOL Getting Key Items Right When You Measure Program Outcomes (.doc 89 KB)

Check if your evaluation uses key data collection approaches that produce valid evidence of program impact for a rigorous evaluation.

TIP

“ A good way to get 100% response rates from teachers is to administer the survey during a staff meeting. We’ve had success bringing them all to the computer room to complete the survey online. Getting the data this way is relatively easy and ideal. ”

—Phale Hale, Phale D. Hale Education Consulting, Inc.

REMEMBER

Different users have different standards for judging quality, which can impact data reliability. When discussing methods in your data collection planning process, make sure to surface and negotiate these differences.

REMEMBER

Credibility of the evidence and the perception of valid data are critical to an evaluation’s success. Without them, an evaluation’s findings will be considered useless by stakeholders.