Data Collection Methods At-A-Glance

In the next few sections, we will provide you will a wealth of information about different data collection methods. But first, review this table. It contains a brief description of each method, and a list of advantages and disadvantages. It can help you decide what methods to focus on.

• more control over obtaining the data and
• the ability to clarify answers by probing matters that emerge in the course of the interview.

The chief disadvantage is cost and time, which can be substantial if the program is a large one. In many circumstances, interviewing may not be feasible at all.¹

This approach can reveal more about what actually takes place than a respondent might report in an interview. Also, observation might reveal valuable unexpected findings since there are no questions to restrict what is learned.

A major disadvantage of direct observation is that it is time-consuming for the evaluator. It can also be difficult to observe several things that occur at the same time, and the presence of an observer can alter events.

Questionnaires have the advantage that each person is answering identical questions, which increases the reliability of responses.² Questionnaires also permit control over the precise phrasing of questions.

Interviewer-completed questionnaires have the advantage that the interviewer is trained in the use of the instrument, which makes the administration easier. Also, the reading level of the respondent is not an issue, since the questions are read by the interviewer. An added advantage of telephone administration is that it allows a degree of anonymity.

The key disadvantage of face-to-face interviews is that, as with individual qualitative interviews, they can be costly and time-consuming. The key disadvantage of telephone interviews is that some groups of people are not likely to have telephones, and other are likely to block calls from unknown sources.

For more information about the advantages and disadvantages of different methods of administration, go to the Questionnaire section by clicking here.

As mentioned, questionnaires have the advantage that each person is answering identical questions, which increases the reliability of responses. Questionnaires also permit control over the phrasing of questions.

Respondent-completed questionnaires have the advantage of anonymity, and allowing the respondent to complete them at the time that is most convenient for them.

The key disadvantage of respondent-completed questionnaires is getting cooperation from respondents. There is often no incentive for completing the questionnaire and returning it on time. This can cause return rates to be very low. Another disadvantage is the possibility that the respondent may have a low level of reading ability. With web-based instruments, it is also possible for a respondent to complete the instrument more than one time.

For more information about the advantages and disadvantages of different methods of administration, go to the Questionnaire section by clicking here.

--------------------
1. Source: National Institute of Justice. (1989). Obtaining information for evaluations. In Evaluating drug control and system improvement projects: Guidelines for projects supported by the Bureau of Justice Assistance. Washington, DC: Prepared for the U.S. Department of Justice, National Institute of Justice by Abt Associates, Inc. (pp. 5-7).

2. Source: Office of Juvenile Justice and Delinquency Prevention. (1989). Special topics in program evaluation. In Evaluating juvenile justice programs: A design monograph for state planners. Washington, DC: Prepared for the U.S. Department of Justice, Office of Juvenile Justice and Delinquency Prevention by Community Research Associates, Inc. (pp. 40-41).