Decide Who Will Conduct the Evaluation

An important step in any evaluation is deciding who will do it. Should it be the program staff or should outside consultants be hired?

Outside consultants are valuable because they will look at your program from a new perspective and thereby provide you with fresh insights. However, outside consultants do not necessarily have to come from outside your organization. Evaluators within your organization who are not associated with your program and who have no personal interest in the results of an evaluation may serve your needs. Here are some important factors to consider:

Professional training and Experience

The background experiences of evaluators can vary considerably. Some evaluators specialize in quantitative methods, others qualitative. Some have experience with one type of evaluation, others with another type. Some consider themselves in partnership with program staff; others see themselves as neutral observers. Some have had formal courses in evaluation; others learned evaluation on the job. Evaluators can even come from different professional disciplines (e.g., psychology, mathematics, or medicine). Find a consultant whose tendencies, background, and training best fit your program’s evaluation goals.

Motivation

Another factor to consider is the consultant’s motivation (beyond receiving a fee). Consultants’ personal motivations will affect their perspective as they plan and implement the evaluation. For example, some consultants may be interested in publishing the results of your evaluation and consequently may shade results toward what they believe would interest journal editors. Other consultants may be interested in using the findings from your evaluation in their own research (e.g., they may be researching why certain people behave a certain way). Find consultants whose professional interests match the purpose of your evaluation. For example, if the purpose of your evaluation is to ensure that the program’s written materials are at the correct reading level for the people you are trying to reach, find a consultant whose interest is in producing data on which management can base decisions, ideally one with experience in assessing reading level.

Consultant Specialties

The following is a list of some areas consultants specialize in:

• Conducting basic research.
• Producing data on which managers can base decisions (data may cover broad social issues or focus on a specific problem).
• Solving problems associated with program management.
• Increasing a program’s visibility to one or more audiences.
• Documenting the final results of programs.

Make sure the consultants you hire have experience in conducting the evaluation methods you need, in evaluating programs similar to yours, and in producing the type of information you seek. Be sure to check all references before you enter into a contract with any consultant.

Below is a list of the characteristics you should look for in a suitable consultant. Note: Although these characteristics are listed with some regard to order of importance, the actual order depends on your program’s needs and the objectives for the evaluation.

Consultant Characteristics

• Is not directly involved in the development or running of the program being evaluated.
• Is impartial about evaluation results (i.e., has nothing to gain by skewing the results in one direction or another).
• Will not give in to any pressure by senior staff or program staff to produce particular findings.
• Will give staff the full findings (i.e., will not gloss over or fail to report certain findings for any reason).
• Has experience in the type of evaluation needed.
• Has experience with programs similar to yours.
• Communicates well with key personnel.
• Considers programmatic realities (e.g., a small budget) when designing the evaluation.
• Delivers reports and protocols on time.
• Relates to the program.
• Sees beyond the evaluation to other programmatic activities.
• Explains both benefits and risks of evaluation.
• Educates program personnel about conducting evaluation, thus allowing future evaluations to be done in-house.
• Explains material clearly and patiently.
• Respects all levels of personnel.