makes one feel so strong as a call for help.”
- George MacDonald
Decide Who Will Conduct the Evaluation
An important step in any evaluation is deciding who will
do it. Should it be the program staff or should outside consultants
Outside consultants are valuable because they will look at
your program from a new perspective and thereby provide you
with fresh insights. However, outside consultants do not necessarily
have to come from outside your organization. Evaluators within
your organization who are not associated with your program
and who have no personal interest in the results of an evaluation
may serve your needs. Here are some important factors to consider:
Professional training and Experience
The background experiences of evaluators can vary considerably.
Some evaluators specialize in quantitative methods, others
qualitative. Some have experience with one type of evaluation,
others with another type. Some consider themselves in partnership
with program staff; others see themselves as neutral observers.
Some have had formal courses in evaluation; others learned
evaluation on the job. Evaluators can even come from different
professional disciplines (e.g., psychology, mathematics, or
medicine). Find a consultant whose tendencies, background,
and training best fit your program’s evaluation goals.
Another factor to consider is the consultant’s motivation
(beyond receiving a fee). Consultants’ personal motivations
will affect their perspective as they plan and implement the
evaluation. For example, some consultants may be interested
in publishing the results of your evaluation and consequently
may shade results toward what they believe would interest
journal editors. Other consultants may be interested in using
the findings from your evaluation in their own research (e.g.,
they may be researching why certain people behave a certain
way). Find consultants whose professional interests match
the purpose of your evaluation. For example, if the purpose
of your evaluation is to ensure that the program’s written
materials are at the correct reading level for the people
you are trying to reach, find a consultant whose interest
is in producing data on which management can base decisions,
ideally one with experience in assessing reading level.
The following is a list of some areas consultants specialize
- Conducting basic research.
- Producing data on which managers can base decisions (data
may cover broad social issues or focus on a specific problem).
- Solving problems associated with program management.
- Increasing a program’s visibility to one or more
- Documenting the final results of programs.
Make sure the consultants you hire have experience in conducting
the evaluation methods you need, in evaluating programs similar
to yours, and in producing the type of information you seek.
Be sure to check all references before you enter into a contract
with any consultant.
Below is a list of the characteristics you should look for
in a suitable consultant. Note: Although these characteristics
are listed with some regard to order of importance, the actual
order depends on your program’s needs and the objectives
for the evaluation.
- Is not directly involved in the development or running
of the program being evaluated.
- Is impartial about evaluation results (i.e., has nothing
to gain by skewing the results in one direction or another).
- Will not give in to any pressure by senior staff or program
staff to produce particular findings.
- Will give staff the full findings (i.e., will not gloss
over or fail to report certain findings for any reason).
- Has experience in the type of evaluation needed.
- Has experience with programs similar to yours.
- Communicates well with key personnel.
- Considers programmatic realities (e.g., a small budget)
when designing the evaluation.
- Delivers reports and protocols on time.
- Relates to the program.
- Sees beyond the evaluation to other programmatic activities.
- Explains both benefits and risks of evaluation.
- Educates program personnel about conducting evaluation,
thus allowing future evaluations to be done in-house.
- Explains material clearly and patiently.
- Respects all levels of personnel.