does not take much strength to do things, but it requires
great strength to decide on what to do.”
-- Elbert Hubbard
Decide What to Evaluate
Programs come in all shapes and sizes, with vast differences
in their complexity and components. Depending on your purpose
and needs, evaluation can help identify the benefits of an entire
program, specific components of the program, or even activities
or services within a component. So, how do you decide which
aspects of your program need to be evaluated to achieve your
purpose? Sometimes, there is evidence of a problem in a specific
program area and you are seeking information to understand what
is wrong. Sometimes the decision is based on available resources.
Other times the decision is made for you, driven by the larger
organization or funders.
It is better to conduct an effective evaluation
of a single program component than to attempt
an evaluation of several components or an entire
program without sufficient resources.1
In order to decide what to evaluate, remind yourself of the
purpose, and then refer to your program’s logic model
(see The Power of
Proof: Setting the Stage). This table shows the questions
you should ask yourself to help you decide what to evaluate
for three different evaluation purposes and examples.2
|To improve the delivery
of the program
|| Are all program
goals (or outcomes or outputs) being achieved? (Refer
to your logic model. See Setting the Stage) If not, which
program goals are not being accomplished?
What inputs, activities, outputs and outcomes
lead to this goal?
activities to meet this goal in place?
Are the necessary inputs present?
Have there been any changes in inputs or activities?
Have any new activities been added
that are having an unknown impact?
formative information gathered for each activity?
|Suppose your program is designed to increase
the adoption of smoke-free workplace policies by employers
in your county. While you have made some progress, only
one employer has adopted a policy in the past two years.
You wonder whether there are any problems that are contributing
to this low adoption rate.
Some activities that occur as a part of your program are
contacting Employee Assistance Program (EAP) directors
by telephone to identify employers with an interest in
the issue, providing draft smoke-free environment policies
to these directors, and then meeting with the EAP directors
of interested organizations to discuss the policies.
You can choose to evaluate whether any one or all of these
activities is working. In addition, these activities require
resources for identifying and contacting EAP directors,
e.g., employer listings, telephone numbers, callers, copies
of the policies, copying equipment, and so forth. Again,
you can evaluate the availability of any or all of these
|To measure the program’s
|| Am I interested
in the overall effectiveness of the program, or am I interested
in a specific output, outcome, or impact?
Has sufficient time passed for the long-term
outcome to be achieved, or is it more appropriate to assess
outputs and short-term outcomes?
I need to know whether the activities that lead to the
output(s), outcome(s), or impact(s) of interest are effective,
or do I already have information about this?
|Let’s use the same example of a program
that is designed to increase the adoption of smoke-free
workplace policies by employers in a county. If the program
has only been in existence for two years, it may be too
early to assess the impact of the program upon lung cancer.
Instead, evaluating an outcome like the number of employers
who have adopted policies or an output such as the number
of EAP directors who have agreed to a visit will provide
early indications of program effectiveness.
|To demonstrate the effective use of resources
|| Am I interested in specific
resources, or resources overall?
the resources of interest link to particular activities
and outputs/ outcomes/impacts?
|In the program designed to increase the
adoption of smoke-free workplace policies by employers
in a county, recall that the activities included contacting
EAP directors to identify employers with an interest in
smoke-free environments, and to provide draft policies
to these persons. One of the resources used was volunteers,
who were used to make phone calls to EAP directors, copy
materials, and mail the materials to the EAP directors.
You want to demonstrate your effective use of these resources
to the volunteer organization. Since the volunteers were
specifically linked to the activities of making phone
calls, copying policies, and mailing materials, your evaluation
can focus on the outputs, outcomes, and impacts of these
1. Source: Child Outcomes Research and Evaluation
Team. (n.d.). How do you prepare for an evaluation? In The
program manager's guide to evaluation. Retrieved January
25, 2004 from the Administration for Children and Families
2. Source: Centers for Disease Control and
Prevention. (2001). Focus the evaluation design.
In Introduction to program evaluation for comprehensive
tobacco control programs (pp. 37-48). Atlanta, GA: the