| Data analysis and interpretation
can be difficult tasks, especially if you have large
amounts of data and limited time and resources. It’s
easy to take short cuts and make mistakes. Carter McNamara,
PhD, lists a few things to watch out for, and pitfalls
to avoid, as you go through this process in his article
entitled Analyzing, Interpreting and Reporting Basic
Research Results.1 |
|
|
- Don't balk at research because it seems far too
"scientific." It's not. Usually the first
20% of effort will generate the first 80% of the plan,
and this is far better than nothing.
- There is no "perfect" research design.
Don't worry about the research design being perfect.
It's far more important to do something than to wait
until every last detail has been tested.
- Work hard to include some interviews in your research
methods. Questionnaires don't capture "the story,"
and the story is usually the most powerful depiction
of the benefits of your products, services, programs,
etc.
- Don't interview just the successes. You'll learn
a great deal by understanding its failures, dropouts,
etc.
- Don't throw away research results once a report
has been generated. Results don't take up much room,
and they can provide precious information later when
trying to understand changes in the product, service
or program.
Other pitfalls to avoid include:
- Failing to match your analysis to your stakeholders
as well as your evaluation questions. Be
sure to consider what the stakeholders will want to
know, and how you can best communicate it to them.
- Treating all of your results equally.
As the person analyzing the data, you are in the best
position to know which results are the strongest and
the most valid. Highlight the most important results.
- Getting lost in complex analyses.
Always keep in mind the evaluation objectives. As
an evaluator you are not doing analysis for analysis’
sake. The analysis is a means to an end, program assessment
and improvement. Keep the analysis as simple as possible.
- Failing to protect confidentiality.
Sometimes, even without a name, a participant may
be identifiable by virtue of age, background, or opinion.
Be especially careful with results that represent
small numbers of people.
- Highlighting problems without suggestions
for solutions. Again, the data analyst is
in the best position to identify solutions to problems.
When you find problems with a program, it is important
to consider how to proceed in solving these.
- Failing to get input from all the
stakeholders. Sometimes what appears to be
a problem may NOT be a problem, and what appears to
be a strength may not be a strength. Remember that
those who are familiar with the various other aspects
of the program are usually able to provide fresh insight
regarding the findings.
For more about pitfalls of data analysis, read the
article Pitfalls
of Data Analysis (or How to Avoid Lies and Damned Lies)
written by Clay Helberg, M.S. from the Research Design
and Statistics Unit at the University of Wisconsin Schools
of Nursing and Medicine.
---------------
1. Source: McNamara, C. (1999). Analyzing,
interpreting and reporting basic research results.
Retrieved July 21, 2004 from The Management Assistance
Program for Nonprofits web site:
http://www.mapnp.org/library/research/analyze.htm |
