substance abuse education Resources

Planning an Evaluation

(Adapted from Community How To Guide On Evaluation, National Association of Governors’ Highway Safety Representatives, 2001.)

• Write a statement defining the purpose(s) of the evaluation.  An unfocused evaluation cannot accomplish its intended goal.  A statement of purpose may be focused on one or more of the activities of the program.

• Determine budgetary and other resource needs.  The type of evaluation to be conducted will be determined by the amount of funding and other resources that are available.  For instance, if your organization is not able to fund an independent evaluation, finding a college or university to donate its services may be part of the plan.  If the plan calls for surveys or focus groups, determine whether there are a sufficient number of people in the organization or coalition to complete those tasks.

• Define the target population.  The target population may vary depending upon the objective of the evaluation.  For example, an evaluation aimed at determining whether your program is changing the attitudes and behavior of youth in your program will include all program participants.

• Write down the type of information to be collected.  A needs assessment conducted before the implementation of your program can form the basis of a baseline for evaluation efforts.  A baseline is essential to show actual improvement.  The information included in the needs assessment probably will include (1) demographics of the population your program is trying to reach and (2) information concerning its attitudes and behaviors concerning alcohol and drugs, including surveys, discussions, and news items.  Information included in pre-tests may be compared with the results of post-tests.

• Choose the type of evaluation to be used. 
  • Formative evaluation – Determines whether a program’s plans, procedures, activities, materials, and modifications will work as planned.
  • Process evaluation – Determines whether a program’s procedures are working as planned.
  • Outcome evaluation –- Determines a program’s progress toward immediate and intermediate goals.
  • Impact evaluation – Determines whether a program has met its ultimate goal of reducing substance use among youth.

• Determine what methods will be used for collecting the information. 
  • Quantitative Methods – Tracking the program (number of people who participated)
  • Qualitative Methods – Personal interviews, focus groups, participant observation.

• Collect the information and analyze the results.  It may be necessary to have a professional evaluator interpret the information and data.  The evaluator may be a college professor or graduate student, a state or local health department evaluator, or someone from a substance abuse prevention and treatment agency.  Other organizations such as advertising, marketing, and public relations agencies may be willing to contribute their expertise in conducting evaluations.

• Write an evaluation report describing the evaluation results.  The report may be simple or complex, depending upon the needs of your organization.  For example, if the evaluation is being used to justify further funding, the report should reflect the elements in the original funding request and should be in the form that is required by the funding agency or organization.