First, complete an Implementer’s Chart (IC) immediately after providing each lesson. The IC measures key skills and components of an effective Prevention Plus Wellness (PPW) program session. Use the IC to monitor the quality of your program implementation. A supervisor can also use the IC to assess a program implementer’s skills during a lesson observation.
Second, monitor program implementation fidelity and immediate pre-post outcomes by collecting a Feedback Form (FF) from each participating youth immediately after they complete their goal plan. The FF is designed to collect critical process data and post-program efficacy measures. By collecting and stapling the Screening Survey to the Feedback Form from each youth participant, these data can then be entered into the free Excel spreadsheet we provide to calculate average and difference scores on post-program data. Open-ended items on the FF provide qualitative data for future program marketing and improvements.
Third, use the Fidelity Checklist on a regular basis, for example every week or month, to ensure you are providing your PPW program with reliability. Evidence-based programs implemented with fidelity have a greater chance of providing positive outcomes reported in published evaluation research and listed on evidence-based databases.
Fourth, implement the PPW program behavioral instrument before the program is provided and 1-3 months post-program to determine pre-posttest behavioral changes. These measures include those addressed in the SPORT and InShape PPW programs, including physical activity, nutrition, sleep, stress control, and alcohol, tobacco and marijuana consumption.
Fifth, a simpler version of option number 4 is to follow up with participating youth 1-3 months post-program by reimplementing the screening survey. While the measures on the screening survey are somewhat limited, this strategy eliminates the need to collect a separate pre-test.
Sixth, a stronger evaluation of behavioral outcomes would be to include two groups of youth. One group receiving the PPW program first, and a second group of youth receiving the PPW program later. These two groups could be matched pairs of youth or youth groups which are similar demographically, or they could be determined by using a lottery or random assignment process. Again, pre-posttest measures could be collected as described in items 4 and 5, or existing data can be used which are available from grant requirements or local or state data collections.
The first three strategies are essential to monitoring the quality of program implementation and immediate outcomes, and as such should always be used. One of the later three strategies are important if your goal is to assess behavioral changes associated with your PPW program implementation. Some version of strategy number 6 is the strongest design because you are providing a comparison between a group of youth receiving the PPW program and one not yet given the program.
For more information about program monitoring, fidelity and improvement: https://preventionpluswellness.com/pages/program-monitoring-fidelity-improvement-support
For questions and help monitoring and evaluating your PPW program: (904) 472-5022, info@preventionpluswellness.com
Please share this important information with others in your region and state. Thank you.