Why Six Rules?
Evidence-Based Programs (EBPs) are interventions with proven efficacy supported by high quality research. Most EBPs also have the advantage of an underlying conceptual model which helps understand the mechanisms by which a program influences substance abuse and health behaviors.
Regardless of the specific evidence-based or evidence-informed program or programs you choose, following a few simple rules during implementation will increase the likelihood of experiencing positive outcomes. Below are some common sense directives for ensuring that you implement your evidence-based program for maximum fidelity and success.
We highlight these critical rules because it has been our experience that they are commonly ignored during implementation. Each discounted rule increases the chances of experiencing dissatisfaction with the evidence-based program or its performance.
Rule #1: Read and follow the directions and protocols
While this would seem an obvious rule for success, it’s amazing how often directions are not read or followed, and protocols for implementation and evaluation are left unused or inappropriately adhered to. Evidence-based programs are efficacious when implemented under certain circumstances. Health and substance abuse specialists increase the probability of achieving positive effects by following directions and protocols provided by program designers and evaluators.
This is not to say you should never deviate from directions or adapt the protocols for your specific setting or population. Adaptation of some type is desirable and in fact necessary in certain cases to best address the unique circumstances of your particular setting and population. However, each alteration interjects an unknown factor during implementation.
We recommend you document any specific changes to the implementation procedures and protocols. In doing so, if problems arise with implementation or outcomes, you can look to these changes as possible reasons for experiencing setbacks.
Rule #2: Get some training
Some evidence-based programs mandate a certain level of training prior to use, others do not. Regardless, it is always a good idea to get some training to make sure you understand the components of the program, and steps for implementation and evaluation, including possible ways to customize the program to your unique situation.
Additionally, it is very helpful to understand the conceptual model underpinning the program, and the quality of research and outcomes reported by studies evaluating the program. Most important, however, is to receive coaching and feedback on how to implement the program with confidence, quality and fidelity.
With the advent of new online technologies and software, it is now easier, more convenient and less expensive to receive training than in the past. Consider taking advantage of web-based training options if you don’t have the time or resources for on-site training workshops.
Rule #3: Practice in advance
Practice makes perfect. We all know this. Even the most experienced of substance abuse and health specialists should practice to become familiar with a new evidence-based program.
Getting practice implementing the program will help you gain familiarity and become comfortable with the content and protocols. It will also increase your confidence to implement the program, and enhance your skills to present it in a natural manner.
You can get practice by role playing with other staff taking the role of participants, or implementing it with readily available samples of your target population. Don’t wait until the last minute to plan time to practice before launching your program full scale.
Rule #4: Run a pilot program before full implementation
It’s also essential to conduct a pilot run before implementing your new program large scale. The purpose of a pilot test is to identify potential problems or glitches that might arise during full scale implementation, causing you delays, cost-overruns or worse.
By running a pilot test prior to full implementation you will save yourself time, money and aggravation by identifying possible stumbling blocks in advance. It will also allow you time to make corrections before they can really hurt.
Run the pilot program as close to the real program as possible. This should include using all program implementation protocols, and collecting both process and outcome data to ensure all of your program components are being implemented with fidelity and your participants both like and report the program as effective.
Rule #5: Collect both process and outcome data
Once you launch your program full scale, continue to collect both process and outcome data. Process data should be monitored regularly (e.g., daily or weekly), to ensure the program is being implemented as it should, and participants are reacting positively to it.
Meanwhile, both short and medium-term outcome data should be collected. These data are designed to assess targeted health behaviors, behavior indicators like intentions or willingness to engage in key behaviors in the future, as well as risk and protective factors associated with the underlying conceptual model.
Regardless of what evidence-based program you use, it is important to monitor the ongoing quality of program implementation and program effects for your particular setting, implementers and population. By evaluating your program during and after implementation you’ll be armed with solid evidence that you are making a significant impact in the lives of your participants. This evidence is critical to receiving continued funding and support from key community stakeholders.
Rule #6: Make adjustments based on data collected
Even with practice and pilot testing, it may be necessary to make alterations to your program implementation, or even the program itself, if you find participants are not responding positively to it, and effects not as expected. It’s a good idea to contact the program designer to help you determine possible adjustments if you find your process or outcome data results are less than optimal.
By continuing to collect and regularly monitor data assessing the quality and effects of your program, you’ll have the information necessary to make changes to your implementation strategy or program. Tracking reactions to your program among participants, implementers and other key stake holders over time, and making changes based on your data, is critical to ensuring both short and long-term program success.
Conclusion
Choosing an evidence-based or evidence-informed program is the first step toward increasing the effectiveness of your substance abuse and health efforts. The second step is following the aforementioned six rules designed to ensure you will reap the full benefits of your program investment.
By following these six rules you will increase your chances of implementing any evidence-based program with fidelity and success. Omitting any of the rules will, on the other hand, increase the likelihood of running into problems with achieving your goals of having a positive experience with and results from your program.