Community-based behavioral health agencies seek to promote positive outcomes for individuals they serve. However, many agencies lack the capacity to collect evaluation or continuous quality improvement data, evaluate the impact of services, or communicate effectively about data. This makes it difficult for agencies to institute data-driven program changes, retain funding in an era of increasing accountability, or compete for new funds. In addition, agencies that lack capacity for program evaluation may also not have staff with essential data-related skills to adequately implement evidence-supported practices that require ongoing monitoring of program processes, outcomes, and impacts.
The Building Agency Capacity for Program Evaluation initiative addresses these issues by enabling a minimum of four agencies in the Greater Philadelphia Area to receive evaluation training and consultation services for two years (July 2014 – June 2016) and participation in the newly established Greater Philadelphia Evaluation Learning Collaborative after June 2016. With the support provided by the Thomas Scattergood Foundation the initiative has been revamped to include more ongoing training for the existing grantees and to guarantee space for four new agencies to participate in this upcoming cycle. The Scattergood Foundation is actively working to expand the program to eight spaces with matching funds. The Scattergood Foundation will not know whether there will be additional funds and more than four spaces until after the deadline for the applications has passed.
Agencies selected to participate will receive training and consultation in practical program evaluation skills to track program processes and outcomes, and to communicate effectively about evaluation and quality assurance data. Training and consultation include:
a) an overview of program evaluation
b) introduction to logic models and their uses
c) helping agencies identify program evaluation needs, goals, methods, and measures appropriate for their program; and
d) assisting agencies to identify organizational capacities and barriers to program evaluation capacity and sustainability.
Unique features of this initiative are to tailor capacity building for program evaluation to the needs of individual agencies and to provide ongoing training and consultation to support changes in capacity over several years.
In the first two years (July 2014-June 2016), agencies selected to participate in this initiative, will be provided with a combination of group training and individual consultation as follows:
1) group training of four (4) half-days with up to 3 staff per agency in the Fall, Winter, and Spring;
2) individual consultation and technical assistance throughout the year for up to 8 staff to supplement group sessions for a total of three (3) individual sessions of up to two (2) hours each per agency; and,
3) ongoing phone and email consultation tailored to individual agency needs.
A combination of two group trainings and three individual on-site consultations are provided in the program’s second year.
After the end of the program’s second year, agencies will also be eligible to participate in the Greater Philadelphia Evaluation Learning Collaborative in which evaluation consultants conduct monthly online meetings with screen sharing (e.g., WebEx, Go-To-Meeting, etc.) that cover a range of evaluation topics that address specific evaluation challenges faced by former grantees. Online meetings would extend for up to 90 minutes and include opportunities for grantees to receive answers to specific questions that can then be followed up with email support. In addition, once each year, former grantees would be eligible to participate in a half-day of evaluation training hosted through the Evaluation Collaborative in which members can attend concurrent workshops to sustain capacity or build new skills.
Agencies of any size may apply to participate in this initiative because the emphasis will be on building agency capacity for one agency program. Agencies that submit an application will be expected to fully participate in all of the scheduled trainings and consultations.
Application Submission Requirements and Review Process
Agencies in the Greater Philadelphia Area seeking to participate in this initiative should submit their responses to each question outlined in the application. Applications must be accompanied by a cover letter from the agency director or senior leadership that indicates the agency’s commitment to full program participation.
Please email completed applications by 11:59PM Monday, July 28, 2014 to Alyson Ferguson at: firstname.lastname@example.org. Applicants will be notified of their acceptance on August 27, 2014. Agencies selected to participate will be expected to participate in the Evaluation Forum sponsored by the Foundation in mid-September 2014.
Review of applications will begin immediately upon receipt by Scattergood Foundation staff and the consultants retained for the initiative, Drs. Jacob Tebes, Joy Kaufman, and Cindy Crusto, and Ms. Amy Griffin, all from The Consultation Center, Yale University, who have provided training and consultation services for the initiative the past three years.