Our Evaluation Methods

RiseVT uses an evidence-based model to implement, measure, and evaluate our statewide programmatic work. We draw upon a variety of evaluation methods to ensure our program is consistently implemented across the state and that we are continuously monitoring our impact on the health of Vermonters.

Mid-and-Long-Term Population Health Data Trends

RiseVT has selected key population health indicators that are available for all Vermont counties to monitor mid- and long-term health trends. We compare local and state data to help us identify areas of both strength and need to guide our local programming and ensure our organization and partners are highly effective. These indicators include self-reported measures around nutrition and physical activity, food insecurity rates, tobacco usage, and health outcomes, including rates of obesity, diabetes, heart disease, chronic conditions, and cholesterol levels. These data are curated from the Youth Risk Behavior Survey (YRBS), the Behavioral Risk Factor Surveillance System (BRFSS), and Map the Meal Gap. This data is collected bi-annually into state and county data snapshots.

Key Informant Interviews to Assess RiseVT Program Impact

Our statewide team conducts biannual key informant interviews with RiseVT Program Managers to evaluate alignment with the Centers for Disease Control and Prevention’s Strategies for the Prevention of Obesity and Overweight. These interviews are conducted using a secure data capture platform known as RedCap. The interviews assess projects type, ages served, partnerships, level of change (policy, infrastructure, program), evidence-based model implementation, and alignment with obesity prevention best practice. The interviews also serve as opportunity for the Program Advisor and the Program Design and Implementation Manager to provide ongoing technical assistance around program implementation and fidelity.

Height & Weight Measurement Study

RiseVT conducts a biennial longitudinal measurement study surveilling the BMI of 1st, 3rd, and 5th graders in 20 elementary schools in 2 pilot counties. This effort began in the fall of 2017 when RiseVT collected height and weight measurements for over 1,700 children in Franklin and Grand Isle counties. In the first round of this measurement effort data indicated that over 40% of the children in pilot communities were overweight or obese, far higher than the self-reported data from most recent the Youth Risk Behavior Survey data.  In the fall of 2019, RiseVT conducted a second round of measurement, capturing BMI on over 1,700 schoolchildren. Find here the results of this second wave of measurement released in January 2020.

Dose Calculation

RiseVT is applying a dose calculation to quantify the intensity of RiseVT programming. This involves calculating the intensity score of unique RiseVT interventions using the Community Programs and Policy Index scoring methodology developed through the Healthy Communities Study (Fawcett et al., 2015). Dose provides a quantifiable way to measure community programs and policies by assigning an intensity score developed using duration, reach, and strength of an intervention.  Key Findings from the Healthy Communities Study indicate that:

  • An average BMI difference of 1.4 kg/m2 was observed between communities with the highest and lowest observed Community Programs and Policy Index (CPPI) intensity scores;
  • Higher intensity community programs implemented across multiple sectors is associated with lower BMI in communities;
  • Programs targeting a greater number of distinct behaviors were associated with a lower childhood BMI (Strauss et al., 2018).

Dose combines duration, reach, and type of behavioral intervention strategy to create a single measure of the potential impact of an intervention.  This metric has been applied to RiseVT work across Vermont communities and will continue to be used for both evaluation and prospective program planning.

Results-Based Accountability (RBA)

RiseVT uses a results-based accountability framework to quantify our program interventions.  This lens has allowed us to ask:

  • How much did we do?
  • How well did we do it?
  • Is anyone better off?

Local RiseVT Program Managers are collecting this data and are reporting on it to inform our Year in Review.  We create individual town and county reports at the end of each year to share our results.

AMPLIFY GRANT TRACKING

RiseVT awards micro-grants to community partners for aligned projects that meet one or more of the CDC’s 24 Strategies to Reduce Obesity. Funds are to be used for programmatic work and are not to be used for an organization’s core business function or overhead.  Partners report on projects funded by the grants to RiseVT statewide which are then promoted through RiseVT’s digital networks and local news stories when appropriate. We track all CDC strategies the grants address and total amount awarded by community.

 

Behavior Change Marketing

RiseVT conducts behavior change marketing campaigns to influence the health behaviors of the broader Vermont population. Pre– and post– evaluation is crucial to the development and measurement of behavior change marketing campaigns. We have conducted pre campaign research using the Center for Research and Public Policy as our research partner and will conduct post campaign surveys to infer the success of this work

 

MEET OUR EVALUATION TEAM

Our evaluation team is made up of our Program Design & Implementation Manager, Program Advisor, and Research & Evaluation Advisor. Each work to collect, analyze, and disseminate data on how our programming impacts the health and well-being of Vermonters.

Dr. Jennifer Laurent

Dr. Jennifer Laurent

Research & Evaluation Advisor

Dr. Jennifer Laurent serves as the Research & Evaluation Advisor for the RiseVT Statewide team’s monitoring and evaluation efforts. Dr. Laurent is also a practicing family nurse practitioner and has been practicing in a Vermont rural family practice working with individuals and families since 1997. In addition, Dr. Laurent serves on the Vermont State Board of Nursing and APRN subcommittee for the Vermont Board of Nursing.

Denise Smith, MS

Denise Smith, MS

Program Advisor

Also the Regional Director of RiseVT – Franklin & Grand Isle Counties, Denise brings to the team years of experience evaluating the impact of RiseVT initiatives. Her primary role on the Evaluation Methods team is to conduct Key Informant Interviews with Program Managers across Vermont to ensure we are measuring our work in our evidence-based model. Denise also spearheads the Measurement Study in our pilot communities.

Emmy Wollenburg, MPH

Emmy Wollenburg, MPH

Program Design & Implementation Manager

Emmy brings extensive public health program planning, management, and evaluation experience to the team. Her primary role on the Evaluation Methods team is to design evaluation methods, strategies, and procedures, synthesize local and national data, and disseminate our findings to staff, partners, and the public.