Collaborative Evaluation with THIS-WIC
THIS-WIC will lead the evaluation of the projects in collaboration with the funded WIC SAs. Applicants do not need to be research experts or have prior experience in evaluation, but will be expected to work with the THIS-WIC team to finalize the evaluation design for funded projects and to provide data necessary to evaluate the impact, feasibility, and acceptability of the intervention.
Throughout the project period, THIS-WIC will build WIC SAs evaluation capacity through trainings, webinars, and direct communication (see more information below in Expectations for Funded WIC SAs). WIC SAs can use grant funding to support current staff or hire new staff needed to carry out evaluation-related activities. Throughout implementation and evaluation of the funded projects, USDA WIC program requirements will be met (i.e., usual WIC care; resources will not be withheld from participants).
A key goal of THIS-WIC is to generate robust evidence of effective strategies for implementing telehealth solutions in WIC. Learnings from THIS-WIC research will help improve WIC services for participating SAs and help them to ensure high-quality programming and program access. These applied research findings will also inform other WIC SAs and local agencies more broadly on ways to meet program needs of the evolving WIC participant population.
Creating Your Evaluation Strategy
The goal of the evaluation strategy is to provide THIS-WIC with more information about evaluation designs applicants would be willing to implement as well as potential challenges applicants might encounter in implementing the proposed evaluation.
Specifically, applicants will be asked to consider and explain whether they can:
- implement the proposed intervention in a randomized (i.e., coin toss) fashion, by randomly assigning WIC participants or sites (i.e., local agencies or clinics) to receive either the telehealth intervention or usual care OR
- implement the proposed intervention using treatment and comparison groups (i.e., not randomized)
In determining whether they can implement the proposed telehealth intervention in a randomized fashion, applicants should consider key stakeholders, particularly those who will be involved in implementing the proposed intervention, including local agency/clinic directors, program managers, and staff.
Research Designs to Consider
The research designs below are organized from strongest (1) to weakest (3).
1. Randomized design for individuals
- Participants are randomly assigned to either intervention or control group
- Measure outcomes for both groups
2. Randomized design for clinics
- Clinics are randomly assigned to either intervention or control group
- Measure outcomes for both groups
3. Quasi-experimental design
- Participants or clinics are not randomly selected, but you still have an intervention and control group
- Measure outcomes for both groups
How to Choose a Research Design
Identifying the right research design takes time, and it is important to gather input from key stakeholders, particularly those who will be involved in implementing the proposed intervention, including local agency/clinic directors, program managers, and staff.
Your design selection may depend on many factors. Below are some important aspects to consider:
- Can you randomize participants? Randomized experimental designs are the most robust study design because of their ability to support cause and effect statements.
- Is a quasi-experimental design a reasonable alternative when randomization is not possible? Quasi-experimental designs include an intervention and control group but rely on options other than randomization to select participants or clinics.
- Including a control allows you to measure what happened after exposure to the intervention compared to what would have occurred without exposure to the intervention.
Use the diagram below to think through key steps in the design selection process.
Randomized Design for Individuals
Randomized experimental designs are the most robust study design because of their ability to support cause and effect statements.
Here’s a simple overview on how to lead a randomized experimental design study:
- Select a study sample from the target population
- Randomly select participants to either intervention or control group
- Provide the intervention to one group while the other group receives no intervention or usual care
- Measure outcomes from both groups
Below is a randomized-experimental design example for a WIC setting.
WIC randomized experimental design examples:
Randomized Design for Clinics
With a group-randomized design, the intervention is randomly assigned to a site instead of individual participants. Outcomes are still measured by assessing participants at each intervention and control sites.
Here’s a simple overview of how to lead a group-randomized design study:
- Select clinics from the target population
- Randomly assign clinics to be either an intervention or control clinic
- Clinics provide the intervention to intervention participants while the control clinic participants receive no intervention or usual care
- Measure outcomes from intervention and control clinics
Below is a group-randomized design example for a WIC setting.
Group or cluster-randomized trials examples from WIC:
Quasi-experimental Design
Quasi-experimental designs are a reasonable alternative when random selection is not possible. Assignment typically occurs by self-selection, which may occur when individuals sign up for a program, or by administrator selection (e.g., clinic director decides which individuals will receive the program).
Here’s a simple overview of how to lead a quasi-experimental design:
- Select clinics from the target population
- Clinics recruit participants
- Eligible participants self-select either to participate in the intervention or to receive usual care
- Staff provide the intervention to self-selected participants while the control participants receive usual care
- Measure outcomes for intervention and control participants
Below is a quasi-experimental design example for a WIC setting.
Quasi-experimental design examples from WIC research:
