Lilian Tutegyereize, Technical Officer, Science Facilitation, FHI 360
Aubrey Weber, Technical Officer, Science Facilitation, FHI 360
To date, the CHOICE Collaboration has conducted three virtual HIV Prevention Ambassador Trainers’ Workshops with more than 75 trainers from four countries, and a fourth workshop is planned later this fall. From the outset, we knew how important monitoring and evaluation (M&E) of the training-of-trainer workshops would be, especially in a virtual setting. User feedback would be crucial to understanding whether our training methods were resulting in real impact.
We chose the Kirkpatrick Model for training evaluation, developed more than 50 years ago, as a framework for our M&E plan. Through a comprehensive M&E approach, we hoped to learn what was working to prepare HIV Prevention Ambassador trainers to conduct their own trainings with adolescent girls and young women (AGYW) in their communities. We also wanted to know what was not working, and how we might create a better, more sustainable workshop.
What we measured
We conducted pre- and post-tests, facilitator debriefs and participant feedback surveys, and interviews to learn how we needed to adjust and improve our training program. Six months after each workshop, a post-training follow-up was done via virtual interviews with trainers, their supervisors, and program managers. We gathered information about post-training activities (such as number of training events held and number of ambassadors trained), the skills trainees are using or wish to develop, and ongoing support activities (such as office hours or mentoring groups) they have accessed or need.
What we learned
During our debriefs, we identified that participant performance was negatively affected—especially during the teach-back activities—by the short time participants were given to prepare. Participants expressed a need for greater support, guidance, and orientation to the HIV Prevention Ambassador Toolkit package. To address these issues, we added a robust mentorship component, a package overview section, and more detail about expectations for the teach-backs.
Based on feedback from participants and pre- and post-test results, we moved away from content focused on HIV prevention and pre-exposure prophylaxis (PrEP) as participants had a high level of understanding in those areas. We shifted the curriculum toward training delivery and facilitation skills, managing gender-based violence, increasing self-care, and handling challenging participants and situations.
The post-training follow-up revealed that all trainers carried out ambassador trainings after the workshop. Trainers made further recommendations including:
- Post-training follow-ups to trainers needed to be the most robust in the first quarter after the workshop because trainers were not yet confident in their skills.
- Trainers expressed a need for support from master trainers during the ambassador trainings. They wanted assurance and guidance during the training to confirm that, as one trainer said, “I am doing the right job.” Some trainers noted they needed continuous interaction with master trainers until they were confident in their ability to execute the work.
- Some trainers expressed they struggled with simplifying concepts in the local language and felt their greater knowledge made it difficult to connect to ambassadors.
- Some suggested increasing sustainability by including district trainers in the training of trainers (cascading the role of training facilitator) because projects are time bound.
- Trainers mentioned that simple visual job aids could help ambassadors better understand the program and their roles.
Next steps
Determining what impact our trainings have had on HIV prevention for AGYW has been the most challenging to evaluate because we cannot draw a direct line from our training program activities to the larger objective of reduced levels of HIV transmission among AGYW.
Nonetheless, we plan to continue capturing activities and outputs, such as number of ambassadors trained, ambassador training events that have occurred post-training, program impact/output, and outcomes such as increased skills and confidence of ambassador trainers in leading ambassador training, on a quarterly basis. Our M&E plan will help to identify inevitable challenges so quick and effective solutions can be implemented.
This is the eighth blog post in a series on the design, development, implementation, and iterative processes of the interactive and virtual HIV Prevention Ambassador Trainers’ Workshop, led by the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR)- and the United States Agency for International Development (USAID)-supported Collaborative for HIV Prevention Options to Control the Epidemic (CHOICE) through the terms of cooperative agreements of the EpiC and RISE projects. In this series, we share our experience and reflect honestly on the ups and downs of designing and delivering a comprehensive training virtually. Access the HIV Prevention Ambassador Trainers’ Workshop materials here.