Restrictions brought on by COVID-19 stalled or delayed travel and gatherings across the globe. For evaluation specialists, these restrictions introduced incredible challenges and, in some cases, effectively halted in-person monitoring and evaluation (M&E) activities. In March 2020, Guidehouse was at the height of data collection activities for a global evaluation commissioned by the U.S. Department of State’s (DOS) Bureau for Educational and Cultural Affairs (ECA) when we had to quickly pivot to address such obstacles and maintain momentum towards completing the evaluation. The challenge became – in addition to logistical troubleshooting, how do you continue data collection activities virtually while the world adjusted to such dramatic and emotional events in so many peoples’ lives?
In October 2018, ECA commissioned Guidehouse to evaluate the Mandela Washington Fellowship for Young African Leaders, the flagship program of the Young African Leaders Initiative (YALI). The program’s purpose is to invest in the next generation of African leaders as they spur growth and prosperity, strengthen democratic governance, and enhance peace and security across sub-Saharan Africa. This evaluation sought to understand the impact the Mandela Washington Fellowship has had on its alumni, their home communities, and U.S. communities since its inception in 2014.
The evaluation design included collecting quantitative and qualitative primary data from a variety of sources, including surveying and interviewing Fellows, U.S. organizers of the Fellowship, U.S. community members who engaged with the Fellowship, and individuals from Fellows’ home communities.
Additionally, the Guidehouse evaluation team conducted social media research and analysis (SMRA) to understand and identify trends.
The Guidehouse team conducted international in-person interviews between August 2019 and November 2019. In March 2020, when U.S.-based data collection (survey and interviews) were scheduled to start, the team was grounded. Guidehouse and ECA knew that for the safety of communities and the research team, interviews would have to be done virtually.
To address the constraints of shifting to all-virtual data collection, Guidehouse and ECA proactively reacted by adjusting our approach to conscientiously account for the new operating environment and keep empathy a critical focus of our approach. Below are three key steps we took to ensure the evaluation could continue in line with its original timeline.