October 5, 2022
The frequency of flooding and droughts has increased in recent years across Africa, and scientific forecasts show that extreme weather events may happen more often in the future (Masih et al., 2014). Rural populations, whose livelihoods depend heavily on agriculture are likely to be particularly hard-hurt by these occurrences. This is presumably the case in Zimbabwe, for example, where agriculture accounts for 90.5% of employment in rural areas, and almost 70% of food consumed by households is produced on-site (Food and Nutrition Council, 2011).
Extreme weather events can affect access to food, health and nutrition (e.g. Hoddinott and Kinsey, 2001; WHO, 2008), local economies, political stability, and education outcomes (e.g. Alderman et al., 2006; Maccini and Yang, 2009). Although we know that weather can affect these and other outcomes, our ability to credibly measure the impact of any given shock is typically limited by a lack of data.
How can we measure the impact of a drought?
Field experiments and randomized controlled trials have become a preferred tool for economists and social scientists trying to measure “impact.” In these experiments, an intervention is often randomly assigned so only some potential recipients are exposed to the program. This randomization allows us to reliably measure the impact of the intervention by providing a “counterfactual” that tells us what would have happened without the intervention. But some things, like weather, cannot be randomly assigned. At the same time, the timing and location of droughts are difficult to predict, making it nearly impossible to purposely collect detailed data on flood or drought victims just before they are hit with the weather shock, as needed for careful counterfactual analysis.
In a new research study, we show how the key to measuring the impact of extreme weather events may be the social programs and international development projects that are already underway in regions when extreme weather events occur (Nordstrom and Cotton, 2022). Depending on the intervention, such projects are often already collecting detailed data on health, employment, or educational outcomes within the communities in which they operate. They can provide baseline data for outcomes of interest within the communities leading up to the drought or flood, and data on how these outcomes change during and after the weather event. As long as the shock is unexpected and some communities experienced it while others do not (or they experience it with different levels of intensity), then such program data is effectively identical to what we could get from a randomized field experiment.
Our study uses program data from a UK government Girls Education Challenge (GEC) project in rural Zimbabwe to measure the impact of a severe drought on education outcomes. The GEC prioritized the collection of detailed data on student education performance and characteristics, including observations from both treatment areas exposed to the GEC program and control areas that were not exposed to the program. The GEC’s IGATE project was implemented from 2013 to 2016. In the final year of the program, during the 2015-16 growing season, a major drought hit the region, disrupting agriculture production and the livelihoods of many families, particularly in the rural regions where the project was implemented. Importantly, the intensity of the drought was seemingly random; some communities experienced normal growing conditions and others experienced severe drought conditions.
We combine the GEC program data with detailed remote-sensing satellite data from the National Oceanic and Atmospheric Administration (NOAA). This allows us to explore how rainfall anomalies impact local agricultural productivity (vegetation coverage) and, ultimately, student academic outcomes.
Figure: Vegetation Health Anomalies and Program Data Communities in Zimbabwe.
We highlight countervailing effects of drought exposure on the educational outcomes of youth.
First, we show that droughts increase the quantity of schooling. Students exposed to droughts have higher attendance and are more likely to advance to the next grade.
This finding stands in contrast to a narrative in which droughts and the economic hardships that come with them cause kids to leave school to work. That is not the case here. Droughts do cause economic hardship, but the declining agriculture production reduces the incentive for students to stay home to either work or contribute labor during harvest time. There are fewer opportunities for kids to work, which seemingly pushes them into the classroom.
Second, we show that the increase in attendance and progression does not correspond to a similar increase in learning outcomes, as measured through math and literacy assessments. Despite more schooling, we see no evidence of additional learning in drought-affected communities.
Specifically, we do not see a significant change in learning overall, suggesting that the positive effects of increased attendance may be offset by negative effects of hunger, stress, economic hardship, or other drought-induced challenges. If we take a closer look at the poorest students, we see that drought exposure leads to significant decreases in literacy performance. The students who had previously reported experience with food insecurity saw the biggest detrimental impacts from being exposed to a drought.
These findings show how the definition of academic performance matters. Some studies define academic success based on attendance, progression, or total years of schooling completed. Other studies define academic success based on math or literacy assessments. Not many studies have data on both types of performance measures.
Had we only used data on the quantity of schooling, our analysis would have strongly concluded that droughts are beneficial for schooling. On the other hand, had we only used data on numeracy and literacy performance, our analysis would have concluded that droughts have only negative impacts, and these are concentrated on the most marginalized students. It is only when we include both types of measures that we avoid jumping to conclusions and discover a more nuanced story. This kind of nuance is critical for those working in the education sector, who may be interested in designing programs that allow students to be more resilient when droughts affect their communities. For example, these findings suggest that policymakers target resources in ways that support student learning during these kinds of events. This could take the form of school feeding programs or other school-based outreach programs since students are more likely to be enrolled and attend school after a drought.
We have shown how project data can offer important insights beyond their original intent. This highlights the potential for implementers or funders to use their existing data sets to better understand the environmental factors and barriers faced within the communities in which they operate. As with any individual data, however, steps need to be taken to ensure the privacy of program participants is respected. The consent and data cleaning processes can be adapted to ensure anonymity and adherence to ethical standards before data collection begins.
Christopher Cotton is a senior economist and research officer at Limestone Analytics and a Professor and the Jarislowsky-Deutsch Chair in Economic & Financial Policy at Queen’s University. Ardyn Nordstrom is a Research Associate at Limestone Analytics and an Assistant Professor of Program and Policy Evaluation in the School of Policy Studies at Carleton University. They have collaborated on several research projects to understand barriers to education and the most effective programs and policies for improving education among marginalized communities.
Ardyn Nordstrom and Christopher Cotton, 2022, “Impact of a severe drought on education: More schooling does not imply more learning,” Queen’s University working paper.
Masih et al., 2014, “A review of droughts on the African continent: A geospatial and long-term perspective,” Hydrology and Earth System Sciences, v18(9).
Food and Nutrition Council, 2011, “Zimbabwe rural livelihood baselines: National synthesis report,” Government of Zimbabwe.
Hoddinott and Kinsey, 2001, “Child growth in the time of drought,” Oxford Bulletin of Economics and Statistics, v63(4).
WHO, 2008, “Climate change and health,” World Health Organization, Regional Office for South-East Asia.
Alderman et al., 2006, “Long term consequences of early childhood malnutrition,” Oxford Economic Papers, v58.
Maccini and Yang, 2009, “Under the weather: Health, schooling, and economic consequences of early-life rainfall,” American Economic Review, v99.
Photo credit: Colin Crowley / Save the Children
There are various methods for the analysis of risk and uncertainty, but how can we combine them to properly integrate into cost-benefit analysis?
Click here to learn more about the guide produced by Limestone under USAID’s LEAP III project: https://limestone-analytics.com/project/usaid-guidelines-for-the-analysis-of-risk-and-uncertainty/