By Iqbal Dhaliwal, Executive Director, and Rebecca Toole, Senior Policy Associate, Abdul Latif Jameel Poverty Action Lab (J-PAL)

 

At the United Nations Climate Change Conference in December 2018 (COP24), parties agreed to a rulebook that lays out how governments will measure, report and verify emissions under the Paris Agreement. Now countries need to act — and know whether policies and programmes are meeting their climate goals.

Thanks to innovations in research design, improvements in measurement technology and an increasing political will to know what works, more opportunities to rigorously evaluate and learn from real-world environment and energy policies exist than many might think.

Take, for example, our work at the Abdul Latif Jameel Poverty Action Lab (J-PAL) to ensure that policy is informed by scientific evidence. J-PAL is anchored by a network of 171 affiliated professors at more than 50 universities who conduct randomised impact evaluations to answer critical questions in social policy. Our Environment & Energy sector measures the real-world impacts of environmental and energy policies on everything from pollution reduction to climate change mitigation and resilience.

For instance, on investments in greenhouse gas sinks, our researchers conducted several evaluations of payments for ecosystem services (PES) programmes, a popular policy tool that compensates people for protecting or improving their local environment, such as conserving forested land. In Uganda, J-PAL affiliate Seema Jayachandran, along with Joost de Laat, Eric Lambin and Charlotte Stanton, partnered with a local NGO to test a programme that paid landowners cash if they did not cut trees on their property. The study used satellite data to measure impacts on deforestation and found that the programme was highly cost-effective. Landowners offered contracts to conserve forests cleared 4% of forested land, compared to 9% in villages where the programme was not offered. This equated to delaying the release of 183.5 metric tons of carbon dioxide into the atmosphere per eligible landowner over the study period.

Ultimately, the effectiveness of PES depends on how contracts are structured and how people respond to the contract terms. J-PAL affiliate Kelsey Jack, along with Paulina Oliva, Samuel Bell, Christopher Severen and Elizabeth Walker, evaluated a PES program that paid farmers to plant trees in Zambia and found that most farmers abandoned the trees they planted, even after paying for the seedlings. Over time, farmers learned that caring for their seedlings was more costly than they initially expected and not worth maintaining their PES contracts, which paid farmers only if a set number of trees survived. This illustrates the importance of piloting and evaluating new interventions to understand their effectiveness and to get the details right.

Beyond payments for ecosystem services, policy makers could choose from any number of strategies: promoting energy efficiency, increasing the use of renewable energy, enacting regulations to reduce emissions. Impact evaluations also speak to these approaches, sometimes with counterintuitive results.

Consider how many think that residential energy efficiency measures are a straightforward way to reduce energy consumption and save people money. But several recent evaluations show these programmes’ impacts in the real world can fall short of engineering projections of their benefits.

J-PAL affiliated researchers Meredith Fowlie, Michael Greenstone and Catherine Wolfram evaluated the Weatherization Assistance Program (WAP) in Michigan, a US Department of Energy programme to support low-income homeowners retrofit their homes. The results showed that the programme fell short of both its anticipated impact on CO2 and electric bill savings for programme participants. The researchers found that the tool used to project the impact of the retrofits on energy consumption vastly overestimated potential emissions reductions. The programme also received many fewer applications than expected, due to the large amount of time needed to enroll. By testing what actually works in the field, and not just the lab, evaluations can help improve programmes or steer policy makers towards more efficient approaches.

So far, we’ve only highlighted consumer or household-level programmes, but large-scale emitters like industrial plants are just as important for reducing emissions, if not more. J-PAL affiliated researchers increasingly conduct evaluations at scale with governments and firms to help improve the design of environmental regulation.

In Gujarat state in India, for example, J-PAL affiliates Esther Duflo, Michael Greenstone, Rohini Pande and Nick Ryan worked with the state environmental regulator and over 470 industrial plants on a randomised evaluation that found that making environmental auditors more independent reduced corruption and industrial pollution. A common arrangement in third-party audit systems is to allow firms to hire their auditors directly, which can create a conflict of interest and incentives to misreport pollution. In this evaluation, auditors were instead randomly assigned to industrial plants, paid from a common pool, monitored for accuracy and paid a bonus for accurate reports. The study found that auditors were 80%less likely to submit a false pollution reading. In response to more accurate audits, industrial plants reduced emissions of air and water pollutants by 28%, and the highest-polluting plants drove this effect.

In an ongoing study in Maharashtra state in India, J-PAL affiliates Michael Greenstone, Rohini Pande and Nick Ryan, along with Anant Sudarshan, are now testing whether publicly naming and shaming large industrial polluters can persuade firms to pollute less. Together, the researchers and the Maharashtra Pollution Control Board (MPCB) designed a five-star rating programme on a scale from one (the biggest polluters) to five (the cleanest firms). Ratings are made publicly available, so plants know how they compare to other firms and regulators, and citizens are equipped to call for stricter compliance.

So even with a rulebook as guidance, accomplishing Paris Agreement targets will require policy makers to make important decisions about which environmental programmes and policies to enact. Rigorous impact evaluations, like those described here, can provide insights on the real-world impacts of such environmental programmes and policies and how to design them well, a key input to accomplishing climate goals.