Impact Evaluation of Energy Interventions. David A. Raitzer

to applying impact evaluation techniques to infrastructure projects are somewhat larger than for other sectors such as health, education, or social programs.

      Nevertheless, increasing numbers of innovative impact evaluations are emerging in the energy sector to address an expanding array of questions. Many energy program and project documents state intentions to improve socioeconomic welfare, such as increased incomes, better education, and improved health. Recent studies have helped to provide causal evidence of such benefits and the pathways through which they occur, but additional impact evaluations are needed to understand how these outcomes may be enhanced. Impact evaluation of energy sector reforms is a frontier area of research. Previous studies offer important insights on which reforms have been effective, but only a tiny share of possible reforms have been subject to rigorous testing. Energy efficiency is increasingly targeted to address sustainability outcomes. Impact evaluation can help test interventions to this end, but the array of options tested to date remains limited.

      Impact evaluation techniques can test effects at the levels of both energy consumers and energy suppliers. For example, governments now offer many incentives to promote clean energy supply. Impact evaluation can help to test their effectiveness. Similarly, impact evaluation can test alternative means of ensuring cost recovery, such as separation of power supply feeders for irrigation and for households. These types of insights on responses to measures taken can help in refining energy programs.

      This report aims to help orient future impact evaluation work in the energy sector by reviewing what has been produced against what might be possible. To do so, this report offers a brief review of impact evaluation considerations for the energy sector, impact evaluations performed to date, and future directions for impact evaluation studies. It does so by first characterizing energy sector programming and theories of change to identify potential impact evaluation questions. Then it reviews impact evaluations performed to date in terms of interventions and outcomes covered, as well as methods applied. From these two sets of information, gaps in intervention and outcome/impact coverage are identified, and future directions for impact evaluation are proposed.

       2. Types of Energy Sector Interventions

      Energy is one of the principal areas for development assistance expenditure globally. In many senses, the Asian Development Bank (ADB) typifies the global pattern. ADB has provided assistance to its developing member countries in the energy sector for more than 40 years. Energy sector operations now make up the largest share of ADB lending, and they accounted for over a quarter of ADB’s total lending commitments from 2008 to 2018.

      ADB’s first Energy Policy was developed in 1981 as a response to the energy crisis of the 1970s. The ADB (2009) Energy Policy aims to help developing member countries provide reliable, adequate, and affordable energy for economic growth in a socially, economically, and environmentally sustainable way. The policy emphasizes energy security, facilitating transition to a low-carbon economy, providing universal access to energy, and working toward achieving ADB’s vision of a region free of poverty.

      The dominant focus of investment in the energy sector is the electricity subsector. Major hardware investment areas in this subsector include power generation, transmission, and distribution. New power generation is often intended to be from more environmentally friendly and efficient sources than the existing energy mix, frequently with more use of renewable sources (Frankfurt School–UNEP Center/Bloomberg New Energy Finance 2018). Transmission investments generally seek to reduce transmission losses and improve transmission capacity, while distribution helps to extend electricity to new areas.

      Complementing the hardware investments are investments in policy improvements. Appropriate tariff structures to encourage electricity conservation and minimize peak-time consumption are often promoted, as are reforms to reduce subsidies and make them transparent, appropriate, and targeted. New technologies are also piloted to demonstrate how electricity can be generated more sustainably.

      Beyond electricity, development investments can seek to improve supply of cleaner fuels, such as natural gas or geothermal energy for heating and/or cooking. They may also target cleaner and improved cookstoves to reduce local air pollution, or provide more efficient public heating services to substitute for individual heating facilities. Interventions, such as biodigesters or interventions to scale up biofuel production, may generate new sources of fuels. Other interventions, such as fossil fuel subsidy reforms, may seek to promote more efficient and environmentally friendly substitution for transport fuels.

      Programming is increasingly focused on ensuring that possibilities from energy sector development can be equitably realized (ADB 2014). For example, energy access interventions for rural electrification are increasingly being coupled with complementary interventions, such as skills training for women and poor stakeholders.

      As energy programs become progressively more oriented toward objectives of inclusion and sustainability, the number of assumptions underpinning their design is increasing. Moreover, a rising share of those assumptions is behavioral, and those behavioral assumptions can often only be truly tested using impact evaluation methodologies.