In Cape Town, South Africa, the municipal electric utility had a problem.
Too many bills were going unpaid, and they were forced to devote time and resources they didn’t have to delinquent bill collection. They had started to shift to prepaid services to cut the problem off at the pass, but what would be the impact? How would it affect revenue? Or the end-users?
Through a partnership with a research team led by Kelsey Jack (University of California Santa Barbara and J-PAL) and Grant Smith (University of Cape Town), the city decided to launch an experiment to measure these impacts. They divided the city into smaller sections and phased in the prepaid approach, simultaneously collecting and analyzing data on energy usage and payment schedules.
We have shown the power of evidence-informed policy to a lot of the data providers in government and beyond. For the next five years, my hope is that we can partner with many more players—including in the private sector—to safely and ethically access and analyze admin data to fight poverty and climate change.
Iqbal Dhaliwal Global Executive Director The Abdul Latif Jameel Poverty Action Lab (J-PAL)
It turns out shifting to prepaid metering may have benefits for both consumers and utility companies. Overall, although household electricity use declined under prepaid meters, the utility was able to recover more of its costs. Consumers may benefit from prepaid electricity by gaining the ability to control their own consumption, and utility companies may benefit from prepaid meters by recovering a larger share of the money that customers owe to them, improving the sustainability of coverage to low-income households.
It’s a seemingly simple example with a not-so-simple backstory. It required relationship building, navigating data use agreements, and bringing an academic and research lens to public services. But with the leadership of J-PAL Africa affiliated researchers and staff, the City of Cape Town and a growing network of communities now have a roadmap for creating this kind of system for data-driven decision-making.
The Challenge
J-PAL is a global research center working to reduce poverty by ensuring that policy is informed by scientific evidence. They connect a network of 261 affiliated professors at almost 100 universities around the world who conduct randomized impact evaluations to measure the impact of development programs to understand which are most effective (or not), and why, in the fight against poverty and climate change. Due to the absence of relevant and reliable administrative data, J-PAL studies typically require bespoke data collection through field surveys, an expensive and time-consuming process.
Rapid digitization has led to increased amounts and variety of administrative data, collected and stored electronically as part of routine work by governments, private companies, and nonprofits. Due to advancements in point-of-use data collection systems, this data is more accurate and reliable than in the past. Yet the administrative data collected by these providers—like immunization rates or tax delinquency, for example—continue to be underutilized and underanalyzed. J-PAL believes there is significant potential for harnessing these datasets and using them to inform and improve decision-making for the public good.
Global Executive Director Iqbal Dhaliwal explains that there are several barriers that must be broken down to enable the use of data to transform lives through improved social policy. J-PAL tackles those barriers through their Innovations in Data and Experiments for Action Initiative (IDEA).
- First, data providers like governments often do not have a clear vision for how they wish to use the data they collect. The result is data that is not made accessible or usable for analysis to researchers, who are often seeking to partner with governments to improve existing programs or develop new ones.
- Second, concerns about privacy and confidentiality compound the issue, holding data providers back from entering into data use agreements with such researchers.
- Third, the bench of technical talent in these departments is often shallow, with limited capacity for data deep dives, even if policymakers do have a clear vision.
Moreover, there is significant variability in how data sharing is handled in different countries, or even by states within the same country. In J-PAL’s seven offices around the world, you may get entirely different responses when requesting information.
The J-PAL team began to see more and more interest by policymakers to improve decision making using data and engaging with researchers to do so. This fueled the mission of IDEA in supporting providers who want to use their data for good.
The Solution
The IDEA initiative brings providers—and governments, in particular—along on the journey through several approaches.
The IDEA Handbook provides case studies on how data can be safely and ethically accessed, how partnerships between researchers and data providers can be navigated, and some innovative data analysis that has helped understand the impact of development programs. The IDEA Handbook is a go-to reference for researchers seeking to use administrative data and for providers looking to make their data accessible for research.
The IDEA Lab will seed the innovative use of admin data for improved policy making.
The Handbook is free and accessible to all. It has 16 chapters produced by 40 authors from the top academic institutions in the world, covering everything from model data use agreements and data security to how to collaborate with Institutional Review Boards at universities.
The Handbook is likewise designed to be accessible and digestible, and is accompanied by 15 author-produced webinars that further break best practices down. These webinars were viewed more than 2,500 times between fall 2020 and fall 2021, going beyond a knowledge-building exercise to create a community of practice. The J-PAL team has been particularly excited to see chapter authors attend other authors’ presentations, and share how they’ve learned from one another.
But the IDEA Lab goes beyond the handbook—it helps set up long-term partnerships with providers to strengthen the use of administrative data, thereby creating a pipeline of successful use cases.
“There’s a lot of desire and interest in using that data but we don’t see that many actual applications. Through the IDEA Lab, we hope to provide technical advisory to data providers that are interested to make their data more accessible and analyze it to improve decision making, and to introduce innovative ideas that can be piloted and rigorously evaluated to measure their impact using this admin data. Eventually the most successful programs and policies can be scaled up, this time using the admin data to monitor the process,” said Aparna Krishnan, project director at the IDEA Lab at J-PAL South Asia.
Collectively, these IDEA strategies serve to illustrate that the challenges of government data sharing are solvable, and in the end can open up countless opportunities to help solve important societal problems.
The Takeaway
In Cape Town, both sides experienced that lightbulb moment, where the value of better data analysis and collaboration became clear.
“There was a realization from both the city side and the researcher side that there were tremendous gains from cooperation. The researchers were bringing interesting questions, data science capabilities, and experimental capabilities, that the government didn’t have,” said Shawn Cole, the John G. McLean professor of business administration at Harvard Business School, co-chair of the J-PAL Board of Directors’ Research, Education, and Training Committee, and a principal editor for the IDEA Handbook.
In the wake of the utility payment experiment, Cole was encouraged to see Cape Town maximizing the approach of partnering with researchers to answer persistent questions that have held back their community. Kelsey Jack credits the city’s investment in making data accessible and building long-term partnerships with researchers with the project’s enduring success. The city has received more than 136 requests from academics to utilize administrative data for social good, powering a research agenda that they would not have the capacity to execute on otherwise.
That growing list of requests, which can now be streamlined through a centralized data sharing platform, is one of the things Cole is most proud of, and it’s an evolution that J-PAL hopes to inspire in more partners in the future. Increase demand, strengthen partnerships, build shared knowledge and capacity, and ultimately drive social change.
“We have shown the power of evidence-informed policy to a lot of the data providers in government and beyond. For the next five years, my hope is that we can partner with many more players – including in the private sector – to safely and ethically access and analyze admin data to fight poverty and climate change,” Dhaliwal said. “We want them to ask, ‘what happens when we access and analyze this data?”