It is clear that poverty is a highly complex issue, with many dimensions – income levels, health, access, location, environment and mobility, to name a few. Amartya Sen has done groundbreaking work when it comes to measuring poverty.
But one big problem persists: Poverty analysis is backwardly focused. To put it bluntly, we can measure how it happened, but it is very difficult to measure it as it happens.
Statistics are often months or even years old. When people apply for social welfare, the personal crisis has already occurred. But how can the symptoms be measured before they even happen?
Such a new approach is to use real time data (or big data) to measure poverty as it is happening.
As we enter an era of massive data collection, thanks to the Internet and mobile devices, how can such data be also used to understand the many dimensions of poverty?
One word of caution: Better analysis can lead to better action to address poverty, but it does not lift people out of poverty by itself.
Data dive in Vienna
As part of a big data exploration – the UN Global Pulse, UNDP, QCRI and the World Bank have teamed up on a series of projects, challenges, and competitions to unearth key questions, explore data sources (both open and big data), and take a fresh look at old problems – I had the pleasure to facilitate a data dive in Vienna at the end of Februrary (together with the Open Knowledge Foundation Austria)
We were happy to have 26 participants from various backgrounds such as:
Business intelligence, data protection and programming
The overall good turnout of participants, for such a niche topic, shows the high interest in big data analysis and proves the potential to reach out to external resource persons.
It is promising to see data specialists investing their free time to participate on such events. (See: Q&A: The day a big data scientist met a development organization)
Since big data analysis is in its infancy within development organizations, this is a chance to combine inside expertise with outside data specialists.
Personally, I think it is particularly helpful to invest in a community for that topic, instead of relying on business intelligence solutions.
My first observation during the discussion was a bit of a clash of mindsets between classical data collection for statistics and new opportunities with big data and data mining approaches.
How can these two approaches can be used together?
How can one support the other for better poverty analysis?
But that did not stop participants from brainstorming interesting ideas to measure vulnerabilities.
One idea was to look at data from retail chains to track consumption patterns to analyze the risk of poverty on a regular basis. For example, having exact figures from recent weeks how certain commodities have been purchased. How staple food is maybe replaced by luxury food items and vice versa.
Bonus-points-collecting services have fine granular data for millions of consumers, and larger and smaller shifts in consumption patterns can be analyzed and localized in real time.
As well as identifying consumer interests for marketing purposes, maybe this data could also be used for a good cause. But to what extent are the consumption behaviours of people affected by poverty tracked (particularly in developing countries) ?
Another approach is data from insurance companies, which could be used as an early indicator for vulnerabilities. Insurance companies have a long tradition of risk assessment, and through microinsurance, their data reaches far into rural areas – also in developing countries.
A participant proposed measuring the distance of daily commuting and how this might change due to economical constraints; or energy providers could provide data for payments as an early indicator when electricity or heating bills cannot be paid.
With access to companies’ data, the flow of remittances could be analyzed in real time to see larger and smaller trends of money transfers and what it might indicate about poverty.
All these ideas lead to the challenge of closed or private data.
Open data is one driver to hope for more access, but maybe more could be done under the umbrella of corporate social responsibility or through the concept of data philanthropy.
But is there any non-social media company offering a real time API yet? There is need for precise data requirements, solutions for data portability and data protection – especially mobile phone data with movement logs of its users contains highly sensitive data and cannot, as different studies show, be fully anonymized.
This is the great side-effect about such personal data. How can it really be protected?
Many participants shared the hope that corporate data philanthropy would become the norm soon and perhaps a requirement to bid successfully on contracts.
It was suggested that development organizations need to make a better case for why corporations should share data with them.
So far, the open data movement has focused on governments, so maybe a shift to companies could help here?
We also talked about data collection or producing data in need, since most data is not at hand and is more or less closed data.
Could crowdsourcing approaches help to get accurate data about poverty?
There was an interesting discussion on personal data philanthropy and the growing willingness of individuals to ‘donate’ personal data for the public good. (See: Would you give up your personal data for development? and check out the Quantified Self)
Efforts for crowdsourcing in recent years show that user reported data can be an alternative to traditional surveys.
Can these approaches be scaled to the level needed to get accurate results?
The development of low-cost sensors is widening the landscape of data tracking. Looking at the growth of smartphone usage in developing countries, this is a potential venue for data collection in the near future.
This out-of-the-box thinking shows the potential for using data to help us better understand complex problems.
It also became clear how little data is available in the public space. And to address poverty, data analysis is only a tool to then act on. This second step is far bigger and even more complex.