Can big data help deliver better operational results?

by , and

Filed under: Development Development 2.0 Poverty

Ready to take a dive?

Ready to go for a dive?

The seminal United Nations Global Pulse paper last year made the case quite clearly and a sleuth of predictions for 2013 just reiterated the point: big data has an enormous, but yet untapped potential for development (and beyond).

As a Harvard Business Review article put it recently: “data-driven decisions are better decisions” – it is as simple as that. Using big data enables managers to decide on the basis of evidence rather than intuition.

So how can we help turn that potential into reality?

We are big fans of “quick dos” as a way to demystify buzzwords, so we thought that the best way to learn about big data for development would be to actually sink our teeth into it.

And what better place to start than operations to test whether big data can make a difference on the ground? What if we could come up with a set of tool and resources that can be applied to operational processes immediately?

The World Bank finance team initially came up with the idea, and our team at UNDP and the UN Global Pulse gladly joined forces. And we hope that the list won’t end here!

What type of questions could big data help answer to inform the work of operations managers? Here’s what we came up with from an initial brainstorming:

  1. Can you measure poverty in real time (using alternative techniques made possible by the widespread availability of ‘high-frequency’ data)
  1. Is it is possible to create a prediction model (or early warning systems) to determine the likelihood of funds being used for purposes intended (thus helping make better decisions BEFORE money leaves the donor and improving our assurance processes)?
  1. Is it possible, with any degree of certainty, to guess which projects are likely to report new improper payment (or fraud/corruption – in a limited sense of the word) cases within the next 12 months: Can such an approach lead to better operational risk management in general?
  1. Is it possible to anticipate the rating of a project, say, three years before the actual rating exercise is conducted?

We know that these questions are rather ambitious and may require a good dose of reality check from data specialists. This is why we envisage that the first stage in our exploration of big data will be one (or perhaps more) data dive(s) to refine the questions, look at availability of datasets.

We are aiming for the end of February. This will be followed by a big data competition in April or May to develop tools and techniques with proven application to development operations. We hope that the results could be a public good for the whole development community.

Are you a data geek interested in contributing to development issues?

Is your organization interested in hosting a data dive?

Would you like to help refine the questions so that they have the maximum impact for the development community?

We’d love to hear from you.

  • Christian

    Thanks for this interesting points. Can you elaborate further on point 2? I wonder how that could be measured? Maybe if 90% of organization plan to invest in one sector and you plan to invest in another sector, although all indicators tell you that is not the sector, which is in need of most help.

    Problem I see is that we are far away from anything called big data. CRS data set I worked with here offene-entwicklungshilfe,de has 1GB. Most data sets are tiny and give you only a small picture of activities. And as long as we have organizations such as Global Pulse, who sit on their big data and do not realease anything not much is gonna change.

    • http://twitter.com/jurakhrapunov Jura Khrapunov

      Hi Christian,

      As you can see Global Pulse is a part of the game and we are trying to get as much as possible out of this partnership for public benefit.

      Concerning question #2 – we don’t know either at this moment, but would like to tap into the expertise out there to find out if it’s possible at all. Internal expertise is always limited so the main point of the challenge is to involve data scientists and engineers throughout the world to work on the issues.

      BTW, we are working with OKFN Austria to organize a data dive in Europe at the end of February. Drop me a line if you are interested

      • Christian

        Yes, let me know what is planned.

        • prasannalaldas

          Tough questions Christian but that’s what this exercise needs. As Jura writes, the chief reason to organize the data dives is the fact that we don’t have all the answers or the data and we see these events as an impetus to engage more people in solving the questions, and to identify and encourage the publication of more data that can help answer the questions (even if tangentially). At the Bank, we’re working with a few folks to identify data around Question 2 – fingers crossed – and there seems to be great interrest in Question 1 (Global Pulse can be really helpful there). Baby steps, but…

    • Andi Pawelke

      Hi Christian,

      UN Global Pulse cannot release any “big data” simply because they don’t have any. They’re working with companies that hold interesting data to explore ways how their digital services and data could be used for the social good. See this piece by the Global Pulse Director Robert Kirkpatrick on what they call “Data Philanthropy” http://onforb.es/ncnGTw Also you might want to check out their FAQ
      section for further details http://www.unglobalpulse.org/about/faqs

      For the upcoming data dive series, a Global Pulse partner from the private sector will be giving access to some of its data for participants to explore the potential of such data for international development. The details will be announced on the UN GP website in due time.

      Full disclosure: I’m currently doing research for UN GP on the challenges related to private sector data sharing.

  • http://twitter.com/stefankasberger Stefan Kasberger

    hey,

    sounds really interesting. I have a question about this, so we can better prepare for the data dive in vienna, me and some friends will participate in.

    What is your definition of poverty or is this up to us to work out a proper definition? The topic is quite big and I’m a little bit sceptic about what can be done from this four points, but let’s have a look at it and get deeper into the issue.

    greetz, stefan

  • Akmal

    i liked this post very much. It raises very interesting and actual
    questions. I want to respond to the question 1. I think the term poverty is too strong and a bit misleading in the current context of the big data discussions. I think it would be appropriate to focus on how Big Data approach can be deployed and effectively used in crises and emergency situations. This is not directly connected with poverty but it connects to situations when people get suddenly vulnerable to poverty due to either having no access to common goods or extraordinary increase of prices for commodity goods and
    services. In this sense, the Big Data is very useful and innovative approach to detect increasing vulnerability and to monitoring the extraordinary situations providing real time input in devising effective and targeted policy and operations interventions at fields.

  • aLan

    Last month’s Harvard Business Review outline how big data is being used in marketing to understand the impact of various components of advertising intiatives. Since both marketing and development historically suffer from the same shortcoming – how to attribute success to specific actions – this same approach could be used to assess the impact of development interventions.