Counting the uncountable

In this episode, we discuss the role of evidence in resource allocation decisions, focusing on the new Joint and Intersectoral Analysis Framework (GF 2.0). We explore how this approach aims to address flaws in traditional needs assessments and improve transparency in humanitarian responses.

You can find more on the JIAF 2.0 here: https://knowledge.base.unocha.org/wiki/spaces/hpc/pages/3992944652/JIAF  

And the most recent estimate of global humanitarian need, based on the JIAF 2.0 method, is here: https://www.unocha.org/publications/report/world/global-humanitarian-overview-2024-enarfres  
 

Guests:
Leila Oliveira, Independent expert and former GF 2.0 advisory lead
Katie Rickard, Impact Initiatives
Helen Pittam, REACH, with on-the-ground perspectives from crisis-affected countries

Co-hosts:
Alice Obrecht, Head of Research & Impact, ALNAP 
Emmeline Kerkvliet, former Research Officer, ALNAP

ALNAP’s A Matter of Priorities: a podcast on tough choices in humanitarian funding 

Join our WhatsApp channel

Transcript

Alice Obrecht: Welcome back to A Matter of Priorities, a podcast about the prioritisation challenges facing the humanitarian aid sector today and how to address them.

(Upbeat Music)

Back to A Matter of Priorities. In our first episode, we heard that the humanitarian sector is facing a financial cliff that is unprecedented in the 25 years for which comparable data is available.

And this has led to a perceived funding gap between the money that's available and the things that people need to survive, which is making agencies and donors have to make really hard choices about where to put their money.

In today's episode, we're turning our attention to the role of evidence in resource allocation decisions. Here, we're going to try to answer two questions. One, how do humanitarians go about identifying who is most in need? And two, could a more scientific approach to identify needs help humanitarians prioritise more ethically and effectively?

Alice Obrecht: Hi, Emma. Nice to see you again.

Emmeline Kerkvliet: Hi, Alice.

Alice Obrecht: Emma, can you talk us through how the humanitarian system has traditionally defined, analysed and understood needs?

Emmeline Kerkvliet: So typically, the first thing that humanitarian agencies do in a crisis, such as the ongoing conflict in Gaza or the landslides in India, is an analysis of what goods and services people are lacking, which is called a needs assessment. And humanitarian agencies are often specialized in different services, which we call sectors, which are things like food security, water and sanitation, protection, shelter, health, so on. And these agencies will do a needs assessment for their respective sector. And then this gets put forward, accompanied by a request for funding to deliver assistance to meet those identified needs. But there are several known issues with this, one being a lack of coordination over the methods used to count the needs, which can create confusion and concerns about double counting.

And another problem is that there's a slight conflict of interest in that the agencies conducting the needs assessments are the same ones that are asking for the money to respond to those needs, which creates a lack of trust in the accuracy of the data. And so there's a new method for needs analysis that has been piloted this year called the Joint and Intersectoral Analysis Framework, otherwise known as the GF 2.0, which aims to provide more objectivity and methodological robustness.

Alice Obrecht: Thank you, Emma. That's, I think, a really good potted history and also for pronouncing the Joint and Intersectoral Analysis Framework correctly, because I've been saying "JAF" a lot. So now I know it's pronounced "GF". So we've been speaking to two people who have been closely involved in the GF 2.0, Katie Ricard of Impact Initiatives, an independent needs assessment agency, and Leila Oliveira, who led the independent body that developed the GF 2.0.

So our first question to Leila was to tell us where the GF 2.0 came from and what it's trying to achieve.

Leila Oliveira: So the GF initiative came from the Grand Biennium Agreement, when the key donors, UN agents and NGOs, agreed that we need a more standard robust, impartial, and comparable needs assessment. The GF 1.0 was an effort towards that and did great into bringing the partners together. We know that for us to really have strategic response, we need very coherent and complement needs analysis, and the GF was really an initiative, an effort to bring the partners together, to make an assessment that's, it's very important, it's credible, we know that that a big question of the assessment list can lead to us that assessment. What are they saying? How must we understand that? And if we had different agents, different sectors, going each way and doing their own thing, it comes back that in the end, the response, I wasn't really doing the needs analysis, but was really to bring it together in a system that it is responsible and responsive to needs.

Alice Obrecht: Great, thank you. Could you give a bit more detail and examples of where some of the challenges with the previous methods lay? So for example, one that I understand is that we might be interested if we're trying to think about prioritizing resources. We would want to know how many people in a given country or crisis area, crisis affected area, how many people need to be served. My understanding is that there's been for many years issues with getting to an accurate number because different agencies will maybe assess people for particular types of issues or needs like food security or water and sanitation or shelter. And then they each come up with their own number of people who need, you'll have a certain amount of people who need shelter support, a certain amount of people who need food security or food aid support,
and etc., etc., and they were using different methods to get there and it wasn't clear how many were double counted. Is that one of the examples of the issues that was going on before and are there others that you can describe as well?

Leila Oliveira: Exactly. There was a bit of independence of the sectors and the agencies to make their own numbers and many times the meaning was different. One example, I'm not saying that is what happened, but you have an example. Even within the sectors or within the UN agency, many times there were different counters doing different things. So for example, you had at one stage the food security sector making an absolute scale, meaning that the population that were in phase five were really in phase five with the definition of phase five, while you had other sectors making a relative scaling, meaning to distribute all the populations among and you would always end up with phase five. Then when we're looking, it looks like one sector is worse than another sector, but not because they're really indifferently severity level in these sectors because of the method.

Alice Obrecht: So this is the problem that the GF is trying to solve for. Essentially, how do we get to a more objective or more robust picture on who needs humanitarian assistance, where, and what kind of assistance they need, and then base that on some kind of shared way of measurement so that we can hopefully draw comparisons across populations and potentially countries. Here's Katie explaining the basics of how GF works.

Katie Rickard: GF 2.0's starting premise is that sectoral need is the building blocks of how we respond and understand humanitarian need as a sector. So the way it works is that every single sector at the country level will design their own method of how they define sectoral need, following of course their own global guidelines, and they then produce a sectoral population in need figure. So the number of people in wash, the number of people in food security, the number of people in shelter need. Then we take those numbers and we basically try and work out at the area level what the overall population in need is, using a premise that at least the sector with the highest number in need, that should be our kind of starting point because we know that at least 100,000 of its food security are in need and then we need to ideally work out those overlaps. Because we haven't yet fixed this issue of identifying overlaps, at the moment there is a big risk that we're underestimating need when we do this process. And there's a series of checks and balances that's a bit too technical to explain now that we try and use to kind of estimate some of these problems and try and come up with a fair overall number that reflects sort of severity of conditions. Then separately we've also identified that we want to understand as a humanitarian system severity. And severity here is really about trying to understand the urgency of assistance that's required, whether populations at risk of losing their lives tomorrow if we don't assist versus maybe in a year's time if we don't. And so here the GF has a second method that's completely distinct from the first that produces area level severity and they look at some kind of key outcome indicators that we know are really good sort of pieces of evidence of severity such as mortality rates, malnutrition rates, things that we know occur when really really bad conditions are present. And then basically that second process produces an area level severity and that should hopefully inform decision makers in terms of the severity and how quickly and urgently they should prioritize assistance to that population in need that's been identified through the first process. This is a very top line summary but I hope it gives you kind of a sense of how GF 2.0 works in practice.

Alice Obrecht: Right, so I mean should I try to recap what I'm hearing from Leila and Katie about what are the aims and the direction of the GF 2.0?

Yes please Alice. And maybe add in a bit that that we talked about in the interviews that we haven't included here. I mean essentially what I'm understanding is that we want to get to some kind of objective measure, a more objective measure, of the the severity of needs that people are facing in a crisis. And a lot of this actually comes from the food security sector. There's been a long-standing frameworks of measurement that allow for an internationally determined and verified approach to defining when a population is in famine or not, which is defined very loosely or generally as an extreme deprivation of food. So in the food security classification system which is known as the integrated phase classification system you have these five phases and based on a set of clear measures around availability of food and excess mortality you can determine whether a particular area of a country or a particular population is actually in a state of famine or not. And then that's used as the basis for these formal international declarations of famine. And so the idea here is you know these kinds of classifications are important because they help focus the attention right. So it's horrible what's happening to this population in Sudan but by being declared a famine that hopefully means we're going to have higher levels of attention, more resources, more funding going to that population in order to make sure that they get out of famine as quickly as possible with as few deaths as possible. And that's supposed to be kind of the idea or the principle here around humanitarian action as a whole. That we really want to go to where it's most needed where people are facing the most extreme challenges and life-threatening situations and try to prioritize them when we don't have enough funding to kind of go around it and support everybody. And so that's really where GF has come in to say okay we have this for the food security sector but what if we had a similar kind of phase classification for water and sanitation or for shelter right. These other areas of the humanitarian services that have not had a similar level of severity ratings or traffic light system. And so GF 2.0 is coming in and saying okay we want to now have a five-phase system for assessing water and sanitation and these other areas. And then as Katie was just saying the big idea is could we then compare across those different needs. So is it the case that someone who's facing a severe you know acute need for water and sanitation so let's say a phase five is that need comparable to the severity of being in famine. That's kind of the idea. And the GF 2.0 has just had one year to try and make this new system work but it has been rolled out quite significantly. So it's been rolled out across all of the countries over 2023 and 2024. All of the countries who participated in the humanitarian program cycle use the GF 2.0.

Emmeline Kerkvliet: That all sounds very good in theory Alice but in practice if humanitarians are going to start using these figures to provide assistance to one population over another because they've been judged to be in a higher category of severity then you really do need to know that your data is good and reliable and accurate. Absolutely. Absolutely. So to give us some of the country-level realities we spoke to Helen Hittem who's one of Katie's colleagues from Reach who works at country level in a number of crises and she told us more about the ways that these numbers can come under pressure.

Helen Pittam: Yeah I absolutely think there are some flaws in the way that analysis is sometimes done and I think one of the biggest risks is more political practical considerations going into analysis. I actually I won't name the country here but I have sat in meetings where a particular cluster has been asked like okay you've put the whole population in phase three severity. They openly said to like global rep centres oh yeah so I can get more funding. Interesting as well that also didn't go into further levels of severity as all kind of put in the bad but not horrific category and so there's certainly like some concerns around that. I do also think like any analysis is fallible there will also be with all the like best intent in the world the data is going to be limited and sometimes it will be a bit difficult. I would say I think there is an argument for a no regrets policy in terms of if you know there's uncertainty but there is the potential for very high risk. I think that is still worth flagging and I do worry just seeing the way that like the worst kinds of crisis can lead to much less reliable data. The other concern is around how regardless of the data like the data like how other factors also go into prioritization outside of like the analysis and the numbers such as like geopolitical interests or how much attention something will get in the media. I've been very frustrated trying to like speak to people. Contacts I have who work in media saying maybe think about doing anything on Sudan. It's currently the world's largest displacement crisis and I'll get told people are a bit sick of war right now. This is never a perfect process and the most important thing is try and make sure it's at least an objective and sensible one.

Emmeline Kerkvliet: So Helen has given us a really good illustration there of how geopolitical interests and organizational incentives can influence the needs analysis process and that's the problem that the GF is intended to solve and as Helen says trying to provide a process that's as objective and sensible as possible. And on the positive side both Leila and Katie have mentioned examples where the GF is making a difference shedding more light on these pressures and helping analysts to protect their figures.

Leila Oliveira: One beautiful test was for long there was one sector that came there with like a number that was like twice the highest of the second number and then they start and then people said no but look this is twice people found the sector not even original so that's what was interesting and then they start trying to explain and when they breathe the indicator it's obvious then the indicator that they were showing it was cumulative after all the months so they were taking what happened every month and summing and when they present someone said but it is cumulative you are summing that's why it's so high and then they said ah okay okay okay let's look back again what and then we give the time for that so you see so when you go back to the evidence and to the standards and give the truth of the process the discussion is more constructive and we can neutralize more the politics that's what I would say.

Emmeline Kerkvliet: And here's Katie discussing the impacts the GF rollout has had this year. So I think the the GF at least as we see it promises a singular analytical framework that we can use to kind of measure needs to help us prioritize the most vulnerable both within crises and across crises and I think by having this framework in place the hope is we'll have a more transparent and more accountable approach to prioritization in terms of where we put our our resources as humanitarians.

Katie Rickard: In terms of kind of whether that promise has achieved reality I think we're still in really really early days so 2023 we rolled out GF 2.0 for the first time and I think if we're honest and I suspect most would agree I hope with this it wasn't really consistently applied across crises so I think when we look at 2023 results and when I say results I mean specifically the population in need figures I don't think the figures coming out from each crises was a comparable analysis of need and I think that's caused some problems last year because I think it means that we're sort of still struggling as a humanitarian community to have a big picture perspective on on who is most in need and where so we can kind of hold decision makers to account to ensure that the most vulnerable are prioritized first. That said so that's the kind of the bad news the good news is that the kind of introduction of this process and the new analytical framework is a lot more clear in its methods it's a lot more transparent in its methods and so while we're still kind of trying to roll this out in a consistent way I think it's invited and enabled a lot more scrutiny on these numbers.

Alice Obrecht: But I also saw that there's in the in the HRP process the humanitarian response plan process that's that's using these numbers to inform estimated people in need and how much it's going to cost to reach them or to serve them in each country. I think I saw that they were saying they still haven't been able to resolve overlapping numbers of people in need across the sectors. Is that the case that we still don't exactly know these total numbers or how close are we getting to getting a better picture on total numbers in each particular crisis?

Katie Rickard: Yeah so the current method at the moment can't answer those questions because we've decided to try and use as a starting point an estimation of sectoral need like you've described food security need or water need. We then use that as our starting point but that means that we can't necessarily at the moment in the way that the analysis process is designed understand how those needs interact. So the decision was taken when we were deciding these designing these methods was to try and choose the maximum possible population in need for any given sector. So in your example the decision was that a hundred thousand would be in need because at least a hundred thousand people are in need of food security but the risk is that we're actually underestimating need because as you've said if those two population groups don't overlap and actually it's 85,000 people on top of the hundred thousand people that have food security needs we could be quite seriously underestimating need in a crisis and this is one of the the flaws of the GF and something that we're hoping to look into fixing in the years ahead.

Alice Obrecht: Okay so it's clear the methodology still requires some ironing out but seems to be moving in a positive direction overall in terms of creating a stronger more shareable comparable evidence based for understanding needs and that's going to mean we have a better evidence base for deciding where who and what to prioritise. My question still remaining was what does this really mean for decision makers and how can they use this information particularly for donors? I put this question to Katie and here's what she had to say.

Katie Rickard: Yeah so I have quite a lot of sympathy for some of these decision makers because I think they're faced with these really really impossible choices often sitting in these capital offices far away from the realities of the crisis and they're running blind at the moment because we as a system are not collectively doing our job to provide them with the necessary inputs to take principled choices. The really depressing reality is that when you look at the official population in need numbers versus the proportion funded it's actually looking like we're a needs-based system that we are prioritizing according to need. It's only when you look at the data that we actually see that we're not at all prioritizing according to who's most acutely in need and that indicates that right now as a system we're not providing them with the necessary inputs to be able to take those types of principled choices and if we really invested in tightening up our data collection processes to make them participate in robust tightening up the analysis process to make it actually consistently applied rigorous and and not seen as as a sort of political tool we have a fighting chance of giving them population in needs numbers that could actually ensure that those prioritization and allocation decisions are based on need and I think kind of going back to your point despite this sort of growth of more right-wing governments and potentially less appetite for international development and humanitarian aid in a lot of the kind of traditional western donor countries it's kind of impressive that we still see quite a large amount of money still going to humanitarian um crises and I think that shows that there is appetite both politically and domestically to make sure that we don't leave the kind of world's worst populations behind and so I think it's our job as humanitarians to capitalize on the appetite and do our jobs to ensure that we we know who those populations are and we we don't leave them behind.

Alice Obrecht: So in terms of the donor perspective that's Katie Ricard from Reach reminding donors that they still need to be investing in a system that uses this higher quality data in order to make decisions that are properly based on needs and not other factors like the ones that perhaps Helen was talking about that she's seeing at country level but I also wanted to know did donors get what they wanted out of GF 2.0 um was it the system that they had signed up for at the beginning so I asked Layla this question here's what she had to say. I want to maybe ask you a fairly direct and pointed question if that's okay if you're if you're up for it um so agencies for a long time are have been tasked with or have chosen to undertake needs assessments themselves and then they use that data for the basis for their fundraising drives with donors and that presents as many have been arguing in the sector for decades now a conflict of interest because the people who are measuring the size of the problem are also the ones who are getting um the financial resources and payment to then address that problem. I'm wondering just thinking about the GF here and the push to get to a more comparative and robust set of needs assessments do you feel that the donors got what they wanted out of GF?

Leila Oliveira: Thank you so the donors were a part of the GF partnership and they're part of the subcommittee level and the advisory group. I must say that in the start there were a lot of discussions especially from the agents to not have it globally comparable numbers for example because they did want a crisis to be forgotten because another crisis was higher so they actually did it and that's the whole thing of the relative scale. I wanted to highlight here two things. I think the one big problem that we had it is that we keep talking about need prioritization and prioritization is one part of strategic response and we should be giving information for a strategic response which does include prioritization sectors, areas, modalities, types you see. I think very quick there are numbers and I'm going to say like both from the sectors and from the intersectoral that get to related to this the number this are the number of people that need humanitarian relief assistance and that's a problem. It's a problem from the analysts because we are not giving enough information to actually see which different kind of responses we need and it's a problem for the donors because giving relief assistance is extremely expensive and then we end up with this number that I'm saying it gets to warning of most 20 people in the globe that needs humanitarian assistance. Maybe yes, warning to it people are facing humanitarian deprivations. Yes, now quickly connect that to it is humanitarian assistance that's a problem and that's why I see why donors have been very much talking about prioritization and what I keep telling even them to be let broaden our statement from prioritization to strategic response.

Alice Obrecht: So I'm just going to stop the audio with Leila there because I mean what you hopefully also heard at the end there is that she's making this interesting distinction which I think kind of challenges some of the thinking we had around this podcast itself right or the topic that we're really trying to explore here which is Leila's talking about prioritisation on the one hand but strategic response is a separate thing that's not being discussed or looked at as much as it could.

Emmeline Kerkvliet: Yeah but what do you think she really means by strategic response?

Alice Obrecht: Yeah it's a good question so let's keep playing it and she explains a bit more what she has in mind.

Leila Oliveira: But the issue of this prioritization, this comparability is that we need to be clear that it's not the most severe crisis that it is the one that needs to be prioritized. It does need the most severe crisis needs prior to our life same efforts. No we're not going to say that we're not going to give life saving to a crisis that is in phase five and we're going to give life saving to a crisis that is in phase three that's obvious we're not going to do this. But remember we're not working on in which the most severe life saving assistance and that's what I'm talking about we need to think of humanitarian assistance as a strategy so let's make sure that we do this in a strategic way thinking out for the future and each time more to mitigate prevent and address food crisis not only to put in the validate or the more severe one so that is when we acknowledge we don't want to prioritize we don't want to compare it but let's not only focus on that.

Emmeline Kerkvliet: If I'm understanding later correctly what she's saying is that we need to not only think about which one is the most severe crisis but we also need to start thinking about the comparative advantage of addressing crisis sooner rather than when it is at its most severe and trying to get in there early before the worst effects happen and thinking more strategically about what we can realistically achieve.

Alice Obrecht: Yeah no I think I think you're right I think there's two things in there right there's one element which is around recognising who you can best work with from an agency perspective it's recognising the countries that you are going to have a better operational presence in be able to deliver better and that might mean that even if there's a country that has perhaps more severe needs that's not a country that you respond to because you're set up better to respond to in other places right and that also kind of works for donors too because donors will have stronger partnerships with certain implementing agencies than others and will therefore be able to have you know a better insights I suppose into delivering an effective response in some countries over others and yeah as you say the other point is interesting around if we actually want to reduce humanitarian needs if you take those numbers that we have for 2024 that have been produced by the UN Office for the Coordination of Humanitarian Affairs, OCHA, the numbers I cited 300 million people in 72 countries. There's this interesting question that's a very important ethical question as well is it better for us to be prioritizing the immediate short-term needs of those people or to maybe address the needs of half of those people and then try to take preventative measures to bring down that number for 2025. That's kind of the sense that I'm getting from Leila that we need to be a bit more strategic and also think about humanitarian assistance itself as one tool in the toolkit for people who are affected by these crises right.

Emmeline Kerkvliet: And the idea that humanitarian action is just one tool in the toolkit is very similar to what others have been talking about how do we define the boundaries of humanitarian assistance and set clearer parameters of what humanitarian aid aspires to achieve.

Alice Obrecht: Really interesting and I think that's exactly where we need to go in our next episode then.

Alice Obrecht: Update. Since recording these interviews and discussions at the start of 2024, the GAF is being applied now for a second time in the needs assessments that are being gathered for humanitarian response plans for 2025. And as a result of that, some of the conversations we were having at the start of the year around these thorny issues of where the boundaries for humanitarian action lie have gained more energy, but also they've become even thornier. We hope to reflect some of this more current thinking and some of the challenges arising from these assessments in a future podcast episode. For now, in the next couple of episodes, you'll be hearing from people at agencies who have had to deal with massive budget cuts over the past year and how they've navigated these challenges, starting with the ICRC and followed up by the World Food Programme. Thank you for listening.

 (Upbeat Music)

Alice Obrecht: Thank you for listening to Matter of Priorities, a podcast by ALNAP. For resources and links related to what we've talked about in today's episode, please check out the description.

For more information on ALNAP, please visit our website, www.alnap.org. And if you enjoyed this episode, please hit subscribe to our series, wherever you get your podcasts.