In one of the most popular TED talks, Nigerian novelist Chimamanda Ngozi Adichie argues that ‘The single story creates stereotypes, and the problem with stereotypes is not that they are untrue, but that they are incomplete. They make one story become the only story’. Although this comment is grounded in stories told in literature, the quote is equally relevant for researchers collecting and analysing data. After all, as Brené Brown says in another popular TED talk: ‘stories are just data with a soul’.
The widespread recognition that mixed methods approaches - combining data or ‘stories’ - can provide a more nuanced and in-depth understanding of the subject matter illustrates the relevance of stories as a metaphor for data. This certainly also holds true for poverty studies and impact evaluations, whereby the mixing of methods allows for fine-grained assessments that tell the story from multiple perspectives.
Some of the most popular reasons for using mixed methods approaches include verification and triangulation purposes: making sure that findings are supported by different sets of data. This follows the recognition that single methods may only provide select pieces of the puzzle, or reflect an incorrect or biased picture, but that a combination of methods allows for drilling down to the core and drawing a comprehensive picture. But this also gives rise to a problem: what if the stories don’t add up? What if the mixing of data and methods reveals different pictures? Are they to be dismissed and discarded? Or might there be truth in disagreement?
Recent mixed methods research into child poverty outcomes in Burundi, Ethiopia and Vietnam highlights the importance of not shying away from discrepant findings when using multiple data sources but of finding truth in exploring such discrepancies. The use of a participatory tool in combining quantitative data from secondary sources and qualitative data that was primarily collected revealed that different data may tell different stories. For example, only roughly one third of children identified as living in monetary poor households but not experiencing multidimensional poverty and vice versa using quantitative survey data were also identified as such using the participatory tool.
Discrepant findings in this research can be explained by methodological challenges, including the use of different sets of indicators and units of analysis in the quantitative and qualitative methods, conceptualisation of child wellbeing and respondents’ reluctance to suggest that parents in their community were unable to secure good child wellbeing. As these challenges relate to both types of methods, it is not possible to say which method is more accurate and therefore more reflective of the ‘truth’. Yet it is in exploring those challenges that stories are revealed that are equally if not more compelling than verified or triangulated pieces of information may be.
The challenge of differential use of units of analysis and resultant differential findings highlight how situations for individual children in the same household may differ. Findings in Burundi showed that while one child may be well fed and regularly attending school, another might find him or herself in a more precarious situation. This emphasises the need to consider children as individuals in poverty analysis or evaluation studies rather than as equal parts of the household unit. The differential set of criteria for child wellbeing in quantitative and qualitative data highlight the local and very context-specific understandings of when a child is living a good life. While helping out at home was considered an important determinant of child wellbeing in Ethiopia, the extent to which children followed parents’ advice was frequently mentioned in Vietnam.
Notwithstanding the power of mixed methods approaches to allow for confirmatory analysis in poverty studies and impact evaluations, the notion that findings from qualitative and quantitative data should add up to the same conclusion is a false one. There is as much truth in agreement as there is in disagreement, and much is to be gained in mixed methods research when giving greater weight to disagreements. To illustrate with one more quote: ‘There are three sides to every story: your side, my side, and the truth’ (Robert Evans).
Image: Illustration of findings from ‘Reducing poverty in the first 18 years of life in Burundi’, by Jorge Martin