Submitted by Sarah Bailey on Fri, 30/10/2020 - 17:20
The COVID-19 pandemic has posed many challenges for researchers. Those who usually work in the field have been particularly affected. Here, TIGR2ESS intern Ellie Brain shares her experience of carrying out research remotely. She outlines key principles used for evaluating development initiatives from afar, considering the caveats based on personal experience, but also remote research’s power to highlight research’s pre-existing limitations and what can be done to change them.
I have recently completed a six-week virtual internship as part of TIGR2ESS Flagship Project 1, evaluating a development initiative by identifying trends in income change amongst tribal communities in rural India.
In a post-COVID-19 landscape, this research project was conducted more than 4,700 miles away from the people whose lives I was investigating. My final report contained quantitative and qualitative data analysis, all completed from my bedroom in Newcastle-Upon-Tyne in the North-East of England.
Development initiatives are projects designed to improve the quality of life of communities in sustainable, transformative, and often innovative and creative ways. I was tasked with evaluating the effectiveness of a development initiative carried out along India’s tribal belt over the past five years. The initiative aimed to improve the lives of communities through drastically rising household income over the time period through a variety of strategies involving technological innovation and agricultural adaptation. Reflecting on the success of such projects is crucial to helping development project leads understand the most effective practices for future development initiatives.
How possible is it to ethically and comprehensively research development initiatives from afar? From my own experience, I have found that remote research is possible, and even bears silver linings. However, it is challenging to capture authentic and legitimate insight. Solutions to this are greatly needed for remote research to be reliable.
Pre-Existing Methods of Evaluation
The quantity and quality of the evidence for the impact of development initiatives are relatively low, but contemporary evaluations appear to embody similar principles deemed as ‘effective’ and ‘ethical’ ways of working.
In line with the OECD’s Guidelines for Project and Programme Evaluations, the quality of an evaluation can be determined using the following criteria: objectivity; impartiality; expert knowledge; transparency; reliability; and completeness. These guidelines advocate for projects to consider the differences amongst those who are the focus of development initiatives, including gender, caste (a form of social division in Hindu society), class, race, age, or ecology. Triangulation, combining different research methods when studying the same phenomenon to increase reliability, is encouraged to overcome possible bias in using just one assessment method.
Development initiative design and implementation increasingly collaborate with those participating in and experiencing the project. This has led to development evaluations also starting to engage with corresponding agencies, stakeholders, and even beneficiaries to design and interpret evaluations of development initiatives.
Caveats
Because I was evaluating a development initiative (a) from afar and (b) in a very short space of time, some of the principles outlined by the OECD were simply not achievable. Without visiting the villages that were the focus of the development project, my contextual knowledge and locational insight were lacking.
My distance from the field made me entirely reliant on the data provided by the charitable trust delivering the initiative. The data that I was provided did not inform me as to the participant’s gender, race, or age, therefore it was not possible for me to investigate if the development initiative was more or less successful between different social groups. Further, I could only analyse the limited number of households for which data was collected. The lack of descriptive qualitative data collection, such as household surveys or interviews, or a diverse range of quantitative development measures, limited my research to just one method of analysis.
As well as limitations to my data, I encountered communication challenges. I could contact the development project leads via email and video call, but I was unable to engage with project workers or beneficiaries on the ground. Access to the internet is a privilege for those who can engage in research remotely, and a weakness for remote development evaluations as they cannot hear the voices of the people whose lives they attempt to assess.
Capabilities?
The limitations I have outlined paint a relatively bleak picture for remote research outcomes and by extension the prospects of research for the foreseeable future. However, in theme with Geographies of Hope, a topic which characterised much of my undergraduate degree at Cambridge, there are some silver linings to the new normal for field-based research.
Capitalising on Quantitative and Secondary Analysis
My dependency on secondary data collected by others and from literature produced by public and charitable organisations meant than I invested much more time understanding existing information than I would have in a traditional development project evaluation.
Taking the time to find information collected by secondary sources led me to discover an abundance of existing knowledge about the specific sites I was working with from think tanks and Non-Governmental Organisations (NGOs).
Making do with the limited quantitative data available also compelled me to think outside of the box for ways to analyse the data. In turn, I reached interesting and surprising conclusions I may not otherwise have reached. Sharing research produced by others and digging deeper was not only necessary for remote research but was also an asset I would transfer to future research conducted on-site.
Taking Time to Reflect
Taking time to reflect on research methods presents an opportunity. Despite conducting quantitative statistical analysis commonly deemed best placed to attain the ‘gold standard’ for research, I have encountered several limitations during my internship.
Opening up an honest, comprehensive conversation about limitations to research methods could pave the way for generating productive solutions. Are reports that claim to evaluate development initiatives, but only use quantitative analysis, adequately capturing the depth of change taking place at those sites? Remote research lays bare the limitations of the OECD’s principles and illustrates that quantitative analysis must be improved to holistically evaluate development initiatives.
A variety of changes could be made; quantitative analysis could be broadened to look at a variety of factors beyond change and qualitative analysis could be included to give a voice to those affected by the project. What is undeniable is that the ‘gold standard’ of research needs to change, and that current engagements in remote research may highlight this need as academia as a whole notices the limitations in their research.
Towards a Remote Research Future: How can we be participatory from afar?
What are the prospects for evaluations that engage with local communities and development initiative beneficiaries in an era where research is taking place almost exclusively over Zoom? How can we engage with all the stakeholders involved in development initiatives? There are no easy solutions. However, my project has highlighted that this is a hurdle that must be tackled if research conducted from afar is to succeed.
I thank my internship supervisor Dr Shreya Sinha and Flagship Project 1 Lead Professor Bhaskar Vira for their support throughout this project. I would also like to thank the whole TIGR2ESS FP1 team for their valuable insights for my final report.