Data collection in the context of the pandemic posed multiple challenges for researchers. At IPA Colombia, this situation led us to explore and test new tools like WhatsApp surveys.1 Here we’d like to share some of the specific situations in which Whatsapp has been a useful alternative to in-person data collection and some of the lessons we’ve learned in the process.
Use of WhatsApp Surveys at IPA Colombia
One of the many advantages of using WhatsApp as a tool to contact participants is how versatile it is. It allows for large-scale simultaneous interactions with participants through not only text but also using media files like images, documents, and audio. This facilitates interactions since it offers multiple, friendlier, and demonstrative ways to communicate messages that participants might not have read, or would have been difficult to understand.
The low cost, effectiveness, and versatility have made it useful for different kinds of projects. For example, one study on the impact of Colombia’s decision to offer legal status to Venezuelan immigrants used WhatsApp surveys on this very mobile population because even if phone numbers change, WhatsApp numbers typically stay stable. This meant we could collect data directly over WhatsApp, but also use it to get their current phone number for follow-up phone surveys.
Projects such as this entrepreneurship training one, and this one on social protection during COVID-19 used WhatsApp to communicate project information and schedule calls for phone surveys. While another project on teaching socio-emotional skills to young children used WhatsApp surveys as a tool to monitor potential spillovers by tracking whether the control group participants have received resources related to the treatment in-between data collections.
Positive Side Effects and Limitations
Similarly, by allowing the sending of personalized messages, this data collection method may help to create a closer relationship between participants, the projects, and IPA as an organization. For example, in the surveys conducted through WhatsApp in the project with Venezuelan migrants mentioned above, we could personalize messages with names and other information we had, with which we intended to offer a more personal sense of connection. But we might have been too successful and had a slight backfire—it wasn’t obvious to some participants that they were interacting with an automated system, rather than a person, and that led to some frustration. We learned to explain explicitly upfront that the responses they were getting to their answers were automatically generated.
There are also some advantages to using a verified WhatsApp business account—we use the IPA logo as our profile picture, which we think cues people to remember who we are when we re-contact them. As we noticed when doing in-person fieldwork with participants who’d been previously contacted through WhatsApp—they seemed to remember IPA from the WhatsApp logo.
Having an official account also means the phone number from which messages will be sent is verified. We think this feature helps to increase participant trust—when the number is verified, participants see the organization name in the chat, instead of the number, with a check symbol that certifies the authenticity of the WhatsApp account. We think this in turn may lead to increased response rates. Of course, it doesn’t work the other way—WhatsApp doesn’t confirm a participant’s identity so if we are re-contacting a number we typically recommend including questions to confirm if the person answering on the other end is the person we’re actually trying to reach.
If you’ve ever sent a WhatsApp message and waited for the double checkmarks to appear confirming receipt, you know how valuable that information can be. The same is true in the survey process. WhatsApp surveys also allow tracking message receipt and reading. The reports allow us to identify failed deliveries due to a phone number out of service, or without a WhatsApp associated, along with messages that were received and read. This is useful for understanding how well the data collection is going, and for identifying when participants are more difficult to reach. It is worth noting, though, that whether read information is available depends on how the individual has set up their privacy preferences.
If you are choosing WhatsApp for data collection, it is worth considering its limitations. First, it is useful with a specific type of population with internet access, smartphones, and some level of technological knowledge, which will limit your sampling to people from specific geographical areas, ages, and socioeconomic backgrounds. This wasn’t an important restriction in the studies described above as most participants had access to WhatsApp and were used to it. It is also difficult to guarantee the quality of the responses for some types of questions, such as multiple selection open-ended questions since there is not an enumerator verifying if the participant fully understood the question. To address this restriction, the projects described above used survey questions that were easy to understand and simple to self-assess.
Every data collection medium has advantages and disadvantages, and we are not comparing all options here (we hope to have another post with more technical details and comparisons soon). But for IPA Colombia, particularly during the disruptions of COVID-19, we have found WhatsApp’s versatility a tremendous asset for data collection, despite some small challenges, and the more experience we get, the better we get at using it. If you think it might be useful for your study, we recommend giving it a try.
The first project that used WhatsApp surveys was implemented in partnership with Stanford University’s Immigration Policy Lab (IPL) and Mercy Corps. IPL designed a webpage, a technical manual, and a code repository that provide case studies from IPL projects, webinars, and instructional videos offering a demonstration of the tool in use, a practical guideline to understanding the minimum must-do's for launching a WhatsApp survey, and a detailed step-by-step covering system setup, survey design, and troubleshooting.
