In a new paper published in JMIR, we explore the feasibility of conducting a full-scale trial to test the impact of automated IVR counseling sessions on contraception take up among women with an unmet need for family planning.
An interesting challenge was that the service was widely available throughout the proposed study area in western Kenya. Anyone could use it. This meant that we could not randomize access to the service as you might in a typical randomized controlled trial with a treatment arm and a control arm.
Instead, we turned to something called a randomized encouragement trial. In this design, you randomize an encouragement to try the service rather than randomizing access or the timing of access. An encouragement can be anything that increases the likelihood that someone encouraged will take up the service. We used SMS messages. Women randomized to the encouragement arm received the messages on the left in Figure 1.
Figure 1: Messages sent to each study arm.
Encouragement trials can be a good choice when take up of the intervention is less than 100%. For example, Devoto et al. (2012) used an encouragement design to study the impact of installing piped water to Moroccan dwellings on household wellbeing. In 2007, a private utility company began offering interest-free loans to connect households to the water supply. They did not expect every eligible household to apply, so they agreed to allow the researchers to randomly “encourage” half of the households. The encouragement consisted of information, loan pre-approval, and paperwork assistance. It worked! 69% of households in the encouragement arm purchased a connection compared to only 10% in the control arm, a difference of 59 percentage points.
We did something similar in this pilot trial and randomly assigned women who enrolled in the study to get a set of encouragement messages or a set of control messages. In our study, 34% of women in the treatment arm tried the intervention compared to only 1.8% of women in the control arm. This is a differential rate of take up of 32 percentage points (see Figure 1).
Figure 2: Impact estimate.
We found that assignment to the treatment group (ie, assignment to receive an encouragement to try the intervention) led to an increase of 13 percentage points in the likelihood of contraception use. This is the reduced form estimate (ie, the effect of the invitation—encouragement—on the uptake of contraception). The causal effect of interest, however, is the ratio of the reduced form estimate to the first stage estimate: 41 percentage points. This effect is known as LATE, and it represents the average causal effect for women whose use of the intervention was determined only through the random encouragement to try the intervention. In other words, it is the effect of using the intervention on contraceptive uptake. The sign of this estimate appears to be positive, but the confidence interval is wide.
This was a small pilot study with substantial attrition, so it’s important not to over interpret these preliminary findings. Check out the full paper for more about what we learned regarding study recruitment, encouragement delivery, and follow-up data collection procedures.
Full paper
Green EP, Augustine A, Naanyu V, Hess AK, Kiwinda L Developing a Digital Marketplace for Family Planning: Pilot Randomized Encouragement Trial. J Med Internet Res 2018;20(7):e10756
DOI: 10.2196/10756
PMID: 30064968
PMCID: 6092593
Data and Code
You can find the data and code for this paper here.
Devoto, Florencia, Esther Duflo, Pascaline Dupas, William Parienté, and Vincent Pons. 2012. “Happiness on Tap: Piped Water Adoption in Urban Morocco.” American Economic Journal: Economic Policy 4 (4): 68–99.