Endnote
How do we conduct a randomized controlled trial?
The method of randomization can be as simple as tossing a coin. It may also be quite a bit more complex. But every method of randomization consists of a sequence of carefully defined steps that allow chances to be specified mathematically. This has two important consequences.
It allows us to account–mathematically–for the possibility that randomization produces treatment and control groups that are quite different from each other.
It allows us to make precise mathematical statements about differences between the treatment and control groups. This in turn helps us make justifiable conclusions about whether the treatment has any effect.
In this course, you will learn how to conduct and analyze your own randomized experiments. That will involve more detail than has been presented in this section. For now, just focus on the main idea: to try to establish causality, run a randomized controlled experiment if possible. If you are conducting an observational study, you might be able to establish association, but any conclusions about causality will be on shaky ground at best. Be extremely careful about confounding factors before making conclusions about causality based on an observational study.
Why don't we conduct RCTs to answer every question?
In many situations it might not be possible to carry out a randomized controlled experiment, even when the aim is to investigate causality. For example, suppose you want to study the effects of alcohol consumption during pregnancy, and you randomly assign some pregnant women to your "alcohol" group. You should not expect cooperation from them if you present them with a drink. In such situations you will almost invariably be conducting an observational study, not an experiment. Be alert for confounding factors.
Should we ever question RCTs?
Yes! RCTs are known as the "gold standard" for causal inference, but they are still vulnerable to many problems unrelated to confounding factors. For example, because PROGRESA was an RCT, researchers could establish that the study caused an increase in investment income among the villages studied. But this does not necessarily hold for other groups outside the study. Cultural or other factors might have made PROGRESA uniquely effective in rural Mexico in the 1990s. We cannot be absolutely sure that a similar program in rural Africa in 2017 would work equally well.
Terminology
- observational study
- treatment
- outcome
- association
- causal association
- causality
- comparison
- treatment group
- control group
- confounding
- randomization
- randomized controlled experiment
- randomized controlled trial (RCT)
Fun facts and good reads
According to an article in the LA Times citing a UCLA study by Matthew E. Kahn, the area around UC Berkeley has the highest rate in California of hybrid car registration: 5.24% in 2009.
In an RCT that started in 2017 in Kenya, the nonprofit GiveDirectly will give people in randomly-chosen villages roughly $22 per month for the next 12 years. You can read about it here. Some people hope that direct cash payments will be more efficient than previous efforts at alleviating poverty. Similar randomized studies have been run before, but with shorter durations. In what ways might a long-running study be more informative?
Poor Economics, the best seller by Abhijit V. Banerjee and Esther Duflo of MIT, is an accessible and lively account of ways to fight global poverty. It includes numerous examples of RCTs, including the PROGRESA example in this section.