Similarity and Randomization¶

An experiment in fuel efficiency¶

Here's one way to think about the difficulty in our previous analysis of fuel efficiency: the cars with high fuel efficiency were not otherwise similar to the cars with low fuel efficiency. The high-efficiency cars had slower acceleration.

Suppose we were free to design an experiment to learn about the causal effect of high fuel efficiency on price. We would want to create many cars identical to each other in every way except their fuel efficiency. Then any observed association between price and fuel efficiency would indicate that fuel efficiency causes a change in price, since nothing else could have made a difference.

Car companies are in the business of building many copies of the same car, so this is a feasible (but potentially expensive) experiment. We might artificially impede the efficiency of some cars (or simply advertise varying levels of efficiency) and observe the effect on the prices buyers are willing to pay.

Randomized experiments¶

In many other domains, it is difficult to create many identical units for study, varying only the treatment of interest. For example, to study the effect of a medical treatment on humans, we would need many identical humans!

Instead, the most commonly-used tactic is to randomly assign varying levels of the treatment. It is simplest to think about random assignment when there are only two levels of treatment, created by performing some change or no change. The individuals receiving the change are called the treatment group, and those receiving no change are called the control group.

The key point of randomized assignment is this: By randomly assigning individuals to treatment or control, we ensure that the two groups are approximately similar in all ways.

If you are able to randomize individuals into the treatment and control groups, you are running a randomized controlled experiment, also known as a randomized controlled trial (RCT). Sometimes, people’s responses in an experiment are influenced by their knowing which group they are in. So if you are studying people, you might want to run a blind experiment in which individuals do not know whether they are in the treatment group or the control group. To make this work, you will have to give the control group a placebo, which is something that looks exactly like the treatment but in fact has no effect.

Randomized controlled experiments have long been a gold standard in the medical field, for example in establishing whether a new drug works. They are also becoming more commonly used in other fields such as economics.