We’ve all probably heard the phrase “correlation is not causation,” which is the idea that just because you observe a relationship between two things, you can’t know for sure that one caused the other.

For us at Vote Forward, that means we can’t just look at raw voter turnout numbers to learn whether our handwritten letters caused people to go out and vote. We need to run an experiment! To do this, we split registered voters into two groups: people who get letters (“treatment”), and people who don’t (“control”). If the people who receive letters vote at a higher rate, we can say with greater confidence that our letters impacted their behavior.

However, we still need to be sure that our letters are the only difference between the “treatment” and “control” groups. Otherwise, some other factor—for example, age—could be behind the observed turnout differences. That’s where randomization comes in. By randomly assigning voters to each group, we know that on average, the groups will look about the same when it comes to a number of important factors. Then, we can be more sure that it really is the handwritten letters, and not any of those other factors, causing higher turnout.

But don’t just take our word for it! Randomized experiments (sometimes called “RCTs”) are considered the gold standard in many fields.

Randomized experiments, or RCTs, are the gold standard for testing a specific type of question: Did our program cause a particular outcome? In our case, did our letters cause more people to turn out to vote? But often we want to learn other things about our programs, particularly as we expand to new, innovative tactics. 

Before we can run an experiment to understand the effect of our new tactic on voters, we need to monitor how it unfolds in practice. To do that, we use interviews, focus groups, surveys, and programmatic data to answer questions like: How many voters were we able to contact? Did volunteers find the work engaging and worthwhile? Where did the process work well, where did it work less well, and what would it look like if we scaled up? 

Answering these questions doesn’t require randomizing voters into a treatment and control group. But, this research is just as important as our RCTs, it just answers very different questions. It helps us refine new tactics and scale effectively. Once a program is up and running, then we can test it with a randomized experiment.

A “control group” is just the group of voters who don’t receive any letters in a given experiment. You might be thinking, “But don’t you want to encourage as many people as possible to vote?” Believe us, we do, and it hurts our hearts a little bit to leave any voters out.

However, as we discussed above, experiments are all about comparison—and that means that if we want to learn whether our efforts impacted turnout, we need to compare the effect of Vote Forward letters to something. That “something” is our control group. To know that we had a meaningful impact, we need to show that lots of people who got Vote Forward letters turned out to vote— and that they voted at a higher rate than the people who didn’t get letters (our control group).

While we don’t love leaving anyone out, the tradeoff is worth it to learn about how letter writing makes an impact. (Plus, we’re still writing millions of letters together.) Running experiments helps us learn faster about what’s working and where we can improve, and it gives our volunteers the evidence to feel confident that their time and energy is well spent.

Because getting people who don’t vote to show up at the polls (or mail in a ballot) is actually pretty hard. Most of the factors that affect whether people vote are tied to big, structural issues (like voter ID laws or polling place accessibility) or deep-seated attitudes (e.g., a lack of faith that elections matter). Given these obstacles, boosting turnout by even a small amount is a real achievement! And, when it comes to politics, we know that many races are decided by tight margins, so a small boost in turnout can translate into a meaningful difference in electoral outcomes.

“Social” campaigns are non-political, “core social good” campaigns. In these campaigns, volunteers typically write letters to registered voters from traditionally underrepresented groups who we believe are unlikely to vote.

“Political” campaigns are political campaigns. In these campaigns, volunteers typically write letters to Democratic-leaning registered voters who we believe are unlikely to vote.

To calculate voter turnout for an election, we rely on public election data that’s held by each state’s government. After an election happens, the government needs time to collect that data, clean it up, and make it available to people like us. Then, our team gets to work analyzing those results. We’re typically ready to share our findings with volunteers a few months after the election has taken place—make sure to sign up for our email updates to receive the latest data and research news.

Our experiments are run by Vote Forward’s in-house Data and Research team. We’ve also had external analysts help us with analyzing our results in the past. Right now, we don’t have the capacity to bring on volunteers to help with our data work—but when we do, we’ll let you know! In the meantime, you can help out our team by making a donation to support this work.