Differences in Differences

Using Regression to Understand Differences, Interactions, and Multiple Explanations

Bogdan G. Popescu

John Cabot University

Intro

How can we identify causal effects using observational data?

Some of the ways to conduct causal analysis with observational data, we need to run

  • differences in differences
  • regression discontinuity design

First Study

Minimum Wage and Number of Jobs

Does raising minimum wage increase the number of jobs? (Card and Kruger, 1993)

Background

  • New Jersey changed the minimum wage from $4.25 per hour to $5.05 in 1992.
  • The number of jobs per fast food restaurant the change was: 20.44
  • The number of jobs per fast food restaurant the change was: 21.03

\[ \text{New Jersey}_{\text{Before}} = 20.44\\ \text{New Jersey}_{\text{After}} = 21.03\\ \Delta = 0.59 \]

Why can’t we interpret \(\Delta = 0.59\) as causal?

First Study

Minimum Wage and Number of Jobs

Is \(\Delta = 0.59\) a causal effect?

  • It is not a causal effect
  • This is because we only look at the treatment group
  • It is impossible to know if the treatment happened because of the treatment (i.e. change in minimum wage) vs. other factors that happen simultaneously or because of the natural volatility of jobs.

First Study

Minimum Wage and Number of Jobs

To answer the question, Card and Kruger compare New Jersey to a neighboring state: Pennsylvania.

\[ \text{Pennsylvania}_{\text{After}} = 21.17\\ \text{New Jersey}_{\text{After}} = 21.03\\ \Delta = -0.14 \]

Is \(\Delta = -0.14\) a causal effect?

  • It is not a causal effect
  • This is because we only look at post-treatment outcomes.
  • It is impossible to know the effect of the treatment: New Jersey and Pennsylvania may be completely different.

Framework

Difference-in-Differences Table

Group Before After
Control A - not treated B - not treated
Treatment C - not treated D - treated

Framework

Difference-in-Differences Table

Group Before After Δ (After - Before)
Control A - not treated B - not treated B - A
Treatment C - not treated D - treated D - C

Δ (After - Before) = within-unit change

Framework

Difference-in-Differences Table

Group Before After Δ (After - Before)
Control A - not treated B - not treated B - A
Treatment C - not treated D - treated D - C
Δ (Treatment - Control) C - A D - B

Δ (After - Before) = within-unit change
Δ (Treatment - Control) = across-group change

Framework

Difference-in-Differences Table

Group Before After Δ (After - Before)
Control A - not treated B - not treated B - A
Treatment C - not treated D - treated D - C
Δ (Treatment - Control) C - A D - B (D - C) - (B - A)

Δ (After - Before) = within-unit change
Δ (Treatment - Control) = across-group change

Framework

Difference-in-Differences Table

Group Before After Δ (After - Before)
Control A - not treated B - not treated B - A
Treatment C - not treated D - treated D - C
Δ (Treatment - Control) C - A D - B (D - B) - (C - A)

Δ (After - Before) = within-unit change
Δ (Treatment - Control) = across-group change

Framework

Difference-in-Differences Table

Group Before After Δ (After - Before)
Control A
23.33
B
21.17
B − A
−2.16
Treatment C
20.44
D
21.03
D − C
0.59
Δ (Treatment − Control) C − A
−2.89
D − B
−0.14
(0.59 − −2.16) or
(−0.14 − −2.89)

Δ (After - Before) = within-unit change
Δ (Treatment - Control) = across-group change

Framework

Difference-in-Differences Table

Group Before After Δ (After - Before)
Control A
23.33
B
21.17
B − A
−2.16
Treatment C
20.44
D
21.03
D − C
0.59
Δ (Treatment − Control) C − A
−2.89
D − B
−0.14
2.75 or
2.75

Δ (After - Before) = within-unit change
Δ (Treatment - Control) = across-group change

Kard and Krueger

Controversy

Conventional wisdom (in economics)
Raising the minimum wage reduces employment due to higher labor costs.

Card & Krueger’s finding:
After New Jersey raised the minimum wage, employment increased slightly at fast-food restaurants compared to Pennsylvania.

Methodological innovation:
They used a natural experiment with a difference-in-differences approach — unusual at the time for labor economics.

Differences in Differences

Differences in Differences

Differences in Differences

Differences in Differences

Differences in Differences

Differences in Differences

Differences-in-Differences

The way we can estimate the causal effect is by running the following model:

1. Model

\[ Y_{it} = \beta_0 + \color{blue}{\beta_1 \cdot \text{Group}_i} + \color{purple}{\beta_2 \cdot \text{Time}_t} + \color{red}{\beta_3 \cdot (\text{Group}_i \times \text{Time}_t)} + \epsilon_{it} \]

2. Code

mod <- lm(outcome ~ group + time + group * time, data = final_new)

Where:

-Group = 1 if this is the treatment group
-Time = 1 if this is the period after intervention
-β₀ – mean of the control group in the pre-treatment period
-β₁ – the increase in outcome across groups
-β₂ – the increase in outcome within groups
-β₃ – the Differences-in-Differences

Framework for Causal Effects

1. Model

\[ Y_{it} = \beta_0 + \color{blue}{\beta_1 \cdot \text{Group}_i} + \color{purple}{\beta_2 \cdot \text{Time}_t} + \color{red}{\beta_3 \cdot (\text{Group}_i \times \text{Time}_t)} + \epsilon_{it} \]

Group Before After Δ (After − Before)
Control β₀ β₀ + β₂ β₂
Treatment β₀ + β₁ β₀ + β₁ + β₂ + β₃ β₂ + β₃
Δ (Treatment − Control) β₁ β₁ + β₃ β₃

\(\Delta\) within units − \(\Delta\) across groups = Difference-in-differences = causal effect

Assumptions

Diff-in-Diff Assumptions

Parallel Trends Assumption

The treatment and the control group have the same trends prior to the intervention.

We assume that the treatment group would have changed like the control group in the absence of the treatment.

Timing
Sometimes, units receive treatment at different times, so this can distort our estimates.

Assumptions

Diff-in-Diff Assumptions

This is an example where the parallel trends assumption holds

Assumptions

Diff-in-Diff Assumptions

Another example where parallel trends hold is the following:

Assumptions

Diff-in-Diff Assumptions

An example where the parallel trend is violated, is the following:

Assumptions

Diff-in-Diff Assumptions

Another example where the parallel trend is violated, is the following:

Treatment Timing

Units can receive observations at different times which can distort our estimate:

Treatment Timing

Units can receive observations at different times which can distort our estimate: