library(stacked)
library(data.table)
data(medicaid)Understanding Kappa Trade-offs in Stacked Difference-in-Differences
How kappa_pre and kappa_post shape the event window, sample, and whether the aggregation choice can matter — with the design screen.
Overview
This vignette explains the kappa_trade_offs() function, which helps researchers understand how different choices of kappa parameters affect their stacked difference-in-differences analysis. Understanding these trade-offs is essential for making informed decisions about your event study window.
* Load the bundled medicaid dataset
stacked use medicaid, clearWhat Are Kappa Parameters?
In stacked difference-in-differences, the kappa parameters define the event study window around each treatment adoption date: - kappa_pre: Number of pre-treatment periods to include (before adoption) - kappa_post: Number of post-treatment periods to include (after adoption)
For example, with kappa_pre = 3 and kappa_post = 2, the event study window spans 6 periods: event times -3, -2, -1, 0, 1, 2.
Timeline for a unit adopting treatment in 2014:
Calendar: 2011 2012 2013 2014 2015 2016
Event: -3 -2 -1 0 1 2
|___________________|____________|
Pre-treatment Post-treatmentWhy Do Kappa Values Matter?
Choosing kappa values involves fundamental trade-offs between the length of your event window and the number of adoption cohorts that can be included.
Feasibility Constraints and External Validity
Larger kappa values require more data on either side of the adoption date. Adoption cohorts near the boundaries of your data become infeasible and must be trimmed from the analysis:
- If your data spans 2008-2021 and you set
kappa_pre = 5, any cohort adopting before 2013 becomes infeasible (they would need data from before 2008) - Similarly, large
kappa_postvalues exclude late adopters
This trimming has implications for external validity: the treatment effect estimates apply only to the adoption cohorts that remain in the analysis. If you trim many cohorts, your estimates may not generalize to the full population of adopters.
Event Window Length
The length of the event window affects what you can learn:
Pre-treatment periods (
kappa_pre): More pre-treatment periods provide stronger evidence about the validity of the parallel trends assumption. If pre-treatment coefficients are near zero, this supports the identifying assumptions.Post-treatment periods (
kappa_post): More post-treatment periods allow you to study dynamic treatment effects and how impacts evolve over time. Short-term and long-term effects may differ substantially.
Statistical Power
Sample size affects the precision of your estimates. The kappa_trade_offs() function reports observation counts to help you assess whether you have adequate statistical power for your chosen specification.
Using kappa_trade_offs()
The function explores all combinations of kappa values you specify:
tradeoffs <- kappa_trade_offs(
data = medicaid,
time_var = "year",
unit_var = "state",
adopt_var = "adopt_year",
kappa_pre_range = 1:4,
kappa_post_range = 1:4,
control_type = "both"
)
# View results
print(tradeoffs)
#> kappa_pre kappa_post n_sub_exp n_unidentified_sub_exp n_event_times n_obs
#> 1: 1 1 5 0 3 354
#> 2: 1 2 4 0 4 400
#> 3: 1 3 3 0 5 425
#> 4: 1 4 3 0 6 480
#> 5: 2 1 5 0 4 472
#> 6: 2 2 4 0 5 500
#> 7: 2 3 3 0 6 510
#> 8: 2 4 3 0 7 560
#> 9: 3 1 5 0 5 590
#> 10: 3 2 4 0 6 600
#> 11: 3 3 3 0 7 595
#> 12: 3 4 3 0 8 640
#> 13: 4 1 5 0 6 708
#> 14: 4 2 4 0 7 700
#> 15: 4 3 3 0 8 680
#> 16: 4 4 3 0 9 720
#> total_treated_obs total_control_obs avg_treated_per_period
#> 1: 114 240 38
#> 2: 140 260 35
#> 3: 165 260 33
#> 4: 198 282 33
#> 5: 152 320 38
#> 6: 175 325 35
#> 7: 198 312 33
#> 8: 231 329 33
#> 9: 190 400 38
#> 10: 210 390 35
#> 11: 231 364 33
#> 12: 264 376 33
#> 13: 228 480 38
#> 14: 245 455 35
#> 15: 264 416 33
#> 16: 297 423 33
#> avg_control_per_period sd_treated_per_period sd_control_per_period
#> 1: 80 0 0
#> 2: 65 0 0
#> 3: 52 0 0
#> 4: 47 0 0
#> 5: 80 0 0
#> 6: 65 0 0
#> 7: 52 0 0
#> 8: 47 0 0
#> 9: 80 0 0
#> 10: 65 0 0
#> 11: 52 0 0
#> 12: 47 0 0
#> 13: 80 0 0
#> 14: 65 0 0
#> 15: 52 0 0
#> 16: 47 0 0
#> adoption_times_str
#> 1: 2014, 2015, 2016, 2019, 2020
#> 2: 2014, 2015, 2016, 2019
#> 3: 2014, 2015, 2016
#> 4: 2014, 2015, 2016
#> 5: 2014, 2015, 2016, 2019, 2020
#> 6: 2014, 2015, 2016, 2019
#> 7: 2014, 2015, 2016
#> 8: 2014, 2015, 2016
#> 9: 2014, 2015, 2016, 2019, 2020
#> 10: 2014, 2015, 2016, 2019
#> 11: 2014, 2015, 2016
#> 12: 2014, 2015, 2016
#> 13: 2014, 2015, 2016, 2019, 2020
#> 14: 2014, 2015, 2016, 2019
#> 15: 2014, 2015, 2016
#> 16: 2014, 2015, 2016* Explore all combinations of kappa values (grid over kpre and kpost)
stacked kappa, time(year) unit(state) adopt(adopt_year) ///
kpre(1/4) kpost(1/4) controltype(both)
* The full grid is returned in r(table)
matrix list r(table)Understanding the Output
The function returns a data.table with one row per (kappa_pre, kappa_post) combination. Here’s what each column means:
Core Parameters
- kappa_pre: Pre-treatment periods in the event window
- kappa_post: Post-treatment periods in the event window
- n_sub_exp: Number of feasible sub-experiments (adoption cohorts)
- n_event_times: Total event-time periods (kappa_pre + kappa_post + 1)
Observation Counts
- n_obs: Total observations in the stacked dataset
- total_treated_obs: Total treated observations across all periods/sub-experiments
- total_control_obs: Total control observations across all periods/sub-experiments
Per-Period Statistics
- avg_treated_per_period: Average treated observations per event-time period
- avg_control_per_period: Average control observations per event-time period
- sd_treated_per_period: Standard deviation of treated observations across periods
- sd_control_per_period: Standard deviation of control observations across periods
Cohort Information
- adoption_times_str: Which adoption years are included
The Key Relationship
The output satisfies a fundamental relationship:
n_obs = total_treated_obs + total_control_obsLet’s verify this:
# Check for kappa_pre=3, kappa_post=2
row <- tradeoffs[kappa_pre == 3 & kappa_post == 2]
print(row)
#> kappa_pre kappa_post n_sub_exp n_unidentified_sub_exp n_event_times n_obs
#> 1: 3 2 4 0 6 600
#> total_treated_obs total_control_obs avg_treated_per_period
#> 1: 210 390 35
#> avg_control_per_period sd_treated_per_period sd_control_per_period
#> 1: 65 0 0
#> adoption_times_str
#> 1: 2014, 2015, 2016, 2019
# Verify the relationship
cat("n_obs:", row$n_obs, "\n")
#> n_obs: 600
cat("total_treated + total_control:",
row$total_treated_obs + row$total_control_obs, "\n")
#> total_treated + total_control: 600
cat("Match:", row$n_obs == row$total_treated_obs + row$total_control_obs, "\n")
#> Match: TRUECompositional Balance in the Stacking Method
A key feature of the stacking method is that it enforces compositional balance: the exact same set of units appears in every event-time period within a sub-experiment.
For example, with state-by-year panel data:
- If New York is a treated state in sub-experiment 1 (the 2014 adoption cohort), then New York appears in sub-experiment 1 for every event-time period (-3, -2, -1, 0, 1, 2)
- If California is a clean control for sub-experiment 2, it appears in every event-time for sub-experiment 2
This means that coefficient estimates for each event time come from the same underlying set of treated units and controls. There are no compositional changes across event times that could confound the interpretation of dynamic treatment effects.
This is different from naive approaches that might include different adoption cohorts in different event-time periods, leading to misleading patterns that reflect compositional changes rather than true treatment effect dynamics.
Verifying with build_stack()
The statistics reported by kappa_trade_offs() match exactly what build_stack() produces:
# Build the actual stacked dataset
stack <- build_stack(
data = medicaid,
time_var = "year",
unit_var = "state",
adopt_var = "adopt_year",
kappa_pre = 3,
kappa_post = 2,
control_type = "both"
)
# Get the prediction from kappa_trade_offs
row <- tradeoffs[kappa_pre == 3 & kappa_post == 2]
# Verify n_obs
cat("Predicted n_obs:", row$n_obs, "\n")
#> Predicted n_obs: 600
cat("Actual nrow(stack):", nrow(stack), "\n")
#> Actual nrow(stack): 600
cat("Match:", row$n_obs == nrow(stack), "\n\n")
#> Match: TRUE
# Verify treated/control counts
cat("Predicted total_treated_obs:", row$total_treated_obs, "\n")
#> Predicted total_treated_obs: 210
cat("Actual sum(treat==1):", sum(stack$treat == 1), "\n")
#> Actual sum(treat==1): 210
cat("Match:", row$total_treated_obs == sum(stack$treat == 1), "\n\n")
#> Match: TRUE
cat("Predicted total_control_obs:", row$total_control_obs, "\n")
#> Predicted total_control_obs: 390
cat("Actual sum(treat==0):", sum(stack$treat == 0), "\n")
#> Actual sum(treat==0): 390
cat("Match:", row$total_control_obs == sum(stack$treat == 0), "\n")
#> Match: TRUE* Build the actual stacked dataset for kpre=3, kpost=2
stacked use medicaid, clear
stacked build, time(year) unit(state) adopt(adopt_year) ///
kpre(3) kpost(2) controltype(both)
* build stores the same counts the kappa table predicts
display "n_obs = " r(n_obs)
display "treated = " r(n_treated)
display "control = " r(n_control)Repeated Cross-Section Data and the Standard Deviation Statistics
The sd_treated_per_period and sd_control_per_period columns require some explanation. For balanced panel data (like the medicaid dataset where we observe each state in each year), these values will be zero:
# For balanced panels, sd is zero
cat("sd_treated_per_period:", row$sd_treated_per_period, "\n")
#> sd_treated_per_period: 0
cat("sd_control_per_period:", row$sd_control_per_period, "\n")
#> sd_control_per_period: 0This is because compositional balance ensures the same states appear in every event-time period, so the count of treated and control observations is identical across periods.
However, the stacking method can also be applied to repeated cross-sectional microdata (e.g., individual-level survey data where different people are sampled each year). In that case:
- Compositional balance still holds at the group level: the same states appear in each event-time period within a sub-experiment
- But the number of individual observations may vary across event-time periods because different individuals are sampled each year
In the repeated cross-section case, sd_treated_per_period and sd_control_per_period will be positive, reflecting this variation in sample sizes across event-time periods. This is not a violation of compositional balance—it’s simply a feature of repeated cross-sectional sampling.
Impact of Control Type
The control_type parameter affects how many control observations are available. Let’s compare the three options:
# Both never-treated and not-yet-treated
tradeoffs_both <- kappa_trade_offs(
medicaid, "year", "state", "adopt_year",
kappa_pre_range = 3, kappa_post_range = 2,
control_type = "both"
)
# Only never-treated
tradeoffs_never <- kappa_trade_offs(
medicaid, "year", "state", "adopt_year",
kappa_pre_range = 3, kappa_post_range = 2,
control_type = "never_treated"
)
# Only not-yet-treated
tradeoffs_notyet <- kappa_trade_offs(
medicaid, "year", "state", "adopt_year",
kappa_pre_range = 3, kappa_post_range = 2,
control_type = "not_yet_treated"
)
# Compare
comparison <- data.table(
control_type = c("both", "never_treated", "not_yet_treated"),
n_obs = c(tradeoffs_both$n_obs,
tradeoffs_never$n_obs,
tradeoffs_notyet$n_obs),
total_treated = c(tradeoffs_both$total_treated_obs,
tradeoffs_never$total_treated_obs,
tradeoffs_notyet$total_treated_obs),
total_control = c(tradeoffs_both$total_control_obs,
tradeoffs_never$total_control_obs,
tradeoffs_notyet$total_control_obs)
)
print(comparison)
#> control_type n_obs total_treated total_control
#> 1: both 600 210 390
#> 2: never_treated 474 210 264
#> 3: not_yet_treated 324 198 126* Compare the three control-type options for kpre=3, kpost=2
stacked use medicaid, clear
stacked kappa, time(year) unit(state) adopt(adopt_year) ///
kpre(3) kpost(2) controltype(both)
stacked kappa, time(year) unit(state) adopt(adopt_year) ///
kpre(3) kpost(2) controltype(never)
stacked kappa, time(year) unit(state) adopt(adopt_year) ///
kpre(3) kpost(2) controltype(notyet)Notice that:
- Treated counts are identical across all control types (the same cohorts are treated regardless of control group choice)
- Control counts vary: “both” has the most controls, as it includes both never-treated and not-yet-treated units
- “both” has the most observations overall
Choosing Kappa Values: Practical Guidance
When choosing kappa values, consider these four key factors:
1. External Validity
Look at n_sub_exp and adoption_times_str to understand which adoption cohorts are included. Ask yourself:
- Are the included cohorts representative of the policy changes you care about?
- Are important early or late adopters being trimmed?
- Would your conclusions change if different cohorts were included?
2. Dynamic Treatment Effects
Consider how treatment effects might evolve over time:
- If effects take time to materialize, you need adequate
kappa_post - If you expect effects to fade or grow, a longer post-period helps distinguish these patterns
3. Testing Parallel Trends
A longer pre-treatment period (kappa_pre) provides stronger evidence about the parallel trends assumption:
- Multiple pre-treatment periods should show coefficients near zero
- A single pre-treatment period provides limited evidence
- Consider whether your setting warrants more scrutiny of pre-trends
4. Statistical Power
Check total_treated_obs and total_control_obs to assess power:
- Very small samples may lead to imprecise estimates
- Consider whether you have enough observations for your research design
# Look for combinations that balance these factors
print(tradeoffs[order(-n_sub_exp, -n_obs)])
#> kappa_pre kappa_post n_sub_exp n_unidentified_sub_exp n_event_times n_obs
#> 1: 4 1 5 0 6 708
#> 2: 3 1 5 0 5 590
#> 3: 2 1 5 0 4 472
#> 4: 1 1 5 0 3 354
#> 5: 4 2 4 0 7 700
#> 6: 3 2 4 0 6 600
#> 7: 2 2 4 0 5 500
#> 8: 1 2 4 0 4 400
#> 9: 4 4 3 0 9 720
#> 10: 4 3 3 0 8 680
#> 11: 3 4 3 0 8 640
#> 12: 3 3 3 0 7 595
#> 13: 2 4 3 0 7 560
#> 14: 2 3 3 0 6 510
#> 15: 1 4 3 0 6 480
#> 16: 1 3 3 0 5 425
#> total_treated_obs total_control_obs avg_treated_per_period
#> 1: 228 480 38
#> 2: 190 400 38
#> 3: 152 320 38
#> 4: 114 240 38
#> 5: 245 455 35
#> 6: 210 390 35
#> 7: 175 325 35
#> 8: 140 260 35
#> 9: 297 423 33
#> 10: 264 416 33
#> 11: 264 376 33
#> 12: 231 364 33
#> 13: 231 329 33
#> 14: 198 312 33
#> 15: 198 282 33
#> 16: 165 260 33
#> avg_control_per_period sd_treated_per_period sd_control_per_period
#> 1: 80 0 0
#> 2: 80 0 0
#> 3: 80 0 0
#> 4: 80 0 0
#> 5: 65 0 0
#> 6: 65 0 0
#> 7: 65 0 0
#> 8: 65 0 0
#> 9: 47 0 0
#> 10: 52 0 0
#> 11: 47 0 0
#> 12: 52 0 0
#> 13: 47 0 0
#> 14: 52 0 0
#> 15: 47 0 0
#> 16: 52 0 0
#> adoption_times_str
#> 1: 2014, 2015, 2016, 2019, 2020
#> 2: 2014, 2015, 2016, 2019, 2020
#> 3: 2014, 2015, 2016, 2019, 2020
#> 4: 2014, 2015, 2016, 2019, 2020
#> 5: 2014, 2015, 2016, 2019
#> 6: 2014, 2015, 2016, 2019
#> 7: 2014, 2015, 2016, 2019
#> 8: 2014, 2015, 2016, 2019
#> 9: 2014, 2015, 2016
#> 10: 2014, 2015, 2016
#> 11: 2014, 2015, 2016
#> 12: 2014, 2015, 2016
#> 13: 2014, 2015, 2016
#> 14: 2014, 2015, 2016
#> 15: 2014, 2015, 2016
#> 16: 2014, 2015, 2016The Design Screen: Will the Aggregation Choice Matter?
Beyond sample sizes, a kappa choice determines something subtler: whether an unweighted stacked regression and the Q-weighted estimator can disagree. Both aggregate the same sub-experiment effects, but with different weights: the Q-weighted estimator uses the treated shares \(w_a^T = m_a^D / \sum_b m_b^D\) (the target), while an unweighted stacked regression implicitly uses precision weights \(w_a^S = S_a / \sum_b S_b\) with \(S_a = m_a^D m_a^C / (m_a^D + m_a^C)\). Both weight vectors are functions of the design alone — no outcome data required — so their total-variation divergence
\[D_\kappa = \sum_a \left| w_a^S - w_a^T \right|\]
can be screened before estimation. When \(D_\kappa\) is near zero the two estimators cannot differ much whatever the treatment effects turn out to be; when it is large, the unweighted estimate is at the mercy of effect heterogeneity. The classic danger configuration is a limited clean-control pool: \(S_a\) saturates at the pool size, so a cohort of 1,000 units and a cohort of 10 receive nearly the same precision weight while their treated shares differ by 100-fold.
Request the screen with screen = TRUE:
screened <- kappa_trade_offs(
medicaid,
time_var = "year",
unit_var = "state",
adopt_var = "adopt_year",
kappa_pre_range = 1:4,
kappa_post_range = 1:4,
screen = TRUE
)
print(screened[, .(kappa_pre, kappa_post, n_sub_exp, D, n_valid_sub_exp,
n_dropped_sub_exp, min_control_mass, max_treated_share,
top_gap_cohort)])
#> kappa_pre kappa_post n_sub_exp D n_valid_sub_exp n_dropped_sub_exp
#> 1: 1 1 5 0.3144696 5 0
#> 2: 1 2 4 0.3125317 4 0
#> 3: 1 3 3 0.2652806 3 0
#> 4: 1 4 3 0.2568542 3 0
#> 5: 2 1 5 0.3144696 5 0
#> 6: 2 2 4 0.3125317 4 0
#> 7: 2 3 3 0.2652806 3 0
#> 8: 2 4 3 0.2568542 3 0
#> 9: 3 1 5 0.3144696 5 0
#> 10: 3 2 4 0.3125317 4 0
#> 11: 3 3 3 0.2652806 3 0
#> 12: 3 4 3 0.2568542 3 0
#> 13: 4 1 5 0.3144696 5 0
#> 14: 4 2 4 0.3125317 4 0
#> 15: 4 3 3 0.2652806 3 0
#> 16: 4 4 3 0.2568542 3 0
#> min_control_mass max_treated_share top_gap_cohort
#> 1: 11 0.7368421 2014
#> 2: 11 0.8000000 2014
#> 3: 16 0.8484848 2014
#> 4: 13 0.8484848 2014
#> 5: 11 0.7368421 2014
#> 6: 11 0.8000000 2014
#> 7: 16 0.8484848 2014
#> 8: 13 0.8484848 2014
#> 9: 11 0.7368421 2014
#> 10: 11 0.8000000 2014
#> 11: 16 0.8484848 2014
#> 12: 13 0.8484848 2014
#> 13: 11 0.7368421 2014
#> 14: 11 0.8000000 2014
#> 15: 16 0.8484848 2014
#> 16: 13 0.8484848 2014* Add the design-screen columns to the kappa grid
stacked use medicaid, clear
stacked kappa, time(year) unit(state) adopt(adopt_year) ///
kpre(1/4) kpost(1/4) screen
* Screen columns (D, n_valid_sub_exp, ...) are in r(table)
matrix list r(table)The new columns:
- D: the divergence \(D_\kappa\) (0 = the aggregation choice cannot matter; the maximum is 2)
- n_valid_sub_exp / n_dropped_sub_exp: cohorts with positive treated and control mass enter D; cohorts with neither cannot be estimated by any stacked regression and are excluded (a large drop count is its own warning)
- min_control_mass: the smallest control mass among valid cohorts — a cohort identified off one or two controls makes any estimate fragile, whatever the weights
- max_treated_share and top_gap_cohort: where the action is
The per-cohort weight vectors behind D are returned as an attribute, which is the table to inspect when D is large:
cohort_wts <- attr(screened, "cohort_weights")
print(cohort_wts[kappa_pre == 2 & kappa_post == 2])
#> kappa_pre kappa_post focal m_treated m_control valid w_target w_precision
#> 1: 2 2 2014 28 18 TRUE 0.80000000 0.64373417
#> 2: 2 2 2015 3 18 TRUE 0.08571429 0.15108047
#> 3: 2 2 2016 2 18 TRUE 0.05714286 0.10575633
#> 4: 2 2 2019 2 11 TRUE 0.05714286 0.09942903* The per-cohort masses and weight vectors are returned in r(cohort_weights)
stacked use medicaid, clear
stacked kappa, time(year) unit(state) adopt(adopt_year) ///
kpre(2) kpost(2) screen
matrix list r(cohort_weights)Here the 2014 Medicaid cohort holds 80% of the treated mass but only 64% of the precision weight. Whether that matters depends on the third ingredient — whether effects differ across cohorts in a way that lines up with the weight gap — which only the estimates can reveal. The screen is necessary, not sufficient: low D settles the question in advance, high D says “check.”
Screen under the masses your target uses
If your estimand weights people rather than units (e.g. weight_type = "pop_total_periods" in build_stack()), screen under the same masses — a design balanced in unit counts can be badly unbalanced in population:
kappa_trade_offs(dt, "year", "state", "adopt_year",
screen = TRUE,
weight_type = "pop_total_periods", pop_var = "pop")* Screen under population person-period masses
stacked kappa, time(year) unit(state) adopt(adopt_year) ///
screen weighttype(pop_total_periods) popvar(pop)weight_type accepts the same options as build_stack(); survey weights go through weight_var.
Balancing
On unbalanced panels, balance = "complete_window" computes the masses only over units observed at all kappa_pre + kappa_post + 1 event times — matching an analysis that enforces compositional balance. This can change the masses substantially and can even change which cohorts are estimable, so screen with the balancing you intend to impose:
kappa_trade_offs(dt, "year", "unit", "adopt_year",
screen = TRUE, balance = "complete_window")* Restrict masses to units observed at every event time in the window
stacked kappa, time(year) unit(unit) adopt(adopt_year) ///
screen balance(complete_window)The screen also works unchanged with duck = TRUE (one extra grouped aggregation inside DuckDB), so it is cheap even on larger-than-memory data. We recommend reporting \(D_\kappa\) and the cohort weight table alongside the event study: it turns the choice of aggregation from an invisible default into a visible design decision.
Summary
The kappa_trade_offs() function is an essential planning tool for stacked difference-in-differences analysis. It helps you:
- Understand feasibility: See which adoption cohorts are included/excluded and assess implications for external validity
- Preview sample sizes: Know exactly how many observations you’ll have for assessing statistical power
- Check composition: Understand the treated/control breakdown, knowing that compositional balance is enforced by the stacking method
- Compare options: Evaluate multiple kappa combinations quickly
- Screen the design: With
screen = TRUE, learn before touching outcomes whether the unweighted and Q-weighted aggregates can disagree - Verify results: Output matches exactly what
build_stack()produces
Use this function before committing to specific kappa values to make informed decisions about your event study specification, balancing the competing demands of external validity, measuring dynamic effects, testing parallel trends, and maintaining adequate statistical power.