Educational experiments often involve assignment of aggregate units such as schools or school districts (statistical clusters) to treatments. Experiments that do so are called cluster randomized experiments. The sensitivity (statistical power, precision of treatment effect estimates, and minimum detectable effect size) of cluster randomized experiments depends on statistical significance level, sample size, and effect size, but also on the variance decomposition among levels of aggregation (as indicated by intraclass correlation or ICC values at each level of aggregation) and the effectiveness of any covariates used to explain variation at different levels of aggregation (as indicated by R2 values at each level of aggregation). We call the ICC and R2 values design parameters because values of these parameters are necessary to design a cluster randomized experiment that has adequate sensitivity.
This database provides empirical estimates of design parameters for two and three level cluster randomized trials that use academic achievement as an outcome variable. These estimates are available for the nation as a whole (based on surveys with national probability samples) and for selected states (based on those states longitudinal data systems, which are essentially an exhaustive sample).
This page provides the information on design parameters that you requested for planning a two level experiment. The search results are organized into three vertical panels. The top panel is a description of the information you requested including the subject matter (math or reading), grade level and school year in which the information was obtained, and a description of the subpopulation that you requested if it is not the whole population.
The bottom two panels give the design parameters for each level of the design. Each row within a panel refers to a different set of covariates that might be used (no covariates, pretest only, demographics only, and both pretest and demographics together).
The key design parameters at level 2 are the intraclass correlation (ICC2) and the R2 value R22. The standard errors SE(ICC2) and SE(R22) reflect the sampling uncertainty of the corresponding design parameters.
The key design parameter at level 1 is the R2 value R12. The standard error (R12) reflects the sampling uncertainty of R12.
This page provides the information on design parameters that you requested for planning a three level experiment. The search results are organized into four vertical panels. The top panel is a description of the information you requested, including the subject matter (math or reading), grade level and school year in which the information was obtained, and the subpopulation that you requested if it is not the whole population.
The bottom three panels give the design parameters for each level of the design. Each row within a panel refers to a different set of covariates that might be used (no covariates, pretest only, demographics only, and both pretest and demographics together).
The key design parameters at level 3 are the level 3 intraclass correlation (ICC3) and the R2 value R32. The standard errors SE(ICC3) and SE(R32) reflect the sampling uncertainty of the corresponding design parameters.
The key design parameters at level 2 are the level 2 intraclass correlation (ICC2) and the R2 value R22. The standard errors SE(ICC2) and SE(R22) reflect the sampling uncertainty of the corresponding design parameters.
The key design parameter at level 1 is the R2 value R12. The standard error (R12) reflects the sampling uncertainty of the estimate of R12.
The tool provides information that is useful for designing experiments with 2 levels (e.g., students within schools), and 3 levels (students within schools within districts) of analysis.
If all of the schools in the study are in the same district, or if district will be used as a fixed blocking effect, then use these design parameters.
If the schools are not all in the same district and districts are not fixed blocking variables, then use these design parameters.
Please provide some basic information to help us understand how the Intraclass Correlation Database is being used. In order to avoid having to register every time you wish to query the Intraclass Correlation Database you can create an account when submitting the registration information.
For more information on how the national estimates were produced, refer to:
Hedges, L. V. & Hedberg, E. C. (2007). Intraclass correlations for planning group-randomized experiments in education. Educational Evaluation and Policy Analysis, 29, 60-87
For more information on how the state estimates were produced, refer to:
Hedges, L. V. & Hedberg, E. C. (2014). Intraclass correlations and covariate outcome correlations for planning 2 and 3 level cluster randomized experiments in education. Evaluation Review.
Acknowledgment: This research was supported in part by grants from the US Institute of Education Sciences and the National Science Foundation