Intraclass Correlation Coefficients Typical of Cluster-Randomized Studies: Estimates From the Robert Wood Johnson Prescription for Health Projects

David M. Thompson; Douglas H. Fernald; James W. Mold

doi:10.1370/afm.1347

Article Figures & Data

Tables

Additional Files

View popup

Table 1

Variables Analyzed

Variable	Level of Measurement	Description
Practice-level variables
Number of physicians (FTE)	Continuous
Number of staff (FTE)	Continuous
Physician turnover rate	Continuous	Calculated as the number who left in past year, divided by total number
Staff turnover rate	Continuous	Calculated as the number who left in past year, divided by total number
Practice type	Binary	Solo or single specialty practice vs multispecialty practice
Use of electronic health record	Binary	Yes/no
Patient-level demographic variables
Age	Continuous	Measured in years
Sex	Binary	Male vs female
Race	Binary	Nonwhite vs white
Patient-level measures of unhealthy behaviors
Average number of drinks per day or month	Continuous
Intention to reduce drinking alcohol	Continuous^a	Applied only to patients who reported drinking at least 10 drinks in the last month
Smoking status	Binary	Smokers were identified as those who smoked at least part of a cigarette in the last 30 days
Intention to quit smoking	Continuous^a	Applied only to patients who reported, on either the pre- or postintervention questionnaire, that they were former smokers, current smokers, or smokers trying to quit
Unhealthy diet	Binary	Unhealthy diet was defined as failure to consume 5 servings of fruit and vegetables a day.
Intent to improve diet	Continuous^a	Applied only to patients whose responses indicated their diet was unhealthy
Physical inactivity	Binary	Physical inactivity was defined as no report of moderate or vigorous activity, nor of a 10-minute period of walking in the last 7 days
Minutes of physical activity on average day	Continuous	Calculated from reports of vigorous and moderate activity along with walking
Intent to start an exercise program	Continuous^a	Applied to patients who reported less than 90 minutes of vigorous or moderate physical activity, including walking, in the last 7 days

FTE=full-time equivalent.
↵a Intention variables were measured on a 5-point ordinal scale, but were treated as continuous measures to calculate intraclass correlation coefficients.

View popup

Table 2

ICCs (and 95% Empirical Bootstrap Cls), Adjusted Cluster Sizes (m), and VIFs Calculated for Patient-Level Variables (N = 5,042 Patients)

		Within 61 Practices			Within 8 PBRNs

Variable	n	ICC (95% CI)	m	VIF	ICC (95% CI)	m	VIF
Age	4,984	0.151 (0.144–0.191)	80.08	12.94	0.054 (0.043–0.071)	554.35	30.63
Sex	5,004	0.050 (0.050–0.089)	80.38	4.99	0.010 (0.006–0.019)	555.93	6.68
Race	5,042	0.265 (0.246–0.296)	81.01	22.23	0.152 (0.133–0.175)	560.20	86.19
Smoking status	4,893	0.118 (0.117–0.187)	78.60	10.19	0.072 (0.059–0.099)	543.43	40.15
Unhealthy diet	4,922	0.206 (0.178–0.252)	79.04	17.11	0.239 (0.197–0.284)	545.48	131.26
Inactivity	4,787	0.064 (0.062–0.095)	70.23	5.43	0.062 (0.054–0.092)	484.4	30.87
Minutes of physical activity per day	4,639	0.053 (0.051–0.094)	74.54	23.49	0.057 (0.046–0.082)	519.25	30.75
Average drinks per day	3,312	0.076 (0.067–0.142)	53.25	4.98	0.076 (0.054–0.111)	360.80	28.23
Average drinks per month	433	0.001 (0.000–0.103)	46.86	1.06	Data from 9 clinics within a single PBRN
Intent to reduce drinking	193	0.207 (0.002–0.600)	18.18	4.56	Data from 9 clinics within a single PBRN
Intent to quit smoking	378	0.000 (0.000–0.075)	40.74	1.00	Data from 9 clinics within a single PBRN
Intent to improve diet	1,355	0.012 (0.003–0.037)	148.26	2.76	Data from 9 clinics within a single PBRN
Intent to increase exercise	917	0.007 (0.000–0.042)	100.27	1.65	Data from 9 clinics within a single PBRN

ICC= Intraclass correlation coefficient; PBRN = practice-based research network; VIF = variance inflation factor.

View popup

Table 3

ICCs (With 95% Empirical Bootstrap CIs) and VIFs Calculated for Practice-Level Variables, Collected on 89 Practices Within 8 PBRNs

	Within Network Statistics

Practice Level Variable	ICC (95% CI)	VIF
Practice type	0.294 (0.187–0.499)	3.82
Use of electronic medical record	0.229 (0.101–0.406)	3.20
Number of physician FTEs	0.053 (0.000–0.427)	1.51
Number of staff FTEs	0.036 (0.000–0.407)	1.35
Number of staff/physician FTEs	0.062 (0.000–0.393)	1.60
Physician turnover rate	0.110 (0.029–0.564)	2.06
Staff turnover rate	0.066 (0.000–0.327)	1.63

FTE=Full-time equivalent; ICC=intraclass correlation coefficient; PBRN=practice-based research network; VIF = variance inflation factor.
Note: crude mean cluster size = 11.125; adjusted mean cluster size = 10.59 clinics per network.

Additional Files

Tables

Supplemental Appendix

Supplemental Appendix. The Variance Inflation Factor (VIF)

Files in this Data Supplement:

Supplemental data: Appendix - PDF file, 1 page, 205 KB
The Article in Brief

David M. Thompson, and colleagues

Background Research conducted in practice-based research networks often randomizes interventions, not by individuals, but according to a natural clustering unit, such as the physician or the practice in which patients receive care. Researchers who conduct cluster-randomized studies must explicitly account for clustering at every stage of design and analysis to avoid underpowered studies. This requires good estimates of clustering effects in the form of intraclass correlation coefficients (ICCs). This study uses data from the Robert Wood Johnson Foundation's Prescription for Health program to estimate ICCs for demographic and behavioral variables and for physician and practice characteristics.

What This Study Found The authors analyzed data from 5,042 patients in 61 practices and 8 practice-based research networks. They found that ICCs for certain measures of health behavior and intent to change those behaviors are small, generally less than 0.1. Clustering is less evident for outcome variables than for other independent and process variables.

Implications

Though small, the ICCs in this report are not trivial; if cluster sizes are large, even small levels of clustering, if unaccounted for, can reduce a study�s statistical power.