This study compared 3 approaches for handling a fourth level of nesting structure when analyzing data from a cluster-randomized trial (CRT). CRTs can include 3 levels of nesting: repeated measures, individual, and cluster levels. However, above the cluster level, there may sometimes be an additional potentially important fourth level of nesting (e.g., schools, districts, etc., depending on the design) that is typically ignored in CRT data analysis. The current study examined the impact of ignoring this fourth level, accounting for it using a model-based approach, and accounting it using a design-based approach on parameter and standard error (SE) estimates. Several fixed effect and random effect variance parameters and SEs were biased across all 3 models. In the 4-level model, most SE biases decreased as the number of level 3 clusters increased and as the number of level 4 clusters decreased. Also, random effect variance biases decreased as the number of level 3 clusters increased. In the 3-level and complex models, SEs became more biased as the weight level 4 carried increased (i.e., larger intraclass correlation, more clusters at that level). The current results suggest that if a meaningful fourth level of nesting exists, future researchers should account for it using design-based approach; the model-based approach is not recommended. If the fourth level is not practically important, researchers may ignore it altogether.
