Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice

Thumbnail image of item number 1 in: 'Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice'.
Thumbnail image of item number 2 in: 'Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice'.
Thumbnail image of item number 3 in: 'Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice'.
Thumbnail image of item number 4 in: 'Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice'.
Showing 1-4 of 86 pages in this dissertation.

PDF Version Also Available for Download.

Description

Reliability generalization (RG) is a method for meta-analysis of reliability coefficients to estimate average score reliability across studies, determine variation in reliability, and identify study-level moderator variables influencing score reliability. A total of 107 peer-reviewed RG studies published from 1998 to 2013 were systematically reviewed to characterize the meta-analytic methods employed and to evaluate quality of reporting practice against standards for transparency in meta-analysis reporting. Most commonly, RG studies meta-analyzed alpha coefficients, which were synthesized using an unweighted, fixed-effects model applied to untransformed coefficients. Moderator analyses most frequently included multiple regression and bivariate correlations employing a fixed-effects model on untransformed, … continued below

Physical Description

iv, 81 pages : illustrations

Creation Information

Holland, David F. (Educational consultant) December 2015.

Context

This dissertation is part of the collection entitled: UNT Theses and Dissertations and was provided by the UNT Libraries to the UNT Digital Library, a digital repository hosted by the UNT Libraries. It has been viewed 1696 times, with 6 in the last month. More information about this dissertation can be viewed below.

Who

People and organizations associated with either the creation of this dissertation or its content.

Author

Chair

Committee Members

Publisher

Rights Holder

For guidance see Citations, Rights, Re-Use.

  • Holland, David F.

Provided By

UNT Libraries

The UNT Libraries serve the university and community by providing access to physical and online collections, fostering information literacy, supporting academic research, and much, much more.

About | Browse this Partner

Contact Us

Corrections Questions

What

Descriptive information to help identify this dissertation. Follow the links below to find similar items on the Digital Library.

Degree Information

Description

Reliability generalization (RG) is a method for meta-analysis of reliability coefficients to estimate average score reliability across studies, determine variation in reliability, and identify study-level moderator variables influencing score reliability. A total of 107 peer-reviewed RG studies published from 1998 to 2013 were systematically reviewed to characterize the meta-analytic methods employed and to evaluate quality of reporting practice against standards for transparency in meta-analysis reporting. Most commonly, RG studies meta-analyzed alpha coefficients, which were synthesized using an unweighted, fixed-effects model applied to untransformed coefficients. Moderator analyses most frequently included multiple regression and bivariate correlations employing a fixed-effects model on untransformed, unweighted coefficients. Based on a unit-weighted scoring system, mean reporting quality for RG studies was statistically less than that for a comparison study of 198 meta-analyses in the organizational sciences across 42 indicators; however, means were not statistically significantly different between the two studies when evaluating reporting quality on 18 indicators deemed essential to ethical reporting practice in meta-analyses. Since its inception a wide variety of statistical methods have been applied to RG, and meta-analysis of reliability coefficients has extended to fields outside of psychological measurement, such as medicine and business. A set of guidelines for conducting and reporting RG studies is provided.

Physical Description

iv, 81 pages : illustrations

Language

Item Type

Identifier

Unique identifying numbers for this dissertation in the Digital Library or other systems.

Collections

This dissertation is part of the following collection of related materials.

UNT Theses and Dissertations

Theses and dissertations represent a wealth of scholarly and artistic content created by masters and doctoral students in the degree-seeking process. Some ETDs in this collection are restricted to use by the UNT community.

About | Browse this Collection

What responsibilities do I have when using this dissertation?

Digital Files

When

Dates and time periods associated with this dissertation.

Creation Date

  • December 2015

Added to The UNT Digital Library

  • March 20, 2016, 10:34 a.m.

Description Last Updated

  • March 14, 2025, 12:44 p.m.

Usage Statistics

When was this dissertation last used?

Yesterday: 0
Past 30 days: 6
Total Uses: 1,696
More Statistics

Interact With This Dissertation

Here are some suggestions for what to do next.

Start Reading

Thumbnail image of item number 1 in: 'Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice'.
Thumbnail image of item number 2 in: 'Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice'.
Thumbnail image of item number 3 in: 'Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice'.
Thumbnail image of item number 4 in: 'Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice'.

PDF Version Also Available for Download.

Citations, Rights, Re-Use

International Image Interoperability Framework

IIF Logo

We support the IIIF Presentation API

Links for Robots

Helpful links in machine-readable formats.

Archival Resource Key (ARK)

International Image Interoperability Framework (IIIF)

Metadata Formats

Images

URLs

Stats

Holland, David F. (Educational consultant). Reliability Generalization: a Systematic Review and Evaluation of Meta-analytic Methodology and Reporting Practice, dissertation, December 2015; Denton, Texas. (https://digital.library.unt.edu/ark:/67531/metadc822810/: accessed June 17, 2025), University of North Texas Libraries, UNT Digital Library, https://digital.library.unt.edu; .

Back to Top of Screen