Challenges in measuring ACGME competencies: considerations for milestones

PDF Version Also Available for Download.

Description

This article extends previous studies of the Accreditation Council for Graduate Medical Education (ACGME)'s measure of residents' training progress to examine the validity of ACGME International competency measures for an international setting across emergency medicine (EM) and neurology, and across evaluators.

Physical Description

6 p.

Creation Information

Natesan, Prathiba; Batley, Nicholas J.; Bakhti, Rinad & El-Doueihi, Philippe Z. September 28, 2018.

Context

This article is part of the collection entitled: UNT Scholarly Works and was provided by UNT College of Education to Digital Library, a digital repository hosted by the UNT Libraries. It has been viewed 25 times , with 9 in the last month . More information about this article can be viewed below.

Who

People and organizations associated with either the creation of this article or its content.

Authors

Organizer of meeting

Publisher

Provided By

UNT College of Education

The UNT College of Education prepares professionals and scholars who contribute to the advancement of education, health, and human development. Programs in the college prepare teachers, leaders, physical activity and health specialists, educational researchers, recreational leaders, child development and family studies specialists, doctoral faculty, counselors, and special and gifted education teachers and leaders.

Contact Us

What

Descriptive information to help identify this article. Follow the links below to find similar items on the Digital Library.

Degree Information

Description

This article extends previous studies of the Accreditation Council for Graduate Medical Education (ACGME)'s measure of residents' training progress to examine the validity of ACGME International competency measures for an international setting across emergency medicine (EM) and neurology, and across evaluators.

Physical Description

6 p.

Notes

Abstract: Background: Measuring milestones, competencies, and sub-competencies as residents progress through a training
program is an essential strategy in Accreditation Council for Graduate Medical Education (ACGME)’s attempts to ensure
graduates meet expected professional standards. Previous studies have found, however, that physicians make global
ratings often by using a single criterion.
Methods: We use advanced statistical analysis to extend these studies by examining the validity of ACGME
International competency measures for an international setting, across emergency medicine (EM) and neurology, and
across evaluators. Confirmatory factor analysis (CFA) models were fitted to both EM and neurology data. A singlefactor
CFA was hypothesized to fit each dataset. This model was modified based on model fit indices.
Differences in how different EM physicians perceived the core competencies were tested using a series of
measurement invariance tests.
Results: Extremely high alpha reliability coefficients, factor coefficients (> .93), and item correlations indicated
multicollinearity, that is, most items being evaluated could essentially replace the underlying construct itself.
This was true for both EM and neurology data, as well as all six EM faculty.
Conclusions: Evaluation forms measuring the six core ACGME competencies did not possess adequate validity. Severe
multicollinearity exists for the six competencies in this study. ACGME is introducing milestones with 24 subcompetencies.
Attempting to measure these as discrete elements, without recognizing the inherent weaknesses in the
tools used will likely serve to exacerbate an already flawed strategy. Physicians likely use their “gut feelings” to judge
a resident’s overall performance. A better process could be conceived in which this subjectivity is acknowledged,
contributing to more meaningful evaluation and feedback.

Source

  • International Journal of Emergency Medicine, 2018. London, UK: BioMed Central, Ltd.

Language

Item Type

Identifier

Unique identifying numbers for this article in the Digital Library or other systems.

Publication Information

  • Publication Title: International Journal of Emergency Medicine
  • Volume: 11
  • Issue: 39
  • Page Start: 1
  • Page End: 6
  • Peer Reviewed: Yes

Collections

This article is part of the following collection of related materials.

UNT Scholarly Works

Materials from the UNT community's research, creative, and scholarly activities and UNT's Open Access Repository. Access to some items in this collection may be restricted.

What responsibilities do I have when using this article?

When

Dates and time periods associated with this article.

Creation Date

  • September 28, 2018

Submitted Date

  • May 23, 2018

Accepted Date

  • August 26, 2018

Added to The UNT Digital Library

  • Oct. 31, 2018, 6:58 p.m.

Usage Statistics

When was this article last used?

Yesterday: 0
Past 30 days: 9
Total Uses: 25

Interact With This Article

Here are some suggestions for what to do next.

Start Reading

PDF Version Also Available for Download.

International Image Interoperability Framework

IIF Logo

We support the IIIF Presentation API

Natesan, Prathiba; Batley, Nicholas J.; Bakhti, Rinad & El-Doueihi, Philippe Z. Challenges in measuring ACGME competencies: considerations for milestones, article, September 28, 2018; London, United Kingdom. (https://digital.library.unt.edu/ark:/67531/metadc1310113/: accessed March 23, 2019), University of North Texas Libraries, Digital Library, https://digital.library.unt.edu; crediting UNT College of Education.