Use of Phillips's five level training evaluation and ROI framework in the U.S. nonprofit sector.

Description:

This study examined training evaluation practices in U.S. nonprofit sector organizations. It offered a framework for evaluating employee training in the nonprofit sector and suggested solutions to overcome the barriers to evaluation. A mail survey was sent to 879 individuals who were members of, or had expressed an interest in, the American Society for Training and Development. The membership list consisted of individuals who indicated association/nonprofit or interfaith as an area of interest. Data from the survey show that training in the nonprofit sector is evaluated primarily at Level 1 (reaction) and Level 2 (learning). It also shows decreasing use from Level 3 (application) through Level 5 (ROI). Reaction questionnaires are the primary method for collecting Level 1 data. Facilitator assessment and self-assessment were listed as the primary method for evaluating Level 2. A significant mean rank difference was found between Level 2 (learning) and the existence of an evaluation policy. Spearman rho correlation revealed a statistically significant relationship between Level 4 (results) and the reasons training programs are offered. The Kruskal-Wallis H test revealed a statistically significant mean rank difference between "academic preparation" of managers with Level 3 evaluation. The Mann-Whitney U test was used post hoc and revealed that master's degree had a higher mean rank compared to bachelor's degree and doctorate. The Mann-Whitney U test revealed that there were statistically significant mean rank differences on Level 1, Level 2, Level 3, and Level 5 evaluation use with the barriers "little perceived value to the organization," "lack of training or experience using this form of evaluation," and "not required by the organization." Research findings are consistent with previous research conducted in the public sector, business and industry, healthcare, and finance. Nonprofit sector organizations evaluate primarily at Level 1 and Level 2. The existence of a written policy increases the use of Level 2 evaluation. Training evaluation is also an important part of the training process in nonprofit organizations. Selecting programs to evaluate at Level 5 is reserved for courses which are linked to organizational outcomes and have the interest of top management.

Creator(s): Brewer, Travis K.
Creation Date: August 2007
Partner(s):
UNT Libraries
Collection(s):
UNT Theses and Dissertations
Usage:
Total Uses: 12,800
Past 30 days: 55
Yesterday: 0
Creator (Author):
Publisher Info:
Publisher Name: University of North Texas
Place of Publication: Denton, Texas
Date(s):
  • Creation: August 2007
  • Digitized: September 13, 2007
Description:

This study examined training evaluation practices in U.S. nonprofit sector organizations. It offered a framework for evaluating employee training in the nonprofit sector and suggested solutions to overcome the barriers to evaluation. A mail survey was sent to 879 individuals who were members of, or had expressed an interest in, the American Society for Training and Development. The membership list consisted of individuals who indicated association/nonprofit or interfaith as an area of interest. Data from the survey show that training in the nonprofit sector is evaluated primarily at Level 1 (reaction) and Level 2 (learning). It also shows decreasing use from Level 3 (application) through Level 5 (ROI). Reaction questionnaires are the primary method for collecting Level 1 data. Facilitator assessment and self-assessment were listed as the primary method for evaluating Level 2. A significant mean rank difference was found between Level 2 (learning) and the existence of an evaluation policy. Spearman rho correlation revealed a statistically significant relationship between Level 4 (results) and the reasons training programs are offered. The Kruskal-Wallis H test revealed a statistically significant mean rank difference between "academic preparation" of managers with Level 3 evaluation. The Mann-Whitney U test was used post hoc and revealed that master's degree had a higher mean rank compared to bachelor's degree and doctorate. The Mann-Whitney U test revealed that there were statistically significant mean rank differences on Level 1, Level 2, Level 3, and Level 5 evaluation use with the barriers "little perceived value to the organization," "lack of training or experience using this form of evaluation," and "not required by the organization." Research findings are consistent with previous research conducted in the public sector, business and industry, healthcare, and finance. Nonprofit sector organizations evaluate primarily at Level 1 and Level 2. The existence of a written policy increases the use of Level 2 evaluation. Training evaluation is also an important part of the training process in nonprofit organizations. Selecting programs to evaluate at Level 5 is reserved for courses which are linked to organizational outcomes and have the interest of top management.

Degree:
Level: Doctoral
Language(s):
Subject(s):
Keyword(s): Evaluation | training evaluation | return-on-investment | ROI
Contributor(s):
Partner:
UNT Libraries
Collection:
UNT Theses and Dissertations
Identifier:
  • OCLC: 191847844 |
  • ARK: ark:/67531/metadc3996
Resource Type: Thesis or Dissertation
Format: Text
Rights:
Access: Public
License: Copyright
Holder: Brewer, Travis K.
Statement: Copyright is held by the author, unless otherwise noted. All rights reserved.