2 Matching Results

Search Results

A Structural and Psychometric Evaluation of a Situational Judgment Test: The Workplace Skills Survey

Description: Some basic but desirable employability skills are antecedents of job performance. The Workplace Skills Survey (WSS) is a 48-item situational judgment test (SJT) used to assess non-technical workplace skills for both entry-level and experienced workers. Unfortunately, the psychometric evidence for use of its scores is far from adequate. The purpose of current study was two-fold: (a) to examine the proposed structure of WSS scores using confirmatory factor analysis (CFA), and (b) to explore the WSS item functioning and performance using item response theory (IRT). A sample of 1,018 Jamaican unattached youth completed the WSS instrument as part of a longitudinal study on the efficacy of a youth development program in Jamaica. Three CFA models were tested for the construct validity of WSS scores. Parameter estimations of item difficulty, item discrimination, and examinee’s proficiency estimations were obtained with item response theory (IRT) and plotted in item characteristics curves (ICCs) and item information curves (IICs). Results showed that the WSS performed quite well as a whole and provided precise measurement especially for respondents at latent trait levels of -0.5 and +1.5. However, some modifications of some items were recommended. CFA analyses showed supportive evidence of the one-factor construct model, while the six-factor model and higher-order model were not achieved. Several directions for future research are suggested.
Date: August 2014
Creator: Wei, Min
Partner: UNT Libraries

Comparing Three Effect Sizes for Latent Class Analysis

Description: Traditional latent class analysis (LCA) considers entropy R2 as the only measure of effect size. However, entropy may not always be reliable, a low boundary is not agreed upon, and good separation is limited to values of greater than .80. As applications of LCA grow in popularity, it is imperative to use additional sources to quantify LCA classification accuracy. Greater classification accuracy helps to ensure that the profile of the latent classes reflect the profile of the true underlying subgroups. This Monte Carlo study compared the quantification of classification accuracy and confidence intervals of three effect sizes, entropy R2, I-index, and Cohen’s d. Study conditions included total sample size, number of dichotomous indicators, latent class membership probabilities (γ), conditional item-response probabilities (ρ), variance ratio, sample size ratio, and distribution types for a 2-class model. Overall, entropy R2 and I-index showed the best accuracy and standard error, along with the smallest confidence interval widths. Results showed that I-index only performed well for a few cases.
Date: December 2015
Creator: Granado, Elvalicia A.
Partner: UNT Libraries