Speaker: Dr. David Flora, York University
Department of Psychology

Title: Because it might not make a big DIF: Assessing differential test functioning

Abstract: There is an enormous literature on the detection of differential item functioning (DIF), but comparatively little research has been dedicated to the consequences of DIF for ensuing test scoring. These consequences of DIF for test scoring then have ramifications for subsequent applications using tests that display DIF across known groups. If a set of items in a test has DIF effects which consistently favor one group over another, then the overall impact of DIF may be substantial. But if DIF effects are weak, or a test has many items and only a few of them have DIF, then the impact of DIF on the overall test scores may be negligible. In other situations, there might be large DIF effects in one direction for some items, but these effects are canceled out by DIF in the opposite direction for other items. Therefore, detection of DIF for one or more items does not necessarily imply that the overall test itself is substantially biased. Thus, it can be useful for researchers to investigate the extent to which DIF manifests itself at the level of the overall test scores to produce differential test functioning (DTF). If the DTF appears negligible, then there may be no need to drop items with DIF (and thereby reduce the test’s reliability and content validity). This presentation introduces new DTF measures that properly account for sampling variability in item parameter estimates. These novel DTF measures can be effectively applied to determine the extent to which individual DIF effects accrue to cause biased scoring for a test as a whole.