In an effort to understand students and classroom environments, teaching practitioners and educational researchers often measure intangible human constructs such as conceptual understanding, self-efficacy, and belonging. However, unlike many physical phenomena, these constructs cannot be measured directly. As a result, researchers and practitioners must use indirect forms of measurement such as questionnaires, assessments, surveys, etc.—referred to as instruments—to probe for these constructs. Due to the nature of language and the diversity of human experience, it can be difficult to create an instrument that consistently provides high quality data. To ensure that data collected from an instrument is credible, instruments must be rigorously evaluated each time they are used. Instrument creators and users must establish that the instrument not only measures the correct construct (validity) but also that the instrument does so consistently (reliability). When practitioners and researchers look for instruments it is important that this validity and reliability information is easily assessible so that high quality instruments can be identified and utilized whenever possible. As part of our efforts to assemble the Chemistry Instrument Review and Assessment Library (CHIRAL), we have documented the various ways in which instruments appear in the literature as well as the extent to which the chemistry education community provides evidence for the quality of data collected using these instruments. We used four seed documents containing lists of instruments from the chemistry education literature to generate an initial set of 180 publications containing instruments. We coded for various publication and instrument features (e.g., topic of measurement, intended population, and related instruments). We also cataloged the manner in which instruments appeared in the literature, (i.e., use of a preexisting instrument or the creation and evaluation of an entirely new instrument). Most significantly, we coded for presentation of evidence for validity and reliability using common methods (e.g., response process interviews, factor analysis, internal consistency, etc.). In this presentation, we report the results of our cataloging and describe issues that emerged during our investigation of the literature. We close with practical recommendations for the reporting of instrument usage or development in chemistry education research.

Patterns and trends in instrument development and usage in the chemistry education literature

ACS Fall 2020 Virtual Meeting

The official archive and repository for all ACS Meetings & Expositions abstracts and select posters or presentations.