The ability of the ChemWiki, an Open Educational Resource, to supplant a traditional textbook in a General Chemistry Course was tested at UC Davis in the spring of 2014. The experiment consisted of one class using the traditional textbook and an experimental class using the ChemWiki as the primary resource for readings and homework. To determine if the ChemWiki was a viable alternative, a non-inferior testing model was used. The non-inferior testing model was adapted from studies comparing a competitor’s new prescription drug to currently available prescription pharmaceuticals. Traditional comparative tests place the burden of proof on a difference that is large enough to unlikely be due to chance, but does not test whether a small difference is due to the intervention or not. Non-inferiority testing is used to test whether a new product of intervention (e.g. ChemWiki) is a suitable substitute for an existing competitor/standard acceptable practice (e.g. textbook), and can attribute a small margin of change to the intervention. Non-inferiority testing relies on an equivalence margin, the maximum difference that one would be willing to accept as a difference between the traditional treatment and the new treatment. Non-inferiority is established when the 90% confidence interval, calculated using traditional statistical models, of the two groups falls within the equivalency margin established at the onset of the study. Using this methodology, the ChemWiki was shown to be non-inferior to the traditional textbook based on student course assessments. To our knowledge this is the first time non-inferior testing strategies have been applied in an educational experiment. This methodology can be applied to a wide range of chemical education studies that are not necessarily looking for large differences, but ones in which a small difference or no difference is a positive result.

Use of non-inferiority testing in chemical education research

251st National Meeting (2016)

The official archive and repository for all ACS Meetings & Expositions abstracts and select posters or presentations.