While there exist various knowledge discovery tools to support designers' learning during design space exploration, there is no established definition of what is expected to be learned and how it should be measured. Measuring learning is important, as it enables assessing and comparing different knowledge discovery methods. In this paper, we review the major categories of learning goals that are introduced in the field of education. Then, 7 different measures are developed to target specific learning goals relevant to design space exploration. Different learning goals are targeted by modifying the domain of the knowledge that is tested by each measure, and the specific task that the user is asked to perform. A human-subject experiment is conducted to measure how these metrics are related. Specifically, the consistency and correlations between different combinations of the measures are examined. Based on the observations made in this study, we discuss the implications and issues for future usage.


Measuring Human Learning in Design Space Exploration to Assess\x0bEffectiveness of Knowledge Discovery Tools

Division / Sponsors: 
Design Engineering Division and Computers & Information In Engineering Division


Paper Proceedings: https://asmedigitalcollection.asme.org/IDETC-CIE/IDETC-CIE2020/volume/83907

IDETC2020 - Virtual Conference

Supporting the need among researchers worldwide for additional online and multimedia content, ASME now offers the ASME Conference Video Collection. This platform will collect and provide long-term access to conference presentation videos, encompassing all technology areas within ASME's technical conference and events program. About 30,000-40,000 engineers across all mechanical engineering and related disciplines participate in ASME conferences each year. This includes authors of approximately 6,000 to 8,000 technical papers published by ASME, the majority of whom also submit video presentations for our virtual conferences. The video collection will be available as an add-on to existing product packages or as a separate subscription.