As technology and businesses evolve and the outcomes of an engineering design impact society’s lives, a design project’s constraints move from strictly technical and economical to social, environmental, and ethical conditions. Among these, one important consideration is to incorporate the fairness dimension in the decision-making process. For example, businesses apply machine learning technologies and big data analytics to study the customers, their needs, and preferences based on vast market data and social media data. This helps enterprises better understand their current market systems and make improved design decisions to cater to customers’ needs. However, these data can be embedded with biases towards protected or sensitive classes, such as race, gender, and age. This often leads to unintended consequences and potentially causes discrimination, inappropriately hurting personal feelings. There is a need to incorporate the concept of fairness in decision-based design (DBD) to support inclusive design decisions.This study aims to introduce the definitions and statistical measures of fairness such as group fairness, demographic parity, equalized odds, calibration testing, and fairness through unawareness to the engineering design research community. From these definitions, we approach two methods to quantify fairness in a dataset. The first method is the Disparate Impact (DI) analysis, commonly used in legal domains to evaluate whether a decision-making system is free of disparate impact (indirect discrimination). The higher the DI index value, the lower the chance of bias. The second method uses fairness testing of calibration scores based on sensitive attributes, predicted probability and outcome from machine learning models, and the actual outcome. In our case study, we use a benchmark dataset, the Adult Income dataset, in a binary classification setting to quantify the presence of unfairness using two supervised learning models: Logistic Regression (LR) and CatBoost (CB) classifiers. The income variable in this data is&nbsp;the target variable (Y), i.e., a binary variable indicating an individual’s annual income where Y=1&nbsp;if the income earned is higher than $50,000. We train the dataset using the two classifiers to predict the income for new individuals. If an individual (new customer or follower) receives a predicted outcome of Y^=1, he/she is chosen as a target audience.It is observed that both the classifiers revealed that Gender and Ethnicity are the two critical sensitive attributes that can affect the predictions made in decision models for test data. First, our results indicate that the CB model outperforms the LR model with a prediction accuracy of 83%. This means that for 83% of individuals from the test data, the model correctly predicts whether this individual earned more than 50K or not. Second, to achieve a fair prediction, eligible individuals (with income higher than 50K) from unprivileged groups with the same features as the privileged group members need to receive the binary prediction of 1, irrespective of their sensitive attribute membership. This will classify algorithmic decision-making to be fair and objective. With the disparate impact (DI) analysis, CB turns out to have a higher index value than LR, indicating better performance in fairness in predictions.Results from the study conclude that quantification of fairness in data is possible using fair ML. However, a thorough understanding is required to bridge the gap between fairness application and DBD to best fit design ecosystems’ characteristics. Many studies have established the connections between consumer behavior, market competitor, and product design [8]. Yet, there exists a knowledge gap to quantitatively understand the effects of enterprise marketing strategies (fair or unfair) within such connections established, as shown in Figure 2.&nbsp;We are working on various design solutions to create links between the design of marketing strategies and choice modeling with fairness consideration in M1 and M2. This proposed framework will help develop a data-driven analytical approach to quantitatively predict the effects of enterprises’ marketing strategies on customers’ choice behaviors by considering the complex customer-product-enterprise relations.

Towards Fairness-Aware Design Decision-Making

Paper Proceedings: https://asmedigitalcollection.asme.org/IDETC-CIE/IDETC-CIE2021/volume/85369

ASME 2021 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE2021)

Supporting the need among researchers worldwide for additional online and multimedia content, ASME now offers the ASME Conference Video Collection. This platform will collect and provide long-term access to conference presentation videos, encompassing all technology areas within ASME's technical conference and events program. About 30,000-40,000 engineers across all mechanical engineering and related disciplines participate in ASME conferences each year. This includes authors of approximately 6,000 to 8,000 technical papers published by ASME, the majority of whom also submit video presentations for our virtual conferences. The video collection will be available as an add-on to existing product packages or as a separate subscription.