AI伦理：公平与可解释的机器学习-与IBM的投资者座谈会摘要

1、Ethical AI - Fair & Explainable Machine LearningDeepFin Investor Workshop Summary with IBMGlobal Quantitative & Derivatives Strategy 18 September 2020In this research note we summarise our September 2020 DeepFin Investor Tutorial on Fair and Explainable AI with IBM held over video conference from Lo

2、ndon. As Machine Learning and AI continue to proliferate, we explore how to remove unfair bias in AI and how to engender trust through explainable AI models.What is Machine Learning and AI?Machine Learning by its nature is a way of statistical discrimination. The discrimination becomes objectionable

3、 when it places privileged groups at a systematic advantage and certain unprivileged groups at a systematic disadvantage. Extensive evidence has shown that AI can embed human and societal bias (e.g. race, gender, caste, and religion) and deploy them at scale, consequently many algorithms are now bei

4、ng re-examined due to illegal bias.Trusted AI how can we trust AI?Fundamental questions arise around how can we trust AI? And, to what level of trust can, and should we, place on AI? Forming the basis of trusted AI systems, IBM introduces four pillars: 1) Explainability: knowing and understanding ho

5、w AI models arrive at specific decisions; 2) Fairness: removing / minimising bias in the model or data; 3) Robustness: ensuring the model is safe and secure; and 4) Lineage: as models are continually evolving, we should track and maintain the provenance of data, metadata, models and test results. Se

6、e cover chart. To remove unfair bias in Machine Learning, we can intervene before the models are built (pre-processing algorithms), to a model during training (in-processing algorithms), or to the predicted labels (post-processing algorithms).Removing unfair bias in Machine Learning and explaining A

7、I models During the practical sections of the workshop, we used Python packages from IBM in the IBM Cloud to remove unfair bias from an AI pipeline, and to help explain machine learning model predictions and data. Removing Unfair Bias in Machine Learning: using German Credit Risk data, we measured b

8、ias in the data and models, and applied a fairness algorithm to mitigate bias. By having access to the training data, we used a pre-processing algorithm to remove bias (age 25 years): there was subsequentlynodifference in the rate of favourable outcomes received by the unprivileged group to the priv

9、ileged group. Explain Machine Learning Models: how can we explain model predictions? We worked through explaining the Iris dataset predictions using SHAP, explaining German Credit Risk predictions using LIME, explain Proactive Retention decisions using TED, and analysed and explained CDC Income Data

10、 using ProtoDash we explain each of these practical approaches in more detail within the research note.Figure 1: Four Pillars of Trusted AIPillars of trust, woven into the lifecycle of an AI applicationBig Data & AI Stategies Ayub Hanif, PhD AC(44-20) 7742-5620 HYPERLINK mailto:ayub.hanif ay HYPERLI

11、NK mailto:ub.hanif ub.hanifBloomberg JPMA HANIF J.P. Morgan Securities plcKhuram Chaudhry AC(44-20) 7134-6297 HYPERLINK mailto:khuram.chaudhry khuram.chaudhryBloomberg JPMA CHAUDHRY J.P. Morgan Securities plcGlobal Head of Quantitative and Derivatives StrategyMarko Kolanovic, PhD AC(1-212) 622-3677

12、HYPERLINK mailto:marko.kolanovic marko.kolanovicJ.P. Morgan Securities LLCSource: J.P. Morgan Quantitative and Derivatives Strategy, IBMSee page 20 for analyst certification and important disclosures, including non-US analyst disclosures.J.P. Morgan does and seeks to do business with companies cover

13、ed in its research reports. As a result, investors should be aware that the firm may have a conflict of interest that could affect the objectivity of this report. Investors should consider this report as only a single factor in making their investment decision. HYPERLINK / Table of Contents HYPERLIN

14、K l _bookmark0 Ethical AI Fair & Explainable Machine Learning 3 HYPERLINK l _bookmark7 Removing Unfair Bias in Machine Learning 6 HYPERLINK l _bookmark9 Explore bias in the data 7 HYPERLINK l _bookmark10 Exploring bias metrics 7 HYPERLINK l _bookmark11 Select and transform features to build a model

15、7 HYPERLINK l _bookmark12 Build models 8 HYPERLINK l _bookmark14 Remove bias by reweighing data 8 HYPERLINK l _bookmark17 Explain Machine Learning Models 10 HYPERLINK l _bookmark19 Understanding model predictions with SHAP 11 HYPERLINK l _bookmark25 Understanding model predictions with LIME 13 HYPER

16、LINK l _bookmark29 Understanding model predictions with TED 14 HYPERLINK l _bookmark32 Understanding data with ProtoDash 16 HYPERLINK l _bookmark36 Takeaways 19Ethical AI Fair & Explainable Machine LearningWith increased proliferation / ubiquity of AI, comes increased scrutinyHow can we trust AI?How

17、 can we build trust in AI?As the hype surrounding advancements in Machine Learning and Artificial Intelligence (AI) starts to deliver, we have seen AI being increasingly used in many decisions-making applications e.g. credit, employment, admissions, sentencing and healthcare. Although Machine Learni

18、ng, by its very nature, is a way of statistical discrimination, the discrimination becomes objectionable when it places certain privileged groups at a systematic advantage and certain unprivileged groups at a systematic disadvantage.Extensive evidence has shown that AI can embed human and societal b

19、ias and deploy them at scale, consequently many algorithms are now being re-examined due to illegal bias: e.g. biases in training data, due to either prejudice in labels or under- over-sampling, yields models with unwanted bias HYPERLINK l _bookmark2 1.The fundamental questions thus are: how can we

20、trust an AI system? How can an AI system explain itself? How does unfair human and societal bias leak into an AI machine? IBM describe four pillars of trust, see HYPERLINK l _bookmark1 Figure 2, forming the basis for trusted AI systems HYPERLINK l _bookmark3 2.Explainability: knowing and understandi

21、ng how AI models arrive at specific decisions.Fairness: removing / minimising bias in the model or data.Robustness: ensuring the model is safe and secure.Lineage: as models are continually evolving, we should track and maintain the provenance of data, metadata, models (with hyperparameters) and test

22、 results.Figure 2: Four Pillars of Trusted AIPillars of trust, woven into the lifecycle of an AI applicationSource: J.P. Morgan Quantitative and Derivatives Strategy, IBMSo, what is fairness? There are numerous definitions of fairness, many of which conflict. Consider if we were to afford positive w

23、eight to any notion of fairness through social policies may sometimes lead to reducing the well-being of every person in society HYPERLINK l _bookmark4 3. Simply put, the way you define fairness impacts bias.1 Barocas, Solon, and Andrew D. Selbst. Big datas disparate impact. Calif. L. Rev. 104 (2016

24、): 671.2 Arnold, Matthew, Rachel KE Bellamy, Michael Hind, Stephanie Houde, Sameep Mehta, A.Mojsilovi, Ravi Nair et al. FactSheets: Increasing trust in AI services through suppliers declarations of conformity. IBM Journal of Research and Development 63, no. 4/5 (2019): 6-1. 3 Kaplow, Louis, and Stev

25、en Shavell. The conflict between notions of fairness and the Pareto principle. American Law and Economics Review 1, no. 1 (1999): 63-77.The following are some fairness terms used in ethical AI:Protected Attribute an attribute that partitions a population into groups whose outcomes should have parity

26、 (e.g. race, gender, caste, and religion)Privileged Protected Attribute a protected attribute value indicating a group that has historically been at systemic advantageGroup Fairness groups defined by protected attributes receiving similar treatments or outcomesIndividual Fairness similar individuals

27、 receiving similar treatments or outcomesFairness Metric a measure of unwanted bias in training data or modelsFavourable Label a label whose value corresponds to an outcome that provides an advantage to the recipientIn HYPERLINK l _bookmark5 Figure 3 we show example group fairness metrics. We have t

28、wo groups, unprivileged and privileged and are measuring their favourable outcome rates. In the sample scenario, 6 of the unprivileged individuals have a favourable outcome, whilst there are 7 privileged individuals with a favourable outcome. We can measure fairness from a number of different perspe

29、ctives.Statistical Parity Difference measures the difference in positive rates.Disparate Impact expresses the unprivileged positive outcomes in relation to the privileged positive outcomes.Equal Opportunity Difference measures the difference in true positives and false negatives in the privileged /

30、unprivileged groups.Mitigate often, mitigate earlyFigure 3: How to measure fairness? Some group fairness metricsSource: J.P. Morgan Quantitative and Derivatives Strategy, IBMWhere can we intervene in the AI pipeline to mitigate bias? If we can modify the training data, then we use a pre-processing a

31、lgorithm. If we can modify the learning algorithm, then an in-processing algorithm can be used. And if, we can only treat the learned model as a black-box and cannot modify either the training data or learning algorithm, then a post-processing algorithm can be used.Figure 4: Where can you intervene

32、in the AI pipeline?Source: J.P. Morgan Quantitative and Derivatives Strategy, IBMNeed to tradeoff between the bias / accuracy of the model with respect to your legal, ethical and trust guidelinesIt is important, however, to note there are some important tradeoffs to consider when mitigating bias vs.

33、 accuracy in AI models:Is your model doing good or bad things to people?If your model is handing out loans, it may be better to have more False Negatives than False PositivesIf your model is sending people to jail, it may be better to have more False Positives than False NegativesDetermine your thre

34、shold for accuracy vs. fairness based upon your legal, ethical and trust guidelinesUltimately, preventing bias is difficult though through careful monitoring and management can be minimised / removed:Work with your stakeholders early on to define fairness, protected attributes and thresholdsApply th

35、e earliest mitigation in the pipeline that you have permission to applyCheck for bias as often as possible using any metrics that are applicableSystematised help (e.g. using the AI Fairness package below) should only be used with defined data sets and well-defined use casesRemoving Unfair Bias in Ma

36、chine LearningFor more details on packages used in the workshop please see HYPERLINK l _bookmark37 Python Packages on page HYPERLINK l _bookmark37 19.How do you remove bias and discrimination in the machine learning pipeline? In the first practical of the workshop we learn about de-biasing technique

37、s that can be implemented using the open source toolkit AI Fairness 360.The chart to the right provides a brief introduction to a confusion matrix for a binary classifier, alongside definitions of some commonly used metricsThe toolkit enables measuring, understanding and removing AI bias. It contain

38、s widely used bias metrics, bias mitigation algorithms, and metric explainers from leading AI fairness researchers across industry and academia. In the workshop we:Apply a practical use case of bias measurement and mitigationMeasure bias in data and modelsApply fairness algorithms to reduce biasFor

39、the practical we use the German Credit Risk dataset. The dataset classifies people described by a set of attributes as good or bad credit risks. By default, the code converts age to a binary value where privileged is age 25 and unprivileged isage = 25 is considered privileged.The dataset also contai

40、ns a protected attribute for sex that is not consider in this evaluation. If we were to set the protected attribute to be sex, then sex = male is considered privileged.Next we split the original dataset into training and testing datasets.Set two variables for the privileged (1) and unprivileged (0)

41、values for the age attribute. These are key inputs for detecting and mitigating bias.Figure 7: Exploring bias in the dataSource: J.P. Morgan Quantitative and Derivatives Strategy, IBMExploring bias metricsWe will measure Mean Difference (an alias of Statistical Parity Difference from HYPERLINK l _bo

42、okmark5 Figure 3) as the difference of the rate of favourable outcomes received by the unprivileged group to the privileged group. A negative value indicates less favourable outcomes for the unprivileged groups. The ideal value of this metric is 0 where fairness for this metric is between -0.1 and 0

43、.1. Secondly we will measure Disparate Impact which is the ratio of rate of favourable outcome for the unprivileged group to that of the privileged group.Running these metrics on the training set we findmean_difference = -0.169905disparate_impact = 0.766430Select and transform features to build a mo

44、delWe know we are dealing with a classification problem (an applicant is either a good credit risk or a bad credit risk): the output prediction will be either 0 or 1 summarised as a binary classification.There are a number of options in terms of models we can use including decision trees, random for

45、ests, Bayesian networks, support vector machines, neural networks and logistic regression. Before picking a model we need to scale and normalise the features.Standardisation of datasets is a common requirement for many machine learning estimators in scikit-learn. StandardScaler implements the Transf

46、ormer API to compute the mean and standard deviation on a training set so as to be able to later reapply the same transformation on the test set.We build a logistic regression classifier on the original training dataThe classifier on the original data had Precision of 64%, Recall of 40% and Accuracy

47、 of 75%.Build modelsFirst we train on the original training data. For sake of simplicity and to aid in understanding we build a logistic regression HYPERLINK l _bookmark15 5 classifier and predictor, using a train / test split of 70 / 30. The models accuracy (lmod.score(x_test, y_test) from scikit-l

48、earn) on the training data is 0.753.As discussed earlier, to assess the actual classification against the predicted classification we use a confusion matrix - see HYPERLINK l _bookmark13 Figure 8. What we see here is 36 applicants were predicted to be Good Credit and were Good Credit (i.e. True Posi

49、tives) as 20 were predicted to be Good Credit but proved to be Bad Credit (False Positives). Simultaneously there were 54 False Negatives and 190 True Negatives. What we know is given the bias in the data, despite an applicants e.g. income being the same, just because of their age they may not get a

50、 loan.Figure 8: Confusion matrix on original dataSource: J.P. Morgan Quantitative and Derivatives Strategy, IBMRemove bias by reweighing dataAs we have access to the training data and are able to modify it we use a pre- processing algorithm HYPERLINK l _bookmark16 6. Reweighing is a pre-processing t

51、echnique that weights the examples in each (group, label) combination differently to ensure fairness before classification. This technique is particularly powerful in that it attempts to remove / mitigate discrimination without relabelling instances.5 Logistics regression estimates the parameters of

52、 a binary logistic model which has a dependent variable with two possible values i.e. pass/fail, win/lose represented by the values labelled 0 or 1.6 See HYPERLINK l _bookmark6 Figure 4: Where can you intervene in the AI pipeline? above.Revisiting the bias metrics from above: Original training data:

53、mean_difference = -0.169905disparate_impact = 0.766430Transformed training data:mean_difference = 0.000000disparate_impact = 1.000000We have removed age-bias from the data.The reweighing process has indeed removed bias (age bias) from the data: there is no difference of the rate of favourable outcom

54、es received by the unprivileged group to the privileged group mean_difference = 0 and ratio of the rate of favourable outcomes for unprivileged group to that of the privileged group Disparate Impact = 1.Given the encouraging transformation on the training data we build a new logistic regression clas

55、sifier on the reweighed data. Model accuracy has gone up from 0.753 to 0.766. Assessing the confusion matrices: we now have 38 applicants as True Positives i.e. the overall Good Credit labelling has not changed only that the classifier now clears a further two people. Similarly the model now correct

56、ly identifies a further 2 people as Bad Credit i.e. True Negatives.Comparing the original model vs. the reweighed model we find Precision has improved from 64% to 68%, Recall hasimproved from 40% to 42% and Accuracy has improved from 75% to 77%.Figure 9: Confusion matrix on original dataSource: J.P.

57、 Morgan Quantitative and Derivatives Strategy, IBMFigure 10: Confusion matrix on reweighed dataSource: J.P. Morgan Quantitative and Derivatives Strategy, IBMThough the changes are on the margin i.e. 2 additional people have been classed as Good Credit, it must be remembered this is on a small sample

58、 size. In a more realistic setting, with 10s of thousands of applicants we can see how these changes scale up.Finally, in the example we used a pre-processing algorithm as we were able to apply mitigation processes to the training data. Some other techniques which can be explored include:In-processi

59、ng algorithmsAdversarial De-biasingReject Option ClassificationPost-processing algorithmsOdds EqualizingExplain Machine Learning ModelsTrust comes from explainability, and different stakeholders need different explanations.The choice on explainability technique boils down to the consumer of the expl

60、anation.In many applications, trust in an AI system will come from its ability to explain itself. But when it comes to understanding and explaining the inner workings of an algorithm, one size does not fit all. Different stakeholders require explanations for different purposes and objectives, and ex