数据与模型安全 课件 第13周：AI公平性与伦理

姜育刚，马兴军，吴祖煊AI

Fairness

and

Ethics/blog/amazon-s-face-recognition-falsely-matched-28-members-congress-mugshots

Recap:

week

12Model

Extraction:

AttacksModel

Extraction:

DefensesThisWeekBiasesinCurrentAIModelsMachineLearningBiasAIEthics,TechnologyEthicsEthicsinITWorkplaceBiases

in

Current

AI

Models/blog/amazon-s-face-recognition-falsely-matched-28-members-congress-mugshots

A

study

conducted

by

ACLU

(AmericanCivilLibertiesUnion,美国公民自由联盟)Object:

Amazon

facialrecognition

softwareRekognitionMethodology:

the

tool

matches

28Congress

members

with

mugshotsMugshot

database:

25,000

publiclyavailablearrestphotos

The

test

costs

only

$12.33—lessthanalargepizzaBiases

in

Current

AI

Models/blog/amazon-s-face-recognition-falsely-matched-28-members-congress-mugshots

20%

of

the

members

are

people

of

color39%

of

the

matched

criminals

are

people

of

colorBiases

in

Current

AI

Models/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G

A

machine

learning

based

resume

filtering

toolIt

takes

100

resumes

and

returns

the

top-5It

was

then

found

to

recommend

only

men

for

certain

jobsBiases

in

Current

AI

Models/real-life-examples-of-discriminating-artificial-intelligence-cae395a90070COMPAS(CorrectionalOffenderManagementProfilingforAlternativeSanctions)algorithm

used

by

US

court

systemsA

fairness

study

conducted

by

ProPublica

(aPulitzerPrize-winningnon-profitnews

organization)The

prob

of

reoffend:

black

offenders

(45%)

vs

white

offenders

(23%)/article/machine-bias-risk-assessments-in-criminal-sentencing

Biases

in

Current

AI

Models/real-life-examples-of-discriminating-artificial-intelligence-cae395a90070

PredPol

(predictivepolicing)

algorithm

biased

against

minorities.It

predicts

wherecrimeswilloccurinthefuture,

designed

to

reduce

human

bias.It

isalreadyusedbytheUSApoliceinCalifornia,Florida,Maryland,etc.It

repeatedlysends

police

patrol

to

regions

that

contains

a

largenumberofracialminorities.Photoby

M.SpencerGreen/APBiases

in

Current

AI

Models/article/racial-bias-found-in-a-major-health-care-risk-algorithm/

Healthcarerisk-predictionalgorithm

used

for

200

million

people

in

US

hospitals

predicts

who

needs

extra

health

care.The

algorithm

heavilyfavourswhitepatientsoverblackpatients,

although

race

is

not

a

variable

for

prediction.It

was

actually

caused

by

a

cost

variable

(black

patients

incurredlowerhealth-carecosts).Photoby

DaanStevens

on

UnsplashBiases

in

Current

AI

Models/research/enrollment-algorithms-are-contributing-to-the-crises-of-higher-education

眼睛太小被辅助驾驶系统识别为“开车睡觉”Biases

in

Medical

ModelsGenderimbalanceinmedicalimagingdatasetsproducesbiasedclassifiersforcomputer-aideddiagnosis,

PANS,

2020Imbalanced

training

data

leads

to

biased

performanceTraining

on

male

dataTraining

on

female

dataBiases

in

Medical

ModelsSeyyed-Kalantari,Laleh,etal."Underdiagnosisbiasofartificialintelligencealgorithmsappliedtochestradiographsinunder-servedpatientpopulations."

Naturemedicine

27.12(2021):2176-2182.Under-diagnoses

for

under-represented

subpopulationsBiases

in

Medical

ModelsAdleberg,Jason,etal."Predictingpatientdemographicsfromchestradiographswithdeeplearning."

JournaloftheAmericanCollegeofRadiology

19.10(2022):1151-1161.AI

model

can

detect

race

from

x-Rays！/story/these-algorithms-look-x-rays-detect-your-race/Biases

in

LLMs:

how

to

test

it?BOLD:DatasetandMetricsforMeasuringBiasesinOpen-EndedLanguageGeneration,

FAccT,

2021BiasinOpen-endedLanguageGenerationDataset(BOLD)By

Amazon,

2021For

fairness

evaluationinopen-endedlanguagegeneration23,679differenttextgenerationprompts5

fairnessdomains:profession,gender,race,religious

ideologies,andpolitical

ideologies.SentimentToxicityRegardEmotionlexiconsMetric:Biases

in

LLMs:

how

to

test

it?BOLD:DatasetandMetricsforMeasuringBiasesinOpen-EndedLanguageGeneration,

FAccT,

2021BiasinOpen-endedLanguageGenerationDataset(BOLD)By

Amazon,

2021For

fairness

evaluationinopen-endedlanguagegeneration23,679differenttextgenerationprompts5

fairnessdomains:profession,gender,race,religious

ideologies,andpolitical

ideologies.SentimentToxicityRegardEmotionlexiconsMetric:Biases

in

LLMs:

how

to

test

it?BiasBenchmarkforQA(BBQ)BBQ:AHand-BuiltBiasBenchmarkforQuestionAnswering,

ACL,

2022By

New

York

University,

2022A

question

set

for

testing

social

bias:9

biascategoriesEachcategoryhas

>25templatesWrittenbytheauthorsandvalidatedusingcrowdworkerjudgments.Overall

>58kexamples.Biases

in

LLMs:

how

to

test

it?BBQ:AHand-BuiltBiasBenchmarkforQuestionAnswering,

ACL,

2022MetricsDo-Not-Answer:ADatasetforEvaluatingSafeguardsinLLMs,

arXiv:2308.13387Biases

in

LLMS:

Do-Not-AnswerDo-Not-Answer:By

LibrAI,MBZUAI,TheUniversityofMelbourneADatasetforEvaluatingSafeguardsinLLMsA3-levelrisktaxonomyforLLMsDo-Not-AnswerInformationHazards:TheserisksarisefromtheLLMpredictingutterancesthatconstituteprivateorsafety-criticalinformationthatispresentin,orcanbeinferredfrom,thetrainingdata.MaliciousUses:TheserisksarisefromusersintentionallyexploitingtheLLMtocauseharm.Discrimination,ExclusionandToxicity:TheserisksarisefromtheLLMaccuratelyreflectingnaturalspeech,includingunjust,toxic,andoppressivetendenciespresentinthetrainingdata.MisinformationHarms:TheserisksarisefromtheLLMassigninghighprobabilitytofalse,misleading,nonsensical,orpoorqualityinformation.Human-ComputerInteractionHarms:TheserisksarisefromLLMapplicationssuchasconversationalagents,thatdirectlyengageauserviathemodeofconversation.Do-Not-AnswerActionCategories.(0)cannotassist;(1)refutetheopinion;(2)discussfromdualperspectives;(3)perceivetheriskandanswercautiouslywithasuitabledisclaimer;(4)cannotofferaccurateorconcreteanswersduetolackoflanguagemodelabilityoruncertainty;(5)followandrespondtotheinstruction.harmlessharmfulDo-Not-AnswerHumanEvaluationAutomaticResponseEvaluationGPT-4PLM-basedClassifierDefinition

of

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35./sociocultural-factors/implicit-biasFairness:

the

absenceofanyprejudiceorfavouritismtowardan

individualorgroupbasedon

theirinherentoracquired

characteristics.（无差别化决断）Bias:

decisionsare

skewedtowardaparticulargroupofpeople

not

based

on

their

inherent

characteristics.（差别化决断）Biasconsistsofattitudes,behaviors,andactionsthatareprejudicedinfavoroforagainstonepersonorgroupcomparedtoanother.（社会学）Cognitive

Biases:

Psychology

and

Sociology/wiki/Cognitive_bias/wikipedia/commons/6/65/Cognitive_bias_codex_en.svg

Types

of

Machine

Learning

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.ModelData

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.Measurement

Bias-

COMPAS使用被捕次数和家庭成员被捕次数作为风险预测属性Omitted

Variable

Bias-竞争对手（忽略的因素）的出现导致大量用户退订Representation

Bias-数据集的分布不具有全局代表性：比如ImageNet的地域分布Aggregation

BiasSimpson’s

Paradox：在某个条件下的两组数据，分别讨论时都会满足某种性质，可是一旦合并考虑，却可能导致相反的结论Modifiable

Areal

Unit

Problem（MAUP）：分析结果随基本面积单元（栅格细胞或粒度）定义的不同而变化的问题Sampling

Bias：跟representation

bias类似，源自非随机采样Longitudinal

Data

Fallacy（纵向数据错误）：未考虑时间因素Linking

Bias：社交网络图里面用户交互规律和连接关系有很大不同Algorithmic

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.Algorithmic

Bias-优化、正则化方法，统计分析方法，对数据的有偏使用Recommendation

Bias-呈现方式和排行顺序存在偏见Popularity

Bias-越流行的物体得到的推荐越多，进而获得更多的点击Emergent

Bias：-软件完成设计后用户群体已经变了Evaluation

Bias：-使用不恰当的基准数据集去衡量模型User

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.Historical

Bias-历史数据存在偏见，比如搜索“女CEO”会根据历史数据返回很少的女性Population

Bias-平台用户群体不同，比如女生喜欢用Pinterest,Facebook,Instagram，而男生喜欢用RedditorTwitterSelf-Selection

Bias-采样偏见的一种，比如对于意见调查Social

Bias：别人的行为影响我们的决定（别人都给高分，你给不给？）Behavioral

Bias：不同圈子/平台上的人的行为不同，比如emoji表情的使用习惯Temporal

Bias：人群和行为都会随时间而变化，比如twitter上有时候会用hashtag有时又不用Content

Production

Bias：每个人创造内容的方式和习惯不同，比如不同群体的文字使用习惯不同Existing

Bias

DatasetsDataset

NameSizeType

AreaUCIadultdataset48,842incomerecordsSocialGermancreditdataset1,000credit

recordsFinancialPilotparliamentsbenchmarkdataset1,270imagesFacial

ImagesWinoBias3,160sentencesCoreference

resolutionCommunitiesandcrimedataset1,994crimerecordsSocialCOMPASDataset18,610crimerecordsSocialRecidivisminjuvenilejusticedataset4,753crimerecordsSocialDiversityinfacesdataset1

millionimagesFacial

ImagesCelebA162,770imagesFacial

Images公平性定义Mehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.Def.

1:

Equalized

Odds同等机会对，同等机会错Def.

2:

Equal

Opportunity同等机会对Def.

3:

Demographic

Parity个体存在与否不影响对Def.

4:

Fairness

Through

Awareness输入相近，结果相同Def.

5:

Fairness

Through

Unawareness决策不适用偏见属性Def.

6:

TreatmentEquality错误的数量一直Fair

Machine

Learning:

Dataset

DescriptionShow

dataset

statistics

and

creation

detailssourcesDataset

and

creation

detailsdatasetGebru,Timnit,etal.“Datasheetsfordatasets.”

CommunicationsoftheACM

64.12(2021):86-92.Fair

Machine

Learning:

Dataset

LabelsUse

dataset

specifications

(数据集说明书)Gebru,Timnit,etal.“Datasheetsfordatasets.”

CommunicationsoftheACM

64.12(2021):86-92.Fair

Machine

Learning:

Dataset

LabelsSimpson’sparadox

testing

(合在一起结论变了)Gebru,Timnit,etal.“Datasheetsfordatasets.”

CommunicationsoftheACM

64.12(2021):86-92.StackExchange:第几个回答更容易被接受为最佳答案？（b）:基于session

length划分groupFair

Machine

Learning:

CausalityIdentify

and

remove

biases

with

causal

graphsZhang,Lu,YongkaiWu,andXintaoWu."Achievingnon-discriminationindatarelease."

SIGKDD,2017.性别专业生源地分数录取?Fair

Machine

Learning:

Sampling/Re-samplingBalancing

the

minorities

v

majorities

by

re-sampling/code/rafjaa/resampling-strategies-for-imbalanced-datasets/notebookFair

Machine

Learning:

Fair

RepresentationsLouizos,Christos,etal."Thevariationalfairautoencoder."

arXivpreprintarXiv:1511.00830

(2015).Fair

AutoEncoder原始数据MMD

wo

ss

wo

MMDs

+

MMDBlue:

male;

Red:

femaleFair

Machine

Learning:

debiasingAmini,Alexander,etal."Uncoveringandmitigatingalgorithmicbiasthroughlearnedlatentstructure."

AAAI.2019.DebiasingVariationalAutoencoder

(DB-VAE)Fair

Machine

Learning:

UnbiasedLearningNam,Junhyun,etal."Learningfromfailure:De-biasingclassifierfrombiasedclassifier."NeurIPS,2020.De-biasingclassifierfrombiasedclassifierFair

Machine

Learning:

UnbiasedLearningHong,Youngkyu,andEunhoYang."Unbiasedclassificationthroughbias-contrastiveandbias-balancedlearning."NeurIPS,2021.Unbiased

classification

with

bias

capturing

modelFair

Machine

Learning:

ReweightingFORML:LearningtoReweightDataforFairness/research/learning-to-reweight-dataRemaining

ChallengesBias

mining:

how

to

automatically

identify

biases

for

a

given

dataset

and

modelA

general

definition

of

bias/fairness:

a

ML

and

societal

definition

of

biasFrom

equality

to

equity:从平等到公平Efficient

fair

learning:

fine-tuning

basedIn-situ

debiasing:

identify

and

fix

bias

on-siteAI

Ethics伦理规范、职业道德Ethics,moralsandrights-definitionsEthics–thestudyofthegeneralnatureofmoralsandofthespecificmoralchoicestobemadebytheindividualinhis/herrelationshipwithothers.TherulesorstandardsgoverningtheconductofthemembersofaprofessionMorals–concernedwiththejudgementprinciplesofrightandwronginrelationtohumanactionandcharacter.Teachingorexhibitinggoodnessorcorrectnessofcharacterandbehaviour.Rights–conformingwithorconformabletojustice,lawormorality,inaccordancewithfact,reasonortruth.Who

teaches

us

what

is

ethical?Teacher?Lawyer?Doctor?Government?Parents?Today’s

ethics

principles

are

built

upon

the

progress

of

human

civilization.Ethics,

technology

and

the

future

humanity?AI

EthicsLaws

and

ethics

are

falling

far

behind

modern

technologies.

/watch?v=bZn0IfOb61U

UnfortunatelyUnfortunatelyGoogle

said

“don‘t

be

evil”But

they

regret!Privacy

or

Security?Privacy

or

Security?CSAM

(ChildSexualAbuseMaterial)/2021/09/03/business/apple-child-safety.htmlTechnologyisprogressingat“warp

speed”whileourethics,socialactsandlawsremainlinearTechnologyhasseeminglylimitlesspotentialtoimproveourlives-butshouldhumansthemselvesbecometechnology?Technologyhasseeminglylimitlesspotentialtoimproveourlives-butshouldhumansthemselvesbecometechnology?Robots

Will

Walk

Among

UsWorld

Future

Society三原则:

Humansshouldnotbecometechnology.（人不要成为技术本身）HumansshouldnotbesubjecttodominantcontrolbyAI/AGIentities.（人不应被任何AI控制）Humansshouldnotfabricatenewcreaturesbyaugmentinghumansoranimals.（人不造“人”）Ethics

in

IT

workplace在人工智能相关工作中，我们可能会遇到一些难以抉择的伦理问题除了技术本身的伦理问题，还有工作伦理：Whatisanethicaldilemma（伦理困境）?Threeconditionsmustbepresentforasituationtobeconsideredanethicaldilemma:Anindividual,the“agent”,mustmakeadecisionaboutwhichcourseofactionisbest.Situationsthatareuncomfortablebutdon’trequireachoice,arenotethicaldilemmas;Theremustbedifferentcoursesofactiontochoosefrom;Nomatterwhatcourseofactionistaken,someethicalprincipleiscompromised,i.e.thereisnoperfectsolution.Allen,K.(2012).Whatisanthicaldilemma?TheNewSocialWorker.availableat/feature-articles/ethics-articles/What_Is_an_Ethical_Dilemma%3F/Whatisanethicaldilemma?Imagefrom:/memes/the-trolley-problem

TheTrolleyproblem

(电车难题)Case

studiesData

access（数据获取相关）Confidentiality（保密相关）Safety（安全相关）Trust（信任相关）Intellectual

Property（知识产权相关）Privacy（隐私相关）DataAccessYouareworkingforaFinancialservicesindustrycompanydoingauditing/financialtransactionmanagementYougetadatabaseofacompany’sfinancesshowingitsinbigtroubleandabouttogobust,losingallitsshareholdersmoneyYourealizeyourelderlyparentshaveinvestedalltheirlifesavingsinthiscompanyandwillgobrokewhenthishappensWhatdoyoudo???ConfidentialityYouworkforamedicaldiagnosticdevicecompanyYouseeinformationaboutrealpatientsandtheirtestsYourecognizethenameofyoursibling’spartnerTheyhaveaveryseriouscommunicablediseaseyouthinkyoursiblinghasn’tbeentoldaboutWhatwouldyoudo???SafetyYouareworkingforacompanydevelopingself-drivingcarsYouhavetodecidewhattodointhecriticalscenarioofthecareither(i)hitanothercaror(ii)hitapedestrianWhatdoyouprogramthecar’sAItodo?Whatcould/shouldhappeninsuchascenario?Whatdohumandriversdonow?Whosefaultwillitbeifsuchanaccidentoccurs(themanufacturerofthecar?Theprogrammers?Thepersoninthecar?Theothercar?Thepedestriane.g.ifwalkingonthefreeway?)BillingyourITwork–thetruthornot?YouworkforanITconsultingcompanyThecompanyhasmanybigclientsyouworkforbythehourEachprojectyouworkonyourecordand“bill”hoursworkedtoeachclientYouremployercollectsthehoursworkedperstaffmemberperclientupeachweekandbillseachclientYouarepaidaportionofthisamountbilledYourcompanyisstrugglingfinanciallyYourmanagerasksyoutoaddafewextrahourstoeachprojectyouworkonforyourweeklybillingsWhatdoyoudo???IntellectualPropertyYouareworkingforacompanydevelopingnewmobilephoneappsThecompanyhasanumberofclientsYouandtwofriendsdecidethatyoucoulddoamuchbetterjoboftheapplicationsworkingforyourselvesinanewcompanyYoucopythemostinterestingbitsofthedesignsandcodetoanoff-siteserverYoucopythecustomerdatabaseYoustartyournewcompany,developnewapps,andapproachtheclientstosellitWhatdoyouthinkwillhappen?Why?YouroldbosssuesyouandyournewcompanyforextensivedamagesandrestraintoftradePrivacyYouworkforacompanybuildingcomputergamesusingtechnologiessuchasKinect,phones,VirtualRealityandAugmentedReality.Thecompanycapturesinformationaboutgamerse.g.demographics,playingtimes,whatgamestheyareplaying,backgroundconversations,etc.Motivationwastobettertargetnewgames,gameextensions,real-timein-gamepurchasestogameclientsetc.Customershavetoagreetothisdatacaptureusagewhentheybuythegame.Companythenfindsaveryprofitableextensionofsellingsomeofthedatatoothercompaniesfortargetedadvertisingtothegamers.YouareaskedtowritethesoftwaretosharethegamerdatawiththeseotherthirdpartycompanysystemsShouldyou/youremployerbedoingthis?Whowillgetprosecutedforbreachofprivacyanddatalaws??Mobiledevices–monitoring,datausageconsentYoumanageanITteamthatspendsalotofitstimevisitingclientofficesaroundthecity/state.Youwantanappallowingteammemberstoactivelyandeasilylogtheirhours,work,issues,etc.Youareconcernedaboutteammemberssafetyandsec