解读欧盟人工智能法案（英文）-毕马威

Decoding

theEUAIAct

UnderstandingtheAIAct’s

impactandhowyoucan

respond

KPMG.MaketheDifference.

KPMGInternational|

/trustedai

05

Executivesummary

17

Nextsteps

11

UnravellingtheAI

Act’skeycomponents

03

Introduction

Contents

07

ExaminingtheAI

Act’simpactand

scope

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.

DecodingtheEUAIAct2

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

Introduction

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.

DecodingtheEUAIAct3

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

ArtificialIntelligence(AI)isofferingnewbenefitstosocietyandbusinesses,aimingtotransformtheworkplaceandmajorindustriesalongtheway.

Simplyput,theraceisontoembracetheremarkableandevolvingpowerof

AIandautomation.

AsKPMG’s

GlobalTechReport2023

reveals,most

globalexecutives(62percent)reportanincreasein

performanceorprofitabilityfromdigitaltransformationinitiativesrelatedtoAIandmachinelearningoverthe

past24months.And68percentsaythesetechnologieswillplaya‘vital’roleinhelpingthemachievetheir

businessobjectivesoverthenextthreeyears,while57percentbelieveAIandmachinelearningwillbe‘important’inmeetingshort-termobjectives.

ButastheworldwideAIproliferationinbusinessand

oureverydaylivesunfolds,thereisacriticalneed

forguardrailsandlegislationtodealwithsignificant

newrisksregardingtheappropriateandethicaluse,

developmentanddistributionofAI.Accordingto

Trust

inartificialintelligence

,aglobalsurveyconductedby

KPMGAustraliaandtheUniversityofQueensland,threeinfivepeoplearewaryabouttrustingAIsystems,and71percentexpectAItoberegulated.Morerecently,

CEOsfromglobaltechgiantscalledforgreaterAI

regulationatameetingonCapitolHilltoprotectpeoplefromtheworsteffectsofAI.

Inresponse,theEuropeanUnion(EU)hasreached

aground-breakingprovisionalagreementona

comprehensiveArtificialIntelligenceAct(AIAct)thattakesarisk-basedapproachtoprotectingfundamentalrights,democracy,theruleoflawandenvironmentalsustainability.1Thoughit'sexpectedtobecomelaw

in2024,withcomplianceexpectedby2025,thislegislation—thefirstofitskind—isanticipatedtoemergeasthede-factonewglobalstandardforAIregulation.

WiththeintroductionoftheAIAct,theEUaimsto

strikeabalancebetweenfosteringAIadoptionand

ensuringindividuals’righttoresponsible,ethicaland

trustworthyuseofAI.Inthispaper,weexplorewhattheAIActmaymeantoyourorganizationandexaminethestructureoftheAIAct,theobligationsitimposes,thetimelinesforcomplianceandtheactionplanthatorganizationsshouldconsider.

Organizationalleadership

shoulddriveinitiativesinline

withtheAIAct,company

brand,valuesandrisktolerancetopromoteresponsibleuse

ofAI.Thiscanhelppromote

ethicaldevelopment,regulatorycompliance,riskmitigationand

stakeholdertrust.

DavidRowlands

GlobalAILeader

KPMGInternational

AIshouldbedevelopedandusedwithafocusonsafetyandethics,turningtechnologicaladvancementintoa

positiveforceforsociety.TheEUAIActwillhelpfosterinnovationwhile

protectingend-users.

LaurentGobbi

GlobalTrustedAILeader

KPMGInternational

1EuropeanParliament.(December9,2023).ArtificialIntelligenceAct:dealoncomprehensiverulesfortrustworthyAI[Pressrelease].

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.DecodingtheEUAIAct4

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

Executive

summary

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.

DecodingtheEUAIAct5

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

TheAIActaimstoregulatetheethical

useofAI

AIholdsimmensepromisetoexpandthehorizonof

whatisachievableandtoimpacttheworldforour

benefit—butmanagingAI’srisksandpotentialknownandunknownnegativeconsequenceswillbecritical.TheAIActissettobefinalizedin2024andaimsto

ensurethatAIsystemsaresafe,respectfundamentalrights,fosterAIinvestment,improvegovernance,andencourageaharmonizedsingleEUmarketforAI.

MostAIsystemsneedtocomplywiththeAIActbythefirsthalfof2026

TheAIAct'sdefinitionofAIisanticipatedtobebroad

andincludevarioustechnologiesandsystems.Asa

result,organizationsarelikelytobesignificantlyimpactedbytheAIAct.Mostoftheobligationsareexpected

totakeeffectinearly2026.However,prohibitedAI

systemswillhavetobephasedoutsixmonthsafter

theAIActcomesintoforce.Therulesforgoverning

general-purposeAIareexpectedtoapplyinearly2025.2

Providersandusersofhigh-riskAIsystemsfacestringentobligations

TheAIActappliesarisk-basedapproach,dividingAIsystemsintodifferentrisklevels:unacceptable,high,limitedandminimalrisk.3

High-riskAIsystemsarepermittedbutsubjecttothemoststringentobligations.Theseobligationswillaffectnotonlyusersbutalsoso-called

‘providers’ofAIsystems.Theterm‘provider’in

theAIActcoversdevelopingbodiesofAIsystems,includingorganizationsthatdevelopAIsystemsforstrictlyinternaluse.Itisimportanttoknowthatanorganizationcanbebothauserandaprovider.

Providerswilllikelyneedtoensurecompliancewith

strictstandardsconcerningriskmanagement,data

quality,transparency,humanoversight,androbustness.

UsersareresponsibleforoperatingtheseAIsystemswithintheAIAct’slegalboundariesandaccording

totheprovider'sspecificinstructions.Thisincludesobligationsontheintendedpurposeandusecases,datahandling,humanoversightandmonitoring.

GuardrailsforgeneralAIsystems

Newprovisionshavebeenaddedtoaddresstherecentadvancementsingeneral-purposeAI(GPAI)models,

includinglargegenerativeAImodels.4Thesemodels

canbeusedforavarietyoftasksandcanbeintegratedintoalargenumberofAIsystems,includinghigh-risksystems,andareincreasinglybecomingthebasisformanyAIsystemsintheEU.Toaccountforthewide

rangeoftasksAIsystemscanaccomplishandtherapidexpansionoftheircapabilities,itwasagreedthatGPAIsystems,andthemodelstheyarebased

on,mayhavetoadheretotransparencyrequirements.

Additionally,high-impactGPAImodels,whichpossess

advancedcomplexity,capabilities,andperformance,

willfacemorestringentobligations.Thisapproach

willhelpmitigatesystemicrisksthatmayarisedueto

thesemodels'widespreaduse.5

TheAIActdoesnotaffectexisting

Unionlaw

ExistingUnionlaws,forexample,onpersonaldata,

productsafety,consumerprotection,socialpolicy,

andnationallaborlawandpractice,continuetoapply,

aswellasUnionsectorallegislativeactsrelatingto

productsafety.CompliancewiththeAIActwillnot

relieveorganizationsfromtheirpre-existinglegal

obligationsintheseareas.

UnderstandingtheAIAct’simpact

onyourorganizationwillbepivotal

tosuccess

Organizationsshouldtakethetimetocreateamapof

theAIsystemstheydevelopanduseandcategorize

theirrisklevelsasdefinedintheAIAct.Ifanyoftheir

AIsystemsfallintothelimited,highorunacceptable

riskcategory,theywillneedtoassesstheAIAct’s

impactontheirorganization.Itisimperativeto

understandthisimpact—andhowtorespond—as

soonaspossible.

2EuropeanCommission.(December12,2023).ArtificialIntelligence—QuestionsandAnswers[Pressrelease].

3EuropeanCouncil.(December9,202).ArtificialIntelligenceActTrilogue:Pressconference—Part4.[Video].

4EuropeanParliament.(March2023).General-purposeartificialintelligence[Backgroundmaterial].

5EuropeanCommission.(December12,2023).ArtificialIntelligence—QuestionsandAnswers[Pressrelease].

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.DecodingtheEUAIAct6

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

Examiningthe

AIAct’simpact

andscope

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.

DecodingtheEUAIAct7

Introduction

Throughourlens:ThepotentialimpactoftheAIAct

Stimulatethepositive

•Stimulateinnovationthroughregulatory

sandboxeswheresmallandmedium-sizedenterprisescantesttheirAIsystemswithoutimminentregulatoryscrutiny.

•Promoteharmonizationofstandards,codesofconductandcertification.

•OffergreatertransparencyregardingAIsystems.

•Createalevelplayingfieldforthoseinvolved.

•SafeguardfundamentalrightsandprovidelegalcertaintyforindividualsresidingintheEU.

Adoptbestpractices

•CategorizeyourAIsystemsandunderstandtheassociatedrisks.

•Imposemorestringentrequirementsforhigh-riskAIsystems(obligatoryriskmanagement,data

governance,technicaldocumentation,etc.).

•Carryoutconformityassessmentsandpost-marketmonitoringforhigh-riskAIsystems.

•Establisheffectiveoversightandenforcementmechanisms.

Manageandreducerisks

•ProhibitunacceptablerisksinAIsystems.

•Avoidfundamentalrightsviolations.

•Preventtheuseofsubliminalorunethical

techniquesthatmightinfluenceordistorta

person’sbehaviorinsuchawaythatitcausesharmtothatpersonoranotherperson.

•Minimizebiasthatcouldresultinunfairorinadequateoutcomes.

•Restricttheexploitationofvulnerablepeopleorgroupsduetotheirage,disability,politicalopinionorotherfactors.

Summary

Impactandscope

Keycomponents

Nextsteps

TheEuropeanCommission(EC)proposedtheAIActinApril2021.AsofDecember2023,theEuropeanParliament,theEuropeanCouncilandtheEuropeanCommissionhavereachedaprovisionalagreementtomaketheAIActlaw.

TheproposedAIActisexpectedtoreshapehow

wethinkaboutandmanageAIsimilarlytowhathas

happenedindataprivacyoverthelastcoupleofyears.

Expectedtobecomelawin2024,theAIActwill

likelyhaveanimmediatewide-rangingimpactonany

businessoperatingintheEUthatoffersAIproducts,

servicesorsystems.Thelawintroducesadefinitionfor

AIintheEU,categorizesAIsystemsbyrisk,laysout

extensiverequirementsandnecessarysafeguarding

mechanismsforAIsystems,andestablishes

transparencyobligations.

Whatitaimstodo?

TheECaimstobalancepromotingAIdevelopment

andboostinginnovationwithmanagingemerging

riskseffectively.Thisisreflectedintheobjectivesof

theproposal:6

•EnsuringthatAIsystemsontheEUmarketaresafeandrespectpublicrightsandvalues.

•ProvidinglegalcertaintytofacilitateinvestmentandinnovationinAIsystems.

•Enhancinggovernanceandeffectiveenforcementofethicsandsafetyrequirements.

•FacilitatingthedevelopmentofasingleEUmarketforlawful,safe,trustworthyAIapplicationswhilepreventingmarketfragmentation.

6EuropeanCommission.(April21,2021).ProposalforaRegulationoftheEuropeanParliamentandoftheCouncil,“LayingDownHarmonisedRulesonArtificialIntelligence(ArtificialIntelligenceAct)andAmendingCertainUnionLegislativeAct.”

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.DecodingtheEUAIAct8

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

Toachievetheseobjectives,theAIActappliesa

risk-basedapproach.Thisallowsforestablishing

specificminimumrequirementstoaddresstherisks

andproblemslinkedtoAIsystemswithoutunduly

constrainingorhinderingtechnologicaldevelopmentordisproportionatelyincreasingcostsrelatingtoplacingAIsystemsonthemarket.

Whowillbeaffected?

Mostorganizations,bothinsideandoutsidetheEU,aredevelopingorusingAIsystemsthatwilllikelyqualify

asAIunderthescopeoftheAIAct.Giventheshort

implementationperiod,however,organizationsshouldgainaprofoundunderstandingoftheAIsystemstheyaredevelopingand/ordeployingandhowtheymeasureuptotheAIAct’srequirements.

Whatpartiesarecovered?

•AnyproviderplacingAIsystemsonthemarketorputtingthemintoservicewithintheEU,regardlessoflocation.

•AnyproviderofAIsystemslocatedoutsidetheEU,whosesystemoutputcanorisintendedforuseintheEU.

•AnyproviderofAIsystemslocatedintheEU.

•AnyimporterordistributorplacingAIsystemsonthemarketormakingthemavailablewithintheEU.

•ProductmanufacturersplacingproductswithAI

systemsonthemarketorputtingthemintoservicewithintheEUundertheirnameortrademark.

•UsersofAIproductsandserviceswithintheEU.

Whatisnotcovered?

•AIsystemsdevelopedorusedexclusivelyformilitarypurposes.

•AIsystemsusedbypublicauthoritiesor

internationalorganizationsinnon-Unioncountrieswhenusedforlawenforcementorjudicial

cooperationwiththeEUunderaframeworkofinternationalagreements.

•AIsystemsdevelopedandusedforthesolepurposeofscientificresearchanddiscovery.

•AIsystemsintheresearch,testing,and

developmentphasebeforebeingplacedonthemarketorputintoservice(thisincludesfree

andopen-sourceAIcomponents).

•PeopleusingAIforpersonaluse.

InthesamewaytheGeneralDataProtection

Regulation(GDPR)isenforced,theECunderstands

thatnon-Europeanentitiessellingtheirproductsin

Europeanmarketsshouldberegulatedsimilarlytothememberstates.TheEUisexpectedtobethecentergroundforglobalAIstandards,withdivergenceintheUSandpossiblytheUK.LiketheGDPR,theAIActwillhaveandextra-territorialeffect.

Whoisaffectedinyourorganization?

Executiveswhomanagecompliance,data

governanceandthedevelopment,deploymentand

useofAItechnologieswilllikelyseetheirrolesand

responsibilitiesimpactedbytheAIAct.Beyond

seniorrolesintheorganization,theBoardofDirectors

andvariousGovernanceCommitteesmayalsobe

affected,andtheyshoulddevelopawarenessand

knowledge.GiventhebroaddefinitionofAIandthe

currentpaceofproliferation,organizationsshould

takeaholisticapproach.Seniorexecutivesshould

collaborateonpurposefulinnovationanddevelopment,

riskmanagement,andgovernanceofAIsystemsto

achievecompliancewiththeAIAct.

Howitwillbeenforcedandwhatare

thepenalties

TheEChasproposedastructureforenforcingAI

providerrequirementsbyestablishinganArtificial

IntelligenceBoardandExpertGroup.BothpartiessitattheEUlevelandareresponsiblefor:

•Contributingtoeffectivecollaborationwithnationalsupervisoryauthorities.

•Providingrecommendationsforbestpractices.

•Ensuringconsistentapplicationoftheregulation.

Eachmemberstatewillbeexpectedtocreateor

designateaNationalCompetentAuthoritytoensure

theimplementationoftheregulationandtosafeguard

theobjectivityandimpartialityoftheiractivities.

TheEU’sproposedregulationwilllikelyhavea

far-reachingimpactonallorganizationsleveraging

thevastpowerofAI,andtheconsequencesof

noncompliancecouldrangefromrestrictingmarket

accesstosignificantfinesdependingonthelevel

ofnoncompliance.Finesmayrangefrom35million

eurosor7percentofglobalturnoverto7.5millionor

1.5percentofturnover,dependingontheinfringement

andsizeofthecompany.7

7EuropeanParliament.(December9,2023).ArtificialIntelligenceAct:dealoncomprehensiverulesfortrustworthyAI[Pressrelease].

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.DecodingtheEUAIAct9

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

Tracking

theEU’s

legislative

journey

Prohibitionson‘unacceptable

risk’AIsystemswill

June2023–December2023

FinalAIActnegotiationsoccur

betweentheCouncil,Commission,and

Parliament.Aprovisionalagreementto

finalizetheproposedrulesisreachedin

December2023.

December2022

June2023

itscommonposition

(‘generalapproach’)

ontheAIAct.

0April2021

TheEuropeanCommissionunveilsa

proposalforanewArtificialIntelligenceAct.

Spring2026

ThefinalAIActtakeseffectinitsentirety.

Wearehere

Thefinaltextisexpectedinthefirsthalfof2024.

TheParliamentadoptstheirnegotiationpositionforthedraftAIAct.

Severalobligationstogeneral-purposeAIwillapply.

TheCouncilhasadopted

apply.

.

Late2024

Mid2025

Whenwillitapply?

MostoftheobligationsoutlinedintheAIActare

expectedtobecomeeffectivebythefirsthalfof

2026.Prohibitionsareanticipatedtotakeeffectby

theendof2024,andobligationsregardinggeneral-

purposeAI(GPAI)areexpectedtotakeeffectas

earlyas2025.GPAIreferstoAIsystemsthatperform

generallyapplicablefunctions,suchasimageand

speechrecognition,audioandvideogeneration,

patterndetection,questionanswering,translationand

others,butcanhaveawiderangeofpossibleuses,

bothintendedandunintended.Thesesystemsmay

beutilizedashigh-riskAIsystemsorincorporatedas

componentsofotherhigh-riskAIsystems.

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.

DecodingtheEUAIAct10

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

Unravellingthe

AIAct’skey

components

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.

DecodingtheEUAIAct11

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

TheAIActisacomprehensivedocumentdesignedtohelpprovideacleardefinitionofartificialintelligence,enabling

EU-widealignmentandconsistencywithotherUnionlawsandregulations.TheAIAct'sprimarygoalistoestablisha

uniformandhorizontallegalframeworktopromotetheuptakeofAIsystemswhileprovidingahighlevelofprotection

againsttheirharmfuleffects.ThisframeworkcanhelptobuildtrustinAItechnologyandgiveindividualsandorganizationsgreaterconfidenceinusingit.

Definingartificialintelligence

TheAIActappliesabroaddefinitionofanAIsystemderivedfromtherecentlyupdatedOrganisationforEconomicCo-operationandDevelopment(OECD)definition.WhiletheAIAct’stextisnotyetpubliclyavailable,theOECDdefinitionisasfollows:

“AnAIsystemisamachine-basedsystemthat,for

explicitorimplicitobjectives,infers,fromtheinputitreceives,howtogenerateoutputssuchaspredictions,content,recommendations,ordecisionsthatcan

influencephysicalorvirtualenvironments.DifferentAIsystemsvaryintheirlevelsofautonomyand

adaptivenessafterdeployment.8”

Thisdefinitionisdeliberatelykeptbroadtocover

thewholespectrum,fromsimplertechnologiesandsystemsfocusingonsingle-usecasestoadvancedapplicationsofdeeplearningandgenerativeAI.Asaresult,theAIAct'sscopebecamemuchwider

thaninitiallyanticipated,extendingsignificantly

beyondourmorerecentunderstandingofadvancedand

generativeAI.TheAIAct'sdefinedscopehasseveral

exemptionsforAIsystems,suchasthoseusedformilitaryordefensepurposesandlimitedexemptionsforfreeandopen-sourcesystems.

AIriskframeworkandrequirements

TheAIActdefinesaframeworktounderstandtherisksassociatedwithAI.ItclassifiesAIsystemsbasedontheirpotentialrisksanddividesthemintodifferentcategoriesdependingonthedatatheycapture,andthedecisionsoractionstakenwiththatdata.

EUobligationswillvarydependingonthecategoryofAI

beingused.Whileanagreementonthecontexthasbeenreached,thefinaltextoftheregulationsisnotyetavailable.However,thefollowingsectionssummarizetheobligationsstipulatedundertheAIAct,basedonthepublicly

availableinformation.9

8NotethattheearliestversionoftheAIActdefinesAIsystemsassystemsthatweredevelopedusingoneofthefollowingtechniquesandapproaches:

(a)Machinelearningapproaches,includingsupervised,unsupervised,andreinforcementlearning,usingawidevarietyofmethods,includingdeeplearning.(b)Logic-andknowledge-basedapproaches,includingknowledgerepresentation,inductive(logic)programming,knowledgebases,inferenceand

deductiveengines,(symbolic)reasoningandexpertsystems.

(c)Statisticalapproaches,Bayesianestimation,searchandoptimizationmethods.

9EuropeanCommission.(April21,2021).ProposalforaRegulationoftheEuropeanParliamentandoftheCouncil,“LayingDownHarmonisedRulesonArtificialIntelligence(ArtificialIntelligenceAct)andAmendingCertainUnionLegislativeAct.”

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.

DecodingtheEUAIAct12

Introduction

Minimalrisk

Noobligations●

Limitedrisk

Highrisk

Summary

Impactandscope

Keycomponents

Nextsteps

TheAIActtakesarisk-basedapproach

Prohibited

Unacceptablerisk

ContraveneUnionvalues,

likefundamentalrights.

Conformityassessment

Highrisktohealth,safety,

environmentand

fundamentalrights.

Transparencyobligation

Riskofimpersonation

ordeception.

Nohighrisks.

Examplesofunacceptable-riskAIsystems

•Behavioralmanipulation.

•Exploitationofvulnerable

characteristicsofpeople.

•Socialscoringbypublicauthorities.

•Real-timeremotebiometricidentificationfor

lawenforcement.

Examplesofhigh-riskAIsystems:

•Evaluationofeligibilitytocredit,healthorlifeinsuranceorpublicbenefits.

•Analysesofjobapplicationsorevaluationofcandidates.

•Productsafetycomponents.

Examplesoflimited-riskAIsystems:

•AIsystemsthatinteractwithconsumers.

•GenerativeAI*:AIsystemsgeneratingor

manipulatingcontent(image,audioorvideo).

Examplesofminimal-riskAIsystems:

•Spamfilters.

•AI-enabledvideogames.

*FurtherspecificobligationstogenerativeAIandfoundationmodelswillapplyoutsideofthisrisk-basedapproach

©2024CopyrightownedbyoneormoreoftheKPMGInternationalentities.KPMGInternationalentitiesprovidenoservicestoclients.Allrightsreserved.

DecodingtheEUAIAct13

Introduction

Summary

Impactandscope

Keycomponents

Nextsteps

Unacceptable-riskAIsystems

WhichAIsystemsarecovered?AIsystemsthat

enablemanipulation,exploitationandsocialcontrol

practicesareseenasposinganunacceptablerisk.

ThiscategoryprohibitsAIforthefollowingpurposes:

•ManipulationthatharmsorislikelytoharmanAIuseroranotherperson.

•Exploitingvulnerabilitiesofaspecificgroupofpersons.

•Socialscoringleadingtodetrimentalorunfavorabletreatmentinsocialcontexts.

•Indiscriminatescrapingoffacialimages.

•Emotionrecognitionsoftwareintheworkplaceandeducation(withsomeexceptions).

•UseofAIthatcategorizespersonsbasedon

sensitivetraitssuchasrace,politicalopinionsorreligiousbeliefs.

•Predictivepolicingonindividuals(riskscoringfor

committingfuturecrimesbasedonpersonaltraits).

•Remotebiometricidentificationofpeople(partialbanwithsomeexceptionsinlawenforcement).

Whataretheobligationsrelatedtothiscategory?

SincetheAIsystemsinthiscategoryposeanunacceptablerisk,theiruseisprohibited.

High-riskAIsystems

WhichAIsystemsarecovered?AIsystemsthat

negativelyaffectsafetyorfundamentalrightswill

beconsideredhighriskandwillbedividedintotwo

categories:1)AIsystemsthatareusedinproductsfalling

undertheEU’sproductsafetylegislation.Thisincludestoys,aviation,cars,medicaldevicesandelevators.2)AIsystemsfallingintospecificareasthatwillhavetoberegisteredinanEUdatabase.8Theseinclude:

•Criticalinfrastructure,suchasthesupplyofutilities.

•Educationalandvocationaltraining,forexample,

automatedscoringof—orexclusionfrom—exams.

•Employment,workersmanagementandaccesstoself-employment,forexample,automated

recruitmentandapplicationtriage.

•Accesstoessentialprivateandpublicservices

andbenefits(e.g.healthcare),creditworthiness

evaluationofnaturalpersons,andriskassessmentandpricinginrelationtolifeandhealthinsurance.

•Law-enforcementsystemsthatmayinterferewithfundamentalrights,suchasautomatedriskscoringregardingpotentialoffenders,deepfakedetectionsoftwareandevidencereliabilityscoring.

•Migration,asylumandbordercontrolmanagement,forexample,verificationofauthenticityoftravel

documentsandvisaandasylumapplicationexaminations.

•Administrationofjusticeanddemocratic

processes,forexample,legalinterpretationtoolstoassistjudicialauthorities.

Mostorganizationsusethesehigh-ri