2024欧盟人工智能法案对欧洲中东和非洲地区医疗科技领域的影响分析报告 How the EU AI Act is Reshaping Medtech in EMEA

HowtheEUAIActisReshapingMedtechinEMEA

Alphasense

What’sInside

1Introduction

3CHAPTER1

ANeedforNewRules

5CHAPTER2

UnderstandingtheAIAct

7CHAPTER3

RiskClassificationUndertheAIAct

11CHAPTER4

ExistingRegulationOverlapswiththeAct

15CHAPTER5

StayonTopoftheEvolvingAILandscapewithAlphaSense

17CHAPTER6

LookingForward

Alphasense

Introduction

SincetheCOVID-19pandemicgrippedtheworld,globalmedical

marketshavehadtodramaticallyreinventthemselvestowithstandanumberofunpredictableandvolatileevents.In2020,once

unshakeablesupplychain-efficiencystoodatastandstillinthewake

ofworldwidelockdowns.Newgeopoliticalconlictsthataroseafterthepandemic’sinception(e.g.,Russia-Ukrainewar,Israel-

Palestineconlict,etc.)posedthechallengeofprovidinglife-saving

medicalcareinthemidstofwarfare.Likewise,demandforchronic

conditionstreatmentsskyrocketed.Thebottomline:innovationwithin

healthcarewasdesperatelyneeded,astraditionalorexistingmedicalinterventionwasfallingshort.

Cueartificialintelligence(AI)andthequick-pacerolloutofgenerative

AI(genAI).Inamatterofmonths,thistechnologyreimaginedhowdrugdiscovery,R&D,massivedataanalysis,vaccinedevelopment,

andmuchmorecouldbeconductedmoreefficientlyandeffectively.ExcitementremainspalpableforgenAIasresearchersandmedtech

developerswieldittofulfilloncetedious,time-consumingprocesses.

However,thisexcitementhasledtosomespeculationaboutwherethistechcouldbetroublesomefortheentireindustry.

ToputparametersaroundtheuseofAIinhealthcareandquash

qualmsofpotentialprejudicialbiasorprivacyrisksassociatedwithit,theEUrecentlyintroducedtheArtificialIntelligenceAct(AIAct),“theworld’sfirstcomprehensiveAIlaw.”HavingbeenpassedthisMaybytheCounciloftheEuropeanUnion,theActisleadingmedtechleaderstonotonlyquestionhowthesestipulationsaffecttoolsalreadyin

1

circulationwithinthehealthcaresystems,butwhatitmeansforafuturewhereAIisattherootoftheindustry.

UsingtheAlphaSenseplatform,weunveilhowtheAIActwillreshape

medtechdevelopmentinthecomingmonthsandyears.Discoverinsightsthatwillequipyouwiththeintelligenceneededtostay

attheforefrontofthemedtechindustryandremainagiletorisingtechnologyandnewlegislation.

2

CHAPTER1

ANeedforNewRules

Simplyput,theAIActaimstobuildtrustamongEuropeans

regardingthebenefitsofAI.AlthoughmanyAIsystemspresent

littletonoriskandhavethepotentialtoaddressvarious

societalchallenges,someAItechnologiesintroducerisksthatneedtobemanagedtopreventnegativeconsequences.

AstheEUCommissionstates,“itisoftennotpossibleto

findoutwhyanAIsystemhasmadeadecisionorpredictionandtakenaparticularaction.So,itmaybecomedifficulttoassesswhethersomeonehasbeenunfairlydisadvantaged,suchasinahiringdecisionorinanapplicationforapublicbenefitscheme.Althoughexistinglegislationprovides

someprotection,itisinsufficienttoaddressthespecificchallengesAIsystemsmaybring.”

TheActaimstoaccomplishthefollowing:

RecognizeanddefuserisksgeneratedbyAIapplicationsDisarmAIpracticesthatposeunacceptablerisks

Identifyalistofhigh-riskAIapplications

EstablishrequirementsforAIsystemsthatentailhigh-riskapplications

Setspecificobligationsfordeployersandprovidersofhigh-riskAIapplications

EnactaconformityassessmentbeforeagivenAIsystemisputintoserviceorplacedonthemarket

Alphasense

3

EnforcelegislationforAIsystemsoncethey’veenteredthemarket

BuildagovernancestructureattheEuropeanandnationallevel

Buthowdoesthisimpactmedtechcompanies?

Essentially,itinluencestheircomplianceresponsibilitiesandsafeguardstheirfinancialinterests.AnybusinesswhoseAI

system’soutputisusedintheEUmustadheretothislaw,regardlessofthecompany’sorsystem’slocation.

“Inthatsense,thereisreallynowayarounditasa

developer,[they]willhavetocomplywiththisact.It

reallywillbeanewlyregulatedspaceforthesespecific

AIproducts,withtheonlyexceptionthatthisonlyappliestothehigh-riskAIproducts,whichisthatmostofthe

productsthatareusedinhealthcarearehighrisk.

Productscanalsobelowrisk,forexample,ifit’sonly

usedinthehospitaladministrationprocessanddoesn’tengagedirectlywithpatients,andthenyoudon’thavetocomplywitheverypartoftheAIAct,youjusthavetoshowthatit’sasafeproductlikeanyotherproductsyouwouldputonamarket.”

—IndustryExpert|ExpertTranscript

Andformedtechcompanies,thetimeisnowtofamiliarizethemselveswiththeActandadoptresponsibleAIpractices.AccordingtotheWorldEconomicForum,implementing

responsibleAIprocessestypicallytakesaroundthreeyears.Companiesthatstartearlywillbeinastrongerposition

forcompliance,gainingstakeholdertrust,optimizingAI’sbusinessadvantages,andmore.

Alphasense

4

Act

CHAPTER2

UnderstandingtheAI

Thewide-spreaddistributionofgenAIanditsapplication

totoolswieldedbynearlyeveryprofessionaltoday(i.e.

lawyersusingChatGPTtocitecasesforcourt)hasonly

raisedconcernsaboutthelimitationofthistech—orlack

thereof.Withinthehealthcareindustry,manyskepticshaveraisedprivacyconcernsregardingpatientinformation,

biasedorprejudicialalgorithms,andunethicalusageof

genAI,leadingtheEUtocraftalegislativeframeworkthatsafeguardsbothpatientsandproviders.

Atthemostfundamentallevel,theAIActsetsparameters

forhowartificialintelligenceandgenAIisleveragedand

distributedwithinthehealthcaresectors.TheNational

LibraryofMedicinestatesthattheAct“aimstosetout(a)

uniforminternalmarketrules‘forthedevelopment,placingonthemarket,puttingintoserviceandtheuseofartificial

intelligence’inamannerthat(b)promotes‘humancentric

andtrustworthyartificialintelligence’and(c)‘ensuresahighlevelofprotectionofhealth,safety,fundamentalrights.’”

Initsfirstyearofenactment,theActwillinitiallyreform

general-purposeAIsystems,andintwoyears’time,extendtothosethatincludeAI-enableddigitalhealthtools(DHTs),andeventuallycoverhigh-riskAIsystems(i.e.DHTs).

Itsimultaneouslyintroducesarangeofnewobligations

fordevelopers,deployers,notifiedbodies,andregulators,particularlythoseworkingwiththenewlycreatedAIOfficeandtheEuropeanCommission.LikeotherEUregulations,itpertainstoDHTsdevelopedthataremarketedandused

5

Alphasense

outsidetheEU,ormorespecifically,“AIsystemsandmodelsspecificallydevelopedandputintoserviceforthesole

purposeofscientificresearchanddevelopment.”

RegulatorsandmedtechtitansalikeremainconfusedabouttheexactanddefiniteparametersoftheAct.According

toNatureMedicine:“Duetothehorizontalnature,widescope,andrapidlychangingnatureofhealthcareAI,

therewillbemanyproblemscreatedbyambiguitiesand

uncertainintersectionswithexistinglaws.Here,lexibility

andintelligenceoftheresponsiblebodies,includingthe

Commission,theEUAIOffice,thecommitteesresponsible

forcreatingguidelinesformedicaldevicesoftwareunder

theMDR,andevenofthecourtswillbeessentialtomeetingthestatedaimoftheAIAct,topromoteratherthanto

decimateAIsectorentrepreneurialactivityandinnovation.”

6

Al

phasen

se

CHAPTER3

RisksClassifications

UndertheAct

TheregulatoryframeworkoftheAIActdefinesfourlevelsofriskforAIsystems:

UnacceptableRisk

Socialscoring,facialrecognition,darkpatternAI,manipulation

HighRisk

Safetycomponentsincriticalinfrastructure,

employmentandperformanceinwork,accessto

education,accesstopublicservices,useininsurance,creditscoring,bordercontrol,justicesystems

LimitedRisk

GeneralpurposeAIandAIsystemswith

specifictransparencyrequirementssuchaschatbots,emotionrecognitionsystems

MinimalRisk

AI-enabledvideogames,spamfilters

GraphicsourcedfromForvisMazars

UnacceptableRisk

AnunacceptableriskisattributedtoAIsystemsthatviolatecoreEuropeanUnionprinciples,includingrespectforhumandignity,freedom,equality,democracy,andtheruleoflaw,aswellas

fundamentalrightsrecognizedbytheUnion.Consequently,AIsystemscategorizedunderthisriskarebanned.

Alphasense

7

Inotherwords,thesesystemsconductprocesses,leverageorproduceinformationdeemedunlawfulbytheCommissioninthefollowingways:

AIsystemsthatmanipulateindividualsintoperformingdangerousorunintendedactions,ormakingdecisionstheyotherwisewouldnot,areproblematic.

Additionally,systemsthattakeadvantageofthe

vulnerabilitiesofcertaingroups,suchasthosebasedonage,disabilities,orspecificsocialoreconomic

conditions,poseethicalconcerns.

Biometriccategorizationsystems,whichanalyzedatalikefacialfeaturesorfingerprintstoinfercharacteristicssuchaspoliticalviews,religiousbeliefs,orsexualorientation,alsoraiseissues.

Similarly,theuseofreal-timebiometricidentification

inpublicspacesforlawenforcementpurposes,thoughtherearesomeexceptions,iscontroversial.

Systemsthatclassifypeopleovertimebasedontheirsocialbehavior,knownassocialscoring,areanother

concern.Moreover,AItechnologiesdesignedtoassessemotionalstatesinworkplaceoreducationalcontextsshouldbebanned.

HighRisk

TheseAIsystemstypicallypossesshighimpactcapabilities

and,therefore,carrysystemicrisksthatcanaffectthehealth,safety,andfundamentalrightsofpersonsintheUnion.

Highrisksystemsareusedinavarietyoftoolsrelevant

tothemedtechandhealthcareindustry,andinclude

medicaldevices(i.e.anyAIsystemthatisaClassIIaor

highermedicaldeviceorusesAIasasafetycomponent),emergencytriageAIsystems(usedforemergencypatienttriagetreatmentorsurgery),andAIsystemsmonitoringorassessingemotionrecognition.

Alphasense

8

Morespecificexamplesofthesesystemsinclude:

AI-assistedX-RayDiagnosis

AsylumHealthRiskScreening

MedicalTrainingAssessmentAISoftware

HealthcareWorkforceManagementAISystems

High-riskAIsystemsmustadheretotheobligationssetforthbytheCommissionbeforeenteringthemarket.Soastoensuretheefficacyandreliabilityofthesesystems,developersmustincorporateadequateriskassessmentandmitigationsystems,aswellashighqualityreferencedatasetstominimizerisksanddiscriminatoryoutcomes.

Foraccountabilitypurposes,asystemmustalsologallactivitytoensuretraceabilityofitsresults,whiledevelopersmust

providedetaileddocumentationonthesystemanditspurposeforauthoritiestoassesscompliance.Whenitcomesto

consumerorend-userprotection,appropriatehumanoversightmeasuresoverasystemarerequiredtominimizerisk,in

tandemwithahighlevelofrobustness,securityandaccuracy.

LimitedRisk

LimitedriskreferstosystemsthatlacktransparencyintheirAIusage.TheActinstillstransparencyobligationstoensurethatconsumersareinformedwhennecessaryinorderto

fostertrustwithanAIsystem,likeachatbot.Forinstance,

anend-usershouldbemadeawarethattheyareinteractingwithamachinesotheycanmakeaninformeddecisionon

howtoassesstheinformationbeingpresentedtothem.

ProvidersalsohavetoensurethatAI-generatedcontentisidentifiable.AnyAI-generatedtextpublishedintendedtoinformonmattersofpublicinterestmustbelabeledasartificiallygenerated,includingaudioandvideocontentconstitutingdeepfakes.

Alphasense

9

MinimalorNoRisk

AIsystemsunderminimalriskdonothaveanyrestrictions

ormandatoryobligations.ExamplesofminimalornoriskAIsystemsincludespamfiltersandAI-enabledvideogames.

AccordingtotheEU,“avastmajorityofAIsystemscurrentlyusedintheEUfallintothiscategory.”TheCommissionandBoard,however,encouragedrawingupcodesofconduct

intendedtofosterthevoluntaryapplicationofrequirementstolow-riskAIsystems.

Ultimately,theCommission’srisk-basedstrategyrequires

organizationstoassesstheAIsystemstheycurrentlyuseorplantoimplementanddeterminetheriskleveleachsystempresents.CreatinganinventoryofAIsystemscanserveasaninitialstepinthisprocess.Oncetherisklevelsareidentified,conductingagapassessmentinrelationtotheAIActhelpsorganizationsevaluatehowtheircurrentpracticescanbe

adaptedtomeetcompliancerequirements,allowingforthedevelopmentofastructuredplan.Additionally,publicsectorentitiesmustconsidergovernment-issuedguidelinesontheethicalandresponsibleapplicationofAItechnologies.

10

Alphasense

CHAPTER4

ExistingRegulation

OverlapswiththeAct

WhiletheAIActsetsoutnobleintentionstosafeguardsocietyfrompotentialAIblunders,confusionamongst

medtechprofessionalsregardingtheproposedbroad-

basedAIActandhowitwillcoexistorwork-in-tandem

withexistinglegislation—andnothinderproductionandinnovation—isn’tnew.

TheAIActbroadlyappliestocorporationswholeverage

AIandgenAIforconsumeruse,itsetsparametersin

accordancewithEUpolicyto“protectandimprovehealth,givingequalaccesstomodernandefficienthealthcareforallEuropeans,andcoordinatinganyserioushealththreatsinvolvingmorethanoneEUcountry.”

However,manyexpertsandindustryparticipantsbelievethatinsteadofaone-size-fits-allapproach,theexistingsector-

specificregulations—namely,theMedicalDeviceRegulation(MDR)andtheInVitroDiagnosticRegulation(IVDR)—couldbealteredtoaddressthenuancesofAItechnologies.

Currently,thesesector-specificlawsclassifyAI-powered

softwareasatypeofmedicaldeviceifitisdesignedto

assistinmakingmedicaldecisions.Thisincludesfunctions

relatedtodiagnosing,preventing,monitoring,predicting,

prognosing,treating,oralleviatinghealthconditions,aswellassupportinglifestylechangesrelatedtohealth.

11

Alphasense

TheActincorporatesmanyprovisionsthatechothosefoundintheMDRandEUIVDR,suchasthemandatesforaqualitymanagementsystem,technicaldocumentation,andusage

instructions.Medtechmanufacturerswhohavealready

achievedcertificationundertheEUMDRandIVDRwilllikelyneedtoupdatetheirtechnicaldocumentationtoalignwiththenewstipulationsoftheEUAIAct.

InadditiontoexistingEUMDRandIVDRrequirements,theActintroducesseveralnewstipulationsforAIsystems,including:

Governanceanddatamanagementprotocolsfortrainingandtestingdatasets

Enhancedrecord-keepingpractices,suchasautomatic

eventloggingthroughoutthesystem’slifecycle

Designtransparencytoensurethatuserscanunderstand

andproperlyutilizethesystem’soutputs

RequirementsforhumanoversightindesignStandardsforaccuracyandcybersecurity

Butwhatmedtecharegripingaboutisthetime-consuming,costlyburdensofMDRandIVDRcertification,andhowtheAIActcouldcomplicatethisprocess.Accordingtoglobal

lawfirmLoganHovell:

“Theconformityassessmentofmedicaldevicesunderthe

MDRorIVDRcanbealengthy(andcostly)process(18

monthsormoreonaverage).Whiletheindustryislooking

attheEUregulatorstofindsolutionstomakethecurrentCEmarkingprocessmoreefficient,therearesomefearsthattheapplicationoftheAIActmayaddanadditionalburdentothereviewbyNotifiedBodiesandnegativelyaffectthetimelinenecessarytoaffixtheCEmarktomedicaldevicesinthe

EU.ConsideringthecurrentworkloadofcertainMDR/IVDRNotifiedBodies,suchpotentialriskcannotbedismissed.”

WhilethegoaloftheAIActistobalancethepromotion

ofinnovationandthenecessityofsafeguardingusersfrom

potentialrisksposedbyAI,italsoleavesagrayareainitsriskclassificationfordevicesbetweentheAIAct,MDR,andIVDR

Alphasense

12

SubjecttothrdpartyassesmentunderUnion

Harmonisation

law(i.e.anotifiedbody)

ManufacturersofhighriskAI

systemswillbe

requiredto

complywiththeMDR/IVDR+AIA

AIusedasasafetycomponentortheAIsystemisitselfaproduct

CoveredunderUnion

Harmonisationlaw(i.e.theMDR)

HighriskAI

medicaldevicesystem

!!HighriskundertheAIAdoesnotmeanriskundertheMDR!!

AI-poweredmedicaldevicesmaybeclassifiedashigh-riskAI

systemsiftheyfunctionasasafetycomponentofaproductoriftheyareconsideredproductsthemselvesunderUnionHarmonisationLaw,asoutlinedinAnnexIoftheAIAct.This

includesregulationssuchastheMDRandIVDR,especiallywhenthird-partyassessmentsarerequired.Insuchcases,theAI

medicaldevicemustthencomplywithboththeAIActandtheMDRorIVDR.

GraphicsourcedfromLoganHowell

Despitelawmakers’effortstostreamlineoverlapping

regulations,manyquestionsremainunresolved.For

example,therelationshipbetweentheAIAct’sruleson

substantialmodificationsandthemodificationrequirements

intheMDRandIVDRisstillunclear.It’salsouncertain

whetherdevicesundergoingtrials,suchasperformance

evaluationsorclinicalinvestigations,willneedcertificationundertheAIActbeforeuseinthesetrials.

Alphasense

13

“Oneofthechallengesformedtechmanufacturers

isthatsomuchoftheAIActdoesnotdirectlyapplytomedicaldevices.OnenoteworthyprovisioninthedrafttextisfoundinRecital30,whichstipulatesthatanymedicaldeviceincorporatingAIcomponentsandrequiringtheinvolvementofanotifiedbodyduring

aconformityassessmentundertheMDRorInVitro

DiagnosticsRegulation(IVDR)wouldautomaticallybeclassifiedasahigh-riskdeviceundertheAIAct.”

–ClarivatePlc|‘RegulatoryLasagne’andtheImpactoftheEuropeanAIActonMedtech

Inthecomingmonthsandyears,itisexpectedthatthe

differencesbetweentheEUAIAct,MDR,andIVDRwill

becomeclearer.HowcanmedtechcompaniessuccessfullylaunchanAIsystemthatcomplieswithallthreeregulatoryframeworks?Thesolutionliesinregulatorysandboxes.

TheEUAIActproposescreatingcoordinatedAI“regulatorysandboxes”toencourageinnovationacrossEurope.

Thesesandboxesallowbusinessestotestandrefinetheirnewproducts,services,orbusinessmodelsunderthe

supervisionofregulators.Thiscontrolledenvironment

givesinnovatorsasafespacetoexperimentwhilehelpingregulatorsunderstandnewtechnologiesandultimately

expandconsumerchoices.However,topreventmisuseorexploitation,regulatorysandboxesneedasolidlegalframeworktoensuretheirproperfunctioning.

“RegulatoryoverlapwiththeMDRandIVDRislikelytopersistinthefinalizedAIActtext.Howeverthoroughpreparationbymedtechcompaniesandtheuseof

regulatorysandboxescouldhelpaddresstheneedfordualconformity.”

–MedtechInsight|EUAIActRegulatoryOverlap“LikelytoPersist”:ExpertPresentsSolutionsforDualConformity

Alphasense

14

CHAPTER5

StayonTopoftheEvolvingAI

LandscapewithAlphaSense

It’struethatintoday’smedtechmarket,ifyou’renotabreastofeverynewdevelopment,legislation,andcompetitor,

you’llnotonlystruggletotakealeadinyourrespectiveindustry,butalsomatchthecaliberofcompetition.

InthecaseoftheAIAct,wherelawmakersanddevelopers

arenavigatingthelegislation’scomplexitiesonadaily

basis,traditionalresearchmethodswillleaveyououtofthisevolvingindustryconversation.That’swhymoreandmore

professionalsareembracingtheAIerabyinvestinginmarketintelligenceplatformslikeAlphaSense.

AlphaSensesurfacesalltheindustryinsightsyouneedsothat

youcanbenchmarkagainstpeers,findoutaboutanyemergingcompetitors,legislation,ortechnologies,generateideasfor

newproductdevelopment,andidentifyrisks—allinoneplace.

AlphaSensehousesavastuniverseofover10,000trustedcontentsources,includingbutnotlimitedto:

Companydocumentssuchaseventtranscripts,globalfilings,pressreleases,companypresentations,productbrochures,andESGreports,inadditiontoprivate

companydocuments

Industryresearchfromtheworld’stopfinancialresearchfirms,includingCowen,MorganStanley,BankofAmerica,Jefferies,andJPMorgan,aswellasourproprietary

15

equityresearchcontentsetWallStreetInsights®thatisspecificallytailoredforcorporateclients

Alphasense

Uniqueindustryexpertperspectivesfromcustomers,competitors,medicalexperts,doctors,andpartners

Thousandsofcuratednewssources,tradepublications,andregulatoryinformationsuchas510(k)s

TheAlphaSenseplatformalsodeliversunmatchedAI-search

capabilitiesandfeaturesforanalyzingqualitativeand

quantitativeresearch,andcanmineunstructureddataforthemostcriticalinsights,including:

Automatedandcustomizablealertsfortrackingregulatoryfilings,companies,industries,andpotentialinvestments

TableexporttoolsthatsupportM&Aworklowsliketargetlistsandduediligence

SmartSynonyms™technologythatensuresyounevermissasourceimportanttoyourresearch

SmartSummaries,ourfirstgenerativeAIfeature,summarizeskeyinsightsfromearningscallsforfasteranalysis

e

Alphasens

16

CHAPTER6