生成式人工智能专利态势报告

Contents

Acknowledgments5

Keyfindingsandinsights7

Introduction13

GenerativeAI–newsystemswithalonghistory13

Motivationofthisreport16

1GenerativeAI:Themainconcepts19

Backgroundandhistoricalorigins19

Deeplearning20

Discriminativeversusgenerativetasks20

WhatGenAImodelsexist?22

WhatareGenAImodes?23

2GlobalpatentingandresearchinGenAI33

Globaldevelopment33

Toppatentowners35

Keylocationsofinventors40

Keyfilingjurisdictions43

3PatenttrendsinGenAImodels45

Globaldevelopment45

Toppatentowners46

Keylocationsofinventors49

4PatenttrendsinGenAImodes51

Globaldevelopment51

Toppatentowners52

Keylocationsofinventors55

ConnectionbetweenGenAImodelsandGenAImodes55

5PatenttrendsinGenAIapplications57

Globaltrends57

Toppatentowners59

Keylocationsofinventors62

Connectionbetweencoremodelsandapplications63

Connectionbetweenmodesandapplications65

3

Furtherconsiderations67

ConcernsabouttheuseofGenAI67

LimitationsandfutureofpatentanalysisinrelationtoGenAI69

Appendices71

A.1Methodologyforpatentanalysis71

A.2Patentindicators73

A.3Interdependencebetweenmodels,modesandapplications74

A.4Patentsearches77

A.5Prompts82

A.6ScientificpublicationquerywithTheLens86

A.7Miningsoftwareanddatasetmentionsinthenon-patent

literaturecorpus86

A.8Descriptions/examplepatentsforGenAIapplications87

References105

4

Acknowledgments

ThisPatentLandscapeReportonGenerativeArtificialIntelligencewaspreparedunderthe

stewardshipofMarcoAlemán(AssistantDirectorGeneral,IPandInnovationEcosystems

Sector)andunderthedirectionofAlejandroRocaCampaña(SeniorDirector,IPforInnovators

Department)andAndrewCzajkowski(Director,TechnologyandInnovationSupportDivision),

andwasledbyChristopherHarrison(PatentAnalyticsManager,IPAnalyticsSection,TechnologyandInnovationSupportDivision)withthegenerousfinancialsupportbyFunds-In-TrustJapan

IndustrialPropertyGlobalprovidedbytheJapanPatentOffice.

ThereportwaspreparedbyaprojectteamledbyChristopherHarrisonandLakshmiSupriya

(PatentAnalyticsOfficer,IPAnalyticsSection,TechnologyandInnovationSupportDivision)

thatincludedKaiGramke,JochenSpuck,KlausJankandMichaelFreunek(allfromEconSight),PatriceLopez(science-miner)aswellasHongKan(PatentAnalyticsOfficer,IPAnalyticsSection,TechnologyandInnovationSupportDivision),AleksandrBelianovandCraigDsouza(former

YoungExperts,TechnologyandInnovationSupportDivision).

OurthanksalsogotoUlrikeTill(Director,IPandFrontierTechnologiesDivision)forreviewingthereportandprovidingvaluableinput.Finally,ourgratitudetotheWIPOeditorialanddesignteamledbyCharlotteBeauchamp(Head,PublicationsandDesignSection).

email:

ip.analytics@

5

Keyfindingsandinsights

ThereleaseofOpenAI’sChatGPTchatbotinNovember2022hasgreatlyincreasedpublic

enthusiasmforgenerativeAI(GenAI).Ithasbeendescribedbymany,includingNvidiaCEOJen-HsunHuang,asan“iPhonemoment”forGenAI.ThisisbecausetheOpenAIplatformhasmadeiteasierforalluserstoaccessadvancedGenAIprograms,particularlylargelanguagemodels(LLMs).Thesemodelshavereachednewlevelsofperformance,demonstratingthepotential

forvariousreal-worldapplications,triggeringawaveofresearchanddevelopment,andlargecorporateinvestmentsinGenAI.

ThisWIPOPatentLandscapeReportprovidesobservationsonpatentingactivityandscientificpublicationsinthefieldofGenAIandbuildsonthe2019WIPOTechnologyTrendspublicationonArtificialIntelligence.Itaimstoshedlightonthecurrenttechnologydevelopment,itschangingdynamicsandtheapplicationsinwhichGenAItechnologiesareexpectedtobeused.Italso

identifieskeyresearchcountries,companiesandorganizations.

significantlysince2017

GenAIpatentfamiliesandscientificpublicationshaveincreased

TheriseofGenAIoverthepastfewyearshasbeendrivenprimarilybythreefactors:more

powerfulcomputers,theavailabilityoflargedatasetsasasourceoftrainingdata,andimprovedAI/machinelearningalgorithms.DevelopmentssuchasthetransformerarchitectureinLLMs

havesignificantlyadvancedGenAI.Thishasmadeitpossibletodevelopcomplexapplicationsinmanydifferentfields.

ThetechnologicaladvancesinGenAIarereflectedbythesharpincreaseinpatentingactivity.Overthepast10years,thenumberofpatentfamiliesinGenAIhasgrownfromjustonly733in2014tomorethan14,000in2023.Sincetheintroductionofthetransformerin2017,thedeep

neuralnetworkarchitecturebehindtheLargeLanguageModelsthathavebecomesynonymouswithGenAI,thenumberofGenAIpatentshasincreasedbyover800%.Thenumberofscientificpublicationshasincreasedevenmoreoverthesameperiod,fromjust116in2014tomorethan34,000in2023.Over25%ofallGenAIpatentsandover45%ofallGenAIscientificpaperswerepublishedin2023alone.

7

8

GenAIpatentfamiliesGenAIscientificpublications

2014l 2015l 20162017201820192020202120222023

15,00010,00050000500010,00015,00020,00025,00030,00035,000

WhicharethetoporganizationswiththemostpatentsinGenAI?

1.Tencent

2.PingAnInsuranceGroup

3.Baidu

4.ChineseAcademyofSciences

5.IBM

Tencent,PingAnInsuranceGroupandBaiduownthemostGenAIpatents.TencentplanstoaddGenAIcapabilitiestoitsproductssuchasWeChattoimprovetheuserexperience.Ping

AnfocusesonGenAImodelsforunderwritingandriskassessment.BaiduwasoneoftheearlyplayersinGenAIandrecentlyunveileditslatestLLM-basedAIchatbot,ERNIE4.0.TheChineseAcademyofSciences(fourth)istheonlyresearchorganizationinthetop10ranking.Alibaba(sixth)andBytedance(ninth)areotherChinesecompaniesinthetop10.

PatentLandscapeReport–GenerativeArtificialIntelligence

IBM(fifth),Alphabet/Google(eighth)andMicrosoft(10th)arethetopUScompaniesintermsofGenAIpatents.IBMhasdevelopedaGenAIplatform,watsonx,whichenablescompaniestodeployandcustomizeLLMswithafocusondatasecurityandcompliance.Alphabet/Google'sAIdivisionDeepMindrecentlyreleaseditslatestLLMmodel,Gemini,whichisgraduallybeingintegratedintoAlphabet/Google'sproductsandservices.MicrosoftisanotherkeyplayerinGenAIandaninvestorinOpenAI.OpenAIitselfhasonlyrecentlyfileditsfirstGenAIpatents.Roundingoutthetop10iselectronicsconglomerateSamsungElectronics(seventh)fromtheRepublicofKorea.

CorporateUniversity/researchorganization

9

(US)

IBMMicrosoft

(US)

PingAn

Alb)etiiyInsurance

Adobe(China)Group

(US)(China)

Sony

()H(CuiChina

Mobile

(China)

Baidu(China)BBKN(Es

(China)Ele(cssuir

(Jn)(China)ByteDance

(China)

AlibabaGroup(China)

Chinese

Academy

ofSciences

(China)

TencentHoldings (China)

WhichinstitutionspublishedthemostscientificpublicationsonGenAI?

TheChineseAcademyofSciencesisclearlyintheleadintermsofscientificpublicationswithmorethan1,100publicationssince2010.TsinghuaUniversityandStanfordUniversityfollowinsecondandthirdplacewithmorethan600publicationseach.Alphabet/Google(fourth)istheonlycompanyinthetop20(556scientificpublications).

However,whenmeasuringtheimpactofscientificpublicationsbythenumberofcitations,

companiesdominate.Alphabet/Googleistheleadinginstitutionbyawidemargin,andsevenothercompaniesarepresentinthetop20.ThecaseofOpenAIisalsonoteworthy.InourGenAIcorpusofscientificpublications,thecompanyhaspublishedonly48articles(325thinstitutionintermsofnumberofpublications),butthesepublicationshavereceivedatotalof11,816citationsfromotherscientificpublications(13thoverall).

WherearethemostGenAItechnologiesinvented?

1.China

2.UnitedStates

3.RepublicofKorea

4.Japan

5.India

6.UnitedKingdom

Keyfindingsandinsights

7.Germany

InventorsbasedinChinawereresponsibleformorethan38,000patentfamiliesbetween

2014and2023,basedontheinventoraddressespublishedonpatents.Since2017,Chinahaspublishedmorepatentsinthisfieldeachyearthanallothercountriescombined.

PatentLandscapeReport–GenerativeArtificialIntelligence

10

Witharound6,300patentfamiliesbetween2014and2023,theUSisthesecondmostimportantresearchlocationforGenAIpatenting.TheAsiancountriesRepublicofKorea,JapanandIndia

areotherkeyresearchlocationsforGenAI,allrankinginthetop5countriesworldwide(third,fourthandfifthrespectively).TheUnitedKingdomistheleadingEuropeanlocation(sixth

globally),with714patentspublishedinthesameperiod.However,Germanyisclosebehind(708patentfamilies)andhaspublishedmoreGenAIpatentsthantheUKinrecentyears.Thesetopinventorlocationsaccountforthemajority(94%)ofglobalpatentingactivityrelatedtoGenAI.

China

US

RepublicofKorea

Japan

India

UK

Germany

Restofworld

Canada

Israel

France

WhichGenAImodelhasthemostpatents?

Inrecentyears,anumberofGenAIprograms,ormodels,havebeendeveloped.AmongthemostimportantGenAImodelsare:

1.generativeadversarialnetworks(GANs)

2.variationalautoencoders(VAEs)

3.decoder-basedlargelanguagemodels(LLMs)

However,notallGenAIpatentscanbeassignedtothesethreespecificcoremodelsbasedonavailableinformationfrompatentabstracts,claimsortitles.

AmongtheseGenAImodels,mostpatentsbelongtoGANs.Between2014and2023,therewere9,700patentfamiliesofthismodeltype,with2,400patentfamiliespublishedin2023alone.

VAEsandLLMsarethesecondandthirdlargestmodelsintermsofpatents,witharound1,800and1,300newpatentfamiliesrespectivelybetween2014and2023.

Intermsofpatentgrowth,GANpatentsshowthestrongestincreaseoverthepastdecade.

However,thishassloweddownrecently.Incontrast,diffusionmodelsandLLMsshowmuch

highergrowthratesoverthelastthreeyears,withthenumberofpatentfamiliesfordiffusionmodelsincreasingfrom18in2020to441in2023andforLLMsincreasingfrom53in2020to881in2023.TheGenAIboomcausedbymodernchatbotssuchasChatGPThasclearlyincreased

researchinterestinLLMs.

WhatarethemaintypesofdatausedinGenAIpatents?

ThemainGenAIdatatypesinclude:

–Image

–Video

–Speech

–Sound

–Music

AmongthedifferentGenAImodes,orthetypeofdatainputandoutput,mostpatentsbelongtotheimage/videocategory.Image/videodataisparticularlyimportantforGANs.Patents

involvingtheprocessingoftextandspeech/sound/musicarekeydatatypesforLLMs.The

remainingmodes:3Dimagemodels,chemicalmolecules/genes/proteinsandcode/software

havefarfewerpatentssofar.AswithpatentsrelatedtoGenAIcoremodels,somepatents

cannotbeclearlyassignedtoaspecificdatatype.Inaddition,somepatentsareassignedto

morethanonemodebecausecertainGenAImodels,suchasmultimodallargelanguagemodels(MLLMs),overcomethelimitationofusingonlyonetypeofdatainputoroutput.

TopapplicationareasofGenAIpatents

ThekeyapplicationareasforGenAIpatentsinclude:

1.Software

2.Lifesciences

3.Documentmanagementandpublishing

4.Businesssolutions

5.Industryandmanufacturing

6.Transportation

7.Security

8.Telecommunications

GenAIisboundtohaveasignificantimpactonmanyindustriesasitfindsitswayintoproducts,servicesandprocesses,becomingatechnologicalenablerforcontentcreationandproductivityimprovement.Forexample,therearemanyGenAIpatentsinlifesciences(5,346patent

familiesbetween2014and2023)anddocumentmanagementandpublishing(4,976).OthernotableapplicationswithGenAIpatentsrangingfromaround2,000toaround5,000overthesameperiodarebusinesssolutions,industryandmanufacturing,transportation,securityandtelecommunications.

Inthelifesciencessector,GenAIcanexpeditedrugdevelopmentbyscreeningand

designingmoleculesfornewdrugformulationsandpersonalizedmedicine.Indocument

managementandpublishing,GenAIcanautomatetasks,savetimeandmoney,andcreatetailoredmarketingmaterials.Inbusinesssolutions,GenAIcanbeusedforcustomerservicechatbots,retailassistancesystems,andemployeeknowledgeretrieval.Inindustryand

manufacturing,GenAIenablesnewfeatureslikeproductdesignoptimizationanddigitaltwinprogramming.Intransportation,GenAIplaysacrucialroleinautonomousdrivingandpublictransportationoptimization.

However,manypatentfamilies(around29,900patentfamiliesbetween2014and2023)cannotbeassignedtoaspecificapplicationbasedonthepatentabstract,claimsortitle.Thesepatentsareinsteadincludedinthecategorysoftware/otherapplications.

Keyfindingsandinsights

11

12

PatentLandscapeReport–GenerativeArtificialIntelligence

Introduction

GenerativeAI–newsystemswithalonghistory

Artificialintelligencetechnologieshaveseenadramaticincreaseinpublicandmediaattentioninrecentyears.However,AIisnotanewfieldofresearch.USandUKscientists–including

theoreticalmathematicianAlanTuring–werealreadyworkingonmachinelearninginthe

1930sand1940s,althoughthetermAIdidnotbecomepopularuntilthe1950s(McCarthyetal.2006).1The1950sand1960ssawasurgeofinterestinmanyAIareasincludingnaturallanguageprocessing,machinelearningandrobotics.Somescientistsatthetimepredictedthatamachineasintelligentasahumanwouldexistwithinageneration(Minsky1967).Thesepredictions

provedtobeoverlyoptimistic.Progressstagnatedbecauseofthelimitationsofcomputingpowerandalgorithmicapproachesavailableatthetime.Asaresult,researchfundingdriedup,whichledtothefirst“AIwinter”inthe1970s.Inthefollowingdecades,periodsofhighAIresearchintensityalternatedwithperiodsofloweractivity.

Foralongtime,AIalgorithmsandsoftwareweredevelopedforspecificpurposes,basedon

clearrulesoflogicandparametersspecifiedbyprogrammers.Evennow,manyAIapplicationsrelyonrule-baseddecisions:ifthis,thenthat.Forexample,virtualassistants(Siri,Alexa,etc.)areessentiallycommand-and-controlsystems.Theyonlyunderstandalimitedlistofquestionsandrequestsandfailtoadapttonewsituations.Theycannotapplytheir“knowledge”tonewproblemsordealwithuncertainty.

AIinthe21stcentury

ThemodernAIboomstartedatthebeginningofthe21stcenturyandhasbeenonanupward

trajectoryeversince.Today,AIandmachinelearningisusedincountlessapplications,includingsearchengines,recommendationsystems,targetedadvertising,virtualassistants,autonomousvehicles,automaticlanguagetranslation,facialrecognitionandmanymore.TheriseofAIhasbeendrivenmainlybythefollowingfactors:

–Morepowerfulcomputers:In1965,GordonMooreobservedthatthenumberoftransistorsoncomputerchipsdoublesapproximatelyeverytwoyearsandpredictedthatthiswould

continueforanother10years(Moore1965).Hislawhasheldtrueformorethanhalfa

century.ThisexponentialgrowthtranslatedintomoreandmorepowerfulAIsystems,oftenwithAI-specificenhancements.

–Bigdata:Second,theavailabilityofdatahasincreasedsimilarlyexponentially.ThishasprovidedapowerfulsourceoftrainingdataforAIalgorithmsandhasmadeitpossibletotrainmodelswithbillionsofimagesorahundredbilliontokens2oftext.

–BetterAI/machinelearningalgorithms:NewmethodsthatallowAIsystemstobetterusedataandalgorithmstolearnthewayhumansdo,suchasdeeplearning,haveenabled

breakthroughsinareassuchasimagerecognitionornaturallanguageprocessing

(WIPO2019).

1Theterm“ArtificialIntelligence”hasbeenvastlyinfluencedbyJohnMcCarthyatDartmouth,whoco-organizedwithMarvinMinskytheDartmouthSummerResearchProjectonArtificialIntelligencein1956.

13

2Tokensarecommonsequencesofcharactersfoundinasetoftext.Tokenizationbreakstextintosmallerpartsforeasiermachineanalysis,helpingAImodelsunderstandhumanlanguage.

PatentLandscapeReport–GenerativeArtificialIntelligence

14

Learningwithexamplesratherthanrules

TheheartofmodernAIismachinelearning,whencomputersystemslearnwithoutbeing

specificallyprogrammedtodoso.ModernAImodelsarefedwithexamplesofinputdata

andthedesiredoutcome,allowingthemtobuildmodelsorprogramsthatcanbeappliedto

entirelynewdata.Machinelearningexcelsathandlingmassivedatasetsanduncoveringhiddenpatternswithinthem.

Apowerfulapproachwithinmachinelearningiscalleddeeplearning.Itleveragescomplex

structurescalledartificialneuralnetworks,looselymodeledafterthehumanbrain.These

networksidentifypatternswithindatasets.Themoredatatheyhaveaccessto,thebettertheylearnandperform.Informationflowsthroughnumerouslayersofinterconnectedneurons,

whereitisprocessedandevaluated.Eachlayerrefinestheinformation,connectingand

weightingitthroughnodes.Essentially,AIlearnsbycontinuouslyreassessingitsknowledge,

formingnewconnectionsandprioritizinginformationbasedonnewdataitencounters.

Thetermdeeplearningreferstothevastnumberoflayersthesenetworkscanutilize.Deep

learning-poweredAIhasachievedremarkableadvancements,especiallyinareaslikeimage

andspeechrecognition.However,itssuccesscomeswithadrawback.Whiletheaccuracyoftheresultsisimpressive,thedecision-makingprocessremainsunclear,eventoAIexperts.Thislackoftransparencyisacontrasttoolderrule-basedsystems.

ModerngenerativeAI(GenAI):thenextlevelofAI

GenerativeAI(GenAI)hasbeenanactiveareaofresearchforalongtime.JosephWeizenbaumdevelopedtheveryfirstchatbot,ELIZA,inthe1960s(Weizenbaum1966).However,GenAIasweknowittodaywasheraldedbytheadventofdeeplearningbasedonneuralnetworks.

Today,GenAIisoneofthemostpowerfulexamplesofmachinelearning.Comparedtoold

rule-basedAIapplicationsthatcouldonlyperformasingletask,modernGenAImodelsare

trainedondatafrommanydifferentareas,withoutanylimitationsintermsoftask.Because

theamountoftrainingdataissolarge–OpenAI’sGPT-3wastrainedonmorethan45terabytesofcompressedtextdata(Brownetal.2020)–themodelsappeartobecreativeinproducing

outputs.Forexample,traditionalchatbotsfollowscriptedresponsesandrelyonpre-definedrulestointeractwithusers,makingthemsuitableonlyforspecifictasks.Incontrast,modernGenAIchatbotssuchasChatGPTorGoogleGeminicangeneratehuman-liketext,allowingforconversationsthatcanadapttomanytopicswithoutbeingconfinedtoapredeterminedscript.Inaddition,thesemodernchatbotscanproducenotonlytext,butalsoimages,musicand

computercodebasedonthedatasetonwhichtheyweretrained.

ThereleaseofChatGPTin2022wasaniPhonemomentforGenAI

InNovember2022,OpenAIreleasedChatGPT(ChatGenerativePre-trainedTransformer)to

thepublic,whichgreatlyincreasedpublicenthusiasmforGenAI.MorethanonemillionpeoplesigneduptouseChatGPTinjustfivedays.A2023surveybyauditingandconsultingfirm

Deloittefoundthatnearly61%ofrespondentsinSwitzerlandwhoworkwithacomputeralreadyuseChatGPTorotherGenAIprogramsintheirdailywork(Deloitte2023).TheChatGPTreleasehasbeendescribedbymany,includingNvidiaCEOJen-HsunHuang,asan“iPhonemoment”forGenAI(VentureBeat2023).ThisispartlybecausetheplatformmadeiteasierforuserstoaccessadvancedGenAImodels,specificallydecoder-basedlargelanguagemodels.3Thesemodels

havedemonstratedthepotentialformanyreal-worldapplicationsandhavesparkedawaveofresearchanddevelopment.ManycompaniesareheavilyinvestinginGenAI,withthesenewermodelsreachinganewdimensionofcapabilities.

3SeenextchapterforanoverviewanddescriptionofthedifferentmodernGenAImodels.

AbrieftimelineofGenAI15

1957•FrankRosenblattintroducestheperceptron,thefundamentalbuildingblockofneuralnetworks

(Rosenblatt1957)

1972

•

Amari-HopfieldNetworksmakerecurrentneuralnetworksabletolearn,asaformofassociativememory(Amari1972,Hopfield1982)

1997

•

LongShortTermMemory(LSTM)recurrentneuralnetworksarepublished,whichwillbecomeoneofthemostsuccessfuldeeplearningarchitecturesinthe2010s(HochreiterandSchmidhuber1997)

1990

•

MarkovnetworksandotherstatisticallanguagemodelsledtoeffectiveAIcommercialsystems,suchasthefirstversionsofGoogleTranslate

2013

•

VariationalAutoencoders(VAEs),anAuto-encoderapproachabletogeneratenewrealisticimagesamplesfrominputimages(KingmaandWelling2013)

2014

•

GenerativeAdversarialNetworksaredescribed,whichwillleadtovariousgenerativeapplicationsaroundphotorealisticimages

2016

•

WaveNetbyDeepMind,anoveldeepneuralnetworkapproachforrealistichumanspeech(vandenOordetal.2016)

2017

•

AteamfromGoogleResearchintroducesthetransformer,thedeepneuralnetworkarchitecturebehindtheLargeLanguageModels(Vaswanietal.2017).

2018

•

GPT,thefirstgenerativelanguagemodelofOpenAI,atransformerof120millionparameters(OpenAI2018)

2