绝对初学者的机器学习

＿

MachineLearningForAbsoluteBeginners

OliverTheobald

SecondEdition

Copyright©2017byOliverTheobald

Allrightsreserved.Nopartofthispublicationmaybereproduced,distributed,ortransmittedinanyformorbyanymeans,includingphotocopying,recording,orotherelectronicormechanicalmethods,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembodiedincriticalreviewsandcertainothernon-commercialusespermittedbycopyrightlaw.

Contents

INTRODUCTION

WHATISMACHINELEARNING?MLCATEGORIES

THEMLTOOLBOXDATASCRUBBING

SETTINGUPYOURDATAREGRESSIONANALYSISCLUSTERING

BIAS&VARIANCE

ARTIFICIALNEURALNETWORKSDECISIONTREES

ENSEMBLEMODELINGBUILDINGAMODELINPYTHONMODELOPTIMIZATIONFURTHERRESOURCESDOWNLOADINGDATASETSFINALWORD

INTRODUCTION

MachineshavecomealongwaysincetheIndustrialRevolution.Theycontinuetofillfactoryfloorsandmanufacturingplants,butnowtheircapabilitiesextendbeyondmanualactivitiestocognitivetasksthat,untilrecently,onlyhumanswerecapableofperforming.Judgingsongcompetitions,drivingautomobiles,andmoppingthefloorwithprofessionalchessplayersarethreeexamplesofthespecificcomplextasksmachinesarenowcapableofsimulating.

Buttheirremarkablefeatstriggerfearamongsomeobservers.Partofthisfearnestlesontheneckofsurvivalistinsecurities,whereitprovokesthedeep-seatedquestionofwhatif?Whatifintelligentmachinesturnonusinastruggleofthefittest?Whatifintelligentmachinesproduceoffspringwithcapabilitiesthathumansneverintendedtoimparttomachines?Whatifthelegendofthesingularityistrue?

Theothernotablefearisthethreattojobsecurity,andifyou’reatruckdriveroranaccountant,thereisavalidreasontobeworried.AccordingtotheBritishBroadcastingCompany’s(BBC)interactiveonlineresourceWillarobottakemyjob?,professionssuchasbarworker(77%),waiter(90%),charteredaccountant(95%),receptionist(96%),andtaxidriver(57%)eachhaveahighchanceofbecomingautomatedbytheyear2035.

[1]

Butresearchonplannedjobautomationandcrystalballgazingwithrespecttothefutureevolutionofmachinesandartificialintelligence(AI)shouldbereadwithapinchofskepticism.AItechnologyismovingfast,butbroadadoptionisstillanuncharteredpathfraughtwithknownandunforeseenchallenges.Delaysandotherobstaclesareinevitable.

NorismachinelearningasimplecaseofflickingaswitchandaskingthemachinetopredicttheoutcomeoftheSuperBowlandserveyouadeliciousmartini.Machinelearningisfarfromwhatyouwouldcallanout-of-the-boxsolution.

Machinesoperatebasedonstatisticalalgorithmsmanagedandoverseenbyskilledindividuals—knownasdatascientistsandmachinelearningengineers.Thisisonelabormarketwherejobopportunitiesaredestinedfor

growthbutwhere,currently,supplyisstrugglingtomeetdemand.IndustryexpertslamentthatoneofthebiggestobstaclesdelayingtheprogressofAIistheinadequatesupplyofprofessionalswiththenecessaryexpertiseandtraining.

AccordingtoCharlesGreen,theDirectorofThoughtLeadershipatBelatrixSoftware:

“It’sahugechallengetofinddatascientists,peoplewithmachinelearningexperience,orpeoplewiththeskillstoanalyzeandusethedata,aswellasthosewhocancreatethealgorithmsrequiredformachinelearning.Secondly,whilethetechnologyisstillemerging,therearemanyongoingdevelopments.It’sclearthatAIisalongwayfromhowwemightimagineit.”

[2]

Perhapsyourownpathtobecominganexpertinthefieldofmachinelearningstartshere,ormaybeabaselineunderstandingissufficienttosatisfyyourcuriosityfornow.Inanycase,let’sproceedwiththeassumptionthatyouarereceptivetotheideaoftrainingtobecomeasuccessfuldatascientistormachinelearningengineer.

Tobuildandprogramintelligentmachines,youmustfirstunderstandclassicalstatistics.Algorithmsderivedfromclassicalstatisticscontributethemetaphoricalbloodcellsandoxygenthatpowermachinelearning.Layeruponlayeroflinearregression,k-nearestneighbors,andrandomforestssurgethroughthemachineanddrivetheircognitiveabilities.Classicalstatisticsisattheheartofmachinelearningandmanyofthesealgorithmsarebasedonthesamestatisticalequationsyoustudiedinhighschool.Indeed,statisticalalgorithmswereconductedonpaperwellbeforemachinesevertookonthetitleofartificialintelligence.

Computerprogrammingisanotherindispensablepartofmachinelearning.Thereisn’taclick-and-dragorWeb2.0solutiontoperformadvancedmachinelearninginthewayonecanconvenientlybuildawebsitenowadayswithWordPressorStrikingly.Programmingskillsarethereforevitaltomanagedataanddesignstatisticalmodelsthatrunonmachines.

Somestudentsofmachinelearningwillhaveyearsofprogrammingexperiencebuthaven’ttouchedclassicalstatisticssincehighschool.Others,perhaps,neverevenattemptedstatisticsintheirhighschoolyears.Butnottoworry,manyofthemachinelearningalgorithmswediscussinthisbookhaveworkingimplementationsinyourprogramminglanguageofchoice;noequationwritingnecessary.Youcanusecodetoexecutetheactualnumber

crunchingforyou.

Ifyouhavenotlearnedtocodebefore,youwillneedtoifyouwishtomakefurtherprogressinthisfield.Butforthepurposeofthiscompactstarter’scourse,thecurriculumcanbecompletedwithoutanybackgroundincomputerprogramming.Thisbookfocusesonthehigh-levelfundamentalsofmachinelearningaswellasthemathematicalandstatisticalunderpinningsofdesigningmachinelearningmodels.

Forthosewhodowishtolookattheprogrammingaspectofmachinelearning,Chapter13walksyouthroughtheentireprocessofsettingupasupervisedlearningmodelusingthepopularprogramminglanguagePython.

WHATISMACHINELEARNING?

In1959,IBMpublishedapaperintheIBMJournalofResearchandDevelopmentwithan,atthetime,obscureandcurioustitle.AuthoredbyIBM’sArthurSamuel,thepaperinvestedtheuseofmachinelearninginthegameofcheckers“toverifythefactthatacomputercanbeprogrammedsothatitwilllearntoplayabettergameofcheckersthancanbeplayedbythepersonwhowrotetheprogram.”

[3]

Althoughitwasnotthefirstpublicationtousetheterm“machinelearning”perse,ArthurSamueliswidelyconsideredasthefirstpersontocoinanddefinemachinelearningintheformwenowknowtoday.Samuel’slandmarkjournalsubmission,SomeStudiesinMachineLearningUsingtheGameofCheckers,isalsoanearlyindicationofhomosapiens’determinationtoimpartourownsystemoflearningtoman-mademachines.

Figure1:Historicalmentionsof“machinelearning”inpublishedbooks.Source:GoogleNgramViewer,2017

ArthurSamuelintroducesmachinelearninginhispaperasasubfieldofcomputersciencethatgivescomputerstheabilitytolearnwithoutbeingexplicitlyprogrammed.

[4]

Almostsixdecadeslater,thisdefinitionremainswidelyaccepted.

AlthoughnotdirectlymentionedinArthurSamuel’sdefinition,akeyfeatureofmachinelearningistheconceptofself-learning.Thisreferstotheapplicationofstatisticalmodelingtodetectpatternsandimprove

performancebasedondataandempiricalinformation;allwithoutdirectprogrammingcommands.ThisiswhatArthurSamueldescribedastheabilitytolearnwithoutbeingexplicitlyprogrammed.Buthedoesn’tinferthatmachinesformulatedecisionswithnoupfrontprogramming.Onthecontrary,machinelearningisheavilydependentoncomputerprogramming.Instead,Samuelobservedthatmachinesdon’trequireadirectinputcommandtoperformasettaskbutratherinputdata.

Figure2:ComparisonofInputCommandvsInputData

Anexampleofaninputcommandistyping“2+2”intoaprogramminglanguagesuchasPythonandhitting“Enter.”

>>>2+2

4

>>>

Thisrepresentsadirectcommandwithadirectanswer.

Inputdata,however,isdifferent.Dataisfedtothemachine,analgorithmisselected,hyperparameters(settings)areconfiguredandadjusted,andthemachineisinstructedtoconductitsanalysis.Themachineproceedstodecipherpatternsfoundinthedatathroughtheprocessoftrialanderror.Themachine’sdatamodel,formedfromanalyzingdatapatterns,canthenbeusedtopredictfuturevalues.

Althoughthereisarelationshipbetweentheprogrammerandthemachine,theyoperatealayerapartincomparisontotraditionalcomputerprogramming.Thisisbecausethemachineisformulatingdecisionsbasedonexperienceandmimickingtheprocessofhuman-baseddecision-making.

Asanexample,let’ssaythatafterexaminingtheYouTubeviewinghabitsofdatascientistsyourmachineidentifiesastrongrelationshipbetweendata

scientistsandcatvideos.Later,yourmachineidentifiespatternsamongthephysicaltraitsofbaseballplayersandtheirlikelihoodofwinningtheseason’sMostValuablePlayer(MVP)award.Inthefirstscenario,themachineanalyzedwhatvideosdatascientistsenjoywatchingonYouTubebasedonuserengagement;measuredinlikes,subscribes,andrepeatviewing.Inthesecondscenario,themachineassessedthephysicalfeaturesofpreviousbaseballMVPsamongvariousotherfeaturessuchasageandeducation.However,inneitherofthesetwoscenarioswasyourmachineexplicitlyprogrammedtoproduceadirectoutcome.Youfedtheinputdataandconfiguredthenominatedalgorithms,butthefinalpredictionwasdeterminedbythemachinethroughself-learninganddatamodeling.

Youcanthinkofbuildingadatamodelassimilartotrainingaguidedog.Throughspecializedtraining,guidedogslearnhowtorespondinvarioussituations.Forexample,thedogwilllearntoheelataredlightortosafelyleaditsmasteraroundobstacles.Ifthedoghasbeenproperlytrained,then,eventually,thetrainerwillnolongerberequired;theguidedogwillbeabletoapplyitstraininginvariousunsupervisedsituations.Similarly,machinelearningmodelscanbetrainedtoformdecisionsbasedonpastexperience.

Asimpleexampleiscreatingamodelthatdetectsspamemailmessages.Themodelistrainedtoblockemailswithsuspicioussubjectlinesandbodytextcontainingthreeormoreflaggedkeywords:dearfriend,free,invoice,PayPal,Viagra,casino,payment,bankruptcy,andwinner.Atthisstage,though,wearenotyetperformingmachinelearning.Ifwerecallthevisualrepresentationofinputcommandvsinputdata,wecanseethatthisprocessconsistsofonlytwosteps:Command>Action.

Machinelearningentailsathree-stepprocess:Data>Model>Action.

Thus,toincorporatemachinelearningintoourspamdetectionsystem,weneedtoswitchout“command”for“data”andadd“model”inordertoproduceanaction(output).Inthisexample,thedatacomprisessampleemailsandthemodelconsistsofstatistical-basedrules.Theparametersofthemodelincludethesamekeywordsfromouroriginalnegativelist.Themodelisthentrainedandtestedagainstthedata.

Oncethedataisfedintothemodel,thereisastrongchancethatassumptionscontainedinthemodelwillleadtosomeinaccuratepredictions.Forexample,undertherulesofthismodel,thefollowingemailsubjectlinewouldautomaticallybeclassifiedasspam:“PayPalhasreceivedyourpaymentforCasinoRoyalepurchasedoneBay.”

AsthisisagenuineemailsentfromaPayPalauto-responder,thespamdetectionsystemisluredintoproducingafalsepositivebasedonthenegativelistofkeywordscontainedinthemodel.Traditionalprogrammingishighlysusceptibletosuchcasesbecausethereisnobuilt-inmechanismtotestassumptionsandmodifytherulesofthemodel.Machinelearning,ontheotherhand,canadaptandmodifyassumptionsthroughitsthree-stepprocessandbyreactingtoerrors.

Training&TestData

Inmachinelearning,dataissplitintotrainingdataandtestdata.Thefirstsplitofdata,i.e.theinitialreserveofdatayouusetodevelopyourmodel,providesthetrainingdata.Inthespamemaildetectionexample,falsepositivessimilartothePayPalauto-responsemightbedetectedfromthetrainingdata.Newrulesormodificationsmustthenbeadded,e.g.,emailnotificationsissuedfromthesendingaddress“

payments@

”shouldbeexcludedfromspamfiltering.

Afteryouhavesuccessfullydevelopedamodelbasedonthetrainingdataandaresatisfiedwithitsaccuracy,youcanthentestthemodelontheremainingdata,knownasthetestdata.Onceyouaresatisfiedwiththeresultsofboththetrainingdataandtestdata,themachinelearningmodelisreadytofilterincomingemailsandgeneratedecisionsonhowtocategorizethoseincomingmessages.

Thedifferencebetweenmachinelearningandtraditionalprogrammingmayseemtrivialatfirst,butitwillbecomeclearasyourunthroughfurtherexamplesandwitnessthespecialpowerofself-learninginmorenuancedsituations.

Thesecondimportantpointtotakeawayfromthischapterishowmachinelearningfitsintothebroaderlandscapeofdatascienceandcomputerscience.Thismeansunderstandinghowmachinelearninginterrelateswithparentfieldsandsisterdisciplines.Thisisimportant,asyouwillencountertheserelatedtermswhensearchingforrelevantstudymaterials—andyouwillhearthemmentionedadnauseaminintroductorymachinelearningcourses.Relevantdisciplinescanalsobedifficulttotellapartatfirstglance,suchas“machinelearning”and“datamining.”

Let’sbeginwithahigh-levelintroduction.Machinelearning,datamining,computerprogramming,andmostrelevantfields(excludingclassical

statistics)derivefirstfromcomputerscience,whichencompasseseverythingrelatedtothedesignanduseofcomputers.Withintheall-encompassingspaceofcomputerscienceisthenextbroadfield:datascience.Narrowerthancomputerscience,datasciencecomprisesmethodsandsystemstoextractknowledgeandinsightsfromdatathroughtheuseofcomputers.

Figure3:ThelineageofmachinelearningrepresentedbyarowofRussianmatryoshkadolls

Poppingoutfromcomputerscienceanddatascienceasthethirdmatryoshkadollisartificialintelligence.Artificialintelligence,orAI,encompassestheabilityofmachinestoperformintelligentandcognitivetasks.ComparabletothewaytheIndustrialRevolutiongavebirthtoaneraofmachinesthatcouldsimulatephysicaltasks,AIisdrivingthedevelopmentofmachinescapableofsimulatingcognitiveabilities.

Whilestillbroadbutdramaticallymorehonedthancomputerscienceanddatascience,AIcontainsnumeroussubfieldsthatarepopulartoday.Thesesubfieldsincludesearchandplanning,reasoningandknowledgerepresentation,perception,naturallanguageprocessing(NLP),andofcourse,machinelearning.MachinelearningbleedsintootherfieldsofAI,includingNLPandperceptionthroughtheshareduseofself-learningalgorithms.

Figure4:Visualrepresentationoftherelationshipbetweendata-relatedfields

ForstudentswithaninterestinAI,machinelearningprovidesanexcellentstartingpointinthatitoffersamorenarrowandpracticallensofstudycomparedtotheconceptualambiguityofAI.Algorithmsfoundinmachinelearningcanalsobeappliedacrossotherdisciplines,includingperceptionandnaturallanguageprocessing.Inaddition,aMaster’sdegreeisadequatetodevelopacertainlevelofexpertiseinmachinelearning,butyoumayneedaPhDtomakeanytrueprogressinAI.

Asmentioned,machinelearningalsooverlapswithdatamining—asisterdisciplinethatfocusesondiscoveringandunearthingpatternsinlargedatasets.Popularalgorithms,suchask-meansclustering,associationanalysis,andregressionanalysis,areappliedinbothdataminingandmachinelearningtoanalyzedata.Butwheremachinelearningfocusesontheincrementalprocessofself-learninganddatamodelingtoformpredictionsaboutthefuture,dataminingnarrowsinoncleaninglargedatasetstogleanvaluableinsightfromthepast.

Thedifferencebetweendataminingandmachinelearningcanbeexplainedthroughananalogyoftwoteamsofarchaeologists.Thefirstteamismadeupofarchaeologistswhofocustheireffortsonremovingdebristhatliesinthewayofvaluableitems,hidingthemfromdirectsight.Theirprimarygoalsaretoexcavatethearea,findnewvaluablediscoveries,andthenpackuptheirequipmentandmoveon.Adaylater,theywillflytoanotherexoticdestinationtostartanewprojectwithnorelationshiptothesitethey

excavatedthedaybefore.

Thesecondteamisalsointhebusinessofexcavatinghistoricalsites,butthesearchaeologistsuseadifferentmethodology.Theydeliberatelyreframefromexcavatingthemainpitforseveralweeks.Inthattime,theyvisitotherrelevantarchaeologicalsitesintheareaandexaminehoweachsitewasexcavated.Afterreturningtothesiteoftheirownproject,theyapplythisknowledgetoexcavatesmallerpitssurroundingthemainpit.

Thearchaeologiststhenanalyzetheresults.Afterreflectingontheirexperienceexcavatingonepit,theyoptimizetheireffortstoexcavatethenext.Thisincludespredictingtheamountoftimeittakestoexcavateapit,understandingvarianceandpatternsfoundinthelocalterrainanddevelopingnewstrategiestoreduceerrorandimprovetheaccuracyoftheirwork.Fromthisexperience,theyareabletooptimizetheirapproachtoformastrategicmodeltoexcavatethemainpit.

Ifitisnotalreadyclear,thefirstteamsubscribestodataminingandthesecondteamtomachinelearning.Atamicro-level,bothdataminingandmachinelearningappearsimilar,andtheydousemanyofthesametools.Bothteamsmakealivingexcavatinghistoricalsitestodiscovervaluableitems.Butinpractice,theirmethodologyisdifferent.Themachinelearningteamfocusesondividingtheirdatasetintotrainingdataandtestdatatocreateamodel,andimprovingfuturepredictionsbasedonpreviousexperience.Meanwhile,thedataminingteamconcentratesonexcavatingthetargetareaaseffectivelyaspossible—withouttheuseofaself-learningmodel—beforemovingontothenextcleanupjob.

MLCATEGORIES

Machinelearningincorporatesseveralhundredstatistical-basedalgorithmsandchoosingtherightalgorithmorcombinationofalgorithmsforthejobisaconstantchallengeforanyoneworkinginthisfield.Butbeforeweexaminespecificalgorithms,itisimportanttounderstandthethreeoverarchingcategoriesofmachinelearning.Thesethreecategoriesaresupervised,unsupervised,andreinforcement.

SupervisedLearning

Asthefirstbranchofmachinelearning,supervisedlearningconcentratesonlearningpatternsthroughconnectingtherelationshipbetweenvariablesandknownoutcomesandworkingwithlabeleddatasets.

Supervisedlearningworksbyfeedingthemachinesampledatawithvariousfeatures(representedas“X”)andthecorrectvalueoutputofthedata(representedas“y”).Thefactthattheoutputandfeaturevaluesareknownqualifiesthedatasetas“labeled.”Thealgorithmthendecipherspatternsthatexistinthedataandcreatesamodelthatcanreproducethesameunderlyingruleswithnewdata.

Forinstance,topredictthemarketrateforthepurchaseofausedcar,asupervisedalgorithmcanformulatepredictionsbyanalyzingtherelationshipbetweencarattributes(includingtheyearofmake,carbrand,mileage,etc.)andthesellingpriceofothercarssoldbasedonhistoricaldata.Giventhatthesupervisedalgorithmknowsthefinalpriceofothercardssold,itcanthenworkbackwardtodeterminetherelationshipbetweenthecharacteristicsofthecaranditsvalue.

Figure1:Carvaluepredictionmodel

Afterthemachinedecipherstherulesandpatternsofthedata,itcreateswhatisknownasamodel:analgorithmicequationforproducinganoutcomewithnewdatabasedontherulesderivedfromthetrainingdata.Oncethemodelisprepared,itcanbeappliedtonewdataandtestedforaccuracy.Afterthemodelhaspassedboththetrainingandtestdatastages,itisreadytobeappliedandusedintherealworld.

InChapter13,wewillcreateamodelforpredictinghousevalueswhereyistheactualhousepriceandXarethevariablesthatimpacty,suchaslandsize,location,andthenumberofrooms.Throughsupervisedlearning,wewillcreatearuletopredicty(housevalue)basedonthegivenvaluesofvariousvariables(X).

Examplesofsupervisedlearningalgorithmsincluderegressionanalysis,decisiontrees,k-nearestneighbors,neuralnetworks,andsupportvectormachines.Eachofthesetechniqueswillbeintroducedlaterinthebook.

UnsupervisedLearning

Inthecaseofunsupervisedlearning,notallvariablesanddatapatternsareclassified.Instead,themachinemustuncoverhiddenpatternsandcreatelabelsthroughtheuseofunsupervisedlearningalgorithms.Thek-meansclusteringalgorithmisapopularexampleofunsupervisedlearning.ThissimplealgorithmgroupsdatapointsthatarefoundtopossesssimilarfeaturesasshowninFigure1.

Figure1:Exampleofk-meansclustering,apopularunsupervisedlearningtechnique

IfyougroupdatapointsbasedonthepurchasingbehaviorofSME(SmallandMedium-sizedEnterprises)andlargeenterprisecustomers,forexample,youarelikelytoseetwoclustersemerge.ThisisbecauseSMEsandlargeenterprisestendtohavedisparatebuyinghabits.Whenitcomestopurchasingcloudinfrastructure,forinstance,basiccloudhostingresourcesandaContentDeliveryNetwork(CDN)mayprovesufficientformostSMEcustomers.Largeenterprisecustomers,though,aremorelikelytopurchaseawiderarrayofcloudproductsandentiresolutionsthatincludeadvancedsecurityandnetworkingproductslikeWAF(WebApplicationFirewall),adedicatedprivateconnection,andVPC(VirtualPrivateCloud).Byanalyzingcustomerpurchasinghabits,unsupervisedlearningiscapableofidentifyingthesetwogroupsofcustomerswithoutspecificlabelsthatclassifythecompanyassmall,mediumorlarge.

Theadvantageofunsupervisedlearningisitenablesyoutodiscoverpatternsinthedatathatyouwereunawareexisted—suchasthepresenceoftwomajorcustomertypes.Clusteringtechniquessuchask-meansclusteringcanalsoprovidethespringboardforconductingfurtheranalysisafterdiscretegroupshavebeendiscovered.

Inindustry,unsupervisedlearningisparticularlypowerfulinfrauddetection

—wherethemostdangerousattacksareoftenthoseyettobeclassified.Onereal-worldexampleisDataVisor,whoessentiallybuilttheirbusinessmodelbasedonunsupervisedlearning.

Foundedin2013inCalifornia,DataVisorprotectscustomersfromfraudulent

onlineactivities,includingspam,fakereviews,fakeappinstalls,andfraudulenttransactions.Whereastraditionalfraudprotectionservicesdrawonsupervisedlearningmodelsandruleengines,DataVisorusesunsupervisedlearningwhichenablesthemtodetectunclassifiedcategoriesofattacksintheirearlystages.

Ontheirwebsite,DataVisorexplainsthat"todetectattacks,existingsolutionsrelyonhumanexperiencetocreaterulesorlabeledtrainingdatatotunemodels.Thismeanstheyareunabletodetectnewattacksthathaven’talreadybeenidentifiedbyhumansorlabeledintrainingdata."

[5]

Thismeansthattraditionalsolutionsanalyzethechainofactivityforaparticularattackandthencreaterulestopredictarepeatattack.Underthisscenario,thedependentvariable(y)istheeventofanattackandtheindependentvariables(X)arethecommonpredictorvariablesofanattack.Examplesofindependentvariablescouldbe:

Asuddenlargeorderfromanunknownuser.I.E.establishedcustomersgenerallyspendlessthan$100perorder,butanewuserspends$8,000inoneorderimmediatelyuponregisteringtheiraccount.

Asuddensurgeofuserratings.I.E.AsatypicalauthorandbookselleronA,it’suncommonformyfirstpublishedworktoreceivemorethanonebookreviewwithinthespaceofonetotwodays.Ingeneral,approximately1in200Amazonreadersleaveabookreviewandmostbooksgoweeksormonthswithoutareview.However,Icommonlyseecompetitorsinthiscategory(datascience)attracting20-50reviewsinoneday!(Unsurprisingly,IalsoseeAmazonremovingthesesuspiciousreviewsweeksormonthslater.)

Identicalorsimilaruserreviewsfromdifferentusers.FollowingthesameAmazonanalogy,Ioftenseeuserreviewsofmybookappearonotherbooksseveralmonthslater(sometimeswithareferencetomynameastheauthorstillincludedinthereview!).Again,Amazoneventuallyremovesthesefakereviewsandsuspendstheseaccountsforbreakingtheirtermsofservice.

Suspiciousshippingaddress.I.E.Forsmallbusinessesthatroutinelyshipproductstolocalcustomers,anorderfromadistantlocation(wheretheydon'tadvertisetheirproducts)caninrarecasesbeanindicatoroffraudulentormaliciousactivity.

Standaloneactivitiessuchasasuddenlargeorderoradistantshippingaddressmayprovetoolittleinformationtopredictsophisticated

cybercriminalactivityandmorelikelytoleadtomanyfalsepositives.Butamodelthatmonitorscombinationsofindependentvariables,suchasasuddenlargepurchaseorderfromtheothersideoftheglobeoralandslideofbookreviewsthatreuseexistingcontentwillgenerallyleadtomoreaccuratepredictions.Asupervisedlearning-basedmodelcoulddeconstructandclassifywhatthesecommonindependentvariablesareanddesignadetectionsystemtoidentifyandpreventrepeatoffenses.

Sophisticatedcybercriminals,though,learntoevadeclassification-basedruleenginesbymodifyingtheirtactics.Inaddition,leadinguptoanattack,attackersoftenregisterandoperatesingleormultipleaccountsandincubatetheseaccountswithactivitiesthatmimiclegitimateusers.Theythenutilizetheirestablishedaccounthistorytoevadedetectionsystems,whicharetrigger-heavyagainstrecentlyregisteredaccounts.Supervisedlearning-basedsolutionsstruggletodetectsleepercellsuntiltheactualdamagehasbeenmadeandespeciallywithregardtonewcategoriesofattacks.

DataVisorandotheranti-fraudsolutionprovidersthereforeleverageunsupervisedlearningtoaddressthelimitationsofsupervisedlearningbyanalyzingpatternsacrosshundredsofmillionsofaccountsandidentifyi