IBM SPSS Modeler 14.2 for Association Analysis 用于关联分析的 IBM SPSS Modeler 14.2

PAGE

LastUpdated

DATE\@"M/d/yyyyh:mm:ssam/pm"

12/8/201112:57:58PM

Page

LastUpdated

DATE\@"M/d/yyyyh:mm:ssam/pm"

12/8/201112:57:58PM

Page

DataMiningwithIBMSPSSModeler14.2

UniversityofArkansas

DavidDouglas

AssociationAnalysis

NotesonAssociationAnalysisusingIBMSPSSModeler14.2

AssociationRulesUsingClementine

IBMSPSSModeler14.2hasthreedifferentalgorithmsforgeneratingassociationrules.Inputdataformatcanbeeithertabularortransactional.Themodelsare:

Apriori–alldatamustbecategorical

Carma–categoricalconsequentsbutcanhavenumericinputs

Sequential–sequentialassociationrules

Apriorialsoproducesassociationrulesinaveryefficientmanner.Italsohastheadvantageofhavingoptionsthatprovidechoicesinthecriterionmeasurementsusedtoguidedetectingtherules.However,ithasamajordisadvantageinthatonlycategorical(symbolic)fieldsareallowedasinputs.

Carma,unlikeGRIandApriori,offersoptionsforruledetectionthatincludessupportforboththeantecedentandtheconsequence;plusithandlesdataintransactionformat.Additionally,itallowsruleswithmultipleconsequents,oroutcomeandisnotlimitedtocategoricaldata.

Sequentialassociationanalysistakesintoaccountthesequenceofevents.Itworkswitheithertransactionortabledata.

NotesonDataFormatsforAssociationAnalysis

Marketbasketisanaturalforassociationanalysisandtherearetwogeneralformatsofdatarepresentationformarketbasketanalysis.Thefirstissometimesreferredtoasthetransactionaldataformatandthesecondisthetabulardataformat.Thetransactionaldataformatrequiresonlytwofields—anidfieldandacontentfield.Forexample(ignorequantitiespurchasedfornow),

TransactionID Items

Broccoli

1 GreenPeppers

1 Corn

2 Asparagus

2 Squash

2 Corn

3 Corn

3 Tomatoes

… …

Noteinthiscasethatasingletransactionrequiresseveralrecords.SASEM6.1requiresthisformat,unlessyouhaveitsdatawarehousingsoftware—whichwedonothave.

Inthetabulardataformat,eachrecordisatransaction(alsoignoringquantitiespurchasedfornow)andaflag(0/1orT/F)torepresentapurchaseornot.Forexample,

TransId

Asparagus

Beans

Broccoli

Corn

GreenPeppers

Squash

Tomatoes

1

0

0

1

1

1

0

0

2

1

0

0

1

0

1

0

3

0

1

0

1

0

1

1

…

…

…

n

1

1

0

0

1

0

1

Notethatthisdataformatcanbecomeverycumbersomeforalargenumberofproductsandalargenumberoftransactions—andwilltypicallybeaverysparsematrix.Thus,twoapproachesforminingalargenumberoftransactionswithalargenumberofproductsaregenerallytaken.

SQLwillbeusedtocreateatransactiondataformatfilethatwillbeusedforthemarketbasketassociationanalysis

Asoftwareproductwillbeusedthatdoesin-databasedatamining

IBMSPSSModeler14.2doesin-databaseminingviaODBCandwithdatabasevendors’products.Forexample,IBMSPSSModeler14.2canbeusedforin-databasedataminingforDB2andlikewiseworkswithSQLServerandOracle.

IBMSPSSModeler14.2forAssociationAnalysis

Aprori&CarmaModelinTabularFormat

TheAprorimodelwillbeillustratedfirst.PlaceanExcelnodeonthemodelcanvasasshownabove,opentheeditwindowandimporttheBaskets1n.xlsfile.ClicktheTypestabandclicktheReadValuesbutton.AddaTypenodeandconnecttheExcelnodetotheTypenode.RemembertoclicktheReadValuesbuttonontheTypetaboftheExcelnode.EdittheTypenodeanddothefollowing:

SettheCardIdvariableDirectiontoNone

SetalltheContinuousvariables’Directiontoinput

Setallthefoodcategorical(Flag)variables’toBoth

Seethesettingsbelow.

OpentheAproriBasketsnode—shownbelow.RecallthattheAprorimodelrequiresallvariablesbecategorical.

ThisexampleprovidesacustomnamesetintheAnnotationstab.IntheFieldstab,setthecategoricalvariablesthatarepossiblefortheConsequents.Notethatpmethod,sex,homeown,incomeandagewouldnotbeaconsequentbutcouldbeanantecedent.

ClicktheModeltab.Anumberofoptionsareavailableforthemodelertoadjustasappropriate.FirstistheMinimumantecedentsupport.IncreasingthisvaluewillresultinfewerrulesaswillincreasingtheMinimumruleconfidence.Inthiscase,theMaximumnumberofantecedentshasbeensetto5.ExecutetheApririBasketsnode.Double-clickthemodelnuggettoreviewtheresults.

FirstnotethattheConsequentandAntecedentcolumns.Thefirstrulesaysthatmalesthebuybeerandfrozenmealsalsobuycannedvegetables95.27%ofthetime.Supportforthisruleisalsodisplayed--14.8.NotethattherulesaresortedbytheConfidencecolumn—youmaywishtosortonadifferentcolumn.Also,theShow/HideCriteriaMenuallowsmoredatatobeshown.ThebottomtwomenuoptionsareShowallandHideall.SelecttheShowalloption.

Liftmeansthesamehereasinothermodels.ClicktheGeneratemenuoptionandselecttheRuleSet.

ThisexamplehasbeengiventheRulesetnameofAproriFrozenMeal,theTargetfieldissettofrozenmealandtheDefaultvaluehasbeensetto0.ClicktheOKbuttonandthegeneratedrulesetnodewillbeplacedontheupperleftofthestreamcanvas.DragthegeneratedmodeltherightoftheTypenodeandconnecttheTypenodetoit.

Openthegeneratedrulesetnode.Recallthatthissetofruleshasatargetfieldorvariableoffrozenmeal.Forthistarget,thereare8rules—rule1indicatesthatifonebuysbeer,thentheywillalsobuyfrozenmeals58%ofthetime.Locaterule8whichhasa94%probability.Malesthatbuybeerandcannedvegetableshavea94%probabilityofbuyingfrozenmeals.

Forconvenience,addaFilternodetotherightofthegeneratednodeandconnectthegeneratednodetothefilternode.Thepurposeofthefilternodeistoeliminatethefieldsnotusedinthe8generatedrulesThevariablesnotusedinthe8rulesare:value,pmethod,income,age,fruitveg,freshmeat,diary,canndemeat,wine,softdrin,fish,confectionery.Notethattwonewvariableshavebeencreatedatthebottomofthevariablelist--$A-11fieldsand$AC-11fields.Donotfilteroutthesetwovariablesorcardidascardidwillhelpfindrecordsinthedata.ConnecttheFilternodetoaTablenodeandexecutetheTablenode.

Asshownbelow,eachrecordinthedatasethasbeensettoTiftheconfidenceofafrozenmealinoneoftherulesisgreaterthan50%.TheT/Fisinthe($A-11fieldscolumn)andtheconfidenceisinthe$AC-11fieldscolumn.Forexample,row3(cardid=10872)hasaconfidence0.747.Canyougobacktotherulesetgeneratednodeandfindtherulethatmadethistrue?

CarmaModel

FortheCarmamodel,allinputsareconsideredtohavearoleofboth.Thus,onlythepurchasableitemsshouldbeincludedinthemodelastheinputsoftheFieldstab.OpentheCarmanode,clicktheFieldstab,selecttheUsecustomsettingsoptionandselecttheinputsforthemodel.

.

AcceptallotherdefaultsandruntheCarmanode.

DoubleclicktheCarmanuggettoviewtheresults.TheformatoftheoutputisidenticaltothatoftheApriorimodelsothedatapresentedthereneednotbeexplainedagain.AswiththeGRImodel,generateaRuleSet(usethesametargetvalueoffrozenmeal),dragittotherightoftheTypenodeandconnecttheTypenodetoit.Openthegeneratedrulesetnodeandnotethatthenumberofrules,three,islessthanintheApriorimodel.Therulewiththehighestconfidenceisthosethatbuybeerandcannedvegetableshavea87.4%probabilityofalsobuyingfrozenmeals.

FinishoutthestreambyaddingtheFilterandTablenodes.Remembertofilteroutallthevariablesnotusedintherulestomakeiteasiertoreadthetable.Aportionofthetablenodeisshownbelow.Record10872hasTthesameastheApriorimodelbutwithlessconfidence;.675inthiscase.

Basket1nData

Basketsummary:

cardid.Loyaltycardidentifierforcustomerpurchasingthisbasket.

value.Totalpurchasepriceofbasket.

pmethod.Methodofpaymentforbasket.

Personaldetailsofcardholder:

sex

homeown.Whetherornotcardholderisahomeowner.

income

age

Basketcontents—flagsforpresenceofproductcategories:

fruitveg

freshmeat

dairy

cannedveg

cannedmeat

frozenmeal

beer

wine

softdrink

fish

confectionery

AtransactionfileformatwillbeusedtoillustrateIBMSPSSModeler14.2–Apriori,CarmaandSequence.Thefile,GroceryTrans1-Time.xlscontainstransactiondatawithasequencecolumn,Time,asshownbelow.NoticethattheProductvariableRolehasbeensettoBoth.

RecallthatonlytheSequencemodelmakesutilizationofTimesoitwillnotbeusedintheAprioriandCarmaanalysis.CreateaIBMSPSSModeler14.2streamasshownbelow.

BecausetheTimefieldcannotbeusedbytheCarmaorApriorimodel,aTypenodeisusedtoremovethisfieldfromtheanalysis.Todothis,ensurethattheRolefortheTimevariableissettoNone.

Then,opentheCarmanodeandclicktheFieldstab.ThiswillrequiresettingtheIDvariableaswellastheContentvariable—thehowtodothismaynotobviousuntilyouclicktheUseCustomSettingsoption.ClicktheUsecustomsettingsoptionandthenclicktheUsetransactionformatcheckbox.

AfterclickingtheUsetransactionformatcheckbox,theCarmanodeproducesthedropdownboxestoallowtheusertoentertheIDfieldandtheContentfield.Inthisexample,theID