广告调查（第二版）英文 指导手册 Davis-Instructors-Manual-ch20 Post-Production Testing Optimization

AdvertisingResearch:Instructor’sManual

Copyright©2012PearsonEducation,Inc.publishingasPrenticeHall

PAGE

PAGE

AdvertisingResearch:Instructor’sManual

Copyright©2012PearsonEducation,Inc.publishingasPrenticeHall

20.Post-ProductionAdvertisingTestingandOptimization

ChapterGoals

Whenstudentshavecompletedthischaptertheyshouldhaveabetterunderstandingof:

• howresearchcontributestoimprovedadvertisingdecision-makingaftercommunicationsareproducedandaired.

• thesituationsinwhicheachofthefourmaintypesofpost-productionresearcharemostappropriate.

• thestrengthsandlimitationsofeachtypeofpost-productionresearch.

• howcopytestingisusedtodetermineadvertisingdeceptiveness.

NotestotheInstructor

TheChapterLectureprovidesaguidetokeytopicsandcontent.ThePowerPointslidesarenamed:davis_adresearch_ch20.ppt.

Twoonlinesupplementalreadingsprovideexamplesoftestingperformedatbothendsofthepost-productioncontinuum:copytestingandcampaigntrackingevaluationresearch.

MapesandRossNaturalExposureCopyTest

Asdiscussedearlierinthischapter,acopytestmustsatisfyseveralcriteriainordertobeusefultoanadvertiser.Thetestmusthaveasound,validatedmethodologyandappropriatenorms.Additionally,theresultsmustbereportedinawaythatallowstheadvertisertoclearlyseetheimplicationsofthefindingsfordecision-making.TheMapesandRosscopytesttoplinereportprovidedintheonlinesupplementalreadingssatisfiesallthesecriteria.ThereportpresentstheresultsofacopytestoftheGeneralElectric(GE)televisionadvertisement

“GEReveal®-LightBulb.”Notehowtheorganizationofthereportaddresseskeyresearch-

relatedissuesdiscussedinChapter1:therationalefortheresearchisexplicitlystated,keyevaluativemeasuresareidentified,themethodologyisdescribedinsufficientdetail,samplecharacteristicsarenoted,andtheresultsandtheirimplicationsareclearlycommunicated.

Thetoplinereport’sdetailedfindings(reportpages9through13)illustratethecontributionofcopytestingtodecisionsregardingwhetherornottoairaparticularcommercial.

• Persuasion(theprimarymetric)isaddressedonpage9,wheretwotypesofanalysesarepresented.First,thetestcommercial’spersuasionscoreiscomparedtotwoofMapesandRoss’overallnorms,inthiscasethenormforcomparativeproducts(consumerpackagedgoodsmiscellaneousproducts)andtheoverallallproductsnorm.Second,thecommercial’spersuasiveness

amongimportantsubgroupsisnoted.Inallcases,thetestcommercial’spersuasionscoresignificantlyexceededthenorms.

• Aspectsofcommercialrecallareaddressedonpages10through12.ProvenRecallreflectsthepercentageofindividualswhowereabletospontaneouslymentionauniquevisualortextualcomponentofthecommercial.UnaidedBrandNameRecallreflectsthepercentageofindividualswhowereable(withoutprompting)torecalltheGEbrandnameaftercommercialviewing.Notehowinbothcasescomparisonsaremadetooverallnormsforboththetotalsampleandforkeysubgroups.VisualElementRecallprovidesinsightintowhichcommercialcomponentsorideasweremoreattention-gettingormemorable,ashigherscoresindicateagreaterpercentageofindividualsmentioningthespecificaspectofthecommercial.

• Priortocopytestinganadvertisement,itisimportanttoidentifythecommercial’sintendedcommunication.Thisallowsthecopytestingmethodologytodeterminetheextenttowhichthisintendedcommunicationwasactuallycommunicatedtothetargetaudience.TheanalysisofMainIdeaCommunicationispresentedonpage13,wherethereportnotesoverallcommunicationoftheintendedmainideaandthencomparesthisleveltorelevantnorms.

Keepinmindthatwhileeachindividualmeasureisimportant,thefinaldecisiononcommercialairingtypicallyisbasedontheoverallpatternofresponse.Whileallmeasuresinthiscasearequitehigh,imagineacasewherepersuasionishighbutrecallislow,orrecallishighbutpersuasionislow.Itistheoverallpatternofresponse,especiallyresponsesamongkeysubgroupsthatleadsthedecision-makingprocess.

Finally,itisimportanttorememberthatitisthecopytestingcompany’sresponsibilitytoprovideadetailedandobjectiveanalysisnarrowlyfocusedontheresearchresults.Itistheadvertiser’sresponsibilitytobringanindependentpointofviewtothereport,detailingareasinwhichs/heagreesordisagreesandprovidingdeeperbackgroundwithregardtodatainterpretation.GEmight,forexample,placetheinterpretationofthepersuasionscoresinthecontextofitsmarketshare.GEmightnotethat:

GeneralElectrichasthedominantshareofthelightbulbcategory.Asaresult,mostconsumersareeitheralreadypositivetowardGEorareGElightbulbpurchasers.Giventhissituation,thescopetoconvinceindividualsto“switch”toGE,issmall.Understandingthis,itwasthereforeGE’smandatetodevelop

anadthatnotonlyhelpedpromotethenewlightbulb“Reveal”insuchawaythatitwasbothpersuasiveandmemorable,butalsotoconveythemessageofpurecleanlight.Despitethestrongmarketdominance,GE’screativityworkedhardandachievedthisbyexceedingtheapplicableMapesandRossnormsforcommercialpersuasiveness.

NationalTobaccoYouthCampaignEvaluation

TheAustralianDepartmentofHealthandAgeinghasthegoalofhelpingallAustraliansachievebetterhealththroughouttheirlifespan.TheDepartmentachievesthisgoalthroughstrengthenedevidence-basedpolicyadvising,improvedprogrammanagement,research,regulationandpartnershipswithothergovernmentagencies,consumersandstakeholders.Inthiscontext,aspecificfocusisontheencouragementofhealthylifestylesandthepreventionorreductionofunhealthylifestyles.

TheNationalTobaccoYouthCampaign(NTYC)isonespecificwaythattheDepartmentofHealthandAgeingattemptstoencouragehealthylifestyles.TheNTYCwasanational,multimediacampaignthataimedtoreducetheinitiationandprevalenceofsmokingamongyoungAustralians.Thecampaign’sprimarygoalwastoencourageyouthaged12to24yearstorejectsmoking.Thesecondarygoalofthecampaignwastoencourageparentstoquitsmokinginordertodiscouragetheirchildrenfromsmoking.

Trackingevaluationresearchwasusedtodeterminecampaigneffectiveness.Baselinedatawascollectedbeforethecampaignran(November/December,2006)andpost-exposuredatawascollectedafterthecampaignhadrun.(April/May,2007)Bothsurveysmeasuredthetargetaudience’sknowledge,attitudes,intentionsandbehaviorswithrespecttotobaccosmokingaswellastheirawarenessoftheNTYCcampaignandmessages.Thetotalsamplesizewas

nearly3,000.

Thefullcampaignevaluationreportisprovidedintheonlinesupplementalreadings.Theresearchreportedinthereportisasuperiorexampleofwell-designedtrackingresearch.Notehowtheuseofthebaselinesampleallowsonetoclearlyassesscampaignimpactandhowtheareasexploredareexplicitlydefinedanddirectlyrelatedtothecampaign’sgoalsandobjectives.Beyondtheresearchitself,thereportisamodelofexcellenceindatapresentation.Thereportisorganizedtoallowthereadertomovethroughasignificantamountofdatainawaythatfostersunderstanding.Inspiteoftheamountofdatapresented,thereaderisneveroverwhelmedbythedetail.Finally,notehowthedatawascollected.Questionnairepre-testingwasusedtomakecertainthattherewerenoproblemswithquestionnairefloworwordingandthequestionnaireitself(providedinthereport’sappendices)illustratemanyoftheguidelinesdiscussedinChapters12and13.

ChapterLecture

Onceanadisproduced,copytestingresearchisconductedtodetermine

whetherornotthatadvertisementshouldbeplacedinmedia.Copytestingisthereforeevaluativeinorientationandisusedtomake“go-nogo”decisionsaboutaparticularadvertisement.Itismostcommonlyperformedontelevisionadvertising,althoughmagazineandradioadsmayalsobecopytested.

I.CopyTestingTelevisionAdvertising

Almostalwaysconductedbyindependentresearchcompanieswhichvaryacrossfourdimensions:

Slide20-

2

Slide20-

3

Slide20-

4

Slide20-

5

Slide20-

6

• Thenaturalnessoftheviewingsituation.Companiesgenerallytakeoneoftwoapproachestoadvertisingexposure.Someexposerespondentstotestcommercialincontextofactualtelevisionprogrambroadcast.OthersprovideuseaVHStapeor

aDVDthatcontainsaprerecordedtelevisionprograminwhichatestadhasbeeninserted.

• Thenumberofadvertisingexposures.Datacollectioncantakeplaceafteroneormoreexposurestotestadvertising.

• Thetimingofdatacollection.Companiesvaryintermsofamountoftimeelapsedbetweencommercialexposureanddatacollection.Somecollectdataimmediatelyafterexposurewhileothersmaywaitoneormoredays.

• Thetypesofdatacollected.Availablemeasuresvary.Normativemeasurescanincludeoneormoreoffollowing:advertisingrecall,messagecommunication,brandattitudeshift,buyingintention,pre-postbrandpreference,attitudestowardtheadvertising,attitudetowardthemessage.

DifferencesinapproachestotelevisioncopytestingareillustratedinthemethodologiesusedbyGallup&RobinsonandMapesandRoss.

A.Gallup&RobinsonInTeleTest

Gallup&Robinson’stestsadinprerecordedtelevisionprogramdistributedtorespondentsviaVHSorDVD.

Sampletypically150menor150womenaged18andolderchosenfromtengeographicallydispersedmetropolitanareas.Personallyrecruitedattheirhomesunderguiseofstudytomeasuretelevisionviewinghabits.RespondentsgivenVHSorDVD,instructedtowatchitthatday,andtoldafollow-uptelephoneinterviewwilltakeplacefollowingday.Nextdayrespondentscontactedbytelephoneandinterviewedregardingtestad’sintrusiveness,

(recall)persuasion,aswellaspost-exposurebrandratingandcommercialreactions.Measuresareoperationalizedas:

Slide20-

7

Slide20-

8

Slide20-

9

Slide20-

10

• Intrusivenessisdefinedascommercial'sabilitytobreakthroughclutterandcommunicateadvertiserorbrandname.Threelevelsofintrusiveness:(1)UnaidedRecallispercentageofviewerswho,whenpromptedbytheproductorsubjectcategory,claimtohaveseencommercialandcorrectlyidentifysponsororbrandname;(2)ProvedUnaidedRecallisthesubsetofthepriorgroupwhoarealsoabletodescribeaccuratelythecommercial;and(3)ProvedBrandAidedRecallispercentageofviewerswho,aidedbyacompanyorbrandnamecue,canproverecallofthecommercial.

• Persuasionandbrandratingsmeasurecommercial'sabilitytofosterfavorableattitudestowardtheadvertisedproduct.Twolevelsofpersuasion:(1)FavorableBuyingAttitudemeasurespurchaseintentaftersingleexposureandreflectspercentofsamplewhosaidthattheirinterestinpurchasingtheproductincreasedsomewhatorconsiderably,and(2)BrandRatingispercentofsamplewhoratebrandexcellentorverygood.

• Commercialreactionsmeasuretwoaspectsofconsumers'affectiveresponsestocommercial:(1)CommercialLikingmeasuredon

five-pointscale(withendpointsof"Ilikeditverymuch"to"I

dislikeditverymuch"),and(2)CommercialExcellenceusesfour

pointagree-disagreescaletomeasureagreementwiththestatement:"ThiscommercialisoneofthebestI'veseenrecently."

NormsareavailableforallstandardInTeleTestmeasures.Afterthese

questions,respondentsviewtestcommercialagainandareaskeddiagnosticandcustomquestions.

B.MapesandRossNaturalExposure

Exposesrespondentstotestadincontextofanon-airprogram.Recruitsasampleofbetween150and200individualstoviewaspecifictelevisionprogramundertheguiseoftelevisionprogramevaluation.Respondentstoldthattheirreactionstotheprogramarethefocusofresearch.

Methodologyallowsdatatobecollectedacrossrangeofcommercialresponse:persuasion(theprimarymetric),recall,mainideacommunication,anddetaileddiagnosticsthatmeasurehowwellthesalesideasarecommunicatedinalikeablemannerandtheaditselfonvariousattributes.

Measuresareoperationalizedasfollows:

Slide20-

11

Slide20-

12

Slide20-

13

Slide20-

14

• Persuasion(theprincipalmetric)measuresad’sabilitytochangeormaintainpreferenceandinteresttowardstheadvertisedbrand.

• Unaidedrecallmeasuresrespondents’abilitytolinkadvertisedbrandtocommercialfromcategorydescription,forexample,“brandsofautomobiles.”Provenrecallmeasuresrespondents’abilitytorememberanddescribespecificand/oruniqueelementsandideasfromcommercial.

• Ideacommunicationmeasuredbyaskingrespondentstodescribewhattheysawandheardinordertodeterminehowwellvisualelementsattractattention.Alsoaskedaboutwhattheybelieveisthemainideaofthecommercialandtoratetheimportanceofthemainidea.Respondentslastlyaskedtowhatextentcommercialhasalteredtheirinterestinbuyingadvertisedbrand.

Recruitedrespondentsarethosewhoareintheadvertisement’stargetaudience,generallydefinedonbasisofdemographicsandcategory/brand-usage

behaviors.Attimeofrecruitment,MapesandRossobtainsbrandpreferencesinsixproductcategories(includingthecategoryofthetestcommercial)using

open-endedquestioning.Open-ended,nondirectivequestionsinarangeofcategoriesareusedtoreduceanypotentialrespondentsensitizationtothetest

productortestcommercial.Thetestcommercialappearsduringprogramtherespondentsareinstructedtowatch.

Dayfollowingcommercialexposure,eachrespondentiscontactedbytelephoneand,afterconfirmingprogramviewership,post-exposurebrandpreferencesarecollected.Percentageshiftfrompre-topost-exposurelevelsofbrand

preferenceinterpretedascommercial'spersuasiveness.Greatershiftsinpurchaseintentfrompre-topost-exposureindicategreaterlevelsofpersuasiveness.Followingpreferencequestions,recalllevelsareobtainedinresponsetocategoryandbrandprompts.AllrespondentswhoclaimrecallarequestionedtodetermineifthespecificcontentNormsforallkeymeasuresareavailable.

C.EvaluatingAlternativeCopyTestingMethodologies

Whenevaluatingrelativeappropriatenessofalternativecopytestingmethodologies,keepthefollowinginmind:

• Thecloseramethodologyisto"realworld"exposure,themoreaccuratelytheresultswillreflect"realworld"advertisingperformanceandconsumerresponse.

• Respondentsaremorelikelytoprovide"realworld"responsestocommercialswhentheyarenotawarethattheyareparticipatingincopytestingresearch.

• Multiplemeasuresanddiagnosticsprovideamultifacetedlookatcommercialperformance.Therefore,methodologiesthatprovidemultiplemeasuresaregenerallymoreinformativeandusefultoanadvertiserversusthosethatprovideasinglemeasure.

• Amongcopytestingmethodologies,measureswithsamenamemaynothavesamedatacollectionmethodology.Importanttobeawareofmethodologyunderlyingthecollectionofdataforspecificmeasure.Differentmethodologiesmeasurepersuasiondifferently.Preferredmethodologyforanyparticularcopytestisonewhoseapproachtomeasurementbestcorrespondstoadvertiser'sviewofhowcommercialisintendedtoaffecttargetconsumer.

Slide20-

15

Slide20-

16

Finally,preferredcopytestingmethodologiesprovidenormsandevidenceofvalidation.

• Normsallowadvertiserstocomparereactionstotheirtestcommercialtoanobjective,externalcriterion.Importantthatnormsarecurrent,reflect

asampleofadequatesize,andareappropriatetotheproductandproductcategoryshowninthecommercial.

• Validationempiricallydemonstratesacceptablelevelsofreliabilityandvalidity.

II.Split-RunandA/BTests

Approachespermitadvertisertocompareacurrentapproachtoadvertisingtooneormorepotentialalternatives.“Split-run”islabelgiventothistestwhenconductedinprintmediawhile“A/Btest”islabelwhenconductedinanonlineenvironment.

A.Split-RunTests

Printyellowpagesandmagazinesusesplit-runtests.“Split-run”referstopublishersplittingtestadswithintheadvertisingmedium.Yellowpagespublisherwouldconductsplit-runtestbyprintingcurrentadinhalfofcity’sdirectoriesandtestadinremaininghalf.Directorieswouldthenbedistributed

astheynormallywould.Similarly,magazinepublishercoulddivideaparticular

issueofthemagazineintotwogroups,withthetestadbeingprintedinone

groupandthecontroladbeingprintedinthesecondgroup.

Slide20-

17

Slide20-

18

Slide20-

19

Split-runtest,asillustratedinSlide20-17,isvariationofpost-testonlyresearchdesignwheredatafromtworandomlyselectedgroupsiscollectedafteradvertisingexposure.

Twoimportanttaskswithregardtoasplit-runtest.

• “Better”mustbeoperationalized.Advertisermustdeterminecriteriabywhichadswillbecomparedand“better”adidentified.Incaseofyellowpages“better”mightbeoperationalizedasnumberoftelephonecallsgeneratedbyeachad.Inthecaseofmagazinead“better”mightbeoperationalizedasnumberofwebsitevisitsgeneratedbyeachad.

• Oncetheadvertiserhasoperationalized“better,”adsmustbecodedinawaythatallowsaccuratecollectionofrequireddata.Forayellowpagessplit-runtest:

Eachyellowpagesadinasplit-runtestwouldcontainatelephonenumberuniquetothatad,thatis,thetelephonenumberappearingintheadwouldnotbeavailablefromanyothersource.Theuniquetelephonenumberineachadisthenmonitoredbyaspecializedresearchcompanyhiredbythedirectorypublisheroradvertiser.Whenaconsumerplacesacall(usingthespecialnumberintheyellowpagesad)thecallisseamlesslytransferredtotheregularbusinessnumberoftheadvertiser,andishandledintheusualway.Neitherthe

consumercallingin,northepersonatthebusinessansweringthe

call,areawarethatthecallisbeingmeasured.

Foramagazinesplit-runtest:Twomagazineadsmaybothhavethegoalofmotivatingvisitstoadvertiser’swebsite.Similartotheapproachusedintheyellowpagessplit-runtest,auniqueURLwouldbeplacedineachad.VisitorsusingeachURLwouldbecountedandthenimmediatelyforwardedtotheadvertiser’swebsite.

Statisticaltestsareusedtocompareeachad’sperformance.

B.A/BTests

Takessamegeneralapproachasasplit-runtestandisalsoavariationofthepost-testonlyresearchdesign.A/Btestsaremoreflexibleversussplit-runtestsinthatA/Bcantestanycontrolstimulusagainstatestcounterpart,forexample:

• Aproposedbanneradvertisementcanbetestedagainstthecurrentbannerad.

• AproposedwordingchangeinaGoogleAdwordsadcanbetestedagainstthewordingappearinginthecurrentad.

• Thecurrentwebsitelandingpagecanbetestedagainstaproposedrevisedpage.

• Anewemailsubjectlinecanbetestedagainstthesubjectlineusedincurrentemailing.

• Arevised“PurchaseNow”buttoncanbetestedagainstthebuttoncurrentlyused.

• Atwo-stepcheckoutprocedurecanbetestedagainstthecurrentthree-stepprocedure.

Similartosplit-runtests,advertisermustfirstoperationalize“better”andthenmustcreateawaytocollectthedataappropriateforthisoperationalizedmeasure.

Slide20-

20

Slide20-

21

Slide20-

22

Example:testingtwoGoogleAdwordsadswherecurrentad(the“A”ad)usesword“free”withoutmentionofbrandnameswhilesecondad(the“B”ad)doesnotuse“free”butrathermentionsthreebrandnames.Googlerotatestheadssotheyhaveequivalentexposure.Onewaytoevaluate“better”istotrackthepercentageofexposedindividualswhoclick-througheachad.ThisdataisshowninSlide20-21.

Statisticaltestsindicatethatwhilepercentofindividualsclicking-throughisgreaterforthecurrentad,individualsclicking-throughfromthetestadaremuchbettercustomers.Percentageofindividualsmakingapurchaseafterclick-throughandtotalrevenuegeneratedbypurchasersshowsthattestadissignificantly“better”thancontrolad:moreindividualsmakeapurchaseandtheseindividualsmakealargeraveragepurchase.

Advertiserwouldlikelyselectthetestadforfurtheruse.

C.AdditionalA/BTestConsiderations

Beyonddecisionsrelatedtooperationalizationof“better,”advertisersengagedinA/Btestsarefacedwiththreeadditionaldecisions.

1.LengthofTest

LengthofanA/Btestshouldreflecttheamountoftimerequiredforparticipantstotakeactionandamountoftimerequiredforasufficientsamplesizetobeobtained.

Slide20-

23

Slide20-

24

Slide20-

25

Slide20-

2.ExposureRotation

AlwaysariskinA/Btestingthatteststimuluswillperformpoorerthancontrol.Giventhisrisk,exposingequalnumbersofindividualstotestandcontroladsmayhaveaseriousnegativebusinessimpact.Asaresult,advertisersusingA/Btestingtypicallyexposearelativelylowpercentageoftheirtotalaudienceto

testadandthenrunthetestlongerinordertoacquiresufficientsamplesize.

3.CharacteristicsofControlandTestStimuli

Advertiserhastwooptionswithregardtocharacteristicsofteststimuli:

• First,usingcontroladasthefoundation,incrementalchangescanbemadeoneatatimetothetestad.Approachworkswellwhenitisimportanttoidentifytheeffectofasinglechangetothecurrentapproach.Disadvantageisthatittakesmultiple,sequentialA/Bteststodeterminetheimpactofseveralproposedchanges.

• Second,anentirelynewadvertisementcouldbecreatedandtested.Advantageisthatdoesnotrestricttestadtoonlyincrementalmodificationsofcurrentad.Disadvantageisthattheresultsaredifficulttointerpret.Whileresearchmightindicatetestadis“better,”verydifficultifnotimpossibletoisolatespecificreasonswhyadperformedasitdid.

III.FullFactorialDesigns

Appropriatewheneveradvertiserneedstodeterminesimultaneouseffectofrelativelyfewmanipulationswithmultiplelevels.ThesituationshowninSlide

20-27showsoneuseofthefullfactorialdesign.Fullfactorialdesignsappropriatewhen:

• Thetotalnumberofmanipulationsislow;allofthesituationshaveonlytwoorthreemanipulations.

• Thetotalnumberofcellsislessthanten.Sincestimulusmaterialsneedtobecreatedforeachcell,tenisgenerallytheupperpracticallimit.

Advantageofthefullfactorialdesignliesinitsabilitytoisolatetheeffectsof

26

Slide20-

27

Slide20-

28

Slide20-

29

Slide20-

30

eachmanipulationindependentlyaswellasinteractionbetweenfactors.Example:

ImaginethatChevronOilwantstoairanadvertisingcampaignthatdescribesChevron’scommitmenttotheenvironment.ThecampaigngoalistoimproveattitudestowardChevron.Chevronisunsurehowtoapproachthecampaign.Companyhasidentifiedfourpossiblefociforthemessages:thecompany’srecyclingefforts,thecompany’scommitmenttohabitatpreservation,thecompany’sdonationstoNongovernmental(NGO)environmentalorganizations,andthecompany’scommitmenttoelementaryschoolenvironmentaleducation.Inaddition,Chevronisunsureastohowtodescribethiscommitment.Oneoptionistodiscussdonationsintermsofapercentageofsaleswhileasecondoptionnamesaspecificmonetaryamount.

Fullfactorialtestoftheadvertisingoptionsisrecommendedwhereeightdifferentadswouldneedtobecreated.Thisisbecausethefullfactorialdesignhaseightcells(theresultsoffourmessagestrategiesandtwodonationoptions.

ThetableshowninSlide20-27showsalleightcombinationsandthenamesgiventoeachcombination.

Individualsfromarandomlyselectedsamplearerandomlyassignedtotheeightconditionsandshownanadvertisementappropriateforcondition.Afterviewingthead,eachrespondentcompletessurveywhichasksforagreement/disagreementwitheightstatementsdesignedtoelicitattitudestowardChevron.

Keyoutcomemeasureistheaverageofthesescaleswherehighernumbersindicatemorepositivebrandimage.

Firststepinanalysiscalculatesaveragesforeachfactorindependentlyandforeachcombinationoffactors.OutcomeofthesecalculationsareshowninSlide

20-28.

F-valuesandlevelsofsignificancethencomputedforeachfactorandforinteractionasshowninSlide20-29.

Thepriortwochartsareinterpretedasfollows:

• Therewasasignificantinteractionbetweenthetwofactorsaslevelofsignificancefortheinteractionislessthan.05.

• TypeofDonationasamaineffectexertedasignificant,independentinfluenceonattitudestowardChevron.MeanscoresindicatethatattitudestowardChevronweresignificantlymorepositivewhendonationtypewasastraightdonationratherthanapercentageofsales.However,significantinteractiontermindicatesthatitisnecessarytoexaminescoresofindividualadspriortodrawingafinalconclusion.Here,itcanbeseenthatacombinationofhabitatandsetamounthadthemostpositiveimpactonattitudeswhilecombinationofhabitatandsaleshadmostnegativeimpactonattitudes.

• MessagefocusasamaineffectexertednoindependentinfluenceonattitudestowardChevron(aisgreaterthan.05).

FindingsindicatethatthebestapproachforthecampaignwouldbetheHabitat-SetAmountapproach.

IV.MultivariateTesting

Slide20-

31

Slide20-

32

Fullfactorialdesignshavepracticallimits.Asthenumberoffactorsandlevelsincreasessodothemethodologicaldemands.Wouldbenearlyimpossible,forexample,toexecuteafullfactorialdesignforsixfactorswithmultiplelevelsperfactors.Asixfactordesignofform3x3x5x2x5x4wouldrequire

1,800differentstimuliandasamplesizeof180,000withonly100individualspercell.

Multivariatetestingprovidessolutionbyusingonlyasubsetofallpossiblecombinationsinordertoestimatetheeffectsofallcombinations.

A.AnEmailMultivariateTest

Example:ImaginethatAmericanSavingsAssociation(ASA)wantstoimproveefficiencyofitsemailcampaigns,withparticularinterestineffectsofrecipientpersonalizationandpresence/absenceofimageswithintheemail.“Click-through”fromanemailtotheASAwebsitewillbehowefficiencyismeasured.

ASAidentifiedfiveareasofmanipulationfortheemailtestwithtwooptionsforeachlevel,asshowninSlide20-32.Notethatonl