广告调查（第二版）英文 指导手册 Davis-Instructors-Manual-ch11 Measurement

AdvertisingResearch:Instructor’sManual

Copyright©2012PearsonEducation,Inc.publishingasPrenticeHall

PAGE

PAGE

AdvertisingResearch:Instructor’sManual

Copyright©2012PearsonEducation,Inc.publishingasPrenticeHall

11.Measurement

ChapterGoals

Afterreadingthischapter,studentsshouldbeableto:

• describethemeasurementprocess.

• explainthefourdifferentlevelsofmeasurement.

• determineameasure'sreliabilityandvalidity.

NotestotheInstructor

TheChapterLectureprovidesaguidetokeytopicsandcontent.ThePowerPointslidesarenamed:davis_adresearch_ch11.ppt.

ChapterLecture

Slide11-

2

Slide11-

3

Slide11-

4

Measurement:Thewaybywhichabstract,typicallyunobservableconceptsarelinkedtoobservableevents.Asystematicapproachofmovingfromaconcepttoaneventincreaseslikelihoodthatconclusionsdrawnfromtheresearchareappropriate,reliable,andvalid.

Thereareseveralstepsinmeasurementprocess.Canbegroupedintothreemajortasks:

• Identifyanddefinetheconceptofinterest

• Specifyanobservableevent

• Evaluateandrevisetheobservableevent

I.IdentifyandDefinetheConceptofInterest

Entailsthreesequentialsteps.Beginswithageneralnotionoftheareatobe

exploredandendswithadetailed,explicitoperationaldefinitionofwhatistobemeasured.

A.IdentifyConceptofInterest

Slide11-

5

Slide11-

6

Slide11-

7

Slide11-

8

Slide11-

9

Slide11-

Themeasurementprocessbeginsbyidentifyingtheconcept(s)ofinterestforstudyandexploration.

• Aconceptisaninventednameforapropertyofanobject,person,state,orevent.

Someconcepts(i.e.,age,gender,andincome)presentfewmeasurementproblemsbecausetheyhavewell-definedmeanings.

Otherconceptsexploredinadvertisingresearch,suchasadvertisingawareness,brandloyalty,andwebsiteengagementarelessconcreteandpresentgreatermeasurementchallenges.Here,itisimportanttoclearlydefinetheconceptthroughcreationofconceptualandoperationaldefinitions.

B.DevelopConceptualDefinition

Conceptualdefinitionexpressestheconcept’scentralorcoreidea.Itclearlystatestheconcept’smajorcharacteristicsanddistinguishesthetargetconceptfromsimilarbutdifferentconcepts.

Conceptsmayhavemorethanoneconceptualdefinition.Conceptualdefinitionsforbrandloyaltymightbe:

• aconsumer'scommitmenttorepurchasethebrandacrossrepeatedbuyingoccasions.

• aconsumer’spredispositiontoconsistentlyspeakpositivelyaboutthebrandtofriendsandothers.

• adispositiontowardabrandthatissofavorablethatitcreatesa

“barriertoexit,”makingitdifficultforotherbrandstocompete.

• greaterconsumptionofaparticularbrandversusconsumers’

averagelevelofconsumption.

Consensusistypicallyusedtoselectfromamongmultipleoptions.

C.DevelopOperationalDefinition

Operationaldefinitiontranslatesconceptualdefinitionintooneormore

10

Slide11-

11

Slide11-

12

Slide11-

13

Slide11-

14

Slide11-

15

observableeventsbyexplicitlydescribingtheconcept’smostimportantobservable,definingcharacteristics.

Whiletherecanbesomedebateamongresearchersregardingaconcept’sconceptualdefinition,therecanbegreatdiversityofopinionastoaconcept’soperationaldefinition.

Example1:Canstartwithconceptualdefinitionofaffectionas“atenderfeelingoremotionforanother.”Imaginethatfourpeople-Mary,Pete,JohnandMeg-allagreewiththisconceptualdefinition.

However,intermsofoperationalization:

• Maryfeelsthatherboyfriend,Pete,hasaffectionforherbecausehisobservableandmeasurablebehaviorsmatchheroperationaldefinition.

• JohnfeelsthatMegisnotaffectionatebecauseherobservableandmeasurablebehaviorsdonotmatchhisoperationaldefinition.Thereverseisalsotrue.Thereisaproblembecausetheiroperationaldefinitionsdonotmatch.

Example2:Advertisingresearchersstartingwiththesameconceptualdefinitioncanalsodivergeastheyoperationallydefinetheconcept.Slide11-

13showshowfourdifferentresearchersmightoperationalize“attitudetowardthead.”

Theprocessofmovingfromtheconceptualtooperationaldefinitioninvolvesseveralsteps:

1.Explicitlyspecifytheconceptofinterest.

2.Exploredifferentaspectsoftheconcept’smeaning.Inthepriorexample,theconceptualdefinitionwas:"apredispositiontorespondinafavorableorunfavorablemannertoaparticularadvertisingstimulusduringaparticularexposuresituation."Themostimportantportionofthisconceptualdefinitionisthespecificationofresponseas"favorable"or"unfavorable."

3.Explicitlyspecifywhatcanbeobserved.Inthepriorexample,theresearchermustspecifyhowfavorableorunfavorablepredispositionscanbeoperationalizedandmademoreconcrete.Notethatnotalloperationaldefinitionsforthesameconceptneedeventobemeasuredinthesameway:thefirstthreeapproachesusedsurveyquestionswhilethefourthapproachused

physiologicalmeasures.

Slide11-

16

Slide11-

17

4.Evaluateandselectoneormoreofthealternatives.Selectedalternativeoralternativesbecomeanoperationaldefinition.Inthepriorexample,thefirstoperationaldefinitionspecifiedthatfavorableandunfavorablepredispositionstowardanadvertisementcanbeobservedbyaskingtherespondentabout

fivedimensionsofthead:good/bad,interesting/boring,

creative/uncreative,liked/disliked,andinformative/uninformative.

1.ImpactofOperationalDefinitiononDataCollection

Theoperationaldefinitionimplicitlydefinesthetypeofdatathatwillneedtobecollected.

Considerthisconceptualdefinitionfor“websiteengagement”:“theextenttowhichindividualsrespondfavorablytositecontent.”Thisconceptualdefinitioncanleadtodifferentoperationaldefinitions,forexample:

Engagementisreflectedintheamountoftimeanddepthofinteractionexhibitedduringawebsitevisit.Higherlevelsofengagementarereflectedinmoreoveralltimespentonthesite,click-throughtodeepersitepages,andgreateroverallpageviews.

Engagementisreflectedinconsumerattitudesaftersitevisit.Higherlevelsofengagementarereflectedinmorepositiveresponsestosurveyquestionsthatmeasurepersonalreactionstositecontent(i.e.,interest,relevanceandappropriateness),feelingstowardthesite(i.e.,enjoymentandlikeability),and

self-reportingofsitebehaviors(i.e.,relativeamountoftimespentonsiteversuscompetitivesites).

Engagementisreflectedinheightenedphysiologicalresponsetoawebsite.Thisentailsincreasedheartrateandincreased

amountofbetawaveactivity.

Allthreeoperationaldefinitionsof“engagement”arereasonable,giventheconceptualdefinition.Each,however,leadstoadifferentformofdatacollection.Thefirstrequirescollectionofactualrespondentbehaviors.Thesecondrequirescollectionofdataviaasurveyorothertypeofquestionnaireandthethirdrequirescollectionofphysiologicalmeasures.

II.SpecifyanObservableEvent

Secondstageofthemeasurementprocesscreatestheobservableevents.Aresearcherdetermineswhatspecifictypesofquestionsorotherdatagatheringsourceswhichwillbeusedtocollecttheinformationspecifiedintheoperationaldefinition.Thisinvolves:(1)identifyingappropriatelevelofmeasurementand(2)selectingquestiontype.

Slide11-

18

Slide11-

19

Slide11-

20

Slide11-

21

Slide11-

22

Slide11-

23

Slide11-

24

A.IdentifytheAppropriateLevelofMeasurement

Therearefourlevelsofmeasurement:nominal,ordinal,interval,andratio.Slide11-18showstheusesofeachlevel.Notehoweachhigherlevelofmeasurementincludesalltheinformationprovidedbyalllowerlevels.Intervalmeasures,forexample,canbeusedtoclassify,putinorder,anddeterminedifferences.

1.NominalLevelMeasurement

Occurswheneverthegoalistheclassificationofthemeasuredcharacteristicorattribute.Itclassifiesbyassigningeachlevelofacharacteristicorattributetoadistinctcategory,forexample,gender.

Alsocommoninobservationalorbehavioraldatacollection:click-throughisanominallevelmeasurementusedtoevaluatebanneradeffectiveness.Acookieorotherunobtrusivedatacollectioninstrumentwouldrecordthepresenceorabsenceofclick.Ineffect,theautomatedsystemmonitoringbannerresponsewouldanswerthequestion“Whatbanneradwasclickedon?”

Or,awebbusinesscanevaluatetheappealofanemailnewsletteroptionbyautomaticallyrecordingtheanswertothequestion“Didthesitevisitorregisterforthenewsletter?”

Nominallevelofmeasurementhasthreedefiningcharacteristics.

• Categoriesaremutuallyexclusiveandcollectivelyexhaustive.Everylevelofthecharacteristicorattributebeingmeasuredfitsintooneandonlyonecategoryandeverycharacteristicorattributefitssomewhere.

• Forpurposesofdataanalysis,numbersareassignedtoeachresponsecategory.However,numbersaremerelylabelsforthecategoriesthattheyrepresent–theyhavenonumericvalue.

• Assumesinternalcategoryequivalence.Allobjectsorpeopleassignedtothesamecategoryandrepresentedbythesamenumberareassumedtobethesame.

2.OrdinalLevelMeasurement

Slide11-

25

Slide11-

26

Slide11-

27

Slide11-

28

Arrangescharacteristicsorattributesaccordingtotheirmagnitudeinanorderedrelationshipalongsomeexplicitdimension,typicallyfromgreatertosmallerorfrommoretoless.

ExampleordinallevelquestionshowninSlide11-25.PossibleoutcomeshowninSlide11-26.

Numbersonordinallevelhavesomemathematicalmeaning.Anumberrepresentsanobject'spositionalongthedimensionofinterestrelativetoallotherobjects.Ordinalmeasuresthereforeprovideinsightsintotherelativestandingoforderedcharacteristics.

Ordinalmeasurescannotprovideanyinsightsintotherelativedistance

betweenrankedobjects.Ontheordinallevelofmeasurementnumbersareonlysymbolsthatrepresentaplaceinanorderedarray.Assuch,theyarenotsubjecttomathematicalcomputationssuchasadditionandsubtraction.

Considerperformanceofthesamethreehorsesinthreedifferentraces.Theorderisalwaysthesamewhere“SamFast”alwayswinsbyaconsiderabledistanceover“BeckySlow”and“KarlWash,”whichalwaysfinishsecondandthirdveryclosetoeachother.Theaveragesforthehorsesforthethreeraceswouldbe:

SamFast 1.0

BeckySlow 2.0

KarlWash 3.0

TheseaverageswouldindicatethattherewasthesamedistancebetweenSamandBeckyasbetweenBeckyandKarl,butclearlythisisnotthecase.Averagingtheordinalmeasuresdistortsourinterpretation.

Ordinalmeasurescanalsoappearoutsideofsurveyresearch.Eye-trackingdatacan,forexample,indicatetheorrderinwhichelementsarenoticed.Itcannot,however,drawtheconclusionthattheitemnoticedsecondisnoticedtwiceasfastastheitemnoticedfourth.

3.IntervalLevelMeasurement

Intervalmeasurementpossessesallfeaturesofnominalandordinalmeasurementwiththeadditionalcharacteristicthatdistanceormagnitudebetweenanytwoadjacentpointsonthescale(orthenumbersthatrepresentthepointsonthescale)isassumedtobeequivalent.Intervalmeasuresallowyoutodetermineboththerelativerankingofobjectsandthedistancebetweentheobjects.

Examples,thermometerandcalendar.Thedifferenceintemperaturebetween

34°and35°isexactlythesameasthedifferencebetween64°and65°.Similarly,thesameamountoftimepassedbetweentheyears1901and1903aspassedbetweentheyears1979and1981.

Slide11-

29

Slide11-

30

Slide11-

31

Slide11-

32

Slide11-

33

Slide11-

34

Itisimportanttonote,however,thatintervalscalesdonothavea"true"zeropointandasaresultitisnotpossibletomakestatementsabouthowmanytimeshigheronescoreisthananother.Equaldifferencesonthetimescalerepresentequaldifferencesintime,buttheyear2000isnottwiceasoldastheyear

1,000.

Whenproperlywritten,pointsonscalesusedonaquestionnaireareassumedtohaveequaldistance,asshowninSlide11-30.

Scaleassumesthatresponseoptionsrepresentacontinuumonwhichdistancebetweenpointsisequivalent.Itassumesthatthedistancebetween"verybelievable"and"slightlybelievable"isthesameasthedistancebetween"slightlybelievable"and"slightlyunbelievable."

Thepowerofintervalscalesversusnominalandordinalscalesisillustratedinanexamplethatfocusesonsoftdrinkpreferences.

Nominaldatasimplyasksifeachsoftdrinkisliked(yesorno).Providestheleastamountofinsightanddifferentiationinbrandattitudes.

Ordinaldataisanimprovementbyaskingindividualstoranktheirpreferences.Thechartshowstheallocationof“mostpreferred”rankingforeachbrand.But,thechartcanbemisleading,asitcannottelldistancesbetweenbrands.

Intervaldataprovidesthebestinsights,showingorderanddistancewhererespondentsuseda4-pointscaleofstronglyliked/stronglydislikedtoshowattitude.

4.RatioLevelMeasurement

Ratiomeasureshaveallthepowerofnominal.Ordinalandintervalmeasures,pluspermitsresearcherstomakecomparisonsamongquantities.Onecaninterpreta“10”scoreonaratiolevelmeasuretohavetwiceasmuchofthemeasuredcharacteristicsasonewitha“5.”Thisisbecauseratiomeasureshaveequaldistancesbetweenpoints(asinintervalmeasurement)andhaveameaningfulzeropoint.

• Examsthatyoutakeinclassareaformofaratioscalewhereyoucanreceiveanypercentagebetweenzeroand100andascoreof80is

consideredtobetwiceashighasascoreof40.

Slide11-

35

Slide11-

36

Slide11-

37

Acommonformofratiomeasureisaconstantsumscalewheretherespondentisgivenanumberofpointsandistoldtodistributethosepointsamongasetofobjectsaccordingtoaspecificcriterion,asshowninSlide11-35.

Numbersontheratiolevelofmeasurementindicatetheactualamountofthepropertybeingmeasured.Ameasureofzeroonaratiomeasuretrulyindicatesthetotalabsenceofthecharacteristicorattributebeingmeasured.

Ratiolevelmeasuresareverycommoninresearchthatfocusesonrespondentbehaviorsforexample,thenumberofadsviewedinaperiodoftime,thenumberofadsviewedofaparticularbrand,thenumberofvisitstoparticulartypesofwebsites,thenumberofTwittermessagessentinthelastday,thenumberofwebsitepagesviewedinthepastday,theamountoftimespentonspecificwebsites,thenumberofvideosforwardedtofriends,orthenumberofbanneradsclicked-on.Notethatinallcases“zero”isapossibleanswer.

5.SelectingaLevelofMeasurement

The“best”levelofmeasurementisdeterminedbythecharacteristicorattributebeingmeasured,desireddepthofdetailrequiredtoanswerresearchinformationneeds,andthelevelofdetailrequiredforplannedorpotentialdataanalysis.However,becausehigherlevelsofmeasurementcontainalldescriptivepower

oflowermeasuresageneralruleofthumbistrytocollectinformationatthehighestappropriatelevelofmeasurement.

Example,bothnominalandratiolevelmeasurescanbeusedtodetermineanindividual'sage.

Advantagesofcollectingdataathigherlevelsofmeasurement(inthiscaseratiolevel)are:

• Datacollectedathigherlevelsislesslimiting.

• Datacollectedathigherlevelsofmeasurementcanalwaysbeturnedintolowerleveldata.

• Ratiolevelmeasures(versusnominallevelmeasures)canbeexaminedwithmorepowerfulstatisticaltechniquesleadingtomoreconfidenceintheconclusionsandinsightsdrawnfromtheresearch.

III.IdentifyAppropriateQuestionType

Surveyresearchcaneitherbeclosed-endedoropen-ended.

Slide11-

38

Slide11-

39

Slide11-

40

Slide11-

41

Slide11-

42

A.Open-endedQuestions

Thesecollectinformationbyallowingindividualsorobserverstoreplyintheirownwordswithoutuseofafixed,predeterminedsetofanswersasshowninSlide11-39.

1.Advantages

• Permitrespondentstoanswerintheirownwords.Canstateexactly

whatisontheirmindwithoutinfluence,asmightoccurwhentheymust

selectananswerfromapredeterminedlistofresponseoptions.

• Canprovideacontextforinterpretinganindividual'sanswerstoclosed-endedquestions

• Goodwaytointroduceatopicarea

2.Disadvantages

• Responsescanbeincomplete,irrelevantand/orincomprehensible

• Takemoretimetoadminister

• Placeagreaterdemandontheverbalskillsoftherespondent

• Analysisismoretimeconsuminganddifficult(versusclosed-endedquestions)

B.Closed-endedQuestions

1.Advantages

Presentasetoffixedalternativeanswersand(versusopen-endedquestions):

•producelessvariabilityintherangeofresponse

•tendtobeeasierforarespondenttoanswer

•arequickertoadminister

•areeasiertoexamineandanalyze

2.Cautions

Slide11-

43

Slide11-

44

Slide11-

45

Slide11-

46

Slide11-

47

Slide11-

48

Shouldonlybeusedwhenfullrangeofoptionsisknownandpresented.

Havethepotentialto"prime"or"cue"therespondent.Open-endedquestionsareappropriatewhenmeasurementgoalisattainmentofunaidedrecall,opinions,orattitudes.Closed-endedquestionsareappropriatewhenmeasurementgoalisattainmentofaidedrecall,opinions,orattitudes.

IV.EvaluateandRevisetheObservableEvent

Aspecificsurveyquestionorobservationalinstrumentyieldsuseful,accurateinformationonlyifitisreliableandvalid.

A.ReliabilityandValidity:AnOverview

Reliabilityandvalidityaretwoindependentyetrelatedaspectsofmeasurement.

• Reliabilityisanassessmentoftheextenttowhichameasureconsistentlyprovidesthesameresults.Areliablemeasureisconsideredtobestable,thatis,freefromrandomerrorinyieldingcomparableresultsovermultipleadministrations.

• Validityisanassessmentofthe“goodness”ofthemeasure.Avalidmeasureisonethataccuratelymeasureswhatwethinkitismeasuring.

Ameasureorsetofmeasurescanhaveoneoffourcombinationsofreliabilityandvalidity.Imaginethatbulletsareshotatatarget.Outcome(representedbydots)canbe:

• reliablebutnotvalid:theoutcomesareconsistent(indicatinghighreliability);butplacementisinthewrongarea(indicatinglowvalidity)

• validbutnotreliable:alloftheshotshitnearthecenterofthetarget(indicatinghighvalidity),butthegreatdiversityinplacementindicatesthatthereislittlereliabilityfrominstancetoinstance

• neithervalidnorreliable:theshotsareneitherclusterednordotheyhitnearthecenterofthetarget

• bothreliableandvalid:shotplacementisconsistentandallshotshitthetargetinthedesiredplace

B.AssessingReliability

Therearetwomainapproachestoassessingreliability:test-retestandalternativeform.

Slide11-

49

Slide11-

50

Slide11-

51

Test-retestreliabilityrepeatstheadministrationoftestitemunderequivalentconditionstosamegroupofpeople.Theresultsoftwoadministrationsarecompared.Greatersimilarityinresponsebetweentwoadministrationsmakestheitem'sreliabilitygreater.

Thealternativeformsimilartotest-retestmethodasthesameindividualsparticipateinmultipleadministrations.Thismethod,however,usesanalternativeformofthefirsttestadministeredforthesecondtesting.Becauseitisassumedthatthetwoformsofthetestaredesignedtomeasurethesamething,reliabilityisestimatedbycomparingresponsestothetwoforms.Thegreaterthecorrespondencethegreaterthereliability.

Thedefinitionofreliabilityisthesameforpassivelyobtainedbehavioralmeasures(suchasrecordingthenumberofwebsitevisitsthroughcookies)asitisforsurveyitems.However,theapproachusedtoassessreliabilityisdifferent.Here,ameasure’sreliabilityisconsideredtobeareflectionofrecordingaccuracyanditssubsequentabilitytoprovideconsistentinformationovertime.

C.AssessingValidity

Validityiscommonlydeterminedinoneofthreeways:face,concurrentandpredictive.

Facevalidityisthemostbasicformofvalidityassessment.Measurehasfacevaliditywhensubjectivejudgmentofprofessionalsorthosewithexpertiseagreethatmeasureaccuratelytranslatesoperationaldefinitionintoanobservedevent,thatis,whenthereisaconsensusthatthemeasuredoesinfactmeasurewhatitissupposedtomeasure.

Concurrentvalidityisassessedbycomparingresultsobtainedfromnewmeasurementwithresultsofanacceptedmeasurementtakenatsamepointintime.

Predictivevalidityisestimatedbydeterminingtheextenttowhichperformanceononevariable(measuredtoday)accuratelypredictsperformanceonanothervariable(tobemeasuredinthefuture).SATscores,forexample,areoftenusedaspredictorsforfirstyearcollegeperformance,asmeasuredbyastudent’sGPA.

ResponsestoApplicationExercises

1.Eachofthethreeresearchershascreatedadifferentoperationaldefinition.

Terry’squestionreflectsanoperationaldefinitionthatdescribes“brandloyalty”as“themajorityofpurchasesmadeforaspecificbrand.”Numbersareusedproperly.Theproblemwiththisquestionliesinthefactthatapersoncouldhavepurchasedasingle128ouncesizeofAllandthree24ouncesizesofCheer.Basedonthenumberofpurchases,itwouldappearthattheconsumerisloyaltoCheer,whichmightnotbethecaseatall.Abetterapproach

mightaskaboutthetotalnumberofouncesofeachbrandpurchased,wherethebrand