全面的自动化测试知识体系(ABOK)

第第页全面的自动化测试知识体系(ABOK)ccessfulintestautomationfirstrequiresanunderstandingofwhattestautomationis,andwhereitfitsintotheoveralltestinglifecycle.

Differencebetweensoftwaretestingandsoftwaretestautomation软件测试自动化与软件测试之间的区别

SoftwareTestingisadisciplineinwhichtestlogicisdesignedtobeimplementedbyahumanbeing.Asopposedto,SoftwareTestAutomationdesignsscriptlogicthatallowsthatmanualtestlogictobeimplementedbyatool.Anexpansionofthisconceptassertsthatsoftwaretestautomationinvolvestoolsupportforallaspectsofatestproject,notjustautomationoftestexecution.Withthatbeingsaid,asatestautomator,youshouldbeawareofallphasesofthetestinglifecycle,alongwithtoolsthatsupportthosephases.

TestToolAcquisitionandIntegration测试工具选购与整合

TheTestToolAcquisitionistheprocessofidentifyingthetoolgoalsandobjectives,makinganeffectivebusinesscaseandnarrowingdownagroupofcandidatetoolstotheselection(s)thatbestmeet(s)theorganizationalneeds.TestToolIntegrationis,conversely,theprocessofgraduallyaddressingtoolconsiderationsandwideningthetool’sacceptance,useandbenefits.Thevarioussourcesprovideextensivedirectionsonhowtoconductthetoolselectionandacquisitionprocess.Mostsourceswillagree,however,thatthisprocessshouldataminimumincludethefollowing:

Makingawell-informeddecisiontoautomate

Conductingatesttoolevaluation

Addressingtoolimplementationconsiderations

Pilotingthetoolimplementation

Often,organizationsarepushedintomakingadecisionabouttheintroductionorexpansionoftestautomationbasedontheabilitytobuyaspecificautomatedtesttool.It’simportanttoexpandyourvisiontobeabletoidentifyseveraloptions–bothcommercialandnon-commercial–andtobeknowledgeableonhowtoperformaneffectiveevaluationoftheseoptionssothatamoreinformeddecisioncanbemadeabouttestautomationimplementation.

AutomationBenefitsandMisconceptions自动化的益处与误解

Behindeverydecisiontointroducetestautomationisawidevarietyofstatedorimpliedgoalsandexpectationsforwhattheautomationmustaccomplish.Withoutaproperunderstandingofreasonabletestautomationgoals,theautomationeffortwillnotsurvive,andwillbedestinedtositontheproverbialshelfofyourtestorganization.Understandingtheimpactoftestautomationmeanshavingafirmgrasponthebenefitsoftestautomation,accompaniedbyanunderstandingofthecommonmisconceptionssurroundingtestautomation.BeingequippedwiththisknowledgehelpsaTestAutomatormoreeffectivelyimplementatestautomationeffort,“sell”theautomationtothepowers-that-bebypresentingandtrackingabusinesscase,andmanageeffort-endingmisconceptionsandunrealisticexpectationsheldbytheorganizationdecisionmakers.

AutomationReturn-on-Investment(ROI)自动化的ROI计算

Softwaretestautomationisaninvestmentintimeandmoney,andlikeanyinvestment,thestakeholders–normallymanagersorcustomers–areexpectingapositivereturnfromthatinvestment.Iftheirexpectationsarenotsufficientlymet,stakeholderswillprobablydiscontinuetheinvestinginautomation.Theinvestmentisjustifiedbyidentifyingthepotentialreturn-on-investment(ROI),aratioofbenefitstocosts.ThereareseveralapproachesforcalculatingROIincluding:

SimpleROICalculation–Apercentagecalculatedintermsofthemonetarysavingsthattestautomationprovides.

EfficiencyROICalculation–Examinestestautomationbenefitsintermsoftimethatissavedoncertainportionsofthetestingeffort.

RiskReductionROICalculation–Apercentagecalculatedintermsoftheincreasedqualitythatresultsfromtheincreasedtestcoverageprovidedtestautomation.

SkillCategory2:TestAutomationTypesandInterfaces测试自动化的类型和接口

TestAutomationTypes测试自动化的类型

Forsimplicitythedifferenttypesoftestautomation–functional(regression),unit,integration,performance,etc.–areoftenlumpedintoone‘testautomation’category.It’simportanttounderstandsomeofthebasicdifferencesamongthesedifferenttypes,evenifyoudon’tspecializeinallofthem,sothatyoucanspeakintelligentlyaboutthemwhenquestionedbymembersoftheorganization.Sinceallautomatorsareoften“paintedwiththesamebrush”,beingtotallyoblivioustobasicconceptsineachtypeofautomationcannegativelyimpactallautomationeffortsintheorganization.Inaddition,beingabletodisplayminimalknowledgeinvarioustypesofautomationmayprovidemanagementwithenoughconfidenceinyoutoallowyoutopilotanewautomationeffort,whichwouldhelpyoutoultimatelyexpandyourskills.

TestAutomationInterfaces测试自动化的接口

Themaininterfacesmadeavailableforapplicationautomationare:

CommandLineInterface命令行接口

ApplicationProgrammingInterface应用程序编程接口

GraphicalUserInterface图形用户界面

TheGraphicalUserInterfaceisprobablythemostpopularinterfaceforautomationimplementation,duetothefactifmorecloselysimulateshowausermightusethetool.TheCommandLineInterfaceandApplicationProgrammingInterface,however,areadvantagesgiventhefactthattheymakeitsimplertoimplementtestautomation.

SkillCategory3:AutomationTools自动化工具

Atestautomationprofessionalalsomustbepreparedtoprovideinsightintotoolsthatsupportallaspectsofthetestinglifecycle.Thisrequiresanunderstandingofthedifferenttypesandcategoriesoftoolsthatsupportthetestinglifecycle,aswellasthecriteriaforassessingthesedifferenttypesoftools.

Thesedifferenttypesoftoolsinclude:

SoftwareConfigurationManagementTools

Business/SystemModelingTools

RequirementsManagementTools

UnitTestingTools

TestManagementTools

DefectTrackingTools

CodeCoverageAnalyzerTools

FunctionalSystemTestAutomationTools

PerformanceSystemTestAutomationTools

SkillCategory4:TestAutomationFrameworks测试自动化框架

AutomationScope自动化的范围

Totalautomationcostiscomposedofbothdevelopmentcostsandmaintenancecosts.(自动化的成本由开发成本和维护成本组成)Astheautomationframeworkbecomesmoredefined,scriptingincreasesincomplexity.Whilethiscausesdevelopmentcoststoincrease,italsocausesmaintenancecoststodecrease.Asthescopewidens,maintenancebecomesincreasinglyimportant.(随着测试范围的扩大，维护成本变得重要起来)Also,asmaintenancebecomesincreasinglyimportant,theautomationframeworkmustbemoreadvancedinordertoreducetotalautomationcosts.Therefore,beforemakingadeterminationoftheframeworkthatwillbeusedfortestautomation,anevaluationmustbeconductedontheimpliedand/orstatedscopeoftheorganization’sautomationeffort.

RolesandResponsibilities角色和职责

Eachframeworktypewillhaveoneormoreofthefollowingrolesforcreationandimplementationoftheframework:

TeamLead测试主管

TestEngineer测试工程师

LeadAutomationArchitect自动化测试架构师

CrossCoverageCoordinator组织协调人员

AutomationEngineer(Automator)自动化测试工程师

Veryoften,onepersonmayholdmultipleroles.

FrameworkGenerations框架的发展

Overtheyears,automatedtestinghasevolved,becomingincreasinglydefinedandsophisticatedwitheachnewevolutionaryphase.Thisevolutionmaybediscussedintermsofthreegenerations:

·FirstGeneration–ThisgenerationiswhatthisbookreferstoastheCommonApproach.Itisprimarilydrivenbyrecordandplayback.第一代–录制回放

SecondGeneration–Thesecond-generationlearnsfromtheproblemsofthefirstgeneration.Itinvolvescreatingaframeworkinwhichtoautomatetests.Italsoincorporatestheconceptsoffunctionaldecomposition,whichsimplymeanscreatingmodularfunctionsforexecutingfundamentalactionsthatoccurintheexecutionofthetestsinagiventestbed.Theseactionsmaythenbecalleduponandexecutedindependentofoneanother.Thisgenerationalsomoregreatlyseparatestestdatafromautomationcode,throughgreateruseofparameterization.第二代–功能分解

·ThirdGeneration–Thethird-generationofautomationbuildsupontheconceptsofthesecondgeneration.Itstronglydependsonfunctionaldecomposition,butevaluatesthefunctionsatamuchhigherlevelofabstraction,sothatcodemaybereusedbytestswithinthesameapplication,andbytestsindifferentapplications.Thisisaccomplishedbyseparatingtestdataandspecificapplicationfunctionalityfromautomationcode.第三代–数据与代码分离

Duringthedesignphaseadeterminationmustbemadeaboutwhichgenerationwilldefineyourautomatedtestframework.

SkillCategory5:AutomationFrameworkDesign自动化框架设计

Theprocessofdesigninganautomatedtestframeworkisnotanexactscience.Itispossibletodefinitivelyidentifythedifferenttypesofframeworks,buttheprocessusedtoselectandimplementaparticularframeworkisalittlemoredifficulttopinpointinsuchawaythatmostoftheindustryexpertsagree.Themostimportantthinghowever,atthispointinautomationhistoryistoensurethatawellthoughtoutapproachbasedoncommon,successfullyimplementedindustrypracticesisusedandhonedwithinagivenorganization.Thisskillcategoryidentifiesanapproach,fromahighenoughlevelthatitwillfitwithwheretheITindustrycurrentlyisrelativetotestautomation,whilestillremaininglow-levelenoughtobeusefulforimplementation.Thisapproachinvolvesthefollowingsteps:

SelectingaFrameworkType选择框架类型

IdentifyingFrameworkComponents确定框架组成部分

IdentifyingFrameworkDirectoryStructure确定框架目录结构

DevelopingImplementationStandards建立实施标准

DevelopAutomatedTests开发自动化测试

SkillCategory6:AutomatedTestScriptConcepts自动化测试脚本思想

TestSelection测试用例选择

Choosingwhatandwhentoautomateinvolvesananalysisofthefollowing:

Automateitemsthataretediousformanualexecution,butrelativelyeasytoautomate.对于那些手工测试比较繁琐而又容易实现自动化的优先考虑自动化实现。

Itemsthatneedtobeexecutedoverandoveragain.对于那些需要重复又重复执行的测试用例实现自动化。

Itemsthatwillincreasecoveragebeyondwhatwillrealisticallybedonemanually对于那些能有效提高测试覆盖率的测试用例实现自动化。

Inaddition,thedecisiondependsonthegoalsoftheorganizationsuchascostgoals,efficiencygoals,andriskmitigationgoals.

AutomatedTestDesignandDevelopment自动化测试的设计和开发

Automatedtestdesignanddevelopmentaretieddirectlytotheinterfaceinwhichthetestswillbecreated,thewayqualityattributesareappliedatthetestlevel,theelementsthatarecommontoautomatedtests,andthestandardsusedforcreationoftests.

AutomatedTestExecution,AnalysisandReporting自动化测试的执行、分析和报告

Thebiggestitemstoaddressregardingtestexecutionare:

Howmanymachineswillbeusedforautomatedtestexecution

Errorhandling

Howtheresultsarereportedandanalyzed

SkillCategory7:QualityAttributeOptimization质量优化

Thiscategorydefinesvariousqualityattributesoftheautomatedtestsuiteandidentifieswaysofaddressingtheseattributesbasedonprioritiesandconstraintssurroundingeach.

ThefollowingaresomeQualityAttributesthatmaybeconsidered.

Maintainability

Portability

Flexibility

Robustness

Scalability

Reliability

Usability

Performance

SkillCategory8:ProgrammingConcepts编程思想

Whetheryouuseatoolwithascriptinglanguage,treestructureand/orkeywords,fundamentalprogrammingconceptsremainparamountineffectivelyautomatingtests,andincreasingthedistancethetestcancoverthroughincreasedsystemcoverageandtestflexibility.Conceptssuchasvariables,controlflowstatements(if..then..else,for..next,etc.),andmodularityarediscussedinthiscategory.

SkillCategory9:AutomationObjects自动化对象

RecognizingApplicationObjects识别应用程序对象

Onceuponatime,functionalsoftwaretestautomationprimarilyusedan“analog”approachthatreliedonspecificcoordinatelocationsofascreenorapplicationwindowforperformingmouseandkeyboardoperationsthatsimulateduseractivitiesontheapplication.Thiswasslightlyunreliable,anddifficulttomaintainonalargescale,becausewhenthescreen,windoworapplicationcomponentsmovedeversoslightly,theautomatedtestwouldfailbecauseitwastryingtooperateonanelementinapositioninwhichitnolongerexisted.Moderntestautomationconverselytakesadifferentapproachthatlocatestheobjectsonthescreenbasedonpropertiesthatdefinethatobjectandthenperformsthedesiredoperationsoncetheobjectisfound.

ObjectMaps对象映射

OneofthesimplestwaystoboostthemaintainabilityandrobustnessofanautomatedtestsuiteisthroughtheintroductionofObjectMaps.ObjectMapsreducetheamountofinformationthatisbeingmaintainedinanautomatedtestbystoringobjectpropertiesinanexternalfileandreferencingobjectsaccordingtovariablenames(alsoknownasLogicalNames)thatarethentiedtotheproperties.

ObjectModels对象模型

Manysoftwareapplicationsarecomposedofseveralinterrelatedobjects,andanobjectmodelisanabstractrepresentationofthehierarchicalgroupofrelatedobjectsthatdefineanapplicationandworktogethertocompleteasetofapplicationfunctions.Theadvantagesofferedbyobjectmodelsinclude:

·Increasedapplicationunderstanding

·Increasedscriptingpower

DynamicObjectBehavior动态对象行为

Oneofthebiggestchallengesfacedbyautomatedtestengineersisthatofdynamicobjectbehavior.Peopleprocessinformationabouttheapplicationbasedonvisualinspection,whileautomatedtestsuseobjectproperties.Peoplecan,therefore,easilymakeadjustmentswhenthereisaslightchangefromavisualstandpoint,andcanignoremanypropertychanges.Thecomputer,however,cannotadjustaseasily.Thenecessaryadjustmentshavetobeanticipatedupfrontandprogrammed.

SkillCategory10:DebuggingTechniques调试技巧

TypesofErrors错误类型

Regardlessofhowwellautomatedtestsaredesignedandcreated,problemswilloccur.Sometimestheproblemsarerelatedtothescripts,andsometimestheyarerelatedtotheapplication,buttherootcauseisnotalwayssimpletofind.Theinabilitytoeffectivelydebugscriptscanseverelydelayschedules,andcanevenbringautomationtoascreechinghalt.Thefirststepinscriptdebuggingisinunderstandingthetypesoferrorsthatmaybeencountered.Thereare4maingenerictypesoferrorsthatmaybeencountereduponscriptexecution:

SyntaxErrors语法错误

Run-timeErrors运行时错误

LogicErrors逻辑错误

ApplicationErrors应用程序错误

DebuggingTechniques调试技巧

Thetrickiestpartofdebuggingisfindingoutthetruesourceofanerror.Thisinvolvesensuringtheerrorisreproducible,andthentheprocessoffindingthemainsourceoftheerrormustbegin.Veryoftenwhatseemstobethesourceoftheerrorisactuallyjustasymptomofthetrueerror,sothereareseveraltechniquesthatmaybeemployedtofindthesourceofanerror.Atthatpointadeterminationmaybemadeonwhetherornottheerrorisduetoanapplicatio