Agile Testing StrategiesSQAForums：敏捷测试策略sqaforums

1、agile testing strategiesscott ambleras scott shows here, the quality of your system is only as good as the quality of your testing efforts.scott is a ddj senior contributing editor and author of numerous it books. he can be contacted at / scottambler.html. 1 philosophical groundworko

2、ver the past few months, ive had more and more people ask me about how we go about testing on agile projects. agile developers are definitely test infected, and this month, i explore several strategies for testing on agile software development projects. a word of warning: although we clearly dont fo

3、llow our fathers serial testing methodology of yesteryear, i suspect that we can still learn a few tricks from dear old dad.lets start by setting the philosophical groundwork: first, you want to test as early as you possibly can because the potential impact of a defect rises exponentially over time

4、(this isnt always true, but its something to be concerned about). in fact, many agile developers prefer a test-first approach. second, you want to test as often as possible, and more importantly, as effectively as possible, to increase the chance that youll find defects. although this increases your

5、 costs in the short term, studies have shown that greater investment in testing reduces the total cost of ownership of a system due to improved quality. third, you want to do just enough testing for your situation: commercial banking software requires a greater investment in testing than membership

6、administration software for your local girl scouts group. fourth, pair testing, just like pair programming and modeling with others, is an exceptionally good idea. my general philosophy is that software development is a lot like swimmingits very dangerous to do it alone.2. testing throughout the lif

7、ecyclefigure 1 presents a high-level view of the agile lifecycle for the purpose of testing (see initiating an agile project at /dept/architect/188700850 for details). agile projects go through an often short initiation phase (iteration 0) where we set the foundation for the project; a co

8、nstruction phase where we develop the system in an evolutionary (iterative and incremental) manner; an end game phase where we transition our system into production; and a production phase where we operate the system and support users. dont fear the serial boogeyman: the initiation phase is not a re

9、quirements phase, nor is the end game a testing phase. click image to view at full sizefigure 1: test activities during the agile lifecycle. testing activities vary throughout the lifecycle. during iteration 0, you perform initial setup tasks. this includes identifying the people who will be on the

10、external investigative testing team, identifying and potentially installing your testing tools, and starting to schedule scarce resources such as a usability-testing lab if required. if your project has a deadline, you likely want to identify the date into which your project must enter the end game.

11、 the good news is that youll discover that increased testing during construction iterations enables you to do less testing during the end game.a significant amount of testing occurs during construction iterationsremember, agilists test often, test early, and usually test first. this is confirmatory

12、testing against the stakeholders current intent and is typically milestone-based at the unit level. this is a great start, but its not the entire testing picture (which is why we also need investigative testing that is risk-based at more of an integration level). regardless of the style, your true g

13、oal should be to test, not to plan to test, and certainly not to write comprehensive documentation about how you intend to hopefully test at some point. agilists still do planning, and we still write documentation, but our focus is on high-value activities such as actual testing.during the end game,

14、 you may be required to perform final testing efforts for the release, including full system and acceptance testing. this is true if you are legislated to do so (common in life-critical situations such as medical software development) or if your organization has defined service-level agreements with

15、 customers who require it. luckily, if youve tested effectively during the construction iterations, your final testing efforts will prove to be straightforward and quick. if youre counting on doing any form of serious testing during the end game, then youre likely in trouble because your team wont h

16、ave sufficient time to act on any defects that you do find.3. testing during a construction iterationthe majority of testing occurs during construction iterations on agile projects. your testing effort, just like your system, evolves throughout construction. figure 2 depicts two construction iterati

17、ons, indicating that there is confirmatory testing performed by the team, and in parallel, investigative testing efforts ideally performed by a independent test team (ive adopted the terms confirmatory and investigative testing from michael bolton, a thought leader within the testing community). alt

18、hough it isnt always possible to have an independent test team, particularly for small projects, it is highly desirable. confirmatory testing focuses on verifying that the system fulfills the intent of the stakeholders as described to the team to date, whereas investigative testing strives to discov

19、er problems that the development team didnt consider. figure 2: incremental testing throughout the agile development lifecycle. there are two aspects to confirmatory testing: agile acceptance testing and developer testing, both of which are automated to enable continuous regression testing throughou

20、t the lifecycle. confirmatory testing is the agile equivalent of testing to the specification, and in fact, we consider acceptance tests to be the primary part of the requirements specification and our developer tests to be the primary part of the design specification. both of these concepts are app

21、lications of the agile practice of single sourcing information whenever possible.agile acceptance testing is a mix of traditional functional testing and traditional acceptance testing because the development team and their stakeholders are doing it collaboratively. developer testing is a mix of trad

22、itional unit testing and traditional class/component/service integration testing. developer testing strives to verify both the application code and the database schema (for more information, see my article ensuring database quality; /architect). your goal is to look for coding errors, per

23、form at least coverage if not full path testing, and to ensure that the system meets the current intent of its stakeholders. developer testing is often done in a test-first manner, where a single test is written and then sufficient production code is written to fulfill that test (see www.agiledata.o

24、rg/essays/ tdd.html for details). interestingly, this test-first approach is considered a detailed design activity first and a testing activity second. automation is an important aspect of construction testing due to the increased need for regression testing on evolutionary projects. the fitnesse te

25、sting framework (), arguably a requirements documentation tool, is often used to automate agile acceptance tests. it is also possible to generate acceptance test cases from use cases and scenario definitions or from process diagrams such as uml activity diagrams or flow charts, and t

26、ools are beginning to emerge to do exactly this. the xunit frameworkjunit () for java and vbunit () for visual basicis used to automate developer tests. commercial testing tools such as hp mercurys testdirector (/us/products/quality-center/testdirector) or i

27、bm rationals testmanager (/developerworks/rational/ products/testmanager) are also good options to consider, as they often prove more sophisticated than their open source alternatives. static code analysis tools, such as findbugs (), are often included in autom

28、ated testing runs to help identify potential quality problems in the source code.4. investigative testingas i was writing this column, i was lucky enough to attend a presentation given by dr. cem kaner to the toronto association of systems and software quality (tassq). kaner, coauthor of lessons lea

29、rned in software testing (wiley, 2001), described his thoughts and experiences in software testing, and as a result, helped me to conceptualize some of my own ideas. in particular, i had been struggling to properly describe the activities of investigative testing efforts during construction iteratio

30、ns, and kaners presentation helped to coalesce my experiences.a separate test team? preposterous you say! actually, there is significant value to be gained by submitting your system to an independent test team at intervals throughout the lifecycle so that they can verify the quality of your work. ag

31、ile teams produce working software at the end of each construction iteration; therefore, you have something new to test at that point. a common practice is to provide a new version of the system at least once a week, regardless of your iteration length, a particularly good strategy the closer you ge

32、t to the end game. the investigative test teams goal should be to ask, what could go wrong, and to explore potential scenarios that neither the development team nor business stakeholders may have considered. theyre attempting to address the question, is this system any good? and not, does this syste

33、m fulfill the written specification? the confirmatory testing efforts verify whether the system fulfills the intent, so simply repeating that work isnt going to add much value. kaner promotes the idea that good testers look for defects that programmers missed, exploring the unique blind spots of the

34、 individual developers. investigative testers describe potential problems in the form of defect storiesthe agile equivalent of a defect report. a defect story is treated as a form of requirementit is estimated and prioritized and put on your requirements stack. the need to fix a defect is a type of

35、requirement, so it makes perfect sense to address it just like any other requirement. as you would expect, during the end game, the only requirement type that youre working on is defect stories.your investigative testing will address common issues such as load/stress testing, integration testing, an

36、d security testing. scenario testing, against both the system itself and the supporting documentation, is also common. you may also do some form of usability testingthe user interface is the system to most end users; therefore, usability is critical to success. the ui includes both the screens that

37、people interact with and the documentation that they read, implying that you need to test both. good investigative testing efforts reveal any problems that developers missed long before they become too expensive to address. it also provides feedback to management that the team is successfully delive

38、ring high-quality working software on a regular basis. kaner pointed out that theres no one right way to go about investigative testing, nor is there one correct list of techniques to employ. your efforts must reflect the goals of the project team that youre supporting. for example, is the goal to d

39、etermine whether the system is ready to be shipped? is it to ensure that the system interoperates with other existing systems? is it to help the developers to identify problems in their own testing efforts by pointing out the causes of defects that they missed? is it to minimize the chance of a laws

40、uit against your organization or its managers? the context in which you are testing will determine what and how you testnot only will the context be different for each project; the context also changes over the life of the project. the type of confirmatory testing performed by agile teams is only one part of the testing pictureit is the agile equivalent of traditional smoke testing. this is a great start, and having automated regression testing provides the safety net required by evolutionary development techniques. investigative testing enables you to explore the critical big