创业管理(李雪灵)pa

1、,Part V: Working with Test Documentation,SE-307 Software Testing,Contents,Planning Your Test Effort Writing and Tracking Test Cases Reporting What You Find Measuring Your Success,Part V: Working with Test Documentation,Planning Your Test Effort,REMINDER The goal of a software tester is: The goal of

2、a software tester is to find bugs, find them as early as possible, and make sure they get fixed.,The Goal of Test Planning,The software test plan is the primary means by which software testers communicate to the product development team what they intend to do. The IEEE Standard 8291998 for Software

3、Test Documentation states that the purpose of a software test plan is as follows: To prescribe the scope, approach, resources, and schedule of the testing activities. To identify the items being tested, the features to be tested, the testing tasks to be performed, the personnel responsible for each

4、task, and the risks associated with the plan.,The Goal of Test Planning,The test plan is a by-product of the detailed planning process thats undertaken to create it. Its the planning process that matters, not the resulting document. The ultimate goal of the test planning process is communicating (no

5、t recording) the software test teams intent, its expectations, and its understanding of the testing thats to be performed.,Test Planning Topics,Many software testing books present a test plan template or a sample test plan that you can easily modify to create your own project-specific test plan. The

6、 problem with this approach is that it makes it too easy to put the emphasis on the document, not the planning process. A list of important topics that should be thoroughly discussed, understood, and agreed to among your entire project team including all the testers.,Test Planning Topics,High-Level

7、Expectations Theyre fundamental topics that must be agreed to by everyone on the project team Whats the purpose of the test planning process and the software test plan? What product is being tested? What are the quality and reliability goals of the product? A result of the test planning process must

8、 be a clear, concise, agreed-on definition of the products quality and reliability goals. The goals must be absolute so that theres no dispute on whether they were achieved.,Test Planning Topics,People, Places, and Things Test planning needs to identify the people working on the project, what they d

9、o, and how to contact them. Similarly, where documents are stored (what shelf or file server the test plan is sitting on), where the software can be downloaded from, where the test tools are located, and so on need to be identified. If hardware is necessary for running the tests, where is it stored

10、and how is it obtained? If there are external test labs for configuration testing, where are they located and how are they scheduled?,Test Planning Topics,Definitions Getting everyone on the project team to agree with the high-level quality and reliability goals definition of a bug goal of a softwar

11、e tester common terms Build. Test release document (TRD). Alpha release. Beta release. Spec complete. Feature complete. Bug committee.,Test Planning Topics,Inter-Group Responsibilities Inter-group responsibilities identify tasks and deliverables that potentially affect the test effort. The test team

12、s work is driven by many other functional groups programmers, project managers, technical writers, and so on. The easiest way to plan these and communicate the plan is with a simple table (see Figure 17.1).,Test Planning Topics,What Will and Wont Be Tested The planning process needs to identify each

13、 component of the software and make known whether it will be tested. If its not tested, there needs to be a reason it wont be covered. It would be a disaster if a piece of code slipped through the development cycle completely untested because of a misunderstanding.,Test Planning Topics,Test Phases T

14、o plan the test phases, the test team will look at the proposed development model and decide whether unique phases, or stages, of testing should be performed over the course of the project. The test planning process should identify each proposed test phase and make each phase known to the project te

15、am. This process often helps the entire team form and understand the overall development model.,Test Planning Topics,Test Strategy The test strategy describes the approach that the test team will use to test the software both overall and in each phase.,Test Planning Topics,Resource Requirements Plan

16、ning the resource requirements is the process of deciding whats necessary to accomplish the testing strategy. The following things could possibly be used for testing: People. Equipment. Office and lab space. Software. Outsource companies. Miscellaneous supplies.,Test Planning Topics,Tester Assignmen

17、ts Planning the tester assignments identifies the testers (this means you) responsible for each area of the software and for each testable feature.,Test Planning Topics,Test Schedule The test schedule takes all the information presented so far and maps it into the overall project schedule. One way t

18、o help keep the testing tasks from being crunched is for the test schedule to avoid absolute dates for starting and stopping tasks. Table 17.2 is a test schedule that would surely get the team into a schedule crunch.,Test Planning Topics,Test Cases Writing and Tracking Test Cases, will go further in

19、to detail about them. The test planning process will decide what approach will be used to write them, where the test cases will be stored, and how theyll be used and maintained.,Test Planning Topics,Bug Reporting Reporting What You Find, will describe the techniques that can be used to record and tr

20、ack the bugs you find. Exactly what process will be used to manage the bugs needs to be planned so that each and every bugs is tracked from when its found to when its fixed-and never, even forgotten.,Test Planning Topics,Metrics and Statistics Metrics and statistics are the means by which the progre

21、ss and the success of the project, and the testing, are tracked. Examples of test metrics that might be useful are Total bugs found daily over the course of the project List of bugs that still need to be fixed Current bugs ranked by how severe they are Total bugs found per tester Number of bugs foun

22、d per software feature or area,Test Planning Topics,Risks and Issues As a software tester, youll be responsible for identifying risks during the planning process and communicating your concerns to your manager and the project manager. These risks will be identified in the software test plan and acco

23、unted for in the schedule. Some will come true, others will turn out to be benign. The important thing is to identify them early so that they dont appear as a surprise late in the project.,Highlights of this chapter include,The purpose of test planning Why its the planning, not the plan, that matter

24、s The areas to consider in the planning process What a new testers role is with the test plan,farucalgary.ca,4,Part V: Working with Test Documentation,Writing and Tracking Test Cases,Contents,The Goals of Test Case Planning Test Case Planning Overview Test Case Organization and Tracking,The Goals of

25、 Test Case Planning,Carefully and methodically planning test cases is a step in that direction. Doing so is very important for four reasons: Organization. Repeatability. Tracking. Proof of testing (or not testing).,Test Case Planning Overview,Test Case Planning Overview,Test Design The overall proje

26、ct test plan is written at a very high level. It breaks out the software into specific features and testable items and assigns them to individual testers, but it doesnt specify exactly how those features will be tested. The following topics, adapted from the IEEE 829 standard, address this purpose a

27、nd should be part of the test design specs that you create: Identifiers. Features to be tested. Approach. Test case identification. Pass/fail criteria.,Test Case Planning Overview,Test Cases IEEE 829 states that the test case specification documents the actual values used for input along with the an

28、ticipated outputs. A test case also identifies any constraints on the test procedure resulting from use of that specific test case.“,Test Case Planning Overview,The IEEE 829 standard also lists some other important information that should be included: Identifiers. Test item. Input specification. Out

29、put specification. Environmental needs. Special procedural requirements. Intercase dependencies.,Test Case Planning Overview,Test Procedures IEEE 829 states that the test procedure specification identifies all the steps required to operate the system and exercise the specified test cases in order to

30、 implement the associated test design.,Test Case Planning Overview,The test procedure or test script spec defines the step-by-step details of exactly how to perform the test cases. Identifier. Purpose. Special requirements. Procedure steps. Log. Setup. Start. Procedure. Measure. Shut down. Restart.

31、Stop. Wrap up. Contingencies. Detail Versus Reality,Test Case Organization and Tracking,One consideration that you should take into account when creating the test case documentation is how the information will be organized and tracked. Think about the questions that a tester or the test team should

32、be able to answer: Which test cases do you plan to run? How many test cases do you plan to run? How long will it take to run them? Can you pick and choose test suites (groups of related test cases) to run on particular features or areas of the software? When you run the cases, will you be able to re

33、cord which ones pass and which ones fail? Of the ones that failed, which ones also failed the last time you ran them? What percentage of the cases passed the last time you ran them?,farucalgary.ca,4,Part V: Working with Test Documentation,Reporting What You Find,Contents,Getting Your Bugs Fixed Isol

34、ating and Reproducing Bugs Not All Bugs Are Created Equal A Bugs Life Cycle Bug-Tracking Systems,Getting Your Bugs Fixed,The reasons listed in Chapter 3 for not fixing a bug were Theres not enough time. Its really not a bug. Its too risky to fix. Its just not worth it. Ineffective bug reporting.,Get

35、ting Your Bugs Fixed,Fundamental principles for reporting a bug: Report bugs as soon as possible. Effectively describe the bugs. Minimal. Singular. Obvious and general. Reproducible. Be nonjudgmental in reporting bugs. Follow up on your bug reports.,Isolating and Reproducing Bugs,Isolating and repro

36、ducing bugs is where you get to put on your detective hat and try to figure out exactly what the steps are to narrow down the problem.,Isolating and Reproducing Bugs,The suggestions in isolating the bug: Dont take anything for granted. Look for time-dependent and race condition problems. White-box i

37、ssues of boundary condition bugs, memory leaks, and data overflows can be slow to reveal themselves. State bugs show up only in certain states of the software. Consider resource dependencies and interactions with memory, network, and hardware sharing. Dont ignore the hardware.,Not All Bugs Are Creat

38、ed Equal,trade-offs must be made and risks must be taken in every software project to decide what bugs to fix and what bugs not to fix or to postpone to a later release of the software. Youll classify your bugs and identify in a short, concise way what their impact is. The common method for doing th

39、is is to give your bugs a severity and a priority level.,Not All Bugs Are Created Equal,Of course, the specifics of the method vary among companies, but the general concept is the same: Severity indicates how bad the bug is; the likelihood and the degree of impact when the user encounters the bug. P

40、riority indicates how much emphasis should be placed on fixing the bug and the urgency of making the fix. NOTE A bugs priority can change over the course of a project. A bug that you originally labeled as Priority 2 could be changed to Level 4 as time starts to run out and the software release date

41、looms. If youre the software tester who found the bug, you need to continually monitor the bugs status to make sure that you agree with any changes made to it and to provide further test data or persuasion to get it fixed.,A Bugs Life Cycle,In many instances, this is as complicated as a software bug

42、s life cycle gets: a bug report is opened, resolved, and closed. In some situations, though, the life cycle gets a bit more complicated, as shown in Figure 19.3.,Bug-Tracking Systems,By now it should be clear that the bug-reporting process is a complex beast that requires a great deal of information

43、, a high level of detail, and a fair amount discipline to be effective. A bug-tracking system that allows you to log the bugs you find and monitor them throughout their life cycle.,Bug-Tracking Systems,The Standard: The Test Incident Report The following list shows the areas that the standard define

44、s, adapted and updated a bit, to reflect more current terminology. Identifier. Summary. Incident Description. Impact.,Bug-Tracking Systems,Manual Bug Reporting and Tracking Notice that this one-page form can hold all the information necessary to identify and describe a bug. Automated Bug Reporting a

45、nd Tracking A bug-tracking database provides a central point that an entire project team, not just the testers, can use to communicate the status of the project, tell whos assigned what tasks to perform, and, most importantly, assure that no bug falls through the cracks. Its the culmination of every

46、thing youve learned in this chapter about how to report the bugs you find.,farucalgary.ca,4,Part V: Working with Test Documentation,Measuring Your Success,Contents,Using the Information in the Bug Tracking Database Metrics That Youll Use in Your Daily Testing Common Project-Level Metrics,Using the Information in the Bug Tracking Database,The projects bug database can tell you what has happened in the past, whats happening now, and allows you to look at the data to make an educated guess