软件测试技术：Part IV Supplementing Your Testing

1、Part IV: Supplementing Your Testing SE-307 Software Testing ContentsAutomated Testing and Test ToolsBug Bashes and Beta TestingPart IV: Supplementing Your Testing Automated Testing and Test ToolsContentsThe Benefits of Automation and ToolsTest ToolsSoftware Test AutomationRandom Testing: Monkeys and

2、 GorillasRealities of Using Test Tools and AutomationThe Benefits of Automation and ToolsBackgroundRegression testing: This process of rerunning your tests is known as regression testing. The work of regression testing is enormousSoftware test tools and automation can help solve this problem by prov

3、iding a more efficient means to run your tests than by manual testing. The Benefits of Automation and ToolsThe principal attributes of tools and automation areSpeed. Efficiency. Accuracy and Precision. Resource Reduction. Simulation and Emulation. Relentlessness. NOTESoftware test tools arent a subs

4、titute for software testers they just help software testers perform their jobs better.Test ToolsAs a software tester youll be exposed to a wide range of testing tools. The types of tools that youll use are based on the type of software that youre testing and whether youre performing black-box or whi

5、te-box tests.Important distinction between two types of tools non-invasive and invasive. If a tool is used only to monitor and examine the software without modifying it, its considered non-invasive. If, however, the tool modifies the program code or manipulates the operating environment in any way,

6、its invasive. Test ToolsThe major classes of testing tools and how theyre used. Viewers and Monitors A viewer or monitor test tool allows you to see details of the softwares operation that you wouldnt normally be able to see. Code coverage analyzers, communications analyzer, the code debuggers that

7、come with most compilers. Test ToolsDrivers Drivers are tools used to control and operate the software being tested. Batch file, replace the keyboard and mouse of the system being tested with an additional computer. Stubs Stubs, like drivers, were mentioned in Chapter 7 as white-box testing techniqu

8、es. EmulatorTest ToolsStress and Load Tools The Stress program allows you to individually set the amounts of memory, disk space, files, and other resources available to the software running on the machine. Load tools are similar to stress tools in that they create situations for your software that m

9、ight otherwise be difficult to create. Interference Injectors and Noise GeneratorsTheyre similar to stress and load tools but are more random in what they do. Test ToolsAnalysis Tools Word processing softwareSpreadsheet softwareDatabase softwareFile comparison softwareScreen capture and comparison s

10、oftwareDebuggerBinary-hex calculatorStopwatchVCR or cameraTest Tools测试设计工具用例生成，随机抽取测试管理工具测试运行跟踪，缺陷跟踪工具Software Test AutomationMacro Recording and PlaybackThe most basic type of test automation is as you run recording your keyboard and mouse actions your tests for the first time and then playing them

11、 back when you need to run them again. Macro recorders and players are a type of driver tool. Drivers are tools used to control and operate the software being tested. On the Mac you can use QuicKeys; on Windows the shareware program Macro Magic is a good choice. Be careful: The biggest problem is la

12、ck of verification. A slight change in the GUI could throw off your captured steps. Software Test AutomationProgrammed Macros Create programmed macros by programming simple instructions for the playback system to follow. Programmed macros such as this one have some real advantages over recorded macr

13、os. Although they still cant perform verification of the test results, they can pause their execution to prompt the tester (see Figure 15.8) with an expected result and a query for her to okay whether the test passed or failed. Software Test AutomationProgrammed Macros (continue)Listing 15.1. A Simp

14、le Macro That Performs a Test on the Windows Calculator1: Calculator Test #22:3: 4: 123-100=5: 6: Software Test AutomationProgrammed Macros (continue)Youre still missing two important pieces, though, to perform complex testing.Programmed macros are limited to straight-line execution they can only lo

15、op and repeat. Variables and decision statements that youd find in a regular programming language arent available. You also dont have the ability to automatically check the results of your test. For these, you need to move to a comprehensive automated testing tool.Software Test AutomationFully Progr

16、ammable Automated Testing Tools The most important feature that comes with these automation tools is the ability to perform verification, actually checking that the software is doing whats expected. There are several ways to do this: Screen captures. Control values. File and other output. Software T

17、est AutomationFully Programmable Automated Testing Tools (continue)To get more information about several of the popular test automation products available, visit the following websites:Software Development Technologies at Mercury at Segue Software at Random Testing: Monkeys and GorillasAnother type

18、of automated testing, though, isnt designed to help run or automatically run test cases. Its goal is to simulate what your users might do. That type of automation tool is called a test monkey. When your software is released to the public, it will have thousands or possibly millions of people using i

19、t. Despite your best efforts at designing test cases to find bugs, some bugs will slip by and be found by those users. Random Testing: Monkeys and GorillasDumb MonkeysIt just clicks or types randomly. 例：1：RANDOMIZE TIMER 2: FOR i=1 TO 10000 3: PLAY “CLICK”+STR$(INT(RND*640)+”,”+STR$(INT(RND*480)+” 4

20、:PLAY CHR$(RND*256) 5:NEXT i Random Testing: Monkeys and GorillasDumb MonkeysIt doesnt seem to make sense that simple random clicking and typing could find a bug, but it does for a couple reasons:Given enough time and attempts, just like the monkeys writing Shakespeare, the random inputs will eventu

21、ally stumble onto a magic sequence that the programmers and testers didnt think of. A dumb monkey, with its continuous repetition and use, can expose bugs such as memory leaks that might not occur until many hours or days of normal use.Random Testing: Monkeys and GorillasSemi-Smart Monkeys Dumb monk

22、eys lack a few important features （如故障的重现）, though, that would make them even more effective. Adding these features （如日志或屏幕录像 ）raises your monkeys IQ a bit, making him semi-smart. The solution is to add logging to your monkey so that everything it does is recorded to a file. When the monkey finds a

23、bug, you need only to look at the log file to see what it was doing before the failure. Its also a good idea to program your monkey to operate only on the software youre testing. Another good feature to make your monkey semi-smart is crash recognition. Random Testing: Monkeys and GorillasSmart Monke

24、ys Such a monkey takes the effectiveness of random testing from his less-intelligent brothers and adds to that an awareness of his surroundings. He doesnt just pound on the keyboard randomly he pounds on it with a purpose. A smart monkey can examine data as it goes, checking the results of its actio

25、ns and looking for differences from what it expects. If you programmed in your test cases, the smart monkey could randomly execute them, look for bugs, and log the results. Realities of Using Test Tools and AutomationTest automation isnt a panacea. When it is properly planned and executed it can mak

26、e your testing immensely more efficient and find bugs that would have otherwise gone undiscovered. However, countless test automation efforts have been abandoned and cost their projects dearly when they went astray. Realities of Using Test Tools and AutomationYou should consider these important issu

27、es before you begin using the techniques described in this chapter:The software changes. Theres no substitute for the human eye and intuition. Verification is hard to do. Its easy to rely on automation too much.Dont spend so much time working on tools and automation that you fail to test the softwar

28、e. If youre writing macros, developing a tool, or programming a monkey, youre doing development work. Some tools are invasive and can cause the software being tested to improperly fail. Highlights of this chapter includeWhy test tools and automation are necessaryExamples of simple test tools you can

29、 useHow using tools migrates to test automationHow to feed and care for monkeysWhy test tools and automation arent a panaceaPart IV: Supplementing Your Testing Bug Bashes and Beta Testing Bug Bashes and Beta TestingHaving Other People Test Your SoftwareTest SharingBeta TestingOutsourcing Your Testin

30、gHaving Other People Test Your SoftwareYoure likely under a tight schedule, you find as many bugs as possible in the time you have, but someone else can come in, test the same code, and find additional bugs. There are several reasons and solutions for it: Having another set of eyes look at the softw

31、are helps break the pesticide paradox. Similarly, people dont just see differently from each other, they go about their testing differently, too. Having someone assist you in your testing helps eliminate boredom. Watching how someone else approaches a problem is a great way to learn new testing tech

32、niques.Test Sharing One common approach is to simply swap test responsibilities with another tester for a few hours or a few days. Think of it as You run my tests and Ill run yours. A fun way to share the testing tasks is to schedule a bug bash. A bug bash is a period of time (usually a couple hours

33、) in which the entire test team stops their normally assigned testing tasks to participate in the bash. In a bug bash, a specific area of the software is selected and all the testers concentrate their testing on that one area or set of features. A bug bash could determine if the bugs have been misse

34、d by normal testing or if its just well written code. Its a good idea to get your product support team helping you test the product early in the design cycle to help identify and fix usability bugs. Beta TestingBeta testing is the term used to describe the external testing process in which the softw

35、are is sent out to a select group of potential customers who use it in a real-world environment. Beta testing usually occurs toward the end of the product development cycle and ideally should just be a validation that the software is ready to release to real customers.Beta TestingThere are several t

36、hings to think about when planning for or relying on a beta test:Who are the beta testers? Similarly, how will you know if the beta testers even use the software? Beta tests can be a good way to find compatibility and configuration bugs. Usability testing is another area that beta testing can contri

37、bute to if the participants are well chosen a good mix of experienced and inexperienced users. Besides configuration, compatibility, and usability, beta tests are surprisingly poor ways to find bugs. A beta test program can take up a lot of a testers time. Outsourcing Your Testing Configuration and compatibility testing are typically good choices for outsourcing. Localizatio