创业管理pa(1)

1、.Part IV: Supplementing Your Testing .nAutomated Testing and Test ToolsnBug Bashes and Beta Testing.Automated Testing and Test Tools.nThe Benefits of Automation and ToolsnTest ToolsnSoftware Test AutomationnRandom Testing: Monkeys and GorillasnRealities of Using Test Tools and Automation.nBackground

2、nRegression testing: This process of rerunning your tests is known as regression testing. nThe work of regression testing is enormousnSoftware test tools and automation can help solve this problem by providing a more efficient means to run your tests than by manual testing. .nThe principal attribute

3、s of tools and automation arenSpeed. nEfficiency. nAccuracy and Precision. nResource Reduction. nSimulation and Emulation. nRelentlessness. nNOTEnSoftware test tools arent a substitute for software testers they just help software testers perform their jobs better.nAs a software tester youll be expos

4、ed to a wide range of testing tools. The types of tools that youll use are based on the type of software that youre testing and whether youre performing black-box or white-box tests.nImportant distinction between two types of tools non-invasive and invasive. nIf a tool is used only to monitor and ex

5、amine the software without modifying it, its considered non-invasive. nIf, however, the tool modifies the program code or manipulates the operating environment in any way, its invasive. .nThe major classes of testing tools and how theyre used. nViewers and Monitors nA viewer or monitor test tool all

6、ows you to see details of the softwares operation that you wouldnt normally be able to see. nCode coverage analyzers, communications analyzer, the code debuggers that come with most compilers. .nDrivers nDrivers are tools used to control and operate the software being tested. nBatch file, replace th

7、e keyboard and mouse of the system being tested with an additional computer. nStubs nStubs, like drivers, were mentioned in Chapter 7 as white-box testing techniques. nEmulator.nStress and Load Tools nThe Stress program allows you to individually set the amounts of memory, disk space, files, and oth

8、er resources available to the software running on the machine. nLoad tools are similar to stress tools in that they create situations for your software that might otherwise be difficult to create. nInterference Injectors and Noise GeneratorsnTheyre similar to stress and load tools but are more rando

9、m in what they do. .nAnalysis Tools nWord processing softwarenSpreadsheet softwarenDatabase softwarenFile comparison softwarenScreen capture and comparison softwarenDebuggernBinary-hex calculatornStopwatchnVCR or camera.nMacro Recording and PlaybacknThe most basic type of test automation is as you r

10、un recording your keyboard and mouse actions your tests for the first time and then playing them back when you need to run them again. nMacro recorders and players are a type of driver tool. Drivers are tools used to control and operate the software being tested. nOn the Mac you can use QuicKeys; on

11、 Windows the shareware program Macro Magic is a good choice. nBe careful: nThe biggest problem is lack of verification. nA slight change in the GUI could throw off your captured steps. .nProgrammed Macros nCreate programmed macros by programming simple instructions for the playback system to follow.

12、 nProgrammed macros such as this one have some real advantages over recorded macros. Although they still cant perform verification of the test results, they can pause their execution to prompt the tester (see ) with an expected result and a query for her to okay whether the test passed or failed. .n

13、Programmed Macros (continue)nListing 15.1. A Simple Macro That Performs a Test on the Windows Calculatorn1: Calculator Test #2n2:n3: n4: 123-100=n5: n6: .nProgrammed Macros (continue)nYoure still missing two important pieces, though, to perform complex testing.nProgrammed macros are limited to strai

14、ght-line execution they can only loop and repeat. Variables and decision statements that youd find in a regular programming language arent available. nYou also dont have the ability to automatically check the results of your test. For these, you need to move to a comprehensive automated testing tool

15、.nFully Programmable Automated Testing Tools nThe most important feature that comes with these automation tools is the ability to perform verification, actually checking that the software is doing whats expected. There are several ways to do this: nScreen captures. nControl values. nFile and other o

16、utput. .nFully Programmable Automated Testing Tools (continue)nTo get more information about several of the popular test automation products available, visit the following websites:nSoftware Development Technologies at nMercury at nSegue Software at .nAnother type of automated testing, though, isnt

17、designed to help run or automatically run test cases. Its goal is to simulate what your users might do. That type of automation tool is called a test monkey. nWhen your software is released to the public, it will have thousands or possibly millions of people using it. Despite your best efforts at de

18、signing test cases to find bugs, some bugs will slip by and be found by those users. .nDumb MonkeysnIt just clicks or types randomly. n例：1：RANDOMIZE TIMERn 2: FOR i=1 TO 10000n 3: PLAY “CLICK”+STR$(INT(RND*640)+”,”+STR$(INT(RND*480)+”n 4:PLAY CHR$(RND*256)n 5:NEXT i .nDumb MonkeysnIt doesnt seem to

19、make sense that simple random clicking and typing could find a bug, but it does for a couple reasons:nGiven enough time and attempts, just like the monkeys writing Shakespeare, the random inputs will eventually stumble onto a magic sequence that the programmers and testers didnt think of. nA dumb mo

20、nkey, with its continuous repetition and use, can expose bugs such as memory leaks that might not occur until many hours or days of normal use.nSemi-Smart Monkeys nDumb monkeys lack a few important features （如故障的重现）, though, that would make them even more effective. Adding these features （如日志或屏幕录像 ）

21、raises your monkeys IQ a bit, making him semi-smart. nThe solution is to add logging to your monkey so that everything it does is recorded to a file. When the monkey finds a bug, you need only to look at the log file to see what it was doing before the failure. nIts also a good idea to program your

22、monkey to operate only on the software youre testing. nAnother good feature to make your monkey semi-smart is crash recognition. .nSmart Monkeys nSuch a monkey takes the effectiveness of random testing from his less-intelligent brothers and adds to that an awareness of his surroundings. He doesnt ju

23、st pound on the keyboard randomly he pounds on it with a purpose. nA smart monkey can examine data as it goes, checking the results of its actions and looking for differences from what it expects. If you programmed in your test cases, the smart monkey could randomly execute them, look for bugs, and

24、log the results. .nTest automation isnt a panacea. When it is properly planned and executed it can make your testing immensely more efficient and find bugs that would have otherwise gone undiscovered. nHowever, countless test automation efforts have been abandoned and cost their projects dearly when

25、 they went astray. .nYou should consider these important issues before you begin using the techniques described in this chapter:nThe software changes. nTheres no substitute for the human eye and intuition. nVerification is hard to do. nIts easy to rely on automation too much.nDont spend so much time

26、 working on tools and automation that you fail to test the software. nIf youre writing macros, developing a tool, or programming a monkey, youre doing development work. nSome tools are invasive and can cause the software being tested to improperly fail. .nHighlights of this chapter includenWhy test

27、tools and automation are necessarynExamples of simple test tools you can usenHow using tools migrates to test automationnHow to feed and care for monkeysnWhy test tools and automation arent a panacea.Bug Bashes and Beta Testing .nHaving Other People Test Your SoftwarenTest SharingnBeta TestingnOutso

28、urcing Your Testing.nYoure likely under a tight schedule, you find as many bugs as possible in the time you have, but someone else can come in, test the same code, and find additional bugs. nThere are several reasons and solutions for it: nHaving another set of eyes look at the software helps break

29、the pesticide paradox. nSimilarly, people dont just see differently from each other, they go about their testing differently, too. nHaving someone assist you in your testing helps eliminate boredom. nWatching how someone else approaches a problem is a great way to learn new testing techniques.nOne c

30、ommon approach is to simply swap test responsibilities with another tester for a few hours or a few days. Think of it as You run my tests and Ill run yours. nA fun way to share the testing tasks is to schedule a bug bash. A bug bash is a period of time (usually a couple hours) in which the entire te

31、st team stops their normally assigned testing tasks to participate in the bash. In a bug bash, a specific area of the software is selected and all the testers concentrate their testing on that one area or set of features. nA bug bash could determine if the bugs have been missed by normal testing or

32、if its just well written code. nIts a good idea to get your product support team helping you test the product early in the design cycle to help identify and fix usability bugs. .nBeta testing is the term used to describe the external testing process in which the software is sent out to a select grou

33、p of potential customers who use it in a real-world environment. nBeta testing usually occurs toward the end of the product development cycle and ideally should just be a validation that the software is ready to release to real customers.nThere are several things to think about when planning for or

34、relying on a beta test:nWho are the beta testers? nSimilarly, how will you know if the beta testers even use the software? nBeta tests can be a good way to find compatibility and configuration bugs. nUsability testing is another area that beta testing can contribute to if the participants are well c

35、hosen a good mix of experienced and inexperienced users. nBesides configuration, compatibility, and usability, beta tests are surprisingly poor ways to find bugs. nA beta test program can take up a lot of a testers time. .nConfiguration and compatibility testing are typically good choices for outsourcing. nLocalization testing is another example that often lends itself