软件测试技术：part 2-blackbox

1、Part II: Testing Fundamentals farucalgary.caSoftware Testing ContentsTypes of software testingExamining the SpecificationTesting the Software with Blinders OnExamining the CodeTesting the Software with X-Ray GlassesPart II: Testing Fundamentals Types of software testingTypes of testingOne possible c

2、lassification is based on the following four classifiers:C1: Source of test generation. C2: Lifecycle phase in which testing takes placeC3: Goal of a specific testing activityC4: Characteristics of the artifact under testC1: Source of test generationC2: Lifecycle phase in which testing takes placeC3

3、: Goal of specific testing activityC4: Artifact under testPart II: Testing Fundamentals Examining the Specification ContentsGetting StartedPerforming a High-Level Review of the SpecificationLow-Level Specification Test TechniquesGetting StartedThe Software Development Process: big-bang, code-and-fix

4、, waterfall, and spiral. In each model, except big-bang, the development team creates a product specification from the requirements document to define what the software will become.The only way to assure the end product which is what the customer required and to plan the test effort （测试投入/测试工作）prope

5、rly is to thoroughly describe the product in a specification.Getting StartedAn excerpt from the Windows Calculator (see Figure 4.1) :The Edit menu will have two selections: Copy and Paste. These can be chosen by one of three methods: pointing and clicking to the menu items with the mouse, using acce

6、ss-keys (Alt+E and then C for Copy and P for Paste), or using the standard Windows shortcut keys of Ctrl+C for Copy and Ctrl+V for Paste.The Copy function will copy the current entry displayed in the number text box into the Windows Clipboard. The Paste function will paste the value stored in the Wi

7、ndows Clipboard into the number text box.As a tester youll also have a document as a testable item. You can use it to find bugs before the first line of code is written.Getting StartedBlack-Box and White-Box TestingBlack-box testing is sometimes referred to as functional testing or behavioral testin

8、g.white-box testing is sometimes called clear-box testingGetting StartedStatic and Dynamic TestingStatic testing refers to testing something thats not running examining and reviewing it.Dynamic testing is what you would normally think of as testing running and using the software.Getting StartedStati

9、c Black-Box Testing: Testing the SpecificationYou can test a specification with static black-box techniques no matter what the format of the specification. It can be a written or graphical document or a combination of both. You can even test an unwritten specification by questioning the people who a

10、re designing and writing the software.（没有specification）Performing a High-Level Review of the Specification The first step in testing the specification isnt to jump in and look for specific bugs. The first step is to stand back and view it from a high level. Examine the spec for large fundamental pro

11、blems, oversights, and omissions. Performing a High-Level Review of the SpecificationPretend to Be the CustomerIts important to understand the customers expectations. Remember that the definition of quality means meeting the customers needs. As a tester, you must understand those needs to test that

12、the software meets them. Dont forget about software security when pretending to be the customer. Performing a High-Level Review of the SpecificationResearch Existing Standards and GuidelinesThe difference between standards and guidelines is a matter of degree. A standard is much more firm than a gui

13、deline. Guidelines are optional but should be followed.Corporate Terminology and Conventions. Industry Requirements. Government Standards. Graphical User Interface (GUI). Security Standards.Performing a High-Level Review of the SpecificationReview and Test Similar SoftwareOne of the best methods for

14、 understanding what your product will become is to research similar software. This could be a competitors product or something similar to what your team is creating.Some things to look for when reviewing competitive products includeScale. Complexity. Testability. Quality/Reliability. Security.Low-Le

15、vel Specification Test Techniques Specification Attributes ChecklistA good, well-thought-out product specification, with all its ts crossed and its is dotted, has eight important attributes:Complete. Accurate. Precise, Unambiguous, and Clear. Consistent. Relevant. Feasible. Code-free. Testable. Low-

16、Level Specification Test TechniquesSpecification Terminology ChecklistThe spec may go on to clarify or elaborate on them, or it may leave them ambiguous in which case, youve found a bug.Always, Every, All, None, Never. Certainly, Therefore, Clearly, Obviously, Evidently. Some, Sometimes, Often, Usua

17、lly, Ordinarily, Customarily, Most, Mostly. Etc., And So Forth, And So On, Such As. Good, Fast, Cheap, Efficient, Small, Stable. Handled, Processed, Rejected, Skipped, Eliminated. IfThen(but missing Else). Highlights of this chapter includeWhat is black-box and white-box testingHow static and dynami

18、c testing differWhat high-level techniques can be used for reviewing a product specificationWhat specific problems you should look for when reviewing a product specification in detailPart II: Testing Fundamentals Testing the Software with Blinders OnDynamic Black-Box TestingYoure entering inputs, re

19、ceiving outputs, and checking the results. Another name commonly used for dynamic black-box testing is behavioral testing because youre testing how the software actually behaves when its used.To do this effectively requires some definition of what the software does namely, a requirements document or

20、 product specification. Once you know the ins and outs of the software youre about to test, your next step is to start defining the test cases.Contents Dynamic Black-Box Testing: Testing the Software While BlindfoldedTest-to-Pass and Test-to-FailEquivalence PartitioningBoundary value analysisDecisio

21、n Table Based testingTest generation from predicatesState TestingOther Black-Box Test TechniquesDynamic Black-Box TestingTest cases are the specific inputs that youll try and the procedures that youll follow when you test the software.Selecting test cases is the single most important task that softw

22、are testers do. Improper selection can result in testing too much, testing too little, or testing the wrong things. Intelligently weighing the risks and reducing the infinite possibilities to a manageable effective set is where the magic is.Test-to-Pass and Test-to-FailThere are two fundamental appr

23、oaches to testing software: test-to-pass and test-to-fail. When you test-to-pass, you really assure only that the software minimally works. Designing and running test cases with the sole purpose of breaking the software is called testing-to-fail or error-forcing. A common class of test cases is one

24、that attempts to force error messages. You know the ones like saving a file to a floppy disk but not having one inserted in the drive. These cases actually straddle the line between test-to-pass and test-to-fail. Equivalence class partitioningTest selection using equivalence partitioning allows a te

25、ster to subdivide the input domain into a relatively small number of sub-domains, say N1, as shown (next slide (a). In strict mathematical terms, the sub-domains by definition are disjoint. The four subsets shown in (a) constitute a partition of the input domain while the subsets in (b) are not. Eac

26、h subset is known as an equivalence class. Equivalence partitioningSubdomainsThe equivalence classes are created assuming that the program under test exhibits the same behavior on all elements, i.e. tests, within a class. This assumption allow the tester to select exactly one test from each equivale

27、nce class resulting in a test suite of exactly N tests. Program behavior and equivalence classesThe entire set of inputs to any application can be divided into at least two subsets: one containing all the expected, or legal, inputs (E) and the other containing all unexpected, or illegal, inputs (U).

28、 Each of the two subsets, can be further subdivided into subsets on which the application is required to behave differently (e.g. E1, E2, E3, and U1, U2). Faults targetedEquivalence class partitioning selects tests that target any faults in the application that cause it to behave incorrectly when th

29、e input is in either of the two classes or their subsets. Faults targeted (cont.)Consider an application A that takes an integer denoted by age as input. Let us suppose that the only legal values of age are in the range 1.120. The set of input values is now divided into a set E containing all intege

30、rs in the range 1.120 and a set U containing the remaining integers. All integers1.120Other integersExample 1Further, assume that the application is required to process all values in the range 1.61 in accordance with requirement R1 and those in the range 62.120 according to requirement R2. Thus E is

31、 further subdivided into two regions depending on the expected behavior. Similarly, it is expected that all invalid inputs less than or equal to 1 are to be treated in one way while all greater than 120 are to be treated differently. This leads to a subdivision of U into two categories. Example 1 (c

32、ontd.)All integers62-1201.61120Example 1 (contd.)It is expected that any single test selected from the range 1.61 will reveal any fault with respect to R1. Similarly, any test selected from the region 62.120 will reveal any fault with respect to R2. A similar expectation applies to the two regions c

33、ontaining the unexpected inputs.Tests selected using the equivalence partitioning technique aim at targeting faults in the application under test with respect to inputs in any of the four regions, i.e. two regions containing expected inputs and two regions containing the unexpected inputs. Example 1

34、 (contd.)The effectiveness of tests generated using equivalence partitioning for testing application A, is judged by the ratio of the number of faults these tests are able to expose to the total faults lurking in A. As is the case with any test selection technique in software testing, the effectiven

35、ess of tests selected using equivalence partitioning is less than 1 for most practical applications. The effectiveness can be improved through an unambiguous and complete specification of the requirements and selected tests carefully using the equivalence partitioning technique described in the foll

36、owing sections. EffectivenessConsider that wordCount method takes a word w and a filename f as input and returns the number of occurrences of w in the text contained in the file named f. An exception is raised if there is no file with name f. This example shows a few ways to define equivalence class

37、es based on the knowledge of requirements and the program text.Example 2beginString w, fInput w, fif (not exists(f) raise exception; return(0); / 异常if(length(w)=0)return(0);if(empty(f)return(0);return(getCount(w,f);endUsing the partitioning method described in the examples above, we obtain the follo

38、wing equivalence classes.Example 2 (contd.)Equivalence classwfE1non-nullexists, not emptyE2non-nulldoes not existE3non-nullexists, emptyE4nullexists, not emptyE5nulldoes not existE6nullexists, emptyExample 2 (contd.)Note that the number of equivalence classes without any knowledge of the program cod

39、e is 2, whereas the number of equivalence classes derived with the knowledge of partial code is 6. Of course, an experienced tester will likely derive the six equivalence classes given above, and perhaps more, even before the code is available Example 2 (contd.)In some cases the equivalence classes

40、are based on the output generated by the program. For example, suppose that a program outputs an integer. It is worth asking: “Does the program ever generate a 0? What are the maximum and minimum possible values of the output?”These two questions lead to two the following equivalence classes based o

41、n outputs:Equivalence classes based on program outputE1: Output value v is 0.E2: Output value v is the maximum possible. E3: Output value v is the minimum possible. E4: All other output values. Based on the output equivalence classes one may now derive（源于） equivalence classes for the inputs. Thus ea

42、ch of the four classes given above might could lead to one equivalence class consisting of inputs.Equivalence classes based on program output (contd.)Equivalence classes for variables: rangeEq. ClassesExampleOne class with values inside the range and two with values outside the range.speed 60.9050,

43、75, 92area: floatarea0.0-1.0, 15.52age: int-1, 56, 132letter : boolJ, 3ConstraintsClassesEq. ClassesExampleAt least one containing all legal strings and one all illegal strings based on any constraints.firstname: string, Sue, Loooong NameConstraintsClassesEquivalence classes for variables: stringsEq

44、. ClassesExampleEach value in a separate classautocolor:red, blue, greenred, blue, greenup:booleantrue, falseConstraintsClassesEquivalence classes for variables: enumerationEq. ClassesExampleOne class containing all legal arrays, one containing the empty array, and one containing a larger than expec

45、ted aName: new int3; , -10, 20, -9, 0, 12, 15ConstraintsClassesEquivalence classes for variables: arraysArrays in Java and records, or structures, in C+, are compound types. Such input types may arise while testing components of an application such as a function or an object. While generat

46、ing equivalence classes for such inputs, one must consider legal and illegal values for each component of the structure. The next example illustrates the derivation of equivalence classes for an input variable that has a compound type.Equivalence classes for variables: compound data typestruct trans

47、criptstring fName; / First name.string lName; / Last name.string cTitle 200; / Course titles.char grades 200; / Letter grades corresponding to course titles. In-class exercise: Derive equivalence classes for each component of R and combine them!Equivalence classes for variables: compound data type:

48、ExampleOne way to partition the input domain is to consider one input variable at a time. Thus each input variable leads to a partition of the input domain. We refer to this style of partitioning as unidimensional equivalence partitioning or simply unidimensional partitioning. This type of partition

49、ing is commonly used.Unidimensional partitioningAnother way is to consider the input domain I as the set product of the input variables and define a relation on I. This procedure creates one partition consisting of several equivalence classes. We refer to this method as multidimensional equivalence

50、partitioning or simply multidimensional partitioning. Multidimensional partitioning leads to a large number of equivalence classes that are difficult to manage manually. Many classes so created might be infeasible.（不可行的） Nevertheless, equivalence classes so created offer an increased variety of test

51、s as is illustrated in the next section. Multidimensional partitioningConsider an application that requires two integer inputs x and y. Each of these inputs is expected to lie in the following ranges: 3 x7 and 5y9. For unidimensional partitioning we apply the partitioning guidelines to x and y indiv

52、idually. This leads to the following six equivalence classes.Partitioning ExampleE1: x7y ignored.E4: y9x ignored.For multidimensional partitioning we consider the input domain to be the set product X x Y. This leads to 9 equivalence classes.Partitioning Example (contd.)E1: x3, y5 E2: x3, 5y9E3: x9E4

53、: 3x7, y9E7: 7, y7, 5y9E9: x7, y9Partitioning Example (contd.)6 equivalence classes:9 equivalence classes:E1: x3, y5 E2: x3, 5y9E3: x9E4: 3x7, y9E7: 7, y7, 5y9E9: x7, y9Partitioning Example (contd.)1. Identify the input domain: Read the requirements carefully and identify all input and output variab

54、les, their types, and any conditions associated with their use. Environment variables, such as class variables used in the method under test and environment variables in Unix, Windows, and other operating systems, also serve as input variables. Given the set of values each variable can assume, an ap

55、proximation to the input domain is the product of these sets.Systematic procedure for equivalence partitioning2. Equivalence classing: Partition the set of values of each variable into disjoint subsets. Each subset is an equivalence class. Together, the equivalence classes based on an input variable

56、 partition of the input domain. partitioning the input domain using values of one variable, is done based on the expected behavior of the program. Values for which the program is expected to behave in the “same way” are grouped together. Note that “same way” needs to be defined by the tester. System

57、atic procedure for equivalence partitioning (contd.)The equivalence classes are combined using the multidimensional partitioning approach described earlier.3. Combine equivalence classes: This step is usually omitted and the equivalence classes defined for each variable are directly used to select t

58、est cases. However, by not combining the equivalence classes, one misses the opportunity to generate useful tests. Systematic procedure for equivalence partitioning (contd.)For example, suppose that an application is tested via its GUI, i.e. data is input using commands available in the GUI. The GUI

59、 might disallow invalid inputs by offering a palette 调色板 of valid inputs only. There might also be constraints in the requirements that render certain equivalence infeasible. 需求中定义了一些不可实现的等价类（输入）4. Identify infeasible equivalence classes: An infeasible equivalence class is one that contains a combin

60、ation of input data that cannot be generated during test. Such an equivalence class might arise due to several reasons. Systematic procedure for equivalence partitioning (contd.)Command temp causes CS to ask the operator to enter the amount by which the temperature is to be changed (tempch). Values