i2 Integrated Testing Strategy [IBM—华为供应链全套方案]

1、 2001 i2 technologies, proprietary and confidential huawei technologies huawei aps project integrated testing strategy revision 1.0 / 4-september-2002 overall solution architecturehuawei aps project 2001 i2 technologies, proprietary and confidential table of contents purpose. 3 introduction. 3 scope

2、. 4 functions/scenarios. 5 architecture. 10 test environment requirements.10 demand planning to forecast netting.10 forecast netting to s verify the readiness of application for system integration verify that data has been migrated as per the defined mappings system integration to test the coexisten

3、ce of products and applications that are required to perform together in the production-like operational environment (hardware, software, network) to ensure that the system functions together with all the components of its environment as a total system to ensure that the system releases can be deplo

4、yed in the current environment verify the ability of the application to be integrated with other systems and modules user acceptance to ensure the system meets the business user requirements and business processes verify the integrated system meets business requirements including service levels oper

5、ations acceptance to ensure the product can operate in the production environment to ensure the product meets the acceptable verify that the application can operate in the production environment. operability tests will be concurrent with, user overall solution architecturehuawei aps project page 16

6、of 28 levelfocusbenefit level of service as per service level agreement to ensure the product operates as stated in the operations standards to ensure the system can be recovered / restarted as per standards to ensure that unix scripts are as per standard acceptance tests. for the aps br1 implementa

7、tion project the following levels of test will be required: levels of testwhere needed unit test all data migration components all components where code has been changed integration test all separate and unique functions and transactions within the data migration suite all customized transactions an

8、d functions within the oracle modules all type and 2 systems that required change to functions and transactions systems integration test (sit) the integrated suite of implemented oracle 11i modules oracle 11i and all of the interfacing type 1, 2 and 3 applications user acceptance test (uat) the tota

9、l solution plus desktop procedures, business processes scope of test levels and test types one of the key principle behind all tests is to ensure that tests are performed as early as possible so that errors can be corrected early which in turn reduces the cost and time required for fixes. given that

10、 principle, the following types of test should be included in the scope of each level of test. levels types unitintegrationsituat audit & control conversion/migration documentation and procedures error-handling function installation overall solution architecturehuawei aps project page 17 of 28 level

11、s types unitintegrationsituat interface/inter-system regression transaction flow usability backup and recovery contingency disaster recoveryn/a job stream operational performance security stress/volume test phases the current approach to sit and uat is based on the criticality of the modules for go

12、live. the following diagram indicates the different phases of test and the occurrences. 8/26 9/2 9/9 9/16 9/23 9/30 10/7 10/14 10/21 10/28 11/4 11/11 11/18 11/25 12/2 12/9 12/16 12/23 12/301/6 1/13 1/20 1/27 2/3 dpdpdpdp fnfnfnfndpdpdpdp fpfpfpfpfnfnfnfn dfdfdffpfpfpfp sccscc sccdfdfdfdf mpmpmpsccsc

13、cscc scc mpmpmpmp dpdpdpdpdpdpdpdpdp fnfnfnfnfnfnfnfnfn fpfpfpfpfpfpfpfpfp dfdfdfdfdfdfdfdf sccscc scc sccsccsccscc scc mpmpmpmpmpmpmpmp erpaps 2003 develop integration programs 2002 unit testing eu material prep eu training uat c3uat c2uat c1 data prep extended sit limited uatfull uat core sit envi

14、ronment aps uat 2 aps uat 1 erp uat c3 erp sit erp uat c1 aps sit 1 erp uat c2 aps sit 2 overall solution architecturehuawei aps project page 18 of 28 test responsibilities the following chart identifies the groups responsible for performing the testing activities at each level of test: level of tes

15、ttest activity responsibilities planningpreparationexecutionreporting unit module team integration team module team integration team module team integration team module team integration team integrationmodule team integration team module team integration team module team integration team module team

16、 integration team systems integration test acceptance team integration team erp & satellite application teams module team key users data team acceptance team integration team erp & satellite application teams module team key users data team acceptance team integration team erp & satellite applicatio

17、n teams module team key users data team acceptance team, integration team erp & satellite application teams module team key users data team user acceptance test acceptance team integration team erp & satellite application teams module team end users representatives data team acceptance team integrat

18、ion team erp & satellite application teams module team end users representatives data team acceptance team integration team erp & satellite application teams module team end users representatives data team acceptance team integration team erp & satellite application teams module team end users repre

19、sentatives data team testing process the basic testing process consists of four basic steps. they are: plan for the tests prepare for tests execute the tests report the results testing is an iterative process. these steps are followed at the overall project level and repeated for each level of testi

20、ng required in the development life cycle. overall solution architecturehuawei aps project page 19 of 28 test planning planning in itself is a process. performing each step of the planning process will insure that the plan is built systematically and completely. documenting these steps will ensure t

21、hat the plan is itself testable by others who have to approve it. during test planning activities, the test focus, test levels and test types specific to the application module being tested will be identified and a testing strategy developed that is aligned with this test strategy and the needs/risk

22、s of the specific application module. the testing will be based on the business and technical requirements for the application. testing techniques and processes there are several testing techniques and processes that will be used. these include: review procedures, code walkthrough procedures, test c

23、ase design, test results evaluations, problem/variances tracking, problem/change management, configuration management of test objects, migration to user acceptance test environment, acceptance of the tested system. some of these processes are defined through the project control office, while others

24、are specific to testing and will be defined and documented by the integration/acceptance team. test documentation and approvals there are several documents that together constitute the test documentation. the detailed test plan for each module for sit or uat will identify the documents required for

25、the project and will identify the acceptor for each document. the project deliverables will require formal acceptance, while the intermediate work products will require a less formal acceptance. the acceptance test cases is the most critical of the work products, as it forms the basis for the user a

26、cceptance testing. this will require input from the key users in their development and formal signoff of the results once the test cases have been executed and results reviewed. examples of other documentation that may be required are test log, fault (problem/ variance) log, backup/restore log, test

27、 change log, and deliverable acceptance forms. test preparation the test design must consider whether the tests involve on-line or batch systems, and whether they are input or output driven. test cases would be built to accommodate these system characteristics. another consideration to take into acc

28、ount will be the grouping of valid and invalid test conditions. the integration approach (top down, bottom up, or a combination of both) can sometimes influence the test case design. it is recommended that the development of regression test packages be considered part of the test design. test cases

29、should always be built for reuse. test case design strategy entails the following steps: overall solution architecturehuawei aps project page 20 of 28 examine the development / testing approach. consider the type of processing - whether on-line, batch, conversion program etc. determine the technique

30、s to be used - white box / black box, and error guessing. develop the test conditions. develop the test cases. create the test script. define the expected results. develop the procedures and data (prerequisites, steps, expectations, and post-test steps). in developing a list of conditions that shoul

31、d be tested, it is useful to have a list of standard conditions to test for some common application types. test data preparation and management test data set up is a very tedious and time-consuming process. after the test case design has been selected, tester will have to consider all the available

32、sources of data and build new or modify existing test data. frequently, the testers tend to look at the functional specifications and set up data to test the specifications. this tends to overlook how the software will be used. it is therefore important that the production data are analyzed to under

33、stand the types, frequency and characteristics of data so that they can be simulated during testing. during the test preparation process consideration may need to be given to the following sources while setting up test data: production data data from previous testing data from centralized test beds

34、(if available) data used in other systems (for interface testing) in many situations, data from the selected source(s) have to be supplemented to cover additional conditions and cases. when setting up effective test data, the goal is to provide adequate requirements and conditions coverage. consider

35、ation should be given to some of each of the following types: frequently occurring types and characteristics of data that have high risk. for example, deposit or withdrawal type of transactions at a banking terminal. frequently occurring types of data with very little exposure. for example, maintena

36、nce- type of transactions such as name and address changes. low frequency errors that have very little consequence. overall solution architecturehuawei aps project page 21 of 28 low frequency errors resulting in heavy losses. for example, the printing of a few additional zeros in a financial report.

37、 once the data source has been identified, testers have to determine the files and sizes of files to use. data is then extracted using the appropriate methods. the key to successful testing is to state the expected results when defining the test conditions and documenting the test scripts. documenta

38、tion should include: what is being tested? how is testing done (procedures)? where to find the test data? the expected results. test data management will be critical during the integration, sit and uat phases of the project. the module teams will define prerequisite data for test cases as part of th

39、e test preparation process. it is critical that this data is created and stored in a database that is quarantined from any other data and accessible only by systems test. a procedure to backup and restore this data to multiple base points will require to be implemented. the module team will require

40、adding to this data, deleting from the base data, and modifying any of the data elements as required. configuration management it is essential that there is an adequate configuration management process adopted for this entire project. items under test must be isolated from the development environmen

41、t and fully documented as to their content and configuration. code should be released from development with fully documented release notes that identify not only the content of the release, but also any known variances to behavior, defect fixes or change control items and any known issues that might

42、 affect testing. test training application training training is required to familiarize the testers with the various application system components prior to test case and data preparation. these requirements will be identified and communicated in the detailed test plans. test process education approp

43、riate test process education will be provided to the test teams for the purposes of test design preparation and execution. the isc acceptance/integration manager is responsible to ensure that all test team members have undertaken the necessary test process education. overall solution architecturehua

44、wei aps project page 22 of 28 risk assessment risk assessment a part of the test planning process. the reason for risk analysis is to gain an understanding of the potential sources and causes of failure and their associated costs. measuring the risk prior to testing can help the process in two ways:

45、 high-risk applications can be identified and more extensive testing can be performed. risk analysis can help draw attention to the critical components and/or focus areas for testing that are most important from the system or user standpoint. in general, the greater the potential cost of an error in

46、 a system, the greater the effort and the resources assigned to prevent these errors from occurring. the table below lists the risks that have been identified at this stage of the testing process. further risks, as identified, will be documented in the detailed test plans and managed in the same man

47、ner as the risk management process defined for the project. riskprobabilityimpactmitigation completion of all specifications in time to allow test scope to be defined 33attempts must be made to ensure specifications are signed off in a timely manner acceptance of all tests levels prior to uat. 15acc

48、eptance to be completed after each test stage. key user staff not available during key testing activities. 55huawei must ensure staff available. if not agree that the testing and acceptance will be conducted by current sme who is within the project. staff not available for integration and system int

49、egration testing 55huawei must allocate more staff to assist the smes should their tasks conflict with testing preparation and execution. not all tests cases defined in the inventory will be completed. 33each test case will be prioritized and risk assessment completed performance testing will be the

50、 first time that transactions are tested with near live frequency and volumes of data. 55a mechanism will need to be defined to allow for these functional errors to be passed back to the teams responsible for taking corrective action. business processes are finalizes to allow test scenarios to be de

51、fined 45huawei and the deployment teams must focus on the finalization of business processes to allow the uat detailed test planning to occur in a timely manner key oracle application decisions are signed off prior to testing activities e.g. multiple organizations, chartered accounts 33huawei must m

52、ake a decision on these areas and signoff. no changes will be allowed once development commences. delivery of solution from the satellite systems to meet the scheduled rest cycles of sit 45it is essential that detailed monitoring of the development of satellite systems is undertaken to ensure that t

53、he sit test schedules are not compromised overall solution architecturehuawei aps project page 23 of 28 test levels - entry and exit criteria the table below lists the entry and exit criteria at a high level for each test level. detail entry and exit criteria will be defined in the detailed test pla

54、ns and in the test cases. testing levelentry criteriaexit criteria unit test cases internal application design 100% of allocated test cases completed no outstanding severity 1 problems log of problems integration test plan test cases test scripts detailed requirements external application design uni

55、t testing completed 100% of allocated test cases completed no outstanding severity 1 problems log of problems test report system integration (sit) test plan test cases test scripts detailed requirements external application design integration testing completed 100% of allocated test cases completed

56、no outstanding severity 1 problems log of problems action plan to correct remaining problems test report user acceptance (uat) test plan test cases test scripts business procedures / workflow system integration testing completed 100% of allocated test cases completed no outstanding severity 1 and 2

57、problems log of outstanding problems test report suspension criteria and resumption requirements suspension criteria suspension of testing may occur if the number or severity of defects reaches a stage that further testing cannot continue or is compromised to the extent that re-testing of entire are

58、as of functionality will be required. the criteria for determining this are: there are no areas of application functionality unaffected by defects severity 1 defects prevent further testing suspension of testing will be need to be determined by the acceptance manager and recommended to the project m

59、anager for approval. at this point, a risk analysis may be required to determine if testing should be suspended. overall solution architecturehuawei aps project page 24 of 28 resumption criteria testing may be recommenced after a period of suspension when: sufficient defect fixes have occurred so th

60、at testing can continue with out compromising the integrity of the test results the defective application functionality preventing further testing has been fixed test tools & metrics during the course of test planning consideration should be given to the use of tools that will allow the development,