完整word版语言测试学资料4word文档良心出品

1、Chapter 4（第四章）The Steps of Language Testing语言测试的设计步骤Statement of the problemProviding a solution to the problemValidation of the testStatement of the problemWhat s the essential first step in testing?To make oneself perfectly clear about what it is one wants to know and for what purpose.When we are

2、going to administer a test, we have to answer the following questions:What kind of test is it to be? Achievement (final or progress),proficiency, diagnostic, or placement?What is its precise purpose?What abilities are to be tested?How detailed must the results be?How accurate must the results be?How

3、 important is backwash?What constraints are set by unavailability of expertise, facilities, time (for construction, administration and scoring)?Providing a solution to the problemWriting specifications for the testThe first form that the solution takes is a set of specifications for the test.This wi

4、ll include information on: content, format and timing, criterial levels of performance, and scoring proceduresCONTENTThis refers not to the content of a single, particular version of a test, but to the entire potential content of any number of versions. Samples of this content will appear in individ

5、ual versions of the test.The fuller the information on content, the less arbitrary should be the subsequent decisions as to what to include in the writing of any version of the test.There is a danger, however, that in the desire to be highly specific, we may go beyond our current understanding of wh

6、at the components of language ability are and what their relationship is to each other.For instance, while we may believe that many subskills contribute to the ability to read lengthy prose passageswith full understanding, it seems hardly possible in our present state of knowledge to name them all o

7、r to assess their individual contributions to the more general ability.We cannot be sure that the sum of the parts that we test will amount to the whole in which we are generally most directly interested.At the same time, however, teaching practice often assumessome such knowledge, with one subskill

8、 being taught at a time.The safest procedure is to include in the content specifications only those elements whose contribution is fairly well established.The way in which content is described will vary with its nature.e.g.The content of a grammar test may simply list all the relevant structures.The

9、 content of a test of a language skill, on the other hand, may be specified along a number of dimensions.The important thing is that content should be as fully specified as possible.The following provides a possible framework for doing this.Operations (the tasks that candidates have to be able to ca

10、rry out)For a reading test these might include, for example: scan text to locate specific information; guess meaning of unknown words from context.Types of textFor a writing test these might include: letters, forms, academic essays up to three pages in length.AddresseesThis refers to the kinds of pe

11、ople that the candidates is expected to be able to write or speak to (for example native speakers of the same age and status); or the people for whom reading and listening materials are primarily intended (for example native-speaker university students).TopicsTopics are selected according to suitabi

12、lity for the candidate and the type of test.FORMAT AND TIMINGThis should specify test structure (including time allocated to components) and item types/elicitation procedures, with examples. It should state what weighting is to be assigned to each component. It should also say how many passageswill

13、normally be presented(in the case of reading or listening) or required (in the case of writing), how many items there will be in each component.CRITERIAL LEVELS OF PERFORMANCEThe required level(s) of performance for (different levels of) success should be specified. This may involve a simple stateme

14、nt to the effect that,to demonstratemastery , 80 per cent of the items must be responded tocorrectly. It may be more complex: the Basic level oral interaction specifications of the Royal Society of Arts (RSA) Test of theCommunicative Use of English as a Foreign Language will serve as an example. The

15、se refer to accuracy, appropriacy, range, flexibility, and size.(Basic)AccuracyPronunciation may be heavily influenced by L1 and accented though generally intelligible. Any confusion caused by grammatical /lexical errors can be clarified by the candidate.AppropriacyUse of language broadly appropriat

16、e to function, though no subtlety should be expected. The intention of the speaker can be perceived without excessive effort.RangeSeverely limited range of expression is acceptable. May often have to search for a way to convey the desired meaning.FlexibilityNeed not usually take the initiative in co

17、nversation. May take time to respond to a change of topic. Interlocutor may have to make considerable allowances and often adopt a supportive role.SizeContributions generally limited to one or two simple utterances are acceptable.SCORING PROCEDURESThese are most relevant where scoring will be subjec

18、tive. The test constructors should be clear as to how they will achieve high scorer reliability.Writing the testSAMPLINGIt is most unlikely that everything found under the heading ofContent in the specifications can be included in any one version of the test. Choices have to be made. For content val

19、idity and for beneficial backwash, the important thing is to choose widely from the whole area of content. One should not concentrate on those elements known to be easy to test. Succeeding versions of the test should also sample widely and unpredictably.ITEM WRITING AND MODERATIONThe writing of succ

20、essful items (in the broadest sense, including, for example, the setting of writing tasks) is extremely difficult. No one can expect to be able consistently to produce perfect items. Some items will have to be rejected, others reworked. The best way to identify items that have to be improved or aban

21、doned is through teamwork. Colleagues must really try to find fault; and despite the seemingly inevitable emotional attachment that item writers develop to items that they have created, they must be open to, and ready to accept, the criticisms that are offered to them. Good personal relations are a

22、desirable quality in any test writing team.What critical questions may you ask? Is the task perfectly clear? Is there more than one possible correct response? Can candidates show the desired behavior (or arrive at the correctresponse) without having the skill supposedly being tested? Do candidates h

23、ave enough time to perform the task(s)?In addition to moderation, as this process is called, an attempt should be made to administer the test to native speakers of a similar educational background to future test candidates. These native speakers should score 100 per cent, or close to it. Items that

24、prove difficult for comparable native speakers almost certainly need revision or replacement.WRITING AND MODERATION OF SCORING KEYOnce the items have been agreed, the next task is to write the scoring key where this is appropriate. Where there is intended to be only one correct response,this is a pe

25、rfectly straightforward matter. Where there are alternative acceptable responses,which may be awarded different scores, or where partial credit may be given for incomplete responses, greater care is necessary. Once again, the criticism of colleagues should be sought as a matter of course.PretestingE

26、ven after careful moderation, there are likely to be some problems with every test. It is obviously better if these problems can be identified before the test is administered to the group for which it is intended. The aim should be to administer it first to a group as similar as possible to the one

27、for which it is really intended.Problems in administration and scoring are noted. The reliability coefficients of the whole test and of its components are calculated, and individual items are analysed.It has to be accepted that, for a number of reasons, pretesting is often not feasible. In some situ

28、ations, though a suitable group exists, it may be thought that the security of the test might be put at risk.It is often the case, therefore, that faults in a test are discovered only after it has been administered to the target group. Unless it is intended that no part of the test should be used ag

29、ain, it is worthwhile noting problems that become apparent during administration and scoring, and afterwards carrying our statistical analysis of the kind described in the later content.Example 1An achievement testStatement of the problemThere is a need for an achievement test to be administered at

30、the end of a presessional course of training in the reading of academic texts in thesocial sciences and business studies (the students are graduates who are about to follow postgraduate courses in English-medium universities).The teaching institution concerned (as well as the sponsors of the student

31、s) wants to know just what progress is being made during the three-month course. The test must therefore obviously be sufficiently sensitive to measure gain over that relatively short period. While there is no call for diagnostic information on individuals, it would be useful to how, for groups, whe

32、re the greatest difficulties remain at the end of the course, so that future course may give more attention in these areas. Backwash is considered important; the test should encourage the practice of the reading skills that the students will need in their university studies. This is in fact intended

33、 to be only one of a battery of tests, and a maximum of two hours can be allowed for it. It will not be possible at the outset to write separate tests for different subject areas.SpecificationsCONTENTTypes of text The tests should be academic (taken from textbooks and journal papers).neutral as poss

34、ible, sincAddressees Academics at postgraduate level and beyond.Topics The subject areas will have to be as the students are from a variety of social science and business disciplines (economics, sociology, management etc.)Operations These are based on the stated objectives of the course, and include

35、 broad and underlying skills:Broad skills:1. Scan extensive written texts for pieces of information.2. Construe the meaning of complex, closely argued passages.Underlying skills: (Those which are regarded as of particular importance for the development of the broad skills, and which are given partic

36、ular attention in the course.)3. Guessing the meaning of unfamiliar words from context.4. Identifying referents of pronouns etc., often some distance removed in the text.FORMAT AND TIMINGScanning 2 passages eacch. 3,000 words in length.15 short-answer items on each, the items in the order in which r

37、elevant information appears in the texts. Controlledresponseswhere appropriate.e.g. How does the middleman make a large profit in traditional rural industry?Heand thenTime: 1 hourDetailed reading 2 passages eachc. 1,500 words in length.7 short-answer questions on each, with guidance as to relevant p

38、art of text.Controlled responses where appropriate.e.g. Cross-sectional studies have indicated that intelligence declines afterthe age of thirty. Part of the explanation for this may be that certain capable as we grow elligence test tasks requires,omething of which we are less5 meaning-from

39、-context items from one of the detailed reading passages e.g. For each of the following, find a single word in the text with an equivalent meaning. Note: the word in the text may have an ending such as -ng, -s, etc.highest point (lines 20-35)5 referent-identification items from one of the detailed r

40、eading passages e.g. What does each of the following refer to in the text? Be very precise.the former (line 43)CRITERIAL LEVELS OF PERFORMANCEItems are written such that anyone completing the course successfully should be able to respond correctly to all of them. Allowing for performance errors on t

41、he part of candidates, a criterial level of 80 s objectives.per cent is set. The number of students reaching this level will be the number who have succeeded in terms of the courseSCORING PROCEDURESThere will be a detailed key, making scoring almost entirely objective. There will nevertheless be ind

42、ependent double scoring. Scorers will be trained to ignore irrelevant (for example grammatical) inaccuracy in responses.SAMPLINGTexts will be chosen from as wide a range of topics and types of writing as is compatible with the specifications. Draft items will only be written after the suitability of

43、 the texts has been agreed.ITEM WRITING AND MODERATIONItems will be based on a consideration of what a competent non-specialist reader should be able to obtain from the texts.Considerable time will be set aside for moderation and rewriting of items.KEYThere will be a key with details of partial cred

44、it for incomplete responses. This too will be subject to thorough moderation.PRETESTINGPretesting of texts and items sufficient for at least two versions will be carried out with students currently taking the course.Example 2A placement testStatement of the problemA commercial English language teach

45、ing institution needs a placement test. Its purpose will be to assign new students to classes at five levels: false beginners; lower intermediate; middle intermediate;upper intermediate; advanced. Course objectives at all levels are expressed in rather general communicativeterms, with no one skill b

46、eing given greater attention than any other. As well as information on overall ability in the language, some indication of oral ability would be useful. Sufficient accuracy is required for there to be little need for changes of class once teaching is under way. Backwash is not a serious consideratio

47、n. More than a thousand new students enroll within a matter of days. The test must be brief (not more than 45 minutes in length), quick and easy to administer, score and interpret. Formal interviews or other direct tests of oral ability are out of the question.SpecificationsCONTENTStyles and topics

48、of texts will reflect the kinds of written and spokenformtexts that are to be found in teaching materials at the various levels.As testing will be indirect, operations are dealt with underFORMAT AND TIMINGClose-type tests A series of five 20-item close-type passages presented in increasing order of

49、difficulty, with blanks selected to representGrammar items and vocabulary items, as well as to some extent testing overall comprehension.Time: 30 minutes (Note: this seems very little time, but the more advanced students will find the early passages extremelyeasy, and will take very little time. It

50、does not matter whether lower level students reach the later passages.)Partial dictationA 20-item partial dictation test (part of what is heard by the candidates already being printed on the answer sheet). Stretches to be transcribed being ordered according to difficulty. Spelling to be ignored in s

51、coring. To be administered in the language laboratory. Since no oral test is possible, this is intended to give some indication of oral ability.Time: 10 minutes e.g. The candidate hears:Particular consideration was given to the timing of the reportand has to complete:Particular consideration. timing

52、15In addition, information should be collected on previous training inEnglish, as well as comments of the person who carries out the registration using English as far as possible (the registration procedure can be used as an informal, and not very reliable, oral test).SAMPLINGAs noted above, the tex

53、ts are meant to provide a representative sample of the kinds of spoken and written English found at the various levels. Choice of texts will be agreed before deletions are made.Scoring will be objective and may be carried out by non-teachers.CRITERIAL LEVELS OF PERFORMANCEThese will be determined du

54、ring pretesting.ITEM AND KEY WRITING (AND MODERATION)Cloze-type tests Once texts have been chosen, deletions are made and keys constructed. The mutilated passages are then presented to the moderating group. It is particularly important that there is only one acceptable response for most items (preferably no more than t