英日机器翻译系统中的英语兼类词消歧策略.doc

英日机器翻译系统中的英语兼类词消歧策略原文地址：英日机器翻译系统中的英语兼类词消歧策略作者：1926267847这个图片是姑娘头像还是老人头像?英日机器翻译系统E-to-J原语分析中的兼类词消歧策略冯志伟语言文字应用研究所计算语言学研究室100010北京朝内南小街51号e-mail：摘要本文介绍了商品化英日机器翻译系统E-to-J中兼类词的消歧策略。作者根据机器翻译的实践，把英语中同形兼类词归纳为29种类型，提出了基于上下文环境的处理这些兼类词类型的消歧方法。关键词：机器翻译，原语分析，兼类词，消歧我最近与日本JEC公司的研究人员合作，开发了一个英日机器翻译系统E-to-J，已经于1998年11月在日本市场推出，实现了商品化。这是我们研制的第一个从外语到外语的商品化机器翻译系统，主要是为了满足国外市场的需要。商品化的机器翻译系统的研制离不开机器翻译的理论。在开发E-to-J机器翻译系统之前，我们开发部的研究人员对于机器翻译的策略在理论上作了认真的探讨，使得我们的商品化机器翻译系统的研制在理论上有一个坚实的基础。这个机器翻译系统采用基于复杂特征的短语结构语法来分析英语，首先造出短语结构树(phrase structure tree，简称P-树)，然后将短语结构树转化为从属树(dependency tree，简称D-树)，采用了P-树到D-树的转化策略，最后根据从属关系语法(即配价语法)来进行日语生成。这种基于复杂特征的短语结构语法和从属关系语法相互结合的策略，适合于英日机器翻译的特点，取得了比较好的效果。在英语分析中，尽管我们采用了富田算法(Tomita Algorithm)实现子树共享以及局部歧义紧缩等技术，提高了分析的效率，但是，由于英语中存在大量兼类词，如果兼类词的词类判断有误，分析中往往会走弯路，严重地影响到分析的速度。兼类词的判定实际上是在词汇中进行词类的消歧，兼类词的消歧，是英语自动分析中必须认真对待的问题。在机器翻译中，所谓兼类词是指广义的兼类词。词形相同，词义相同，只是词类不同的词算兼类词，如plan，可以用作动词，也可以用作名词，但是，其词义都是计划，算为兼类词；而且，词形相同，词义和词类不同的词，也算为兼类词，如can，作名词时其词义为罐头，作助动词时其词义为能够，我们也把它算为兼类词。这样，本文中所说的兼类词是意义非常广泛的兼类词，实际上也就是同形兼类词。在机器翻译系统中，兼类词的消歧，实际上就是同形词的辨别。我们对兼类词做这样广泛的规定，完全是从机器翻译形式分析的角度出发来考虑的。因为机器只认形式，只要是形式上相同的词，机器都有必要将它们区别开来。在系统开发的初期，兼类词的消歧是贯穿在整个翻译过程中的，哪里遇到兼类词就在哪里消歧，这是一种见子打子的策略。这种策略的好处是消歧可以收到立竿见影的效果，但是，缺乏系统性，往往会忽略许多重要的兼类现象，由于商品化系统必须面对各种复杂的真实的语言材料，如果一些重要的兼类现象到分析结束时才处理，系统在分析的初始阶段必须付出大量的无谓的开销，系统的空间复杂度和时间复杂度都很高。商品化的机器翻译系统必须讲求效率，如果系统的无谓的开销过大，翻译速度必定不会很高，难以满足市场的需求，在激烈的市场竞争中有可能失利。经过反复的考虑和试验，我们决定把兼类词判别作为系统的一个单独模块，放在形态分析结束而句法语义分析刚开始时来进行，尽量作到在句法语义分析开始之前就对大多数的兼类词进行消歧，从而为后面的分析工作奠定坚实的基础。实践证明，这样的做法大大地降低了系统的时间复杂度和空间复杂度，减少了系统的无谓开销，提高了翻译的速度，满足了日本市场的需要。下面，我们把E-to-J英日机器翻译系统中在兼类词消歧方面的研究作一个简单的介绍。1.英语中词的兼类现象：根据大量的语料分析和翻译试验，我们把英语中的兼类现象初步总结为如下29个方面：1.1 V?N：V表示动词，N表示名词，V?N表示动词-名词兼类。下面的表示方法与此相同。例如，plan是动词-名词的兼类词。在Where do you plan to spend your vacation?中，plan是动词(我们在句子后面标以V)，而在下面的句子中，plan是名词(我们在句子后面标以N)：Have you made any plans for tomorrow night?(N)另外，在N and plan and N(N表示名词)这样的格式中，由于与plan并列的前后词都是名词，故plan也是名词。动词-名词兼类的例子还有(当为动词时，句子后面标以V；当为名词时，句子后面标以N)：work：My daughter works in an office.(V)It is agas works.(N)I read the works of Shakespears.(N)study：I study French.(V)We shall make astudy of Shakespears plays.(N)attack：The enemy could attack at night.(V)The city came under attack during the night.(N)face：The house faces the park.(V)She pulled along face.(N)use：all the paper has been used.(V)A new machine for the kitchen with several different uses.(N)1.2 AUX?N：AUX表示助动词。例如，may：May Ihelp you?(AUX)May Day is first day of May.(N)can：She can speak German.(AUX)He opened acan of beans.(N)will：It will rain tomorrow.(AUX)Free will makes us able to choose our way of life.(N)需要注意的是，will有时还可以用作动词，例如在句子We willed him to stop中，will只能分析为动词。不过，这种情况比较少见，暂不考虑。1.3 N?ADJ：ADJ表示形容词。例如，kind：I like that kind of book.(N)It was very kind of you to do it.(ADJ)standard：We work to ahigh standard of exactness.(N)It is one of the standard books on the subject.(ADJ)utmost：to do ones utmost.(N)She worked with her utmost strength.(ADJ)1.4 WH?PRON：WH表示关系代词，PRON表示一般的代词。例如，which：Nobody wants to have acar which has been used for years.(WH)Which do you like better,summer or winter?(PRON)1.5 DET?N：DET表示限定词。例如，any：Any child would know that.(DET)His gift was unknown to any except himself.(N)(any=anybody)1.6 NUM?N：NUM表示数词。例如，one：It is one meter.(NUM)Lent me that book-the one with the red cover.(N)1.7 PREP?CONJ：PREP表示介词，CONJ表示连接词。例如，for：They have bought some new chairs for the office.(PREP)We must start early,for we have along way to go.(CONJ)than：He is taller than me.(PREP)Nothing is more unpleasant than to find insects in your bath.(CONJ)1.8 PREP?ADV：ADV表示副词。例如，in：in the room.(PREP Open the bag and put the money in.(ADV)down：He runs down the hill.(PREP)It gets cold quickly when the sun goes down.(ADV)on：a ring on my finger.(PREP)If you walk on you shall come to the church.(ADV)about：They walk about the street.(PREP)I am about ready.(ADV)1.9 ADV?CONJ：例如，so：Cut the apples up like so.(ADV)I had aheadache,so Iwent to bed.(CONJ)1.10 PREP?V：例如，regard：She said nothing regarding your request.(PREP)She regarded him curiously.(V)1.11 PREP?TO：TO表示动词不定式前面的to，它在句法功能上不同于作为介词的to。例如，to：He wants to be fed.(TO)He points to the moon.(PREP)1.12 PRON?DET：例如，those：Those tree bear fruit.(DET)Those are our happiest day.(PRON)1.13 DET?ADV：例如，almost：A cry almost human was heard.(DET)It is almost over.(ADV)no：With no fire and no matches they could not light the candle.(DET)He is no ordinary student.(ADV)1.14 AUX?BE：BE表示作为连系动词的be，它与作为助动词的be是不同的。例如，be：I am working now.(AUX)The book is on the table.(BE,connecting verb)1.15 V?ADJ：例如，correct：Please correct my spelling.(V)It is very correct answer.(ADJ)appropriate：The government appropriated alarge sum of money for building hospitals.(V)His bright clothes were not appropriate for afuneral.(ADJ)alternate：Work alternated with sleep.(V)He works on alternate days.(ADJ)1.16 AUX?V：例如，have：he has gone to Beijing.(AUX)This coat has no pocket.(V)1.17 ADJ?ADV：例如，alive：He is very much alive.(ADJ)He kept the memory alive.(ADV)hard：hard duty.(ADJ)The snow was hard frozen.(ADV)1.18 PRON?ADV：例如，ourselves：We consider ourselves of no importance at all.(PRON)We build the house ourselves.(ADV)1.19 ADV?THERE：THERE表示在there is中作为引导词的there。例如，there：Put it there,not here.(ADV)There is apencil on the table.(THERE)下面是同时兼属三类的同形兼类词：1.20 V?N?ADJ：例如，average：My mail averages 5letters aday.(V)On average we receive 5letters each day.(N)What is the average rainfall for July?(ADJ)light：I light on the clue.(V)The light travels faster than the plane.(N)The box is light to carry.(ADJ)express：She expressed her thanks.(V)Send the letter by express.(N)I come here with the express purpose of seeing you.(ADJ)1.21 V?N?ADV：例如，back：She backed the horse.(V)You must iron the backs of the shirts as well as the fronts.(N)She turned her face back.(ADV)1.22 ADJ?ADV?N：例如，enough：We have enough seats for everyone.(ADJ)It is warm enough to swim.(ADV)She doesnt have enough to do.(N)1.23 DET?N?ADV：例如，more：Can Ihave more time to answer your questions?(DET)He liked the girl,and thought hed like to see more of her.(N)His illness was more serious than the doctor first thought.(ADV)all：Please answer all questions on this list.(DET)That ugly little house was all the home that Iever had.(N)She was all worn out.(ADV)1.24 ADJ?ADV?DET：例如，just：He is avery just man.(ADJ)He was sitting just here.(ADV)You have received ajust punishment.(DET)1.25 PREP?CONJ?ADV：例如，as：They all rose together as one man.(PREP)She doesnt run as fast as she used to.(CONJ)Paul runs fast,but Irun as fast.(ADV)before：She stood before him.(PREP)We do want to buy something now before prices go up.(CONJ)He fell silent as before.(ADV)1.26 THAT?DET?PRON：THAT表示引入从句的that。例如，that：There is no proof that she killed him.(THAT)This room is alot warmer than that one.(DET)He kissed me and with that he left.(PRON)1.27 BE?AUX?SS：SS表示所属关系的s。例如，s：Hes astudent.(BE)Its raining hard,isnt it！(AUX)writers ability.(SS)1.28 ADJ?PREP?N：例如，past：I have not been feeling very well for the past few days.(ADJ)The boy rushed past us.(PREP)Our country has aglorious past.(N)还有同时兼属四类的同形兼类词。1.29 PRON?BOTH?DET?ADV：BOTH表示具有前后呼应关系的连接词。例如，neither：Books that are neither of them very interesting.(PRON)Neither my father nor Iwere there.(BOTH)Neither road out of town is very good,but this one is better than the other one.(DET)Just as Ihave not good eyes,so neither has my son,neither have my children.(ADV)2.兼类词的消歧策略：-基于形态的消歧方法：英语中各类词的形态变化不尽相同，因此，对于发生了形态变化的兼类词，我们可以通过它们的形态变化方式来判定它们所属的词类。例如，book是一个动词-名词兼类词，但是，在I have booked aroom中，由于booked采取了过去分词的变化形式，作为名词的book不可能有这样的形式，所以，我们可以判定这个booked是动词，它的词义不是书，而是预定。这种基于形态的消歧方法，基本上用不着考虑上下文，判定起来直接而迅速。-基于上下文环境的消歧方法：词的上下文就是词的分布，词的分布是一种广义的形态，它反映了词的句法功能。例如，英语名词的前面可以出现数词、形容词、限定词，根据这样的分布环境，我们就可以判定动词-名词兼类词是名词。英语形容词的前面可以出现副词，而名词前面不能出现副词，根据这样的分布环境，我们就可以判定形容词-名词兼类词是形容词。英语动词的前面可以出现助动词，根据这样的分布环境，我们就可以判定助动词后面的动词-名词兼类词是动词。在上下文环境X+and+ADJ中，如果X是一个动词-形容词兼类词，由于与它并列地连接的词是形容词ADJ，因此，可以判定X也是形容词。-基于语义的消歧方法：词的语义搭配关系存在着一定的优先关系。例如，动词buy(买)之后的宾语一般为事物(thing)，因此，名词应该优先，具体地说，如果buy后面是动词-名词兼类词book，而book是名词时它在词典中的定义是a collection of sheets of paper fastened together as athing to be read，那么，book是名词的可能性远远大于是动词的可能性，我们可以判定它是名词。在我们的机器翻译系统中，我们综合采用了上述三种方法来进行兼类词的消歧，在本文中，我们着重讨论基于上下文环境的消歧策略。3.E-to-J机器翻译系统的同形兼类词的消歧规则概览：下面，我们把E-to-J机器翻译系统中使用上下文消歧的规则作一简单介绍。3.1 V?N的消歧规则：-in+V?N+of-N在规则表示式中，箭头的左部表示条件，箭头的右部表示结论，需要消歧的兼类词以及消歧的结论，都写在花括号内；具体的单词写在内。这个规则表示：当动词-名词兼类词V?N前面为in，后面为of时，该兼类词可判定为名词N。例如，在in plan(N)of中的plan可判定为N。用这样的表示方法，我们还可以写出如下的规则：-DETN?V-N例如，this plan(N)。-NUM+N?V-(N例如，three works(N)-PREP+N?V-N例如，before study(N)-DETN?VAUX-N例如，that work(N)should-N?V+ADJ+for-N例如，plan(N)important for-DET+N?V+AUX-N例如，the work(N)should-DET+N?V+V-N例如，the work(N)finished-DETN?V+to+V-N例如，the work(N)to finish-DETN?V+of-N例如，the plan(N)of-N+,+N?V+and+N-N例如，draft,plan(N)and manuscript-N+and+N?V+and-N例如，draft and plan(N)and-N?V+why+(N or PRON)+V-N在规则的条件部分，如果有多种可能的情况，用or把各种情况连接起来，并把它们写在圆括号内。如(N or PRON)表示或者为N,或者为PRON例如，wonder(V)why they come here(这里，they为PRON)wonder(V)why students come here(这里，students为N)-(DET or PRON)+N?V-N例如，her face(N),the face(N)-DET+ADJ+N?V-N例如，the good work(N)-PREP+N?V-N例如，regardless ofreward(N)，这里regardless of被看成介词PREP。-AUX+not+N?V-V例如，can not work(V)-AUX+N?V-V例如，shall work(V)，-N+WH+N?V+N+PREP-V例如，student who studies(V)English in the classroom-pleaseN?V-V例如，please work(V)-BE+N?V+by-V例如，is attacked(V)by-N+ADV+N?V-V例如，must slowly work(V)-N+N?V+ADV-V例如，John works(V)slowly-WH+AUX+N+N?V+PUNC-V例如，How should John work(V)?-(N or PRON)+N?V+DET+N-V例如，we use(V)our ingenuity3.2 AUX?N的消歧规则：-AUX?N+(N or PRON)+V-AUX例如，May(AUX)I help you-AUX?N+not+V-AUX例如，can(AUX)not have-as+(N or PRON)+AUX?N-AUX例如，as you can(AUX),-(DET or ADJ)+AUX?N+BE-N例如，the can(N)was,red can(N)broken,-N+AUX?N+BE-AUX例如，John can(N)be-PRON+AUX?N-AUX例如，I can(AUX)3.3 N?ADJ的消歧规则：-DET+N?ADJ+(oforfororto)-N例如，the kind(N)of rose,a medium(N)for amusing-ADV+N?ADJ+(oforfororto)-ADJ例如，very kind(ADJ)of you,very kind(ADJ)for,very kind(ADJ)to-DET+N?ADJ+PUNC|V|AUX|BE-N例如，the red(N),，the American(N)comes-N?ADJ+N?ADJ-ADJ+N这个规则表示，如果前后两个词都是名词-形容词兼类词，则可判定前者为形容词，后者为名词。例如，basic(ADJ)standard(N)，-DET+N?ADJ+ADJ+N-ADJ例如，the kind(ADJ)old man-N+N?ADJ+(toorfor)-ADJ例如，professor kind(ADJ)to student,-DET+N?ADJ+one-ADJ例如，the red(ADJ)one-ADJ+(andoror)+N?ADJ-ADJ例如，nice and red(ADJ)-DET+N?ADJ+N-ADJ例如，the utmost(ADJ)care3.4 WH?PRON的消歧规则：-N+WH?N+BE-WH例如，book which(WH)is3.5 DET?N的消歧规则：-DET?N+N-DET例如，a(DET)student-V+DET?N+PREP-N例如，read many(N)at-DET?N+ADJ+N-DET例如，any(DET)other possibility3.6 NUM?N的消歧规则：-NMU?N+N-NUM例如，one(NUM)meter-NUM?N+AUX-N例如，one(N)cannot3.7 PREP?CONJ的消歧规则：-PREP?CONJthan+that+WH-PREP当PREP?CONJ为that时，使用这个规则。例如，than(PREP)that which-PREP?CONJ+DET+N-PREP例如，for(PREP)the independence-PREP?CONJfor+sure-PREP当PREP?CONJ为for时，使用这个规则。例如，for(PREP)sure-PREP?CONJ+DET+N-PREP例如，before(PREP)the night,-PREP?CONJ+ADJ+N+$END-PREP$ENG表示句子结尾。例如，for(PREP)individual freedom.-DET+PREP?CONJlonging+for-PREP例如，the longing for(PREP)3.8 PREP?ADV的消歧规则：-PREP?ADV+(N or ADJ or DET)-PREP例如，in(PREP)Japan,in(PREP)the country,-PREP?ADVon+DETa+ADJ-PREP例如，on(PREP)a dangerous-PREP?ADVabout+PRON(possessive case)-PREP例如，about(PREP)mine-PREP?ADV+DET+ADJ+N-PREP例如，in(PREP)this wonderful play-PREP?CONJdown+Vgo-ADV例如，down(ADV)go3.9 ADV?CONJ的消歧规则：-seem+ADV?CONJ+ADJ-ADV例如，seem so(ADV)obvious3.10 PREP?V的消歧规则：-PREP?Vregarding+DET+N-PREP例如，regarding(PREP)the content3.11 PREP?TO的消歧规则：-ADJ+PREP?TO+V or BE or AUX-TO例如，able to(TO)read,able to(TO)be read,able to(TO)have-PREP?TOto+PRON(accusative)-PREP例如，to(PREP)us-PREP?TOto+Vlove+Nlabor-TO例如，to(TO)love labor-PREP?TOto+Vdo+N or DET or ADJ-TO例如，to(TO)do something3.12 PRON?DET的消歧规则：-PRON?DET+N-DET例如，those(DET)men-PRON?DET+BE-PRON例如，those(PRON)are3.13 DET?ADV的消歧规则：-BE+DET?ADV+V-ADV例如，She is almost(ADV)killed-DET?ADV+DET+N-DET例如，no(DET)other country,-DET?ADV+N-DET例如，no(DET)thing-DET?ADV+as+ADJ-ADV例如，almost(ADV)as wonderful-DET?ADV+DET+ADJ-DET例如，even(DET)the few event3.14 AUX?BE的消歧规则：-there+AUX?BE-BE例如，there is(BE)-itAUX?BE+ADJ-BE例如，it is(BE)better-AUX?BE+V(ed)-AUX例如，be(AUX)done-AUX+AUX?BE+ADJ-BE例如，must be(BE)red-AUX?BE+ADV+ADJ-BE例如，be really able-AUX?BE+ADV+ADV+ADJ-BE例如，be(BE)often more important-there+have+not+AUX?BE-BE例如，there have not be(BE)3.15 V?ADJ的消歧规则：-DET+ADJ?V+N-ADJ例如，the correct(ADJ)plan-veryorrather+ADJ?V-ADJ例如，very correct(ADJ)-AUX+ADJ?V-V例如，should correct-make+N+ADJ?V-ADJ例如，make draft appropriate(ADJ)-ADJ?V+ADV-V例如，alternate(V)rapidly-ADJ?V+and+ADJ-ADJ例如，correct(ADJ)and concise-ADJ+ADJ?V-ADJ例如，concise and correct(ADJ)-ADJ?V+and+V-V例如，correct(V)and revise-V+and+ADJ?V-V例如，revise and correct(V)-most+ADJ?V-ADJ例如，most correct(ADJ)3.16 AUX?V的消歧规则：-AUX?V+V-AUX例如，have(AUX)done-AUX+AUX?V+BE+V-AUX例如，would have(AUX)be regard-AUX?V+not+V+and+V-AUX例如，have(AUX)not hope and dream-AUX?V+DET+N+V-AUX例如，have(AUX)any writer shown3.17 ADJ?ADV消歧规则：-BE+(ADJ?ADV)+to-ADJ例如，be alive(ADJ)to the dangers-ADJ?ADV&lex！=still+N-ADJ在这个规则中，ADJ?ADV不能为still，！=表示不等于。例如，hard(ADJ)duty-DET+ADJ?ADV+N-ADJ例如，the hard(ADJ)person-AUX+ADJ?ADV+V-ADV例如，The snow has hard(ADV)frozen-V+ADV+ADJ?ADV+PREP-ADV例如，to work very hard(ADV)for the boss-BE+ADJ?ADV-ADJ例如，be hard(ADJ)-V+ADJ?ADV+(.or?or！)-ADV例如，work hard(ADV).3.18 PRON?ADV的消歧规则：-PRON+V+N+PRON?ADV+PUNC-ADV例如，we achieve happiness ourselves(ADV)3.19 ADV?THERE的消歧规则：-ADV?THERE+BE-THERE例如，there(THERE)is3.20 V?N?ADJ的消歧规则：-V?N?ADJ+N+BE-ADJ例如，average(ADJ)work is-V?N?ADJ+N+PREP-ADJ例如，average(ADJ)rainfall for July-V?N?ADJ+V+ADJ+than-N例如，light(N)travels faster than-DET+V?N?ADJ+V-N例如，the light(N)travels-able to+V?N?ADJ+DET+N-V例如，able to express(V)an idea3.21 V?N?ADV的消歧规则：-V+DET+N+V?N?ADV+PUNC-ADV例如，She turned her face back(ADV).-V+DET+N+V?N?ADV+when-ADV例如，She turned her face back(ADV)when some body looks for her3.22 ADJ?ADV?N的消歧规则：-BE+ADJ?ADV?N+to+(PRON or N)-ADJ例如，be enough(ADJ)to me-have+ADJ?ADV?N-ADJ例如，have enough(ADJ)money-(！BE！or！AUX)+ADJ?ADV?N-ADV当ADV之前不是BE，也不是AUX时，ADJ?ADV?N可判定为ADV。例如，warm enough(ADV),-have+ADJ?ADV?N+to+V-N例如，have enough(N)to do3.23 DET?N?ADV的消歧规则：-DET?N?ADV+ADJ-ADV例如，more(ADV)important-V+DET?N?ADV+(N or PRON)+AUX-N例如，learn all(N)I can about3.24 ADJ?ADV?DET的消歧规则：-N+BE+ADJ?ADV?DET+PUCT-ADJ例如，view is just(ADJ).3.25 PREP?CONJ?ADV的消歧规则：-$START+PREP?CONJ?ADV+ADJ+N+as(CONJ)-ADV规则中的$START表示句子开头。例如，as(ADV)many vegetables as-PREP?CONJ?ADV+(,or?or！)-ADV例如，Havent Iseen you before(ADV)?-PREP?CONJ?ADV+(DET or N)-！ADV！ADV表示不为ADV，也就是说，该词为PREP或者CONJ。例如，before(！ADV)the table,before(！ADV)table-PREP?CONJ?ADV+NUM-PREP例如，before(PREP)1997-PREP?CONJ?ADV+PRON(accusative)-PREP例如，before(PREP)me-PREP?CONJ?ADV+PRON(nominative)+(V or AUX or BE)-CONJ例如，before(CONJ)you go,before(CONJ)you should go,before(CONJ)I am astudent-(asorso)+ADJ+PREP?CONJ?ADV-CONJ例如，so cheap