挖掘-textmining计算机科学技术研究所

第九章文档自 技杨建计算机科学技 1 Anexpressofacertain withoutanyexplanationsandcomment.It'sunnecessarytoknowwhowritesthesummary.(ANSI)Aconciseandaccurateexpressofthewithoutanyexplanationandcomment.Asummaryisindependentontheauthorofthesummary.(ISO214-1976(E))2 以性的形式表述文献中信息价值较高的部分，而以指示性的形式表述其余部分的。作 3SummaryClassifiedbyuser'sGenericSummarizationUser-querySummarizationClassifiedbytext ClassifiedbySummarizationBasedonExtractionSummarizationBasedonUnderstandingClassifiedbyneedSupervisedSummarizationUnsupervisedSummarization4 系统适用于5ABriefHistoryofDUC,holdbyNIST,fromByMarkT.MayburyandInderjeet6哥伦比亚大学的多文档自 系统Newsblaster 密西根大学研究开发的个性化的基于Web的多文档自 和内容推荐系统南加利福尼亚大学的信息科 NeATSVivisimo公司 infonetware公司 DUC Understanding 78内部评价方法(IntrinsicMethods)：在提供参考的前提下，以参考为基准评价系统的质量。通常情况下，系统摘要与参考越吻合，其质量越高。外部评价方法(ExtrinsicMethods)：不需要提供参考，利用文档代替原文档够提高应用性能的被认为是质量好的摘9Evaluation-- 得与目标 (coselectionrate)。 基本相似，很相似，完全相似等Evaluation--Edmundson评价的几个基本规 “。”“：”“；”“！”“？”；为使专家与机械具有可比性，只专家和机械的句子都按照在原文 n

Pi/n（Pi为相对于第i个专家的重合率，n为专家的数目原文(题目

RecallNhm/机 系PrecisionNhm机 系专 机专机评评 被广泛应用于DUC 评测任务 UnderstandingConference(DUC)isaseriesofsummarizationevaluationsthathavebeenconductedbytheNationalInstituteofStandardsandTechnology(NIST)since2001.Itsgoalistofurtherprogressinautomatictextsummarizationandenableresearcherstoparticipateinlarge-scaleexperimentsinboththedevelopmentandevaluationofsummarizationSince2008,DUChasmovedtotheText Conference(TAC)QuestionAnswering;RecognizingTextualEntailment; SummarizationKeywordDisplaymostEasytoHardtoread,poorrepresentationofSentenceExtractkeyMediumSummariesoftendon’treadGoodrepresentationofNaturallanguageunderstanding/BuildknowledgerepresentationofGeneratesentencessummarizingHardtodoSomethingbetweenthelasttwoWebWeb</html> Thisis

PlainText

Content:weight=Sumoftopicterms’weights;Length:weight/=Position:weight*=positionConten;Stigmawords;Stigma:Conten;Stigmawords;Redundancy:weight*=(1-&Repair e.g:Edmundson的 取正值 词(Bonus 取负值的惩罚词(Stigma [Luhn,1958]：能够指示文章 实词的个数来计算句子的权值。[V.A.Oswald]主张句子的权值应按其所含 ANES(AutormaticNewsExtractionSystem)作所谓的目次性，这也很受欢统计表明：大部分科技文献的标题都能基本反映叶句，删句留主干句的方 (如气象预报等)。该方法根据用户的 基于理解的自动常包含语法分析、语义分析、信息提取和生成，作者应属于此。分层的，理解比低层理解更为手工人员在编制时并不一定通结束语及其论题句，以发现其，再挑选句子并修饰稍加组织生成。文献，不同用户点和观察角度可能不同，的结果应当不同。SentenceSentenceRepresenteachsentenceasafeatureComputescorebasedonPresentinorderinwhichtheyoccurinPostprocessingtomakesummarymoreEliminateredundantDeletesubordinateclauses,SentenceImportance--StatisticalLinguisticIntegrative [G. 向量空间VSM（VectorSpace 段落的 句子在段落Sentence 句子级 冗余性消除 其中sj 中与si最相似的句子 排序、指代A Sigir95paperonsummarizationbyKupiec,Pedersen,ChenTrainablesentenceProposedalgorithmisappliedtoitsowndescription(thepaper)FeatureFixed-phraseCertainphrasesindicatesummary,e.g.“inParagraphParagraphinitial/finalmorelikelytobeThematicwordRepetitionisanindicatorofUppercasewordUppercaseoftenindicatesnamedentities.Sentencelengthcut-Summarysentenceshouldbe>5Hand-labelsentencesintrainingset(good/badsummarysentences)Trainclassifiertodistinguishgood/badsummarysentencesModelused:NaïveCanranksentencesaccordingtoscoreandshowtopntouser. EvaluationofBaseline(choosefirstnsentences):Overallperformance(42-44%)notveryHowever,thereismorethanoneClusteringbasedClustering 直至不能再划分为更小的子Sentences 及MMRAlgorithmQuery-SpecificAgenericsummarymakesnoassumptionaboutthereader’sinterests.Query-specificsummariesarespecializedforasingleinformationneed,thequery.Summarizationismucheasierifwehaveadescriptionofwhattheuserwants.Recallfromlast -typeexcerpts–simplyshowincontextMMR alMarginalMEAD 方法 MMRArgmax[Sim1(Di,Q)-(1-)maxSim2(Di,DjDiR\ 术和IGR(InformationGainRatio)技术结合起来，称为MMI-MS alMarginalImportance–Multi-Sentence)来选 句MMRArgmax[Sim1(Di,Q)-(1-)maxSim2(Di,DjDiR\ DMMI-MSArgmax[Imp(Si)-(1-)maxSimj(Si,SjSiSS\ IGR_sum(w,D) CCset(D

gain_r(w,CMMR- alMarginalRelevanceMulti- MMR- 关方法:MMR-SS(SemanticSimilaritybased alMarginalRelevance)来选 句，生成关于同 的通 MMR-SSArgmax[Sim1(Pj,T,Di,D)-(1-)maxSim2(Pj,Pm,Di Sim1(Pj,T,Di,D)=W1*(Pj,T)W2*Score(Pj)W3*time-sequence(Ditime-sequence(D

timestamp(Di)-timestamp(Dmintime

mintimeSim2(Pj,Pm,Di,S)=Wa*(Pj,Pm)Wb s-selected(Dip x2s-D

*[Pjw

Q(A,B)

i 2x2y2 i ；TypesofMDSingle trackedoveralongtimeElizabethTaylor’sboutwith GiveextraweighttoMayneedto eMultipleeventsofasimilarMarathonrunnersandMorebroadbrush,ignoreAnissuewithrelatedGunIde