深度学习在自然语言处理的应用..

1、深康学习咨自然语言处理的应用张俊林畅捷通股份有限公司2014.10.3大纲深度学习简介基础问题：语言表示问题-Word Embedd i ng-不同粒度语言单元的表示字符/单字/单词/短语/句子/文档值得重点关注的模型-RAE/Tensor Network/卷积网络 NLP的应用-语言模型-中文分词知识挖掘-情感计算-机器翻译-Paraphrase-IR探讨与思考深度学习（表示学习）Feature representation Learning algorithm摩托 非牢托|足否令十轮乍把手|43深度学习（表示学习）Natural ImagesLearned bases: EdgesTest

2、 exampleR8X Xb36EnnT1人o,0 o, 0.8, ozoz0.3, o,o,0.5 =coefficients (feature representation)Layer-W i se Pre-Tra i n i ng6Deno i s ing AutoencoderHidden code (representation)KL(reconstruction | raw input)ooooOi Aoooo)Corrupted inputRaw inputreconstruction自然语言交互的时代 ZBMWATSONoHi therp Im Cortana.C AKIc N

3、ow I F IIOne-Hot 表示 One Hot表示在传统NLP中很常用r，dogM=l,050. .,0Mcatw = 0?l,00MtheM=0,0.0,Similarity(dog,cat)=0Word Embedd i ng词向量:单词的分布向量表不(Distributional Representation )ndog* =1,0, 0.9, 0.0cat = 15 0,0.5,0.2n the1 =0.1,0.0, 0.0Similaity(dog,cat)Similarity(dog,the)Similarity( “the dog smiles.” , Mone cat

4、cries.)词向量表征了单词使用上下文中的句法语义特征- One-Hot的字面匹配到DR的语义匹配10无监督训练获得单词的WE-word2vec11皱子卑猴ftjffi羚牛子羊R密包背性M牛!起蚱直胡猴熊 大猴山瑚卵狐時肆狮综儕企章黒雌松野交小览家 黑Word: T果 Position in vocabulary: 3620WordCosine distance苹果公司0687991桔子06449240.641999iphono0640282報苹0.6133920 5124741607J根0609675渡梦0.604317230万个0603871草苹0603183ipcd0595989乔布斯

5、059588505949420586100西红怖0.583661rim0.581620iphoncs0580593ipodtouch0578409440万个0.5698810.569102ZUDO05690290566155filcvault亠小0.565740 永後機养黄瓜 丄豆/ma lefector单词：苹果无监督训练获得单词的WE-word2vecWord长颈WE Position In vocaba丄airy: 69904WordCeline068873306595530 658b02064904706226790622402U6201210617492061097

6、20.60850606032120601038060097305974170.596062059447805938680592358059040605903490585549 0 5U43JL2 厶0584111单词:长颈鹿12无监督训练获得单词的WE-word2vecWord: 张 Posirion in vocabulary: 294diWard单字：雯14赵李陈工杨汪邹吕郭靳创荒盪俞薛刘郝姜u;s芸未寿07093550.695515 0653313 0.650116 0.641075 0.611737 0.607666 0506544 C5R1273 0.5753530.5664820.

7、563638 0563406 0.561972 0.56012 0555660 0549661 0.547796 0545319 “C 0.544621/rhd1efactor单字：张无监督训练获得单词的WE-word2vecWordI 銮 Pooxeion xn vooabuloxyi 1777WordCoin dilanc06934960649624064113605963390.573760055612405556560545938052205205168150.51491005137020513169C5109550502604049737504872950486723048462504790090.4636020462975y0456392巻鯉翌y weibo.co m/mg (efactor无监督训练获得单词的WE-word2vec:Je- PoaiL ion i:i vocabular y: 3222WoxxiCoainy ditancw单字：葱15椒芥齋萬梨枯酱柑柚荽菠樱台枸拌植业枣椰奨豌桃0 0 000 000000000000695143 662650 661218 661146 657093 6b441b 642134 638596 625900