朴素贝叶斯python代码实现

1、朴素贝叶斯优点：在数据较少的情况下仍然有效，可以处理多类别问题缺点：对于输入数据的准备方式较为敏感适用数据类型：标称型数据贝叶斯准则：表示某个由心y表云的数据点，该数据点来自类别q的概率 如果p（cY xty）> p（c2 | x7y）,那么属于类别 如果p（cr x,y） < p（c2 |x?y）,那么属于类别6使用朴素贝叶斯进行文档分类朴素贝叶斯的一般过程（1 ）收集数据：可以使用任何方法。本文使用RSS源（2）准备数据：需要数值型或者布尔型数据（3）分析数据：有大量特征时，绘制特征作用不大，此时使用直方图效果更好（4 ）训练算法：计算不同的独立特征的条件概率（5 ）测试算法：

2、计算错误率（6 ）使用算法：一个常见的朴素贝叶斯应用是文档分类。可以在任意的分类场景中使用朴 素贝叶斯分类器，不一定非要是文本。准备数据：从文本中构建词向量摘自机器学习实战。'my','dog','has','flea','problems','help','please',0'maybe',' no t','take','him','to','dog','park',&#

3、39;stupid',1'my','dalmatio n','is','so','cute',T,'love','him',0'stop','posti ng','stupid','worthless','garbage'.'mr','licks','ate','my','steak','how',&#

4、39;to','stop','him',0'quit','buyi ng','worthless','dog','food','stupid'以上是六句话，标记是 0句子的表示正常句，标记是 1句子的表示为粗口。我们通过分析 每个句子中的每个词，在粗口句或是正常句出现的概率，可以找出那些词是粗口。在bayes.py文件中添加如下代码:pyth onview pla in copy1. # coding=utf-82.4.p>ostingList =

5、9;my' , 'dog',h5.'maybe' , 'not','take' , 'him'6.'my','dalmation','is' , 'so'7.'stop','posting','stupid',8.'mr','licks','ate' , 'my',9.'quit','buying',&

6、#39;worthless'10. classVec =0, 1,0, 1,0, 111.returnpostingList, classVec3. def loadDataSet():'has' , 'flea' , 'problems' , 'help','to' , 'dog','park','stupid','cute' , T,'love','him','worthless','

7、garbage','steak' , 'how' , 'to' , 'stop' , 'him','dog' , 'food' , 'stupid' # 1代表侮辱性文字，0代表正常言论'please',12.13. def createVocabList(dataSet):14. vocabSet = set()15. for documentin dataSet:16. vocabSet = vocabSet | set(document)

8、17. return list(vocabSet)18.18. def setOfWords2Vec(vocabList, inputSet):19. returnVec = 0 * len(vocabList)20. for word in inputSet:21. if word in vocabList:22. returnVecvocabList.index(word) = 123. else :% word24. print "the word: %s is not in my Vocabulary!"25. return returnVec运行结果:import

9、 bayesaaa listOPosts, listClasses = bayes*loadDataSet()aaa myVocabList = bsyes*createVocabList(ItstOPosts)aaa myVocabList1 cute1, 1 love' , 1 help' , 1 garbage''quit', 1IT, 1 problemsT, 'is1, 1 park' , 'stop ','flea 1 , 'dalmation1 , 1licks' , 'foo

10、dT, 'not', 1hin' , 'buying' , 'posting', Thas ','worthless' , Tate 1 , 'to1, 'naybe', Tplease', 1 dog' , 'how 1 , 'stupid 1 , 'so1, Tt mke、* 'nr ' steak 'nyr http:/blog. csdn. net/Jay_Xioaaa bayes. setOfWords2Vec(myV

11、ocdbLtst, l_tstOPosts0)o, G, 3, Qt e, ”1, o, o, o, -1, e, o, o, e, o, o, ef -if o, et o, o, -1,0, 6 0. 0, S 第 7aaa bayes * setOfWords2Vec(myVocabLtst a l_istOPosts3)o, o, o, -1, e, 6, o, o, e, -1, e>, e, o, g, o, o, -i, 0, -1, 0, o, 0, a, e, q, -1, 0 趴 0, 0训练算法：从词向量计算概率pyth onview pla in copy1. #

12、朴素贝叶斯分类器训练函数2. # trainMatrix:文档矩阵，trainCategory:由每篇文档类别标签所构成的向量3. def trainNBO(trainMatrix, traincategory):4. numTrainDocs = len(trainMatrix)5. numWords = len(trainMatrix0)6. pAbusive = sum(trainCategory) / float(numTrainDocs)7. p0Num = zeros(numWords);8. p1Num = zeros(numWords);9. p0Denom = 0.0;10.

13、 p1Denom = 0.0;11. for i in range(numTrainDocs):12. if trainCategoryi = 1:13. p1Num += trainMatrixi14. p1Denom += sum(trainMatrixi)15. else :16. p0Num += trainMatrixi17. p0Denom += sum(trainMatrixi)18. p1Vect = p1Num / p1Denom19. p0Vect = p0Num / p1Denom20. return p0Vect, p1Vect, pAbusive运行结果:from n

14、unpy import *reload(bayes)cmodule 'bayes' fron T bayes + pyb >>» listOPosts* listclasses = bayes.loadDa»> nyvocabList = bayes*createvocabLtst(ItstOPosts)aaa trainMat=>» far postinDoc in listQPosts:.trainMat,append(bayes,setOfWords2Vec(nyVocabLtst,posttnDoc)i a- a&#

15、187;> pOV, plV, pAb = byes.trainNBettrainMat* listclasses)pAbp6Varray( 0.0526315S,0.95263150*0.05263158> -8,0.65263158, 0.O5Z6315B, s.05nel9Tay Xio >6.052631S8,0.05263158,0.05263158,0.85263158. ®t-o.,0.19526316, -0. O* 目*05263158,6.05263158, 0.0526315B, -0.* 0.05263158,-o.>0.0526315

16、8,fl.O52631S0, '0,0,eS263158r -fi.,0.052631S8,0.05263158,0.15789474)>» plVarray(-fl., -0., -0. Q,05263158G.05263158,O, -e. -0.t 3*05253158,0.65263158,-e.»-e.* -飢*3.05263154C.0526315Bf0.05263158,0.0526315B,0.05263158, -B”6.1G5263L6,V.,0,0S26315B,0.05263158,弋*t0.10S26S16,Ot 0,15789474

17、, -Q.,9+052&3158>-6., *0.j)-o,测试算法：根据现实情况修改分类器上一节中的trainNBO函数中修改几处：pONum = on es( num Words);pINum = on es( num Words);pODe nom = 2.0;p1De nom = 2.0;p1Vect = log(p1Num / p1De nom)p0Vect = log(p0Num / p1De nom)pyth onview pla in copy1. #朴素贝叶斯分类器训练函数2. # trainMatrix:文档矩阵，trainCategory:由每篇文档类别标签

18、所构成的向量3. def trainNBO(trainMatrix, trainCategory):4. numTrainDocs = len(trainMatrix)5. numWords = len(trainMatrixO)6. pAbusive = sum(trainCategory) / float(numTrainDocs)7. pONum = ones(numWords);8. p1Num = ones(numWords);9. pODenom = 2.0;10.11.pIDenom = 2.0;12.for i in range(numTrainDocs): if trainC

19、ategoryi = 1:13.14.pINum += trainMatrixipIDenom += sum(trainMatrixi)15.else :16.17.p0Num += trainMatrixip0Denom += sum(trainMatrixi)1.p1Vect = log(p1Num / p1Denom)p0Vect = log(p0Num / p1Denom)return p0Vect, p1Vect, pAbusive22.朴素贝叶斯分类函数23.24.def classifyNB(vec2Classify, p0Vec, p1Vec, pClass

20、1):25.p1 = sum(vec2Classify * p1Vec) + log(pClass1)p0 = sum(vec2Classify * p0Vec) + log(1.0 - pClass1)26.27.if p1 > p0:return 128.else :2.defreturn 0testingNB():listOPosts, listClasses = loadDataSet()33.34.myVocabList = createVocabList(listOPosts)trainMat =8.39.for postinDoc i

21、n listOPosts:trainMat.append(setOfWords2Vec(myVocabList, postinDoc)p0V, p1V, pAb = trainNB0(array(trainMat), array(listClasses)40.41.testEntry ='love' , 'my' , 'dalmation' thisDoc = array(setOfWords2Vec(myVocabList, testEntry)42.print testEntry,'classified as:',classi

22、fyNB(thisDoc, p0V, p1V, pAb)6.testEntry = 'stupid' , 'garbage' thisDoc = array(setOfWords2Vec(myVocabList, testEntry)print testEntry,'classified as:',classifyNB(thisDoc, p0V, p1V, pAb)运行结果:»> reloadbayes<module 1 ibayes1 from 'bayes.py' >»

23、> bayes )ove1,，dalmation1 'classified as: - 0'茸tupid'r 'garbage'1准备数据：文档词袋模型词集模型（set-of-words model）:每个词是否出现，每个词只能出现一次词袋模型（bag-of-words model ）:一个词可以出现不止一次pyth onview pla in copy.5.6.7.#朴素贝叶斯词袋模型def bagOfWords2VecMN(vocabList, inputSet):returnVec = 0 * len(vocabList)for

24、 word in inputSet:if word in vocabList: returnVecvocabList.index(word) += 1 return returnVec示例：使用朴素贝叶斯过滤垃圾邮件（1）收集数据：提供文本文件（2）准备数据：将文本文件解析成词条向量（3）分析数据：检查词条确保解析的正确性（4） 训练算法：使用我们之前建立的trainNB0（）函数（5 ）测试算法：使用 classifyNB（），并且构建一个新的测试函数来计算文档集的错误率（6）使用算法：构建一个完整的程序对一组文档进行分类，将错分的文档输出到屏幕上准备数据：切分文本使用正则表达式切分句子&g

25、t;» Import re>>> nySent = 1 This book is the best book on Python or M.L* I have ever laid eyes upon,'>» regEx = pilef *)listOfTokens = regEx.split(nySent)>» listOfTokens1bcok"展由好bodM叩Eon)qykythorT . or1,坤X fLl f' ,Thave', 'ever'?, 1 eyes', T

26、upon',''>» tok.lowerf) for tok in listOfTokens if len(tok) > 0(thts', 1 book" ±1 the' # 1 best1 , * book' # lon, * python'T or*, W, fll #','have', 'ever'# Laid1, 'eyes', Tupon'测试算法：使用朴素贝叶斯进行交叉验证pyth onview pla in copy1.

27、#该函数接受一个大写字符的字串，将其解析为字符串列表2.#该函数去掉少于两个字符的字符串，并将所有字符串转换为小写3.def textParse(bigString):4.import re5.listOfTokens = re.split(r'W*', bigString)6.return tok.lower()for tok in listOfTokensiflen(tok) > 27.8.#完整的垃圾邮件测试函数9.def spamTest():10.docList =11.classList =12.fullText =13.#导入并解析文本文件14.for i

28、in range(1,26):15.wordList = textParse(open('email/spam/%d.txt'% i).read()16.docList.append(wordList)17.fullText.extend(wordList)18.classList.append(1)19.20.wordList = textParse(open('email/ham/%d.txt'% i).read()21.docList.append(wordList)22.fullText.extend(wordList)23.classList.appe

29、nd(0)24.25.vocabList = createVocabList(docList)26.trainingSet = range(50)27.testSet =28.#随机构建训练集29.for i in range(10):30.randIndex = int(random.uniform(0, len(trainingSet)31.testSet.append(trainingSetrandlndex)32.del (trainingSetrandlndex)33.34.trainMat =35.trainClasses =36.for docIndex in trainingSet:37.trainMat.append(setOfWords2Vec(vocabList, docListdoclndex)38.trainClasses.append(classListdoclndex)39.40.pOV, p1V, pSpam = trainNBO(array(trainMat), array(tra